Evaluation of the Mwangaza Mashinani Pilot Project in Kilifi and Garissa Counties, Kenya 2019-2021
Kenya, 2019 - 2021
Get Microdata
Reference ID
KEN_2019-2021_MMPIE_v01_M
Producer(s)
Oxford Policy Management Limited (OPML)
Collection(s)
Metadata
Created on
Mar 08, 2022
Last modified
Mar 08, 2022
Page views
4807
Downloads
57
Identification
Survey ID Number
KEN_2019-2021_MMPIE_v01_M
Title
Evaluation of the Mwangaza Mashinani Pilot Project in Kilifi and Garissa Counties, Kenya 2019-2021
Country/Economy
| Name |
|---|
| Kenya |
Abstract
The Mwangaza Mashinani is a project that aims to enhance energy access for the most vulnerable segment of the Kenyan population. The project is targeted at vulnerable, off-grid households with school-going children who are enrolled in the GoK's cash transfer programme in Kilifi and Garissa. Once enrolled, households may choose to purchase a three-bulb solar home system for lighting and with a USB point for phone charging on a pay-as-you-go basis. To address the issue of affordability of these products, the project provides households with a bi-monthly cash top-up (i.e., a subsidy) which can be used to make repayments for the solar device over a one-year period.
Beyond improving access to clean energy for households, the project aims to increase children's and their families' well-being in terms of: Health - households reduce their use of kerosene and/or candles for lighting resulting in a reduction in indoor air pollution and fire hazards and, ultimately, a reduction in the prevalence of respiratory illness, burns and eye irritation; Education - children can increase the number of hours that they spend studying at night by using solar lighting resulting in improved educational outcomes; and Livelihoods - households can use the solar device to continue their productive activities at night, diversify livelihoods (e.g., by charging mobile phones for a fee), and reduce their net energy expenditure, resulting in an increase in household income.
UNICEF contracted OPM to conduct an independent evaluation of the Mwangaza Mashinani pilot project. The primary purpose of the evaluation was to render a summative assessment of the impact of the pilot project, and to generate scientifically robust evidence on whether and how the project has impact on the quality of life of children and their families in terms of education, health, and livelihoods. The evaluation also had a formative purpose, to improve the processes of the pilot project.
To fulfil these aims, a mixed methods research framework was designed. Broadly, this framework combines four methodologies: i) a quantitative research study to respond to questions of effectiveness and impact, and to support answers on relevance and sustainability. This study is based on a quasi-experimental design (using propensity score matching) and a quantitative household survey, which was implemented through an in-person survey at baseline and endline, and remotely through a mobile phone survey at midline; ii) a qualitative research study to respond to questions of relevance and effectiveness, and to provide initial indications of sustainability. This study is based on interviews with community leaders, community-based project support structures, and male and female household members; iii) an implementation review to respond to questions regarding the relevance and effectiveness of the project. This study is based on interviews with national-level stakeholders, including the GoK, UNICEF, and the implementing consortium and iv) a VfM study to respond to questions of efficiency. It relies on data from UNICEF and the implementing consortium.
Beyond improving access to clean energy for households, the project aims to increase children's and their families' well-being in terms of: Health - households reduce their use of kerosene and/or candles for lighting resulting in a reduction in indoor air pollution and fire hazards and, ultimately, a reduction in the prevalence of respiratory illness, burns and eye irritation; Education - children can increase the number of hours that they spend studying at night by using solar lighting resulting in improved educational outcomes; and Livelihoods - households can use the solar device to continue their productive activities at night, diversify livelihoods (e.g., by charging mobile phones for a fee), and reduce their net energy expenditure, resulting in an increase in household income.
UNICEF contracted OPM to conduct an independent evaluation of the Mwangaza Mashinani pilot project. The primary purpose of the evaluation was to render a summative assessment of the impact of the pilot project, and to generate scientifically robust evidence on whether and how the project has impact on the quality of life of children and their families in terms of education, health, and livelihoods. The evaluation also had a formative purpose, to improve the processes of the pilot project.
To fulfil these aims, a mixed methods research framework was designed. Broadly, this framework combines four methodologies: i) a quantitative research study to respond to questions of effectiveness and impact, and to support answers on relevance and sustainability. This study is based on a quasi-experimental design (using propensity score matching) and a quantitative household survey, which was implemented through an in-person survey at baseline and endline, and remotely through a mobile phone survey at midline; ii) a qualitative research study to respond to questions of relevance and effectiveness, and to provide initial indications of sustainability. This study is based on interviews with community leaders, community-based project support structures, and male and female household members; iii) an implementation review to respond to questions regarding the relevance and effectiveness of the project. This study is based on interviews with national-level stakeholders, including the GoK, UNICEF, and the implementing consortium and iv) a VfM study to respond to questions of efficiency. It relies on data from UNICEF and the implementing consortium.
Unit of Analysis
Individuals
Households
Households
Scope
Notes
The scope of the Evaluation of the Mwangaza Mashinani Pilot Project in Kilifi and Garissa Counties, Kenya 2019-2021 Survey includes the following themes:
Baseline Survey
- Member identification
- Member basics
- Member details
- Household members health
- Education expenses
- Household assets
- Access to energy
- Awareness of alternative sources of energy
- Exposure to intervention
- Woman time use
- Child time use
- Outcome
Midline Survey
- Solar Energy
- Exposure to intervention
- Member livelihoods
- Remittances
- Member education
- Member health
- COVID knowledge and behavior
Endline Survey
- Member basics
- Member details
- Household members health
- Education expenses
- Household assets
- Access to energy
- Solar panel (for non-beneficiary users)
- Solar panel (for beneficiaries)
- Promotion of solar devices
- Covid-19 behaviour and access
- Woman time use
- Child time use
- Covid-19 info for households
- Outcome
Baseline Survey
- Member identification
- Member basics
- Member details
- Household members health
- Education expenses
- Household assets
- Access to energy
- Awareness of alternative sources of energy
- Exposure to intervention
- Woman time use
- Child time use
- Outcome
Midline Survey
- Solar Energy
- Exposure to intervention
- Member livelihoods
- Remittances
- Member education
- Member health
- COVID knowledge and behavior
Endline Survey
- Member basics
- Member details
- Household members health
- Education expenses
- Household assets
- Access to energy
- Solar panel (for non-beneficiary users)
- Solar panel (for beneficiaries)
- Promotion of solar devices
- Covid-19 behaviour and access
- Woman time use
- Child time use
- Covid-19 info for households
- Outcome
Topics
| Topic |
|---|
| Household member demographic information |
| Household member education outcomes and household education expenses |
| Household livelihoods and remittances |
| Household member health |
| Household assets and cooking habits |
| Access to energy for lighting and mobile phone charging |
| Awareness of alternative sources of energy |
| Household use of the solar devices |
| History of repair and maintenance, and willingness to maintain the devices |
| Women’s time use |
| Children’s time use |
| COVID-19 behaviour and access to WASH and health services |
Keywords
| Keyword |
|---|
| Cash Transfer |
| Cash Plus |
| Solar Energy |
| Solar Home Systems |
| Poverty |
| Education |
| Health |
| Livelihoods |
| COVID-19 |
| Kenya |
| UNICEF |
Coverage
Geographic Coverage
Kilifi and Garissa counties
Universe
The following sub-counties are included in the population : Ganze, Magarini and Kaloleni in Kilifi county and Balambala, Dadaab, Fafi and Ijara in Garissa county. The definition of the population is constricted by the circumstances on the ground. Particularly, areas of extreme security risk are excluded from the viable population as the operation of both implementation and evaluation teams would be unsafe. Within these areas, the population of households is restricted to those households eligible for enrolment in the Mwangaza Mashinani pilot project. Specifically, these are CT-OVC and/or OP-CT beneficiary households residing in off-grid communities in Kilifi and Garissa, that have at least one household member enrolled in and attending school and that do not possess a solar device with more than one bulb and who have indicated to be willing to pay 250 Ksh as a deposit for the solar device.
Producers and sponsors
Primary investigators
| Name |
|---|
| Oxford Policy Management Limited (OPML) |
Producers
| Name |
|---|
| Research Guide Africa |
Funding Agency/Sponsor
| Name | Abbreviation |
|---|---|
| United Nations Children's Fund | UNICEF |
| Swedish International Development Cooperation Agency | SIDA |
Sampling
Sampling Procedure
Since a high ratio of the total population eligible for the pilot project was sampled for the evaluation, a sampling methodology was chosen that adheres to the required principles of the representativity of the proposed sample and randomness of the selection. A single stage sampling method was implemented as any optimisation of the costs and field logistics that would have been gained by the multistage approach is offset by the high sampling ratio where most geographical units suitable for primary stage clusters need to be included in the sample. A single sampling method applies the selection algorithm directly on the sample frame in its entirety.
The use of a stratified systematic random selection method would have been ideal for drawing the sample. However, the high sampling ratio (above 50% of the population) in this context precluded the use of systematic methods. To retain control over the structure of the sample and thus ensure representativity, explicit stratification combined with a simple random sampling (SRS) within each stratum was used. Consequently, the allocation of the sample proportion in each explicit stratum is proportional to the size of each stratum in the population.
The definition of the explicit strata is based on the following criteria:
· Cash transfer type - CT-OVC or OP-CT
· Gender of the household head - Male or Female
· Sub-county - Ganze, Magarini, Kaloleni, Dadaab, Fafi, Ijara or Balambala.
As mentioned, the proposed sample includes beneficiaries from both targeted counties: Kilifi and Garissa. The allocation of the sample to the two counties is proportional to the total population in each county. As such, county was imposed as a super-strata with pre-defined allocations of the sample.
The SRS method of selection within each stratum is based on the random number generator.
Given the longitudinal nature of the evaluation, the same baseline respondents were tracked and re-interviewed at midline and endline so as to create a panel of survey respondents. The final quantitative survey sample achievement is shown below, including the distribution by county and treatment status:
Baseline
- Kilifi: 387 (Treatment) and 373 (Control)
- Garissa: 213 (Treatment) and 213 (Control)
Midline
- Kilifi: 321 (Treatment) and 268 (Control)
- Garissa: 180 (Treatment) and 174 (Control)
Endline
- Kilifi: 378 (Treatment) and 366 (Control)
- Garissa: 189 (Treatment) and 181 (Control)
The use of a stratified systematic random selection method would have been ideal for drawing the sample. However, the high sampling ratio (above 50% of the population) in this context precluded the use of systematic methods. To retain control over the structure of the sample and thus ensure representativity, explicit stratification combined with a simple random sampling (SRS) within each stratum was used. Consequently, the allocation of the sample proportion in each explicit stratum is proportional to the size of each stratum in the population.
The definition of the explicit strata is based on the following criteria:
· Cash transfer type - CT-OVC or OP-CT
· Gender of the household head - Male or Female
· Sub-county - Ganze, Magarini, Kaloleni, Dadaab, Fafi, Ijara or Balambala.
As mentioned, the proposed sample includes beneficiaries from both targeted counties: Kilifi and Garissa. The allocation of the sample to the two counties is proportional to the total population in each county. As such, county was imposed as a super-strata with pre-defined allocations of the sample.
The SRS method of selection within each stratum is based on the random number generator.
Given the longitudinal nature of the evaluation, the same baseline respondents were tracked and re-interviewed at midline and endline so as to create a panel of survey respondents. The final quantitative survey sample achievement is shown below, including the distribution by county and treatment status:
Baseline
- Kilifi: 387 (Treatment) and 373 (Control)
- Garissa: 213 (Treatment) and 213 (Control)
Midline
- Kilifi: 321 (Treatment) and 268 (Control)
- Garissa: 180 (Treatment) and 174 (Control)
Endline
- Kilifi: 378 (Treatment) and 366 (Control)
- Garissa: 189 (Treatment) and 181 (Control)
Weighting
No sampling weights are required for the proposed sample design. As the sampling was done using a stratified and proportional simple random selection, all of the sampling units contained within the frame had the same probability of being selected in the final analytical sample.
As the stratification was fully proportional no further post-stratification adjustments are required. This applies to both the initial sample and top-up sample as both samples were drawn to reflect the structure of the underlying population.
For the purpose of the analysis, the complex survey design setup is used in order to account for the stratification and to use the Finite Population Correction (FPC) adjustment for the estimation of the standard errors.
As the stratification was fully proportional no further post-stratification adjustments are required. This applies to both the initial sample and top-up sample as both samples were drawn to reflect the structure of the underlying population.
For the purpose of the analysis, the complex survey design setup is used in order to account for the stratification and to use the Finite Population Correction (FPC) adjustment for the estimation of the standard errors.
Data Collection
Dates of Data Collection
| Start | End | Cycle |
|---|---|---|
| 2019-02-11 | 2019-04-22 | Baseline |
| 2020-07-25 | 2020-08-20 | Midline |
| 2021-04-20 | 2021-06-14 | Endline |
Data Collection Mode
Computer Assisted Personal Interviewing (CAPI) at baseline and endline; and Computer Assisted Telephone Interviewing (CATI) at midline
Data Collectors
| Name | Abbreviation |
|---|---|
| Research Guide Africa | RGA |
Questionnaires
Questionnaires
The baseline, midline and endline survey questionnaires are provided in English, for download as external resources.
Access policy
Contacts
| Name | URL |
|---|---|
| Oxford Policy Management Limited | Link |
Citation requirements
Use of the dataset must be acknowledged using a citation which would include:
- the Identification of the Primary Investigator
- the title of the survey (including country, acronym and year of implementation)
- the survey reference number
- the source and date of download
Example:
Oxford Policy Management Limited (OPML). Evaluation of the Mwangaza Mashinani Pilot Project in Kilifi and Garissa Counties, Kenya (MMPIE) 2019-2021. Ref: KEN_2019-2021_MMPIE_v01_M. Downloaded from [uri] on [date].
- the Identification of the Primary Investigator
- the title of the survey (including country, acronym and year of implementation)
- the survey reference number
- the source and date of download
Example:
Oxford Policy Management Limited (OPML). Evaluation of the Mwangaza Mashinani Pilot Project in Kilifi and Garissa Counties, Kenya (MMPIE) 2019-2021. Ref: KEN_2019-2021_MMPIE_v01_M. Downloaded from [uri] on [date].
Disclaimer and copyrights
Disclaimer
The user of the data acknowledges that the original collector of the data, the authorized distributor of the data, and the relevant funding agency bear no responsibility for use of the data or for interpretations or inferences based upon such uses.
Metadata production
DDI Document ID
DDI_KEN_2019-2021_MMPIE_v02_M
Producers
| Name | Abbreviation |
|---|---|
| Oxford Policy Management Limited | OPML |
| Research Guide Africa, Survey Partner | RGA |
DDI Document version
Version 01 (March 2022)
Assigned World Bank DDI ID and study ID
Added content to the scope, disclaimer, citation, contact information and resource description.
Assigned World Bank DDI ID and study ID
Added content to the scope, disclaimer, citation, contact information and resource description.