Evaluation of the Mwangaza Mashinani Pilot Project in Kilifi and Garissa Counties, Kenya 2019-2021
Kenya, 2019 - 2021
Get MicrodataIdentification
| Name |
|---|
| Kenya |
Beyond improving access to clean energy for households, the project aims to increase children's and their families' well-being in terms of: Health - households reduce their use of kerosene and/or candles for lighting resulting in a reduction in indoor air pollution and fire hazards and, ultimately, a reduction in the prevalence of respiratory illness, burns and eye irritation; Education - children can increase the number of hours that they spend studying at night by using solar lighting resulting in improved educational outcomes; and Livelihoods - households can use the solar device to continue their productive activities at night, diversify livelihoods (e.g., by charging mobile phones for a fee), and reduce their net energy expenditure, resulting in an increase in household income.
UNICEF contracted OPM to conduct an independent evaluation of the Mwangaza Mashinani pilot project. The primary purpose of the evaluation was to render a summative assessment of the impact of the pilot project, and to generate scientifically robust evidence on whether and how the project has impact on the quality of life of children and their families in terms of education, health, and livelihoods. The evaluation also had a formative purpose, to improve the processes of the pilot project.
To fulfil these aims, a mixed methods research framework was designed. Broadly, this framework combines four methodologies: i) a quantitative research study to respond to questions of effectiveness and impact, and to support answers on relevance and sustainability. This study is based on a quasi-experimental design (using propensity score matching) and a quantitative household survey, which was implemented through an in-person survey at baseline and endline, and remotely through a mobile phone survey at midline; ii) a qualitative research study to respond to questions of relevance and effectiveness, and to provide initial indications of sustainability. This study is based on interviews with community leaders, community-based project support structures, and male and female household members; iii) an implementation review to respond to questions regarding the relevance and effectiveness of the project. This study is based on interviews with national-level stakeholders, including the GoK, UNICEF, and the implementing consortium and iv) a VfM study to respond to questions of efficiency. It relies on data from UNICEF and the implementing consortium.
Households
Scope
The scope of the Evaluation of the Mwangaza Mashinani Pilot Project in Kilifi and Garissa Counties, Kenya 2019-2021 Survey includes the following themes:
Baseline Survey
- Member identification
- Member basics
- Member details
- Household members health
- Education expenses
- Household assets
- Access to energy
- Awareness of alternative sources of energy
- Exposure to intervention
- Woman time use
- Child time use
- Outcome
Midline Survey
- Solar Energy
- Exposure to intervention
- Member livelihoods
- Remittances
- Member education
- Member health
- COVID knowledge and behavior
Endline Survey
- Member basics
- Member details
- Household members health
- Education expenses
- Household assets
- Access to energy
- Solar panel (for non-beneficiary users)
- Solar panel (for beneficiaries)
- Promotion of solar devices
- Covid-19 behaviour and access
- Woman time use
- Child time use
- Covid-19 info for households
- Outcome
| Topic |
|---|
| Household member demographic information |
| Household member education outcomes and household education expenses |
| Household livelihoods and remittances |
| Household member health |
| Household assets and cooking habits |
| Access to energy for lighting and mobile phone charging |
| Awareness of alternative sources of energy |
| Household use of the solar devices |
| History of repair and maintenance, and willingness to maintain the devices |
| Women’s time use |
| Children’s time use |
| COVID-19 behaviour and access to WASH and health services |
Coverage
Kilifi and Garissa counties
The following sub-counties are included in the population : Ganze, Magarini and Kaloleni in Kilifi county and Balambala, Dadaab, Fafi and Ijara in Garissa county. The definition of the population is constricted by the circumstances on the ground. Particularly, areas of extreme security risk are excluded from the viable population as the operation of both implementation and evaluation teams would be unsafe. Within these areas, the population of households is restricted to those households eligible for enrolment in the Mwangaza Mashinani pilot project. Specifically, these are CT-OVC and/or OP-CT beneficiary households residing in off-grid communities in Kilifi and Garissa, that have at least one household member enrolled in and attending school and that do not possess a solar device with more than one bulb and who have indicated to be willing to pay 250 Ksh as a deposit for the solar device.
Producers and sponsors
| Name |
|---|
| Oxford Policy Management Limited (OPML) |
| Name |
|---|
| Research Guide Africa |
| Name |
|---|
| United Nations Children's Fund |
| Swedish International Development Cooperation Agency |
Sampling
Since a high ratio of the total population eligible for the pilot project was sampled for the evaluation, a sampling methodology was chosen that adheres to the required principles of the representativity of the proposed sample and randomness of the selection. A single stage sampling method was implemented as any optimisation of the costs and field logistics that would have been gained by the multistage approach is offset by the high sampling ratio where most geographical units suitable for primary stage clusters need to be included in the sample. A single sampling method applies the selection algorithm directly on the sample frame in its entirety.
The use of a stratified systematic random selection method would have been ideal for drawing the sample. However, the high sampling ratio (above 50% of the population) in this context precluded the use of systematic methods. To retain control over the structure of the sample and thus ensure representativity, explicit stratification combined with a simple random sampling (SRS) within each stratum was used. Consequently, the allocation of the sample proportion in each explicit stratum is proportional to the size of each stratum in the population.
The definition of the explicit strata is based on the following criteria:
· Cash transfer type - CT-OVC or OP-CT
· Gender of the household head - Male or Female
· Sub-county - Ganze, Magarini, Kaloleni, Dadaab, Fafi, Ijara or Balambala.
As mentioned, the proposed sample includes beneficiaries from both targeted counties: Kilifi and Garissa. The allocation of the sample to the two counties is proportional to the total population in each county. As such, county was imposed as a super-strata with pre-defined allocations of the sample.
The SRS method of selection within each stratum is based on the random number generator.
Given the longitudinal nature of the evaluation, the same baseline respondents were tracked and re-interviewed at midline and endline so as to create a panel of survey respondents. The final quantitative survey sample achievement is shown below, including the distribution by county and treatment status:
Baseline
- Kilifi: 387 (Treatment) and 373 (Control)
- Garissa: 213 (Treatment) and 213 (Control)
Midline
- Kilifi: 321 (Treatment) and 268 (Control)
- Garissa: 180 (Treatment) and 174 (Control)
Endline
- Kilifi: 378 (Treatment) and 366 (Control)
- Garissa: 189 (Treatment) and 181 (Control)
No sampling weights are required for the proposed sample design. As the sampling was done using a stratified and proportional simple random selection, all of the sampling units contained within the frame had the same probability of being selected in the final analytical sample.
As the stratification was fully proportional no further post-stratification adjustments are required. This applies to both the initial sample and top-up sample as both samples were drawn to reflect the structure of the underlying population.
For the purpose of the analysis, the complex survey design setup is used in order to account for the stratification and to use the Finite Population Correction (FPC) adjustment for the estimation of the standard errors.
Survey instrument
The baseline, midline and endline survey questionnaires are provided in English, for download as external resources.
Data collection
| Start | End | Cycle |
|---|---|---|
| 2019-02-11 | 2019-04-22 | Baseline |
| 2020-07-25 | 2020-08-20 | Midline |
| 2021-04-20 | 2021-06-14 | Endline |
The initial sampling frame used in the first round of data collection was constructed by Energy4Impact (E4I) through a listing exercise that took place in all of the designated sub-counties (both treatment and comparison). The first round of listing was conducted by E4I between December 2018 and early February 2019. Listing was based on in-person visits by teams of enumerators to the beneficiaries known to be living in the designated areas. The questionnaire included all eligibility questions. These data were used to construct the sampling frame by applying the eligibility criteria to the successfully verified beneficiaries. There were 2,629 households included in the sampling frame of which 1,995 households were in treatment sub-counties and 634 households were in comparison sub-counties.
However, due to a change in the beneficiary lists of the Government's cash transfer programme, a supplemental round of listing was conducted. This was run by E4I in all project sub-counties in Garissa and Kilifi but not in the comparison sub-counties. Consequently, the listing in the comparison sub-counties of Balambala in Garissa and Kaloleni in Kilifi was run by Research Guide Africa's survey team as an integrated exercise of the primary data collection. The top-up sample frame was therefore based on two data sources: verification data provided by the implementing partners in the treatment sub-counties and the CCTP-MIS data (which does not include specific data related to the eligibility criteria) in the comparison sub-counties. The treatment sample frame included 1,607 households and the comparison sampling frame included 1,804 unverified households. Based on the rate of eligibility in the treatment sub-counties, it was assumed that 735 of these households were likely to be eligible.
| Name |
|---|
| Research Guide Africa |
Access policy
Use of the dataset must be acknowledged using a citation which would include:
- the Identification of the Primary Investigator
- the title of the survey (including country, acronym and year of implementation)
- the survey reference number
- the source and date of download
Example:
Oxford Policy Management Limited (OPML). Evaluation of the Mwangaza Mashinani Pilot Project in Kilifi and Garissa Counties, Kenya (MMPIE) 2019-2021. Ref: KEN_2019-2021_MMPIE_v01_M. Downloaded from [uri] on [date].
Disclaimer and copyrights
The user of the data acknowledges that the original collector of the data, the authorized distributor of the data, and the relevant funding agency bear no responsibility for use of the data or for interpretations or inferences based upon such uses.
Metadata production
| Name |
|---|
| Oxford Policy Management Limited |
| Research Guide Africa, Survey Partner |
Metadata version
Assigned World Bank DDI ID and study ID
Added content to the scope, disclaimer, citation, contact information and resource description.