Hunger Safety Net Programme Impact Evaluation 2012, Second Follow-up Round
Kenya, 2012
Get MicrodataIdentification
KEN_2012_HSNP-FU2_v01_M
Hunger Safety Net Programme Impact Evaluation 2012
Second Follow-up Round
| Name | Country code |
|---|---|
| Kenya | KEN |
Oxford Policy Management was responsible for the monitoring and evaluation (M&E) of the programme under the pilot phase, with the intention of informing programme scale-up as well as the government’s social protection strategy more generally. The M&E involved a large-scale rigorous community-randomised controlled impact evaluation household survey, assessment of targeting performance of three alternative targeting mechanisms (Social Pension; Dependency Ratio; Community-based Targeting), qualitative research (interviews and focus group discussions) to assess targeting and impact issues less easily captured in the quantitative survey, and on-going operational and payments monitoring to ensure the smooth implementation of the programme. Findings were communicated to the HSNP Secretariat, Government of Kenya and the Department for International Development (DFID) on a regular basis to inform and advise on policy revisions and development. The M&E component used the data it produced to advise the design of HSNP Phase 2, including micro-simulations of different programme targeting scenarios and review of the phase 2 targeting approach which combines proxy means testing with community-based targeting.
The impact evaluation study compares the situation of HSNP and control households at the time of their selection into the programme (baseline), with their situation 12 months (year 1 follow-up) and 24 months later (year 2 follow-up). Over this 24-month period most of the HSNP households covered by the evaluation had received 11 or 12 bi-monthly transfers (initially KES 2,150, increased to KES 3,500 by the end of the evaluation period).
The baseline data collection was completed in November 2010, the first round of follow-up data collection finished in November 2011, and the final round of fieldwork - in November 2012.
Sample survey data [ssd]
- individuals,
- households,
- community.
Version
Version 01
Scope
The scope of the study includes:
- Household demographic characteristics
- Health
- Individual mobility
- Housing and amenities
- Consumption expenditures - food and non-food consumption
- Income from livelihood activities
- Assets
- Food availability and coping strategies
- Subjective poverty assessment
- Formal/informal transfers, food aid and employment programs
- Saving and borrowing
- Community infrastructure, water, shocks, seasonality, prices
- Community HSNP targeting.
Coverage
Counties of Turkana, Marsabit, Wajir, and Mandera.
All persons living within "secure" sub-locations across all counties at the time of sampling (2008; due to sporadic insecurity across the four counties, a small portion of sub-locations were deemed to be insecure when the sample was drawn and so excluded from the sample frame).
Producers and sponsors
| Name |
|---|
| Oxford Policy Management Limited |
| Name | Role |
|---|---|
| Institute for Development Studies | Consortium member HSNP M&E component |
| Research Solutions |
| Name | Role |
|---|---|
| Department for International Development | Programme and Evaluation Funder |
Sampling
At follow-up 2, in addition to attrition, the sample size is further reduced because the follow-up 2 survey covered eight fewer sub-locations, 40 rather than 48. Overall 2,436 households were surveyed (at the baseline, 5,108 households were covered).
The evaluation sub-locations were selected from a sample frame of all secure sub-locations in each district. In each district 12 sub-locations were selected with PPS (Probability Proportional to Size) with implicit stratification by population density such that there is an even number of selected sub-locations per new district.
The evaluation sub-locations were sorted within districts by population density and paired up, with one of the pair being control and one being treatment.
The sampling strategy for the quantitative survey was designed in order to enable a comparison of the relative targeting performance of three different targeting mechanisms. These are:
- Community-based targeting (CBT): The community collectively selected households they consider most in need of transfers, up to a quota of 50% of all households in the community;
- Dependency ratio targeting (DR): Households were selected if individuals under 18 years old, over 55 years old, disabled or chronically ill made up more than a specified proportion of all household members;
- Social pension (SP): All individuals aged 55 or older were selected.
For both the treatment and control sub-locations there are an equal number of CBT, SP and DR sub-locations. Assignment of targeting mechanisms to sub-locations was done randomly across the same pairs that were defined to assign treatment and control status.
In all the evaluation sub-locations, the HSNP Admin component implemented the targeting process. In half the sub-locations the selected recipients started receiving the transfer as soon as they were enrolled on the programme - these are referred to as the treatment sub-locations. In the other half of the evaluation sub-locations, the selected recipients were not to receive the transfer for the first two years after enrolment - these are referred to as the control sub-locations.
The households in the treatment sub-locations that are selected for the programme are referred to as the treatment group. These households are beneficiaries of the programme. In control sub-locations the households that are selected for the programme are referred to as the control group. These households are also beneficiaries of the programme but only begin to receive payments two years after registration. The targeting process was identical in the treatment and control sub-locations.
The following population groups can thus be identified and sampled:
- Group A: Households in the treatment sub-locations selected for inclusion in the programme;
- Group B: Households in control sub-locations selected for inclusion in the programme but with delayed payments;
- Group C: Households in treatment sub-locations that were not selected for inclusion in the programme;
- Group D: Households in control sub-locations that were not selected for inclusion in the programme.
Because targeting was conducted in both treatment and control areas, households were sampled in the same way across treatment and control areas. Selected households (groups A and B) were sampled from HSNP administrative records. Sixty six beneficiary households were sampled using simple random sampling (SRS) in each sub-location (in two of sub-locations this was not possible due to insufficient numbers of beneficiaries in the programme records). In cases of household non-response replacements were randomly drawn from the remaining list of non-sampled households. This process was strictly controlled by the District Team Leaders.
Non-selected households (groups C and D) were sampled from household listings undertaken in a sample of three settlements within each sub-location. These settlements were randomly sampled. The settlement sample was stratified by settlement type, with one settlement of each type being sampled. Settlements were stratified into three different types:
- Main settlement (the main settlement was defined as the main permanent settlement in the sub-location, often known as the sub-location centre and usually where the sub-location chief was based. As there was always one main settlement by definition, the main settlement was thereby always selected with certainty).
- Permanent settlements (permanent settlement is defined as a collection of dwellings where at least some households are always resident, and/or there is at least one permanent structure).
- Non-permanent settlements.
As concern community level data, community questionnaires were conducted in every community for which at least one household interview was attached. A community was defined as a settlement or a sub-section of a settlement if that settlement had been segmented due to its size. Due to missing data, a small proportion of households are not linked to any community data.
The above explanation is taken from "Kenya HSNP Monitoring and Evaluation Component: Impact Evaluation Final Report 2009 to 2012". For more details please refer to this report in Related Materials section.
The reduction in the number of sub-locations surveyed at follow-up 2 was the result of decisions made by the programme and its stakeholders, rather than a technical decision by the evaluation team. This reduction in sample size is unfortunate for a number of reasons. Firstly, it undermines the study design to the extent that the smaller sample size reduces the ability to detect impact with statistical significance. Secondly, it affects the balance of the sample, meaning that treatment and control populations are less balanced at baseline than they were with the original sample structure. Lastly, the sample was designed to be seasonally balanced across the whole calendar year, which is no longer the case as sub-locations that would have been surveyed in the latter and early part of the calendar were dropped. Another implication of the reduced sample at follow-up 2 is that the baseline estimates presented in this report differ from those presented in the baseline and follow-up 1 impact reports. This is because the estimates now relate to slightly different populations.
Two versions of the sampling weight are provided:
- hh_wt sampling weights produce estimates for all households living in sub-locations covered by the evaluation (i.e. the study population). They do not provide estimates for any larger population.
Weights are given by the inverse probability of being selected by strata. For selected households, the weights are given by: wi = Ni /ni, where:
- ni is the number of beneficiary households interviewed in the ith sub-location;
- Ni is the number of beneficiaries listed in the HSNP administrative data for that sub-location.
For non-selected households, the weights are given by:wijk = 1 / [ (aijk/Aijk) (1/bij)(1/cij) ], where where:
- Aijk is the total number of non-beneficiary households of residency status k in the selected segment of the selected type j settlement in sub-location i;
- aijk is the number of households of residency status k in the selected segment of the selected type j settlement in sub-location i that were interviewed;
- bij is the total number of segments in the selected type j settlement in sub-location i (often bij=1);
- cij is the total number of settlements of type j in sub-location i.
- hh_wt_original sampling weights produce representative statistics for the entire population of secure sub-locations within each district.
Community-level variables can be weighted using community weights (cmq_wt), which equal the sum of household weights across the households lnked to that community.
The above explanation is taken from "Kenya HSNP Monitoring and Evaluation Component: Impact Evaluation Final Report 2009 to 2012". For more details please refer to this report in Related Materials section.
Data collection
| Start | End |
|---|---|
| 2012-02 | 2012-11 |
This is the Baseline component of a panel survey and it was conducted between August 2009 to November 2010.
Data Access
The use of the datasets must be acknowledged using a citation which would include:
- the identification of the Primary Investigator (including country name),
- the full title of the survey and its acronym (when available), and the year(s) of implementation,
- the survey reference number,
- the source and date of download (for datasets disseminated online).
Example:
Oxford Policy Management Limited. Kenya Hunger Safety Net Programme Impact Evaluation 2012, Second Follow-up Round. Ref. KEN_2012_HSNP-FU2_v01_M. Dataset downloaded from [URL] on [date].
Disclaimer and copyrights
The user of the data acknowledges that the original collector of the data, the authorized distributor of the data, and the relevant funding agency bear no responsibility for use of the data or for interpretations or inferences based upon such uses.
Contacts
| Name | Affiliation | |
|---|---|---|
| Marta Marzi | Oxford Policy Management Limited | marta.marzi@opml.co.uk |
| Fred Merttens | Oxford Policy Management Limited | fred.merttens@opml.co.uk |
Metadata production
DDI_KEN_2012_HSNP-FU2_v01_M
| Name | Affiliation |
|---|---|
| Development Data Group | World Bank |
| Oxford Policy Management Limited |
2014-01-07
Metadata version
v01 (January 2014)