Hunger Safety Net Programme Survey 2016, Phase 2

HOUSEHOLD SURVEY

The household survey used a two-stage sampling approach, for which the sample frame was defined by sub-locations and households in the HSNP Management Information System (MIS) data. The MIS data are data from a census of nearly all households in the four HSNP counties. The census contains the information that was gathered in respect of these households during the registration for the HSNP programme, their Proxy Means Test (PMT) score and their assignment to the HSNP cash transfers, as well as information about all payments received by all households since the start of Phase 2. The HSNP acknowledges that a small number of the population was recognised to be missed and was registered at a later date. The sampling procedure was intended to cover the different sample requirements of the impact evaluation approaches, including the Local Economy-Wide Impact Evaluation (LEWIE), the quantitative impact evaluation based on the Regression Discontinuity (RD) approach, and the Propensity Score Matching (PSM) back-up.

Drawing the sample consisted of two stages:

1. First stage: sampling of sub-locations
2. Second stage: sampling of households within a sub-location.

First stage of sampling

In the first stage, a first stratification was performed, based on sub-counties within each of the four counties. Sub-locations were then selected within each sub-county. The sampling frame, i.e. list of all sub-locations in each county, was obtained using the HSNP MIS data. An explicit stratification of sub-locations was carried out with the aim of identifying sub-locations that can be defined as towns, nearby villages and remote areas. This was guided by the settlement type classification that has already been made by the programme as part of its sub-location mapping exercise.

Sub-locations were selected using the probability proportional to size (PPS) method. This method implies selecting larger EAs, as defined by the household population, with a higher probability. Sub-locations served as primary sampling units (PSUs) in each county. Before drawing the sample of sub-locations, sub-locations that did not have sufficient households in them to make up the minimum sample size required for the analysis, were dropped. A total of 45 sub-locations from the sample frame that had fewer than 14 households with PMT scores above or below the eligibility cut-off were dropped. In addition, six sub-locations per county (that is, 24 in total) were sampled with certainty, which were county capitals and main trading gateways, as these commercial hubs were important to include in the sample for the LEWIE. After also removing from the sample frame 24 sub-locations which were sampled with certainty, this led to a sample frame size of 433 sub-locations from which to select the remainder of the sub-location sample.

For the remaining sub-locations in our sample frame, the PPS process was implemented. This starts by first generating a list of all sub-locations in the sample frame. This list was sorted into groups for each of the four counties, which amounts to implicit stratification by county. A sampling step was then calculated, based on the cumulative sum of population sizes and the number of sub-locations to be drawn. The sampling step is used to select sub-locations from the list, beginning from a random start.

Due to variation in the population sizes of some sub-locations, the PPS procedure in this instance leads to some sub-locations being selected more than once. If a sub-location was selected more than once then the number of households selected from that sub-location would be increased (doubled, if the sub-location was picked twice, and tripled if selected three times).
Second stage of sampling

In the second stage, a fixed number of households were randomly selected within each sub-location. 24 households were selected for the purpose of the household quantitative impact evaluation. Eight households were added to the sample for the analysis of the HSNP impact on the local economy (i.e. LEWIE sample). The selection of a fixed number of households in the second stage in theory delivers a sample that is self-weighted (compensating for the oversampling of larger sub-locations in the first PPS stage). In practice, analysis weights are still required also to account for non-response, as outlined further below.

The RD approach required a sample of treatment and control households with a PMT score within a small neighbourhood of the HSNP PMT eligibility threshold. To implement this approach, the bandwidth chosen was a distance of 400 above and below the eligibility cut-off. In each sub-location 32 households were sampled, as follows:

· 12 households below the HSNP eligibility cut-off but within the RD bandwidth;
· 12 households above the cut-off but within the RD bandwidth
· Four households below the RD bandwidth and
· Four households above the RD bandwidth.

In some instances, there were insufficient observations below the RD bandwidth to make up the intended LEWIE sample. When this occurred, the shortage was made up by selecting additional households with a PMT above the RD bandwidth. For instance, if there were only two observations below the RD bandwidth in a sub-location, six households would be sampled from above the RD bandwidth.

Sample size

The sampling process yielded a sample of 187 sub-locations, including the 24 that were sampled with certainty. 11 sub-locations were sampled twice and one sub-location was sampled three times. 44 sub-locations were selected in Mandera, 46 in Wajir, 48 in Marsabit and 49 in Turkana. In each sub-location 32 households were sampled. In a few sub-locations there were insufficient households to select the desired LEWIE sample, resulting in fewer than 32 households sampled. Overall, 6,384 households were sampled, and of these 5,979 were successfully interviewed.

Replacement protocol

Within a sub-location, if a selected household could not be surveyed, it was replaced with another household in that sub-location. If some replacement households were selected (due to being unable to identify all initially-sampled households), the number of sampled households in that sub-location increased to a maximum of 48. The households in the replacement pool were further randomised and a sequence of use was determined. Thus, the replacement households were issued in random order. The team supervisor carefully controlled the list of replacement households.

Representativeness

The target population of this research was all the households living in the HSNP counties, comprising both beneficiaries and non-beneficiaries of HSNP cash transfers. The household survey was designed to be representative at the county level.

BUSINESS SURVEY

A business questionnaire was conducted in the three main commercial hubs of each county.

The purpose of the survey was to learn more about local economic activities and livelihoods in the HSNP counties, and the data was used for the LEWIE analysis. The aim was to capture information on three main sectors of the local economy:
· Retailing - shops that sell retail goods on which a price mark-up is applied;
· Services;
· Producers - businesses that transform inputs into outputs.

In each sub-location, a sample of at least seven businesses from each category was targeted. Since no sampling frame for local businesses was available, the survey research teams in each county undertook a listing exercise of all businesses on the main commercial centre of the selected sub-locations. The following categories of businesses were excluded from the listing:
· Temporary stalls or mobile sellers located outside permanent kiosks;
· Banks;
· Education institutions (schools, universities etc.);
· Health facilities.

Once the listing was completed, the team leader sampled the required number of businesses using a step sampling approach. Overall, 282 business questionnaires were administered in the four counties.

Representativeness

The business survey is not representative of any commercial hubs.

LIVESTOCK TRADER SURVEY

Since livestock trading is a very important activity in the HSNP counties, a number of livestock traders have been interviewed to understand better how the market works. In each county, three main livestock markets were targeted for interviews.

Each enumerator team was asked to interview four traders in each of the sub-locations, leading to a total sample size of 12 livestock trader interviews per county. Sampling of livestock traders was mostly done purposively. To the extent possible, team leaders sampled livestock traders in order to achieve a balance between those trading large animals, those trading small or medium value animals, those trading only within the HSNP counties and those who also trade outside the HSNP counties.

Representativeness

The livestock trader survey is not representative of any livestock markets.