Evaluation of the Hunger Safety Net Programme Phase 3: COVID-19 Cash Transfer 2020-2021

The evaluation team implemented a stratified one-stage probability sampling strategy for the selection of survey respondents from the individuals included in the lists covered by Give Directly for the COVID-19 CT. The goal was to select at baseline a sample of 1,000 eligible individuals who would receive the COVID-19 CT, which would then be interviewed by the evaluation team at baseline, midline, and endline.

The sampling strategy considered the following process:

1. The sample was drawn once the COVID-19 CT beneficiaries were considered as enrolled into the intervention. After discussions with Give Directly, it was decided that an individual was considered a future COVID-19 CT recipient when he/she had responded to the short SMS-based survey delivered by Give Directly.

2. The sample was drawn in two separate batches. The first batch of recipients comprised 6,838 vulnerable individuals from informal settlements in Nairobi, while the second batch contained 1,596 vulnerable individuals from Mombasa. We sampled the same number of beneficiaries from the first and second batches (500 individuals from each batch).

3. Explicit stratification was first applied based on the geographical location of the COVID-19 CT recipient. This entailed that we sample 500 individuals from Nairobi from the first batch, and 500 from Mombasa from the second batch. This allowed us to disaggregate our quantitative findings between Nairobi and Mombasa, and produce informative descriptive and regression analyses for each of the two cities included in the intervention.

4. Implicit stratification was then applied based on the following categorical variables: i) local partner from which the eligible beneficiary was selected, and ii) gender of the COVID-19 CT recipient. The goal of this stratification process was to enhance the representativeness of our sample in terms of these variables, so that our evaluation sample resembled as much as possible the distribution of these characteristics in the target population (i.e. the list of beneficiaries of the COVID-19 CT used as sampling frame for our sample).

5. We did not cluster our survey respondents. Apart from spill-over effect issues, which were not a concern due to the lack of a counterfactual in our methodological approach, this is normally a logistical necessity for in-person surveys. This was not an issue either, given the remote nature of the data collection process.

6. Extensive replacement lists were created to maximise efficiency during survey implementation without sacrificing representativeness of the sample. A detailed replacement protocol was elaborated, which took into account the stratification process described above.
   Given the longitudinal nature of the evaluation, the same baseline respondents were tracked and re-interviewed at midline and endline so as to create a panel of survey respondents.The final baseline quantitative survey sample achievement is shown below, including the distribution by county

Sample achievement
Baseline Survey
Nairobi 500
Mombasa 500
Total 1,000

Midline Survey
Nairobi 483
Mombasa 489
Total 972

Endline Survey
Nairobi 463
Mombasa 478
Total 941