Safety Nets Follow-up Panel Survey, 2015

The follow up survey sampling strategy builds on the baseline sampling strategy. The communes covered by the baseline survey include Tibiri and Guecheme in the region of Dosso, as well as Sae Saboua, Guidan Sori, Gangara and Tchadoua in the region of Maradi. In these communes, over 500 villages were eligible for the first phase of the Cash Transfer Program, many more that the project could serve. Given the difficulty to find transparent targeting criteria to prioritize villages within communes, the project team decided to implement public lotteries to select beneficiary villages among all equally eligible villages. Within commune, a randomization procedure is used to select beneficiary villages through public lotteries in presence of village chiefs, commune authorities and program staff. Prior to performing the randomization, small villages were grouped into clusters containing at least 150 households. The randomization was performed by clusters, and stratified to ensure an equal probability of selection for nomadic and sedentary villages. In addition to selecting villages to benefit from the cash transfer project, a number of control villages were also drawn to be sampled at baseline. Since the baseline sample of clusters for the evaluation is obtained through randomization among all clusters of villages selected communes, it is representative of these communes. The final evaluation sample includes 151 clusters (244 villages).

Prior to the baseline survey, a listing exercise was undertaken in all villages in the evaluation sample. Based on this household listing, screening criteria were applied to exclude ineligible households, defined by program documents as those with self-reported income higher than a pre-set threshold. Approximately 20% of households were deemed ineligible based on these criteria. The listing of households eligible to the cash transfer program constitutes the sampling frame for the baseline survey. It is representative of households eligible for the cash transfer program at the commune level. Therefore, the evaluation sample is representative of eligible households in communes eligible to the cash transfer program. The evaluation sample was drawn by taking a random sample of 30 eligible households from the sampling frame in each cluster. The baseline survey successfully interviewed 4330 households.

After the baseline survey was conducted, clusters assigned to receive the cash transfer program were further randomized into a group that would receive the cash transfer only (CT), and a group that would receive the cash transfer plus behavioral change accompanying measures (CT+BCC). In addition, the baseline survey data was merged with administrative data from the cash transfer program in order to identify which households were selected as beneficiaries. Table 3 below summarizes the composition of the baseline sample, including by treatment and control group, as well as by household beneficiary status in the treatment group.

Table 3: Composition of Baseline Sample
C CT CT+BCC Total
Beneficiaries HH 0 558 570 1128
Non-Beneficiaries HH 1469 862 871 3202
Total HH 1469 1420 1441 4330
Total Clusters 52 50 49 151

The sampling strategy for the follow-up survey aimed at ensuring sufficient statistical power to detect impacts between the various treatment and control groups, and including among the sub-sample of beneficiary households between the two treatment groups. Therefore, the follow-up sample was stratified based on the proxy means test score used to determine eligibility to the program. Specifically, all the households with a proxy means test score below 1.04 times the beneficiary selection threshold were selected, while half the households with a proxy means test score equal or greater than 1.04 times the beneficiary selection threshold were selected. Table 4 details the composition of the follow-up panel sample.

Table 4: Composition of Follow-up Panel Sample
C CT CT+BCC Total
Beneficiaries 0 558 570 1128
Non-Beneficiaries 1313 760 752 2825
Total 1313 1318 1322 3953
Clusters 52 50 49 151

For the purpose of the project impact evaluation, an additional booster sample of 1058 beneficiary households was randomly selected from the administrative database of beneficiaries and added to the follow-up sample. That booster sample is only added for the two treatment groups. The booster sample is not part of the follow-up panel survey.