This study is an impact evaluation on the overall effect of the PURP on both project recipients and their neighbors. Key outcomes are household agricultural income, milk yields from cattle, and restriction of herd size to the carrying capacity of the land. A separate but coordinated study will examine the long term effect of the project on land degradation patterns. In Phase 2 areas, eligible applicants to the project were entered into a lottery and half of them were randomly selected to become project beneficiaries. Thus a very clean control group was created to determine the causal impacts of the project.
Kind of Data
Sample survey data [ssd]
Unit of Analysis
Anonymized dataset for public distribution
pollution and protection
income, property and investment
land use and planning
Phase II of PURP consists of two peri-urban areas. The first is the area surrounding the city of Choibalsan in Dornod aimag, within 45 km of the city. The second is surrounding the cities of Arvaikheer and Kharkhorin in Uvurkhangai and Arkhangai aimags, and consists of parts of 10 soums.
All households that were members of the 328 herder groups that applied to the PURP in the Phase II areas. And two neighbor households of these groups that lived within 2 km of the lease area boundaries.
Modification for Wave 2:
Sampling of neighbor households was extended to 5 km distance from lease area boundaries, due to low rates of successful neighbor interview in the baseline survey.
Producers and sponsors
Innovations for Poverty Action
Millennium Challenge Corporation
All households that were members of groups that applied to the project were included in the survey (total 923 interviewed). Moreover two neighbor households were intended to be surveyed for each group, randomly chosen from a list provided by a firm that had mapped the households in the areas (total 420 interviewed). For project applicants, power calculations were performed for one key outcome variable, average daily milk yield per milking cow (liters): Baseline value = 2.19, Std Dev. = 1.3, MDE = 0.25
FOLLOW-UP (Wave 2):
Sample from baseline was augmented by any PURP treatment and control households that were missed in the baseline survey (total 24).
Additionally, since few groups in the baseline survey had two successful neighbor interviews, neighbors were re-sampled in Wave 2. If a group had one or two neighbors interviewed in the baseline survey, then the same neighbors were to be interviewed in Wave 2. If the group did not have a full set of two neighbors in the baseline, then a list of all neighbor households within ten kilometers of the lease area were given by the herder group leaders. The survey firm then attempted to locate these households. The first two households within 5 kilometers of the lease boundary were to be interviewed. If there were not two households within 5 kilometers, then neighbors up to ten kilometers away could be interviewed. In total 143 new neighbors were added to the sample for Wave 2.
Deviations from the Sample Design
Some groups did not have two neighbors within 2km of their lease area boundary. In these cases sometimes less than two neighbors were interveiwed per group. This was especially the case in the Choibalsan peri-urban area, because households are spread very widely apart.
92.6% (Treatment households)
89.1% (Control households)
72.1% (percent of Treatment groups with at least one neighbor interview)
71.7% (percent of Control groups with at least one neighbor interview)
-response rates for neighbors differed substantially by peri-urban area, and were much lower in Choibalsan area (45%) compared to Kharkhorin area (86%)
FOLLOW-UP (Wave 2):
90.3% (Treatment households)
82.4% (Control households)
84.8% (percent of Treatment groups with at least one neighbor interview)
85.2% (percent of Control groups with at least one neighbor interview)
-these are listed as a proportion of the original baseline sample list, augmented with new households discovered during the follow-up survey
-response rates for neighbors differed substantially by peri-urban area, and were much lower in Choibalsan area (64%) compared to Kharkhorin area (96%)
No sample weighting is required for this dataset
Dates of Data Collection
Household & Group Leader Baseline (Wave 1)
Household & Group Leader Follow-up (Wave 2)
Interviewing was conducted by 2 teams of interviewers. Each interviewing team comprised of 1 team leader, 6 interviewers, and 2 drivers. Each team used 2 four wheel drive vehicles to travel from cluster to cluster (and where necessary within cluster).
The role of team leader was to ensure the overall coordination and technical oversight of all the tasks, maintain the communication with IPA and MCA-Mongolia, coordinate with local authorities, and make arrangements for accomodation and travel. Additionally, the team leader was responsible for ensuring the quality of data collection by checking all the paper copy of questionnaires that are filled out by interviewers at the end of everyday.
Wave 1: Joint affiliation of MEC and MCDS LLC
Wave 2: Joint affiliation of MEC and MCDS LLC
Household questionnaire - collects basic socio-economic and key herding related outcomes from the households (in Mongolian)
Herder Group Leader questionnaire - collects information about group members and their joint business activities (in Mongolian)
Soum Governor questionnaire - collects information about soum level demography, livestock census, land information, and development projects (in Mongolian)
Data editing took place at a number of stages throughout the processing, including:
a) Field check by survey firm team leaders
b) Checking the discrepencies of 1st and 2nd data entry
c) Internal logic check by survey firm
d) Manual check by IPA
e) Logic control check by IPA
f) Data cleaning by IPA
All surveys were checked for logical consistency by interviewer supervisors in the field, the day they were collected. If a survey contained inconsistent answers, the respondent was re-interviewed to clarify the problematic questions.
Consistency checking, including skip patterns and restricted value ranges, when applicable, were programmed into the survey software.
Several types of missing value were recorded. These are operationalized as extended missing values in the Stata datasets.
Note: Questions that were accurately skipped (.s) were generally not coded for the baseline survey and were instead left blank. All valid skips were coded in the follow-up survey.
Numeric missing value code is given first (followed by string code in parentheses)
.s (.skip) -- Question was skipped according to a correct skip pattern
.r (.refusal) -- respondent refused to give a valid answer to the question
.d (.don't know) -- respondent did not know the answer to the question
.q (.qre refused) -- respondent refused to give answers to an entire section of the questionnaire
Data entry was conducted by 6 data entry staff. All the data entry was conducted at Survey firm internally developed software, which was approved by IPA. First all the questionnaires were scanned and manually entered twice by 2 different data entry staff. Then the two entries were reconciled to produce a third dataset. Next, IPA took a random sample of 1500 data points fromt his third dataset and manually checked them against the paper copies of the questionnaires. The data was accepted from the data collection contractor when the error rate found by the manual check was below 0.5%.
FOLLOW-UP (Wave 2):
Data collection was conducted on tablets and thus data was entered directly. Interviews were audio-recorded and a random sample of audio recordings were checked against the dataset to verify data integrity. Errors were scored based on their severity and the average error rate was required to fall below a preset threshold.
Monitoring & Evaluation Division of the Millennium Challenge Corporation