Urban Property Rights Project Special Hashaa Plot Survey 2011-2013
Independent Impact Evaluation
The impact evaluation study of the MCA-M PRP will be the first fully randomized evaluation of a large-scale land titling program. Randomization will occur at the geographic level akin to a neighborhood. Mongolian cities are divided up into a number of administrative units - the smallest being the “kheseg”. Khesegs were chosen as the unit of randomization for the study because they are a well-defined unit that is small and numerous enough to allow for sufficient statistical power. The baseline estimation strategy will be a differences-in-differences approach, where we compare the outcomes of households in the treatment group with the control group as well as before and after the completion of the formalization activities. Exposure to treatment was 66% in Darkhan and Erdenet, and 50% in Ulaanbaatar districts. There are no results to report as of now because only the baseline has been conducted so far.
Kind of Data
Sample survey data [ssd]
Unit of Analysis
Anonymized dataset for public distribution
Property and investment
Property and saving
Regionally: Ulaanbaatar, Darkhan and Erdenet
Households living in hashaa plots in the ger districts of Mongolia's three largest cities: Ulaanbaatar, Darkhan, and Erdenet.
Producers and sponsors
Innovations for Poverty Action
Millennium Challenge Corporation
8,552 plots were identified for surveying for the sample. Of these, 6,344 were occupied households and 5,816 were successfully interviewed for a response rate of 68%. 528 households refused to participate in the survey and 2,068 plots were unoccupied, had no one present at the time of any of the survey attempts, or were invalid plots. Plots found to be unoccupied or to be owned or occupied by a business or state entities were deemed unsuitable for the survey and were dropped from the sample. Geographic Information System data on all hashaa plots in the ger areas of the relevant districts of the capital and in Darkhan and Erdenet, were obtained from the PRP PIU. The ownership status of many of these plots was recorded in this GIS data set, though the ownership status information was known to be out of date and inaccurate. The boundaries of administrative units such as city, district, khoroo, and kheseg were also included. IPA processed the GIS data using ArcGIS and Stata computer software.
Once the GIS and administrative cadastral data sets were integrated, sample selection was stratified by kheseg, a geographical unit roughly equivalent to a neighborhood in the United States. First, the number of program-eligible plots per kheseg was calculated. Plots listed as “fully registered” in the GIS data were not included in this calculation since they would not be eligible for project assistance. Weights were then calculated for each kheseg unit that measured the proportion of the total number of eligible plots located in this unit. These weights were then multiplied by 8,000, the total number of plots it was deemed desirable and feasible to include in survey activities, to determine the number of plots to be sampled from each kheseg. After the sample size for each kheseg was determined, plots were randomly selected for inclusion in the survey.
Deviations from the Sample Design
In November of 2010, the survey contractor selected by MCA-M began administering the questionnaire to the households residing on and/or owning the plots selected during the sampling process. Due to the anticipated errors in the Geographic Information System data, not all of the hashaa plots selected for the SHPS sample were occupied. In addition, Mongolian households are extremely mobile. To minimize these challenges, the survey teams were required to make four attempts to locate the hashaa plot to determine the registration status and an additional four attempts to complete the survey questionnaire. Unfortunately, the SHPS had to be suspended after several weeks of data collection due to unforeseen delays in project implementation. The scope of the project was subsequently adjusted and the project implementation areas shifted due to the inflexibility of the data collection contract. The scope of the project was reduced from covering all districts in Ulaanbaatar to covering only the three largest districts, Bayanzurkh, Chingeltei, and Songinokhairkhan.
The response rate was 68%.
Dates of Data Collection
Interviewing was conducted by teams of interviewers. There were up to 4 supervisors, 40 interviewers, hashaa registration teams that comprised of team leader and driver, and field survey teams of 6 and 8 people for Darkhan and Erdenet. There was also a survey data quality team and a database creation team led by an IT manager.The team applied 2 phased approaces: 1 phase hashaa registration and 2 phase administration of main survey. The role of the team leader was to coordinate field data collection activities, maintain relationship with MCA and IPA, manage budgets, prepare and submit reports and oversee sub-contractors. The field manager's role was to recruit data collection team, carry out interview training on data collection, entry, cleaning and processing, organize training on the use of GPS receiver sets, organize all logistics for the field, field data collection by 6 data collection teams, oversee field operation, organize data entry, data processing and handling. The supervisor was responsible for using the information collected during hashaa registration to contact the respondent and confirm a date for the questionnaire to be administered and send an interviewer accordingly. Survey team members were in charge of carrying out actual interviews respondents and following up with them if necessary. Survey data quality team were required to listen to audio recordings of all questionnaires and checking it against hard copy and to check Data Quality Monitor'scomments and fix the errors in data files. Database creation team were in charge of scanning questionnaires, doing data entry, checking data entry against original PDF file, database logical control and translating database into English.
MEC LLC and MCDS LLC
Household questionnaire prepared in both Mongolian and English. The team organized 4 pilot testings involving 109 respondents. Modules: - Log of attempts made to take survey, - 1. Registration section 2. Control section (filled by enumerator) 3. Introduction to survey 4. Basic Information 5. Demographic, education level and residential information of household members 6. Economic activities and incomes of household members 7. Household assets and properties 8. Planned future investments 9. Registration status of plot being surveyed 10. Implementation level of the 2003 amendment to the Land Law. 11. Accessability of land registration information and service quality at General Authority of State Registration 12. Land conflicts 13. Hashaa plot sales and its market value 14. Infrastructure of hashaa plots 15. Household spendings 16. Household business activities 17. Insurance 18. Household loans 19. Government policy and thoughts on its implementation 20. Citizens' involvement and labor in common 21. Risk evaluation
IDs in the dataset were checked against the original sample frame to make sure that they were correctly entered and complete. In addition,team leaders manually inspected each survey to ensure accuarcy of data collected and for logicallly consistancy. Back checks were also performed.
Data entry was conducted by 8 operators of the Database Creation team. The team used ABBYY FlexiCapture Standalone and Distributed version for data entry automated solutions for the questionnaires. Templates of the questionnaire and answer sheets were created using a special program. Then the IT manager created an auto recognition definition template using Document Definition Editor or ABBYY FlexiCapture 9.0 which will automatically identify pages and document types. Two different operators made data entry #1 and data entry #2. The third operator checked and corrected incorrect values for logical and validation rules. After checking every character and group field correction all variables were recognized and verified. Then the data was exported into MS Excel MS Access. The export document includes the imported, recognized, verified and exported dates. Then the third operator compared the database #1 to database #2 to identify differences. After checking the database errors, the third operator provided a logical control using a filter. Finally, an english version of the database is created. The final database is then imported into STATA. There were no problems in the course of finalizing the database.
In addition, Innovations for Poverty Action took a random sample of 1500 data points and manually checked them against the paper copies of the questionnaires. The data was accepted from the data collection contractor when the error rate found by the manual check was below 0.5%.
Monitoring & Evaluation Division
Millennium Challenge Corporation
Shing-Yi Wang, Erica Field and Leigh Linden. 2012. Special Hashaa Plot Baseline Survey.