The evaluation of the Transport Project and Farm to Market Roads Activity aimed to answer whether or not improved conditions throughout the road network:
• Lowered transport costs and travel time for businesses, including farm households;
• Provided better access to a wider range of job opportunities for individuals (labor market effects);
• Lowered the price of consumables and inputs by increasing competition and reducing barriers to entry posed by poor transport infrastructure; and
• Improved access to health establishments and schools
The overall expected result of these changes was an increase in overall incomes and employment at the household level. To comprehensively evaluate the impact of the MCA Honduras Transportation project, the Independent Evaluator used two methods: (i) a model-based approach, in which the treatment effect is represented by change in travel time, and the program impact is represented as a function of change in travel time. The model relies heavily on Geographic Information Systems (GIS) data for several purposes, including the estimation of changes in travel time; and (ii) HDM-IV analysis.
Kind of Data
Unit of Analysis
Anonymized dataset for public distribution
The target population for the household survey was the population of all households in Honduras at the beginning and end of the project. We used a sample frame constructed for the most recent National Census. Because that Census was conducted a number of years ago (2001), the survey field procedures included procedures to ensure that all current households are subject to sampling (e.g., use of systematic sampling over all of the current households of the entire census segment).
Producers and sponsors
University of Chicago
Millennium Challenge Corporation
Please see Annex 3 of Final Evaluation Report for details. NORC used a panel survey with a stratified two-stage sample design in which the first-stage sample units are caserios. Caserios were selected for use as the first-stage sample unit not only because they (like Census segments) are an efficient size for sampling, but also because a substantial amount of GIS data is available for them.
The sample frame was a list of 22,816 caserios stored in the GIS (and the same as those available from the Census). Caserios in the Islas de la Bahia and Gracias a Dios departments, and caserios in protected status, were excluded as out of scope, reducing the sample frame size to 20,467. These are the primary sampling units (PSUs) for the study.
The caserio sample size and the number of households to be selected from each sample caserio were determined by a detailed statistical power analysis. Based on this analysis, NORC decided to select a sample of 20 households from each of 100 sample caserios. That is, the sample design called for a sample of 2,000 households located in 100 caserios, in each of two survey rounds (before and after the program intervention). The same households were to be interviewed in both survey rounds. Because some caserios did not contain 20 households, baseline data were collected on a total of 1,600 households.
Deviations from the Sample Design
There were no deviations from the sample design.
Of the targeted households, 1,408 households, or about 88 percent, yielded completed interviews in the second round of data collection.
Probabilities of selection of caserios (the PSU) are contained in the file named SampleWith SelectionProbabilities.xls. "Base weights" would be calculated as the reciprocal of the variable "Prob," normalized.
Dates of Data Collection
Interviewing was conducted by 17 fieldwork teams. Each team comprised three interviewers (51 total), a field editor, a driver and a supervisor (17 total in each position). Each team used a 4-wheel drive vehicle to travel from cluster to cluster (caserios) and within cluster where necessary. Supervisors' role was to daily prepare visits, introduce interviewers to respondents, coordinate their teams, observe interviews, provide feedback to interviewers, make sure any interviewing mistakes are highlighted and rectified at the beginning of the fieldwork, and handle the answered ballots for the field editor's review.
Responsibilities of supervisors and interviewers are described in the corresponding manuals included in the Transport Project data and documents submission. The field editor's responsibility was to check all answered ballots to find missed questions, fields incorrectly completed, any inconsistencies in the registered data.
The role of the three technical supervisors appointed by the National Statistics Institute (INE) was to provide assistance to supervisors and reviewers about data quality, rectifying discrepancies, incorrect skips and any issues during fieldwork that needed further management beyond the role assigned to supervisors.
Honduran National Institute of Statistics (Instituto Nacional de Estadistica)
Data collection firm
Household survey; Origin and Destination survey; Traffic Volume; Traffic Speed
During the field stage, field editors and INE technical supervisors checked for data inconsistencies in the field and during data entry. The questionnaire included some coding for geographic reference, and once ballots were filled out, field editors checked coding was appropriately done by interviewers and also coded economic activities. Corrections were done manually by visual control of the questionnaire and no software was used.
Data entry was conducted manually by 11 data-entry clerks from March 14 through April 13, 2011 (two rounds of data entry). All data was entered at INE's data processing department using manual data entry. Clerks entered data using 11 desktop computers using a CSPro template developed by the National Statistics Institute (INE), following the HH-survey questionnaire model. Once data was entered, they were validated using CSPro to identify coding and continuous variables inconsistencies (i.e. values outside certain number of standard deviations, non-logical values, etc.), false skips and any other discrepancy from the questionnaire-model programming. All range checks and skips were controlled by the program and data-entry clerks could not override these. Most open-ended responses ("Other" answers) were not entered or coded, except in rare circumstances where the response matched an existing code in the questionnaire.
Double data entry was conducted on 100% of ballots to perform independent verification of all variables. This process also included separate comparison of data followed by modification of one or both datasets to correct keying errors by original operators who first keyed the files.
After verifying inconcistencies and and incorrect values, mistaken-data depuration took place between 5.04.11 and 5.12.11, and was conducted in SPSS.
Estimates of Sampling Error
Standard errors are calculated for all estimates. They are presented in the Final Report for all estimates. The standard errors were estimated using Stata procedures. They are also presented in the Stata .log file that accompanied the project documentation.
Monitoring and Evaluation Unit
Millennium Challenge Corporation
Caldwell, Joseph, Fidel Ordoñez, Varuni Dayaratna, Michael Reynolds, John Felkner, Impact Evaluation of the Millennium Challenge Corporation Transportation Project in Honduras: Merged Baseline and Endline Household Survey Data, NORC at the University of Chicago, December 12, 2013.