Production and Business Services (Impact) 2009-2011
Independent Impact Evaluation
The Productive Development Project (PDP) was one of three large-scale projects financed under the 2006 compact between the Millennium Challenge Corporation (MCC) and the Government of El Salvador. The main objective of the PDP was to assist in the development of profitable and sustainable business ventures for poor individuals in El Salvador's Northern Zone. Between 2008 and 2012, the PDP used nearly $72 million in allocated funds to serve over 13,500 participants and create more than 11,000 full-time equivalent jobs. The PDP comprised three activities: Production and Business Services (PBS), Investment Support, and Financial Services. The PBS Activity -- the subject of this impact evaluation and the largest of the three activities -- provided technical and material assistance to farmers and small-scale producers to support the Northern Zone's dairy, fruit, horticulture, handicrafts, tourism, forestry, and coffee sectors.
The design for the impact evaluation of the PBS Activity is a randomized rollout design, in which some producers were offered PBS assistance several months before other producers were offered similar assistance. The goal of this design is to attain the highest level of rigor possible without significantly altering existing implementation plans. The evaluation is limited to three value chains: (1) handicrafts, (2) dairy, and (3) horticulture. Among all value chains in the PBS, these three chains were most amenable to random assignment, and were expected to yield impacts in the one-year interim timeline specified for the evaluation.
Treatment groups were offered PBS assistance in the first implementation phase (beginning in late 2009 for handicrafts and mid-2010 in the case of the dairy and horticulture chains) and control groups in the dairy and horticulture chains were offered PBS assistance roughly one year after the treatment group. As over 15,000 individuals had participated in the PBS Activity by August 2012, this evaluation sample of 1,736 individuals (518, 593, and 625 in the dairy, horticulture, and handicrafts chains, respectively) includes only 11 percent of the individuals served under the activity.
Kind of Data
Sample survey data [ssd]
Unit of Analysis
Producer level and household level data were collected. The evaluation measured the impact of PBS assistance on employment creation and producers' investment and income, on household income, and intermediate outcomes, such as production levels, business practice adoption, technology adoption, and product diversification.
Income was tracked at the producer and household levels because income at the producer level provides the most direct measure of the economic impact of PBS assistance, whereas income at the household level provides a measure of the ultimate effect of assistance on the well-being of producers' households.
Anonymized dataset for public distribution
Agriculture & Rural Development
The PBS Activity targeted poor farmers, organizations, and micro-, small, and medium enterprises that benefit poor inhabitants of the Northern Zone. Therefore, the sample covered only geographic regions in El Salvador's Northern Zone, including Cabanas, Chalatenango, Custcatlan, Morazan, San Miguel, Santa Ana, and La Union.
The survey covered producers in the handricraft, horticulture, and dairy value chains in the Northern Zone. On average, producers in the handicraft, horticulture, and dairy chain were 40, 45, and 50 years old, respectively.
Producers and sponsors
Mathematica Policy Research, Inc.
Millennium Challenge Corporation
For each value chain, a few weeks prior to the start of the first cycle, Chemonics provided Mathematica with lists of potential beneficiaries required for each value chain. The number of potential beneficiaries on these lists was determined by Chemonics' target number of participants for each implementation cycle, as well as Mathematica's preliminary calculations of the size of the impacts that the evaluation would be likely to detect with those sample sizes. Within the lists, producers were organized by groups -- either the groups in which they already worked or new groups organized by Chemonics for training and assistance purposes.
Then, for each value chain, Mathematica randomized the set of potential beneficiaries into two groups: the treatment group, which would be served in the first cycle, and the control group, which would be served in the second cycle. Groups and individuals designated as exceptions by Chemonics were excluded from randomization. Mathematica sent the randomized lists of assigned potential beneficiaries to Chemonics, and Chemonics informed producers of the start-date of their services.
Baseline and follow-up surveys had response rates above 89 percent, and treatment and control groups in the dairy chain had slightly less than 90 percent of randomized producers in the final evaluation sample.
Statistical weights cannot be used to make the evaluation results truly representative of all PBS participants. This is because the study population was not drawn randomly from the full range of geographic locations served under PBS. Furthermore, the study population was deemed to be eligible only under Phase I selection criteria, as opposed to a substantial portion of PBS beneficiaries, who met stricter Phase II eligibility standards.
While this rigorous evaluation provides an unbiased estimate of the impact of PBS assistance, its primary limitation is the lack of generalizability of its impact estimates. With an analysis sample of less than 2,000 producers in only three value chains, this evaluation's results cannot be extrapolated to the entire population of 16,500 producers assisted all value chains of the PBS activity. In addition, the evaluation's one-year time frame precludes a rigorous evaluation of the impact of PBS assistance over the full multi-year compact period.
Ministry of Economy’s General Office of Statistics and Census
Government of El Salvador
Monitoring & Evaluation Division of the Millennium Challenge Corporation
The user of the data acknowledges that the original collector of the data, the authorized distributor of the data, and the relevant funding agency bear no responsibility for use of the data or for interpretations or inferences based upon such uses.
Copyright 2012, Millennium Challenge Corporation.
DDI Document ID
Millennium Challenge Corporation
Documentation of the Study
Date of Metadata Production
DDI Document version
Version 1.1 (January 2014). This version uses the new MCC metadata template.
Version 2.0 (April 2015). Edited version based on Version 01 (DDI-MCC-SLV-MPR-SLV-2012-v01.1) that was done by Millennium Challenge Corporation.