Health Results-Based Financing Impact Evaluation 2014

Household Survey

The process evaluation applied a retrospective study design and a theory-based evaluation approach that made use of sequential mixed methods. The retrospective design allowed for classification of observations according to the outcomes of interest and retrospectively assessing their exposure and interaction with specific study factors, e.g., contextual factors and intervention design factors. This is facilitated by the theory-based evaluation approach, which examines the interaction between the context, the actors, and the intervention, and then attempts to explain how this interaction works to produce the outcomes of the intervention by interrogating the intervention’s formal theory of change. The theory-driven approach sought to explore the influence of contextual factors on interventions and its outcomes through tracking and validating the program impact pathways.

The DHE (District Health Executives) team members, facility managers, health workers, HCCs and health facility catchment communities within World Bank funded RBF districts constituted the sampling frame from which respondents were purposively drawn to participate in a qualitative inquiry. A multistage sampling approach was used to select the Province, Districts, Facilities and Community Members with each using Purposive Sampling although each had varying “purposes” or specific reasons for selection. The cascade sampling first selected three provinces from the eight rural provinces in which RBF operated. The criteria for selection was based on geographic spread to ensure representation from each geo-region. Then within each of the three selected provinces, one or two districts were selected based on their identification as cases of interest by the project implementing entity. A total of four districts were selected.

Finally, the third stage of sampling involved the selection of one high- and one low-performing facility from each selected district. Of note is that the facilities were in part selected based on performance as defined by their actual earnings relative to expected earnings. The classification of performance therefore entailed initially assessing facility performance using quantitative methods and then proceeding to obtain primary qualitative data. The research team collected primary data through in-depth interviews, focus group discussions, and group interviews. The basic principles of analyzing qualitative data were applied. In particular, the processing of data for each facility made use of a desktop matrix analysis of themes drawn from both the conceptual framework and others emerging from transcripts. A comparison of these qualitative data across facilities enabled the research team to identify trends across facilities and to interpret the findings.