Health Results Based Financing Impact Evaluation 2018, Health Facility Endline Survey
Tajikistan, 2018
Get Microdata
Reference ID
TJK_2018_HRBFIE-FEL_v01_M
Producer(s)
Gil Shapira, Damien de Walque
Collection(s)
Metadata
Created on
Nov 12, 2019
Last modified
Nov 12, 2019
Page views
76853
Downloads
1532
Identification
Survey ID Number
TJK_2018_HRBFIE-FEL_v01_M
Title
Health Results Based Financing Impact Evaluation 2018, Health Facility Endline Survey
Subtitle
Health Facility Endline Survey
Country/Economy
| Name | Country code |
|---|---|
| Tajikistan | TJK |
Study type
Impact Evaluation Survey
Series Information
The Health Results Innovation Trust Fund (HRITF) launched in 2007. It supports results-based financing (RBF) approaches in the health sector to improve maternal and child health around the world. The HRITF is supported by the Governments of Norway through Norad and the United Kingdom through the Department for International Development (DFID). It is administered by The World Bank.
As of September 2016, the total contributions to the trust fund were US$477.6 million equivalent from Norway and the United Kingdom. To date, HRITF has committed US$385.6 million for 35 PBF programs linked to US$2.0 billion in financing from IDA-the World Bank's fund for the poorest. In addition to the RBF programs, the trust fund also finances the evaluation of these programs and other supporting activities. Funded by the HRITF, the Tajikistan Health Results Based Financing Impact Evaluation Endline Survey was conducted in 2018.
The endline (follow-up) survey took place three years after project implementation, which launched in 2015. A baseline survey was conducted 2014-2015. The endline survey is largely based on the HRITF instruments that were modified to the Tajik and project context.
To learn more, please visit the RBF Health website (https://www.rbfhealth.org).
As of September 2016, the total contributions to the trust fund were US$477.6 million equivalent from Norway and the United Kingdom. To date, HRITF has committed US$385.6 million for 35 PBF programs linked to US$2.0 billion in financing from IDA-the World Bank's fund for the poorest. In addition to the RBF programs, the trust fund also finances the evaluation of these programs and other supporting activities. Funded by the HRITF, the Tajikistan Health Results Based Financing Impact Evaluation Endline Survey was conducted in 2018.
The endline (follow-up) survey took place three years after project implementation, which launched in 2015. A baseline survey was conducted 2014-2015. The endline survey is largely based on the HRITF instruments that were modified to the Tajik and project context.
To learn more, please visit the RBF Health website (https://www.rbfhealth.org).
Abstract
The 2018 endline survey of the impact evaluation (IE) for Health Performance-Based Financing (PBF) in Tajikistan sought to ascertain: (i) the impact and cost-effectiveness of the PBF model implemented in Tajikistan; and (ii) whether PBF is more effective or cost-effective if implemented in conjunction with additional low-cost interventions (Collaborative Quality Improvement, Citizen Report Cards). The results from the IE will help inform the Ministry of Health on whether PBF should be scaled-up to additional PHC level institutions in other regions.
The Collaborative Quality Improvement intervention responds to policy concerns that performance incentives may not produce the desired improvements if providers lack the necessary competencies to inform decisions and knowledge. The Citizen Report Card attempts to improve the effectiveness of PBF by strengthening the 'short route' of accountability (e.g., by increasing accountability of health facilities to their local constituents). Since PBF, collaborative quality improvement (CQI), and citizen report cards (CRC) have never been implemented on a large scale in Tajikistan, it is to be expected that the results from the IE will be useful for designing national PHC policy in Tajikistan, and that they will also contribute to the larger body of knowledge on these interventions.
The IE employs both difference-in-difference and experimental approaches to identify the impact of the different combinations of interventions. Assignment to PBF was not random. Three districts in the Sughd region and four districts in the Khatlon region were selected to implement the program. All Rural Health Centers (RHCs) in these seven districts are covered by the program. Nine additional districts (two in Sughd and seven in Khatlon) were selected as control districts. The selection of the control districts was guided by geographical proximity to treatment districts and similarity in terms of number of health facilities and doctors per capita. The districts were also selected such that the number of RHCs in treatment and control groups in each region would be similar.
Within the chosen 16 districts (treatment and control districts), clusters consisting of an RHC and its subsidiary Health Houses were randomly assigned to implement Collaborative Quality Improvement, Citizen Score Cards, or neither of these two interventions. The randomization was blocked by district. In sum, RHCs were assigned into six study arms.
The goal of the facility-based survey is to measure multiple dimensions of quality of care and collect detailed information on key aspects of facility functioning. Household surveys are primarily used to measure health service coverage at the population level as well as select health outcome indicators measured through anthropometry or tests. The surveys also collect broader data on the health of the households, health seeking behaviors and barriers to use of health services. In addition, PBF and other administrative data would be used to track outcomes over time in the treatment groups 1-3 (the ones receiving performance-based payments). The endline (follow-up) survey took place three years after project implementation. The survey is largely based on the HRITF instruments that were modified to the Tajik and project context.
The Collaborative Quality Improvement intervention responds to policy concerns that performance incentives may not produce the desired improvements if providers lack the necessary competencies to inform decisions and knowledge. The Citizen Report Card attempts to improve the effectiveness of PBF by strengthening the 'short route' of accountability (e.g., by increasing accountability of health facilities to their local constituents). Since PBF, collaborative quality improvement (CQI), and citizen report cards (CRC) have never been implemented on a large scale in Tajikistan, it is to be expected that the results from the IE will be useful for designing national PHC policy in Tajikistan, and that they will also contribute to the larger body of knowledge on these interventions.
The IE employs both difference-in-difference and experimental approaches to identify the impact of the different combinations of interventions. Assignment to PBF was not random. Three districts in the Sughd region and four districts in the Khatlon region were selected to implement the program. All Rural Health Centers (RHCs) in these seven districts are covered by the program. Nine additional districts (two in Sughd and seven in Khatlon) were selected as control districts. The selection of the control districts was guided by geographical proximity to treatment districts and similarity in terms of number of health facilities and doctors per capita. The districts were also selected such that the number of RHCs in treatment and control groups in each region would be similar.
Within the chosen 16 districts (treatment and control districts), clusters consisting of an RHC and its subsidiary Health Houses were randomly assigned to implement Collaborative Quality Improvement, Citizen Score Cards, or neither of these two interventions. The randomization was blocked by district. In sum, RHCs were assigned into six study arms.
The goal of the facility-based survey is to measure multiple dimensions of quality of care and collect detailed information on key aspects of facility functioning. Household surveys are primarily used to measure health service coverage at the population level as well as select health outcome indicators measured through anthropometry or tests. The surveys also collect broader data on the health of the households, health seeking behaviors and barriers to use of health services. In addition, PBF and other administrative data would be used to track outcomes over time in the treatment groups 1-3 (the ones receiving performance-based payments). The endline (follow-up) survey took place three years after project implementation. The survey is largely based on the HRITF instruments that were modified to the Tajik and project context.
Kind of Data
Sample survey data [ssd]
Unit of Analysis
Health centers,
Health workers,
Patients (adults & children)
Patient household
Health workers,
Patients (adults & children)
Patient household
Version
Version Description
- v01: Edited, anonymous datasets for public distribution.
Scope
Notes
The scope of the Tajikistan Health Results Based Financing Impact Evaluation 2018 - Health Facility Endline Survey includes:
- Health Facilities: Assessment of key aspects of facility functioning and structural aspects of quality of care such as - facility staffing; infrastructure and equipment; availability of drugs, consumables and supplies; supervision; record keeping and reporting to Health Management Information System; Service volumes
- Health Workers: Roles, responsibilities and characteristics of the interviewed health worker; staff satisfaction and motivation, technical knowledge on maternal child health, and non-communicable diseases.
- Patient-Provider Interactions: assessment of adherence to protocols in terms of IMCI and hypertension management.
- Patient Exit Interviews: patients' perceived quality of care and satisfaction with the care given; socio-economic background, and the general health of the patient.
- Main Household Questionnaire: socio-demographic characteristics, income, transfers, assets, housing, consumption of food and other items, migration of household members in and out of the country, mortality, utilization of health care, and blood pressure measurements for all adults over 18 years.
- Women of Reproductive Age: general health status, pregnancy history, reproductive health, utilization of family planning methods, antenatal care, deliveries, postnatal care, vaccination of children under 5 years, and anthropometric measures of the children under 5 years.
- Adults Over 40: general health status, health-related behaviors (e.g., physical activity, smoking, and alcohol consumption), health care seeking, high cholesterol, and other health conditions.
- Health Facilities: Assessment of key aspects of facility functioning and structural aspects of quality of care such as - facility staffing; infrastructure and equipment; availability of drugs, consumables and supplies; supervision; record keeping and reporting to Health Management Information System; Service volumes
- Health Workers: Roles, responsibilities and characteristics of the interviewed health worker; staff satisfaction and motivation, technical knowledge on maternal child health, and non-communicable diseases.
- Patient-Provider Interactions: assessment of adherence to protocols in terms of IMCI and hypertension management.
- Patient Exit Interviews: patients' perceived quality of care and satisfaction with the care given; socio-economic background, and the general health of the patient.
- Main Household Questionnaire: socio-demographic characteristics, income, transfers, assets, housing, consumption of food and other items, migration of household members in and out of the country, mortality, utilization of health care, and blood pressure measurements for all adults over 18 years.
- Women of Reproductive Age: general health status, pregnancy history, reproductive health, utilization of family planning methods, antenatal care, deliveries, postnatal care, vaccination of children under 5 years, and anthropometric measures of the children under 5 years.
- Adults Over 40: general health status, health-related behaviors (e.g., physical activity, smoking, and alcohol consumption), health care seeking, high cholesterol, and other health conditions.
Coverage
Geographic Coverage
Three districts in the Sughd region and four districts in the Khatlon region were selected to implement the program. All Rural Health Centers in these seven districts are covered by the program. Nine additional districts (two in Sughd and seven in Khatlon) were selected as control districts. The selection of the control districts was guided by geographical proximity to treatment districts and similarity in terms of number of health facilities and doctors per capita. The districts were also selected such that the number of RHCs in treatment and control groups in each region would be similar.
Producers and sponsors
Primary investigators
| Name | Affiliation |
|---|---|
| Gil Shapira | The World Bank - DECHD |
| Damien de Walque | The World Bank - DECHD |
Producers
| Name | Affiliation | Role |
|---|---|---|
| Tashrik Ahmed | Consultant | Field coordinator/research assistant |
| Huihui Wang | The World Bank - GHN03 | |
| Aneesa Arur | The World Bank - GHN02 | |
| Jeanette Walldorf | Consultant |
Funding Agency/Sponsor
| Name | Abbreviation |
|---|---|
| Health Results Innovation Trust Fund | HRITF |
Other Identifications/Acknowledgments
| Name | Affiliation |
|---|---|
| Mahbuba Mstafaevna | Tajikistan Ministry of Health |
| Fatima Gaibova | Tajikistan Ministry of Health |
| Saodat Mirsaburova | Tajikistan Ministry of Health |
| Wezi Msisha | The World Bank |
| Ha Ngyuyen | The World Bank |
| Kate Mandeville | The World Bank |
| Mutriba Latypova | The World Bank |
| Sabrina Qandenova | The World Bank |
| Sarvinoz Barfieva | The World Bank |
| Rouselle Lavado | The World Bank |
Sampling
Sampling Procedure
The major features of the sampling procedure include the following steps (they are discussed in more detail in a copy of the study's report located in "External Resources"):
Health Facilities:
1. Table 6-4 in the study's report presents the number of RHCs selected for the sample for each district. Of the 216 RHC selected for the sample (after randomly excluding some RHCs when the total was not divisible by three), 151 have subsidiary HHs. Forty-three HHs were selected of the sample in Sughd and 107 in Khatlon.
2. While some Rural Health Centers have one or more subsidiary Health Houses in their catchment areas, others do not have any. One Health House from each RHC with subsidiary HHs was to be included in the sample. The selection was random with each health house within a cluster having an identical probability of being chosen. Non-selected health houses were ranked to serve as replacements if the survey cannot be implemented in the selected HHs.
Households:
1. The evaluation relies on two samples of households. As the primary focus of the PBF intervention is on Maternal and Child Health (MCH) services, the main household sample is of households with women who experienced a recent pregnancy. This sample would not be appropriate to study the impact on the coverage of services related to Non-Communicable Diseases (NCD). Therefore, a second sample consists of households with individuals over the age of 40. The household samples are clustered according to the catchment area of each Rural Health Center (and its affiliated health houses).
2. The resulting targeted primary household sample size is of 4,320 households, with 20 in each of the 216 clusters in the six study arms. To be eligible to be included in the household survey sample, households must have had at least one woman aged 15-49 years who has had a child in the preceding three years. The same villages were covered for both the baseline and followed up survey and eligibility was determined at each round by a listing exercise.
3. The resulting targeted sample size for the secondary household sample is 1,584 households, with 22 in each of 72 clusters in two of the six study arms. Eligibility for this sample is determined by an individual over the age of 40 in the household. Eligibility for the two samples is determined by a common listing of households in selected villages. Households which satisfy both eligibility criteria can be randomly selected to count towards the sample size requirements for both.
4. A two-stage cluster sampling methodology was employed to identify random samples. First, villages were randomly selected out of a list of the villages served by each facility. The list was obtained from the MoH. RHCs have either single or multiple villages in their catchment areas while HHs typically serve a single village. If an RHC has at least one affiliated HH, then two villages were selected. One village was directly served by the RHC while the other included in the sub-catchment area of the HH. In each village, 100 households were listed. If the village had over 100 households, a random walk method was used to select the target number. A short questionnaire was conducted at each household to determine households' eligibility for the two samples. From all eligible households, the target sample for each catchment area was selected. In catchment areas in which two villages were included in the sample, half of the households were to be selected from each village.
Health Facilities:
1. Table 6-4 in the study's report presents the number of RHCs selected for the sample for each district. Of the 216 RHC selected for the sample (after randomly excluding some RHCs when the total was not divisible by three), 151 have subsidiary HHs. Forty-three HHs were selected of the sample in Sughd and 107 in Khatlon.
2. While some Rural Health Centers have one or more subsidiary Health Houses in their catchment areas, others do not have any. One Health House from each RHC with subsidiary HHs was to be included in the sample. The selection was random with each health house within a cluster having an identical probability of being chosen. Non-selected health houses were ranked to serve as replacements if the survey cannot be implemented in the selected HHs.
Households:
1. The evaluation relies on two samples of households. As the primary focus of the PBF intervention is on Maternal and Child Health (MCH) services, the main household sample is of households with women who experienced a recent pregnancy. This sample would not be appropriate to study the impact on the coverage of services related to Non-Communicable Diseases (NCD). Therefore, a second sample consists of households with individuals over the age of 40. The household samples are clustered according to the catchment area of each Rural Health Center (and its affiliated health houses).
2. The resulting targeted primary household sample size is of 4,320 households, with 20 in each of the 216 clusters in the six study arms. To be eligible to be included in the household survey sample, households must have had at least one woman aged 15-49 years who has had a child in the preceding three years. The same villages were covered for both the baseline and followed up survey and eligibility was determined at each round by a listing exercise.
3. The resulting targeted sample size for the secondary household sample is 1,584 households, with 22 in each of 72 clusters in two of the six study arms. Eligibility for this sample is determined by an individual over the age of 40 in the household. Eligibility for the two samples is determined by a common listing of households in selected villages. Households which satisfy both eligibility criteria can be randomly selected to count towards the sample size requirements for both.
4. A two-stage cluster sampling methodology was employed to identify random samples. First, villages were randomly selected out of a list of the villages served by each facility. The list was obtained from the MoH. RHCs have either single or multiple villages in their catchment areas while HHs typically serve a single village. If an RHC has at least one affiliated HH, then two villages were selected. One village was directly served by the RHC while the other included in the sub-catchment area of the HH. In each village, 100 households were listed. If the village had over 100 households, a random walk method was used to select the target number. A short questionnaire was conducted at each household to determine households' eligibility for the two samples. From all eligible households, the target sample for each catchment area was selected. In catchment areas in which two villages were included in the sample, half of the households were to be selected from each village.
Response Rate
Health Facilities:
Of 216 RHCs selected for the impact evaluation, 210 were evaluated at both baseline and follow-up. Six RHCs evaluated at baseline were ineligible for selection at follow-up due to closure or re-registration (either upgraded to a district health center or downgraded to health house). These six RHCs and their respective health house and household enumeration areas were replaced before the start of the follow-up survey. A total of 151 health houses were assessed at baseline, and 150 at follow-up. Eleven health houses were close or re-registered as RHCs. Our analyses treat RHCs and health houses as panel data, where it is assumed the observed facility is measured at both time points. Therefore, both the original units which have been replaced and the replacement are excluded in the subsequent difference-in-difference and cross-sectional analyses.
Health Workers:
A total of 1,574 health workers were surveyed in the RHCs included in the analysis sample, 767 at baseline and 807 at follow-up. The average number of health workers fell slightly below the 4 per RHC target, as more remote RHCs did not have four staff members available. In health houses, the two staff per HH was achieved in the baseline sample but narrowly missed in the follow-up survey. Health workers who worked in both the rural health center and health house were treated as RHC employees.
Households:
A total of 10,599 households were surveyed across 230 villages in 210 RHC catchment areas, 4910 at baseline and 5689 during follow-up covering 83,803 household members. Within the two targeted populations, 7048 women 15-49 years of age with a pregnancy in the past three years, and 17,583 adults 40 years or older were surveyed.
Of 216 RHCs selected for the impact evaluation, 210 were evaluated at both baseline and follow-up. Six RHCs evaluated at baseline were ineligible for selection at follow-up due to closure or re-registration (either upgraded to a district health center or downgraded to health house). These six RHCs and their respective health house and household enumeration areas were replaced before the start of the follow-up survey. A total of 151 health houses were assessed at baseline, and 150 at follow-up. Eleven health houses were close or re-registered as RHCs. Our analyses treat RHCs and health houses as panel data, where it is assumed the observed facility is measured at both time points. Therefore, both the original units which have been replaced and the replacement are excluded in the subsequent difference-in-difference and cross-sectional analyses.
Health Workers:
A total of 1,574 health workers were surveyed in the RHCs included in the analysis sample, 767 at baseline and 807 at follow-up. The average number of health workers fell slightly below the 4 per RHC target, as more remote RHCs did not have four staff members available. In health houses, the two staff per HH was achieved in the baseline sample but narrowly missed in the follow-up survey. Health workers who worked in both the rural health center and health house were treated as RHC employees.
Households:
A total of 10,599 households were surveyed across 230 villages in 210 RHC catchment areas, 4910 at baseline and 5689 during follow-up covering 83,803 household members. Within the two targeted populations, 7048 women 15-49 years of age with a pregnancy in the past three years, and 17,583 adults 40 years or older were surveyed.
Data Collection
Dates of Data Collection
| Start | End | Cycle |
|---|---|---|
| 2018-03-01 | 2018-07-31 | Follow-up |
Data Collection Mode
Computer Assisted Personal Interview [capi]
Data Collection Notes
The Committee on Ethics of the Ministry of Health and Social Protection reviewed the study design, fieldwork protocols and the instruments and granted ethical clearance for the study on October 24th, 2014. The Health Results Innovation Trust Fund survey instruments were adapted to the Tajik context. Additional modules were developed to correspond to the project focus on NCD outcomes on top of the MCH outcomes. Representatives of the Ministry of Health, Zerkalo, and members of the World Bank team conducted several field visits to health facilities to inform the adaptation of the survey instruments. The instruments were pretested three times between August and October 2014, before the training of the field team. A local firm, Zerkalo, was selected through an international competitive procurement process to manage all aspects of the data collection. Figure 6.2., in the attached report located in “external resources”, summarizes and illustrates how data collection was organized for the baseline and endline surveys. Identical instruments were used with few adjustments.
Data Collectors
| Name | Affiliation |
|---|---|
| Zerkalo Analytics | Consultant |
Questionnaires
Questionnaires
The Tajikistan Health Results Based Financing Impact Evaluation 2018 - Health Facility Endline Survey includes the following 7 questionnaires.
Facility-Based Surveys:
1. Health facility assessment module
2. Health worker interview module
3. Observation of patient-provider interaction module
4. Patient exit interview modules
Household Survey:
5. Main household questionnaire
6. Women of reproductive age interview questionnaire
7. Adults over 40 years old questionnaire
Facility-Based Surveys:
1. Health facility assessment module
2. Health worker interview module
3. Observation of patient-provider interaction module
4. Patient exit interview modules
Household Survey:
5. Main household questionnaire
6. Women of reproductive age interview questionnaire
7. Adults over 40 years old questionnaire
Access policy
Confidentiality
Users of the data agree to keep confidential all data contained in these datasets and to make no attempt to identify, trace or contact any individual whose data is included in these datasets.
Access conditions
- Licensed datasets, accessible under conditions
Citation requirements
Use of the dataset must be acknowledged using a citation which would include:
- the Identification of the Primary Investigator
- the title of the survey (including country, acronym and year of implementation)
- the survey reference number
- the source and date of download
Example,
Shapira, Gil., Damien de Walque. 2018. Tajikistan Health Results Based Financing Impact Evaluation 2018, Health Facility Endline Survey (HRBFIE-FEL 2018). Ref. TJK_2018_HRBFIE-FEL_01_M. The World Bank. Dataset downloaded from [URL] on [date].
- the Identification of the Primary Investigator
- the title of the survey (including country, acronym and year of implementation)
- the survey reference number
- the source and date of download
Example,
Shapira, Gil., Damien de Walque. 2018. Tajikistan Health Results Based Financing Impact Evaluation 2018, Health Facility Endline Survey (HRBFIE-FEL 2018). Ref. TJK_2018_HRBFIE-FEL_01_M. The World Bank. Dataset downloaded from [URL] on [date].
Access authority
| Name | Affiliation | |
|---|---|---|
| Gil Shapira | The World Bank | gshapira@worldbank.org |
Disclaimer and copyrights
Disclaimer
The user of the data acknowledges that the original collector of the data, the authorized distributor of the data, and the relevant funding agency bear no responsibility for use of the data or for interpretations or inferences based upon such uses.
Metadata production
DDI Document ID
DDI_TJK_2018_HRBFIE-FEL_v01_M_WB
Producers
| Name | Abbreviation | Affiliation | Role |
|---|---|---|---|
| Development Economics Data Group | DECDG | The World Bank | Documentation of the study |
Date of Metadata Production
2019-09-12
DDI Document version
Version 01 (September 2019).