SABER Service Delivery 2017, Measuring Education Service Delivery
Afghanistan, 2017
Get Microdata
Reference ID
AFG_2017_SABER-SD_v01_M
Producer(s)
World Bank
Collection(s)
Metadata
Created on
Mar 04, 2019
Last modified
Jun 11, 2019
Page views
61305
Downloads
20390
Identification
Survey ID Number
AFG_2017_SABER-SD_v01_M
Title
SABER Service Delivery 2017, Measuring Education Service Delivery
Subtitle
Measuring Education Service Delivery
Translated Title
SABER Service Delivery, Afghanistan 2017
Country/Economy
| Name | Country code |
|---|---|
| Afghanistan | AFG |
Study type
Service Provision Assessments
Series Information
Systems Approach to Better Education Results - Service Delivery (SABER SD) consists of a collection of school-based surveys implemented globally. The SABER SD survey work began in 2016. So far, three countries are included in the series: Laos PDR, Afghanistan and the province of Punjab in Pakistan
Abstract
The SABER Service Delivery (SD) tool was developed in 2016 in the Global Engagement and Knowledge Unit of the Education Global Practice (GP) at the World Bank, as an initiative to uncover bottlenecks that inhibit student learning in low and middle-income countries and to better understand the quality of education service delivery in a country as well as gaps in policy implementation. This school survey is aligned to the latest education research on what matters for student learning and how best to measure it. Its main purpose is to provide a mechanism to assess these different determinants of learning through a diagnostic tool and also to uncover the extent to which policies translate into implementation and practice. As a global initiative, SABER SD provides data for the new global lead indicator on learning, which makes it easier to monitor the Sustainable Development Goal of achieving universal primary education.
The foundations of the SABER SD survey build upon two pre-existing World Bank Group initiatives that produce comparative data and knowledge on education systems: SABER (System Approach for Better Education Results) and SDI (Service Delivery Indicators). It also draws upon earlier surveys, namely, QSDS (Quality of Service Delivery Surveys) and PETs (Public Expenditure Tracking). On the one hand, the SABER SD tool builds on the evidence base and captures policy implementation measures from the core SABER domains. On the other hand, the SABER SD tool adapts and extends the surveys that were developed and implemented under the SDI program, which provides a set of metrics for benchmarking service delivery performance in education and health in Africa. There are two main factors that distinguish the SABER SD tool from SDI: (i) it has expanded the measurement of service delivery in primary education outside Africa and into Asia through pilots in Afghanistan, Pakistan and Laos, and (ii) it has adapted and extended the SDI approach by including additional test items from TIMSS and PIRLS, different modality for test administration, different classroom observation modules, and additional questions on student background.
In alignment with the World Development Report (WDR 2018), the SABER SD instrument collects information on the the four key elements of an education system, which are identified as the main determinants for student learning (teachers, school management, school infrastructure, student preparedness). This survey strategically collects information on the most important school inputs and processes that produce learning outcomes. The SABER SD instrument collects data at the school level and asks questions related to the roles of all levels of government (including local and regional). The tool provides comprehensive data on: teacher effort and ability; principal leadership; school governance, management, and finances; community participation; and student performance in Mathematics and Language which includes a classroom observation module.
The foundations of the SABER SD survey build upon two pre-existing World Bank Group initiatives that produce comparative data and knowledge on education systems: SABER (System Approach for Better Education Results) and SDI (Service Delivery Indicators). It also draws upon earlier surveys, namely, QSDS (Quality of Service Delivery Surveys) and PETs (Public Expenditure Tracking). On the one hand, the SABER SD tool builds on the evidence base and captures policy implementation measures from the core SABER domains. On the other hand, the SABER SD tool adapts and extends the surveys that were developed and implemented under the SDI program, which provides a set of metrics for benchmarking service delivery performance in education and health in Africa. There are two main factors that distinguish the SABER SD tool from SDI: (i) it has expanded the measurement of service delivery in primary education outside Africa and into Asia through pilots in Afghanistan, Pakistan and Laos, and (ii) it has adapted and extended the SDI approach by including additional test items from TIMSS and PIRLS, different modality for test administration, different classroom observation modules, and additional questions on student background.
In alignment with the World Development Report (WDR 2018), the SABER SD instrument collects information on the the four key elements of an education system, which are identified as the main determinants for student learning (teachers, school management, school infrastructure, student preparedness). This survey strategically collects information on the most important school inputs and processes that produce learning outcomes. The SABER SD instrument collects data at the school level and asks questions related to the roles of all levels of government (including local and regional). The tool provides comprehensive data on: teacher effort and ability; principal leadership; school governance, management, and finances; community participation; and student performance in Mathematics and Language which includes a classroom observation module.
Kind of Data
Sample survey data [ssd]
Unit of Analysis
The unit of analysis varies for each of the modules. They are as follows:
Module 1, the unit of analysis is the school.
Module 2, the unit of analysis is the teacher.
Module 3, the unit of analysis is the principal/school.
Module 4, the unit of analysis is the classroom/school/teacher.
Module 5, the unit of analysis is the student.
Module 6, the unit of analysis is the teacher.
For modules where the unit of analysis is not the school (i.e., teachers and/or students), it is possible to create an average for the school based on groupings by the unique identifier - the school code.
Module 1, the unit of analysis is the school.
Module 2, the unit of analysis is the teacher.
Module 3, the unit of analysis is the principal/school.
Module 4, the unit of analysis is the classroom/school/teacher.
Module 5, the unit of analysis is the student.
Module 6, the unit of analysis is the teacher.
For modules where the unit of analysis is not the school (i.e., teachers and/or students), it is possible to create an average for the school based on groupings by the unique identifier - the school code.
Version
Version Description
Latest version updated September 30, 2018
Scope
Notes
The scope of the survey includes information on the following education variables among others:
- School facility and infrastructure
- Teacher/Principal background
- Teacher/Principal absenteeism
- Teacher/Principal knowledge
- School management and accountability
- School finance
- Community participation
- Classroom observation
- Student background
- Student assessment
- Teacher assessment
Data was collected across the following strata: 5 regions (Central/ East/ North/ South/West); urban/rural; gender (boys schools/girls schools/coeducational schools)
- School facility and infrastructure
- Teacher/Principal background
- Teacher/Principal absenteeism
- Teacher/Principal knowledge
- School management and accountability
- School finance
- Community participation
- Classroom observation
- Student background
- Student assessment
- Teacher assessment
Data was collected across the following strata: 5 regions (Central/ East/ North/ South/West); urban/rural; gender (boys schools/girls schools/coeducational schools)
Coverage
Geographic Coverage
The Afghanistan SABER SD survey was implemented across 21 provinces in Afghanistan in 200 primary schools, of which 170 are public schools and the remaining 30 are Community Based Education (CBE) schools. The sample of 170 public schools is nationally representative, to the extent possible, of the places in Afghanistan that were secure enough for the teams to visit. Since it was not possible to obtain the sample frame from the universe of CBE schools, the sample of 30 CBE schools is not representative of the universe of CBE schools, but only of on-budget CBE schools. The Afghanistan SABER SD survey covers the following 21 provinces: Balkh, Faryab, Ghazni, Ghor, Hilmand, Hirat, Jawzjan, Kabul City, Kabul Province, Kandahar, Khost, Kunar, Logar, Nangarhar, Nuristan, Paktia, Parwan, Sar i Pul, Sarepul, Takhar and Wardak.
Universe
The target was to have a nationally representative sample. The team had a list of public schools covering just over 1.1 million students in 34 provinces. The final pool of primary schools from which the sample was drawn included those with a Grade 4 population of students.
Producers and sponsors
Primary investigators
| Name | Affiliation |
|---|---|
| World Bank | WBG |
Producers
| Name | Affiliation |
|---|---|
| Ezequiel Molina | World Bank |
| Iva Trako | World Bank |
| Anahita Hosseini Matin | World Bank |
| Eema Masood | World Bank |
| Mariana Viollaz | World Bank |
Funding Agency/Sponsor
| Name |
|---|
| SABER Trust Fund |
Other Identifications/Acknowledgments
| Name |
|---|
| Afghanistan Ministry of Education |
Sampling
Sampling Procedure
Using estimates of Grade 4 enrollment rates by gender, we used sampling with probability proportional to size for all public school sampling. For CBE schools, we did not have this information and therefore, this sample is not representative of the universe of CBE schools in Afghanistan.
First stage — provinces
Six provinces were sampled mechanically: three because of their political importance (Kabul City and Kabul Province in the Central region; Nangarhar in the East region); and three because they represented such a disproportionate fraction of their region that a PPS (probability proportional to size) strategy would always sample them anyway (Balkh in the North; Khost in the South; and Hirat in the West). Of the remaining 28 provinces, 15 were randomly sampled via stratified PPS, with strata simply defined as regions, yielding a total of 21 sampled provinces.
Second stage — public schools
Within the sampled provinces, we assigned public schools to strata defined by three characteristics: region (one of five); rural/urban; and gender (male, female, or coed). The gender category was defined empirically from enrollments reported in the sample frame: if the numbers of either males or females was zero or was very small in both absolute and proportional terms, we considered the school single-sex. If in either absolute or proportional terms neither sex dominated, we considered the school co-educational. We then did stratified PPS to sample 170 public schools, along with a number of replacements in case schools had been closed or the sample frame was in some other way erroneous.
Third stage — gender
Within the sampled mixed (“coed”) schools, we had to decide in advance whether to sample girls or boys. We set the overall fraction of these schools in which we would sample girls (equivalently, boys) to be equal to the overall fraction of the enrollment across all these mixed sex schools that girls (equivalently, boys) comprised. We then randomized so that at each school, the probability of girls (equivalently, boys) being sampled was roughly proportional to the fraction of that school’s enrollment that girls (equivalently, boys) comprised.
Fourth step — CBE schools
Six months after drawing the original sample, we received a final list of CBE schools in the relevant provinces. Unlike typical public data, and specifically unlike Afghanistan’s public school sample frame, this CBE school list did not include enrollments. We thus sampled 30 of these schools in numbers proportional to the number of CBE schools in each sampled province.
Fifth step — Revised security for CBE schools in Khost
A few weeks after drawing the CBE sample, it was revealed that only one in every six CBE schools in Khost province was sufficiently safe for the field teams to visit. This meant re-drawing the CBE sample in Khost among the small minority of schools that were safe enough to visit.
First stage — provinces
Six provinces were sampled mechanically: three because of their political importance (Kabul City and Kabul Province in the Central region; Nangarhar in the East region); and three because they represented such a disproportionate fraction of their region that a PPS (probability proportional to size) strategy would always sample them anyway (Balkh in the North; Khost in the South; and Hirat in the West). Of the remaining 28 provinces, 15 were randomly sampled via stratified PPS, with strata simply defined as regions, yielding a total of 21 sampled provinces.
Second stage — public schools
Within the sampled provinces, we assigned public schools to strata defined by three characteristics: region (one of five); rural/urban; and gender (male, female, or coed). The gender category was defined empirically from enrollments reported in the sample frame: if the numbers of either males or females was zero or was very small in both absolute and proportional terms, we considered the school single-sex. If in either absolute or proportional terms neither sex dominated, we considered the school co-educational. We then did stratified PPS to sample 170 public schools, along with a number of replacements in case schools had been closed or the sample frame was in some other way erroneous.
Third stage — gender
Within the sampled mixed (“coed”) schools, we had to decide in advance whether to sample girls or boys. We set the overall fraction of these schools in which we would sample girls (equivalently, boys) to be equal to the overall fraction of the enrollment across all these mixed sex schools that girls (equivalently, boys) comprised. We then randomized so that at each school, the probability of girls (equivalently, boys) being sampled was roughly proportional to the fraction of that school’s enrollment that girls (equivalently, boys) comprised.
Fourth step — CBE schools
Six months after drawing the original sample, we received a final list of CBE schools in the relevant provinces. Unlike typical public data, and specifically unlike Afghanistan’s public school sample frame, this CBE school list did not include enrollments. We thus sampled 30 of these schools in numbers proportional to the number of CBE schools in each sampled province.
Fifth step — Revised security for CBE schools in Khost
A few weeks after drawing the CBE sample, it was revealed that only one in every six CBE schools in Khost province was sufficiently safe for the field teams to visit. This meant re-drawing the CBE sample in Khost among the small minority of schools that were safe enough to visit.
Deviations from the Sample Design
Sampling in Afghanistan had several special features: high logistical costs, CBE schools, fluid security concerns, and a very specific type of gendered schools.
1. High Logistical costs. It was decided early in the process to sample a subset of provinces to be visited, since visiting every province would have implied high logistical costs that made little sense to incur for a relatively small number of schools.
2. CBE school sample frame. CBE schools were disproportionately important to SABER-SD (in relation to their actual numbers), so a portion of the sample was reserved for schools of this type. However, the sample frame for these non-public-operated schools came from disparate sources and could not be assembled until six months after the sample frame for the public schools. Since the team was unable to obtain the sample frame from the universe of CBE schools (on and off-budget), which would allow the proper random selection and representativeness, it is important to acknowledge that the sample of CBE schools in the SABER SD survey is not representative of all CBE schools, but only of on-budget CBE schools, which are managed by the Afghan Ministry of Education. The 2017 Afghanistan SABER SD surveyed 30 CBE schools: 6 located in the province of Ghor, 13 located in the province of Paktia and 11 located in the province of Khost.
3. Fluid security concerns. When the sampling process was completed, the team did not know which schools would be in areas safe enough to visit at the actual survey time. A few weeks after drawing the CBE sample, it was revealed that only one in every six CBE schools in the Khost province was sufficiently safe for the field teams to visit. This meant re-drawing the CBE sample in Khost among the small minority of schools that were safe enough to visit. Within these restrictions, the resulting CBE sample is representative of the parts of Afghanistan that were secure enough for teams to visit.
4. Gendered Schools. The fourth feature of this environment, gendered schools, meant that while there were both exclusively boys' schools and exclusively girls' schools, there were also co-educational schools that nonetheless kept classrooms sex-segregated. This implied that for each sampled public school, the team decided in advance to randomly sample a male or a female classroom.
1. High Logistical costs. It was decided early in the process to sample a subset of provinces to be visited, since visiting every province would have implied high logistical costs that made little sense to incur for a relatively small number of schools.
2. CBE school sample frame. CBE schools were disproportionately important to SABER-SD (in relation to their actual numbers), so a portion of the sample was reserved for schools of this type. However, the sample frame for these non-public-operated schools came from disparate sources and could not be assembled until six months after the sample frame for the public schools. Since the team was unable to obtain the sample frame from the universe of CBE schools (on and off-budget), which would allow the proper random selection and representativeness, it is important to acknowledge that the sample of CBE schools in the SABER SD survey is not representative of all CBE schools, but only of on-budget CBE schools, which are managed by the Afghan Ministry of Education. The 2017 Afghanistan SABER SD surveyed 30 CBE schools: 6 located in the province of Ghor, 13 located in the province of Paktia and 11 located in the province of Khost.
3. Fluid security concerns. When the sampling process was completed, the team did not know which schools would be in areas safe enough to visit at the actual survey time. A few weeks after drawing the CBE sample, it was revealed that only one in every six CBE schools in the Khost province was sufficiently safe for the field teams to visit. This meant re-drawing the CBE sample in Khost among the small minority of schools that were safe enough to visit. Within these restrictions, the resulting CBE sample is representative of the parts of Afghanistan that were secure enough for teams to visit.
4. Gendered Schools. The fourth feature of this environment, gendered schools, meant that while there were both exclusively boys' schools and exclusively girls' schools, there were also co-educational schools that nonetheless kept classrooms sex-segregated. This implied that for each sampled public school, the team decided in advance to randomly sample a male or a female classroom.
Response Rate
100% response rate from all the 200 schools in the sample. No reserve schools were activated.
Weighting
The final weighting was conducted at the school level.
Data Collection
Dates of Data Collection
| Start | End |
|---|---|
| 2017 | 2017 |
Time periods
| Start date | End date |
|---|---|
| 2017 | 2017 |
Data Collection Mode
Computer Assisted Personal Interview [capi]
Supervision
Fieldwork supervision was conducted by Rahman Safi International (RSI) and the World Bank team based in both Kabul, Afghanistan and Washington, D.C.
Data Collection Notes
The SABER SD survey was carried out in Afghanistan from April to August 2017. The SABER SD team worked closely with the local survey firm (Rahman Safi International (RSI), who was hired to implement the field survey. The team coordinated with RSI throughout the sates of the survey including survey preparation, data collection, managing the quality assurance of the survey process and data entry. The questions in each of the six SABER SD modules were reviewed carefully prior to the training of enumerator teams and adapted to the Afghanistan context. These were further revised after the field testing to ensure that they could be easily understood and the intended meaning could be communicated accurately in the Dari Language to all interviewers. The field data collection was conducted from the months of April to August 2017 in 200 schools across 21 provinces in Afghanistan.
Data Collectors
| Name | Abbreviation |
|---|---|
| Rahman Safi International | RSI |
Data Appraisal
Estimates of Sampling Error
Estimated sampling error: 3.5% (proportionally sampled) and 3.9% (actual with oversampling).
Access policy
Citation requirements
Use of the dataset must be acknowledged using citation which would include:
-the identification of the Primary Investigator
-the title of the survey (including country, acronym and year of implementation)
-the survey reference number
-the source and date of download
Example
World Bank. SABER Service Delivery, Afghanistan 2017. Ref. AFG_2017_SABER-SD_v01_M. Dataset downloaded from [URL] on [date].
In addition, if you use the dataset for published papers, it would greatly help us maintain a citation database if you would inform the primary investigators.
-the identification of the Primary Investigator
-the title of the survey (including country, acronym and year of implementation)
-the survey reference number
-the source and date of download
Example
World Bank. SABER Service Delivery, Afghanistan 2017. Ref. AFG_2017_SABER-SD_v01_M. Dataset downloaded from [URL] on [date].
In addition, if you use the dataset for published papers, it would greatly help us maintain a citation database if you would inform the primary investigators.
Disclaimer and copyrights
Disclaimer
The user of the data acknowledges that the original collector of the data, the authorized distributor of the data, and the relevant funding agency bear no responsibility for use of the data or for interpretations or inferences based upon such uses.
Metadata production
DDI Document ID
DDI_AFG_2017_SABER-SD_v01_M_WB
Producers
| Name | Abbreviation | Affiliation | Role |
|---|---|---|---|
| Development Economics Data Group | DECDG | The World Bank | Documentation of the DDI |
Date of Metadata Production
2019-01-23
DDI Document version
Version 01 (2019)