Quality for Preschool Impact Evaluation 2017, Endline Survey
Ghana, 2017
Get MicrodataIdentification
GHA_2017_QPIE-EL_v01_M
Quality for Preschool Impact Evaluation 2017
Endline Survey
| Name | Country code |
|---|---|
| Ghana | GHA |
Impact Evaluation Study
Baseline Survey: 2015
Midline Survey: 2016
Endline Survey: 2017
The project seeks to increase the quality of preschool education during the two years of universal Kindergarten (KG) in Ghana through intervening in the supply-side (i.e., teacher in-service training) and the demand side (i.e., increasing parental awareness for developmentally appropriate quality early education).
The primary goal of the impact evaluation is to test the efficacy of a potentially scalable (8-day) in-service teacher training to improve the quality of KG teacher practices and interactions with children and to improve children’s development, school readiness and learning in both private and public preschools in the Greater Accra Region of Ghana. Additional goals of this evaluation are: to test the added value of combining a scalable (low-cost) parental awareness intervention with teacher in-service training; to compare implementation challenges in public and private schools; and to examine several important sources of potential heterogeneity of impact, primarily impacts in public vs. private schools.
The current submission is for the Endline Survey, conducted with 3 types of respondents across two phases – School survey and Caregiver [household] surveys. The school survey was conducted from February to March 2017 and consisted of collecting the following data: (a) direct assessments of children’s school readiness, (b) surveys of KG teachers, (c) direct observation of inventory of facilities within KG classrooms [environmental scan]; videotaping of KG classroom processes, teaching, and learning (not being submitted); as well as video coding of KG classroom video recordings using Teacher Instructional Practices and Processes Systems (instrument not being submitted). The caregiver survey was conducted via phone from May to July August 2017 on the primary caregivers of the KG children. The caregiver survey sought information on caregivers’ background, poverty status, involvement or participation in school and home activities, and perception about ECD.
Sample survey data [ssd]
Units of analysis include individuals (KG teachers, children, caregivers), KG classrooms and preschools.
Version
Version 1: Edited, anonymous dataset for public distribution. All Personally Identifying Information (PII) has been removed.
2017-12-15
The data provided is the final version with PII removed.
Scope
The scope of the QP4G Endline survey include:
Child Direct Assessment: Socio-emotional skills, early literacy skills, early numeracy skills, motor skills, and executive function skills; and assessor’s reported ratings.
KG teacher survey: KG teachers background, poverty status, household food security situation, perceptions about ECD, participation in in-service training, work conditions, teacher well-being, and teaching knowledge.
Classroom observation: Direct observation of inventory of facilities within KG classrooms; videotaping of KG classroom processes, teaching, and learning (not being submitted); video coding of KG classroom video recordings using Teacher Instructional Practices and Processes Systems (instrument not being submitted).
School attendance: Child name and unique ID, school term, number of total attendance per term, number of days of absence per term.
Caregiver survey: Caregivers’ background, poverty status, involvement or participation in school and home activities, and perception about ECD.
Coverage
Urban and Peri-Urban Districts, Greater Accra Region
District
The survey universe is 6 poor districts in the Greater Accra Region. We sampled 240 schools, 108 public (Govt.) schools and 132 private schools. The population of interest is KG teachers and children in KG 1 and KG 2 classrooms in these schools, as well as the caregivers of sampled students.
Producers and sponsors
| Name | Affiliation |
|---|---|
| Sharon Wolf | University of Pennsylvania |
| John Lawrence Aber | New York University |
| Jere Behrman | University of Pennsylvania |
| Name | Role |
|---|---|
| Innovations for Poverty Action | Technical assistance in questionnaire design, sampling methodology, data collection and data processing |
| Name |
|---|
| Strategic Impact Evaluation Fund |
| Early Learning Partnership, World Bank |
| UBS Optimus Foundation |
| Name | Role |
|---|---|
| Ghana Education Service | Government Support |
Sampling
This impact evaluation applies a cluster-randomized design. Eligible schools were randomly selected to participate in the study. The eligible population was schools with KG 1 and KG 2 classrooms (the two years of universal preprimary education) in six districts in the Greater Accra Region. In these six districts, we have sampled 240 schools; 108 public schools and 132 private schools in total.
The unit of randomization for this randomized control trial (RCT) is schools, whereby eligible schools (stratified by public and private sector schools) are randomly assigned to: (1) in-service teacher-training program only; (2) in-service teacher-training program plus parental awareness program; or (3) control (current standard operating) condition.
The sampling frame for this study was based on data in the Education Management Information System (EMIS) from the Ghana Education Service. This data was verified in a 'school listing exercise' conducted in May 2015.
Sample selection was done in four stages:
The first stage involved purposive selection of six districts within the region based on two criteria: (a) most disadvantaged (using UNICEF's District League Table scores, out of sixteen total districts); and (b) close proximity to Accra Metropolitan for travel for the training of the KG teachers. The six selected municipals were La Nkwantanang-Madina Municipal, Ga Central Municipal, Ledzokuku-Krowor Municipal, Adentan Municipal, Ga South Municipal and Ga East Municipal.
The second stage involved the selection of public and private schools from each of the selected districts in the Accra region. We found 678 public and private schools (schools with kindergarten) in the EMIS database. Of these 361 schools were sampled randomly (stratified by district and school type) for the school listing exercise, done in May 2015. This was made up of 118 public schools and 243 private schools. The sampling method used for the school listing exercise was based on two approaches depending on the type of school. For the public schools, the full universe of public schools (i.e., 118) were included in the school listing exercise. However, private schools were randomly sampled using probability proportional to the size of the private schools in each district. Specifically, the private schools were sampled in each district proportionate to the total number of district private schools relative to the total number of private schools. In so doing, one school from the Ga South Municipal was removed and added to Ga Central so that all districts have a number of private schools divisible by three. This approach yielded 122 private schools. Additionally, 20 private schools were randomly selected from each of the districts (i.e., based on the remaining list of private schools in each district following from the first selection) to serve as replacement lists. The replacement list was necessary given the potential refusals from the private schools. There were no replacement lists for the public schools since all public schools would automatically qualify for participation.
The third stage involved selecting the final sample for the evaluation using the sampling frame obtained through the listing exercise. A total of 240 schools were randomly selected, distributed by district and sector. Schools were randomized into treatment groups after the first round of baseline data collection was completed.
In the final stage, the survey respondents were sampled using different sampling techniques:
a. KG teachers: The research team sampled two KG teachers from each school; one from KG1 and KG2. KG teachers were sampled using purposive sampling method. In schools where there were more than two KG classes, the KG teachers from the "A" stream were selected. For the treatment schools, all KG teachers were invited to participate in the teacher training program.
b. KG child-caregiver pair: The research team sampled KG children and their respective caregivers using simple random sampling method. Fifteen KG children-caregivers pair were sampled from each school. For schools with less than 15 KG children (8 from KG1, 7 from KG2 where possible), all KG children were included in the survey. KG children were selected from the same class as the selected KG teacher. The survey team used the class register to randomly select KG children who were present on the day of the school visit. Sampling was not stratified by gender or age. The caregivers of these selected child respondents were invited to participate in the survey.
The research team sought informed consent from the school head teacher, caregivers, as well as child respondents.
Schools: Of the 240 randomly sampled schools at baseline, 235 were successfully interviewed during both Midline and Endline. This yielded a school response rate of 98 percent. Only 2 percent of the randomly selected schools did not participate in the study at Midline and Endline. These included three schools that declined to participate in the study and 2 schools that were closed down due to land litigation issues.
KG teachers: At Baseline, 444 preschool teachers were randomly selected for the QP4G study. Of these, 348 teachers, representing 78 percent, were successfully interviewed at Midline. However, 302 out of the 348 teachers were successfully interviewed at Endline for a teacher response rate of 88 percent. Teacher attrition was largely due to the movement/transfer of teachers from school/preschool classes and teachers leaving the teaching profession.
KG children: The evaluation team randomly sampled 3435 KG children at Baseline for the QP4G study. Of these, 2657, representing 89 percent of the 2975 KG children were successfully assessed at Endline. The attrition rate among the children was largely due to the change in schools of children because of family migration outside the catchment districts or region.
Caregivers: Caregiver participation in the QP4G study increased from 2134 at Baseline to 2710 at Midline. This represented a 27 percent increase in the caregiver coverage over the baseline. The increase in the caregiver coverage was largely due to the 576 additional active contact numbers obtained from the caregivers. These caregivers, hitherto, had no contact numbers at baseline. Of the 2710 caregivers interviewed at Midline, 1734 were successfully interviewed at Endline for a caregiver response rate of 64 percent. Caregiver attrition rate was due to non-working contact numbers and refusal to participate in the study.
No weights were used in the analysis.
Survey instrument
Data were collected at Endline Survey using structured questionnaires or forms.
Child Assessment: Child Assessment was conducted using International Development and Early Learning Assessment [IDELA] tool designed by Save the Children. IDELA was adapted based on extensive pre-testing and piloting by different members of the evaluation team. The adapted version measured five indicators of ECD. The indicators were early numeracy skills, language/literacy skills and development, physical well-being and motor development, socio-emotional development, and approaches to learning. IDELA contained 28 items. In addition, one task was added – the Pencil Tap – to assess executive function skills. IDELA was translated into three local languages, namely, Twi, Ga, and Ewe. These local language versions had gone through rigorous processes of translation and back translation. No change was made to the IDELA used.
Environmental Scan: The Environmental Scan tool was designed to take inventories of the facilities in the KG classrooms. No changes have been made to the Endline version of the KG Class Environmental Scan. The class environmental scan also included a video recording of the KG classroom processes and systems.
TIPPS: The video recordings were coded using an early childhood education adapted version of TIPPS. Seidman, Raza, Kim, and McCoy (2014) of New York University developed the TIPPS instrument. TIPPS observes nineteen key concepts of teacher practices and classroom processes that influence children’s cognitive and social-emotional development. The concept sheet was used to code the kindergarten classroom videos.
Teacher Survey: The Teacher Survey measured KG teachers’ attitudes, behaviors, and perceptions on their background, nature and work conditions, depression and anxiety, external control, motivation, job satisfaction, burnout, and perceptions of early childhood education.
School Attendance Records: The School Attendance Records Form was designed to record school attendance information for the sampled KG teachers and children. The School Attendance Records form captured school-specific attendance details such as the active number of school days, the number of national/school-related holidays per term, and child-specific information such as present/absent from school.
Caregiver Survey: The Caregiver Survey was used to solicit information from primary caregivers regarding their background, food insufficiency, household wealth, parent’s involvement with child’s education, perceptions of early childhood development, and child discipline. Three more questions were added to the household wealth to capture the wage and household income of the primary caregivers. These questions were added to aid in the cost analysis of the parental engagement intervention. Four local language dictionaries of keywords and phrases were developed for the Caregiver Survey. The selected languages were Ga, Ewe, Twi, and Hausa.
Questionnaires are provided under the Related Materials tab.
Data collection
| Start | End | Cycle |
|---|---|---|
| 2017-02-06 | 2017-03-31 | All School surveys (teacher survey, child assessment, classroom observation) |
| 2017-05-22 | 2017-07-07 | Caregiver Survey |
| Name |
|---|
| Innovations for Poverty Action, Ghana |
Five different data collection teams were formed for the various Endline surveys. The teams conducted child assessment, teacher interviews, and classroom observation, video coding, school attendance records taking, and caregiver phone survey.
Data collection activities, namely, video coding, school attendance records taking, and caregiver phone survey were done in a centralized location while the others were done in the selected schools.
The school survey activities had seven field teams. Each field team comprised 1 team leader, 3 child assessors, and 1 teacher interviewer/videographers. Two field supervisors managed the field teams: each field supervisor was assigned 3 - 4 field teams. One auditor audited the teacher surveys.
Technically, each school survey team was assigned one school to complete a day. However, in order to ensure team productivity, the field teams were tasked to visit more than one school a day if the number of children in originally assigned school for a particular day was less than 15.
The caregiver phone survey had three auditors; two for auditing and one or sending top-ups to the respondents as a gift. There 12 phone interviewers, each completing 8 caregiver surveys per day.
Each phase of the data collection activity was monitored through field visits. The monitoring team covered IPA’s Research Manager, Research Associate, Field Manager, and Survey Coordinator. Team leaders and field supervisors were also involved in monitoring their assigned team(s). With the exception of the caregiver survey and the video coding activities, field visits were done through accompaniment and spot checks. On-site observations were done for the caregiver survey and video coding activities since these activities were conducted in a central location (i.e., IPA office). The field visit was done to establish whether the protocols were followed and to assess the performance of the field staff. Each field staff was visited and observed multiple times during the data collection period by different monitors. Specific monitoring protocols, namely, IDELA Monitoring Form, Teacher Interviewers Monitoring Form, and Video Quality Form were used (these instruments are not shared).
Specialized data collection teams were hired for each specific data collection activity – child direct assessment, teacher survey, caregiver survey, school attendance records, and classroom observation. Each of these unique data collection teams was provided specialized training based on the focus of the data collection activity, content of the data collection instrument and specific protocols relating to the particular data collection activity. Training of each of these specialized teams included classroom training and school visits or field practice.
Data were collected using different means of administration. Child direct assessment and teacher survey were conducted using in-person interviews through computer-assisted personal interview. The child direct assessment and teacher survey lasted approximately 45 minutes. The caregiver survey was administered via phone using computer-assisted telephone interview. The caregiver survey duration was approximately 50 minutes. This includes a screening of caregivers to determine their eligibility for the interview. Classroom observation was done by physically observing the inventory of facilities in the facilities in the KG classrooms (environmental scan) and videotaping classroom processes and systems. These videos were then coded in a centralized location by trained and certified video coders. School attendance records involved the review of pupil attendance register and teacher attendance book and translating specific information on a number of days present and/or absent from school per term. This activity took about 1 hour per each KG class for the selected children and teachers.
To gain access to the selected schools, prior approval was sought from the Regional Ghana Education Service and the District Education Directors for the selected districts. The research team sought informed consent from the school head teachers, teachers, and caregivers. Each child’s assent was sought before conducting the child direct assessment.
Teacher survey was administered strictly in the English language. Child direct assessment and caregiver survey were administered in the particular language the respondent understands and could speak fluently. The key languages were English, Twi, Ga, and Ewe. The child direct assessment was fully translated and back-translated in the local languages used for the assessment. Key stock of words or dictionary regarding specific variables were developed for the caregiver survey.
After each data collection activity, a debriefing was conducted to learn about the data collection procedures, instrument administration, logistics, challenges and lessons learned during the particular data collection activity.
Unlike the school attendance instrument that was fully piloted, the other data collection instruments were partially piloted based on the modifications done to the baseline instruments.
Data processing
Data consistency checks namely high-frequency checks and backchecks (audits) were conducted for all surveys remotely. Corrections were made during and after data collection after errors were reconciled.
All checks and cleaning was done using STATA and IPA data management systems. IPA possesses all the relevant code.
Data Access
Licensed datasets, accessible under conditions and following review [until the manuscripts submitted for publication has been approved and published].
Use of the dataset must be acknowledged using a citation which would include:
- the Identification of the Primary Investigator
- the title of the survey (including country, acronym and year of implementation)
- the survey reference number
- the source and date of download
Example:
Sharon Wolf, University of Pennsylvania, John Lawrence Aber, New York University, Jere Behrman, University of Pennsylvania. Ghana: Quality Preschool for Ghana Impact Evaluation, Endine Survey 2017, Ref. GHA_2017_GQPIE-EL_v01_M. Dataset downloaded from [URL] on [date]
Disclaimer and copyrights
The user of the data acknowledges that the University of Pennsylvania, New York University, Innovations for Poverty Action, Early Learning Partnership Fund, Strategic Impact Evaluation Fund, the World Bank bear no responsibility for use of this data or for interpretations or inferences based upon such uses.
(c) 2017, Innovations for Poverty Action
Contacts
| Name | Affiliation | URL | |
|---|---|---|---|
| Edward Tsinigo | Innovations for Poverty Action | etsinigo@poverty-action.org | |
| Alaka Holla | World Bank | aholla@worldbank.org | |
| Strategic Impact Evaluation Fund (SIEF) | World Bank | http://www.worldbank.org/en/programs/sief-trust-fund |
Metadata production
DDI_GHA_2017_QPIE-EL_v01_M_WB
| Name | Affiliation | Role |
|---|---|---|
| Development Economics Data Group | The World Bank | Documentation of the DDI |
2019-04-05
Metadata version
Version 01 (March 2019)