Quality for Preschool Impact Evaluation 2016, Midline Survey
Ghana, 2016
Get MicrodataIdentification
GHA_2016_QPIE-ML_v01_M
Quality for Preschool Impact Evaluation 2016
Midline Survey
| Name | Country code |
|---|---|
| Ghana | GHA |
Impact Evaluation Study
Baseline Survey: 2015
Midline Survey: 2016
Endline Survey: 2017
The project seeks to increase the quality of preschool education during the two years of universal Kindergarten (KG) in Ghana through intervening in the supply-side (i.e., teacher in-service training) and the demand side (i.e., increasing parental awareness for developmentally appropriate quality early education).
The primary goal of the impact evaluation is to test the efficacy of a potentially scalable (8-day) in-service teacher training to improve the quality of KG teacher practices and interactions with children and to improve children’s development, school readiness and learning in both private and public preschools in the Greater Accra Region of Ghana. Additional goals of this evaluation are: to test the added value of combining a scalable (low-cost) parental awareness intervention with teacher in-service training; to compare implementation challenges in public and private schools; and to examine several important sources of potential heterogeneity of impact, primarily impacts in public vs. private schools.
The current submission is for the Midline Survey, conducted with 3 types of respondents across two phases – School survey and Caregiver [household] surveys. The school survey was conducted from May to July 2016 and consisted of collecting the following data: (a) direct assessments of children’s school readiness, (b) surveys of KG teachers, (c) direct observation of inventory of facilities within KG classrooms [environmental scan]; videotaping of KG classroom processes, teaching, and learning (not being submitted); as well as video coding of KG classroom video recordings using Teacher Instructional Practices and Processes Systems (instrument not being submitted). The caregiver survey was conducted via phone from August to September 2016 on primary caregivers of KG children. The caregiver survey sought information on caregivers’ background, poverty status, involvement or participation in school and home activities, and perception about ECD. Overall, the Midline Survey was conducted from May to September 2016 for all respondents.
Sample survey data [ssd]
Units of analysis include individuals (KG teachers, children, caregivers), KG classrooms and preschools.
Version
Version 1: Edited, anonymous dataset for public distribution. All Personally Identifying Information (PII) has been removed.
2016-12-15
The data provided is the final version with PII removed.
Scope
The scope of the QP4G Midline survey include:
Child Direct Assessment: Socio-emotional skills, early literacy skills, early numeracy skills, motor skills, and executive function skills; and assessor’s reported ratings.
KG teacher survey: KG teachers background, poverty status, household food security situation, perceptions about ECD, participation in in-service training, work conditions, teacher well-being, and teaching knowledge.
Classroom observation: Direct observation of inventory of facilities within KG classrooms; videotaping of KG classroom processes, teaching, and learning (not being submitted); video coding of KG classroom video recordings using Teacher Instructional Practices and Processes Systems (instrument not being submitted).
School attendance: Child name and unique ID, school term, number of total attendance per term, number of days of absence per term.
Caregiver survey: Caregivers’ background, poverty status, involvement or participation in school and home activities, and perception about ECD.
Coverage
Urban and Peri-Urban Districts, Greater Accra Region
District
The survey universe is 6 poor districts in the Greater Accra Region. We sampled 240 schools, 108 public (Govt.) schools and 132 private schools. The population of interest is KG teachers and children in KG 1 and KG 2 classrooms in these schools, as well as the caregivers of sampled students.
Producers and sponsors
| Name | Affiliation |
|---|---|
| Sharon Wolf | University of Pennsylvania |
| John Lawrence Aber | New York University |
| Jere Behrman | University of Pennsylvania |
| Name | Role |
|---|---|
| Innovations for Poverty Action | Technical assistance in questionnaire design, sampling methodology, data collection and data processing |
| Name |
|---|
| Strategic Impact Evaluation Fund |
| Early Learning Partnership, World Bank |
| UBS Optimus Foundation |
| Name | Role |
|---|---|
| Ghana Education Service | Government Support |
Sampling
This impact evaluation applies a cluster-randomized design. Eligible schools were randomly selected to participate in the study. The eligible population was schools with KG 1 and KG 2 classrooms (the two years of universal preprimary education) in six districts in the Greater Accra Region. In these six districts, we have sampled 240 schools; 108 public schools and 132 private schools in total.
The unit of randomization for this randomized control trial (RCT) is schools, whereby eligible schools (stratified by public and private sector schools) are randomly assigned to: (1) in-service teacher-training program only; (2) in-service teacher-training program plus parental awareness program; or (3) control (current standard operating) condition.
The sampling frame for this study was based on data in the Education Management Information System (EMIS) from the Ghana Education Service. This data was verified in a 'school listing exercise' conducted in May 2015.
Sample selection was done in four stages:
The first stage involved purposive selection of six districts within the region based on two criteria: (a) most disadvantaged (using UNICEF's District League Table scores, out of sixteen total districts); and (b) close proximity to Accra Metropolitan for travel for the training of the KG teachers. The six selected municipals were La Nkwantanang-Madina Municipal, Ga Central Municipal, Ledzokuku-Krowor Municipal, Adentan Municipal, Ga South Municipal and Ga East Municipal.
The second stage involved the selection of public and private schools from each of the selected districts in the Accra region. We found 678 public and private schools (schools with kindergarten) in the EMIS database. Of these 361 schools were sampled randomly (stratified by district and school type) for the school listing exercise, done in May 2015. This was made up of 118 public schools and 243 private schools. The sampling method used for the school listing exercise was based on two approaches depending on the type of school. For the public schools, the full universe of public schools (i.e., 118) were included in the school listing exercise. However, private schools were randomly sampled using probability proportional to the size of the private schools in each district. Specifically, the private schools were sampled in each district proportionate to the total number of district private schools relative to the total number of private schools. In so doing, one school from the Ga South Municipal was removed and added to Ga Central so that all districts have a number of private schools divisible by three. This approach yielded 122 private schools. Additionally, 20 private schools were randomly selected from each of the districts (i.e., based on the remaining list of private schools in each district following from the first selection) to serve as replacement lists. The replacement list was necessary given the potential refusals from the private schools. There were no replacement lists for the public schools since all public schools would automatically qualify for participation.
The third stage involved selecting the final sample for the evaluation using the sampling frame obtained through the listing exercise. A total of 240 schools were randomly selected, distributed by district and sector. Schools were randomized into treatment groups after the first round of baseline data collection was completed.
The survey respondents were sampled using different sampling techniques:
a. KG teachers: The research team sampled two KG teachers from each school; one from KG1 and KG2. KG teachers were sampled using purposive sampling method. In schools where there were more than two KG classes, the KG teachers from the "A" stream were selected. For the treatment schools, all KG teachers were invited to participate in the teacher training program.
b. KG child-caregiver pair: The research team sampled KG children and their respective caregivers using simple random sampling method. Fifteen KG children-caregivers pair were sampled from each school. For schools with less than 15 KG children (8 from KG1, 7 from KG2 where possible), all KG children were included in the survey. KG children were selected from the same class as the selected KG teacher. The survey team used the class register to randomly select KG children who were present on the day of the school visit. Sampling was not stratified by gender or age. The caregivers of these selected child respondents were invited to participate in the survey.
The research team sought informed consent from the school head teacher, caregivers, as well as child respondents.
Schools: Of the 240 randomly sampled schools at baseline, 235 were surveyed at Midline. This represents 97% of the sample schools. Of the remaining 3% (n = 5), three schools dropped out of the study and 2 schools closed down.
KG teachers: 78% of KG teachers were interviewed at Midline survey. Notably, we interviewed 348 of the 444 teachers surveyed at baseline. This is because 96 teachers had either transfered (moved) to other schools (or classes) or had left the teaching profession.
KG children: Of the 3435 children who were assessed at baseline, 2975 were assessed at Midline. This represents 87% coverage rate. Thirteen percent of the children could no be tracked at Midline due largely to change in school [because of family migration outside the catchment districts or region].
Caregivers: Caregiver participation in the QP4G study increased from 2134 at Baseline to 2710 at Midline. This represented 27% increase in the caregiver coverage over the baseline. The increase in the caregiver coverage was largely due to the 576 additional active contact numbers obtained from the caregivers. These caregiver, hitherto, had no contact numbers at baseline.
No weights were used in the analysis.
Survey instrument
Data were collected at Midline Survey using structured questionnaires or forms.
Child Direct Assessment: The KG Child Assessment was conducted using the International Development and Early Learning Assessment (IDELA) tool designed by Save the Children. IDELA was adapted based on extensive pre-testing and piloting by different members of the evaluation team. The adapted version measured five indicators of ECD. The indicators were early numeracy skills, language/literacy skills and development, physical well-being and motor development, socio-emotional development, and approaches to learning. IDELA contained 28 items. In addition, one task was added – the Pencil Tap – to assess executive function skills. Apart from the English language, IDELA was translated and administered into three local languages, namely, Twi, Ga, and Ewe. These local language versions had gone through rigorous processes of translation and back translation. The IDELA tool has not been shared as Save the Children have proprietary rights over this.
KG Class Environmental Scan: The KG classroom observation involved taking inventories of the KG classrooms [environmental scan] and conducting video recordings of the classroom processes. The KG Class Environmental Scan tool was designed to take inventories of the facilities in the KG classrooms. The classroom video recordings have not been shared as they contain PIIs.
TIPPS: The video recordings taken during the classroom observations were coded using an early childhood education adapted version of Teacher Instructional Practices and Processes Systems (TIPPS). Seidman, Raza, Kim, and McCoy (2014) of New York University developed the TIPPS instrument. TIPPS observes nineteen key concepts of teacher practices and classroom processes that influence children’s cognitive and social-emotional development. The concept sheet was used to code the kindergarten classroom videos. The TIPPS tool has not been shared as New York University has proprietary rights over this.
KG Teacher survey: The Midline KG Teacher Survey was based on the modification of the Baseline version. The modifications took into account the data need for measuring outcomes of the teacher training intervention as well as concerns about respondent burden/distress, response rates, and costs. Two modules were added: (a) participation in in-service training (i.e., participation in any in-service training including QP4G teacher training, issues of contamination or spill-over effects, and receipt of text-message intervention); and (b) perceptions of early childhood development. The latter module was culled from the Caregiver Survey. Also, KG teachers who took part in the Baseline II survey were excluded from answering the following time-invariant questions: (a) background characteristics such as local languages of teachers/caregivers, the level of proficiency in speaking and writing in English and local languages; teacher’s paternal and maternal educational level, and (b) English reading knowledge.
Caregiver Survey: The Midline Caregiver survey was based on the modification of the Baseline II Caregiver Survey. The modification involved the removal of the food security, tracking, and mobility updates modules as well as the addition of modules on child discipline and parental participation in the parental awareness raising program. The module on child discipline was adopted from the UNICEF’s MICS 2013 Household Questionnaire. A Call Records Form was also designed to track and screen each caregiver before the actual interview was conducted. Four local language dictionaries of keywords and phrases were developed for the Caregiver Survey. The selected languages were Ga, Ewe, Twi, and Hausa. These languages were selected because they were used extensively in interviewing the caregivers at Baseline II survey.
School Attendance Records: The School Attendance Records Form was designed to record school attendance information for the sampled KG teachers and children. The Form captured school-specific attendance details such as the active number of school days, the number of national/school-related holidays per term, and child-specific information such as present/absent from school.
Questionnaires are provided under the Related Materials tab.
Data collection
| Start | End | Cycle |
|---|---|---|
| 2016-05-16 | 2016-07-11 | Child Direct Assessment |
| 2016-05-16 | 2016-07-11 | Teacher Interviewer and Classroom Observation |
| 2016-05-16 | 2016-07-25 | School Attendance Records |
| 2016-07-20 | 2016-08-12 | Video Coding |
| 2016-08-18 | 2016-09-26 | Caregiver Phone Survey |
| Name |
|---|
| Innovations for Poverty Action, Ghana |
Five different data collection teams were formed for the various Midline surveys. The teams conducted child assessment, teacher interviews, and classroom observation, video coding, school attendance records taking, and caregiver phone survey.
Data collection activities, namely, video coding, school attendance records taking, and caregiver phone survey were done in a centralized location while the others were done in the selected schools. The school survey activities had seven field teams, comprising team leaders, child assessors, and teacher interviewers/videographers. A field supervisor managed at least four teams. One auditor audited the teacher survey. Each school survey team visited a school a day. However, in order to ensure their productivity, they were tasked to visit more than one school a day if the number of children in one of the schools was less than 15.
The caregiver phone survey had three auditors; two for auditing and one or sending top-ups to the respondents as a gift. The number of the phone interviewers were 15.
Each phase of the data collection activity was monitored through field visits. The monitoring team covered IPA’s Research Manager, Research Associate, Field Manager, and Survey Coordinator. Team leaders and field supervisors were also involved in monitoring their assigned team(s). With the exception of the caregiver survey and the video coding activities, field visits were done through accompaniment and spot checks. On-site observations were done for the caregiver survey and video coding activities since these activities were conducted in a central location (i.e., IPA office). The field visit was done to establish whether the protocols were followed and to assess the performance of the field staff. Each field staff was visited and observed multiple times during the data collection period by different monitors. Specific monitoring protocols, namely, IDELA Monitoring Form, Teacher Interviewers Monitoring Form, and Video Quality Form were used (these instruments are not shared).
Specialized data collection teams were hired for each specific data collection activity – child direct assessment, teacher survey, caregiver survey, school attendance records, and classroom observation. Each of these unique data collection teams was provided specialized training based on the focus of the data collection activity, content of the data collection instrument and specific protocols relating to the particular data collection activity. Training of each of these specialized teams included classroom training and school visits or field practice.
Data were collected using different means of administration. Child direct assessment and teacher survey were conducted using in-person interviews through computer-assisted personal interview. The child direct assessment and teacher survey lasted approximately 45 minutes. The caregiver survey was administered via phone using computer-assisted telephone interview. The caregiver survey duration was approximately 50 minutes. This includes a screening of caregivers to determine their eligibility for the interview. Classroom observation was done by physically observing the inventory of facilities in the facilities in the KG classrooms (environmental scan) and videotaping classroom processes and systems. These videos were then coded in a centralized location by trained and certified video coders. School attendance records involved the review of pupil attendance register and teacher attendance book and translating specific information on a number of days present and/or absent from school per term. This activity took about 1 hour per each KG class for the selected children and teachers.
To gain access to the selected schools, prior approval was sought from the Regional Ghana Education Service and the District Education Directors for the selected districts. The research team sought informed consent from the school head teachers, teachers, and caregivers. Each child’s assent was sought before conducting the child direct assessment.
Teacher survey was administered strictly in the English language. Child direct assessment and caregiver survey were administered in the particular language the respondent understands and could speak fluently. The key languages were English, Twi, Ga, and Ewe. The child direct assessment was fully translated and back-translated in the local languages used for the assessment. The key stock of words or dictionary regarding specific variables were developed for the caregiver survey.
After each data collection activity, a debriefing was conducted to learn about the data collection procedures, instrument administration, logistics, challenges and lessons learned during the particular data collection activity.
Unlike the school attendance instrument that was fully piloted, the other data collection instruments were partially piloted based on the modifications done to the baseline instruments.
Issues that occurred in the field were managed on a case-by-case basis in line with the overall focus of the study and specific protocols developed for the Midline survey. Notably, corrective actions were developed and implemented for specific cases relating to the merger of KG units in some schools, splitting of some schools, closure of two schools [and the need to track respondents from those schools], as well as transfer and movement of teachers and children across and within schools.
Data processing
Data consistency checks namely high-frequency checks and backchecks (audits) were conducted for all surveys remotely. Corrections were made during and after data collection after errors were reconciled.
All checks and cleaning was done using STATA and IPA data management systems. IPA possesses all the relevant code.
Data Access
Public use files, accessible to all after registration.
Use of the dataset must be acknowledged using a citation which would include:
- the Identification of the Primary Investigator
- the title of the survey (including country, acronym and year of implementation)
- the survey reference number
- the source and date of download
Example:
Sharon Wolf, University of Pennsylvania, John Lawrence Aber, New York University, Jere Behrman, University of Pennsylvania. Ghana: Quality Preschool for Ghana Impact Evaluation, Midline Survey 2016, Ref. GHA_2016_GQPIE-ML_v01_M. Dataset downloaded from [URL] on [date]
Disclaimer and copyrights
The user of the data acknowledges that the University of Pennsylvania, New York University, Innovations for Poverty Action, Early Learning Partnership Fund, Strategic Impact Evaluation Fund, the World Bank bear no responsibility for use of this data or for interpretations or inferences based upon such uses.
(c) 2016, Innovations for Poverty Action
Contacts
| Name | Affiliation | URL | |
|---|---|---|---|
| Edward Tsinigo | Innovations for Poverty Action | etsinigo@poverty-action.org | |
| Alaka Holla | World Bank | aholla@worldbank.org | |
| Strategic Impact Evaluation Fund (SIEF) | World Bank | http://www.worldbank.org/en/programs/sief-trust-fund |
Metadata production
DDI_GHA_2016_QPIE-ML_v01_M_WB
| Name | Affiliation | Role |
|---|---|---|
| Development Economics Data Group | The World Bank | Documentation of the DDI |
2019-04-05
Metadata version
Version 01 (March 2019)