{"doc_desc":{"title":"Guinea Education Impact Evaluation","idno":"DDI_GIN_2012_PITIE-BL_v01_M_v01_A_PUF_WB","producers":[{"name":"Development Data Group","abbreviation":"DECDG","affiliation":"World Bank","role":"Study documentation"},{"name":"Education - GP","abbreviation":"GEDDR","affiliation":"World Bank","role":"Study documentation"}],"prod_date":"2014-09-17"},"study_desc":{"title_statement":{"idno":"GIN_2012_PITIE-BL_v01_M_v01_A_PUF","title":"Impact Evaluation of Performance-based Incentives for Teachers 2012","sub_title":"Baseline Survey","alt_title":"PITIEL-BL 2012"},"authoring_entity":[{"name":"Marie-Helene Cloutier","affiliation":"World Bank"},{"name":"Deon Filmer","affiliation":"World Bank"},{"name":"Felipe Barrera","affiliation":"Harvard University"}],"production_statement":{"funding_agencies":[{"name":"Strategic Impact Evaluation Fund","abbreviation":"SIEF","role":""},{"name":"International Development Association","abbreviation":"IDA","role":""}]},"distribution_statement":{"contact":[{"name":"Marie-Helene Cloutier","affiliation":"World Bank","email":"mcloutier@worldbank.org","uri":""}]},"series_statement":{"series_name":"1-2-3 Survey, phase 1 [hh\/123-1]"},"version_statement":{"version":"- v01 (Cleaned baseline data as of June 24, 2014).\n\nThe baseline data for this IE was collected to provide a pre-intervention overview of the schools, teachers, and students targeted in the context to the performance-based rewards program and check that the randomization successfully balanced the pre-program characteristics of the three groups of schools. \n\nTwo main data collection activities were carried to collect the data: \n\n- Announced visits to the schools to administer the standardized Math and French tests led by the SNCESE (\"Service National de Coordination de l'\u00c9valuation du Syst\u00e8me \u00c9ducatif\"), a service of the MEPU-A, as well as a principal questionnaire and a teacher questionnaire.\n\n- Unannounced inspection visits to schools led by the General Inspection Department (\"Inspection General de l'Education\", IGE) of the MEPU-A to establish teacher's baseline inspection scores."},"study_info":{"abstract":"This study aims to assess the impacts of two types of interventions to be implemented by the Guinea Ministry of Pre-University and Civic Education (MEPU-EC) using performance-based rewards for teacher as motivation-enhancing strategies and providing simple guidance on effective classroom management strategies to encourage their use. In the context of the incentive scheme, teacher performance will be measured using an objective and comparable performance indicator.\n\nThe evaluation is designed to answer the following research questions: \n- What is the effect of performance-based incentives for teachers on teaching practices and behaviors (absenteeism, time on task) and on student learning outcomes? \n- To what extent do recognition rewards trigger different (better or worse) modifications in teachers\u2019 behavior and practices compared to those triggered by in-king rewards? \n- What is the effect of providing guidance on effective classroom management on teaching practices and student learning outcomes? \n- What is the effect of performance-based incentives when teachers are also provided with guidance on effective classroom management practices? Moreover, cost effectiveness of the different treatment arms will be investigated. \n\nThe proposed evaluation strategy is a randomized control trial that spans two academic years (2012-2013 and 2013-2014) and targets all grade 3 and 4 teachers of a nationally representative sample of 420 schools. The first year of the impact evaluation focused on assessing the impact of performance-based teacher incentives only and comparing the two types of incentives: in-kind and recognition. The second year includes the additional intervention of delivering guidance on classroom management. Data on schools, teachers, and students is collected through (unannounced) attendance checks, time on task and general classroom observations (carried out in person and using a video), official inspection visits, administration of student\u2019s curriculum-based standardized Math and French tests, teacher surveys and content-knowledge tests, as well student and principal questionnaires. Costing data will be collected through the financial reports of the IDA project and of the government budget (for the second year incentives and guidance intervention) since all expenditures related to the interventions evaluated are paid through these channels.\n\nBaseline data is documented here.","coll_dates":[{"start":"2012-05-01","end":"2012-11-01","cycle":"Baseline"}],"nation":[{"name":"Guinea","abbreviation":"GIN"}],"geog_coverage":"National","analysis_unit":"- Teachers","data_kind":"Sample survey data [ssd]","notes":"The scope of the study includes:\n- performance-based rewards \n- school characteristics and environment \n- support and monitoring of teachers\n- teachers' demographic characteristics\n- teachers' work experience and training needs\n- pedagogical practices and languages\n- absenteeism and events disturbing teaching\n- student learning\n- students' demographic characteristics and family background\n- teachers' performance recognition or punishment\n- French test\n- math test"},"method":{"data_collection":{"sampling_procedure":"The sample was designed to be representative, at the national level, of the target grades' teacher population in public French-speaking schools. \n\nThe sampling process at the schools took place as followed: \n- The population of public Francophone schools was extracted from the in 2011-12 Education Management Information System database and used as the sampling frame;\n- Schools were split into the 15 strata defined to capture the school location (8 regions and 2 zones, namely rural-urban).\n- Assuming that the number of teachers and students in grade 2 and 3 in in 2011-12 indicates the numbers of teachers to be expected in grades 3 and 4 in 2012-13, the number of grade 2 and 3 teachers\/classes for each school was calculated.\n- The number of schools to be selected per strata was established using the Markwardt protocol (this represent the average between selecting a proportional and equal number of schools per strata). The selection probability of each individual school was established using the number of targeted teachers in the school and the number of schools in the strata.\n- Using a random starting point and the selection probability, 450 individual schools and 75 replacement schools were selected. \n\nThis sample was completed by adding the 16 pre-identified schools where the instruments were piloted in the 2011-12 academic year. Therefore, before launching the baseline fieldwork, a sample of 466 schools was targeted. Within each school, all grade 3 and 4 teachers and all of their students were targeted.\n\nRandomization is at the school level but target beneficiaries are the teachers. 420 schools, all grade 3 and 4 teachers, all grade 3 and 4 students.","sampling_deviation":"While in the field at baseline, the teams were unable to locate some of the schools and some of the located schools turned out not to have the targeted grades and thus had to be taken out of the sample. The final sample contains 420 schools. No replacement schools were used. The final sample therefore differs from the targeted sample and national representativeness is uncertain. It is important to note that this does not reduce the internal validity of the IE design since the random assignment of schools to the different experiment arms was carried-out once the realized sample of schools was stabilized.\n\nSample of teachers and students: once in the school in May 2012, the aim was to administer a standardized test to all grade 2 and 3 students (who will be grade 3 and 4 students in 2012-13, the first year of the intervention). Furthermore, in October 2012, the aim was to survey\/inspect all grade 3 and 4 teachers for the 2012-13 academic year. At baseline, a total of 416 principal and 1177 teachers were surveyed; 1214 inspected, and 23183 students participated in the test. \n\nWithin each of the targeted teachers\u2019 classes, the objective was that all students should take part in the test. However, once in the field, the teams were faced with larger schools (in terms of the number of students) than expected and thus did not have enough printed copies of the tests to administer it to all students. In about one third of the visited schools, instead of randomly selected students within each of the classroom, only subset of the classrooms (all students) were selected to participate in the test. Furthermore, when class size was too big, a random selection of students within the selected classes was carried out. There is no reason to believe that the selection of students within a class was not random but there is also no certainty that it was. Finally, because of teacher absenteeism and logistics difficulties, the tests were only administered in 353 out of the 420 targeted schools.","coll_mode":["Face-to-face [f2f]"],"research_instrument":"The following survey instruments were used: (i) a questionnaire administered to the school\u2019s principal, (ii) a teacher questionnaire administered to targeted teachers (grade 3 and 4), (iii) a Math test (a few parallel booklets), a French test (a few parallel booklets) administered to students in the targeted teachers\u2019 classrooms, and (iv) an inspection bulletin administered to targeted teachers in the context of two lessons, one in French and one in Math. \n\nBelow is presented a more detailed description of the various instruments. \n\nPrincipal questionnaire (May and October 2012) \n A. School and principal identification \n B. Demographics\n C. Education and professional training\n D. Work experience and training needs\n E. Pedagogical practices and languages\n F. School basic characteristics \n G. School environment\n H. Interaction with colleagues (subordinate, supervisors, etc.)\n I. Support and monitoring of teachers\n J. Motivation\n \nTeacher questionnaire(May and October 2012) \n A. Class and teacher identification\n B. Demographics\n C. Class characteristics\n D. Education and professional training\n E. Work experience and training needs\n F. Pedagogical practices and languages\n G. Interaction with colleagues \n H. Motivation\n I. Absenteeism and events disturbing teaching\n J. Remuneration\n K. Perception of key factors influencing student learning \n L. Performance recognition or punishment\n\nStudent tests (May 2012) \n 1.A Identification of school, class, and teachers\n 2.B. School-related student characteristics\n 1.C. Student environmental and familial backgrounds\n 2. French test questions\n 3. Math test questions\n\nInspection bulletin (October 2012) \n I. Class and inspector identification\n II. Teacher identification\n III. Summary of scores\n VI. General material and spatial classroom arrangement\n V.1 Lesson 1 \u2013 Identification of the lesson\n V.2 Lesson 1 \u2013 Teaching and learning material preparation \n V.3 Lesson 1 \u2013 Lesson planning (according to the Competency-based approach)\n V.4 Lesson 1 \u2013 Delivery of the lesson\n V.5 Lesson 1 \u2013 Analysis of the own\u2019s performance\n VI.1 Lesson 2 \u2013 Identification of the lesson\n VI.2 Lesson 2 \u2013 Teaching and learning material preparation \n VI.3 Lesson 2 \u2013 Lesson planning (according to the Competency-based approach)\n VI.4 Lesson 2 \u2013 Delivery of the lesson\n VI.5 Lesson 2 \u2013 Analysis of the own\u2019s performance","act_min":"All data collection activities were carried under close supervision of the Impact Evaluation World Bank team which oversaw the training sessions and carried-out random spot check in the field during data collection.\n\nDecentralized government staff (inspectors, district and prefectural education officers, planners, etc.) served as survey administrators. Individuals were selected on a competitive basis (using a test) and trained as enumerators and supervisors. To enhance objectivity, inspectors were assigned to a region different from their regular assignment.","weight":"No weight will be used for the analysis and thus no weight variable is included in the datasets.","method_notes":"Teacher performance is measured using an objective and comparable performance indicator. \n\nThis indicator is equal to the sum of two weighted sub-indicators, with respective weights of 0.7 and 0.3, which are: \n- the teacher's annual value-added into his\/her students learning, i.e. the annual gain (in percentage points) in the average score of the teacher's students on standardized tests in Math and French; \n- the teacher's annual improvement in the quality of his\/her lessons preparation and delivery which corresponds to the gain (in percentage points) in the inspection scores attributed to the teacher during inspection visits.\n\nRewards will be determined and allocated to teachers on an individual basis and will take the form of either goods\/in-kind (rice sacks, radios, phones, televisions, and generators) or recognition (honor boards, certificates, a letter from the Minister of Education, public ceremonies, and dissemination of results on community radios). The value of the reward will be determined on an absolute scale - without relative performance comparisons. Guidance on effective classroom management strategies will be provided to selected teachers in verbal and written form by the inspectors to ensure a uniform set of messages reach the teachers on a regular basis."},"analysis_info":{"response_rate":"Response rates varies from high in the case of inspection and questionnaires to a little lower in the case of test. Balance analysis indicates that these response rates were orthogonal to treatment."}},"data_access":{"dataset_use":{"cit_req":"The use of the datasets must be acknowledged using a citation which would include:\n- the identification of the Primary Investigator (including country name);\n- the full title of the survey and its acronym (when available), and the year(s) of implementation;\n- the survey reference number;\n- the source and date of download (for datasets disseminated online).\n\nExample:\n\nMarie-Helene Cloutier, World Bank; Deon Filmer, World Bank; Felipe Barrera, Harvard University. Guinea Impact Evaluation of Performance-based Incentives for Teachers 2012, Baseline Survey (PITIEL-BL). Ref. GIN_2012_PITIE-BL_v01_M_v01_A_PUF. Dataset downloaded from [URL] on [date].","disclaimer":"The user of the data acknowledges that the original collector of the data, the authorized distributor of the data, and the relevant funding agency bear no responsibility for use of the data or for interpretations or inferences based upon such uses."}}},"schematype":"survey","tags":[{"tag":"DOI"}]}