Nonfinancial Extrinsic and Intrinsic Teacher Motivation in Government and Private Schools 2015-2017, Impact Evaluation Surveys

Baseline Respondent Identification and Sampling Strategy:

Delhi:

Teacher Motivation:
STIR initially did a search process of several hundred Affordable Private Schools (APS) in east Delhi. From these schools, STIR passed school names onto IDinsight where the teachers might be interested in working with IDinsight. IDinsight attempted to sample all schools for the Teacher Motivation survey. In total, IDinsight interviewed 1,259 teachers for the Teacher Motivation survey.

Classroom Observation:
From these 1,259 teachers, STIR did an additional round of screening to determine which teachers were the most interested and returned a list of 810 teachers to IDinsight. This list formed the basis of the classroom observation. However, due to attrition and refusals at the school level we were unable to meet our target of teachers and ended up surveying only 342 teachers.

Student Testing:
For sampling students in the classroom, IDinsight sampled 10 students per classroom in classes (of all teachers covered for the classroom observation) with more than 10 students using the attendance register for the day the enumerator came to the class. In classes with fewer than 10 students, all children were sampled.

Uttar Pradesh:

Teacher Motivation:
In Uttar Pradesh, IDinsight obtained a list of all clusters in Raebareli and Varanasi districts that STIR was working in. From this list, IDinsight selected all clusters with more than 16 schools. This was done to ensure that there would be enough schools in the cluster to assign some to the control group while also maintaining enough treatment schools for STIR to form a network. For the Teacher Motivation survey, IDinsight surveyed all teachers in the school, yielding 1,145 teachers.

Classroom Observation:
For the classroom observation, IDinsight sampled roughly 2/3 of the teachers who completed the Teacher Motivation questionnaire, to get a final list of roughly 810 teachers. Teachers were added to this list due to teachers dropping out and the final number was 838 teachers.

Student Testing:
For sampling students in the classroom, IDinsight sampled 10 students per classroom in classes with more than 10 students using the attendance register for the day the enumerator came to the class. In classes with fewer than 10 students, all children were sampled.

Midline Respondent Identification and Sampling Strategy:

For midline, which took place at the beginning of the second academic year, we followed up with teachers and students surveyed at baseline. Teachers were added only in the case where the number of teachers still teaching in the school from our baseline lists fell below a certain number. In Delhi, teachers were added if less than two teachers from our list in a given school were available and in Uttar Pradesh, new teachers were added only if all teachers from our baseline lists in a given school dropped out.

The sampling strategy had two clear advantages:

1. It helped us target teachers and students that have been exposed to STIR for as long as possible since the timeline for the overall evaluation is relatively short.
2. The evaluations are already quite complex and this helped have a clear interpretation and narrative surrounding the results.

Delhi:

Teacher Motivation:
From the list of 1,259 teachers surveyed at teacher motivation baseline, 453 teachers dropped out of schools during the academic year and hence were not available for surveying during midline. A further 65 teachers refused to participate and 84 teachers were not available during the data collection period. Given this, the total number of teachers surveyed at teacher motivation midline was 657. These teachers formed the sample for analyses.

Classroom Observation:
For classroom observations, we attempted to collect data for all 811 teachers on the Delhi original list. For those schools where the number of teachers available from our 811 list fell below two, 148 new teachers were added based on a random selection from those teachers employed at that school as of 1 July 2015. A total of 459 teachers were surveyed as part of the classroom observation midline.

Student Testing:
For testing of student learning levels, all students surveyed at baseline formed the potential sample at midline. Among the 3,367 students from baseline, 1,956 students were tracked and surveyed at midline. 1,127 students had dropped out from the schools. 40 students were absent throughout the course of the data collection, and were not found in schools during any of the five revisits. The remaining 244 students were in schools where we could not survey.

Uttar Pradesh:

Teacher Motivation:
From the 1,145 teachers surveyed at baseline, 288 teachers dropped out of schools during the course of the academic year and were hence not available for data collection. An additional 61 refused to participate in the data collection and 41 were not available through the course of the data collection. The final number of teachers surveyed at midline were 755. This was the sample for analysis.

Classroom Observation:
From the list of 838 teachers surveyed at baseline, we successfully observed the classrooms of 734 of these teachers at midline. Another 13 teachers were added in schools where all teachers from our 838 had dropped out. 12 of these 13 were in Raebareli and 1 was in Varanasi. In total, 747 teachers were surveyed. 82 teachers dropped out of the schools in our sample. 13 teachers refused to participate in the data collection and 14 teachers were absent throughout the survey period and were not available on either of our visits.

Student Testing:
Of the 7,386 students tested at baseline, a total of 4,560 students were also tested at midline. 615 students were absent all days of visits to the schools. 149 students were in the four schools that refused data collection. 2,062 dropped out of the schools in our sample.

Endline Respondent Identification and Sampling Strategy:

For endline, which took place after the end of the second academic year, we followed up with teachers and students surveyed at midline. In Delhi, one teacher was added per school to the classroom observation sample where possible. Additional teachers were added to the teacher motivation sample by offering the survey to all the teachers in our sample schools. The sampling strategy had two clear advantages:

1. It helped us target teachers and students that have been exposed to STIR for as long as possible since the timeline for the overall evaluation is relatively short.
2. The evaluations are already quite complex and this helped have a clear interpretation and narrative surrounding the results.

Delhi:

Teacher Motivation:
From the list of 657 teachers surveyed at teacher motivation midline, 101 teachers dropped out of schools during the academic year and hence were not available for surveying during endline. A further 25 teachers refused to participate and 50 teachers were not available during the data collection period. Given this, the total number of teachers surveyed at teacher motivation midline was 481. These teachers formed the sample for analyses.

Classroom Observation:
For classroom observations, we attempted to collect data for all 459 teachers on the Delhi midline list as well as 102 teachers we surveyed at baseline and couldn't at midline but were hopeful of covering in the last survey. A new teacher was added to each school's sample where possible. A total of 376 teachers were surveyed as part of the classroom observation endline.

Student Testing:
For testing of student learning levels, all students surveyed at midline formed the potential sample at endline. Among the 1,956 students from baseline, 1,843 students were tracked and surveyed at midline. 49 students had dropped out from the schools. 45 students were absent throughout the course of the data collection, and were not found in schools during any of the five revisits.

Uttar Pradesh:

Teacher Motivation:
From the 967 teachers surveyed at midline, 105 teachers were transfered and 17 retired during the course of the academic year and were hence not available for data collection. An additional 36 refused to participate in the data collection and 26 were not available through the course of the data collection. The final number of teachers surveyed at midline were 731. This was the sample for analysis.

Classroom Observation:
From the list of 894 teachers surveyed at midline, we successfully observed the classrooms of 722 of these teachers at endline. 67 teachers were transferred, 7 retired and 32 refused to participate in the data collection. 26 teachers were absent throughout the survey period and were not available on any of our visits.

Student Testing:
Of the 4,560 students tested at midline, a total of 3,175 students were also tested at endline. 168 students were absent all days of visits to the schools. 1,011 graduated out of the schools in our sample and 194 students dropped out of schooling altogether.