| Author(s) |
Heather Lanthorn, Doug Johnson, Pratima Singh, and Varun Chakravarthy |
| Country |
India |
| Language |
English |
| Description |
This report provides estimates of the causal impacts of one year of STIR’s programming on key teacher and student outcomes in two locations in India. This midline report represents the mid-points of two parallel evaluations, after one academic year of programming. The endline report will cover the 2-year impacts.
This report is organized as follows. In Section 2, we introduce the evaluations and the program contexts. In Section 3, we turn to an overview of the programmatic components and underlying logic and assumptions in STIR’s theory of change (ToC). This builds a solid foundation for understanding the evaluation questions, methodology, and findings. In Section 4, we present details of the evaluations including objectives, questions, design, methods and analytical approach. Section 5 lays out the main results from the evaluations so far and in Section 6, we conclude with a brief discussion of the limitations. |
| Abstract |
STIR seeks to improve teachers’ motivation, mindset, and classroom practice to improve student learning outcomes. This happens through voluntary, in-service professional development and non-financial incentives for teachers. Broadly, STIR works to inspire teachers to become agents of change in their classroom, schools, and education systems. STIR’s basic theory of change (as relevant to Year 1 of this evaluation) holds that as teachers change their motivation to teach well and their mindset on their potential to become a better teacher is possible, they will change their level of effort to update their classroom practice, which may change the quantity and/or quality of their teaching. These changes may change student learning outcomes. Changes in classroom practice and student performance will also influence teachers’ motivation.
STIR is both the program designer and implementer. We are conducting two parallel randomized evaluations of STIR’s programming: one with Affordable Private Schools (APSs) in East Delhi and another with government schools in the state of Uttar Pradesh (U.P.). (Please refer to the baseline report for additional details.) There are differences in the program and evaluation designs across these settings. In East Delhi, STIR staff directly deliver programming to teachers. IDinsight is examining 180 APS, of which 120 were randomly selected to receive the treatment (invitation for teachers in a school to join STIR). In U.P., STIR uses a cascade or “training of trainers” approach: government staff (government school teachers) are trained by STIR staff to deliver programming to other teachers. In U.P., out of 270 government schools, we randomly selected 180 to receive the treatment. In both cases, the participation of individual teachers in treated schools is intended to be voluntary.
In each setting, we test two different variations of STIR’s programming – the “core” and the “core-plus” models. Core programming focuses on enhancing intrinsic motivation and professional mindsets among teachers. The core-plus model adds various non-financial incentives to the core programming, thus aiming to increase both intrinsic and extrinsic motivation. The key research questions for both evaluations are to measure the impact of STIR’s programming on the following outcomes:
• teacher motivation (using a motivation questionnaire and teacher attendance);
• quantity of teaching practice (using a modified Stallings classroom snapshot) (Stallings 1977; World Bank 2015);
• quality of teaching practice (using a classroom observation tool of child-friendly behaviors);
• student learning (using a modified version of the ASER tool to assess Hindi and math learning levels) (“Annual Status of Education - Rural” 2005).
In each setting, we estimate both school-wide (“intent-to-treat”) and teacher-level (“treatment-on-the-treated”) effects. School-wide effects capture the overall effect of STIR programming on aggregated school outcomes, including teachers who actively participate in STIR as well as those who don’t. As STIR both encourages participating teachers to influence and inspire other teachers in their schools, as well as works in some cases directly with Head Teachers, the school-wide estimate sheds light on the full potential of STIR’s programming. The school-wide results therefore capture the combined impacts of teachers who attend STIR meetings as well as those teachers who do not but may be affected through multiple other channels of influence. We also explore teacher-level effects — i.e., the impact of STIR on the 40-50% of teachers in treatment schools who actively participated in the program — three ways:
• focusing on small schools (where a higher proportion of teachers are active participants);
• using Instrumental Variable/Local Average Treatment Effect (IV/LATE) estimation;
• conducting an observational study style analysis, comparing only teachers who participated actively in STIR’s programming in treatment schools with all teachers in control schools. |
| Table of contents |
1. Executive Summary
2. Introduction
3. STIR's Programming and Variations
4. Evaluation Objectives, Questions, Approach and Methods
5. Results and Interpretations
6. Limitations |
| Download |
https://microdata.worldbank.org/index.php/catalog/5941/download/63721 |
