The Impact and Sustainability of Community-Led Total Sanitation and Hygiene (CLTSH) 2017-2019

Andilaye Impact Evaluation - Baseline, Midline, and Endline Surveys

We are interested in understanding the impact the Andilaye intervention has on sustained WASH behavior change, diarrhea, and mental well-being. As such, we conducted a sample size determination for mental well-being, as this impact was deemed the most restrictive in terms of required sample size. Our sample size analysis indicated that we should recruit and enroll a total of 30 households from each of 50 study clusters (25 clusters per study arm), for a total of 1,500 enrolled households. During our power analysis, we considered the possibility of increasing the number of study clusters and decreasing the take size within each cluster. However, this approach presented considerable logistical and financial implications. Our final sample size accommodates for 10% of households being lost to follow-up, poor compliance to intervention allocation, inconsistent adherence to the intervention, and/or household level loss to follow-up. With data collected from 1,589 households, our baseline survey sample exceeded the targeted number of required households, per our sample size calculation, and the midline, with 1,496 met expectations with respect to anticipated attrition.

Sampling Methodology:

The Andilaye team employed a structured sampling strategy to randomly select eligible kebele clusters and study households. The primary sampling unit (PSU) for this study was the kebele; specifically, any rural or peri-urban kebele that is accessible throughout the course of the year. The ultimate sampling unit (USU) for this study is the household; specifically, any household residing in a targeted, sentinel gott within a randomly selected study kebele. While we randomly selected eligible study clusters (i.e., kebeles), we purposively selected gott(s) from which we randomly selected study households. We utilized a ‘fried egg’ approach to purposively select one or two gotts that are either situated in/near the center of the kebele (if there are centric gotts) or are not adjacent to any other study kebele (in the event there are no centric gotts). The number of targeted gotts depended only on the number of eligible households that consented to participate in the study. The purposive selection of data collection sites within study clusters via the ‘fried egg’ approach is justified, as it minimizes spill-over of intervention effects and other externalities associated with the research between intervention and control clusters, especially those adjacent to each other. In accordance with our sample size calculation, we randomly selected approximately 30 households total per kebele cluster. Not all households met eligibility criteria for inclusion in the study sample, and some households refused to participate in the study.

Target Study Population:

Our target study population included all households residing in randomly selected rural and peri-urban kebeles that are accessible throughout the course of the year in the Farta, Fogera, and Bahir Dar Zuria woredas. These woredas are located within South Gondar and West Gojjam Zonesin Amhara National Regional State.

Inclusion & Exclusion Criteria:

Kebele-level criteria: Rural and peri-urban kebeles in the Farta, Fogera, and Bahir Dar Zuria woredas that are accessible throughout the course of the year were targeted for selection into the evaluation. Given intervention implementation is being supervised by local government officials (e.g., Woreda Health Officers, Health Center HEW supervisors), it was necessary for the kebeles to be accessible throughout the course of the year, to demonstrate proof of concept regarding the effectiveness of the Andilaye intervention. While sanitation coverage and utilization were originally incorporated as inclusion criteria, the veracity of those data were questionable in many kebeles in which initial visits were made (i.e., only one latrine observed in a community in which sanitation coverage was reportedly over 80%, and community reports of this being the case for as long as people could recall). Due to uncertainty with regard to the sanitation coverage and utilization data, and the fact that it became apparent during formative research that behavioral slippage was rampant even in kebeles previously declared as open defecation free, we decided to drop those criteria from inclusion requirements. The Andilaye team did discuss this change in selection criteria with relevant donors, who agreed the study would demonstrate added value if it could include otherwise eligible kebeles, regardless of their sanitation coverage and use or previous CLTSH triggering status, to explore issues related to behavioral maintenance and prevention of behavioral slippage. This evaluation does not include a mass drug administration (MDA) component and has not sought to influence the timing of such activities in study communities. The presence of MDA in the study area will not alter the research questions assessing the impact of the Andilaye intervention on targeted health impacts, behavior change, or sustainability of improved sanitation and hygiene practices.

Household-level criteria: Inclusion criteria for the Andilaye Impact Evaluation included any household randomly selected from the gott census book that resides in the target gott(s) that: 1. Has at least one child aged 1-9 years and consented to allowing study staff to observe the children, specifically their faces and hands; 2. Provided consent to participate in our study, with at least one adult household member consenting to serve as the survey respondent. We excluded from enrollment in our study any household that: 1. Refused to provide consent to participate in our survey; 2. Was repeatedly vacant or does not have an appropriate member of the household (capable adult) home to serve as the household’s respondent after three attempts to engage the household; and 3. Did not have a household member aged 1-9 years (at baseline) living in the household. After consulting with the field supervisor, the enumerator replaced these households with the next randomly selected household on the eligible household register. Field supervisors and study supervisors from Emory supervised field activities and ensured enumerators were only surveying households within the eligible household sampling frame in order to guarantee the sample was random and equitable. Actual recruitment of households selected for the Andilaye Impact Evaluation took place within the home compounds. The enumerator made contact with adult members of the household; she explained the purpose of the visit, the purpose of the study, and asked the respondent if s/he would like to participate in the study. Enumerators assessed household level eligibility by asking potential survey respondents a series of questions that lead to a determination of eligibility. Potential survey respondents were informed that they could choose not to participate in the study, that they could refuse to answer any question, and that they could stop the survey for any reason at any point in time.

Sampling Frames & Sample Selection:

All kebeles that are rural or peri-urban, and are accessible throughout the course of the year, per Woreda Health Office definition, situated in the three targeted woredas were eligible for inclusion in our study. The enumerated list of all kebeles the three respective Woreda Health Offices maintain served as the first level sampling frame. From this sampling frame, we employed a random number generator and a stratified (at the woreda level) selection approach to identify 50 eligible kebele level clusters from across the three woredas for inclusion in our study. Given each of the three woredas vary with regard to their hydrogeological conditions and the size and number of kebeles, we deemed a stratified selection approach appropriate, and used it to select study clusters. Of the 50 clusters, 22 were selected from Farta, 12 from Fogera, and 16 from Bahir Dar Zuria. An even number of clusters were selected from each woreda to ensure an equivalent sample size between the intervention and control clusters selected from each woreda. Once an appropriate gott was selected from each kebele, the team worked with the HEW to obtain a list of all of the households within the gott, specifically those with a child aged 1-9 years, as per study inclusion criteria. In order to operationalize this in a standardized manner, we used The Carter Center’s (TCC) household census books, which are kept at the Health Post and enumerate all households and all household members (by age) residing within the household. At each Health Post, we obtained all TCC census books pertaining to the relevant gott(s). When there was more than one version of the TCC census book (i.e., books from censuses conducted during different years), the book with the latest census data was used as the gott sampling frame. After all relevant books were gathered, the total number of households in the gott was determined (by counting up the number of households from each of the gott’s TCC census books), and a random number generator was used to generate a list of 60 households per gott that reportedly had a child between the ages of 1-9 years. The list exceeded the total number of households that would be required for enrollment in each gott in order to allow for replacement in the field if the household did not, in reality, meet inclusion criteria (i.e., have at least one child aged 1-9 years), have an eligible respondent available after three attempts, or have an eligible respondent who consented to participate in the study. During data collection, each field supervisor and enumerator were provided a list of households, and instructed to visit each household, starting with the first household on the list, to invite them for enrollment in the study, ask an eligible adult representative to provide informed consent, and administer the household survey. If households were absent, no eligible adult respondent was available or refused to consent, or upon further conversation with the household it became apparent that the household was not eligible (e.g., the death of the only child between 1-9 years, the child was actually older than 9 years), the enumerator electronically recorded the information and notified the field supervisor. If the household was absent or no eligible adult was available, the enumerator visited the household three times prior to replacing that household with the next household on the list - after contacting the field supervisor to confirm the replacement. If the household was otherwise ineligible (e.g., refused consent, no children within the targeted age range), the enumerator replaced that household with the next household on the list - after contacting the field supervisor to confirm the replacement.

Field Procedures:

During the household surveys, enumerators sought out adult (i.e., person over the age of 18 years who was capable of understanding and undergoing the informed consent process) respondents within selected households, with preference going first to the primary female caretaker of the index child (i.e., the youngest child between the ages of 1-9 years residing in the household), as she would tend to know the most about the latrine use, defecation, and personal hygiene practices of most members of her household. If she was not available, enumerators sought out other household members in the following order: eldest available female caretaker, eldest available female household member, eldest available male caretaker, or eldest available male household member. All household members present during survey administration were asked to self-report on their own sanitation and hygiene habits, and hand cleanliness and facial and hand cleanliness were assessed for the primary survey respondent and all children under the age of ten years, respectively. If, after three attempts at a household, no eligible adult respondent was available or agreed to consent to participate in the study, the household was replaced by the next randomly selected household in the gott.