Pacific Labor Mobility Survey 2021-2023

Wave 1

Rigorous quality control measures were put in place during the implementation of the PLMS.
Enumerators were selected based on their performance during training sessions. Their performance during survey implementation was regularly assessed; briefing, feedback and corrective actions were provided when issues were detected.

The questionnaire for the face-to-face segment of the household survey was scripted in Survey Solutions, a CAPI application developed by the World Bank. For implement the phone-based segment of the household survey and the worker survey, the questionnaires were scripted in TELSIA (Telephone Survey Integrated App), an all-in-one CATI application. Both Survey Solutions and TELSIA allow us to capture a wide range of data with ease, with features allowing enumerators to synchronize collected data to a secured server, and supervisors to monitor progress and detect issues in real time.

The scripted questionnaires were checked and tested by the core survey team. Once the first version of a scripted questionnaire was ready, the field team conducted a survey plot with about 40-50 households. It assessed the questionnaire duration, the CAPI/CATI scripts, the survey protocols, and all operating procedures. The CAPI/CATI applications were adjusted during and after the pilot. The enumerators were then trained again on a fully debugged application.

Data collection was carried out under stringent protocols to maximize response and reach the target sample size. This includes:

1. Emphasis on the initial contact and introduction
2. Emphasis on the voluntary nature and confidentiality of participation in the survey
3. For phone-based data collection, for each phone contact enumerators attempted least seven calls at different times of the day on different days of the week to optimize rate of response. This was then followed by another call by the supervision team to those who refused to participate in the survey.

In terms of data management and data quality assurance, a Comprehensive Field Quality Control (CFQC) system ran over the entire data collection period to detect errors and undesirable interviewer effects, and to take corrective actions on time. The CFQC system includes:

1. Computerized built-in validation checks in the CAPI/CATI application detected issues in the collected data, based on a wide range of criteria on consistency, validity, and value range.
2. A comprehensive monitoring dashboard provided daily statistics on collected data as well as performance of each enumerator.
3. A composite Interviewer Risk Index synthesized the survey-specific data quality indicators and swiftly identified underperforming interviewers. This led to the elimination of interviews by one interviewer in Vanuatu and two interviewers in Kiribati.
4. Non-response items were closely monitored to minimize potential nonresponse bias.
5. Double-blind back-calls were used to verify the survey implementation by enumerators and to check the accuracy of the collected data.
6. For face-to-face data collection, a map of all interview locations based on built-in GSP coordinates were used to monitor the consistency between fieldwork and sample design.
7. For phone-based data collection, random audio supervision by a team of auditors monitored compliance with the calling and interviewing protocols as well as the accuracy of the registered answers.
8. The principal investigators had direct access to all CFQC elements and collected data in real time. The investigators had weekly supervision meetings with the field team to discuss and address undesirable issues when detected.