Living Standards Measurement Survey 2004 (Wave 3 Panel)

A first data cleaning took place in Albania and implemented by INSTAT in collaboration with ISER and Government of Albania consultants. The cleaning process has involved following activities:

1. defining data checking routines and writing the syntax code of the cleaning programs;
2. generating lists of outliers and inconsistencies for each module to be checked against paper questionnaires;

During the first few days, data cleaning operators have been working on the Export Procedure of the Data Entry Program to check if data export succeeded and to finalize the English version of the dictionaries and error messages. Some changes were made to the Export Procedure due to a problem on the “Agriculturea2” file conversion and to the dictionary structure to check over correct labelling of exported data. The dictionary used during data entry was in Albanian language. So, an accurate comparison of the Albanian and English versions was done to ensure consistency (except for the labelling) between the two. This work was performed by using a freeware software called “Winmerge”, which underlines all the differences between two text files.

Phase two has been devoted to update the Batch Edit (BE) procedure of the Data Entry Program, where a little correction was required to avoid some error messages incorrectly issued by the BE. Afterwards, the routine was applied to check all the errors, and a program in Access was run to associate PSU and Data-entry operator code to each questionnaire selected by the BE. Once obtained the procedure report, a pool of four people from INSTAT started to check all the reported errors and make the necessary adjustments. A copy of all original data in CSPRO software was made. During this work, some atypical circumstances were reported: sometimes errors or warning detected possible data-entry or interviewer problems. For these cases, no correction was made and the occurrence was highlighted in the report. Most of the problems reported by the BE were referred to the “distance that seemed to be inconsistent with the walking time” and “number of hours worked per week” higher than 70. All these situations were checked and corrected if differences between recorded values and paper values were found. At the end of these operations 10 problems (4%) were corrected out of 278 reported errors or warnings. Other 8 strange cases (3%) were underlined on the report.

The following steps were followed to check for questionnaire consistency. An SPSS program was written to check individual information present in the roster (Module 1) and coherence in the dwelling section (Module 2). No questionnaire was found to violate consistency in Module 1 and only one violating one of the dwelling rules.

Afterwards, a check of each qualitative variable value was carried out, by tabulating frequencies by variable and verifying if values were in their expected range. Any problem of this phase was reported, except for some “99” (meaning “not remember”) values assigned to some date variables such as months or years.

Different criteria were used to check for outliers in quantitative variables. A variation of the classic interval around the mean was used for these cases. Since some very asymmetric distributions were found, a skewed interval around the median was adopted. This interval involved MAD (Median Absolute Deviation) and an asymmetric measure based on quartiles. For each module of the questionnaire a SPSS program was run to check the questionnaire consistency related to quantitative variables and to identify outliers. As in the case of Data-Entry checks, all these cases were verified and adjusted if a difference between recorded value and paper value was found. In suspected cases no change was introduced.

The third step was to check split-off households. The consistency dealt with verifying that all individuals not coded as present household members were considered as valid components only if they join a split-off household; otherwise they were considered “refused interview” or “impossible to be contacted”. Only two cases of individuals without split-off household were found and the related corrections made.

Besides the preliminary check implemented right after the survey completion, additional controls were performed at a later stage. No major data entry errors were found, while some inconsistencies were highlighted and fixed. Furthermore, original files were reshaped to obtain individual or household-level dataset, as some initial data files were organized in a different way (e.g. by plot in the case of agriculturea or by income source in the socialassistance). Value labels for occupation and activity, whose coding was provided by INSTAT, were assigned to code-variables. A number of variables were created to better detect and trace households, and to enhance comparability across waves -e.g. cfinloc, which allows selecting households at their final destination, see below-. A careful check of each variable value was fulfilled, by tabulating frequencies and verifying if all values and codes were consistent. After the discrepancies were fixed, the general dissemination files were created.

Finally, an analysis of the differences in the sections’ content between the two waves of APS was performed. This may be useful for analysts aiming at using the longitudinal panel of the three waves, if used in conjunction with the Variable_reconciliation LSMS_PANEL_final document of wave 2. Codebooks for the individual and household-level files were created and are part of the documentation.