Screening and Matching with Psychometric Assessments: Randomized Experiments 2011-2013

First experiment

The experimental job candidate sample consisted of 1,354 recent graduates of either community college or university in Jordan, who participated in Jordan NOW 2.0 between December 2011 and November 2012. The sample was obtained through advertising the program through advertisements in local newspapers, radio stations, cafes, and Facebook, along with a telephone campaign to recent graduates from 14 universities and community colleges in and around Amman. Youth were told this program would be free to participate in, and had the goal of helping match them to jobs. Eligibility was restricted to Jordanians who had graduated from community college or university since May 2009. In the first four months the program was restricted to females only, before also being opened up to males.

Candidates signed up and completed a screening process on a rolling basis throughout 2012. Researchers worked with a Jordanian psychologist and Business Development Center (BDC), a leading training services firm in Jordan, to develop and test a labor market matching service for educated graduates. The college graduates received four hours of tests that measured their mental reasoning, English proficiency, Excel proficiency, soft skills such as the ability to work well in groups, and personality traits.

The research team then stratified the sample by gender and screening batches (110 strata), and randomly assigned three-quarters of the candidates within each strata to a treatment group that would be attempted to be matched to jobs, and one-quarter to a control group that would not. This continued until December 2, 2012, resulting in 343 individuals being assigned to the control group, and 1,011 to the treatment group.

The firm sample consists of a primary, experimental sample, and a secondary, booster, sample. The experimental sample consists of 2,279 small and medium firms, who were selected via a listing survey that screened firms according to whether they planned on hiring a worker in the next six months, and whether they would consider young workers and female workers for these positions. we then randomly assigned half the firms to treatment, which would be attempted to be matched to job candidates, and half to control, which would not. After no hires occurred in a first phase, the control group was also offered the treatment, and a secondary booster sample of 175 larger firms that were potentially interested in hiring workers through the program was added.

Second experiment

The job candidate sample consists of 2,002 recent graduates from community colleges and universities, who participated in the new program Yalla Watheefa (Let's Go Get a Job). The program was advertised through flyers, brochures and Facebook. Yalla Watheefa ambassadors would also recruit candidates on university campuses around Amman. Yalla Watheefa was advertised as a free program for fresh graduates to inform them of job openings. Recruitment of participants took place in April and May 2013, just before graduation. Both males and females were eligible to participate, with eligibility limited to individuals graduating between May 2012 and May 2013. Individuals who were already employed, who were not seeking a job, or who were older than 26 were excluded from the sample. To enroll individuals filled out an application form that served as a baseline survey, providing demographic information, field of study, university, prior work experience, job type preferences, and reservation wage.

The sample of businesses consisted of 33 firms with 51 job vacancies. Some vacancies were for more than one worker in that job, so in total these firms were seeking to fill 178 positions. High prestige job openings included openings for accountants, credit officers, IT staff, nurses, engineers, and managers. Low prestige job openings include sales jobs, waiters, data entry, receptionist, and bagging groceries and are jobs that do not require a specific academic major and can typically be done by individuals without a college degree. This threshold was chosen to approximately split the sample of graduates in half, resulting in 958 individuals who said they were willing to work in low prestige jobs, and 1,047 who said they were only willing to work in high prestige jobs.

The research team randomly sent announcements of vacancies to recent graduates, sending a total of 9,820 announcements.

For each job vacancy, firms provided information on the position, qualifications required, and a salary range. Then the research team used information on gender, academic major, and job type prestige to classify candidates as a "good fit", "bad fit", or "no fit" for each job. The job announcement for each job opening was sent to an average of 205 randomly chosen candidates, stratifying this randomization so that two-thirds of those receiving the announcement would be deemed good-fits, and one-third bad-fits. For the first two jobs researchers piloted both phone calls and SMS messages and had identical response rates. As a result, SMS was used to inform candidates about the remainder of the jobs for convenience and costs purposes.

A week after the job announcement was sent, researchers followed up with a random sample (typically 100) of the candidates who had received job announcements to monitor their self-reported application rates and the reasons for not applying if they didn't apply. The research team also followed up with the firms to monitor how many applications they received, and at the end of the hiring process, how many interviews the firms scheduled, and whether job hiring took place.