Written by Dr Norma Wong. She is a postdoctoral researcher at the Centre for Intercultural Communication, at VID Specialized University (Stavanger), Norway and a member of MIGREL research group.
I still remember that late afternoon in May when Oleksandr Ryndyk and I huddled in front of our computer screens in a virtual meeting, rushed with adrenaline when he hit that button to activate our Lime Survey and Facebook ad. The survey is the result of months of designing and tweaking. Unlike semi-structured or life story interviews, surveys do not allow for adaptation along the way. Any blind spot or mistake might only surface much later in the process of analysis and by then we could do nothing about it.
The survey I am referring to is for our research project about the employment situation of the newly arrived refugees in Norway (www.maviproject.no). The project takes a multi-stakeholder and mixed-method approach to offer a thorough investigation on the topic from different perspectives. In addition to interviews and participant observation, this survey is an important source of quantitative data for our project.
Placing Facebook Ads to recruit participants
Deviating from the conventional way of random sampling using a list from the official registry, we experimented with using Facebook as our recruitment tool. This is a relatively new method but starts to be discussed and evaluated seriously within the field of migration studies[i], as the global coverage of Facebook has become more ubiquitous. Facebook’s database often gives rather accurate estimates on the size of different segments of the population, and its algorithms have been professionally designed and perfected over the years to effectively reach the right population.
Figure 1: A mockup of our Facebook Ad on different platforms. Our ad, as well as the survey, was written in Arabic to appeal to the target population.
The struggle against biases
Fast-forward to eight weeks after that moment Oleksandr hit the button. We are more than pleased by the result. Over the course of eight weeks, we have experimented with four different configurations for the ad, which have received over 12 000 clicks, and attracted over 2300 to start the survey; one in three of which went on to complete the full survey. This would have been unlikely to be achieved within such a short period if we had created and distributed the survey the conventional way. Not only are we pleased by the number of full responses, but we are also thrilled that there is a good male to female respondent ratio, as we have planned by budgeting it based on known statistics. (See Figure 2)
Figure 2: Result summary from our ad campaign from 21st May to 8th July 2021 on Facebook
However, it appears that we are not able to circumvent another inherent bias, that is a high proportion of our respondents who are more highly educated and economically active. This is inherent in online surveys, that they tend to catch the attention of those who are more active on social media are more technologically skilled.
A common tool in statistical analysis in situations as such is to use the method of weighting on our dataset to rebalance the disproportionally large number of the highly educated. We will do this by comparing the profile of our data with that from the national statistics on the newly arrived refugees. Reflecting on our experiment, Oleksandr and I have already started writing an article on the different configurations of Facebook advertisements, and their effects on the representativeness of the respondents.
Upcoming plans for the dataset
Going forward as planned, we will go beyond the preliminary descriptive observation, and start to go into modelling our data for more concrete findings. With the data given to us from over 2000 respondents, our quest is to answer the question “what factors correlate with a successful labour market integration amongst the newly arrived refugees in Norway?” In particular, what would help a newly arrived refugee find a skill-matching job? What are the likely effects of social networks and organized social activities in the process? Lastly, we would also like to look further and consider how our quantitative analysis could be combined with the interviews our research team have conducted earlier, which contains rich qualitative data. With such knowledge, we hope that we will find evidence-based answers that would help more refugees to be able to find their way in the labour market and stand on their feet.
[i] See for example, Potzschke and Braun (2017) Migrant Sampling Using Facebook Advertisements: A Case Study of Polish Migrants in Four European Countries. Social Science Computer Review 35(5) p.633-653 DOI: 10.1177/0894439316666262 In the article, the authors have documented carefully their decisions concerning the content and the setting of campaigns. We have taken a lot of references from their work while designing ours.
To be notified every time we write a new blog post subscribe to our blog at the top of this page!