Home / Topics / Research Design / Theory and Method / Web-based / Exploring Online Recruitment for Web-based Research Studies

Exploring Online Recruitment for Web-based Research Studies

Posted on June 6, 2011

Joel Moskowitz (bio) and Diana McDonnell (bio) discuss the benefits and challenges of using online recruitment methods for a community-based online smoking cessation intervention for Korean Americans.

 

Q: What were the objectives of the study?
A: The primary goals were to implement and evaluate an online smoking cessation intervention specifically geared toward Korean Americans (KAs) and to explore the motivators for quitting smoking.

But the study also provided an opportunity for our team to experiment with – and evaluate – various recruitment methods, including traditional, online, free, and paid.

Our goal was to recruit about 1,130 participants, but we exceeded that target by over 100, with a total of 1,261 participants.
Q: Who funded the study?
A: The project was funded by the Centers for Disease Control and Prevention as a project of the Prevention Research Centers, a network of academic, community, and public health partners that conducts applied public health research.
Q: Why did you decide to use primarily online recruitment methods for this study?
A: First of all, almost two thirds of all KAs have access to the Internet, so we felt that an online approach would enable us to quickly and affordably reach a large number of potential participants across the country.

Secondly, research suggests that more traditional recruitment strategies – like television, newspaper, radio, and billboard ads – tend to reach only a tiny percentage of the target population in community-based cessation trials and online cessation programs. Online ads have been more effective overall.

And lastly, since our intervention was web-based, it just made sense to recruit participants through online mechanisms. For example, online ads can take prospective participants directly to the study information with a simple click or two, whereas more conventional – that is, non-online – recruitment methods require people to record contact information and then act upon that information later. This additional step can result in a notable drop-off of potential participants.
Q: What conventional recruitment strategies did you use?
A: We used both free and paid conventional recruitment methods, zeroing in on venues specific to KAs. These efforts included the following:

Free:
  • We posted fliers at local grocery stores and restaurants in Oakland and San Francisco, focusing on those catering to our target population.
  • We held a press conference geared toward KA media outlets, which resulted in a few articles in KA newspapers that included the project's website address.
Paid:
  • We ran a television campaign on two local KA cable channels.
  • We ran four large ads in the free weekly newspaper associated with these KA cable channels.
Q: What web-based recruitment tools did you use?
A: We used both free and paid online recruitment methods, including the following:

Free:
  • We requested help from various KA college student organizations, asking them to spread the word about the study via e-mails to their member lists.
  • We asked our community partners, the Korean Community Advisory Board (KCAB), to send recruitment e-mails to their networks. KCAB is a group of KAs who have a long history of collaborating with us in community-based participatory research projects with the KA community.
  • We contacted KA and Asian American community organizations about placing free ads on their websites to promote the project.
Paid:

Our paid online advertising included both text and graphic ads.
  • For our text ads, we used Google AdWords and Yahoo! Search Ads. These programs enabled us to create our own ads and then choose keywords that link users to the ads when they use those terms in a search. These ads were pay-per-click, meaning that we paid only when users clicked on our ads. We found the Google program to be more effective because it gave us more leeway in choosing keywords than the Yahoo! program, and unlike the Yahoo! program, it allowed us to use Korean language text. Also, our Google ads showed up as "Sponsored Links" in both search results as well as websites with content related to our ad. The ads were much more effective on websites than in search results, probably because very few KAs actually search online for help with smoking cessation.
  • We placed graphic ads, created by a KA graphic designer, on the websites of the Korea Times, Korea Daily, Radio Korea, and SF Korean (an online KA business directory) and nine of its affiliate sites.
We also hired an e-mail marketing service to send study information to almost 30,000 U.S. e-mail addresses associated with Korean ethnic codes.
Q: Did you offer any incentives for participants?
A: We learned through some preliminary qualitative research that significant cash incentives would be an effective motivator for participation in the project. With this in mind, we offered up to $100 per participant to complete 11 brief online surveys (assessing participants' smoking status) over the course of a year.
Q: What cultural considerations did you have to take into account when formulating your recruitment strategy?
A: The smoking rate among KAs is very high, but they are generally unlikely to participate in smoking cessation interventions or even search online for information about quitting smoking. Therefore, we not only had to develop an intervention that would be keenly well-suited to this community, but we had to be particularly strategic in our recruitment efforts.

Language also proved to be an issue because we didn't want to exclude any potential participants based on whether they read English or Korean. However, in order to ensure that our participants were Korean American, and to increase the appeal of the intervention itself, we created primarily Korean ads, resulting in the limited participation of non-Korean readers. The program and survey materials themselves were available in both Korean and English, but only about 20% of the participants used the English materials.

On the other hand, the recruitment e-mail message we sent to 30,000 recipients was in English due to technical constraints, which enabled us to reach quite a few people who may not have otherwise learned about the study.
Q: What was the cost of recruitment?
A: The cost of advertising was much more substantial than we expected, totaling approximately $84,000, or $66.50 per study enrollee:
  • Online graphic ads - $65,000 over 17 months
  • Online text ads (Google Ads) - $14,500 over 27 months
  • Online text ads (Yahoo! Ads) - $250 over 14 months
  • E-mail campaign - $2,500
  • TV ads - $1,200
Additional recruitment costs included:
  • Ad development by graphic designer - $1,580
  • Incentives for participation - $75,000 (average $60 per participant)
Q: What were the most effective recruitment strategies for this study?
A: Unsurprisingly, most of our participants found out about the study through one of our paid mechanisms. Free advertising resulted in very few participants. Almost half of our participants learned about the study from a text link (e.g., Google ads) and about a third of enrollees found the study through a graphic ad (e.g., the ads in one of the online newspapers).

While our search engine ads yielded few results, the Google ads that showed up on websites related to our ad content proved quite cost-effective.

And although we did not specifically evaluate the effect of the incentives on our recruitment efforts, we believe that these payments contributed to our recruitment success. We're currently developing a study that will specifically test the effect of incentives, based on the promising results of this trial.
Q: How did you evaluate your online recruitment strategy?
A: Fortunately, Google Analytics became available fairly early in our study, and this tool was instrumental in helping us track the progress of our online campaigns. We were even able to evaluate the efficiency of individual websites on which our Google ads appeared, and we used this information to customize our Google ad campaign to optimize study enrollment results.

We also asked participants to indicate how they found out about the study, and we analyzed this information as well.
Q: What were some of your key takeaways with regard to online recruitment?
A: Online study recruitment methods can be effective, but unpaid mechanisms such as free online ads are much less productive. One possible explanation for this is that unpaid ads are less likely to target a specific population, which can vary in importance depending on the study. In our case, being able to hone in on the KA population through our paid online advertising was extremely beneficial.

We've also come to realize that paid online ads afford the broad exposure necessary for getting the sought-after results in a cost-effective manner. Many people were exposed to these ads many times, but with the Google and Yahoo! ads, we paid only when users click on the ads. Achieving this repeat exposure through more conventional print-based efforts would have been significantly more expensive.

And lastly, we've learned the value of being flexible and creative in developing recruitment strategies tailored to the target population, especially when that population is typically difficult to reach.

 

 


McDonnell, D.D., Lee, H-J., Kazinets, G., & Moskowitz, J.M. (2010). Online recruitment of targeted populations: Lessons learned from a smoking cessation study among Korean Americans. Social Marketing Quarterly, 16(3), 2-22. doi:10.1080/15245004.2010.500441

Additional References:

Baek, D. H. (2007, January 5). Quit smoking and earn money. Korea Times. Retrieved from http:// www.koreatimes.com/article/358442

Madden, M. (2003). America's online pursuits: The changing picture of who's online and what they do. Edited by L. Rainie (Ed.). Washington, DC: Pew Internet & American Life Project.

National Tobacco Cessation Collaborative. (2007). Innovations in building consumer demand for tobacco cessation products and services. Washington, DC: Academy for Educational Development.

Feedback

Please note that the feedback is viewed only by 4researchers staff and is not intended for communication with individual contributors.

 

Use the form below to submit feedback about this article. If you would like a response, please be sure to include your e-mail address.


More About "Web-based"

1:28

Benefits of Web-Based Data Collection

1:59

Easing Data Entry with Web-Based Tools

Show All...