Become a Smarter Sample Buyer: A Retrospective, 5 Years Later.

by Brad Franz

In 2015, I wrote an article highlighting what I considered the most important questions sample buyers should ask their providers to better understand commissioning sample for their research projects. The intent was simple – help buyers of online sample understand how decisions that are made when purchasing sample (and the types of sample purchased) can significantly affect their downstream research objectives. Smarter buyers hold suppliers accountable, requiring them to provide better solutions – a rising tide lifts all ships mentality.

Five years later, the current state of the data collection industry has grown increasingly complex and it has become even harder to navigate for the sample buyer. The proliferation of sample exchanges and marketplaces, DIY platforms, and aggregation technologies enables easier access to respondents (an undeniable benefit) yet makes it harder to understand the sample considerations that can directly impact your data.

Maneuvering through modern data collection strategies is complicated enough without requiring a sample buyer to fully understand a series of nuanced questions.  As such, I’m shifting my focus to a few simple concepts, which can be implemented immediately to aid you in becoming a better sample buyer.

Define What Quality Means to You

Quality is arguably the most overused and undefined word in the data collection industry. The term is plastered on every promotional email and touted on every provider’s website. The word “quality” means something very different for each stakeholder along the research process and is dependent on one’s perspective.

It’s important that you define what quality means to you, and hold others accountable to that standard. A simple approach to this complex concept is to assess quality at two phases of the data collection process – pre-survey entry and in-survey quality checks. In the sample process funnel, the better the input, the better the data output. If you monitor your quality metrics over time, you will quickly see trends fall out, differentiating sample suppliers.

Sample Source Matters!

All sample providers are inherently different due to the way they recruit, manage, and engage their panelists. As a result, multi-source sample frames have become an industry best practice. Technology solutions have aided in the ability to manage multiple sources, but have also decreased the attention given to the individual providers. With appropriate control and monitoring, you can expect an increase in reliability and consistency of supply over time.

Additionally, sample fraud is no longer limited to inattentive respondents. Imperium, one of the leading technology firms in sample quality and fraud detection, recently reported that up to 38% of survey respondents are duplicates, fraudsters, or bots.¹  While in-survey quality tools are critical in a holistic sample quality strategy, this data illustrates the importance of intentional source selection for your research projects. Pre-survey fraudulent activity can be significantly reduced by carefully selecting and controlling the data streams that supply respondents to your studies. Source selection, in my opinion, is the single most important factor in establishing a data collection strategy, and is a foundation to build your quality standards.

Sample Source x Respondent Experience = Data Quality 

Don’t forget this equation – it’s often overlooked! Source selection, in combination with the user experience, will directly affect your data quality. Ensuring quality and dependable sourcing, while providing respondents with a positive and inclusive survey experience, will result in more reliable and accurate survey data. Unfortunately, the inverse is also true, and inattention to either factor can reduce confidence in your survey responses. Several factors can play into the respondent’s overall survey experience; some of the most impactful factors include survey topic, incentive compensation, length of interview, questionnaire design, and mobile optimization.

Be Empathetic to Respondents 

Never forget there is a human being on the other side of the survey. That may sound obvious, but the entire data collection industry has commoditized respondents as a result of more efficient sample access paired with competitive cost per interviews (CPIs).  Admittedly, it’s easy to be enamored by a low-cost offering. Marketing research is inherently an expensive undertaking, and the data collection budget is only one factor in the bottom line.

To help put yourself in the respondents’ shoes, ask your providers what incentives are being offered and consider your personal response. What is your time worth? Would you provide thoughtful responses knowing the going rate of the CPI you are commissioning? Price does not guarantee quality, but as sample buyers we need to better understand what impact we have on the respondent’s experience.

Obtaining better data through sampling requires a dedication to data collection standards and best practices. Simple, small changes in collection strategies, with a focus on sample sourcing and participant experience, can go a long way in creating more reliable and dependable data. Sampling is an underserved aspect of the research process and too often overlooked as the first line of defense in the fight for data quality. Choose your providers wisely and partner with those who cultivate and encourage appropriate participation standards. Better sampling is closer than you think!

Interested in reading more? Check out Brad’s other article:

Become a Smarter Sample Buyer – Essential Questions to Ask Your Providers

As always, you can follow Burke, Inc. on our LinkedIn, Twitter, Facebook and Instagram pages.

Sources:

1. “Imperium Announces Pioneering Data Quality Certification Program.” Imperium, 21 Sept. 2020, www.imperium.com/about/news/imperium-announces-pioneering-data-quality-certification-program/.

Feature Image – ©Rawpixel.com – stock.adobe.comstock.adobe.com