ESOMAR is an organization that standardized 28 questions to assist researchers in their procurement process of online sample.
1. What experience does your company have in providing online samples for market research?
Centiment LLC was formed in 2015. Since that time we have been involved in building our panel from the ground up employing the most up-to-date technologies for profiling, targeting, and engaging respondents in applicable survey studies.
2. Please describe and explain the type(s) of online sample sources from which you get respondents. Are these databases? Actively managed research panels? Direct marketing lists? Social networks? Web intercept (also known as river) samples?
Centiment directly recruits and maintains its own database of panelists. Our recruitment sources include social media (e.g. LinkedIn and Facebook) as well as various affiliates. All panelists are pre-recruited and profiled ahead of being targeted for a survey study. We also integrate with select vetted panel partners to expand our reach when needed.
3. If you provide samples from more than one source: How are the different sample sources blended together to ensure validity? How can this be replicated over time to provide reliability? How do you deal with the possibility of duplication of respondents across sources?
We use a technology called fingerprinting which looks at a variety of sources including IP address, device type, screen size, and cookies to ensure only unique panelists enter your survey. We also employ ReCaptcha and various in-survey measures, such as attention checks and response validation, to ensure all panelists are engaging thoughtfully in studies
4. Are your sample source(s) used solely for market research? If not, what other purposes are they used for?
Our panelists are used solely for market research.
5. How do you source groups that may be hard to reach on the internet?
We utilize the targeting capabilities of our recruitment sources to directionally target and recruit respondents on the basis of demographic and firmographic information. Once onboarded, we send respondents through an extensive profiling exercise that we rely upon to precisely target respondents for applicable surveys.
6. If, on a particular project, you need to supplement your sample(s) with sample(s) from other providers, how do you select those partners? Is it your policy to notify a client in advance when using a third-party provider?
We automatically select the panel sources of best fit for a project based on the targeting criteria employed. This may utilize our direct sample, that of a vetted partner, or a combination of both. When expanding to partner panels, the two most important factors are i) the sample meets the targeting criteria of the study and ii) we have consistently reached respondents that fulfill our quality standards.
7.What steps do you take to achieve a representative sample of the target population?
Our system has the ability to employ extensive quotas to provide a sample set reflective of census data when needed.
8. Do you employ a survey router?
While we provide suggested follow-up surveys after the completion or screen-out of a prior survey, we do not define our survey introduction system as a survey router. These suggested surveys are only made available to respondents that have profile data that satisfy the targeting requirements of the survey. We communicate the duration and associated reward of a survey ahead of introducing a respondent to a survey, just as we would on an offer wall.
Conversely, some survey routers will introduce a respondent directly into a survey without communicating any summary detail about a survey, such as the expected time to complete. Not setting this expectation can lead to decreased engagement as the respondent does not understand the expected time commitment.
9. If you use a router: What measures do you take to guard against, or mitigate, any bias arising from employing a router? How do you measure and report any bias?
Not applicable.
10. If you use a router: Who in your company sets the parameters of the router? Is it a dedicated team or individual project managers?
Not applicable.
11. If you use a router: Who in your company sets the parameters of the router? Is it a dedicated team or individual project managers?
Not applicable.
12. What profiling data is held on respondents? How is it done? How does this differ across sample sources? How is it kept up-to-date? If no relevant profiling data is held, how are low-incidence projects dealt with?
Profile data is collected and stored on our servers. Hundreds of firmographic and demographic data points are gathered. We cross-map our profile data against those of our partners to ensure our respondents can participate in surveys beyond those we directly provide. These profile questions are periodically re-introduced. We look for inconsistencies in select profile data as a component of our security to ensure respondents are accurately reporting profile data over time.
13. Please describe your survey invitation process. What is the proposition that people are offered to take part in individual surveys? What information about the project itself is given in the process? Apart from direct invitations to specific surveys (or to a router), what other means of invitation to surveys are respondents exposed to? You should note that not all invitations to participate take the form of emails.
When a respondent has stored profile data that meets the criteria of a study, that study is automatically unlocked and made available on their offer wall. To drive traffic to a study, we will run periodic batches of invites via email or SMS.
The only details about a survey that are shared are the associated reward and expected minutes to complete the study. No additional information, such as the topic of the survey, is shared to avoid any selection bias.
14. Please describe the incentives that respondents are offered for taking part in your surveys. How does this differ by sample source, by interview length, by respondent characteristics?
The majority of our respondents opt to be compensated via PayPal. At times we may offer alternative methods of compensation such as gift cards or enabling a respondent to pledge their earnings to the non-profit of their choice.
In general, longer studies with highly specific audience targeting correlate with a higher incentive to respondents.
15. What information about a project do you need in order to give an accurate estimate of feasibility using your own resources?
Target audience, question count, estimated minutes to complete, and any quota requirements.
16. Do you measure respondent satisfaction? Is this information made available to clients?
While we regularly solicit feedback from our respondents on their experience with us, we do not make this information available to researchers. We also do not ask for feedback from respondents on a survey-by-survey basis.
17. What information do you provide to debrief your client after the project has finished?
Any specific metrics that need to be communicated post-data delivery should be requested upfront to ensure we are storing that data to make it readily available.
18. Who is responsible for data quality checks? If it is you, do you have in place procedures to reduce or eliminate undesired within survey behaviors, such as (a) random responding, (b) Illogical or inconsistent responding, (c) overuse of item non-response (e.g. “Don’t Know”) or (d) speeding (too rapid survey completion)? Please describe these procedures.
We have extensive fraud scoring and quality assessment measures to ensure we’re maintaining a healthy panel. If you need to cancel (reconcile) responses, please email your project manager an Excel export of data including your “aid” (answer ID) and associate response data.
To mitigate the need to cancel any responses, we suggest an attention check be included in the first 3 minutes of your survey to ensure respondents that aren’t fully engaged in your survey are screened out in real-time. Standard acceptable thresholds for canceling a complete include open text that is not related to the subject matter, straight-lining, or speeding. Speeding is deemed as completing a survey in under 30% of the median completion time. Any reconciliations must be submitted to your project manager for review within 10 calendar days from the date your data set was delivered.
19. How often can the same individual be contacted to take part in a survey within a specified period whether they respond to the contact or not? How does this vary across your sample sources?
We monitor engagement levels across our platform. Respondents that are deemed to provide the highest quality responses will be invited to more surveys than others. To ensure respondents are not participating in too many surveys, we set daily and weekly caps on the number of surveys that can be completed. These limits are not absolute as they may need to be temporarily relaxed to enable a very niche audience to participate in a survey.
20. How often can the same individual take part in a survey within a specified period? How does this vary across your sample sources? How do you manage this within categories and/or time periods?
While in some instances a respondent may be permitted to take two surveys consecutively, they can only complete a particular survey one time. Our fingerprinting technology focuses on ensuring only unique respondents complete a particular survey.
21. Do you maintain individual-level data such as recent participation history, date of entry, source, etc., on your survey respondents? Are you able to supply your client with a project analysis of such individual-level data?
If requested ahead of a study, we can provide detail to a researcher on a number of factors such as the date of entry to that particular study. To adhere to privacy constraints, we do not provide details on a panelist beyond those applicable to a researcher’s individual study.
22. Do you have a confirmation of respondent identity procedure? Do you have procedures to detect fraudulent respondents? Please describe these procedures as they are implemented at sample source registration and/or at the point of entry to a survey or router. If you offer B2B samples what are the procedures there, if any?
Across both B2B and consumer audiences, we employ a range of deduplication methods through our fingerprinting technology. After targeting a region, we verify the location through IP address and also engage proxy detection. We maintain a fraud score on each respondent based on canceled completes, speeding, and other quality assessment measures. While we gather profile data and monitor the consistent reporting of that profile data, to ensure privacy, respondents participate in surveys on an anonymized basis.
23. Please describe the ‘opt-in for market research’ processes for all your online sample sources.
Respondents must agree to a Terms of Use and Privacy Policy to participate in surveys.
24. Please provide a link to your Privacy Policy. How is your Privacy Policy provided to your respondents?
Centiment’s Privacy Policy can be found here. Respondents must agree to a Privacy Policy ahead of participating in a survey.
25. Please describe the measures you take to ensure data protection and data security.
As described above, we employ fingerprinting technology to avoid respondent duplication, and extensively monitor respondent participation in surveys through our fraud scoring system.
26. What practices do you follow to decide whether online research should be used to present commercially sensitive client data or materials to survey respondents?
Centiment complies with all applicable data protection laws and regulations including GDPR and CCPA.
27. Are you certified to any specific quality system? If so, which one(s)?
Not applicable.
28. Do you conduct online surveys with children and young people? If so, do you adhere to the standards that ESOMAR provides? What other rules or standards, for example, COPPA in the United States, do you comply with?
Centiment adheres to all applicable laws and regulations in the respective geography research is being conducted. In the US, children under the age of 13 are not permitted to participate in surveys.