TGM Research answers ESOMAR’s 20 Questions to Help Buyers of AI-Based Services for Market Research and Insights

Generative AI in market research is expanding, bringing new opportunities and challenges. To ensure clarity and trust, ESOMAR has developed "20 Questions to Help Buyers of AI-Based Services for Market Research and Insights." These questions provide a framework for responsible AI use.

TGM Research proudly presents its comprehensive responses to ESOMAR's questions. As a leader in AI-driven insights, TGM Research is committed to innovation, accuracy, and quality. Our responses highlight how we use AI to deliver cutting-edge, trustworthy data analysis and data creation solutions.

A. Company profile

1. What experience and know-how does your company have in providing AI-based solutions for research?

TGM Research has been a leader in using AI and machine learning to improve market research for over four years. Our team of data experts works closely with our research staff to create and use AI tools in all parts of the research process, from designing studies to collecting and analyzing data and making reports. We've used AI in more than 50 successful projects for clients in different industries.

2. Where do you think AI-based services can have a positive impact for research? What features and benefits does AI bring, and what problems does it address?

At TGM Research we believe AI can revolutionize market research in several key areas:

a) Intelligent survey/sampling design to boost representativeness and engagement
b) NLP and sentiment analysis for richer insight extraction from unstructured data
c) Predictive modeling to forecast consumer trends and market dynamics
d) Monitoring data quality related elements and flagging outliers for the human supervision (HITL)
e) Automation of time-consuming data processing and analysis tasks

3. What practical problems and issues have you encountered in the use and deployment of AI? What has worked well and how, and what has worked less well and why?

In our experience using AI for market research:

What worked well:

Using high-quality, relevant data to train AI models and continuously improving them has been key to getting accurate results.
Having our experts guide AI development and check outputs has helped prevent issues like AI making things up and ensured AI aligns with research goals.
Using AI techniques that can be explained has helped build trust among clients and stakeholders by showing how AI makes decisions.

What didn't work as well / key learnings:

Using generic AI models without tailoring them to market research led to less accurate results and poor fit with research needs.
Not having strong data security, especially when using external AI support, risked client confidentiality and data integrity. We only use closed systems with strict controls.
Overestimating what AI can do and underestimating the need for human oversight resulted in some flawed outputs and misinterpretation of results.

Key lessons:

Data quality, relevance, and security are critical.
Humans and AI models need to work together for reliable and appropriate results.
AI models need continuous monitoring and improvement to maintain performance and alignment with changing research needs.

B. Is the AI capability/service explainable and fit for purpose?

4. Can you explain the role of AI in your service offer in simple, non-technical terms in a way that can be easily understood by researchers and stakeholders? What are the key functionalities?

Key capabilities:
At TGM Research, we leverage AI to enhance the market research process while maintaining a strong focus on data quality. Our modular AI tools cover the research workflow end-to-end:

Automated Data Coding: We use NLP to automate the coding of unstructured data. Our data quality monitoring system regularly assesses the accuracy of NLP outputs against human-coded samples.
We employ data cleansing and validation procedures, and our monitoring system checks for data completeness, consistency, and outliers.
Predictive Modeling: Our predictive simulator enables what-if scenario exploration and trend forecasting. We ensure the quality of historical training data and continuously monitor the model's performance against real-world outcomes.

Across all our AI functionalities, we have integrated a comprehensive data quality monitoring solution Research Shield. This system allows us to enforce strict data quality rules, monitor quality metrics in real-time, and maintain data lineage records. By continuously monitoring and assuring the quality of the data that feeds our AI models, we deliver reliable, accurate, and trustworthy insights.

Our commitment to data quality and rigorous validation sets us apart. By combining AI with robust data quality measures, we deliver cutting-edge research solutions that provide deep, actionable insights while upholding the highest standards of data integrity and reliability.

5. What is the AI model used? Are your company’s AI solutions primarily developed internally or do they integrate an existing AI system and/or involve a third party and if so, which?

Our AI solutions combine proprietary models developed fully in-house by our data science teams with best-in-class open source components (e.g. BERT, GPT).

We do not outsource core AI development but may engage specialist vendors for niche tools.

6. How do the algorithms deployed deliver the desired results? Can you summarise the underlying data and the way in which it interacts with the model to train your AI service?

Our models are tailored for specific market research applications, trained extensively on our own historical project data and real-time market feeds. We do not train models on client’s datasets.

Techniques span supervised learning, NLP, time-series modeling, etc. We employ transfer learning to adapt models across categories.

C. Is the AI capability/service trustworthy, ethical and transparent?

7. What are the processes to verify and validate the output for accuracy, and are they documented? How do you measure and assess validity? Is there a process to identify and handle cases where the system yields unreliable, skewed or biased results? Do you use any specific techniques to fine-tune the output? How do you ensure that the results generated are 'fit for purpose'?

Rigorous testing is core to our AI development process. We verify model outputs against internal benchmarks, with extensive back-testing. Key metrics tracked include accuracy, F1 scores and human evaluations. Outlier detection and adversarial probing identify potential failure modes. Models are continuously monitored and retrained.

8. What are the limitations of your AI models and how do you mitigate them?

We are open about what our AI models can and cannot do. We keep detailed records of what the models are meant for and how well they perform. When the AI confidence is low, we rely on our human experts to make the call. We also give clients advice on how to use insights from AI responsibly.

9. What considerations, if any, have you taken into account, to design your service with a duty of care to humans in mind?

Our AI ethics guidelines, created with input from people inside and outside the company, guide how we develop and use AI. The main principles are:

Making sure humans are still in control
Thinking about the impact on society as a whole

D. How do you provide Human Oversight of your AI system?

10. Transparency: How do you ensure that it is clear when AI technologies are being used in any part of the service?

At TGM Research, transparency is a core principle in our approach to integrating AI into our market research and data collection services. We ensure clarity on AI usage through:

Clear documentation: We provide detailed methodology documents specifying which components involve AI, the models used, and AI's role in generating results.
Flagging AI-generated content: All outputs with AI-generated content are clearly labeled, with disclaimers on limitations and potential biases.
Transparent reporting: Our reports include sections detailing AI methodologies, data sources, assumptions, and human oversight measures.

Our aim is to use AI in a responsible and transparent way, making sure clients can clearly see where AI is supporting human knowledge. We are dedicated to using AI ethically and in a way that can be explained, to improve market research.

11. Do you have ethical principles explicitly defined for your AI-driven solution, and how in practice does that help to determine the AI's behaviour? How do you ensure that human-defined ethical principles are the governing force behind AI-driven solutions?

Yes, TGM Research has explicitly defined ethical principles that align with ESOMAR's Code of Conduct to guide the development and use of our AI-driven market research solutions. These principles cover key areas such as human oversight, fairness, explainability, privacy, and societal benefit.

To ensure these principles determine AI behavior in practice, we:

Embed ethical considerations into every stage of our AI development process
Employ human-in-the-loop frameworks for continuous monitoring, validation, and fine-tuning of AI models
Continuously audit AI systems for fairness, accuracy, and reliability
Engage with diverse stakeholders for input and feedback on AI practices

We ensure human-defined ethical principles are the governing force behind our AI solutions through alignment with ESOMAR's principles in leveraging AI responsibly to enhance market research insights while upholding trust and ethical standards

12. Responsible Innovation: How does your AI solution integrate human oversight to ensure ethical compliance?

In addition to human-guided data curation and model training, we employ human-in-the-loop frameworks where researchers and other team members dynamically provide input to guide AI inferences. This could include specifying decision guardrails, providing active feedback for model fine-tuning, and flagging unexpected outputs for review.

E. What are the Data Governance protocols?

13. Data quality: How do you assess if the training data used for AI models is accurate, complete, and relevant to the research objectives in the interests of reliable results and as required by some data privacy laws?

We have strict data provenance tracking and quality control, with data lineage records maintained. All personal data is de-identified. Models are trained only on vetted, representative data with regular refreshes.

14. Data lineage: Do you document the origin and processing of training or input data, and are these sources made available?

Data sources and processing steps are documented, with data flow maps available to clients on request. We have data sharing agreements with providers specifying usage rights.

15. Please provide the link to your privacy notice (sometimes referred to as a privacy policy). If your company uses different privacy notices for different products or services, please provide an example relevant to the products or services covered in your response to this question.

Our privacy policy details what data we collect, how it is used and secured, and processes for inquiries/corrections. We adhere to global regulations like GDPR, CCPA, etc.

16. What steps do you take to comply with data protection laws and implement measures to protect the privacy of research participants? Have you evaluated any risks to the individual as required by privacy legislation and ensured you have obtained consent for data processing where necessary or have another legal basis?

We conduct regular privacy impact assessments and obtain necessary consents. Access controls and encryption secure personal data, with audit trails maintained. Retention schedules enforce timely deletion.

17. What steps do you follow to ensure AI systems are resilient to adversarial attacks, noise and other potential disruptions? Which information security frameworks and standards do you use?

Our InfoSec and data science teams jointly monitor AI systems for vulnerabilities. We adopt a defense-in-depth strategy spanning network, application and data-level security controls.

18. Data ownership: Do you clearly define and communicate the ownership of data, including intellectual property rights and usage permissions?

We explicitly specify in client agreements any data ownership/usage rights granted to us to develop AI solutions. Clients retain ownership of their raw data and resulting insights (including code lists for open-ended questions).

19. Data sovereignty: Do you restrict what can be done with the data?

We comply with client requirements on data residency and cross-border restrictions. Client data is not commingled without consent. Our data processing agreements define usage limitations in accordance with applicable regulations.

20. Ownership: Are you clear about who owns the output?

Insights and reports generated for a client are owned by them. We do not repurpose client-specific outputs without approval. Any general AI models we develop are owned by TGM Research.

Get better responses. Faster.

With TGM, you have more than 130 markets and multiple local teams helping to execute your global project