Anthropic, an artificial intelligence company, has conducted a large-scale survey utilizing AI to gather insights from over 81,000 participants across 70 languages. While the approach demonstrates significant speed and breadth, it has drawn criticism from experts in the field of digital technology.
The company tested a specialized AI interviewer designed to collect detailed textual responses. This program posed fundamental questions to participants, such as their recent experiences with chatbots or their preferences for technological advancements, and then generated follow-up questions based on the initial answers.
According to Anthropic, this method allows sociologists to conduct in-depth interviews on an unprecedented scale. By replacing lengthy human moderation with an algorithm, the AI was able to simultaneously collect subjective opinions from tens of thousands of individuals worldwide, ensuring rapid data processing.
However, digital technology researchers specializing in qualitative assessment have raised concerns that while AI tools can generate extensive text, they do not constitute genuine qualitative research. They highlight three primary reasons for this skepticism:
- Algorithmic Interaction Over Genuine Dialogue: Quantitative research requires strict controlled conditions, while qualitative interviews rely on flexibility. A human interviewer can adapt to conversational nuances, interpreting tone, mood, and body language to steer the discussion. In contrast, Anthropic’s AI operates solely within the confines of prompt engineering and fixed developer instructions, leading to what researchers describe as a “mechanical execution of tasks.”
- Lack of Personal Perspective: Human researchers analyze responses through the lens of their own experiences, gender, race, and beliefs, which helps contextualize issues such as financial difficulties or social challenges. The AI, devoid of identity or biography, produces a homogenized viewpoint, lacking neutrality as it replicates stereotypes from its training data.
- Inability for Self-Reflection: During interviews, researchers continually monitor their biases and assess how their status or profession may influence respondent openness. The AI model cannot evaluate its impact on individuals, lacks empathy, and is unable to establish a trusting emotional connection.
Experts conclude that while the AI interviewer from Anthropic serves as an efficient tool for rapid opinion collection, it views research merely as a means of “data acquisition.” In contrast, qualitative engagement with individuals necessitates mutual understanding and a shared human experience.
Anthropic's AI-driven survey, which gathered insights from over 81,000 participants, has sparked criticism from researchers who argue that AI lacks the emotional depth and contextual understanding necessary for genuine qualitative research. Experts emphasize the importance of human interaction in understanding complex social issues.