Séminaire ayant lieu dans le cadre du Doctorat en informatique cognitive, en collaboration avec le centre de recherche CRIA        

TITRE : Do LLMs pass the Turing test? And what does it mean if they do?

Cameron JONES

Jeudi le 6 novembre 2025 à 10h30

Local PK-5115 (Il est possible d'y assister en virtuel en vous inscrivant ici)        

RÉSUMÉ

Large Language Models (LLMs) seem well designed for the Turing test in that they can produce fluid naturalistic text. Many have suggested that they would pass the test or implicitly already have. We addressed this question empirically by evaluating several LLMs in a standard three-party 5 minute Turing test. Two models, when prompted to adopt a humanlike persona, achieved a pass rate of 50%: suggesting that interrogators were no better than chance at distinguishing between humans and LLMs. One of these models (GPT-4.5) was judged to be human 73% of the time, significantly more often than the real humans it was being compared to. These results suggest that LLMs pass the Turing test, but what does that mean? I will discuss potential interpretations of these results, including whether they suggest that LLMs are intelligent, produce humanlike behaviour, or are merely exploiting superficial cues.

BIOGRAPHIE

Cameron JONES is an Assistant Professor in the Psychology Department at Stony Brook University. His research focuses on the intersection between psychology and AI: using paradigms from psychology to compare human and AI behaviour, using AI to understand how people interact with each other, and investigating the impact that AI might have on our psychology longer term. His recent work has focussed on evaluating social intelligence in LLMs (including theory of mind and more interactive social tasks), investigating the extent to which AI systems can manipulate and deceive people (as well as the role that trust and rapport play in those interactions), and evaluating LLMs in the Turing test.

RÉFÉRENCES

Jones, C. R., & Bergen, B. K. (2025). Large language models pass the turing test. arXiv preprint arXiv:2503.23674.

Jones, C. R., Rathi, I., Taylor, S., & Bergen, B. K. (2025). People cannot distinguish GPT-4 from a human in a Turing test. In Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency (pp. 1615-1639).