Are you talking to a real person? How do you know?
Jason Davis, research professor and co-director of the Newhouse Synthetic Media Lab at Syracuse University, examines these questions.
Faculty Bio:
Dr. Jason Davis is a research professor in the office of Research and Creative Activity at the S.I. Newhouse School of Public Communications and co-director of both the Newhouse Synthetic Media Lab and the Emerging Insight Lab. At Syracuse University, Dr. Davis research focuses on the development of AI tools for the detection of synthetic media and disinformation well as STEM focused communication and professional development.
Prior to joining Syracuse University, Dr. Davis was director of Business Development and Operations at the Center for Biotechnology and Interdisciplinary Studies at Rensselaer Polytechnic Institute where he developed research partnerships in the area of upstream and downstream biologics manufacturing and served as Academic co-chair for the National Institute for Innovation in Manufacturing Biopharmaceuticals (NIIMBL) workforce development committee. Prior to RPI, he served as global training and development leader at General Electric’s Global Research Center, and as a principal scientist leading research efforts in water purification technologies, CO2 capture, nano-particle drug delivery and DNA and RNA stabilization technology. Prior to his role at GE, Davis was a Senior Research Scientist at Curia Global where his research focused on drug discovery in the areas of oncology, Alzheimer’s disease, and diabetes.
Davis’s current research focuses on the detection of misinformation and disinformation using AI/ML tools and the development of detection systems for synthetic media including text, images, audio and video as well as explainability frameworks for AI and Human-in-the-Loop decision making.
Dr. Davis holds more than 10 patents in the areas of diabetes, water purification technology, anti-fouling coatings, and RNA sample stabilization technology.
Davis earned a Ph.D. in Chemistry from McGill University, and a B.Sc. in Biochemistry (with honors) from Laurentian University.
Transcript:
Over the past year my collaborator Dr. Gina Luttrell and I have been asking a deceptively simple question: Can humans still tell if a person even exists?
To answer this question, we first wanted to find out if people could tell when a short profile was written by a machine rather than a human. To find out, we assembled one hundred profiles, half authored by real journalism students and subject matter experts, and half generated by four different large language models. We asked people to read the profiles and label them as “human” or “synthetic.” The result was striking. Participants were correct only 54% of the time, just a shade above the 50% level that equates to random guessing.
At the same time, we built an AI detector trained on a separate set of 3,000 profiles. When we ran that detector over the same hundred profiles, it correctly identified 46 of the 50 synthetic profiles and 42 of the 50 human profiles, for an overall accuracy of 88%.
What do these numbers tell us? First, as large language models become more fluent, human heuristics, or the gut reactions we take for granted, for spotting deception erode quickly. Second, with the arrival of generative AI, the need to determine if something is real or synthetic represents a new layer of complexity for humans to navigate. Our research suggests breaking this task into three distinct components can help:
The first is Detection: Is the content human-generated or synthetic?
Second is Attribution: Is it coming from who it says it’s coming from?
Finally, Characterization: What is the intent? Is it malicious deception, genuine content, or something in between?
By developing robust, trained AI analytics with clear, human readable evidence, we can assist humans with the first two tasks and help preserve digital trust and transparency. This lets us return to the deeply human task of deciding if the content is good or bad and what to do about it.
As AI creates new challenges in digital trust and transparency, it also illuminates a broader reality: that perhaps the safest path forward is collaboration, not competition, between humans and machines.
