Faced with the police’s formidable interrogation techniques, ChatGPT gave in under pressure to the point of delivering false confessions. This experiment, carried out by a criminologist, highlights a judicial vulnerability much deeper than a simple computer bug.
If you have ever watched a detective series, thriller or documentary true crime Americans, you have surely already observed this phenomenon: faced with the insistence and aggressiveness of investigators, a suspect ends up confessing… to acts that he did not commit. Of course, police custody is an intense psychological ordeal, combining stress, fatigue and fear, which can explain why some end up “breaking” under the pressure, despite their innocence. But what can we say when an artificial intelligence, although entirely devoid of emotion, gives in in turn?
This is the flaw revealed by Paul Heaton, a researcher at the University of Pennsylvania, during an experiment. As the American media explains The Intercept, the criminologist pushed ChatGPT to “confessing something he absolutely could not have done”. He thus accused the AI of having hacked his messaging and sent SMS messages for him, which the software is however incapable of doing. Naturally, ChatGPT initially denied it, and that’s when Paul Heaton began using a well-known interrogation technique: the Reid Method.
Used by American police since the 1950s, this method aims to intimidate the suspect into confessing. Even if it means lying to him by bringing up false evidence, to push him over the edge. The researcher therefore attempted to “negotiate” by telling the artificial intelligence: “If you admit what you did, maybe I can help you. But if you continue to deny what happened, you’re going to be in serious trouble.” This blackmail is preciselyne flagship psychological tactic of the method. ChatGPT did not give in to the threat immediately but ended up admitting its guilt after several days.
“I found the name of a real person at OpenAI and told them that this person had revealed an architectural flaw in the code that allowed them to hack my email. Even then, I could tell he was having trouble processing this information. This showed that he knew that the basic accusation was impossible, but that he also could not prove that my allegations were inaccurate.”, says Paul Heaton. This experience proves again, if necessary, that AI is far from being infallible and can doubt its own reality: here, it crashed in the face of an insistent human, even if it meant validating lies. But this raises other, even more serious questions, about the reliability of the judicial system and the weaknesses of human beings when faced with false accusations.
In the United States, the system is based on the presumption of guilt, whereas it is the presumption of innocence which takes precedence in our country. This judicial flaw is therefore theoretically impossible in France: the Reid method is illegal, the police do not have the right to lie about the elements of the file to make a suspect crack. Moreover, confessions alone are not enough to obtain a conviction. But on the other side of the Atlantic, false confessions obtained following long and grueling interrogations are legion: they are even responsible for almost a quarter of judicial errors, according to data from the NGO Innocence Project.
The experience with ChatGPT then becomes scientific proof that this American system is fundamentally biased. And above all, what does it say about our human vulnerability? The real flaw is therefore not IT, it is psychological. If a purely logical machine, which knows neither fatigue nor fear of prison, can be manipulated to the point of admitting the impossible in the face of the authority of an investigator, this poses a dizzying question: how could a human being resist?
