Deepfakes threaten every aspect of society. Our ability to identify fake content is crucial for nullifying disinformation, but as AI technology improves, who can we trust to detect deepfakes: man or machine?
The Dangers of Deepfakes
As AI technology advances,the dangers of deepfakespose an increasing threat to all of us. Here’s a quick summary of some of the most pressing issues deepfakes pose:
Deepfakes will only become more convincing, so we need robust tools and processes for detecting them. AI is providing one such tool in the form of deepfake detection models. However, likealgorithms designed to identify AI-generated writing, deepfake detection tools aren’t perfect.
At this time, human discretion is the only other tool we can rely on. So, are we any better than algorithms at identifying deepfakes?
Can Algorithms Detect Deepfakes Better Than Humans?
Deepfakes are a serious enough threat that technology giants and research groups are dedicating vast resources to research and development. In 2019, the likes of Meta, Microsoft, and Amazon offered $1,000,000 in prizes during aDeepfake Detection Challengefor the most accurate detection model.
The top-performing model was 82.56% accurate against a dataset of publicly available videos. However, when the same models were tested against a “black box dataset” of 10,000 unseen videos, the top-performing model was only 65.18% accurate.
We also have plenty of studies analyzing the performance of AI deepfake detection tools against human beings. Of course, the results vary from one study to the next, but generally, humans either equal or outperform the success rate of deepfake detection tools.
One 2021 study published onPNASfound that “ordinary human observers” achieved a slightly higher accuracy rate than the leading deepfake detection tools. However, the study also found that the human participants and AI models were susceptible to different types of mistakes.
Interestingly, research carried out byThe University of Sydneyhas found the human brain is, unconsciously, more effective at spotting deepfakes than our conscious efforts.
Detecting Visual Clues in Deepfakes
The science of deepfake detection is complex, and the required analysis varies, depending on the nature of the footage. For example, the infamous deepfake video of North Korean leader Kim Jong-un from 2020 is basically a talking head video. In this case, the most effective deepfake detection method might be analyzing visemes (mouth movements) and phonemes (phonetic sounds) for inconsistencies.
Human experts, casual viewers, and algorithms can all perform this kind of analysis, even if the results vary. TheMITdefines eight questions to helpidentify deepfake videos:
The latest AI deepfake detection tools can analyze the same factors, again, with varying degrees of success. Data scientists are constantly developing new methods, too, such as detecting natural blood flow in the faces of on-screen speakers. New approaches and improvements to existing ones could result in AI deepfake detection tools consistently outperforming humans in the future.
Detecting Audio Clues in Deepfakes
Detecting deepfake audio is a different challenge entirely. Without the visual cues of video and the opportunity to identify audiovisual inconsistencies, deepfake detection relies heavily on audio analysis (other methods like metadata verification can also help, in some cases).
A study published byUniversity College Londonin 2023 found humans can detect deepfake speech 73% of the time (English and Mandarin). As with deepfake videos, human listeners often intuitively detect unnatural speech patterns in AI-generated speech, even if they can’t specify what seems off.
Common signs include:
Once again, algorithms can also analyze speech for the same deepfake signals, but new methods are making tools more effective. Research byUSENIXidentified patterns in AI vocal tract reconstruction that fail to emulate natural speech. It summarizes that AI voice generators produce audio matching narrow vocal tracts (roughly the size of a drinking straw) without the natural movements of human speech.
Earlier research from theHorst Görtz Instituteanalyzed genuine and deepfake audio in English and Japanese, revealing subtle differences in the higher frequencies of genuine speech and deepfakes.
Both the vocal tract and high-frequency inconsistencies are perceptible to human listeners and AI detection models. In the case of high-frequency differences, AI models could, theoretically, become increasingly accurate—although the same could also be said for AI deepfakes.
Humans and Algorithms Are Both Fooled by Deepfakes, but in Different Ways
Studies suggest humans and the latest AI detection tools are similarly capable of identifying deepfakes. Success rates can vary between 50% and 90+%, depending on the test parameters.
By extension, humans and machines are also fooled by deepfakes to similar extents. Crucially, though, we’re susceptible in different ways, and this could be our biggest asset in tackling the dangers of deepfake technology. Combining the strengths of humans and deepfake detection tools will mitigate the weaknesses of each and improve success rates.
For example,MITresearch found humans were better at identifying deepfakes of world leaders and famous people than AI models. It also revealed the AI models struggled with footage with multiple people, although it suggested this could result from algorithms being trained on footage featuring single speakers.
Conversely, the same study found AI models outperformed humans with low-quality footage (blurry, grainy, dark, etc.) that could be intentionally used to deceive human viewers. Likewise, recent AI detection methods like monitoring blood flow in particular facial regions incorporate analysis humans aren’t capable of.
As more methods are developed, AI’s ability to detect signs we can’t will only improve, but so will its ability to deceive. The big question is whether deepfake detection technology will continue to outpace deepfakes themselves.
Seeing Things Differently in the Age of Deepfakes
AI deepfake detection tools will continue to improve, as will the quality of deepfake content itself. If AI’s ability to deceive outpaces its ability to detect (like it has with AI-generated text), human discretion may be the only tool we have left to battle deepfakes.
Everybody has a responsibility to learn the signs of deepfakes and how to spot them. Aside from protecting ourselves from scams and security threats, everything we discuss and share online is vulnerable to disinformation if we lose grasp of reality.
