Understanding deepfakes - Creation and implications

Growing concerns surround the advancing technology of deepfakes, enabling the creation of fabricated scenes. Celebrities have unwittingly become subjects of fake pornography, while politicians have been depicted in videos appearing to say words they never uttered.

These worries have spurred the development of countermeasures. New legislation aims to prevent the creation and dissemination of deepfakes. Earlier this year, major social media platforms such as Facebook and Twitter prohibited deepfakes on their networks. Conferences focused on computer vision and graphics are filled with presentations detailing strategies to combat them.

So, what exactly are deepfakes, and why do they provoke such concern?

Definition of a deepfake:

Deepfake technology allows for the seamless integration of anyone into a video or photo in which they did not actually participate. Such capabilities have existed for decades; for instance, the late actor Paul Walker was digitally resurrected for "Fast & Furious 7." However, previously, creating these effects required entire studios of experts and a year's worth of work. Now, new automatic computer graphics or machine learning systems can generate images and videos much more quickly.

The term "deepfake" is often misunderstood and is widely disliked by experts in computer vision and graphics. It has become a catch-all term encompassing everything from state-of-the-art videos generated by AI to any image that appears potentially fraudulent.

Much of what is labeled as a deepfake is not accurate. For example, a controversial "crickets" video from the U.S. Democratic primary debate, released by former presidential candidate Michael Bloomberg's campaign, was created using standard video editing techniques. Deepfake technology was not involved.

How deepfakes are created

Machine learning is the key component in creating deepfakes, enabling their production faster and more affordably. To create a deepfake video of someone, a creator typically trains a neural network on extensive real video footage of the person, providing a realistic "understanding" of their appearance from various angles and lighting conditions. This trained network is then combined with computer graphics techniques to overlay the person onto a different actor.

While AI accelerates the process significantly, creating a believable composite still requires time and manual adjustment of the program's parameters to avoid imperfections. Many assume that generative adversarial networks (GANs), a class of deep-learning algorithms, will drive future deepfake development. GANs can generate faces that are nearly indistinguishable from real ones. However, GANs are challenging to work with, requiring extensive training data and struggling with temporal consistency in videos.

Despite the focus on GANs, most deepfake videos today do not heavily rely on them. According to Siwei Lyu of SUNY Buffalo, GANs are not the main technique used for creating deepfakes. Instead, a combination of AI and non-AI algorithms is commonly used. For example, when Canadian AI company Dessa (now owned by Square) created audio "deepfakes" of talk show host Joe Rogan, GANs were not involved.

What are deepfakes used for?

The primary threat posed by deepfakes at present is to women, with nonconsensual pornography making up 96 percent of the current deepfakes circulating on the Internet. While most target celebrities, there is a growing number of cases involving deepfakes used to create fake revenge porn, according to Henry Ajder, head of research at the detection firm Deeptrace in Amsterdam.However, women are not the only potential targets of harassment. Deepfakes could facilitate bullying more broadly, whether in educational institutions or workplaces, as individuals can be placed in absurd, risky, or compromising situations.Corporations are concerned about the potential for deepfakes to amplify scams. There have been unverified reports of deepfake audio being utilized in CEO scams to deceive employees into sending money to scammers. Extortion could become a significant application. Identity fraud was the primary concern regarding deepfakes for over three-quarters of respondents in a cybersecurity industry poll conducted by the biometric firm iProov. Respondents were chiefly worried that deepfakes would be used for fraudulent online transactions and to hack personal banking services.

For governments, the major fear is that deepfakes pose a threat to democracy. If a female celebrity can be depicted in a pornographic video, the same can be done to a politician seeking reelection. In 2018, a video emerged of João Doria, the governor of São Paulo, Brazil, who is married, allegedly participating in an orgy. He claimed it was a deepfake. Other instances have occurred, such as in 2018 when the president of Gabon, Ali Bongo, who was long thought to be ill, appeared in a suspicious video to reassure the population, triggering an attempted coup.

The uncertainty surrounding these unverified cases underscores the most significant danger of deepfakes, regardless of their current capabilities: the "liar's dividend," which means that the mere existence of deepfakes allows individuals to dismiss any evidence of wrongdoing as a deepfake, providing a blanket excuse for their actions. "That is something you are absolutely starting to see: that liar's dividend being used as a way to get out of trouble," says Farid.

Who created deepfakes?

Some of the most notable deepfake examples often originate from university laboratories and the startups they spawn. For instance, a highly publicized video featuring soccer star David Beckham fluently speaking in nine languages, despite only speaking one in reality, is based on technology developed at the Technical University of Munich in Germany.

However, these are not the types of deepfakes that concern governments and academics. Deepfakes do not need to be sophisticated or high-tech to have a damaging impact on society, as demonstrated by nonconsensual pornographic deepfakes and other problematic variants.The term "deepfake" originates from a seminal example in 2017 by a Reddit user known as r/deepfakes, who used Google's open-source deep-learning library to swap the faces of porn performers with those of actresses. The codes used in do-it-yourself (DIY) deepfakes commonly seen today are largely derived from this original code. While some may be viewed as entertaining experiments, none can be considered truly convincing.So, why the widespread concern? According to Hany Farid, a digital forensics expert at the University of California, Berkeley, technological advancements continually improve. There is no consensus in the research community regarding when DIY techniques will become sophisticated enough to pose a genuine threat, with predictions ranging from 2 to 10 years. However, experts agree that eventually, anyone will be able to use a smartphone app to create realistic deepfakes of others.

How do we stop malicious deepfakes?

Over the past year, several U.S. laws regarding deepfakes have come into effect. States are introducing bills to criminalize deepfake pornography and prohibit the use of deepfakes in elections. Texas, Virginia, and California have criminalized deepfake porn, and in December, the president signed the first federal law as part of the National Defense Authorization Act. However, these laws are only effective when the perpetrator resides in one of those jurisdictions.

Outside the U.S., China and South Korea are the only countries taking specific actions to prohibit deepfake deception. In the United Kingdom, the law commission is reviewing existing laws for revenge porn to address various ways of creating deepfakes. However, the European Union does not seem to view this as an immediate issue compared to other forms of online misinformation.

While the U.S. is at the forefront, there is little evidence that the proposed laws are enforceable or prioritize the right areas.Many research labs have developed innovative ways to identify and detect manipulated videos, such as incorporating watermarks or blockchain technology. However, creating deepfake detectors that cannot be easily manipulated to create more convincing deepfakes remains challenging.Tech companies are making efforts to combat deepfakes. Facebook has recruited researchers from Berkeley, Oxford, and other institutions to build a deepfake detector and enforce its new ban. Twitter has also made significant policy changes and is reportedly planning to tag any deepfakes that are not removed outright. YouTube reiterated in February that it will not allow deepfake videos related to the U.S. election, voting procedures, or the 2020 U.S. census.Outside these platforms, two programs, Reality Defender and Deeptrace, aim to prevent deepfakes from causing harm. Deeptrace is working on an API that acts as a hybrid antivirus/spam filter, screening incoming media and diverting obvious manipulations to a quarantine zone, similar to how Gmail filters out spam. Reality Defender, a platform being developed by the company AI Foundation, also aims to tag and identify manipulated images and videos before they can cause harm. "We think it's really unfair to put the responsibility of authenticating media on the individual," says Ajder.