Recorded Future analyzed data from the Recorded Future® Platform, dark web, information security reporting, and other open-source intelligence (OSINT) sources to identify the use and prevalence of how threat actors are attempting to advertise, discuss, sell, and purchase deepfake-related services and products that facilitate fraudulent activities. In this report, we define deepfakes as synthetically generated visual and audio content that is being used offensively to target individuals, companies, and security systems. This report is part of our series on the business of fraud.

Executive Summary

Threat actors have begun to use dark web sources to offer customized services and tutorials that incorporate visual and audio deepfake technologies designed to bypass and defeat security measures. Furthermore, threat actors are using these sources, as well as many clearnet sources such as forums and messengers, to share tools, best practices, and advancements in deepfake techniques and technologies. As reported by Insikt Group’s Criminal and Underground Team throughout 2020, threat actors are developing customized deepfake products.

We believe they will continue to develop these products, as the demand is likely to increase due to corporations incorporating visual and audio recognition technologies into their security measures. Within the next few years, both criminal and nation-state threat actors involved in disinformation and influence operations will likely gravitate towards deepfakes, as online media consumption shifts more into “seeing is believing” and the bet that a proportion of the online community will continue to be susceptible to false or misleading information.

Key Judgments

• Deepfake technology used maliciously has migrated away from the creation of pornographic-related content to more sophisticated targeting that incorporates security bypassing and releasing misinformation and disinformation. Publicly available examples of criminals successfully using visual and audio deepfakes highlights the potential for all types of fraud or crime, including blackmail, identity theft, and social engineering.
• English- and Russian-language dark web forums were identified as the main sources for users to advertise, discuss, share, and purchase deepfake-related products, services, and topics. The most widely used forums were found to be low- to mid-tier forums that have lower barriers to entry, but activities were also found on high-tier forums. Deepfake topics were also identified on Turkish-, Spanish-, and Chinese-language forums.
• The most common deepfake-related topics on dark web forums included services (editing videos and pictures), how-to methods and lessons, requests for best practices, sharing free software downloads and photo generators, general interests in deepfakes, and announcements on advancements in deepfake technologies.
• There is a strong clearnet presence and interest in deepfake technology, consisting of open-source deepfake tools, dedicated forums, and discussions on popular messenger applications such as Telegram and Discord.
• Discussion on most publicly available forums and messengers relating to deepfakes surrounds the education and genuine interest in deepfake technology, in addition to users sharing content and refining their craft, in line with discussions identified on closed dark web sources. In the future, we believe that this otherwise relatively benign community can serve as a basis for individuals to venture into illicit criminal activity using learned deepfake skills.

Editor’s Note: This post was an excerpt of a full report. To read the entire analysis, click here to download the report as a PDF.