January 9, 2019 • The Recorded Future Team
Editor’s Note: The following blog post is a summary of a presentation from RFUN 2018 featuring OSINT subject matter expert Chief Warrant Officer 3 Nathan McKeldin of the U.S. Army.
The term “artificial intelligence” (AI) has become ubiquitous in the security world. If you believe the sales brochures, practically every security product on the market has some level of AI functionality.
Many types of security solutions tout the benefits of AI, including threat intelligence. But what benefits does AI really bring to the intelligence gathering process?
Late last year, we held our seventh annual Recorded Future User Network (RFUN) conference in Washington, D.C. During the conference, attendees were treated to a presentation on AI applications for open source intelligence (OSINT) by subject matter expert CW3 Nathan McKeldin of the U.S. Army.
Over the past five years, Nathan has used OSINT extensively to inform operational planning, fuel counterintelligence investigations, and support tactical military units with insights, warnings, and targeting information. In his presentation, Nathan explained how AI fits into the intelligence gathering process and how we can expect it to evolve over the next two decades.
To get things started, Nathan took the audience through four key definitions.
Since OSINT is Nathan’s primary focus, that’s where things began. According to Public Law 109-163, Sec. 931, OSINT is:
The key phrase here is “publicly available.” OSINT is founded on information that’s intended for general public consumption, which means no covert techniques or forced entry is required to collect it.
The Department of Defense Manual 5240.01 (August 2016) defines publicly available information as information that is:
Simply put, if a regular person can access a piece of information without doing anything illegal, you can reasonably classify it as “publicly available.”
Nathan defines AI as “cognitive reasoning determined through data-driven analysis and algorithmic functions.”
Put another way, an information system can be described as intelligent if it makes an informed, non-random decision algorithmically.
Common subcategories of AI include:
However, AI is not the same thing as automation, which is where a system automatically responds to an expected input (or set of inputs) by producing a desired output. The automated doors at your local Walmart may be helpful, but they aren’t intelligent.
One of the simplest applications of AI in intelligence gathering is to increase the efficiency of a human analyst. Machine-aided analysis describes the application of certain tenets of AI to a data set to execute tasks a human is capable of, but at a much greater volume and velocity.
Put another way, machine-aided analysis could be described as an “easy button” that takes away the heavy lifting from time-consuming, analytical tasks. For example, it could be used to digest a large volume of documents and automatically produce an output of people, places, and things.
Once he’d covered the basic terminology of AI and OSINT, Nathan gave the audience an overview of how AI is being used to improve military operations.
As he explained, from a military perspective, there are five domains:
Naturally, while each of these domains is important in its own right, all five domains are heavily interconnected. Airplanes fly, but they have to land on an airstrip or aircraft carrier. Troops are regularly moved around by air and sea. And, for obvious reasons, the cyber domain has become heavily intertwined with each of the other domains over the past few decades.
So where does AI fit into war fighting? Right at the center.
As AI has evolved, dozens of applications have been identified across each of the primary domains. Cyber is the most obvious candidate, since AI comes primarily from machines, networks, and computers connected to the internet, but there are also plenty of applications for AI in traditional military domains.
Over the next 10 to 20 years, we’ll see a huge increase in the use of AI technologies to improve army operations. Some of the most valuable advances will likely include:
In addition to the use cases described above, AI will be used to control drones for a variety of military, foreign, and domestic purposes.
AI will also be used to control so-called “drone swarms” to overwhelm opposing forces. The AI involved would be similar to that used to control drones at the South Korean Winter Olympic opening ceremony.
There are plenty of applications for AI in space, but perhaps the most obvious will be systems designed to help space-based assets avoid asteroids, and the development of kinetic strike vehicles.
Once he’d covered the frontline military applications of AI, Nathan took a step back to focus on how AI can be used to enhance intelligence operations.
Consider the intelligence cycle. In simple terms, it’s a feedback loop which starts with a set of requirements, and ends when those requirements are met with an actionable intelligence product.
This cycle has been going on, in some capacity, for as long as wars have been going on, and up until recently, it was a heavily manual process. However, over the last 50 years, a number of programs and capacities have been developed to automate portions of the process.
For example, here are some of the ways that AI can fit into the intelligence cycle:
Note that in the first phase of the cycle, Nathan has highlighted machine-aided analysis as the tool set of choice. This is because it’s important to retain human involvement in the loop, particularly when the outputs of the process will be used to inform real-world operations. We’ll look at how machine-aided analysis is enhancing OSINT practices shortly.
Beyond this, however, there are a huge number of applications for intelligent technologies throughout the intelligence cycle. From automated data collection via AI-powered drones and sensors (in the real world) and web crawlers and spiders (in the cyber domain), right through to automated and semi-intelligent dissemination mechanisms to the intended recipient, AI will increasingly be used to enhance and accelerate the intelligence cycle, particularly in a military context.
In fact, AI even has a role to play in improving the intelligence cycle itself.
In the diagram above, note the addition of machine learning to the center of the cycle. As Nathan explained, in the traditional intelligence cycle, humans provide the feedback loop to ensure the process is refined over time. In the future, machine learning will perform this function automatically and iteratively train collection and analysis algorithms by figuring out what’s working and what isn’t based on AI-fueled analysis of massive data sets.
That brings us to OSINT, a clear candidate for AI enhancement. The single greatest problem faced by OSINT collectors is the sheer volume of data available.
To give the audience an idea of scale, Nathan went over some stats on current global internet usage pulled from Internet Live Stats. In 2018 alone, internet users around the world have:
As Nathan put it: “Because of the massive amount of data available, working with OSINT can be like trying to find a needle in a stack of needles. It can be very hard to get down to that one particular needle you need to find.”
For this reason, applying AI and machine-aided analysis to OSINT has a whole host of benefits. For example:
Although effusive about the power and value of intelligent technologies to enhance OSINT processes, Nathan was quick to point out that it will never be able to replace an analyst’s ability to determine the “so what?” of information — nor should it.
To help put everything into context, Nathan covered a hypothetical scenario of a military analyst using AI-powered OSINT to solve a real-world problem: tracking the activity of extremist groups.
The process could run as follows:
In this example, the entire process, from collection to dissemination, is facilitated by AI. Threat intelligence solutions like the Recorded FutureⓇ Platform are already being used to fulfill intelligence processes very much like the one Nathan described, both for military and cybersecurity purposes.
To round things off, Nathan covered some of the ways AI could impact OSINT in the coming years.
From a positive perspective, AI could power personal assistant technologies with access to open source or classified databases — essentially like Siri with a security clearance. Imagine an in-field operator asking, “Hey Siri, when was the last time [extremist organization] was active in this area?” and receiving an immediate, accurate response.
Similarly, AI could power wearable technologies such as a smart contact lens that tracks eye motion, reads documents, discovers relationships, makes recommendations, provides analyses, and pushes everything to a HUD — think next-generation Google Glass built into a tactical visor.
Of course, not all applications of AI will be forces for good.
From an adversarial perspective, fake news and “deep fakes” — the use of AI to quickly create relatively convincing fake audio or video, like a clip of the president giving a speech he never actually gave — will continue to be a huge concern. We’ve already seen this to some extent with Russian interference and propaganda during the 2016 U.S. presidential campaign, and things are only going to become murkier. In particular, fake videos are getting better all the time, and are already becoming very difficult to identify.
Fortunately, as Nathan pointed out during the presentation, there will also be applications for AI to distinguish between genuine and fake media by analyzing signatures and tracking their propagation online.
Ultimately, Nathan’s sentiment was clear. AI has a huge role to play in the future of intelligence gathering, both for military and domestic organizations, and it will become increasingly mainstream as time moves on.