Making Threat Intelligence Less Like Manual Labor

February 13, 2018 • The Recorded Future Team

Key Takeaways

  • Many organizations continue to rely on manual processes to collect and make sense of threat intehttps://www.recordedfuture.com/threat-intelligence/lligence, but doing so is exhausting and repetitive. Security analysts are no match for the scale of data they are confronted with today.
  • Manually sifting through data has left many security operations centers feeling overwhelmed. According to a recent survey conducted by ESG, 51 percent of organizations believe they have a shortage of skilled staff, and less than half actually rely on threat intelligence to make decisions when responding to threats.
  • To respond to some of the most pressing needs identified by organizations today, threat intelligence solutions need advanced analytics to automate the most repetitive tasks, namely, data collection and processing. This automation should result in fewer, more intelligent alerts, making the jobs of analysts easier — not simply broader and more frequent alerts that add to the analysts’ burdens.

The enemy of critical thinking is dull repetition. Do the same thing over and over again, and your brain begins to take shortcuts, automating the process to save energy and work more efficiently. For security analysts who have to manually sift through endless streams of data to identify and prioritize threats, the dangers of “falling asleep at the wheel,” or simply getting exhausted by the effort it takes to approach each new piece of information with a critical eye, are some of the biggest factors explaining why nearly three quarters of cybersecurity professionals say that security analytics and operations is harder today than it was two years ago.

To work more effectively, security operations centers must incorporate threat intelligence solutions that include advanced analytics to automate the “manual labor” parts of the job, including collecting and processing data, so that analysts can focus on doing the actual analysis instead of getting burnt out.

Overwhelmed by Data

One of the most emphasized features of an effective cybersecurity program by experts in the cyber threat intelligence community is human-machine teaming. Often, this advice is given as a word of caution toward those who might put their trust in threat intelligence solutions that rely solely on automation. And it’s true that the best algorithms in the world are far, far away from competing with an expert’s ability to identify patterns from incomplete data sets and get into the mind of a threat actor. But neither can any one human or team of humans keep up with the massive — and growing — amount of data that every organization is now confronted with every day.

A recent survey conducted by ESG found that 72 percent of cybersecurity professionals believe their work has become more difficult today than it was two years ago. One of the biggest reasons they feel this way is because of a shortage of expertise — the security operations centers of more than half of all organizations surveyed feel short-staffed, leaving them overburdened with tasks and overwhelmed with a plethora of alerts and notifications.

It may feel like manual processes are an inescapable part of the generation of threat intelligence — in fact, in the traditional threat intelligence lifecycle on which the modern cyber threat intelligence cycle is based, there was no better way to gather intelligence than by sending personnel out into the field to make observations firsthand. Getting information used to mean sending people to take pictures, listening in on conversations, meeting contacts, hearing reports, and manually breaking codes and decrypting communications. All of these methods took years of experience to perform effectively, despite often being tedious and time-consuming tasks.

Many of these information-gathering tasks have analogues in the world of cyber threat intelligence, and they can be equally exhausting and time consuming, if not more so. Before threat intelligence can be benefited from, it must be collected, correlated, contextualized, and enriched — and according to ESG’s research, some 39 percent of organizations still rely on manual processes to perform these labors. This means some analysts are still expected to look at streams of data coming from various sources and showing up in different formats, monitor social media and forums to keep an eye out for new threats, know when to listen to and when to ignore alerts and identify false positives, and more.

It’s not enough to say that this takes experience to do effectively — it simply doesn’t scale. According to one report released by Ponemon Research, only 46 percent of organizations said their threat incident responders actually relied on threat data when responding to threats, and an even smaller 27 percent felt they used that data effectively when they did use it.

At the same time, when asked about the top objectives that they had for their threat intelligence programs, 33 percent of organizations surveyed stated that they wished to improve their risk management efficiency and effectiveness, and 31 percent stated that they wanted to use threat intelligence to automate remediation — that is, use threat intelligence solutions to identify indicators of compromise and then trigger automated remediation tasks.

Both of these objectives, which were the top two listed by the organizations surveyed, are substantially easier to achieve using threat intelligence solutions with advanced analytics that can automate the most repetitive labors.

Let Advanced Analytics Do the Grunt Work

The answer to the problem of manual labor is a threat intelligence solution that includes advanced analytics. The ultimate goal here is to have a tool that results in far fewer alerts — but of much higher quality — appearing on the user end, eliminating the need for your security analysts to sort out the genuine threats from the false alarms by themselves. Threat intelligence tools with machine-learning algorithms can help contextualize the flow of data, and therefore narrow it down. For example, a threat intelligence tool might use advanced analytics to determine which types of threats most commonly target certain industries. A wide-scale attack that targets the credit card information stored by many businesses may have little relevance to the security operations center of an organization that doesn’t store the credit card information of their clients, for example, but if they are using a threat intelligence solution that does not have a central management and analysis portal or relies on sufficiently intelligent algorithms to tell apart a critical alert from an irrelevant one, they may waste precious time and mental stamina evaluating that alert.

Many leading threat intelligence platforms will also use artificial intelligence to analyze the techniques, tactics, and procedures used by threat actors and identify the most common indicators of compromise associated with certain types of attacks, or most commonly leveraged against particular industries. With that information, it becomes much easier for security analysts to plan for future attacks and take the necessary precautions. A threat intelligence tool with advanced analytics should act as a “helper app,” says ESG, guiding organizations toward the right priorities and the fastest responses. It’s far easier to point a machine to the types of intelligence of most value to your business and let machines automatically determine what’s relevant. This is the difference between guessing web URLs and using Google to at least point you in the right direction.

What’s needed most of all are solutions that play on the strengths of both machines and humans. Automated processes are not meant to add to the volume of work that security analysts must perform, and analysts cannot reasonably be expected to keep up with the work that machines can now do. Their roles must be complementary, not overlapping.

To learn more about finding a threat intelligence solution that can help you move away from manual processes, download your copy of ESG’s “Operationalizing Threat Intelligence With a Complete Solution.”