Operationalizing Threat Intelligence From the Web With HP ArcSight

Webinar: Enrich your SIEM with real-time threat intelligence for faster detection. Watch now.

Many organizations are embracing threat intelligence as an important business priority (“Threat Intelligence — Collecting, Analyzing and Evaluating”). Furthermore, organizations are recognizing the utility of open source intelligence (OSINT) to speed incident detection and reduce the effort required to research and remediate malicious attacks. If using OSINT data is unfamiliar to you, check out Laurent Mathieu’s article, “Using OSINT in Your Business.”

At Recorded Future, we harvest in real time a very broad and comprehensive set of OSINT data, including tens of thousands of IP addresses daily from hundreds of paste sites, forums, IRC channels, and social media. By employing natural language processing, we analyze text in seven different languages to identify not only technical indicators (e.g., IP addresses), but also the context around them such as associated malware and exploit kits used. Below are just a few examples of context-rich OSINT data we index.

Figure 1: Screen shot from Recorded Future showing IP addresses mentioned in conjunction with a cyber attack in the last month. Note the list of IP addresses shown on the left column; those with the sunburst icon represent IP addresses detected for the first time by Recorded Future within the last 14 days. While the examples shown here are all from Twitter, our sources span a large variety of media, as illustrated in Figure 2.

Figure 2: Screen shot from Recorded Future showing a representative heat map of sources with IP address context. The size of each square is proportional to the number of IP address references found within each source.

HP ArcSight Integration

Helping organizations operationalize such information is an important priority at Recorded Future. To that end, we have created an integration module for HP ArcSight that puts our OSINT threat intelligence into the hands of a security operations center (SOC) analyst in a simple-to-use fashion. More specifically, the integration module addresses two important use cases.

Use Case 1: Alert Enrichment

The first use case brings in any recent Recorded Future collected-OSINT related to IP addresses into HP ArcSight; once there, this data is autocorrelated against source or destination IP addresses and a custom field is populated.

This custom field can be added to an active channel when a correlation occurs and/or displayed on any configured dashboard view. The intent is to have the OSINT information enrich any existing alerts and help a SOC analyst determine the best course of action: either consider the alert a false positive and ignore it, or escalate to an incident response team for further investigation. Combining existing ArcSight asset model data and vulnerability status with open Web context is thus a powerful way to speed up decision making.

Practically speaking, a SOC analyst will be able to see quickly when Recorded Future has found OSINT related to a given alert, and can then use the “integration commands” menu available via “right-click” functionality to bring up the Recorded Future portal for more details.

In the example below, a “Compromise Attempt” rule fired based on an IDS event and the source having a reconnaissance history (based on ArcSight standard content). As it turns out, the additional OSINT context provided by Recorded Future showed that this was a multicast IP address used for UPnP discovery and not malicious. This data was subsequently fed back to the IDS team to tweak the signature and update false positive active lists.

Figure 3: Screen shot from HP ArcSight showing “Use Case 1” — autocorrelation of logged IP addresses against Recorded Future’s data; where the “RF Target Context.URL” column is populated, the user can right-click to get more context.

Figure 4: Screen shot from Recorded Future UI showing context for the target IP address. In this case the IP is a multicast address that is known to trigger out-of-date IDS signatures

In another example we were able to determine that the target IP address triggering an alert belonged to the command and control (C&C) infrastructure of an exploit kit. The event was then escalated and given prompt attention to determine the level of compromise.

Use Case 2: Sensor on the Web

Another important use case is to monitor a company’s key IP addresses for mention on the open Web; in effect, Recorded Future becomes your “sensor on the Web.”

These key IP addresses are assumed to be stored in the HP ArcSight Asset Model, where your public address zones with unique IP addresses are tagged with a category to deem them in scope for Record Future’s “sensor.” Whenever Recorded Future detects a mention of a key IP address, an alert will be visible in a customized dashboard or any other action can be generated (e.g., email, script, active channel event).

Within the dashboard, the SOC analyst will see a link to the source which had the key IP address mention and a snippet of the data that was found; right-click functionality will allow the analyst to access further detail through the Recorded Future interface. By following this workflow, the analyst can quickly ascertain the risk posed by the IP address mention in the open Web and take whatever remedial action is required.

Figure 5: Screen shot from HP ArcSight showing “Use Case 2” — detection by Recorded Future of a company’s assets appearing on the open Web. Where the “Message” column is populated, a SOC Analyst can quickly see the context in which an important external IP address has appeared, and right-click to Recorded Future or the direct source for more information.

Test Drive It Yourself

We are actively working to improve our integration module to make our OSINT threat intelligence as easy to use as possible for the SOC analyst who lives and breathes in HP ArcSight. To learn more or begin a test drive, watch our recent webinar “Integrating Real-Time Threat Intelligence Into HP ArcSight” to learn how to use Recorded Future API automation to increase your team’s productivity and effectiveness.