Signals and Anomalies Abundant in News Data

Posted: 25th April 2011
Signals and Anomalies Abundant in News Data

When you have a data set as large and diverse as we’ve built and continue to collect here at Recorded Future, the subject matter for analysis is pretty close to limitless. There really are telltale signals and anomalies to be found everywhere.

I’m going to use a geopolitical issue, but the potential to identify actionable moments related to product sentiment, sales figures, political campaigns, medical intelligence or financial trends all seem plausible with this same recipe: large cuts of data from online media, issues involving rival or conflicting interests, and a look at how those two (or more) interests line up over time.

As an exercise, I took aggregate data (count, momentum and sentiment) spanning the beginning of the year through early April from the Recorded Future news analytics API on two distinct sets of entities. First, countries in the Middle East and North Africa experiencing civil unrest, and second, ministers of foreign affairs from a variety of European countries.

I began with a relatively simple goal: partner up momentum data for the above mentioned countries with that of European foreign ministers to see if any surprising signals appeared, potentially giving away a country’s political interests.

Aside from watching the momentum rise and fall for Tunisia, then Egypt, then Libya over the first three months of the year, I noticed that online momentum for France’s Minister of Foreign Affairs Michele Alliot-Marie, who was pretty modest in terms of media volume compared to the countries in turmoil, twice exceeded the momentum of any of those in view countries.

The first momentum spike occurred in late January, just after Ben Ali fled Tunisia, and was largely driven by Alliot-Marie’s tour of the Middle East including meetings in Israel, Palestine, Jordan and Saudi Arabia.

French Foreign Minister and Middle EastFrench Foreign Minister with Highest Online Momentum

By the end of February, which was the second time Alliot-Marie’s online momentum was extremely high, she resigned over ties to the since departed Ben Ali regime in Tunisia. Inordinate momentum for a typically quiet individual is usually a lead worth following, and perhaps the rise in coverage during January indicates there was a concerted effort, using Alliot-Marie’s trip to the Middle East, to mask her questionable ties to Tunisia.

Moving in a different direction, while I was looking at this data several weeks ago, Libya was really hot after NATO voted to impose a no-fly zone over the country. Using a motion chart for analysis, one official’s name, Turkish Minister of Foreign Affairs Ahmet Davutoglu, seemed to move quite a bit whenever attention on Libya shifted.

So, following the trail of a seemingly active player, I looked at Davutoglu in comparison to the other ministers during February. When I set up sentiment on separate axes, here was a moment that stood out.

French Foreign Minister and Middle East Media Sentiment on Dimitris Droutsas and Ahmet Davutoglu Davutoglu and Droutsas Network from February Davutoglu vs Droutsas Network from Mid-February

Was it a well timed move by Turkish officials to be consumed by national interests in getting citizens out of Libya? Perhaps. The country was quickly destabilizing.

However, one can imagine a savvy political play with the country’s foreign minister traveling to New Dehli while Greek minister Droutsas met with UN Secretary-General Ban Ki-Moon over the long contentious issue of Cyprus.

In the end, this is an example that I walked into while looking at a hot topic of the day. Turn this around though, and think, what are the competing issues or entities that are critical to understand in order for my campaign (or business or product or trading strategy) to succeed? The web probably reveals more on these issues than you ever expect!