The Nature of (Pre)News and Event Detection
By Chris on July 22, 2011
Being able to detect events early, or even before they occur, can carry substantial value, some times in the form of significant financial gain, other times in the ability to save lives, or even in providing the opportunity to change the nature of what is to happen.
As the world has moved into an always-on, real-time mode, traditional methods of “news”, i.e. the morning delivered printed newspaper or even the hourly newscast, have been overtaken in their ability to provide early indications of what is happening by information traveling rapidly through media such as Email, SMS, Twitter, Facebook updates, etc.
The increased speed of delivery and accessibility to news creates opportunities to better understand developing scenarios even as the growing volume of content creates challenges in sifting, filtering and identifying actionable information about the future.
For example, large scale search/data mining capabilities are allowing practically anyone to find minuscule mentions of subtle indications about what is to come and detect early signals of such events.
The world’s collective set of media – news, blogs, social media, government filings, etc. – is all available to anybody interested in detecting what is to make news or even better, what is about to happen.
Some examples of what we’re seeing now include:
- Detecting future corporate trouble in company conference calls months ahead of hitting the news (PDF).
- The first signals from the attack on the Osama Bin Ladin compound being transmitted and detected via Twitter.
- News of financial indicators such as Unemployment numbers being delivered literally in micro seconds to trading algorithms.
The race to being the first to deliver news, pre-news, or even better, future events is on!
In the figure below, we try organize the challenge into four areas:
- Complex and distributed signals indicating what’s to come – e.g. weeks to months in advance
- Rumors and buzz pre-events – e.g. days to hours to minutes in advance
- Detecting and capturing information content out of an event post it happening
- News recycling where information literally is regurgitated over and over for days to weeks
It goes without saying that the ability to capture value (be it economic, strategic, tactical) is directly proportional to how early one can detect and execute.
We will step through each of the areas mentioned above and discuss what value there is, and we will do so in reverse order:
4. The traditional approach to news consumption is to read what everybody else is reading, perhaps with the hope of reading it early. An event will potentially generate story after story over days and weeks. Ultimately, there’s not a lot of value in this related to capturing early signal. It might be educational and insightful, it might be enjoyable, but you will not find early signal.
3. Between the time period of an event – be it a bomb explosion or the release of a financial figure – and it actually being reported in an online story there may be a time period of 10-20 minutes these days. There are opportunities to detect this event, whether from a Taiwanese blogger seeing the semiconductor factory exploding or a sudden co-occurrence of tweets or inferred information in a collection of option trades, and being early here can certainly capture value. However, we can expect the delay from event to news story to keep shrinking rapidly. This is frankly yet another race to the bottom.
2. In the run up to an event – days to hours to minutes ahead – there are rumors and subtle signals. These may be explicit, say “We hear AAPL will post great iPad sales tonight,” or subtle signals like suddenly Google mentions growing rapidly and positively. Given these signals pre-event, there is ability to maneuver be it ahead of a price change in a stock or a protest event leading to instability.
1. Well ahead of an event – weeks to months – we may see a series of signals including:
- Explicit future signals
- “…Apple to release iPhone 5 on September 7…”
- Covert behavior
- A corporate layoff event followed by insiders selling stock indicating a fall in stock price
- Obscure signals
- E.g. clinical indications from a blog: “…this drug is making me sick …”
The early nature of such signals obviously makes them very attractive. At the same time, these are subtle signals, and it will take judgment, statistical rigor, or the like, to take advantage of effectively and confidently.
Tricky issues also remain in identifying prescient signals. These range from the technical (efficiently and accurately organizing references to time in news) to the psychological (how we go about researching and analyzing information that may indicate a future event).
In summary, the nature of news continues to change, and the game of analyzing it for actionable information as is shifting from news to pre-news to early event detection – that’s where the future is and the value lies. At Recorded Future, we’d like to think we can play a substantial role in this, and in changing the paradigm of how we seek out and understand future events around the world.