Tag Archives: D2DCRC

beat the news

Beat the News with digital footprints

Every day we produce an almost unfathomable amount of data. Posting on Twitter, Facebook, Instagram and YouTube. Commenting in chat rooms; blogging; trading stock tips; and decorating hacks in niche forums. We broadcast what we’re eating, feeling and doing from our GPS-equipped smartphones, sharing maps of our runs, photos from shows, and news that gets us cranky or inspired.

The details of our passing moods are all there, creating a vital if virtual public pulse.

Dr Brenton Cooper’s Data to Decisions (D2D) CRC team checks this pulse and, by extracting signals from our collective digital footprint, shows where we’re going next.

Are we gearing up to strike? Or celebrate? Is disease spreading? What effect will an interest rates hike have? Are we about to toss out the government, or move money out of the market?

Whatever the social disruption, D2D CRC’s Beat The News ™ forecasting system can issue a warning – before it happens. In March 2016, it accurately forecasted the impact of an anti-coal port protest in Newcastle, NSW. The following May, no ships could move during the protest blockade, costing an estimated $20 million.


“This warning system tells you what might happen, when it will happen and why.”


Social media monitoring is already a billion-dollar industry, and Cooper, who is D2D CRC’s Chief Technology Officer, knows “there are plenty of tools that help you understand what’s happening right now. But this tells you what might happen, when it will happen and why.”

This sort of heads-up will be invaluable. D2D CRC’s first collaborators are Australia’s defence and national security agencies, whose analysts now have a Beat The News ™ dashboard that sifts through about two billion data points a day.

“These are people paid to understand the political climate, but they can’t read everything,” explains Cooper. “That’s where machine-enablement certainly helps.”

Maybe the agencies are watching Indonesian politics and want to know if there might be some unrest in the capital Jakarta. Beat The News ™ analyses a huge volume of open-source information, combining structured and unstructured data from a wide range of sources. It geo-locates posts, extracts key words, topics, trends and hashtags, and measures sentiment.

“Once we’ve done those types of data enrichments, we then pump it through a variety of models,” says Cooper, “to automatically and accurately predict the answer.”

The potential applications are many, so the CRC recently trademarked Fivecast™ – “as in forecast, only one better,” says Cooper – to take the system to market, whether as a spin-off company, licensing to a partner, or licensing the IP to a third party.

US company Dataminr has raised more than US$130 million from investors for its real-time analytics, but Cooper says Fivecast™ will offer a further capacity – event prediction. It’s the only predictive geopolitical risk analytics platform on the market. Corporate risk consultancies are already interested. Their clients include global mall conglomerates alert to anything that might stop people enjoying their shopping.

Find out more about Beat The News ™ at d2dcrc.com.au

– Lauren Martin

You might also enjoy ‘Disrupting Terrorism and Crime’. Sanjay Mazumdar, CEO of the Data to Decisions CRC (D2D CRC), takes a look at what the national security sector can learn from Big Data disruption.

national security data

Disrupting terrorism and crime

When people think about digital disruption they usually think of the peer-to-peer accommodation network AirBnB, or the inexpensive ride-sharing app Uber. These businesses have redefined their respective markets – with big data analytics1 underpinning their success.

Despite the fear that disruptive tech will bring with it new threats to security, Australia’s national security has much to benefit from the type of disruption brought about by big data – particularly when it comes to fighting terrorism and crime.

The national security sector faces the most imminent and complex big data challenges. This is because a powerful weapon of today’s terrorist or criminal is their ability to hide in data. They can plan and coordinate an attack or crime with impunity.

The ability for criminals to “hide in data” means that national security agencies are often faced with the daunting task of finding the “needle in the haystack” – where the haystack is growing at a phenomenal rate. In fact, people often comment that national security data analysts are “drowning in data, but starving for information”.

Big data analysts often need to find connections in vast, disparate volumes of data, where connections are imperceptible to humans but can be discovered using smart analytics and machine enablement.

The challenge is made greater by the wide variety of data sources (e.g. texts, voices, images, videos), the ever-increasing size and scale of the data collected, and the organisational and legislative silos impacting data agencies.

The effect of big data means that national security data analysts often spend most of their time collecting data, formatting it for analysis and generating reports, and less of their time doing the analysis. This is referred to as the “bathtub curve”.

The application of big data analytics is aimed at “inverting the bathtub”, which means automating the collection and processing of data to form intelligence. The generation of intelligence reports can also be automated via digital technologies, which enables analysts to spend more time analysing intelligence and making decisions.

The D2D CRC is developing applications to maximise the benefits that Australia’s national security sector can extract from Big Data. They are helping agencies generate timely and accurate intelligence as a powerful weapon against national security threats.

By addressing their big data challenges and applying high-performance analytics, the D2D CRC hopes it can support agencies in predicting threats rather than reacting to catastrophic aftermath. 

Sanjay Mazumdar

CEO, Data to Decisions CRC

Read next: Victoria’s Lead Scientist, Dr Amanda Caples, reveals the major flaw in traditional government approaches to disruption. 

Spread the word: Help Australia become digital savvy nation! Share this piece on digital disruptors using the social media buttons below.

More Thought Leaders: Click here to go back to the Thought Leadership Series homepage, or start reading the Women in STEM Thought Leadership Series here.

1 Big data is a term for any collection of data sets so large and complex they becomes difficult to store, process and analyse using current technologies. Big data analytics is the process of examining these data sets to uncover hidden patterns, unknown correlations, trends and other useful business information.