Everyone has heard about big data. Some have nothing but praise for it; others are not so sure. Many are still waiting to better understand its implications. It takes a little imagination to understand that big data is already impacting on almost every aspect of commercial enterprise, social and government sectors alike. In this week’s guest blog, Andrew Berry and Zoran Milosevic from DEONTIK provide insights to stimulate creative thinking in respect to collaborative partnerships.

The last few years have seen an unprecedented rise in big data and analytics. This rise, which some refer to as a renaissance in IT, is driven by the increasing supply of data on one hand, and the availability of new wave of technology which enables unlocking the value of big data for businesses, government, individuals and society.

The phrase ‘big data’ does not only refer to the sheer volume of data. The phrase is broader and also refers to the complexity, velocity and variety of data. Velocity is about speed of data creation and the need to analyse and react promptly while variety is about different types of data beyond traditional structured data, covering natural text, audio, images, video and other forms of data.

The new big data and analytics technologies coupled with the availability of cloud infrastructure, can be seen as enablers of new types of collaborations. These are further stimulated by new government legislations and incentives to promote data release and accessibility, as well as the push towards standardisation and ease of use.

Consider for example the healthcare industry. The standardisation of eHealth data and improved business and technical interoperability among healthcare providers, allows better quality of healthcare delivery and improved patient safety – while making use of new technology to monitor interactions between providers and behaviour of patients in near real-time.  The new legislations and new technologies also enable improved handling with privacy policies, as they allow more effective de-identification of patient’s personal information. This opens up many opportunities for better collaboration between clinical research organisations, healthcare providers and pharmaceutical organisations. The ultimate goal is to develop more effective and efficient care delivery. Further, the availability of more datasets and advanced monitoring technologies allows new possibilities to better monitor healthcare service outcomes, and thus enable government to provide proper incentives for providers. This is increasingly possible on much faster time scales than was the case in the past.

In addition to such enterprise data, the social media channels such as Twitter and Facebook provide new sources of information, which, when coupled with new analytics capabilities, facilitate new ways of rapid detection of disease outbreaks by monitoring social chats within specific communities or regions. This direct involvement of community in providing ‘close to the source’ information about potential health risks can be regarded as part of the establishment of a broader resilience community strategy that can also involve non-government organisations and businesses.

In summary, new types of collaboration can emerge not only within specific domains, e.g. healthcare, but also across domains, such as between healthcare, emergency management organisations and universities.

The US government provides leadership in the use of big data at broader scales, as reported in the article White House Makes Collaboration a Big Data Priority, http://www.applieddatalabs.com/white-house-big-data-collaboration. Australian Government too has recently issued a Big Data Issues Strategy paper stating that ‘the pursuit of big data technologies by government agencies will see a number of potential collaboration opportunities between agencies that will strengthen existing networks and help develop new partnerships’.

There is an increasing opportunity for innovation in the area, opening up possibilities for smaller organisations to participate. In the US, there were some 200 start-up companies in big data and analytics in healthcare only from 2010-2013, as reported in http://www.mckinsey.com/insights/health_systems_and_services/the_big-data_revolution_in_us_health_care.

There are some initial signs of similar trends in Australia, in spite of the fact that in general the Australian government needs to support innovations much more in the future, as recently reported in the ABC Inside Business,

http://www.abc.net.au/news/2013-06-09/australian-business-in-27rigor-mortis27/4740964.

Andrew Berry and Zoran Milosevic are founders of a young Queensland IT company, Deontik Pty Ltd, with a real-time analytics product called EventSwarm which enables organisations to identify and act on opportunities in real time. http://www.deontik.com/Products/EventSwarm.html.

Stay up to date with Roadmender by clicking on the ‘Follow’ button, below right.