PBT Group thought leadership

Research firm, Gartner, defines dark data as the information assets organisations collect, process and store during regular business activities, but generally fail to use for other purposes. Considering how data has grown over the last few years, and with research predicting that worldwide data will grow 61% to 175 zettabytes by 2025 – businesses must start considering how their dark data sets are growing, how they need to manage and control these and importantly, ask if this so called dark data can be of any value.

With emphasis naturally placed only on data perceived to be of good quality, that holds a purpose for a business within that moment in time (and rightly so for effective analysis), the likelihood of dark data growth across businesses in all sectors is very high – and in some cases without an organisation even realising it.

In fact, a past IBM study stated that over 80% of all data is dark and unstructured. IBM also estimated that this figure will rise to 93% by 2020. We can link such predicted growth back to the fact that technology is consistently evolving, with businesses investing in technological aspects like Machine Learning and the Internet of Things (IoT) etc. – all generating large amounts of data. The reality is that a digital business today is being hit with data from all sides.

As not all data is of immediate value, much of it is stored and becomes ‘dark’. While there is merit in data storage, there is little to be gained if sufficient processes are not formed around dark data and if the dark data is completely forgotten about. When it comes to dark data realities, businesses need to take careful consideration and planning into account – to not only avoid a missed data opportunity, but to mitigate risk that dark data can pose.

An initial first step to be considered is to ensure that the business has full sight of, or is well aware of, all the data being generated and coming into the business. A solid data foundation plays a key role in getting this right.

Following this, data storage strategies should not only be considered for quality data that holds a purpose, but also for data that can be considered dark. While the dark data may not hold an immediate value or purpose, it should still be placed into a storage strategy that takes key aspects into account – to ensure data protection.

Compliance and data security come to mind. Not only are businesses required, from a regulatory standpoint, to have certain compliance factors bedded down within data storage strategies – but with the consistent growth in cybercriminal activity, they must consider how at risk their dark data is and how they need to protect it. Although data may be considered dark to one business, it could still hold significant value to cybercriminals and their cybercrime-based activities.

Furthermore, although considered dark on the onset, a business that pays no attention to its dark data can in fact miss out on valuable analysis and detail rich information opportunities later down the line. Dark data, relooked at regularly, could hold the insights a business could use to gain a competitive advantage it may be seeking.

A business should not let its dark data go unnoticed. Sound data management processes, backed by the right technological tools, compliance, data security and management systems for dark data become important for a business to not only see the value in its dark data, but to also mitigate some of the risk.