Eighty-five percent of the matter in the universe is invisible to us, and little is known about it. Scientists call it “dark matter.” Dark matter interacts with the rest of the universe—in fact, scientists say that we would not exist without it—but it does so in very mysterious ways.
Executive Summary
Insurance carriers have a lot of dark data hiding in plain sight. Here, Actuary Peter Bothwell notes that while carriers are starting to use NLP text mining to turn their unstructured dark data in claim notes and underwriting evaluations into structured data, even system logs contain valuable dark data. The system log data can be used by operations managers and application developers to optimize carrier policy, billing and claim operating processes.The same is likely true of much of the data that your organization produces and saves. Although it exists and interacts with your business, the business itself is largely unaware of its potential value. Hence, today it remains “dark data.” If you can be one of the first to find and use your dark data, you have an opportunity to create real competitive advantage.
The concept of dark data is becoming widely used and accepted. Gartner defines dark data as “information assets that organizations collect, process and store in the course of their regular business activity but generally fail to use for other purposes.” Wikipedia goes on to say: “The ability of an organization to collect data can exceed the throughput at which it can analyze the data. In some cases, the organization may not even be aware that the data is being collected.” (Original source: “The API Briefing: the Challenge of Government’s Dark Data,” www.digitalgov.gov, U.S. General Services Administration, June 17, 2015 by Bill Brantley)
