Benefits and Challenges of Data Discovery

Benefits and Challenges of Data Discovery

Written By Nathan Roberson (@theMrobot) - October, 27th 2013

One of the newest creations of the data tools and technology industry is data discovery. While the new digital golden child has been met with a lot of excitement, there are still professionals in the data industry who have yet to fully reap the benefits of the new discovery. Some of the more recent and most heard complaints about DD are that the tools take too long to fully set up, are difficult to use and have a limited scope in terms of their overall applications. As with most new discoveries, there are instant benefits to DD as well as challenges that come attached to those benefits.

Definition

Much like business intelligence, also known as BI, data discovery has different meanings to different people. The most basic definition of DD is “discovering what your data can uncover.” A majority of vendors and practitioners have started to find a more tightly focused definition that’s created from their own personal context.

One example of a tightly focused definition is individuals who work in data quality and management will want to concentrate their DD on uncovering important metadata about essential data assets. Their main focus is on making sure that the data they have is thorough, the overall quality of the data and seeing that the data is consistent. A market scientist, on the other hand, will be more concerned with predictive analytics and will see data discovery as an instrument that can be used to identify current trends, observe campaigns and improve models or self-service reporting and BI instruments that are used for marketing. Anyone who specializes in selling specific DD tools might want to focus the definition of the term to resemble the limitations of what their solution can actually accomplish.

The Recent Need for Better Methods and Tools

Modern day professionals who work mainly with data already know how vital DD is, but they also realize the final results and efficiency of DD can differ wildly. There’s a reason the need for improved methods and tools has become so important, including:

• Big Data. DD is both essential and more difficult on projects that involved big data. All of the data has to be processed the correct way in order for the discovery to be more effective, and the different formats and sources that are used have a tendency to make the usual DD methods “seize up.” Instances where big data initiatives also include the swift profiling of heavily incoming big data also makes profiling all of the data more difficult and less feasible through the use of existing toolsets.
• Real-Time Analytics. The continuing trend towards real-time analytics has birthed a brand new class of DD use cases. While these new cases have their merit, they necessitate DD tools that are more powerful, can adapt better and are more automated.
• Nimble BI and Analytics. Business intelligence teams and data scientists have started to use more nimble, iterative ways of utilizing data business value. They’ve started to use DD processes more and in an assortment of different ways.

Best Ways to Use DD

As DD starts to gain more of a foothold as a genuine field, there are certain practices that will become more vital as time passes. Being able to use DD faster to know things sooner about your data will enable you to make enhancements to your individual goals and overarching reasons for using it in the first place.

You’ll also want to have an end goal in mind before you dive into DD so that you don’t run the risk of becoming lost as you start exploring the data. Even if you aren’t sure of what exactly it is that you’re searching for, you’ll still want to have a specific business goal in mind in order to keep from wasting valuable time and resources.

While you might not yet understand DD, it’s something that you should at least think about looking into to see how and if it could prove beneficial to you and your company.

Image Sources:IBM – Power in Data – The Atlantic

Nathan Roberson

Nathan Roberson is a lifelong coffee addict, aficionado of all things punk rock, and is particularly fond of Boxer dogs as well as all things tech, marketing design and internet. follow him on Twitter @theMrobot or add him to your circles on Google +