Unstructured data: the hidden treasure for retailers

MARCH 10, 2022

Understanding what the customer wants, having the right insights to drive the types of decisions that are made every day and at all levels within an organization. Will never cease to be a competitive advantage for retailers.

Customer data is the Holy Grail for customer-oriented businesses, especially in today’s hyper-connected digital environment that has moved things on customer inputs from contact centers and survey results to dozens of touchpoints through the customer journey and across all types of devices. 

An overlooked treasure – As most of the businesses keep relying solely on structured data as quantitative customer reviews, to guide their strategy and improve their customer-facing operations, an abundance of valuable information for the customer preferences, buying behavior remains untouched, with only just 5-10% being adequately assessed, 

While structured data remains an essential piece of information that is clearly defined and easy to report, providing solid input on customers’ place, gender, age, and insights into how they feel about your brand, the vast majority of scattered, unstructured data on the emotions of a customer across the whole customer journey have been left outside the CX-analysis equation.

Failing to find a way to best incorporate those hidden data into their overall plan, keeps businesses away from really connecting with consumers; where structured data offer a place to start while searching for issues that can assess a business’s overall sentiment, unstructured data are set to provide insights on how consumers encounter unique challenges through different levels of brand experience.

Yes, the exploitation of those big, unstructured data forms, as CS calls’ recordings/chat logs, email correspondence, social media posts, reviews, as well as offline data as digital and scanned documents including invoices, purchase orders, contracts, written agreements and policies, survey results, and application forms, is the real challenge for retailers these days, constituting a staggering 90 to 95% of all consumer data that could help businesses pinpoint core challenges and move the organization forward.

Leaving unstructured data in the dark – Unstructured data, or Dark Data as Gartner defines it, is, “The information assets organizations collect, process, and store during regular business activities, but generally fail to use for other purposes’’. These are vast volumes of information that, although extremely useful to a business’ strategy, fail to be analyzed by most of the retailers for two, quite straightforward reasons:

  • Too big to handle – The sheer amount and the variety of data generated can be overwhelming, as it ranges from social media data and various 360 touchpoints to data buried in documents, call records, and transcripts from audio and video files.
  • Don’t have the right tools – The technology required for effectively deriving value from such data for practical production scenarios has been missing.

Unleashing your CX power – With AI and related technologies having already been used for the classification of written texts like spam in emails, the sentiment analysis of social media posts, the recommendations of most relevant news/media by search engines, as well as to help to assess risks/threats to national security in social media posts, the time has come for them to be used as an effective way of identifying relevant information from both digital and offline sources.

As AI unleashes its processing power today and, in the years, to come, encompassing machine learning, natural language processing (NLP), and deep learning, together with the emergence of more distributed computing and storage solutions, dark data mining has been now made possible, ready to become the critical asset of any retail business that wants to offer the full view of the consumer experience, combining the two forms of data. 

Customer Experience platforms are the right tools for this, applying breakthrough artificial intelligence (AI) technologies and machine-learning (ML) algorithms to unstructured customer interaction data, mining the data for insights on datasets that were previously regarded as unusable, together with real-time analysis to uncover patterns, insights on product, customer experience, and customer feedback analysis/satisfaction and sentiment analysis of communications.

Along this data mining process, powerful ‘AI Impact Analysis’ algorithms can for example distinguish between what people talk about most frequently and the mentioned things that have the greatest impact on their behavior, assessing how customers think, feel and act about an experience and revealing how those thoughts and feelings can translate into behaviors, to understand the relative importance of each interaction in their journey in real-time.

Data analysis that is based on the unfiltered voice of the customer served by AI should help companies understand what customers really value, how it feels to be a customer along their multi-touchpoint journey, and most importantly, the ‘why’ behind customer’s behavior, choice, beliefs, and expectations, helping organizations get their hands around their data so that they can use it, either strategically to make important long-term customer-focused decisions, or by leveraging it in real-time to make operational decisions at the moment. 

Driving Change – Unlocking the value of dark data, is not alone the key to CX success, as the organization needs to be able to align these insights and operationalize them across the following functions:

  • Leadership: A leadership team that is empathetic and customer-focused.
  • Alignment: Customer Experience vision that is linked to the overall business strategy and imperatives.
  • Linked metrics: CX and journey metrics that are linked to overall business metrics.
  • Employee engagement: Staff being engaged around the customer, with frontline staff being empowered to proactively interact with customers.
  • Optimized processes: Supporting processes are connected to the customer journey – old processes have been overhauled, streamlined, and internal barriers in delivery, have been removed.

Conclusion Hidden dark data is a real treasure that translates to hidden risks, value, profits, efficiencies, savings for retail organizations, as there is an immense potential to uncover insights from this data and deliver meaningful business value to the enterprise. CX’s success in retail can only come from a commitment from companies to truly put the customer first, by harnessing this valuable customer data, taking intelligent action, and effectively aligning the organization. Thanks to technological advancements, the time is upon us to start out on the journey towards improved automation. This will allow for valuable insights to be gained from this otherwise hidden treasure trove that is dark data.