What can you get out of your unstructured data?

Most stored data is unstructured. However, this does not mean that you cannot use it.

Author: Andri Lareida

«Unstructured data is difficult to analyze». This statement is outdated and not true. Thanks to modern technologies, images, texts, audio as well as video recordings can be processed and analyzed electronically. In Switzerland, too, companies are beginning to put their unstructured data to productive use. Do you also have data in your company that you would like to use? This article shows you some possible use cases of unstructured data.

Unstructured data cannot be meaningfully represented as a table

An image is taken as a numerical value in three color channels, processed, sent and stored, but the relevant content remains hidden in this way: the picture from the last family party, a damage photo from last week's traffic accident, or vacation memories - a person will not group these images according to similar distribution of red, green and blue color values, but according to the same depicted people, objects, destinations and moods. The same is true with a text: A text is a string of letters and words - but the similarity between a letter carrier and a letter carrier can only be recognized by a system that can recognize the connection of words in context. This is the only way to distinguish whether bank means a financial institution or a seat.

Figure 1: Number of AI publications on arXiv.org. Ai Index Report 2019

Research has made tremendous progress in the field of unstructured data over the past decade

The availability of powerful graphics processors has triggered a boom in Deep Learning. Based on huge data sets and with great computational effort, Amazon, Facebook, Google, Microsoft and other large technology groups have designed and trained complex neural networks. These can, for example, understand and translate complex sentences or provide a meaningful answer to questions. In object recognition of images, these systems now regularly achieve higher accuracy than humans. In addition, these systems can recognize objects at a very fine granularity and also understand the overall impression of a scene. In the meantime, these systems can react so naturally to input texts that the answers can hardly be distinguished from the answers of a human being.

More and more powerful AI models are being made available through the community.

AI models allow existing models to be adapted to use cases at a very high level, e.g.:

  • Automatically match emails to the right person
  • Extract content from emails and automatically create a draft quote.
  • Automatically include scanned documents from customers in business processes and make them searchable
  • Automatically transcribe and summarize service center phone calls and store them in the customer relationship management (CRM) system
  • Automatically detect the sentiment of a comment on social media to respond appropriately when needed
  • Automatically check pictures of a car for previous damage before taking out an insurance policy
  • Automatically suggest the amount of insurance needed for a homeowner's policy based on pictures of the furnishings

These are just a few examples of countless possible applications.

Unstructured data can be analyzed and brings added value

Do you have a treasure trove of unstructured data in the form of texts, images or sound recordings in your company and you lack the manpower to analyze them? Or do you think your workforce spends too long analyzing or creating unstructured data? Then you can benefit from the advances in artificial intelligence. Define specific questions and applications you want to answer and implement with your unstructured data. Your data science team can then get to work evaluating the right technologies and models through experimentation.

Your ipt expert

I look forward to hearing from you