Classifying texts with Azure ML

Categorizing customer correspondence is time and resource intensive. We show a solution based on artificial intelligence (AI).

Author: Andreas Schneider

Answering customer concerns quickly and well requires that they are put in the right place. We show how we can achieve this with modern methods from the field of artificial intelligence (AI). Thanks to machine learning (ML) in the cloud, we can develop, deploy, operationalize and ultimately bring the application to market faster than with traditional methods.

Use Case
Language is our most important means of communication. That is why the automated evaluation of texts holds great potential. Text classification is one of the "low hanging fruits" with direct benefits for the user. So let us now pick a fruit from the machine learning tree. 

Our example is an insurance company that wants to assign its customer inquiries, automated with text classification, to the right people. In this way, the e-mails reach the right place more quickly and can be answered promptly. This should now be done by a "learning machine" and no longer manually by one person. The machine tells us with what probability the e-mail belongs in one of the different categories (Figure 1).

Lernmaschine.png
Figure 1: A learning machine automatically estimates the membership probabilities for a given customer mail.

Further use cases for textual data can be found in the blog post on AI-based Natural Language Processing: 7 application areas in modern companies.

Text Classification with Azure ML - 3 phases


Our ML model for e-mail classification learns from sample e-mails, of which we already know the correct classification. We use the methods provided by Azure ML to version data, code, and models so that we can later trace the development of our model.

Successful machine learning is characterized by a clean management of data, models, code and deployments. To achieve this we should keep the ML lifecycle in mind and pay attention to best practices. Azure ML helps us to do this.

Machine Learning Lebenszyklus_Webseite.png
Figure 2: Machine learning life cycle. 1. acquire data, 2. train models, 3. provide models. Repeat.

Machine learning projects, like the one described here, roughly consist of three phases (Figure 2). For each phase there are concepts in Azure ML which simplify the development. Figure 3 gives an overview of the implementation of our use case:

lAblauf Trainingsdaten bis fertiges Modell_Blau.png
Figure 3: Sequence of steps from the training data to the finished model.

1.    Phase: Obtaining data

  • Reading e-mails: We load the labelled mails as dataset to Azure ML.
  • Preprocessing: We convert the email content into a sequence of numbers. Words then correspond to numbers. 

2.    Phase: Training models

  • Training: We create models that we train with the pre-processed texts and categories from the previous step.
  • Evaluation: We evaluate the performance and adjust the models if they are not satisfactory. We select the best model for deployment.

3.    Phase: Deploy model

  • Deployment: Offer model via Rest-API on an Inference Cluster
  • Monitoring: Azure ML collects the input data and predictions to detect data drift and detect performance degradation early.

4.    Phase: Repetition Phase 1 to 3

  • The circle closes as we continue training the model with the newly collected data.

 

Especially for Switzerland, we have to consider multilingualism in all three phases in order to achieve good results. For each language we have different amounts of data, from which we create language-specific models.

Azure ML supports us in the implementation with various concepts that simplify the development. Datasets give us versioned access to our e-mails, regardless of which computing instance we are on. Results, plots and other information that we record during the training can be viewed in the web interface for each run, allowing us to compare different runs. We record the classification models in the Model Registry. It gives us an overview of all models and model versions and simplifies our model selection. With Azure ML we can easily offer finished models via the Internet and have them monitored afterwards. Azure ML creates a REST API for model queries. Pipelines give us the possibility to train existing models with the freshly collected data. Azure ML also offers additional features such as Auto ML to automate the building of models. 

Conclusion

Azure ML gives us a simple infrastructure on which we can develop machine learning models with existing knowledge in the quality required for companies. The acceleration through the cloud not only shortens the time-to-market, but also allows us to train models with special hardware requirements. At the same time, in the cloud we only pay for the services we need.

In addition, Azure ML supports - or almost forces - us to build reproducible and traceable models because data, code and models are consistently versioned. Gone are the days when someone built a model that afterwards nobody knew how it came about.

Thanks to machine learning, our customers can react faster and better to their customers' concerns. Automated categorization is just the beginning. Artificial intelligence offers further possibilities to support customer contact in a personalized and automated way.

Tech Box

Data: E-mail contents, labels for the different categories, read in with pandas.

Model: Time series multi-class classification, LSTM layers with Keras.

Cloud: Azure ML Python SDK, model development as pipeline, training with GPUs, deployment on Inference Cluster.