7 tips for more successful machine learning projects

Studies show that 85% (1) of AI projects fail or do not generate measurable business value. This does not have to be!

Learn 7 tips on how to successfully master machine learning projects.

Author: Dominik Keusch

Tip 1: Use the technology in the right place

It is undisputed that the further developments in the field of Machine Learning (ML) in recent years influence various industries and change the working world. On the other hand, there is the danger of overestimating the hype. ML brings a new tool into the system landscape. Using this tool and the new possibilities in a targeted manner and in the right place is the art. Start from the challenges in business, from real problems and do not use the technology for its own sake.

Tip 2: Develop ML-specific project competence

Even if your company has its own IT department and develops its own software, this does not mean that you are ready for ML projects. An ML project differs from a classic IT project in many ways. It is mainly about data, models and evaluations:

  • Different know-how, different roles in the team, different tools and procedures are used. Both worlds can and should benefit from each other, but make sure that you create a good exchange between the two worlds. 
  • While operationalization in the IT area by means of source code management, CI/CD pipelines, monitoring is very mature, best practices in the ML area are only slowly establishing themselves.
  • This is made more difficult by the fact that not only the source code but also the data, models, parameters and results should be versioned. 

Tip 3: A comprehensive AI & data strategy is a must

Once you want to use the possibilities of AI / ML across departments, you need more than a few ML specialists. Often there is a big gap between the technical developers of ML solutions and the actual users of the application. With an AI & Data Strategy you can get all employees and management on board and build more confidence in data as a whole company. The new possibilities of data-driven approaches require a complete rethink, new processes, creativity, openness. For a refreshing input on this topic, watch the recording of the ML Days keynote by Jeffrey Bohn - Chief Research and Innovation Officer of Swiss Re.  

Tip 4: Assessing and managing expectations realistically

The expectations for AI / ML projects are often unrealistically high. This leads to hasty conclusions, disappointed looks and therefore Proof of Concept (PoCs) are often not even continued. The actual potential of the technology thus remains unused. With ML projects, it is worthwhile to manage expectations from the outset. Get a realistic assessment from experts and make the expected results transparent to your stakeholders.  

Tip 5: Do not underestimate the path from proof of concept to production

Many ML projects start with a PoC in a highly reduced, encapsulated setting. There is basically nothing to be said against this. However, be aware that there is a long way to go from a successful PoC to successful use in a productive system. The main reason for this is that PoC often only focuses on training and evaluating a model. All other necessary steps of operationalization are neglected. Figure 1 clearly shows that the PoC is often limited to the small black block in the middle. 



202010_7 Tipps für erfolgreichere ML Projekte_Hidden technical Dept_02.png
Figure 1: The PoC compared to the implemented ML project in the productive system Source: Hidden Technical Debt in Machine Learning Systems (2), D. Sculley et al.

Tip 6: Generalize at the process level

Most of the algorithms and models in ML have one thing in common: They serve to solve a specific, well-defined problem. The data, the algorithms and the models used are selected to solve the problem in the best possible way. If you try to use the model in a different domain, this often fails. The model for problem A is therefore difficult to transfer to problem B. What does this mean for you and your company? Don't generalize the models or the tasks, but the processes. In contrast to the ML models, the process of how you approach ML projects can be generalized well. Likewise, the skills and the tools you need to successfully implement the projects.  

Tip 7: Handling data consciously and with foresight

Finally a word about the extensive topic of data. The amount and quality of data is of undisputed importance - most people know that. Only few people know how to obtain complete, high-quality data. In practice, I often experience the reason for this: you have to collect the data today that you want to evaluate in the coming years. This presents you with a hen-egg problem. Today you do not know yet which problems you want to solve in the future with ML. But you have to collect the necessary data today. So a blind collecting of all possible data has turned out to be as bad strategy as not collecting the data. A long-term AI & data strategy, which is aligned with the company strategy, helps you to break through the chicken - egg problem. 

Your ipt-Expert

I look forward to hearing from you.