Best practices for data management in microservices

There are different approaches to address the challenges of data management in microservices.

Use of container technology

Author:

When implementing microservice architectures, container technologies should be used consistently. Containers provide the necessary isolation and enable independent deployment of individual microservice instances. The recommended approach to data storage is thus closely linked to best practices from the world of container orchestration.

Container and PaaS infrastructures, such as Red Hat OpenShift, allow the implementation of various approaches to meet the different requirements of microservice applications. Microservices with «Database per Service» can be implemented very easily, especially with OpenShift or Kubernetes. The Pods concept of OpenShift provides independent containers with the business logic and the persistence solution (e.g. MySQL).

Distributed database systems

A central, shared database is not necessarily a suitable approach for microservice architectures. In such a scenario, the use of distributed, decentralized database systems (such as Cassandra ScyllaDB) is recommended. Each microservice is assigned its own space, which de facto corresponds to the «database per service» approach. Thus, this approach could also qualify for data storage in microservices.

Event-driven data management

There are various approaches for the implementation of event-driven microservice architectures. Frameworks such as Vert.X or Lightbend Lagom offer useful support. The Lagom framework, for example, already provides event-driven data management with event sourcing or a message bus API 'out-of-the-box'.

Due to the high level of complexity and the great divergence from conventional architectures, ipt recommends the use of event-driven data management only for new developments. There should be as few external dependencies as possible, especially not to legacy systems. Event-driven architectures are mainly suitable for applications with high scalability requirements (e.g. stream processing or high volume data).

The Apache Kafka product is suitable for implementing an event-driven data management solution. However, this is not intended for the long-term storage of business data. Here, a distributed database such as Cassandra ScyllaDB, which can be filled from the event stream, is still recommended as a supplement.

Event Sourcing is a well suited approach for data storage in microservices. However, it should be noted that solutions that include event sourcing require a steep learning curve. Due to the unfamiliar and multi-layered architecture, such implementations are often perceived as more complex.

Conclusion

There are different approaches to address the challenges of data management in microservices. From ipt's point of view, there is no patent remedy - depending on non-functional requirements, the solution architecture must be adapted with regard to data storage. In a highly distributed system, such as a microservice-based application, a balance must always be struck between business needs and technical limitations. Depending on the intended use, it is certainly sensible to choose a specific pattern or to combine different patterns. The key is that the persistence technology used is tailored to the specific application of the microservice.

Before a transformation to a microservice architecture, it should be thoroughly evaluated whether this new type of architecture pattern is to be adopted in the long term and whether the organizational requirements for this are met. Product-oriented teams in the DevOps model are the ideal basis for the successful implementation of a microservice architecture.