Microservice Architecture: A realization.
Nine out of ten individuals who have come to this article likely have coded a monolith application or definitely at least have used one, and the sole individual who has to google the meaning of monolith is a developer in Netflix who started career a few years back. Yes, there are monoliths everywhere, and you know it, for decades they powered the businesses and are still fueling the public as well as private organizations. However, the competition and need to deliver more and that too more often is driving a push to microservice. Organizations or agencies in various sectors, whether federal or state have spent, or spending, or are willing to pay millions to go on the cloud. This post is not about should-you or should-you-not opt for microservice, instead what you SHOULD do, to migrate a legacy application to an actual microservice world successfully.
If you have read the first few annoying lines, I presume you want to get bored further and will continue reading. Let me get you sleepy by not wasting any more time.
Well, presume you are an architect and have been tasked by the CIO of your organization to modernize the legacy system into the future, or better I not waste a moment to correct myself by paraphrasing what I said- ‘you have been tasked to the future’. Maybe you can give a more appealing phrase.
Your organization’s application is a massive monolith and has an enormous silo relational database in which all the business data lives collectively. To you, the data looks like a vast dorm room virtually partitioned into multiple sections and to go around in the dorm a student has to pass from parts within. You, as an architect, know the principles of ‘Separation of Concerns’ and ‘Divide and Concur.’ These are the principles you want to apply for migrating to microservice. You act like a pro architect and decide to sit with business domain experts and technical leads to start working on dissecting the silo data model into multiple sub-data models based on the business domain. You know that it is not a small and easy task, and by doing this, you will have to compromise on the foreign integrity constraints and therefore will be risking data to corruption. Nonetheless, right data model which can serve the development of the microservices is required. So you go ahead and slice up the schema like a butcher slices up the meat.
Once you have sub data models which can live independently as decoupled entities yet are cohesive enough when weaved together can describe the whole system, only then you could justify the migration to microservices. After weeks of closed-door day-long meetings you now have completed one of the two most strenuous tasks of the assignment, doing it right is no less achievement and should be celebrated. So you go out and celebrate with the team. A right data model is going to drive in the right direction the design, development, configuration, deployment, and testing, of course.
As you have given away the monolith and now have multiple sub-data models, your business transaction will now span various services. To be clear, earlier in monolith application web request which persisted data to multiple tables was in a single transaction as the monolith was using one data source configured in the application server, but now those tables are in multiple schemas, and henceforth there are multiple HTTP calls from one service to another to serve a user request. Moreover, when you realize this, you say to yourself ‘welcome to the distributed transaction management.’
By now, you have understood that transaction management is the next big thing in microservice development; in fact, you learned it much earlier and designed your data model by being considerate of transaction management. At this moment, you take a pause and ponder, did I take into consideration all of the business transactions when I perceived the data model? Is it a microservice data model, or have I inadvertently visioned a distributed monolith? Suddenly you realize that you have promised your family time out in the evening and you pack your laptop and zoom to the home. While driving back home, you ponder on the hard work you and the team did to give shape to the data model and decided to move ahead and work first thing in the morning to learn about handling the distributed transaction management. There are two ways to treat a distributed transaction; one is based on JTA other is based on Saga.
The Java Transaction API, which in short is JTA uses 2-Phase commits to ensuring consistency in a distributed request. There are various implementations of JTA, such as Atomikos, JOTM, Bitronix. While on the other hand, Saga is driven by events which are published by various services to communicate the changes in their respective data sets; it heavily uses message brokers. Being an architect who is driven by data you decide to evaluate both the approaches by doing a small proof of concepts. Based on the results, you get to know that using 2-Phase commit is more comfortable to implement; however, it comes with its cons, which restrict its usage to relational databases. While Saga, which is complex to implement, as one needs to deal quite a bit with the messaging layer and not everyone is comfortable with it. However, with Saga, you can use services which are supported by non-relational data sources such as MongoDB. It is a very crucial decision and it is not easy to make. There are two thoughts which need to be considered when deciding. First is which approach will enable developers to code quickly, and the second one is, which can lead to a flexible microservice architecture.
There is something significant to retrospect; it’s the domain model which you came up earlier. Remember you had a party after the formal sign off. By this time, you have learned that both transaction management and data modeling go hand in hand. If you fail to represent the data in a microservice way, then you will undoubtedly have impaired transaction management which will force you to seek trade-offs, you never intended. You decide to revisit the domain model and see if there is room to further split the model to have the smallest transaction boundary while keeping alive your CIO’s dream of scraping the monolith.
You started with a monolith model which looks like below. The shapes represents the tables, and similar shapes means tables belonging to a particular functionality.
In the first iteration, you were able to split up the monolith model into multiple aggregates. However, you have a transaction which now spans five aggregates henceforth data model demands a revisit.
With additional knowledge about the pros and cons of two different approaches to transaction management, you decide to make short the transaction boundary. To do that you gather the business experts and look to denormalize the aggregates. You ended up moving columns and creating more tables in the aggregates.
After the above exercise, your microservice architecture looks like below. The transaction now spans three services instead of five as other two microservices publish the data events.
Plausibly you may decide to refine the data representation further to isolate a transaction boundary. Your ultimate intent is to have a data model for each service, which will enhance the degree of autonomy of the service, in terms of the business functionality, design choice, programming language choice, data store choice. After, finishing multiple iterations, to refactor the data model, you have developed a belief that no matter which vendor platform you choose to deploy, whether IBM, RedHat, Pivotal, AWS or any cloud or on-premise solution, the hard part you still have to do yourself. And if you will not do it right then, no vendor will do it for you.
Remember, microservice is far far more than a Spring Boot code running in a Docker container.
