Our Approach to Microservices

Often, we hear the import­ance of split­ting soft­ware into smal­ler com­pon­ents when speak­ing about microservices. Inter­est­ingly, these prin­ciples date back to the 70s and refer to the good prac­tices we should adopt in code design. A per­fect example is the Unix philo­sophy, already proven from the­ory to the real world, with its imple­ment­a­tion as the ulti­mate and time-tested proof. Let’s review some of the main prin­ciples fol­lowed, sum­mar­ized by Peter H. Salus:

Write pro­grams that do one thing and do it well.

Write pro­grams to work together.

Write pro­grams to handle text streams, because that is a uni­ver­sal interface.

In the last years, the growth of cloud-nat­ive tools boos­ted microservices adop­tion, bring­ing these prin­ciples back to the spot­lights. Auto­ma­tion, vir­tu­al­iz­a­tion, and con­tain­er­iz­a­tion stand as pil­lars of cloud-nat­ive that enable teams to apply dif­fer­ent devel­op­ment prac­tices with infra­struc­ture in mind. As a con­sequence, it became a trend and a default to imple­ment dis­trib­uted systems.

At synvert we’ve been sup­port­ing many of our cus­tom­ers nav­ig­at­ing this trend in their jour­ney to become bet­ter-per­form­ing engin­eer­ing teams and we’re happy to share some lessons.

A trend that became a problem

We believe everything in soft­ware engin­eer­ing is a tradeoff. There’s no right or wrong and while dis­trib­uted sys­tems and microservices have many advant­ages they also bring down­sides and chal­lenges if not applied cor­rectly, or even worse, if they not are not serving any real need.

We often see teams begin­ning a new pro­ject with unclear scope, try­ing to imple­ment microservices from the get-go. Some­thing we dis­cour­age since it leads to unne­ces­sary costs and main­ten­ance hor­ror stor­ies! Here are the top 5 chal­lenges we see:

1. Com­mu­nic­a­tion com­plex­ity: the sys­tem becomes more com­plex as it com­prises mul­tiple ser­vices that com­mu­nic­ate with each other over a net­work. Instead of simple method calls, your team now needs to think about dif­fer­ent and more com­plex com­mu­nic­a­tion mod­els over the wire
2. Data con­sist­ency: each ser­vice may have its way to read and write data, mak­ing it dif­fi­cult to man­age data con­sist­ency and integ­rity across the sys­tem. Even­tual con­sist­ency is now a needed skill in the team
3. Oper­a­tional over­head: As the num­ber of ser­vices grows, the oper­a­tional over­head increases, includ­ing dis­trib­uted mon­it­or­ing, log­ging, and debug­ging. Teams need to be equipped with the neces­sary skills and tools to man­age the sys­tem effectively.
4. Test­ing com­plex­ity: Test­ing a microservices archi­tec­ture is more com­plex than test­ing a mono­lithic archi­tec­ture. Each ser­vice needs to be tested in isol­a­tion and as part of the lar­ger sys­tem, mak­ing test­ing more time-con­sum­ing and challenging.
5. Cost fal­lacy: While microservices can help organ­iz­a­tions scale and be more agile, imple­ment­ing a microservices archi­tec­ture can go both ways. Unfor­tu­nately, many times the wrong way. It requires sig­ni­fic­ant invest­ment in terms of infra­struc­ture, tools, and personnel.

Over­all, while microservices offer sev­eral bene­fits, we believe the team must be strong in code and sys­tem design to lever­age them. It should also allow the code design to develop and reach a matur­ity level where the code bound­ar­ies and spe­cific ser­vice infra­struc­ture needs become clear. Oth­er­wise, the most likely scen­ario is for the team to end up with a dis­trib­uted mono­lith that cre­ates more prob­lems than it solves.

Intro­du­cing the mod­u­lar monolith

A mod­u­lar mono­lith is essen­tially a mono­lith that’s designed with mod­u­lar­ity in mind. This means that dif­fer­ent com­pon­ents of the sys­tem are organ­ized into sep­ar­ate, cohes­ive mod­ules that can be eas­ily mod­i­fied without affect­ing other parts of the sys­tem. Remem­ber the Unix principles?

In many ways, a mod­u­lar mono­lith can solve most of the chal­lenges that microservices try to address, without the dis­trib­uted complexity.

The real chal­lenge in code and sys­tem design is achiev­ing mod­u­lar­ity and there are dif­fer­ent ways to achieve it, i.e: Domain-Driven-Design. How to achieve it is worth a whole new post (we’re on it…)! The key takeaway we would like to give is that this approach can derisk the devel­op­ment pro­cess, and make it easier to main­tain and modify the sys­tem over time, without rely­ing on the com­plex­ity of microservices, let alone, a dis­trib­uted mono­lith before it’s really needed.

Nev­er­the­less, the won­ders of the mod­u­lar mono­lith aren’t remov­ing the real value microservices can bring to a team and an organ­iz­a­tion. Let’s dis­cuss them below.

What if there’s a real need?

There are many known gen­eric reas­ons for microservices to make sense. From exper­i­ence work­ing with big teams, we always like to bal­ance the his­tor­ical con­text and future needs to decide when to use it.

From our exper­i­ence, it var­ies a lot depend­ing if we’re on a green­field or brown­field pro­ject. But, In the end, it always boils down to when there’s a real need for:

1. Dif­fer­ent DORA and QoS tar­gets per ser­vice & team
2. Split own­er­ship and ser­vice governance
3. Tech stack modernization

If any of the above is becom­ing a press­ing need and there’s no other way around it, microservices (tech­nical dimen­sion) with the right team topo­logy (organ­iz­a­tion and gov­ernance dimen­sion) are our go-to tools to solve them!

Here’s a simple play­book of how we look at it for green­field projects!

1. Achieve mod­u­lar­ity with code design

Mod­u­lar­ity is what we’re look­ing for! By being on the right track in regards to code design, and con­sequently hav­ing clear domain bound­ar­ies pop­ping up while code evolves, with needed com­mu­nic­a­tion mod­els and resi­li­ence needs it becomes increas­ingly easier to identify what should or not become a microservice without pre­ma­ture complexity

2. Fig­ure out the right team topo­logy for gov­ernance reasons

In par­al­lel, if any of the needs start pres­sur­ing we ana­lyze the right team and gov­ernance needed to start split­ting the pre­vi­ously iden­ti­fied domains and bound­ar­ies pop­ping up on step 1.

3. Cre­ate inde­pend­ent release pipelines

With 1. and 2. sor­ted out, it’s time to under­stand the tech­nical changes needed to achieve inde­pend­ence. There are com­mon fronts needed around cre­at­ing inde­pend­ent release pipelines, dis­trib­uted observ­ab­il­ity, and dif­fer­ent com­mu­nic­a­tion mod­els (say good­bye to syn­chron­ous HTTP requests)

Fol­low­ing this play­book, we ensure we keep things as simple as pos­sible while leav­ing the doors open for fur­ther scale!

The story and start­ing points are a bit dif­fer­ent for brown­field pro­jects… that’s the reason why we put in the hoven a sep­ar­ate and ded­ic­ated art­icle address­ing this scenario.

Final thoughts

With this art­icle, we want to raise aware­ness and our exper­i­ence that microservices are a double-edged sword that organ­iz­a­tions and teams should play with carefully.

Many people in the industry are mak­ing it a stand­ard without emphas­iz­ing the com­plex­it­ies and costs they bring. Nev­er­the­less, they also cre­ate immense and impact­ful bene­fits that can change the dir­ec­tion of a whole organ­iz­a­tion when done right. This is where we can help!

Synvert is a valu­able tech­no­logy part­ner that can sup­port defin­ing your tech­no­logy strategy and imple­ment­a­tion and we help organ­iz­a­tions weigh the costs, com­plex­it­ies, and real oppor­tun­it­ies asso­ci­ated with microservices before decid­ing whether and how to adopt them