3 Questions to Ask Before Adopting Microservices

From my perspective as a product manager, I firmly believe that effective products and well-defined processes can resolve any issue, even complex ones like the considerable overhead associated with microservices.

My summer internship at Vertex Ventures US provided an opportunity to validate this conviction. Through conversations with over thirty professionals across various organizations – including Facebook, Fannie Mae, Confluent, Salesforce, and others – and a webinar featuring the founders of PagerDuty, LaunchDarkly, and OpsLevel, we focused on addressing three key inquiries:

1. What is the typical path teams take when implementing microservices?
2. What are the primary difficulties organizations encounter during this transition?
3. What approaches, workflows, and technologies are employed by businesses to mitigate these difficulties?

How do teams adopt microservices?

Based on conversations with numerous organizations, the vast majority are either actively implementing microservices or seriously evaluating their adoption. This aligns with broader industry observations; a survey conducted by O’Reilly involving over 1500 individuals revealed that over 75% have already begun adopting microservices.

It is uncommon for organizations to initiate projects by constructing systems entirely from microservices. Only one of the companies we consulted had taken this approach. While some newer companies, like LaunchDarkly, initially intended to build their infrastructure using microservices, they ultimately opted for a monolithic architecture after recognizing the significant overhead involved.

John Kodumal, CTO and co-founder of LaunchDarkly, explained, “We found ourselves dedicating more effort to constructing and maintaining the systems required for distributed environments than to developing our actual services, so we significantly scaled back that approach.”

He further elaborated, “For instance, tasks we attempted using mesosphere proved unachievable. We were unable to implement effective logging solutions, and achieving zero-downtime deployments was impossible. The underlying infrastructure contained numerous defects, and a substantial amount of time was spent resolving fundamental issues instead of building our services.”

Consequently, a more typical pattern involves organizations starting with a monolithic application and transitioning to microservices as a means of scaling their infrastructure alongside their growing teams. Most companies begin to decentralize control and move towards a microservice architecture when their development team reaches approximately 30 developers.

Larger enterprises with existing monolithic systems are eager to embrace microservices, but the costs can be substantial and the transition often spans several years. Atlassian’s platform infrastructure utilizes microservices, yet older monolithic components remain within Jira and Confluence despite continuous efforts to break them down. Many large companies encounter challenges in completing this transition. However, a robust strategy implemented from leadership, coupled with support from development teams, as demonstrated by Freddie Mac, can facilitate significant advancements.

Certain startups, such as Instacart, have initially adopted a modular monolith, which allows code to remain within a single repository while simultaneously initiating the process of assigning ownership of specific code functionalities to the appropriate teams. This approach helps to reduce the complexities associated with a full microservice architecture by combining the advantages of a centralized repository and release process with the flexibility of distributed code ownership.

What difficulties do teams encounter?

Teams may pursue varied approaches when transitioning to a microservice architecture, but they generally experience a consistent set of obstacles once implemented. John Laban, CEO and co-founder of OpsLevel, a company assisting teams in building and maintaining microservices, explained that “while a distributed or microservices-based architecture allows teams to operate independently, certain potential issues require attention.”

In fact, an O’Reilly survey illustrates that over 25% of respondents share the top 10 challenges organizations face during microservices adoption. Although we previously discussed some barriers to adoption, insights from our conversations emphasized difficulties related to complexity management.

An imprecise definition of what constitutes a service can lead teams to generate unnecessary work by developing numerous, highly similar services or by distributing related services among different teams. We spoke with one organization that excessively decomposed their monolithic application. Their service definitions were overly granular, and upon completion of the decomposition, they were left managing more than 4,000 microservices. They subsequently had to reverse course and consolidate to a more practical quantity.

Defining an excessive number of services results in unnecessary organizational and technical isolation, while simultaneously increasing complexity and overhead. Logging and monitoring are essential for each service, but dispersed ownership across various teams can create challenges in achieving comprehensive observability without standardized tools. It becomes difficult for teams to obtain a unified view when dealing with a large number of interconnected systems and services spanning the entire architecture.

As an illustration, one company previously utilized 10 separate monitoring systems, three distinct logging systems, and additional third-party providers contributing their own data. This proliferation of observability tools generates issues that negatively impact downstream processes, making crucial operations such as incident response considerably more challenging.

“Achieving the appropriate number of services is crucial,” stated Andrew Miklas, co-founder of PagerDuty. “There’s a fundamental reasonableness test for the number of services within an organization. If a developer supports three services, you likely have too many. Conversely, if an entire team of 10 developers supports a single service, it might be time to consider breaking it down.”

Effectively managing a microservice architecture also necessitates managing multiple codebases, each with its own unique dependencies. Each codebase also requires connection to its own release pipeline. As Continuous Integration and Continuous Delivery (CI/CD) practices advance and companies achieve multiple deployments daily, identifying the specific change from which codebase caused an issue when something fails becomes increasingly difficult.

While challenges are common, those we interviewed also identified inventive solutions to overcome difficulties associated with microservices.

What approaches and resources can businesses utilize to address these difficulties?

Conversations with software engineers and engineering leaders revealed key strategies and tools that organizations can employ to effectively manage the challenges associated with microservices:

1. Acknowledging and working with organizational boundaries. The most effective approach involves making it straightforward for teams to collaborate beyond their defined areas when necessary. Implementing consistent standards and unified formats for communication between teams facilitates cooperation when joint efforts are required. As one expert noted, many companies establish broad guidelines, granting teams operational independence provided they adhere to these established best practices.

Regarding tools, utilizing innovative technologies such as feature flags can empower teams to embrace siloed structures and services. The ability to separate code deployment from user-facing exposure provides increased flexibility in rollout strategies, as one source explained.

2. Achieving a balance between generalist and specialist skills within your team can also mitigate the organizational challenges inherent in distributed architectures. While each team benefits from specialists possessing in-depth knowledge of their specific codebase, incorporating generalists can foster connections between teams working on features that span multiple codebases, promote the sharing of best practices throughout the company, and enhance team understanding of how all codebases interact.

3. Standardization is key to simplifying microservices. While independent service ownership allows teams to select the most appropriate technologies for their needs, excessive autonomy can lead to unnecessary complexity.

It’s recommended to introduce new technologies only when they offer a substantial benefit. Focus on seeking improvements that are transformative, rather than incremental. Prior to adopting microservices, define a preferred technology stack for engineering teams to standardize upon.

“When making technical choices, consider the needs of the entire organization, not just the specific software you are currently developing,” one professional suggested. This approach simplifies hiring, knowledge sharing, and concentrates development efforts on the most valuable technologies.

4. A service catalog is a particularly valuable solution for enhancing visibility across the microservice architecture. The primary purpose of a service catalog is to alleviate the management overhead associated with microservices by providing a central repository for developers to access all necessary information about their infrastructure. This includes details on costing, observability, and team ownership, enabling developers to understand how infrastructure aligns with the company’s organizational structure.

Companies such as OpsLevel, Cortex and Effx assist development teams in building superior software by illustrating how their services interconnect and map to the organizational structure. John Laban articulated his vision for managing microservice architectures and the associated challenges: “The ideal scenario is where development teams have complete ownership of their software, encompassing both design and operation. This model works well with a distributed architecture, fostering autonomy and accelerating development. However, this independence necessitates increased responsibility regarding reliability, security, and compliance.”

The most effective tools extend beyond simply tracking service level objectives (SLOs); they facilitate collaboration to create better services. For instance, OpsLevel helps teams identify the programming language used for a specific service, determine the appropriate contact person, understand how to reach them in case of issues, and stay informed about upcoming changes.

These tools can also address practical issues like identifying and eliminating unused services, compiling services for comprehensive legal and security audits, resolving incidents, and promoting technology standardization across the infrastructure.

Consider the scenario where your organization needs to adopt a new technology, such as Kubernetes, or upgrade to a newer version of a language like Java or Python. This process may seem manageable with a small number of services. However, once your organization scales to over approximately 30 services, ensuring standardization across the entire stack becomes exceedingly difficult.

With organizational silos focusing teams on their specific infrastructure components, cross-team engineering efforts require a microservice catalog to ensure all teams are aligned on adoption status. Companies like OpsLevel maintain an updated list of all services and their owners, along with metadata on languages and frameworks, to facilitate complete adoption.

Another critical application of microservice catalogs is in incident response. Interviews revealed the significant difficulties teams encounter when resolving incidents in a microservice architecture. With services distributed across numerous teams and monitoring data fragmented across various siloed products, obtaining a comprehensive understanding of the events leading up to an incident is challenging.

Tools like Effx leverage the service catalog to consolidate data from various services and data sources, providing a complete picture of an incident. For example, a developer on support duty can use Effx to quickly scan their services for incidents using monitoring data from tools like Datadog. Upon identifying an incident, they can construct a timeline of events using monitoring data, past deployments, active feature flags, and provisioning changes to diagnose the root cause.

Bringing It All Together

Microservices have become a broadly recognized approach for enabling organizations to expand their infrastructure in alignment with their organizational structure.

The majority of teams initially begin with a monolithic architecture to minimize initial costs. As a company grows and naturally establishes specialized areas of concentration and responsibility, microservices assist in connecting the appropriate services with the corresponding teams.

Greater intricacy introduces difficulties in overseeing this infrastructure. Uncontrolled service growth, technical and organizational isolation, and managing dependencies can hinder development and diminish the enjoyment of software creation.

However, development teams are inherently focused on finding solutions, and they have created methods to address these challenges. Establishing consistent agreements across teams, appropriately sizing development teams, standardizing processes and tools, and maintaining a service catalog all assist teams in handling the complexity.

These approaches have enabled those we interviewed to scale their systems to support some of the most widely used products available today. We are eager to observe the innovative products that startups will develop to assist teams in fully leveraging the capabilities of microservices. If your organization is seeking input on improving its microservice infrastructure, or if you are developing a new product to simplify microservices management, please leave a comment below or contact us.

Vertex Ventures US holds a financial stake in both LaunchDarkly and OpsLevel.