AWS App Mesh vs. Istio: A Comparison Of Service Mesh

Context

As Microservices architecture isolates system functionalities into multiple independent services, the communication mechanism between these services plays a key role for a scalable, available, and high-performance architecture.

A service mesh is an infrastructure layer that manages service communication in a transparent and language-independent way. It helps to configure how service instances perform critical actions such as service discovery, load balancing, data encryption, and authentication and authorization.

Applicability

The service mesh pattern is more applicable when you have:

• Services in multiple programming languages requiring a common architectural pattern for communication.
• Offloading responsibilities such as security, service discovery, load balancing, authentication, monitoring, etc.
• Particularly useful in synchronous communication (REST, gRPC, WebSocket). For asynchronous or event-driven, it can co-exist along with event-mesh or messaging architecture.

Istio

Istio is the most popular open-source service mesh created by Google, IBM, and Lyft with native integration to Kubernetes, and Envoy as Service Proxy. Istio Control plane handles automatic load balancing, routing, policy management, observability (metrics, logs & traces), authentication and communication, and many more.

Istio is an open source service mesh that helps organizations run distributed, microservices-based apps anywhere. 

Istio as Managed Service is available with, which are top 5 providers to Istio opensource codebase:

• Google Anthos Service Mesh — Earlier Google Cloud provided Istio as a managed service, which is now deprecated. Google recommends migrating to Anthos Service Mesh, which is powered by Istio only.
• Aspen Mesh — As part of F5, it provides a fully hosted SaaS platform for Istio with multi-cluster/multi-cloud capabilities, operational support, security-hardened distribution, RBAC support, object-driven policy, simplified mTLS management, an intuitive UI, and other features.
• Solo Gloo Mesh — It offers enterprise Istio with multi-cluster and hybrid cloud management with a unified control plane, zero-trust security, operational support, and many more. It supports AWS (with EKS Anywhere), Azure, Red Hat, and on-premises deployment.
• Istio on IBM Cloud —It is offered as a managed add-on that integrates Istio directly with an existing Kubernetes cluster. It supports single-click deployment with a production-ready Istio instance on the IBM Cloud Kubernetes Service cluster.
• Red Hat OpenShift Mesh — It provides a production-grade Istio with support for a hybrid cloud enterprise Kubernetes platform and is most suitable if you are using OpenShift Kubernetes distribution.

Click here to see the complete list of all service providers as part of the Istio ecosystem. 

Vendors offering Kubernetes distributions are most likely to have an Istio distribution complete the ecosystem. Synergy between Kubernetes as a PaaS and Istio as a Service mesh solution is an important aspect in the enterprise.

AWS App Mesh

AWS App Mesh is a managed service providing application-level networking and service communication support for Microservices architecture. It also uses Envoy proxy making it compatible with multiple solution approaches. It supports h AWS Fargate, Amazon EC2, Amazon ECS, Amazon EKS, and Kubernetes running on AWS.

Features Comparison

While there is no silver bullet for comparison, it is dependent on system context and applicability to meet functional & non-functional requirements. The below list includes all the key features required for the system under consideration as part of our research with High & Low or Yes & No as relative measures:

Feature	AWS App Mesh	Istio	Notes & Key Observations
Operations Cost	Low	High	Istio provides a self-managed control plane and it has operational overhead if you are not using a managed service.
Portability	Low	High	Istio is built for Kubernetes and also supports VMs and is compliant to SMI (via adapter). It can be integrated with self-managed K8s, EKS, GKE, etc. First-class support for Google Cloud (Istio on GKE)
Flexibility	Low	High	Istio is extensible & vendor-neutral App Mesh – first-class support for EKS, ECS, Fargate. However, even with Kubernetes on EC2, need to use AWS
Ease of setup (with EKS)	Medium	High	Istio – Setting up with EKS requires a learning curve and a deeper understanding of Istio. Click here to see details. App Mesh – With EKS, setting up is easy especially because of AWS support and integration. Click here to see details.
Observability	Medium	High	Istio – Tracing by adding request header is available out-of-the-box.Click here to see details. App Mesh – The application code needs to be updated to enable tracing. Click here to see details.
Kubernetes support	Low	High	Istio natively integrates with Kubernetes
Multi cluster support	High	Yes	AWS App Mesh – Click here to read about using App Mesh in multi-account setup. Istio – Click here to read about multi-cluster setup using Istio.
Weighted routing	Yes	Yes	Useful for A/B, Canary deployments, etc.
Rate Limiting	No	Yes	App Mesh feature planned in App Mesh Roadmap
Fault management	Partial	Yes	Both support timeout, retry circuit breakers. However, Istio also supports variable jitter between retries, health checks, failure injection. App mesh does not allow using custom error codes for retry configuration and only applies default policy.
Traffic Mirroring	No	Yes	Istio allows traffic mirroring currently while App Mesh does not have this feature right now. Click here to read more about Istio Traffic Monitoring.
Security – mTLS and Authentication	Yes	Yes	Both support mTLS. However, Istio offers more granular control over mTLS setup including automatic certificate rollover Also, Istio allows request Authentication and has integration available with Auth0, Keycloak, etc. Click here for more details on Istio. App Mesh has IAM integration which is a plus for AWS workloads and setup.

Other Alternatives

While Istio and AWS App Mesh are great solutions, there are other competing technologies (not in the scope of this topic) worth considering:

• Linkerd — created by Buoyant, it is one of the lightweight service mesh built using Rust. It has reached the graduated maturity level as part of CNCF.
• Open Service Mesh — created by Microsoft, it is one of the lightweight service mesh built using Golang. It is the sandbox project as part of CNCF.
• Consul — built by Hasicorp, it provides a single binary providing both server and client capabilities. It uses the Envoy proxy with a sidecar pattern.
• Kuma — developed by Kong, it is Envoy based service mesh built using Golang and it is a CNCF sandbox project.

Conclusion

Each technology selection including service mesh is contextual – a system under consideration, organizational maturity, architecture maturity, developer skill-set, architecture roadmap, alignment with current organizational technology landscape, security considerations, etc.

Considering the above factors, in my perspective below guiding principles help to make the service mesh selection decision:

AWS App Mesh is more suitable for scenarios where:

• the application architecture is leveraging AWS cloud-native architecture and synergy between cloud services is preferred
• managed service model without minimal operational overhead is preferred
• granular control is not the need of the hour
• engineering skillset is limited in terms of understanding of service mesh
• vendor lock-in is not an area of concern or opensource code is not the need of the hour
• multi-cloud service mesh is not required

Istio Service Mesh is more suitable for scenarios where:

• the application architecture is leveraging common mesh architecture as an underlying infrastructure layer
• vendor lock-in is not preferred and opensource is the key priority
• the industry-standard solution is preferred to be more aligned with multi-cloud architecture
• granular control is needed with flexibility in managing mesh network, policies, configuration, setup, etc.
• the engineering team has service mesh knowledge and is keen to manage or even customize the solution if needed

References

• Introducing AWS App Mesh – service mesh for microservices on AWS
• AWS App Mesh – Application-Level Networking for Cloud Applications
• Istio Introduction
• Guide for Service Mesh by InfoQ
• Service Mesh Feature List Comparison