Service Mesh: Istio & Envoy

How Envoy Works (Sidecar Pattern)

Traffic Flow:

1. App calls http://service-b:8080
2. Envoy intercepts (iptables rules redirect traffic)
3. Envoy applies routing rules, retries, load balancing, mTLS
4. Envoy sends to destination Envoy sidecar
5. Destination Envoy validates mTLS, checks policies
6. Destination Envoy forwards to app on localhost

1. Traffic Management

Canary Deployments (Gradual Rollout)

# Route 90% to v1, 10% to v2 (canary)
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: reviews-route
spec:
  hosts:
  - reviews
  http:
  - match:
    - headers:
        user-type:
          exact: "beta-tester"
    route:
    - destination:
        host: reviews
        subset: v2
      weight: 100
  - route:
    - destination:
        host: reviews
        subset: v1
      weight: 90
    - destination:
        host: reviews
        subset: v2
      weight: 10

---
# Define subsets (versions)
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: reviews-destination
spec:
  host: reviews
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2

Circuit Breaking

# Prevent cascading failures
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: reviews-circuit-breaker
spec:
  host: reviews
  trafficPolicy:
    connectionPool:
      tcp:
        maxConnections: 100
      http:
        http1MaxPendingRequests: 50
        http2MaxRequests: 100
        maxRequestsPerConnection: 2
    outlierDetection:
      consecutiveErrors: 5
      interval: 30s
      baseEjectionTime: 30s
      maxEjectionPercent: 50

Retries & Timeouts

apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: reviews-retries
spec:
  hosts:
  - reviews
  http:
  - route:
    - destination:
        host: reviews
    timeout: 10s
    retries:
      attempts: 3
      perTryTimeout: 2s
      retryOn: 5xx,reset,connect-failure

2. Security (mTLS)

Mutual TLS: Automatic encryption + authentication between services

# Enable strict mTLS for entire namespace
apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
  name: default
  namespace: production
spec:
  mtls:
    mode: STRICT

---
# Authorization: Only service A can call service B
apiVersion: security.istio.io/v1beta1
kind: AuthorizationPolicy
metadata:
  name: reviews-viewer
  namespace: production
spec:
  selector:
    matchLabels:
      app: reviews
  action: ALLOW
  rules:
  - from:
    - source:
        principals: ["cluster.local/ns/production/sa/productpage"]
    to:
    - operation:
        methods: ["GET"]

Benefits:

• Zero code changes - apps use plain HTTP
• Automatic certificate rotation (Istio CA)
• Identity-based security (not IP-based)

3. Observability

What you get automatically:

• Metrics: Request rate, latency, error rate (RED metrics)
• Distributed Tracing: See request path across services (Jaeger, Zipkin)
• Access Logs: Detailed logs of all requests

Example: Prometheus Metrics

# Istio automatically exports metrics:

# Request rate
istio_requests_total{
  source_app="productpage",
  destination_app="reviews",
  response_code="200"
}

# Request duration
istio_request_duration_milliseconds{
  source_app="productpage",
  destination_app="reviews"
}

# Query example
rate(istio_requests_total{destination_app="reviews"}[5m])

Distributed Tracing

Envoy automatically propagates trace headers (B3, W3C Trace Context):

# Trace shows entire request flow:
1. Frontend → Gateway (2ms)
2. Gateway → Product Page (5ms)
3. Product Page → Reviews (10ms)
4. Reviews → Ratings (8ms)
Total: 25ms

# Identify bottlenecks visually!

Q: How does Envoy intercept traffic?

A: Istio uses iptables rules to redirect all inbound/outbound traffic to the Envoy sidecar. The app thinks it's connecting directly, but traffic flows through Envoy transparently.

Q: What's the performance impact?

A: Typically adds ~0.5-1ms latency per hop and ~50-100MB RAM per sidecar. For most apps this is acceptable given the benefits.

Q: How does mTLS work without code changes?

A: App sends plain HTTP to localhost → Envoy intercepts, encrypts with mTLS, sends to remote Envoy → Remote Envoy decrypts, forwards plain HTTP to app. Apps never see encryption.

Q: Can you use Envoy without Istio?

A: Yes! Envoy is standalone. Istio is one way to configure Envoy at scale. You can configure Envoy directly for simpler setups.

Q: Istio vs API Gateway?

A:

• API Gateway: External traffic → internal services (north-south)
• Service Mesh: Service-to-service traffic (east-west)
• Often use both: Gateway for ingress, Istio for internal mesh

Q: How does circuit breaking prevent cascading failures?

A: If Service B is slow/failing, Envoy detects pattern (5 consecutive errors), temporarily stops sending traffic to B, returns errors immediately. This prevents Service A from queueing requests and running out of resources.