Practical guide scope
Who this is for
Platform engineers, SREs, and architects choosing production traffic routing
Where it applies
Nginx, HAProxy, Kubernetes ingress, cloud load balancers, APIs, TCP services, and database failover paths
Problems this guide helps solve
- The chosen load balancer does not match L4/L7, health-check, or failover requirements.
- Health checks mark a process healthy while the customer path is broken.
- Timeouts and retries amplify failures or duplicate requests.
- Routing configuration has no safe validation or rollback process.
Why load balancing architecture matters
Load balancing is not just traffic distribution. In production it becomes the place where availability, latency, rollout safety, and failure isolation meet.
L4 versus L7
L4 balancing is usually simpler and faster. L7 balancing gives better routing control, headers, path rules, and observability, but it also adds more moving parts.
HAProxy and Nginx trade-offs
HAProxy is usually my first choice for explicit traffic control, health checks, connection behavior, and failover patterns. Nginx is excellent when HTTP routing, TLS termination, static delivery, and caching matter.
Production checks
- Health checks must match real readiness, not just open ports.
- Timeouts should be aligned between client, proxy, and backend.
- Retry behavior must not multiply load during partial outages.
- Metrics should include p95/p99 latency, 5xx, queueing, and backend saturation.
Decision matrix
| Approach | Best for | Risk |
|---|---|---|
| L4 | Simple high-throughput TCP | Less routing context |
| L7 | HTTP services and canary | More config complexity |
| HAProxy | Precise failover behavior | Needs careful tuning |
| Nginx | HTTP/TLS edge | Less explicit for some failover cases |
Operational takeaway
Start with the simplest architecture that gives you safe failover and observable behavior. Optimize after you can see the failure modes.
Implementation blueprint
Use this sequence to turn the theory into an auditable production change. Adjust commands, thresholds, and ownership to the real environment before execution.
-
Classify traffic and failure requirements
Document protocols, TLS termination, session behavior, idempotency, timeout budget, health semantics, expected throughput, and failover targets.
- L4 or L7 requirement is explicit
- Retry safety is understood
- Health endpoint represents serving ability
-
Choose routing and health-check architecture
Compare Nginx, HAProxy, ingress, service mesh, and cloud load balancers against operational complexity and failure behavior.
- Control plane failure is considered
- Backend draining is supported
- Observability exposes backend state
-
Test partial failure and rollback
Simulate slow backends, connection refusal, dependency failure, certificate rotation, deploy drain, and restoration of the previous configuration.
- Slow backend does not exhaust all workers
- Bad backend is removed
- Config rollback is tested
Configuration and command examples
Examples are conservative starting points. Review security, version compatibility, failure behavior, and rollback before production use.
HAProxy HTTP baseline with health checks
Tune timeouts to the application budget and avoid retries for non-idempotent requests unless behavior is explicitly safe.
defaults
mode http
timeout connect 3s
timeout client 30s
timeout server 30s
backend api
balance leastconn
option httpchk GET /ready
http-check expect status 200
server api1 10.0.0.11:8080 check inter 2s fall 3 rise 2
server api2 10.0.0.12:8080 check inter 2s fall 3 rise 2Production validation checklist
- A slow or failed backend is removed without taking down healthy capacity.
- Timeouts align with client, proxy, application, and dependency budgets.
- Retries cannot duplicate unsafe transactions.
- TLS, headers, source IP, and request IDs remain correct.
- Deploy draining prevents new traffic before process termination.
- Routing changes pass syntax, smoke, and rollback checks.
Official references
Stable reference
Version, testing scope, and citation
Yuri Osipov. "SteadyOps guide: load balancing comparative architectures." SteadyOps, version 1.0.0, reviewed 2026-07-10. https://steadyops.best/articles/load-balancing-comparative-architectures/ Production reliability review
Need this implemented safely in your environment?
Send the current stack, failure mode, and required outcome. SteadyOps will reply with the inputs needed for a focused review and the safest next step.
Focused request
Need this implemented safely in your environment?
Send your current stack and the production risk. Optional commercial details can be added after the technical context.