Performance Engineering

What to Measure and Why

You cannot improve what you do not measure, but measuring the wrong thing is worse than measuring nothing. Performance engineering begins with choosing the right metrics and understanding what they actually tell you.

Latency and Percentiles

Latency is the time between a request being sent and its response arriving. The trap: reporting average latency hides the experience of your worst-off users. If P50 is 20ms but P99 is 1.2s, one in a hundred users waits 60x longer than the median user.

Throughput

QPS (queries per second) and TPS (transactions per second) measure how much work a system completes per unit time. High throughput with high latency often means requests are queuing. High throughput with low latency is the goal.

Other Critical Metrics

The Math Behind Scaling

Three laws form the theoretical backbone of performance engineering. They tell you what is possible before you write a single line of code.

Amdahl's Law

If a fraction s of your workload is serial (cannot be parallelized), then the maximum speedup with N processors is:

Speedup(N) = 1 / (s + (1 - s) / N)

Example: s = 5% serial
  N = 10  cores  →  Speedup = 6.9x
  N = 100 cores  →  Speedup = 16.8x
  N = ∞   cores  →  Speedup = 20x  (hard ceiling!)

Little's Law

A deceptively simple but universally applicable relationship for any stable system:

L = λ × W

L = average number of items in system
λ = average arrival rate
W = average time each item spends in system

Example: If requests arrive at 500/sec (λ) and each takes 200ms (W):
  L = 500 × 0.2 = 100 concurrent requests in the system

This tells you how many connections, threads, or workers you need. It works for queues, databases, thread pools, and even checkout lines at the grocery store.

Universal Scalability Law (USL)

Extends Amdahl's Law by adding a coherence penalty term: as you add nodes, they must coordinate (cache invalidation, locks, consensus). Beyond a certain point, adding capacity actually decreases throughput.

The USL model: C(N) = N / (1 + α(N-1) + β·N·(N-1)) where α is contention and β is coherence delay. When β > 0, there is a point of negative returns.

Finding the Bottleneck

Profiling answers "where is the time going?" before you optimize. Guessing where bottlenecks are is wrong more often than you think.

CPU Flame Graphs

Flame graphs (invented by Brendan Gregg) visualize stack traces sampled over time. The x-axis is the sampled stack population (wider = more time spent), and the y-axis is stack depth. They instantly reveal which functions dominate CPU time.

# Generate a flame graph on Linux
perf record -g -p <PID> -- sleep 30
perf script | stackcollapse-perf.pl | flamegraph.pl > flame.svg

# For Java (using async-profiler)
./asprof -d 30 -f flame.html <PID>

Memory Profiling

Track heap allocations, object retention, and GC behavior. In garbage-collected languages, excessive allocation rates create GC pressure even if peak memory is fine. Tools: jmap/MAT (Java), memray (Python), pprof (Go), Chrome DevTools (JS).

I/O Profiling

Disk and network I/O are often the real bottleneck but are harder to profile. Use iostat, iotop, strace (syscall tracing), and bpftrace for production-safe tracing on Linux.

Breaking Things on Purpose

Load testing validates whether your system meets performance targets under realistic conditions. Different test types answer different questions.

Tools

# k6 example: ramp to 200 VUs over 2 minutes
import http from 'k6/http';
import { check } from 'k6';

export const options = {
  stages: [
    { duration: '1m', target: 200 },
    { duration: '3m', target: 200 },
    { duration: '1m', target: 0 },
  ],
};

export default function () {
  const res = http.get('https://api.example.com/items');
  check(res, { 'status 200': (r) => r.status === 200 });
}

Performance Killers

These patterns silently degrade performance. They often pass code review because they look correct, but under load they are devastating.

# N+1 query example (Python/SQLAlchemy)

# BAD: N+1 — 1 query for users, then N queries for orders
users = session.query(User).all()
for user in users:
    print(user.orders)  # Each access triggers a lazy-load query

# GOOD: Eager loading — 1 query with JOIN
users = session.query(User).options(joinedload(User.orders)).all()
for user in users:
    print(user.orders)  # Already loaded, no extra query

Amortizing Connection Cost

Establishing a new connection is expensive. A TCP handshake is 1 RTT, TLS adds 1-2 more, and database authentication adds yet another round trip. For a cross-region connection at 80ms RTT, that is 240-400ms before the first byte of useful work.

A connection pool maintains a set of pre-established connections. When your code needs a connection, it borrows one from the pool and returns it when done. This amortizes the setup cost across thousands of requests.

Database Connection Pools

HTTP Connection Pools

HTTP/1.1 keep-alive reuses TCP connections across requests. HTTP/2 multiplexes many requests over a single connection. Most HTTP client libraries manage pools internally, but you must configure max connections per host and idle timeouts.

Pool Sizing

# PgBouncer config example (pgbouncer.ini)
[databases]
mydb = host=localhost port=5432 dbname=mydb

[pgbouncer]
pool_mode = transaction          # Return conn to pool after each transaction
default_pool_size = 20           # Connections per user/database pair
max_client_conn = 1000           # Max client connections to PgBouncer
reserve_pool_size = 5            # Extra conns for burst traffic
reserve_pool_timeout = 3         # Seconds before using reserve pool