Load Testing: Preparing for Traffic Spikes

The Load Test You Skipped

A product launch arrives. Traffic is 10x normal. Your application melts. Requests timeout. Customers leave. Revenue drops.

Load testing prevents this. You simulate 10x traffic in staging, find bottlenecks, fix them, then launch with confidence.

Most teams skip load testing because it's "complex." It's not. It's a few hours of effort that saves weeks of firefighting.

Load Testing vs Stress Testing

Load testing: Apply typical production load. Measure response time, throughput, resource usage.

• Goal: Verify the system handles expected traffic.
• Load: 100% of expected peak traffic.

Stress testing: Apply load beyond capacity. Find breaking points.

• Goal: Discover where the system breaks.
• Load: 150-300% of expected peak traffic.

Both are essential. Load testing verifies normal operation. Stress testing finds the ceiling.

Tools: JMeter, Locust, k6

Apache JMeter:

• Desktop application, or command-line.
• Visual test plan builder (record interactions, replay).
• Supports HTTP, databases, FTP.
• Generates detailed reports.
• Best for: Complex scenarios with multiple request types.

Locust:

• Python-based. Define load tests in code.
• Web UI for running tests and viewing metrics.
• Easy to extend with custom logic.
• Best for: Teams that prefer code over UI.

k6:

• JavaScript-based. Test scripts in JS.
• Cloud-hosted SaaS option (paid) or self-hosted (free).
• Excellent developer experience.
• Best for: Modern APIs and microservices.

Recommendation for startups: k6 for simplicity, Locust if you already use Python, JMeter if you need complex scenarios.

Practical Load Test

Scenario: E-commerce site. Expected peak: 1000 concurrent users browsing, 100 buying per minute.

Load test plan:

1. 100 concurrent users → 10 second ramp-up
2. Each user browses (5 GET requests)
3. Each user searches (1 GET request)
4. 10% checkout (1 POST request)
5. Hold for 5 minutes
6. Measure response times, error rates, throughput

k6 test script:

import http from 'k6/http';
import { check, sleep } from 'k6';

export let options = {
  vus: 100,        // Virtual users
  duration: '5m',  // 5 minutes
  rampUp: 10,      // Ramp up over 10 seconds
};

export default function() {
  // Browse products
  let res = http.get('https://api.myshop.com/products');
  check(res, { 'status is 200': (r) => r.status === 200 });
  sleep(1);

  // Search
  res = http.get('https://api.myshop.com/search?q=laptop');
  check(res, { 'status is 200': (r) => r.status === 200 });
  sleep(1);

  // 10% checkout
  if (Math.random() < 0.1) {
    res = http.post('https://api.myshop.com/checkout', {
      orderId: `order_${Date.now()}`,
      items: ['item_1', 'item_2'],
    });
    check(res, { 'checkout succeeds': (r) => r.status === 200 });
  }
}

Run with: k6 run test.js

Result: You see response times, error rates, throughput. If p95 latency is >1s, your API is too slow. If error rate >1%, something breaks under load.

Finding Bottlenecks

Load test results show symptoms. Now debug.

Symptom: High latency

• Check database connection pool (maybe it's exhausted?)
• Check database query performance (run EXPLAIN ANALYZE)
• Check for N+1 queries (app issues too many queries)
• Check CPU usage (underpowered instances?)

Symptom: Increasing error rate

• Check application logs for patterns
• Check database error logs
• Check infrastructure (disk full? memory exhausted?)
• Look for dependency timeouts (external API slow?)

Symptom: Memory usage climbs

• Check for memory leaks (objects not released)
• Check connection pool size (connections hold memory)
• Check cache size (unbounded cache grows)

Use monitoring during load test: watch Prometheus metrics, application logs, and database queries in real-time. The spike in your metric correlates with the bottleneck.

Capacity Planning

Use load test results to estimate infrastructure.

Example:

• Load test: 100 users, 50ms p95 latency, 2000 requests/sec
• Production expectation: 10,000 concurrent users
• Scale factor: 10,000 / 100 = 100x

Naive calculation: Need 100x the infrastructure.

Reality: Not linear. If database is bottleneck, more app servers don't help. Database scales differently than application layer.

Smart approach:

1. Identify the bottleneck (usually database, cache, or external API)
2. Estimate: at 10,000 users, database will do 20,000 queries/sec
3. Test database with 20,000 queries/sec
4. Use results to size database (instance type, read replicas, caching)
5. Size application servers based on CPU/memory profiles

For most applications: application tier scales linearly. Database tier does not. Cache aggressively. Use read replicas. Consider sharding for extreme scale.

Chaos Testing

Chaos engineering: intentionally break things during load testing to see what happens.

Chaos scenarios:

• Kill a database replica (see if failover works)
• Inject latency into API calls (what if external API is slow?)
• Drop 5% of packets (network issues)
• Fill disk to 90% (see if app handles gracefully)

Run load test + chaos simultaneously. If the system recovers, great. If it cascades into failure, you've found a real problem to fix before production.

Load Test Checklist

• Define realistic load profile (concurrent users, requests per second)
• Create load test scripts (use k6, JMeter, or Locust)
• Run against staging environment (never production)
• Monitor metrics during test (CPU, memory, latency, errors)
• Identify bottlenecks (database, application, external APIs)
• Fix issues and retest
• Document results and capacity limits
• Run monthly (before product launches, before major holidays)
• Run chaos tests (inject failures)
• Communicate limits to team (we can handle 10k concurrent users safely)