The Scaling Journey: From Single Database to Distributed Systems
Most SaaS platforms start with a single database. As user base grows, query times increase, and eventually, the database becomes the bottleneck. Strategic scaling decisions at different growth stages prevent costly rewrites.
Phase 1: Vertical Scaling and Optimization
Before horizontal scaling, optimize your current database:
- Query analysis: Use EXPLAIN ANALYZE to identify slow queries
- Index optimization: Create composite indexes on frequently filtered columns
- Query rewriting: Join optimization, N+1 elimination, and pagination
- Connection pooling: Use PgBouncer or similar to manage connections efficiently
This phase can 3-10x performance with minimal cost. Most SaaS products can serve 100k+ active users with a properly optimized single database.
Phase 2: Read Replicas and Caching
When read workloads exceed write capacity, deploy read replicas:
- Leader-replica architecture: Write to leader, read from replicas
- Replica lag handling: Applications must tolerate temporary inconsistency
- Read balancing: Distribute reads across multiple replicas
- Cache-aside pattern: Check cache before querying replicas
Redis or Memcached act as application-level caches, dramatically reducing database load. Cache invalidation strategies matter: time-based expiry for semi-static data, event-based invalidation for critical data.
Phase 3: Data Partitioning and Sharding
When even replicas can't keep up, shard your data. Sharding splits data across multiple database instances.
Sharding strategies:
- Range-based: Shard by date ranges or numerical ranges (simple but unbalanced)
- Hash-based: Hash key determines shard (balanced but harder to reshard)
- Directory-based: Lookup table maps keys to shards (flexible but adds lookup overhead)
Resharding (moving data between shards) is operationally complex. Plan shard count carefully—typically 2-4x your anticipated peak load.
Phase 4: Managed Database Services
Cloud providers offer managed databases with built-in scaling:
- AWS Aurora: Serverless scaling, automatic backups, multi-region replication
- Google Cloud Spanner: Horizontally scaled relational database
- Supabase: PostgreSQL with built-in replication and scaling
These services reduce operational burden but increase costs and lock-in.
Specialized Databases for Specific Workloads
Not all data fits SQL well:
- Document stores (MongoDB): Flexible schemas, good for user profiles
- Time-series databases (InfluxDB): Optimized for metrics and logs
- Graph databases (Neo4j): Relationship-heavy data
- Search indices (Elasticsearch): Full-text search and analytics
Polyglot persistence—combining database types—adds complexity but solves specific problems elegantly.
Frequently asked questions
When should we move from single database to replicas?
Monitor database CPU and I/O. When CPU consistently exceeds 70% during peak hours, or queries timeout frequently, implement read replicas. This typically happens at 50-100k concurrent users depending on your workload.
How do we handle sharding without losing performance?
Use consistent hashing to minimize resharding when adding shards. Implement a shard discovery service (Redis, Consul) storing shard mappings. Application queries include shard key, allowing direct routing to correct shard without scanning all shards.
What are the risks of moving to managed database services?
Vendor lock-in limits portability. Costs scale non-linearly with load. Data transfer between regions incurs charges. Managed services abstract complexity but reduce control over configuration and debugging. Evaluate tradeoffs carefully for your use case.