Database Sharding

Splitting a large database horizontally into smaller shards, each holding a subset of rows on its own server, so data and traffic can scale beyond one machine. A shard key decides which shard stores each row.

Replication copies the same data to several servers for availability and read scale. Partitioning divides a table into pieces, often within one server. Sharding spreads those pieces across servers so each holds only part of the data. Most large systems combine sharding with replication.

Pick a key with high cardinality and even distribution that matches your most common queries, so most queries touch one shard. Poor keys create hot shards. MongoDB and DynamoDB document shard and partition key design.

Only after simpler options are exhausted: better indexes, query tuning, read replicas, caching and table partitioning. Sharding adds complexity in transactions, joins, resharding and operations. See the Database Performance Tuning and Database Replication pages.

Distributed SQL systems such as CockroachDB, YugabyteDB, TiDB and Spanner, and databases such as MongoDB and Cassandra, shard automatically. Middleware such as Vitess, Citus and ShardingSphere adds sharding to MySQL or PostgreSQL. Building sharding into application code is the most work.

A technique that maps keys and nodes onto a ring so that adding or removing a node moves only a small fraction of keys. It is described in the consistent hashing and Dynamo papers listed on this page.

Links and release facts were checked on 31 October 2026 from GitHub. The MySQL documentation returned 403 to automated checks. Links to the Instagram, Meta and GitHub engineering posts were dropped because they did not resolve. Descriptions follow each project's own pages and are not rankings.