Relational databases slow down as the number of records in a table grows, especially without proper indexing. Sharding addresses this by dividing a large dataset into smaller, manageable segments (shards) spread across multiple servers.

Use cases

  • Large databases that a single server cannot handle efficiently.
  • High traffic: distribute load across servers.
  • Geographic distribution: keep a shard close to its users (for example, a European shard for European users), which also helps with regulation and compliance.
  • Tenant isolation: give each tenant, or a key customer, a dedicated shard.

Each shard can also be scaled vertically on its own, so you upgrade only the shard under pressure.

Advantages

  • Improved performance: each server handles less data, and queries know exactly which shard to search.
  • Increased capacity: add more servers as data grows.
  • Fault isolation: if one shard fails, the others keep working.

Disadvantages

  • Complexity in distributing data and routing queries.
  • Uneven distribution can overload some shards.
  • Cross-shard joins are complex and inefficient.

Choosing shard count and algorithm

Consider data size, query patterns (shard on fields you filter by), and growth rate. Common algorithms are range-based, hash-based, and directory-based sharding.

Where the sharding logic lives

It is usually in the application layer. Some databases, like MongoDB, route queries to the right shard automatically. Amazon RDS does not support sharding natively, so you implement it in your application.

SQL vs NoSQL

Sharding relational databases is harder because of schema design, joins across shards (often solved by denormalization at the cost of extra storage and write complexity), and multi-record ACID transactions. NoSQL databases are often designed with sharding in mind but may lack joins and multi-record transactions.