How database sharding splits the load

Software1 minute 3 secondsOpen on Instagram (opens in a new tab)

When one database gets too busy, sharding splits the data across several smaller databases, so each request only goes to the shard it needs.

Key takeaways

  1. Sharding splits one busy database into several smaller databases called shards.

  2. A rule, like a range of names or the ID's remainder, decides each row's shard.

  3. Each request goes straight to the one shard that holds its data.

  4. Less traffic in one place means faster searches and easier scaling.

One database gets too busy. More and more people show up, queues grow, and at some point you cannot keep making one machine bigger. So instead of making one database handle everything, you split the data into smaller databases. That is database sharding, and each smaller database is called a shard.

One library, many buildings

Think of a database as one big library where every book is in one building, with one front door. As the crowd grows, the lines get longer and the shelves fill up. Sharding opens several smaller libraries instead, and each one keeps a share of the books.

The rule that decides where data lives

Sharding only works if everyone agrees on where each row belongs. A rule decides that, based on a shard key such as a user's name or ID. Two common rules:

  • By range: shard 1 holds users A to H, shard 2 holds users I to P, and shard 3 holds users Q to Z.
  • By remainder: divide the user ID by the number of shards and look at the remainder. With four shards, the remainder is 0, 1, 2 or 3, and that number names the shard.

Every row follows the same rule, so it always lands in the same place. Looking for user 2345? Apply the rule: 2345 divided by 4 leaves a remainder of 1, so only shard 1 is checked. The others do not even wake up.

Why it matters

Now each request goes straight to the shard it needs. That means less traffic in one place, faster searches and a system that is easier to scale: when the shards get busy again, you can open another one and move some of the data over. The tradeoff is extra complexity. Choosing a good shard key matters, because a query that needs data from every shard loses much of the benefit. In short, database sharding turns one big database into several smaller ones that share the work.

All explainers

Loading related explainers