Distributed Systems

Hot Partition

Also known as: hot key, hot shard, partition skew

Definition

A hot partition is a single shard receiving a disproportionate share of traffic, so it saturates while the rest of the cluster idles. Because throughput is bounded by the busiest partition, not the average, a well-provisioned cluster can throttle at a fraction of its nominal capacity.

Last reviewed · Part of the Architecture Glossary

In practice

Hot partitions come from partition keys that mirror real-world skew:

Partition keyFailure
tenant_id in multi-tenant SaaSThe largest customer is 40% of the traffic
date on a time-series tableToday's partition takes 100% of the writes
status = 'PENDING'The whole queue lives in one value
Monotonic id or timestampAll inserts land at the end of the keyspace

Remedies, cheapest first:

  • Add entropy to the key. tenant_id#bucket where bucket = hash(record_id) % 16, then scatter-gather on read. Trades read fan-out for write spread.
  • Cache the hot key in front of the store, so the read amplification never reaches the partition.
  • Isolate it. Give the whale tenant its own shard. Cleaner than pretending the distribution is uniform when it never was.

When it matters

DynamoDB, Cassandra, Kafka (partition = ordering unit), sharded Postgres, and any multi-tenant system with a power-law customer distribution.

Common mistake

Capacity-planning against average per-partition throughput. If one partition takes 30% of the traffic on a 10-shard cluster, you cannot use more than about a third of the cluster's rated capacity — provision for the hot one and design the key to flatten it.

See also

Go deeper