Sometimes a random Reddit thread sends me down a rabbit hole for an entire evening, and this was one of those nights. Someone posted about how companies like YouTube scale their databases, and the comments were full of people saying they genuinely don't understand how a single system can handle that much load. That question stuck with me, so I went digging.
How YouTube actually does it
Turns out YouTube runs on MySQL, sharded using Vitess, a system originally built at YouTube itself to let MySQL scale horizontally across thousands of shards while still looking like one database to the application. That immediately made me look at my own setup and ask: why am I not doing this?
Taking inventory of what I actually run
I have two servers. One runs a shared PostgreSQL instance in Docker, serving multiple small sites. The other is dedicated to Immich, using Postgres with the pgvector extension for similarity search on photos. The only thing running MySQL is my mother's site, doc-agnes.com, on WordPress, and that one I can't touch. I'm not confident enough yet in migrating a live WordPress database without risking her data, and I'm honestly not sure how well WordPress even plays with Postgres in the first place.
Down the Citus and Aurora hole
So I went looking at whether Postgres has its own answer to Vitess. Citus came up, a Postgres extension for distributed/sharded Postgres, but by most comparisons I found, it's slower than Vitess for pure horizontal MySQL-style scaling. That had me grumbling: maybe I should've built on MySQL and Vitess from day one instead of Postgres.
Then Aurora came into the picture. AWS built their own storage engine underneath Aurora rather than relying on stock MySQL or Postgres internals, and the numbers impressed me. It outperformed both in a lot of the benchmarks I read. But when I asked Gemini to weigh the three against each other, it pointed out Aurora's standard configuration can start struggling at very high sustained write throughput (rough figures put a stress point somewhere around 150k writes), which knocked it down a peg for write-heavy workloads at serious scale.
Then I compared operational complexity. MySQL with Vitess is the hardest to run by a wide margin, since you're managing sharding logic, proxies, and connection handling largely yourself. Citus, being a Postgres extension, is comparatively painless. It rides on top of a database I already know instead of asking me to bolt together a whole new distributed system by hand. That's when it clicked why so many companies default to Postgres now, not because it's necessarily faster in every benchmark, but because the operational tax is so much lower. (Aurora Limitless, AWS's newer distributed Postgres offering, looked solid too.)
A market share detour
Somewhere in this spiral I looked up database market share and landed on Postgres sitting at the top of developer-survey rankings, with MySQL close behind, though depending on which survey you look at (installed base vs. what developers say they use and want), the ordering flips. Either way, Postgres' momentum right now is hard to miss.
Redis came up too. I always assumed it was purely a caching layer, and Gemini told me I was mostly right, but technically it's a full in-memory database, not just a cache sitting in front of one. The speed comes from RAM being drastically faster than disk. What I was fuzzy on is that Redis can still persist to disk in the background (snapshotting, or an append-only log of writes), so a restart doesn't necessarily mean losing everything. It's not purely ephemeral the way I assumed.
The reality check
After all that reading, comparing, and grumbling to myself about architecture I'll probably never need, I had to laugh at myself lol. I don't have YouTube's problem. I have about four users per project, people I actually know, and the heaviest thing I'm storing is photos in Immich. No shards, no Vitess, no 150k writes per second. Just me, Postgres, and a very small, very happy dataset. But hey, at least now I understand the theory.