Capstone·18 min read
Scaling a Photo App to a Million Users
Ten pieces, one photo-sharing app, and the exact break that makes each one necessary.
Big Idea
Scaling, load balancers, caching, CDNs, replication, sharding, queues, rate limiting, consistency, failover. Every one of them was added to some real system the moment something broke without it. Nobody designs the million-user architecture on day one, they grow into it one break at a time, and each fix buys capacity by taking on a cost. Knowing the list is not the skill. Knowing which break calls for which fix, and what it will charge you, is.
Why This Exists
Why A List Of Components Is Not A Design
These ten terms are usually taught as a list of components, which makes them look like a checklist to apply. Each one is really a response to a specific symptom, and reaching for it before the symptom exists buys the cost with none of the benefit.
A real system and an interview question ask the same thing: what breaks first, and what does fixing it cost. Someone who shards a 10,000-user database has answered a question nobody asked, and has given up cross-shard queries to do it.
Growing one app from a single server to a million users puts the pieces in the only order that explains them, the order the breaks actually arrive in, so the reason for each one is still attached to it when you meet it again on its own.
Think Of It Like
A Food Cart That Became A Chain
One cart, one cook, one griddle. A queue forms down the block, so you buy a bigger griddle, which is scaling up. The griddle has a size limit, so you open a second cart, which is scaling out, and now somebody has to stand on the corner pointing people at the shorter line, which is the load balancer.
Then the three dishes everyone orders start getting cooked in advance and kept warm, which is a cache. Customers across town get a branch in their own neighbourhood instead of a long walk, which is a CDN. The paperwork that used to happen at the counter while the customer waited moves to a back office, which is a queue.
None of it was planned at the cart. Every piece arrived the week the previous setup stopped coping, and every piece added something new to go wrong: a corner with nobody on it, a warm tray of yesterday’s special, a back office that lost an order.
The Shape Of It
Day One, And It Is Already Correct
Interactive
Watch It Grow, One Break At A Time
One diagram, ten stages. Press play, or step through it. Nothing on the canvas ever moves, it only arrives, and the badge on each box is the stage that put it there.
The Walkthrough
Ten Breaks, And What Each Fix Costs
Worked Example
Emma’s Photo, And The Replica That Had Not Heard
Stage five bought read capacity and stage nine pays for it. Here is the whole bill, in six messages: the write commits on the primary, the refresh is routed to a replica for the good reason that replicas are there to take reads, and the replica answers correctly and uselessly with a world from a few hundred milliseconds ago.
Read routed to a replica
Own content read from the primary
Reference
The Whole Growth Path
Counterpoint
Most Apps Never Leave Stage Three
The ladder above is ten stages long because the story ends at a million users. Almost nothing ends at a million users. A single well-sized database with a cache in front of it comfortably serves a business with real revenue, and the honest version of most roadmaps stops somewhere around stage four and stays there for years.
Which makes the expensive mistake the opposite of the one people worry about. The risk is not failing to scale, it is paying a scaling cost before the break arrives: sharding a database that one machine would have carried, splitting a service to solve a problem a function call did not have, choosing a shard key before you know how the data is read. Those costs are permanent, and they are paid in every feature built afterwards.
The useful reading of this page is not top to bottom. It is: find the symptom you actually have, apply that stage, and leave the rest on the shelf until something breaks.
Seen In The Wild
Teams That Walked This Exact Ladder
Instagram reached tens of millions of users on sharded PostgreSQL, splitting rows across many logical shards keyed by user ID, with Memcached in front absorbing the read traffic. The order of the ladder above is close to the order they actually walked it.
Netflix does not rent a CDN for video, it ships its own Open Connect appliances into ISP data centres. That is stage four taken to its conclusion: the bytes start in the same building as the viewer.
Amazon RDS Multi-AZ keeps a synchronous standby in a second availability zone and promotes it automatically, typically in one to two minutes, which is the realistic shape of stage ten rather than the zero-downtime version people assume.
Nearly every photo or video you have uploaded anywhere went onto a queue: the response came back the moment the file landed, while resizing, transcoding, moderation and notifications ran behind it.
Key Points
What To Carry Forward
Every piece here is a response to a specific break, not a rung on a maturity ladder. The skill is matching the symptom to the fix, not knowing the list.
Reads and writes outgrow a database separately. Replicas fix reads, sharding fixes writes, and most applications need the first years before the second.
A cache and a CDN both store copies but solve different waits: one skips repeated work, the other shortens distance.
A queue does not make slow work fast, it moves it off the request path, which moves unfinished state into the interface.
Consistency is a per-feature decision, not a database setting. A like count can lag, somebody's own post cannot.
Every fix adds a cost, so system design is really choosing which cost your app can live with.
Common Mistakes
Where This Usually Goes Wrong
Sharding at 10,000 users because the design doc said a million, paying for cross-shard queries years before one machine would have struggled with the writes.
Scaling out before making the servers stateless, so sessions sit on local disk and a user who lands on a different server is silently signed out.
Adding a cache with no invalidation plan, then chasing a 'random' wrong value that is really one write path nobody remembered to update.
Reading from a replica immediately after writing to the primary, then filing the missing row as a database bug rather than the replication lag it is.
Enforcing a rate limit per server instead of per client across the fleet, so the real limit is the configured number times the number of servers.
Treating automatic failover as zero downtime, when a promotion still refuses writes for a minute or two and nothing in the stack retries them.
“I buried four bones in the garden against a winter that has never once arrived. One under the shed, one by the wet patch, two I can no longer locate at all. I have dug up the wrong one twice. The bowl, meanwhile, has been filled every single day of my life. I am told this is planning ahead. The bones are not better for being buried, and the garden is now full of holes.”
Try It Yourself
Find Your Own Stage
Further Reading
Where To Go Deeper
How Amazon's own teams build and operate the pieces on this ladder: timeouts and retries, load shedding, health checks, and what failover looks like when it is written by the people paged for it.
The standby-and-promotion model from stage ten, documented by the service most teams actually use for it, including the failover window you are buying.
Quiz Review
Check your understanding
Question 1 of 6
An app has 10,000 users and the database is pinned at 90% CPU. Why is sharding the wrong first move?
- ✓Because sharding is the fix for writes outgrowing one machine, and at 10,000 users almost nothing is write-bound. The load is reads, which a cache or a read replica absorbs without changing the data model at all. Sharding buys write capacity you will not use for years, and charges for it immediately: every query that spans users gets harder, joins and transactions stop being free, and a shard key chosen this early is chosen before you know the access patterns that have to live with it.