Capstone·19 min read

How a Backend Is Built Around One Message

Ten backend pieces, one chat message, and the exact thing each piece is the only one that can do.

Big Idea

Servers, APIs, databases, caches, WebSockets, queues, load balancers, containers, architecture, observability. Drawn as a stack they look like the parts list for a real backend. They are not a list, they are ten answers to ten different questions, and one message moving from one phone to another asks all ten in order. Nadia types running ten minutes late and hits send. Everything below is what has to exist for Theo to read it, and nothing below exists until the version before it runs out.

The Shape Of It

Why The Phones Can’t Just Talk To Each Other

The obvious design is the one that does not work: Nadia’s phone handing the message straight to Theo’s. Phones make terrible delivery targets. Neither one has an address the other could dial, both get a new one every time they move between wifi and cellular, and Theo’s is locked in a pocket most of the day.

So neither phone tries. They both talk to one machine that is none of those things, and that machine is the entire backend on day one.

At two users that single box is the whole backend, and it is the right one: one process to deploy, one place to look when something breaks, and no second machine that can disagree with it. Everything else on this page is what gets added when that stops being enough, in the order it stops being enough.

Interactive

Watch The Backend Grow, One Piece At A Time

One canvas, ten pieces. Press play, or step through it. Nothing ever moves, it only arrives, and the badge on each box is the piece that put it there.

Piece 1ServerA message with nowhere to go

One always-on process, with both phones talking to it instead of to each other.

Piece ten is what a backend diagram usually is: not a design, but a record of ten questions somebody had to answer, drawn all at once so the order is no longer visible. The ten cards below are that order.

The Walkthrough

Ten Pieces, And What Each One Charges

ServerA message with nowhere to go

What it can’t do yet

Nadia's phone holds the message and Theo's phone has to receive it. A phone is the worst delivery target there is: it sits behind carrier NAT with no address anyone can reach, it changes IP the moment it changes network, and it is asleep most of the day.

The piece

A server: a process on a machine that stays on, listening on a port, accepting requests and returning responses. Both phones talk to it and never to each other. Your backend code runs there, in Go, Java, Python, Node or Elixir, and the choice of language changes nothing about this shape.

What it costs

Something now has to stay up, and it is yours: deploys, patches, restarts, and a bill per hour whether anyone sends a message or not. On day one it is also a single point of failure, and every piece below exists because this one process cannot do everything well.

↗Deep dive: The Lifecycle of a Software Request

Message queues, containers and observability do not have a page of their own here yet, so pieces six, eight and ten point at the nearest concept rather than a dedicated one.

Worked Example

Asking, Versus Being Told

Theo’s app has to find out the message exists. Both versions below work. The difference is who speaks first, and that one change removes the empty round trips and the delay at the same time.

Polling: the client asks, over and over

WebSocket: the server speaks first

What the socket costs is on the server, not the phone. That connection is held by one specific process, so an app server is no longer a thing you can restart whenever you like, and a deploy drops every connection it was holding. Which is the next two problems on this page.

Worked Example

The Locked Phone, And Whose Outage It Is

There is no socket, because Theo’s app is not running. Reaching him means calling Apple or Google, which means a request to a company you do not operate. Leaving that call on the request path hands them your send button.

Inline: Nadia waits for Apple

Queued: the slow dependency moves off the request

The queue did not make the push faster, it changed what a slow push means: a retry instead of a failed send. The bill arrives in two places. The interface now has to distinguish sent from delivered, because they happen at different times. And queues are at-least-once, so that worker will eventually run twice on the same job, which has to be harmless rather than a duplicate notification at 3am.

Worked Example

Ten Servers, And The One Holding The Socket

One machine runs out of memory and file descriptors long before it runs out of users, because every socket is held for hours rather than milliseconds. Ten machines fix that and break delivery: a connection lives on exactly one process, and the server handling Nadia’s send is almost never the one holding Theo’s socket.

The load balancer does not help here. It decides where new connections go, and has no way to write into one that is already open somewhere else.

Notice what grows. The fan-out scales with the size of your fleet, not with the number of recipients, so adding servers makes each message cost more internally. Teams past a certain size stop broadcasting and keep a registry of which server holds which connection, which is faster and is now one more piece of state that can be wrong.

Worked Example

Thirty Seconds Late, And Every Dashboard Green

Messages are arriving half a minute after they were sent. API p99 is 60ms, the error rate is zero, every service reports healthy, and all of that is true. Per-service metrics measure components answering for themselves, and nothing was slow inside a component. The message was waiting between them, and a queue is not a service with a latency number.

A trace follows the one message across every hop and puts the time where it actually went:

Every unit of real work on this trace adds up to under a second. The other 29.4 seconds is a wait, and the fix is not faster code, it is more workers or fewer jobs. Two things would have caught it without a trace, and neither is a latency percentile: queue depth, the one metric that describes a backlog, and a request ID carried through every log line so the hops can be lined up after the fact. Instrument the gaps between the boxes, not just the boxes.

Recap

The Whole Path

Read straight down. Each line is what that piece does, and the line between is what it still could not do — which is the piece underneath it.

1Serverruns the code

but it will take literally anything the phone sends

2APIdefines the requests

but a restart wipes every conversation

3Databasesaves the message

but the same handful of reads keeps repeating

4Cachespeeds up reads

but nobody has told Theo anything yet

5WebSocketsdeliver it live

but his phone is locked, and Apple is slow to answer

6Message queuehandles slow work

but one machine cannot hold every connection

7Load balancersspread traffic

but ten machines drift apart from each other

8Containersrun it everywhere

but one codebase is now doing four different jobs

9Architecturesplits it up

and now nobody can say which piece is slow

10Observabilityshows what’s happening

A backend grows one piece at a time. Each new piece gets added right when the last one hits its limit.

Key Points

What To Carry Forward

✓

Each piece answers exactly one question: where does the code run, what may be asked, what survives a restart, what repeats, who gets told, what can wait, which machine, which environment, which codebase, where did the time go.

✓

A database makes a message durable and a cache makes a read cheap. Neither of them delivers anything — delivery is the socket's job, and that separation is the whole reason both exist.

✓

Polling and WebSockets both work. One pays per empty check, the other pays by holding state, and that state is what stops an app server from being freely restartable.

✓

A queue does not make the push notification faster. It moves it off the path the sender is waiting on, and buys retries, duplicates, and a message that is 'sent' before it has been sent.

✓

The moment there are two servers, delivery becomes routing: a socket lives on exactly one process, so every other server needs a channel to reach it.

✓

Observability is the only piece that adds no capability at all. It is what makes the other nine debuggable, which is why it is the one that should never be last.

Quiz Review

Check your understanding

Question 1 of 6

A chat app keeps messages in a list in memory and delivers them over WebSockets. Everything passes in testing. What does the first real deploy break, and which piece fixes which half of it?

✓Both halves of the same restart. The list in memory is gone, which deletes every conversation on the service, and that is what a database fixes: the data outlives the process. The open sockets are also gone, and no storage layer helps there, because a socket cannot be persisted. That half is fixed on the edges of the restart instead: drain connections before the process stops, and have the client reconnect with backoff and ask for anything it missed while it was disconnected. Durability and connectivity look like one bug here and are two different pieces.