Realtime chat: architecture and what it actually costs
The short answer
Getting messages to move between two connected browsers takes an afternoon. What takes months is everything around it: guaranteed delivery when a connection drops mid-send, ordering, offline queues, read receipts, presence, and scaling beyond one server, because WebSocket connections are stateful and do not load-balance the way HTTP requests do. Build the delivery guarantees first. The interface is the easy half.
Why chat is harder than it looks
A demo where two browsers exchange messages proves almost nothing about production. Real chat has to answer questions the demo never asks.
What happens when the sender's connection drops after the message left the client but before the server acknowledged it? Does the recipient get it twice, or not at all? If two messages are sent a millisecond apart from different devices, which one appears first, and does everyone looking at the conversation agree on the order?
These are distributed systems problems wearing a friendly interface. That is why chat projects that look 80% done after two weeks are frequently 30% done, and the remaining work is all in the invisible half.
The delivery problem
Every serious chat system needs three states per message: sent, meaning the client handed it over; delivered, meaning the server persisted it and the recipient's device received it; and read.
The design decision that makes this workable is that the client generates the message ID, not the server. If the client retries after a dropped connection, the server recognises the duplicate and does not store it twice. Without client-generated IDs, retries create duplicates, and duplicates are the bug users notice and complain about first.
The second rule is to persist before acknowledging. A message acknowledged but not written is a message that vanishes when the process restarts, and it vanishes silently, which is the worst kind of data loss because nobody reports it until much later.
Ordering, offline and reconnection
Ordering
Timestamps from client devices cannot be trusted because clocks drift and users change them. The practical approach is a per-conversation sequence number assigned server-side. Everyone reading that conversation sees the same order, which is what users actually care about. Global ordering across all conversations is much harder and almost never needed.
Offline queue
Queue on the server, not the client. Undelivered messages wait server-side until the recipient reconnects, which is the only approach that survives a device being switched off for three days.
Sync on reconnect
Sync by last-seen sequence number, not by timestamp. Give me everything after message 4,182 in this conversation is precise. Give me everything since 10:42 is not, and it produces gaps and duplicates at the edges.
The reconnect storm
If your server restarts, every client reconnects at once. Without jittered backoff, the reconnection traffic takes the server down again, and you get a cycle that looks like a much worse outage than the original restart.
Scaling past one server
This is where most chat projects hit a wall, and it arrives suddenly rather than gradually.
HTTP is stateless, so load balancing is straightforward. A WebSocket connection is a long-lived stateful connection pinned to one server. If user A is connected to server 1 and user B to server 2, server 1 has no direct way to reach user B, and the message simply does not arrive.
The standard solution is a publish-subscribe layer between servers, commonly Redis, so a message received by any server is published to a channel that the server holding the recipient's connection is subscribed to. Two related requirements follow: sticky sessions at the load balancer so a reconnecting client returns to a server that can serve it, and presence tracking held in shared storage rather than in one server's memory.
Presence and typing indicators: cheap to build, expensive to run
- A typing indicator that fires an event per keystroke generates enormous traffic for very little value. Throttle it: one event when typing starts, one when it stops, with a timeout.
- Naive who-is-online checks against every connected user do not scale. Store presence in shared memory with a short TTL that clients refresh.
- Accept that presence will be a few seconds stale. Users do not notice, and the alternative costs far more than the accuracy is worth.
- Read receipts multiply write volume. One message read by twenty group members is twenty writes if implemented naively. Batch them.
What it costs to run
Modest, compared to video. Chat is text and small payloads, and the connection overhead is the dominant cost rather than the data.
The real costs sit elsewhere: the database, because message history grows forever unless you plan retention; file and image storage if you support attachments; and the pub-sub layer once you are on more than one server.
Two decisions belong early. How long messages are retained, and where attachments live. Storing images in the database is the mistake that turns a cheap system into an expensive one, and it is painful to reverse once there are millions of rows. Under India's DPDP framework, retention is also a compliance decision rather than only a cost one, since personal data should be erased once the purpose it was collected for is served.
From our own builds
The realtime chat on AlgoTipp, a platform for an Australian client, runs on Socket.io alongside self-built WebRTC video and voice calling in the same product.
The hard part was not the messaging. It was reconnection and delivery behaviour, which is where a chat feature either feels solid or feels broken. Users forgive a slow message. They do not forgive a missing one, and they remember it for a long time.
The design decision worth passing on is the one about message IDs. Generating them on the client rather than the server is a small change that makes retries safe, and retrying safely is what lets you be aggressive about reconnection without creating duplicates.
Frequently asked questions
Building chat into your product?
We have built production realtime chat with Socket.io alongside self-hosted video calling, including the reconnection and delivery behaviour that decides whether it feels reliable.