How to Design a Real-Time Chat Application (Case Study)

System design rounds often open with a deceptively simple prompt: "Design a chat application." The problem sounds contained, but it quietly touches real-time transport, distributed storage, delivery guarantees, and stateful servers all at once. Knowing how to design a chat application coherently, and defend your tradeoffs out loud, is what separates candidates who clear this round from those who stall halfway through.

Quick answer: To design a real-time chat system, you need (1) a persistent connection layer, almost always WebSockets, so messages flow without repeated HTTP round-trips; (2) a message storage layer that combines an append-only database for history with a cache for hot reads; (3) a fan-out service that routes messages to the right active connections; (4) a presence and delivery-receipt layer on top; and (5) a horizontal scaling strategy that moves connection state off any single server. The sections below cover each layer in the order you would walk through them in an interview.

Clarifying Requirements Before You Design a Chat App

Before drawing a single box, spend two or three minutes on scoping questions. Interviewers specifically watch for this step, and skipping it signals overconfidence.

Functional requirements to confirm:

• Is this one-on-one messaging, group chats, or both? Group chats raise fan-out complexity sharply.
• What media types are expected? Text only, or file attachments and images as well?
• Do messages need to persist across devices and sessions, or is this ephemeral?
• Do you need read receipts, typing indicators, and online presence?
• What scale are you designing for: tens of thousands of concurrent users or hundreds of millions?

Non-functional requirements worth stating explicitly:

• Low-latency delivery: users expect messages to appear in well under a second.
• At-least-once delivery: messages must not drop silently if a recipient is briefly offline.
• Eventual consistency is usually acceptable for chat; strict ordering within a single conversation thread is usually required.
• High availability matters more than strict consistency on the real-time delivery path.

A clear requirements pass frames every architectural decision that follows. If you skip it, the interviewer will surface a constraint you did not account for, and you will need to rework your design mid-explanation.

High-Level Architecture

Once scope is settled, sketch the major components before going deep into any one layer:

Client (mobile / web)
        |
[Load Balancer / API Gateway]
        |
[Chat Servers]  <-->  [Message Queue / Pub-Sub (Kafka or Redis Pub/Sub)]
        |
[Message DB (Cassandra / DynamoDB)]
[User / Presence Service]
[Notification Service (APNs / FCM for offline users)]
[Object Store for media (S3 / GCS)]

The critical insight here is that chat servers are stateful. Each server holds the open WebSocket connections for the clients currently attached to it. This means your routing layer needs to know which server holds a given user's connection so it can deliver incoming messages to the right place.

A message queue or pub-sub layer sits between servers for exactly this reason. Server A receives a message intended for User B, looks up that User B is connected to Server C, and publishes to a channel that Server C subscribes to. The chat servers never call each other directly. This keeps the fan-out logic decoupled and lets you scale individual components independently.

Real-Time Delivery: WebSockets vs. Long Polling

This is the transport decision interviewers most often ask you to defend explicitly.

Long polling works like this: the client opens an HTTP request, the server holds it open until it has something to send, then closes the connection. The client immediately opens a new one. It works with standard HTTP infrastructure but adds latency between the close and the reopen, and it wastes server threads on idle open connections.

WebSockets establish a single persistent, bidirectional TCP connection. Once open, either side can push data without a new handshake. For a real-time chat system design, WebSockets are the standard choice because:

• Latency is much lower once the connection is established.
• The server can push a message the instant it arrives, without waiting for the client to poll.
• A single connection handles both sending and receiving, simplifying the client implementation.

When long polling still appears: legacy environments where WebSocket support is blocked by a corporate proxy, or simple notification-only scenarios where true bidirectionality is unnecessary.

Server-Sent Events (SSE) are worth a brief mention. SSE is one-directional (server to client only), which makes it appropriate for notification feeds or live dashboards, but not for full-duplex chat.

Saying "WebSockets because they are better" is not enough. Show that you understand what you are giving up (sticky routing requirements, WebSocket upgrade support, harder load balancing) in exchange for what you gain.

Data Model and Message Storage

Message record

Each message needs at minimum these fields:

• message_id: a distributed unique ID such as Snowflake or ULID for total ordering
• conversation_id: the partitioning key for a one-on-one thread or group
• sender_id: the author
• content: the text payload, or a pointer to object storage for media files
• created_at: a Unix timestamp in milliseconds
• status: sent, delivered, or read

Why a wide-column store fits the access pattern

Chat messages are write-heavy and read in time-ordered ranges. Cassandra stores rows sorted by a clustering key, which makes paginating through a conversation's history cheap. Partitioning by conversation_id keeps a single conversation's data on the same node, avoiding scatter-gather for most reads.

DynamoDB is another common pick, with conversation_id as the partition key and message_id as the sort key. This gives you fast time-ordered range reads within a conversation at low operational overhead.

An RDBMS works fine at smaller scale. PostgreSQL with a well-indexed messages table is simpler to operate and reason about. Only reach for Cassandra or DynamoDB when write throughput or horizontal partitioning is a genuine constraint, not as a reflex.

Why not rely on timestamps alone for ordering

Two messages can arrive with the same millisecond timestamp across two different servers. Use a distributed ID scheme like Snowflake or ULID that embeds time but adds enough uniqueness bits to give a total order without central coordination.

Fan-Out, Presence, and Read Receipts

Fan-out

When a sender posts a message, every active recipient needs to receive it. For one-on-one chat this is straightforward. For a group with thousands of members, it is not.

Two strategies:

1. Fan-out on write: When the message arrives, immediately write it to each member's inbox. Recipients read only their own inbox. Simple for small groups, expensive for large ones because a single message triggers many writes.
2. Fan-out on read: Store the message once. Recipients fetch it when they next open the conversation. Scales better for very large groups but adds read latency and complexity.

Most production systems use a hybrid: fan-out on write for small groups (under a few hundred members), fan-out on read for groups above a certain size threshold. Name this tradeoff in the interview; it shows you have thought beyond the happy path.

Presence

Presence is a separate service, not part of the message delivery path. A common lightweight approach:

• When a client opens a WebSocket, it sends a heartbeat every 30 seconds.
• The presence service stores {user_id: last_seen_at} in Redis.
• If last_seen_at is older than 60 seconds, the user is considered offline.
• On an intentional disconnect, the client sends an explicit offline event when possible.

Avoid computing presence on every message delivery. Cache the state and invalidate it on heartbeat timeout.

Read receipts

Read receipts require the receiving client to acknowledge delivery. When a client renders a message, it sends an ACK back over the WebSocket. The server updates the message status to read in the database and notifies the sender's connected server via the pub-sub layer.

At scale, these acknowledgment writes can be high-volume. Batch them on the server side (process every 500 milliseconds rather than per-message) to reduce write pressure without noticeably degrading the user experience.

Scaling and Bottlenecks

Chat servers are stateful, which complicates horizontal scaling. Adding a new server gives you capacity, but no existing connections live there. A service registry or consistent-hashing approach must track which server holds each active connection, so the routing and fan-out layers can forward messages correctly.

The message queue is your decoupling point. Chat servers publish to topics and subscribe to topics relevant to their connected users. They do not call each other directly. Kafka works well at very high throughput and provides message replay and durability, useful if a consumer goes down briefly. Redis Pub/Sub is simpler and lower-latency but does not persist messages for offline consumers; you handle offline delivery separately.

Common database bottlenecks:

• Write throughput: batch writes where at-least-once delivery allows it.
• Hot conversations (viral group chats or channels with many concurrent writers): a single partition can become a write hotspot. Use read replicas and cache the most recent 50 to 100 messages in Redis for fast retrieval.
• Old messages: move them to cheaper object storage after a retention window, with a pointer in the primary database for historical lookups.

Offline delivery: if a recipient is not connected, queue the message and trigger a push notification via APNs or FCM. This path belongs in a separate notification service, not the chat server itself. Conflating them makes the chat server harder to scale and test.

Tradeoffs to surface explicitly:

• Consistency vs. availability: chat systems usually prefer availability on the real-time path and accept that a message might appear slightly out of order in rare edge cases.
• Fan-out cost vs. read cost: fan-out on write is expensive for large groups; fan-out on read shifts cost to readers and adds latency.
• WebSocket stickiness: sticky routing means a server crash disrupts all its connected clients until they reconnect. Design robust client-side reconnection logic with exponential backoff.

Frequently Asked Questions

How do I handle message ordering in a distributed system? Use a monotonically increasing ID generated by a scheme like Snowflake rather than relying on server timestamps alone. Within a conversation, sort by this ID. The scheme embeds a timestamp but adds enough uniqueness bits to give a total order that survives clock skew across multiple servers.

When should I use Kafka vs. Redis Pub/Sub for the fan-out layer? Use Kafka when you need replay, guaranteed delivery to temporarily offline consumers, or high throughput with audit or replay requirements. Use Redis Pub/Sub when you want low latency and operational simplicity, and you can accept that messages not received by a subscriber in the moment are lost, handling offline delivery separately through a durable queue.

How do you prevent messages from being lost if a chat server crashes? Acknowledge a message to the sender only after it is durably written to the message store. The client retries on reconnect using the message ID as an idempotency key to prevent duplicates. This gives at-least-once delivery. De-duplication on the read path handles the rare double-delivery case.

Is a relational database ever the right choice for chat storage? Yes, for moderate scale. If the system serves tens of thousands of daily active users rather than hundreds of millions, PostgreSQL with a well-indexed messages table is simpler to operate, easier to query for analytics, and straightforward to migrate. Only move to a wide-column store when write throughput or partition-level horizontal scaling is a real constraint, not a theoretical one.

How do you handle end-to-end encryption in your design? The server stores only ciphertext; cryptographic keys live on the client. This means the server cannot search message content, cannot apply server-side moderation to encrypted messages, and cannot recover messages if a client loses the key. Surface these tradeoffs explicitly rather than treating encryption as a checkbox addition. Interviewers want to see that you understand the operational implications.

Practising This Case Study on Rehurz

System design is a spoken discipline. You need to walk through an architecture clearly, field probes like "what happens when one chat server crashes?" or "how does your fan-out strategy change at 100 million users?", and hold your reasoning together under real pressure. Reading a case study is not the same as delivering it under interview conditions.

On Rehurz, system design is practised as a live spoken round: you talk through your design and defend tradeoffs under adaptive cross-questioning, with "System Design" tracked as a dimension on your technical scorecard. The interviewer persona matches the seniority level you are targeting, so the pressure reflects what you would actually face. After the session, you get per-topic feedback on where your explanation was solid and where it had gaps you need to close.

To run through this case study out loud before your next real interview, book a live technical interview prep session. Your first session is free, and no card is required. Get started at Rehurz.

---

Chat application design earns its place as a system design staple because it forces you to connect every layer, from a single TCP connection to a horizontally partitioned database, into a coherent whole. Walk the requirements clearly, defend your real-time transport choice, justify your storage model, and surface the fan-out and scaling tradeoffs explicitly. That is what gets you the pass.