SLIDE 43: Section Intro, Data Flow
Section 5: Data Flow
Section 4 defined the components. This section defines how data moves between them. A system architecture diagram with 45 boxes is meaningless until you can trace a request through it, name every service it touches, and explain whether each hop is synchronous or asynchronous. Data flow is where the architecture comes alive.
In a ride-sharing platform, there are three primary data flows that exercise fundamentally different patterns: the ride request flow (synchronous orchestration under a latency budget), the location update flow (high-throughput streaming writes), and the trip completion flow (synchronous finalization followed by asynchronous fan-out). Understanding these three flows is enough to explain how the entire system works.
What We’ll Cover:
- The three primary data flows: ride request, location update, trip completion
- The trip state machine: states, transitions, cancellation, and failure
- Synchronous vs asynchronous: which operations and why
- Event-driven processing and dead letter queues
- Data ownership across service boundaries
- Race conditions in real-time matching
This section connects backward to the architecture from Section 4 and forward to every deep-dive section that follows. When we discuss geospatial design in Section 6, matching in Section 7, or database selection in Section 8, we will reference the flows defined here.
SLIDE 44: The Three Primary Data Flows
Three Flows, Three Patterns
Every operation in the system belongs to one of three primary data flows. Each flow exercises a different architectural pattern, has a different latency profile, and stresses different components.
Flow 1: Ride Request
Rider requests a ride. The system finds nearby drivers, scores them, dispatches an offer, and waits for acceptance. This is a synchronous, latency-sensitive orchestration. The entire flow must complete in under 10 seconds, and every service in the chain is on the critical path. If any service is slow, the rider waits.
Flow 2: Location Update
Drivers continuously stream GPS positions every 3-4 seconds. Each update writes to the spatial index and asynchronously appends to location history. This is a high-throughput streaming write. At 1.25 million updates per second, this is the highest-volume data flow in the system by a factor of 5,000x over ride requests. It runs continuously, independent of ride requests.
Flow 3: Trip Completion
Driver marks the trip complete. The system calculates the fare, processes payment, updates trip state, and notifies both users. The synchronous part (fare + payment) must finish quickly so the driver can accept their next ride. The asynchronous part (analytics, receipt email, rating prompt) fans out to downstream services without blocking.
The rest of this section traces each flow step by step, then examines the cross-cutting concerns that apply to all three: the state machine, sync vs async decisions, event processing, data ownership, and race conditions.
SLIDE 45: Ride Request Flow, Step by Step
The Critical Path: Request to Acceptance
This is the flow the rider cares about most, and the one the interviewer will ask you to trace. Every step is synchronous and latency-critical.
Step 1: Rider sends request
The rider taps “Request Ride” with pickup location, dropoff location, and vehicle type. The request hits the API Gateway, which validates the JWT token, checks the rate limiter, and routes to the Trip Service.
Step 2: Trip Service creates the trip
Trip Service creates a trip record with status “requested.” This is the single source of truth for this ride’s lifecycle. The Trip Service then calls the Matching Service.
Step 3: Matching Service queries for nearby drivers
Matching Service calls Location Service, which queries the spatial index (Redis GEOSEARCH) for drivers within a configurable radius (typically 5 km). The spatial index returns a list of candidate drivers with their positions.
Step 4: Matching Service scores and ranks
Candidates are filtered by eligibility (vehicle type, active status) and scored on multiple factors: ETA, driver rating, acceptance rate, heading direction. The top candidate is selected.
Step 5: Dispatch Service sends the offer
Dispatch Service pushes the ride offer to the selected driver via WebSocket. The driver sees pickup location, estimated trip distance, estimated fare, and pickup ETA. A 15-second countdown begins.
Step 6: Driver responds
If the driver accepts, Dispatch notifies Trip Service, which updates the trip to “matched” and pushes confirmation to the rider. If the driver declines or the timer expires, Dispatch cascades to the next candidate on the ranked list. After 3-5 failed cascades, Matching re-queries the spatial index with an expanded radius.
End-to-end target: under 10 seconds from request to acceptance.
SLIDE 45B: Ride Request Flow, Diagram
Tracing the Request Through Services
Every arrow in this diagram is a synchronous call. The rider is waiting for a response at each step. This means every service on this path must meet its latency budget, or the entire flow exceeds the 10-second target.
Latency Budget Breakdown
API Gateway (auth + rate limit): ~50ms
Trip Service (create record): ~100ms
Matching (spatial query + scoring): ~200-500ms
Dispatch (WebSocket push to driver): ~100ms
Driver decision window: up to 15,000ms
Trip Service (update state + notify): ~100ms
The dominant factor is the driver’s decision time. The system’s processing takes roughly 500-900 milliseconds. The remaining budget goes to the driver. This is why the acceptance window is the primary tuning parameter for matching latency, not the backend processing time.
What If No Driver Accepts?
After 3-5 cascade attempts (each adding up to 15 seconds), the system re-queries the spatial index with a wider radius. If still no match after a configurable timeout (typically 60-90 seconds), the trip transitions to “failed” and the rider sees “no drivers available.” The system never hangs indefinitely. Every path has a terminal state.
Interview Tip:
When tracing this flow in an interview, call out the sync/async boundary explicitly: “Everything from request to driver acceptance is synchronous. The rider is waiting. After acceptance, downstream processing like analytics and notification fans out asynchronously via Kafka.” This shows you understand the design decision, not just the sequence.
SLIDE 46: Location Update Flow, Step by Step
The Highest-Volume Flow in the System
This flow runs continuously, independent of ride requests. Every active driver streams GPS positions every 3-4 seconds. At 5 million active drivers, this produces 1.25 million updates per second. For comparison, ride requests peak at roughly 700 per second. The location update flow is 1,800x the volume of the ride request flow.
The Write Path
A driver’s phone captures GPS coordinates and sends them to the WebSocket Gateway over the persistent connection established when the driver went online. The WebSocket Gateway forwards the update to the Location Service.
The Location Service does two things with each update:
First, it writes to the spatial index (Redis GEOADD). This overwrites the driver’s previous position. The spatial index always reflects the most recent known position. There is no history here, only current state.
Second, it asynchronously appends the position to the location history database (time-series DB like TimescaleDB or Cassandra). This is a fire-and-forget write. If the history write is slow or fails, the spatial index is unaffected. The two write paths are completely decoupled.
The Read Path
When a rider requests a ride (or opens the app and sees nearby drivers), the request goes to the Location Service, which executes a GEOSEARCH against the spatial index. This returns all drivers within a specified radius, sorted by distance. The read path never touches the location history database. It reads only from the in-memory spatial index.
Key Design Decision
The write path and read path share the same data store (Redis) but are logically independent. High write volume (1.25M/sec) must not block or degrade read performance (sub-200ms queries). Redis handles this because writes are O(log(N)) GEOADD operations and reads are O(N+log(M)) GEOSEARCH operations, and both are executed in memory.
SLIDE 46B: Location Update Flow, Write Path vs Read Path
Two Paths, One Index
The write path is a continuous stream. The read path is bursty. Both converge on the same spatial index, but they have fundamentally different performance characteristics.
Write Path Characteristics
Sustained high throughput: 1.25 million updates per second, evenly distributed across time. Each write is small (driver_id, latitude, longitude). Writes do not need acknowledgment back to the driver. The driver sends GPS and moves on. If one update is lost, the next one arrives 4 seconds later. This tolerance for occasional loss is what makes the write path simpler than it looks at first glance.
Read Path Characteristics
Bursty, correlated with ride requests: a spike in ride requests means a spike in nearby queries. Each read returns a list of nearby drivers with positions. Read latency is critical because the rider is waiting and the matching pipeline is blocked. Reads must complete in under 200 milliseconds.
Why the Separation Matters
If the spatial index is overwhelmed by writes, reads slow down and matching latency increases. This is why the Location Service design must ensure that write ingestion does not starve read queries. In Redis, this is handled naturally because Redis is single-threaded and operations are fast enough that interleaving writes and reads does not cause contention at this scale. For larger deployments, geographic partitioning (one Redis instance per city) ensures each instance handles a manageable fraction of the global write volume.
This design is covered in depth in Section 6 (Geospatial Systems) for the spatial index and Section 8 (Database Design) for the history storage.
SLIDE 47: Trip Completion Flow, Step by Step
Synchronous Finalization, Then Asynchronous Fan-Out
When the driver taps “Complete Trip,” the system must finalize the trip quickly so the driver can accept their next ride. Speed matters here because every second the driver spends on the completion screen is a second they are not earning.
Step 1: Driver marks trip complete
The Driver App sends a completion signal via WebSocket. The WebSocket Gateway routes it to the Trip Service.
Step 2: Trip Service updates state
Trip Service transitions the trip from “in_progress” to “completed.” This transition is validated against the state machine: you cannot complete a trip that was never started, and you cannot complete a trip that is already completed. The transition is idempotent.
Step 3: Pricing Service calculates the fare
Trip Service calls Pricing Service with the actual distance driven (from location history) and actual time elapsed. Pricing Service applies the fare formula: base_fare + (distance x distance_rate) + (time x time_rate), multiplied by the surge multiplier that was locked at trip start. The calculated fare is returned to Trip Service.
Step 4: Payment Service captures the charge
Trip Service calls Payment Service to capture the pre-authorized amount (or the actual fare if it differs). Payment Service communicates with the external payment gateway (Stripe, Braintree) to finalize the charge. This must succeed or be retried. The rider must be charged exactly once.
Step 5: Both apps updated
Trip Service pushes the fare breakdown to the rider app and the earnings update to the driver app. The driver is now available for the next dispatch.
Step 6: Asynchronous fan-out
Trip Service emits a “trip_completed” event to Kafka. Downstream consumers process it independently: Analytics Service records metrics, Notification Service sends the receipt email, Rating Service prompts both users for ratings. None of these block the driver or rider.
SLIDE 48: Trip State Machine, States and Transitions
The Trip Lifecycle as a Finite State Machine
Every trip in the system is a state machine. At any point, a trip is in exactly one state, and only specific transitions are valid. Enforcing valid transitions prevents impossible states (a trip that is simultaneously “completed” and “in_progress”) and ensures every trip follows a predictable lifecycle.
States
The happy path moves through: requested (rider submitted, matching in progress), matched (driver assigned), driver_en_route (driver heading to pickup), driver_arrived (driver at pickup location), trip_in_progress (rider in the car, metering active), and completed (trip finished, fare calculated, payment captured).
Transitions with Triggers
Each transition has a specific trigger. “Requested” to “matched” is triggered by the Dispatch Service confirming driver acceptance. “Matched” to “driver_en_route” is automatic once the match is confirmed. “Driver_arrived” to “trip_in_progress” is triggered by the driver tapping “Start Trip.” “Trip_in_progress” to “completed” is triggered by the driver tapping “Complete Trip.”
Why a State Machine Matters
Without a formal state machine, you risk invalid transitions. What happens if a network glitch causes a “complete trip” signal to arrive twice? The state machine rejects the second transition because the trip is already in “completed.” What if a “start trip” signal arrives for a trip that was cancelled? Rejected, because “cancelled” has no outbound transitions to “in_progress.” The state machine is the last line of defense against corrupt trip records.
Every transition is logged to a trip_state_history table for auditability and debugging. This connects to Section 8 (Database Design) where we define the schema.
SLIDE 48B: Trip State Machine, Cancellation and Failure Handling
Cancellation Can Happen at Almost Every State
The trip state machine has a “cancelled” state reachable from every state before “trip_in_progress.” Once the rider is in the car, cancellation is no longer available through the app (the driver must complete the trip or report an issue).
Rider Cancellation Rules
Before matching: free cancellation. The trip simply moves to “cancelled” with no charge. This is the most common cancellation: the rider changed their mind while waiting.
After matching but before pickup: cancellation fee may apply. The driver has already started driving toward the rider. The fee compensates the driver for wasted time and fuel. The fee amount is configurable per market.
Driver Cancellation
A driver can cancel (decline to continue) after accepting a match. This affects the driver’s acceptance rate metric, which feeds into future matching scores. Frequent cancellations lower the driver’s priority in the scoring pipeline.
When a driver cancels after accepting, the trip returns to “requested” and the Matching Service re-queries the spatial index for a new driver. The rider sees “finding a new driver” rather than an error.
System Failure Handling
If the Matching Service cannot find a driver within the timeout window (60-90 seconds), the trip transitions to “failed.” The rider is notified with “no drivers available” and can retry.
If a driver accepts but the confirmation never reaches the Trip Service (network partition), the Trip Service times out and re-dispatches. The original driver’s lock expires after the timeout, releasing them back to the available pool.
Every transition must be idempotent. A retry of any state change must produce the same result as the original. This is enforced by checking the current state before applying the transition: if the trip is already in the target state, the operation returns success without modifying anything.
SLIDE 49: Synchronous vs Asynchronous, Which Operations and Why
The Decision Framework
Every inter-service call in the system is either synchronous (the caller waits for a response) or asynchronous (the caller fires and continues). The choice is not arbitrary. It is determined by one question: is the user waiting for this result?
If yes, the call is synchronous. The rider is waiting for a match. The matching pipeline is synchronous. The rider is waiting for the fare at trip completion. Pricing is synchronous.
If no, the call is asynchronous. The analytics pipeline does not need to finish before the rider sees their fare. The receipt email does not need to send before the driver can accept the next ride. These are asynchronous, processed via Kafka events.
The Boundary Between Sync and Async
The boundary runs through the Trip Service. Everything upstream of trip state finalization is synchronous. Everything downstream (events emitted after a state change) is asynchronous. The Trip Service is the sync/async boundary. This is a deliberate design choice: it keeps the critical path fast and pushes non-critical work to background processing.
SLIDE 50: Event-Driven Processing, What Happens After State Changes
The Kafka Event Bus
Every trip state transition emits an event to Kafka. These events are the mechanism by which downstream services learn about what happened without being coupled to the Trip Service.
When a trip moves to “completed,” the Trip Service emits a “trip_completed” event containing the trip ID, rider ID, driver ID, fare amount, distance, duration, and timestamps. Three independent consumers process this event:
Payment Service captures the pre-authorized charge. Analytics Service records the trip for reporting and ML model training. Notification Service sends the receipt to the rider and the earnings update to the driver.
Event-Carried State Transfer
The event carries all the data the consumer needs. The “trip_completed” event includes the fare amount so that Payment Service does not need to call Pricing Service. The event includes the rider email so Notification Service does not need to call User Service. This eliminates synchronous dependencies between downstream services and makes the system more resilient: if User Service is down, Notification Service can still send the receipt because it has the email from the event payload.
Key Events in the System
The major events are: ride_requested (triggers analytics), ride_matched (triggers driver ETA tracking), ride_completed (triggers payment, analytics, notifications, rating prompt), ride_cancelled (triggers refund if applicable, analytics), and payment_processed (triggers driver settlement pipeline).
Each consumer processes events at its own pace. Analytics might lag by seconds. Settlement might lag by hours. This is acceptable because none of these consumers are on the critical path.
SLIDE 50B: Dead Letter Queue for Failed Events
Failed Events Must Not Be Silently Dropped
When a Kafka consumer fails to process an event, the event must not disappear. A dropped “trip_completed” event means a rider is never charged, a driver is never paid, and the trip never appears in analytics. Silent failure is worse than visible failure because no one knows to investigate.
The Dead Letter Queue Pattern
After a configurable number of retry attempts (typically 3-5 with exponential backoff), a failed event is routed to a Dead Letter Queue (DLQ). The DLQ is a separate Kafka topic that holds failed events for investigation.
An operations team monitors the DLQ. For each failed event, they investigate the cause (schema mismatch, downstream service timeout, corrupt payload, unexpected null field), fix the issue, and replay the event from the DLQ back into the original topic.
Common Failure Causes
Schema evolution is the most frequent cause: the Trip Service adds a new field to the event, but the consumer has not been updated to handle it. The consumer fails to deserialize the event and routes it to DLQ.
Downstream timeout is second: Payment Service tries to capture the charge, the external payment gateway times out, all retries fail, the event lands in DLQ.
Corrupt or unexpected data: a null field where a value was expected, a negative fare amount, a trip ID that does not exist in the consumer’s database.
Interview Tip:
Mentioning DLQ in a system design interview demonstrates production awareness. Most candidates describe the happy path. Mentioning “failed events go to a dead letter queue for investigation and replay” shows you have operated real systems, not just designed them on whiteboards.
SLIDE 51: Data Ownership Across Service Boundaries
Each Service Owns Its Data
This is the database-per-service pattern. No service directly queries another service’s database. Trip Service owns trip records. Location Service owns positions. Payment Service owns transactions. If Matching Service needs trip data, it calls Trip Service’s API or consumes Trip Service’s events. It never runs a SQL query against Trip Service’s database.
Why This Matters
If Matching Service directly queries the trips database, a schema change in the trips table can break Matching Service. The two services are now coupled at the database level, and you cannot deploy or scale them independently. Database-per-service eliminates this coupling.
How Services Access Each Other’s Data
Synchronous access: service-to-service API calls (gRPC or REST). Used when the caller needs a response before continuing. Example: Trip Service calls Pricing Service to get the fare.
Asynchronous access: Kafka events. Used when the caller does not need an immediate response. Example: Analytics Service consumes “trip_completed” events to build dashboards.
Cached access: a service maintains a local read-only copy of another service’s data, refreshed via events. Example: Matching Service caches driver ratings locally (refreshed via “rating_updated” events) rather than calling User Service on every match.
Anti-Pattern: The Shared Database
A monolithic database shared by all services is the fastest path to an un-scalable system. Every service competes for connections, every schema change requires coordinating all teams, and a single slow query from one service degrades performance for all others. Database-per-service is the default in any microservice architecture. This connects to Section 8 (Database Design) where we define the specific database technology for each service.
SLIDE 52: Race Conditions in Real-Time Matching
Concurrency Creates Conflicts
In a system processing hundreds of ride requests per second, with millions of drivers updating positions, concurrent operations create conflicts. These are not edge cases. They happen thousands of times per day at scale.
Race Condition 1: Two Riders Matched to the Same Driver
Two ride requests arrive within milliseconds. Both query the spatial index and find driver_A as the nearest eligible driver. Both Matching Service instances attempt to dispatch to driver_A.
Solution: optimistic locking. The first dispatch attempt marks driver_A as “in_dispatch” with a compare-and-swap operation. The second attempt finds the lock and fails fast, falling back to the next candidate. The lock has a short TTL (30 seconds) so it auto-releases if the dispatch flow crashes before cleaning up.
Race Condition 2: Driver Goes Offline During Matching
The spatial index shows driver_B at position (40.7, -73.9). The Matching Service selects driver_B. Between selection and dispatch, driver_B closes the app and goes offline. The WebSocket connection drops, but the spatial index has not yet expired the position (TTL has not elapsed).
Solution: the Dispatch Service checks the WebSocket connection status before sending the offer. If the connection is dead, skip this driver and cascade to the next candidate. Additionally, the TTL on position records (15-20 seconds) ensures stale entries expire quickly.
Race Condition 3: Network Partition During Acceptance
Driver_C receives the dispatch offer and taps “Accept.” The acceptance signal is sent, but a network partition prevents it from reaching the Trip Service. The 15-second timer expires in Trip Service, which cascades to the next driver. Driver_C believes they have the ride. The next driver also accepts.
Solution: the Trip Service uses an idempotent state transition. Only the first valid “matched” transition succeeds. The second acceptance is rejected because the trip is already in “matched” state with a different driver. Driver_C receives a “ride no longer available” notification.
SLIDE 53: Worked Example, Trace a Complete Ride from Request to Payment
One Ride, End to End, with Timestamps
Every line in this trace maps to a service defined in Section 4, a data flow defined in this section, and a database decision we will make in Section 8.
SLIDE 54: Wrong vs Right, Data Flow Presentation
Common Mistake 1: Describing Flows Without Naming Services
Wrong: “The system processes the request and finds a driver.”
What system? What service? “Processes” how? This tells the interviewer nothing about your architecture. It is indistinguishable from someone who has never designed a distributed system.
Better: “The Trip Service receives the request, calls the Matching Service, which queries the Location Service’s spatial index via Redis GEOSEARCH, filters and scores the candidates, and the Dispatch Service sends the offer to the top-ranked driver via WebSocket.”
Common Mistake 2: Only Showing the Happy Path
Wrong: tracing the ride request from request to acceptance with no mention of what happens when things go wrong.
Better: “If the driver declines, the Dispatch Service cascades to the next candidate. After 3-5 failed attempts, Matching re-queries with a wider radius. If no match is found within 90 seconds, the trip transitions to ‘failed.’” Showing one failure branch demonstrates production thinking.
Common Mistake 3: Ignoring Sync vs Async Boundaries
Wrong: describing all operations as if they happen in a flat sequence with no distinction between synchronous and asynchronous calls.
Better: “Everything from request to driver acceptance is synchronous because the rider is waiting. After the trip state changes, a Kafka event triggers analytics, notifications, and settlement asynchronously.” Calling out the boundary shows you made a deliberate design decision.
Common Mistake 4: Treating All Data Flows as Equivalent
Wrong: giving equal weight to the ride request flow and the analytics pipeline.
Better: “The ride request flow is the critical path with a 10-second latency budget. Analytics processing is asynchronous and can lag by minutes or hours without user impact.” Prioritizing flows shows you understand which paths matter most.
SLIDE 55: How to Walk Through a Data Flow in an Interview
Pick One Flow, Trace It Completely
You will not have time to trace all three flows in detail. Pick the most architecturally interesting one (typically the ride request flow) and trace it end to end. Mention the other two flows briefly to show awareness.
The Structure
Start from the user action (“the rider taps Request Ride”), follow the request through each service by name, call out sync vs async at every boundary, show the data that flows between services, mention one failure case, and end with the final state.
What to Emphasize
Name every service the request touches. The interviewer is checking whether your architecture actually works, not just whether the boxes look nice on the diagram. If you drew a Matching Service in Section 4 but never mention it in the data flow, the interviewer notices.
Call out where state is written. “Trip Service writes the trip record to PostgreSQL” is much stronger than “the system saves the trip.”
Call out one failure case per flow. You do not need to enumerate every possible failure. Showing one demonstrates that you think about error handling. “If the driver declines, the offer cascades to the next candidate. If all candidates decline, the system re-queries with a wider radius.”
Time Budget
Data flow walkthrough should take 4-6 minutes of the interview. 3-4 minutes for the primary flow, 1-2 minutes for briefly describing the other flows. This connects directly to Section 14 (Complete Solution Summary) where we cover full interview pacing.
SLIDE 56: Interview Practice, Data Flow
Practice Exercise: Trace the Ride Request Flow
Set a timer for 3 minutes. Draw or narrate the following from memory:
- Start with the rider action and the API Gateway
- Name every service the request passes through
- State whether each hop is synchronous or asynchronous
- Show what happens when the driver declines
- End with the trip state after driver acceptance
Evaluation Criteria
Strong answer: names Trip Service, Matching Service, Location Service, Dispatch Service. Calls out Redis GEOSEARCH for the spatial query. Mentions the 15-second acceptance window. Shows the cascade on decline. States that everything up to acceptance is synchronous.
Adequate answer: names the key services but misses one or two details. Shows the happy path but skips the failure case.
Weak answer: “The system matches the rider with a nearby driver.” No service names, no data flow, no sync/async distinction.
Transition to Next Section
“Now that we have the data flows defined, the next question is: how does that spatial index actually work? When Matching Service asks ‘give me drivers within 5 km,’ what data structure answers that query at 1.25 million updates per second?” That is Section 6: Geospatial Systems.
SLIDE 57: Section Cheat Sheet
Key Numbers
Ride request flow: < 10 sec end-to-end (500-900ms system, rest is driver decision)
Location write path: 1.25M GPS updates/sec sustained
Location read path: sub-200ms nearby queries (bursty)
Trip completion: sync fare + payment, async analytics + notifications
Dispatch cascade: 15-sec window per driver, 3-5 attempts before re-query
Flow Summary
Sync vs Async Decision Rule
State Machine Transitions
How This Connects to Other Sections
Section 4 (Architecture) defined the boxes. This section defined the arrows. Section 6 (Geospatial) designs the spatial index that the location flow writes to and the matching flow reads from. Section 7 (Matching) details the scoring pipeline referenced in the ride request flow. Section 8 (Database Design) selects the specific technologies (PostgreSQL, Redis, TimescaleDB) for each data store referenced here. Section 10 (Dynamic Pricing) designs the surge engine that feeds into fare calculation.