✅ System Design 19: Design of Online Learning Platform | Udemy | Coursera | HLD | LLD

An online learning platform (such as Udemy or Coursera) is a distributed knowledge-sharing ecosystem where global instructors upload instructional videos, assignments, and assessments, and students purchase, stream, and track their progress through courses. This chapter outlines the architectural blueprints, functional workflows, non-functional trade-offs, and deep-dive schema/component designs for building a massive, highly scalable ed-tech platform.

1. Why Do We Need This? (Problem Statement & Scope)

Building an enterprise-grade learning management system (LMS) is deceptively complex. Unlike simple static websites, an online learning platform must coordinate multiple distinct actors simultaneously:

• Students/Users: Browse, search, enroll, stream video lessons, solve quizzes, track progress, and write course reviews.
• Instructors/Creators: Onboard onto the platform, upload massive video files, organize courses into chapters, create quizzes, and monitor enrollments.
• Moderators/Admins: Audit uploaded content, verify instructor credentials, review policy violations, and publish safe courses to the global catalog.

Because of this multi-actor complexity, tackling the system design requires breaking down the requirements methodically into functional scopes, non-functional constraints, core entities, and isolated data flows.

2. Core Mental Model & Real-Life Analogy

Real-Life Analogy: Think of an online learning platform like a Global Academic Marketplace & Central Post-Production Studio.

• Instructors are independent authors dropping raw film reels and textbooks at the receiving dock (Media Uploader & Catalog Service).
• Moderators are quality-assurance editors inspecting the reels for compliance before placing them on the front-shelf display (Course Moderator Service).
• Students are library patrons walking into the showroom, browsing catalog drawers (Elastic Search), checking out media securely with a library card/payment receipt (Enrollment & Payment Service), and viewing films in private screening booths equipped with local projectors (CDN & Video Playback Service).

3. Functional Requirements

Before designing the architecture, we must define the core features expected from both user and instructor perspectives:

Student / User Requirements

1. User Onboarding & Authentication: Users can create secure accounts, log in, and manage their profile metadata.
2. Catalog Browsing & Search: Users can search, filter, and sort courses by category, difficulty level, ratings, and price.
3. Course Enrollment & Access Control: Users can enroll in free courses directly or purchase paid courses via a secure payment gateway.
4. Video Streaming & Playback: Enrolled students can stream course videos smoothly with minimal latency.
5. Progress Tracking: The system tracks and persists the percentage of the course completed by the student.
6. Assessments: Students can take quizzes and submit assignments.
7. Reviews & Comments: Students can leave ratings, comments, and reviews for courses.

Instructor Requirements

8. Instructor Onboarding & Verification: Instructors register and undergo verification (e.g., government ID checks).
9. Course Content Management: Instructors can create courses, upload metadata, structure chapters, attach quizzes, and upload heavy video files.

4. Non-Functional Requirements & CAP Theorem Strategy

To handle millions of active users globally, we must define strict non-functional targets:

• Scale: Target capacity of 100,000+ courses and 1,000,000+ active students globally.
• CAP Theorem Trade-Off:
  • Availability over Consistency ($AP$) for video streaming, course searching, and content browsing. If an instructor publishes a new course, it does not need to appear globally at microsecond zero. High uptime and uninterrupted video playback are paramount.
  • Consistency over Availability ($CP$) for the Payment & Enrollment Flow. If a user pays for a course, access must be provisioned accurately and atomically; we cannot tolerate payment success without course enrollment.
• Large File Support: The system must effortlessly handle multi-gigabyte video files (1-hour 4K/1080p lectures).
• Low Latency Streaming: Video playback startup and chunk delivery must occur within a nominal latency window ($\le 2$ seconds).

5. Core Entities & Database Schema Design

Based on our functional requirements, we identify the primary entities: Users, Instructors, Courses, Enrollments, Payments, Assessments/Quizzes, and Progress.

Relational Database Schemas (PostgreSQL / MySQL)

Assessment & Progress Schemas

6. API Design (REST Endpoints)

While interview constraints often allow skipping exhaustive endpoints, these are the core public REST contracts:

7. High-Level Architecture (HLD)

The system is split into two primary operational zones: the Student Domain and the Instructor Domain, fronted by dedicated Load Balancers and API Gateways.

High-Level Flow Walkthrough:

1. Instructor Ingestion: Instructors upload course metadata via the Catalog Service and heavy video files via the Media Uploader Service into Blob Storage (S3).
2. Moderation: An admin reviews the uploaded assets via the Course Moderator Service before publishing.
3. Student Discovery: Students search for courses using the Course Search Service, backed by Elastic Search.
4. Enrollment & Payment: For paid courses, the Payment Service processes transactions, triggering secure enrollment permissioning.
5. Playback: Enrolled students request videos through the Video Playback Service, streaming transcoded chunks directly from Blob Storage / CDN.

8. Deep-Dive Design: Data Ingestion, Search, and Outbox Pattern

A. Elastic Search & CDC Pipeline for Course Discovery

Searching courses by multiple dynamic filters (category, difficulty, rating, price) across segregated relational tables causes severe latency if joined on the fly.

• Architecture: We introduce an asynchronous CDC (Change Data Capture) Pipeline and an Aggregator Service.
• Flow:
  Structure & Concept Flow

  Copy DiagramCopy

  Course DB (Relational)

  |

  v (CDC Pipeline detects changes)

  Aggregator Service joins tables into clean JSON documents

  |

  v

  Elastic Search Cluster (Precomputed search index)

  |

  v

  Course Search Service queries Elastic Search instantly

   

B. Instructor Verification Flow

When an instructor registers, they must provide government identification (e.g., PAN card, Aadhaar in India) for anti-fraud compliance.

• The User Service routes verification payloads to external third-party identity verification APIs (e.g., UIDAI endpoints).
• The verification flag (verification_status) is updated in the Instructor DB.

C. Large Video Uploads & Transcoding (S3 Integration)

Instructors do not stream multi-gigabyte video files through application servers.

• Signed URL Workflow:
  Structure & Concept Flow

  Copy DiagramCopy

  Instructor -> Media Uploader Service -> requests Pre-signed S3 URL

  |

  v (receives secure upload token)

  Instructor uploads video chunks directly to Blob Storage (S3)

  |

  v

  Transcoding Workers convert video into multiple resolutions (4K, 1080p, 720p, 480p)

   

9. Deep-Dive Design: Resilient Payments & The Outbox Pattern

A critical failure mode in distributed e-commerce systems occurs when a user pays successfully, but the course enrollment database update fails, leaving the user without access.

The Outbox Design Pattern

To guarantee consistency without locking multiple databases across network boundaries:

10. Deep-Dive Design: Video Playback & Real-Time Progress Tracking

A. Secure Video Streaming via CDN

1. When an enrolled student clicks play, the Video Playback Service verifies user access against the Course Enrollment DB.
2. Upon successful verification, the service returns a Manifest File (HLS/DASH protocol).
3. The client player requests video chunks from the Local CDN, falling back to S3 Blob Storage only on cache misses, ensuring sub-2-second latency globally.

B. Debounced Progress Tracking via Kafka

Tracking student progress by firing network requests every single second would overwhelm the database.

• Optimized Event Pipeline:
  Structure & Concept Flow

  Copy DiagramCopy

  Client Player (Fires progress event every 10-15 seconds)

  |

  v

  Kafka Broker (Ingests high-throughput progress events)

  |

  v

  Consumer Service (Runs batch jobs / consumer loops)

  |

  v

  Progress DB (Persists only the latest timestamp and completion % periodically)

   

11. Enterprise-Level Enhancements (Follow-Up Interview Questions)

When designing for enterprise clients (e.g., Microsoft, Google, Flipkart purchasing bulk seats for employees), interviewers often introduce advanced scaling constraints:

1. Tenant Isolation & Database Sharding: Enterprise clients demand strict tenant data segregation. Rather than pooling all organization courses in a single table, data is partitioned or sharded using organization-specific tenant keys.
2. Custom Enterprise Subdomains & SSO: Enterprises require branded landing pages (company.udemy.com) configured via custom DNS subdomains, managed templates stored as JSON configurations, and SAML/OAuth Single Sign-On integration.
3. Subnet & Network Routing: Enterprise traffic is routed through dedicated subnets and API gateways with specialized rate-limiting and enterprise reporting hooks.

Summary and Key Takeaways

• Multi-Actor Domain Complexity: Online learning platforms coordinate distinct workflows for students, instructors, and moderators, requiring decoupled service boundaries.
• Strategic CAP Selection: Prioritize Availability ($AP$) for video streaming, search, and catalog browsing; prioritize Consistency ($CP$) for payment gateways and enrollment permissions.
• CDC & Elastic Search: Precompute and sync relational course data into Elastic Search using CDC pipelines to achieve sub-second search latencies across multiple dynamic filters.
• Direct-to-S3 Uploads: Never pipe heavy video files through application servers; utilize pre-signed S3 URLs and asynchronous transcoding workers.
• Transactional Outbox Pattern: Solve distributed dual-write failures during payments by using database outlook tables paired with Kafka event brokers and permission sync consumers.
• Optimized Progress Tracking: Avoid database thrashing by buffering high-frequency client viewing events in Kafka and flushing aggregated progress checkpoints asynchronously.