Building a Personal DSA Playlist Revision RAG: From 60-Hour Video Content to Timed Vector Search

A comprehensive study guide on converting long-form educational video playlists (such as a 60-hour Data Structures and Algorithms course across 127+ videos) into an interactive Retrieval-Augmented Generation (RAG) assistant. This system allows students to ask conceptual questions in natural language and receive immediate text answers alongside in-browser YouTube playback at the exact timestamp where the topic was taught.

1. The Core Problem: The DSA Revision Bottleneck

Students preparing for technical interviews or studying Data Structures & Algorithms (DSA) frequently encounter a critical revision bottleneck:

• Lack of Written Notes: Students often watch video explanations, understand the core logic dynamically, but skip writing detailed manual notes.
• Content Bloat: Over time, playlists scale to massive lengths (e.g., 127 videos, 60–68+ hours of footage, or 500+ problems across large channels).
• Inability to Locate Context: When solving practice problems days or weeks later, a student might vaguely recall a specific technique (e.g., "Where was the Ransom Note problem solved?" or "When should I use a variable-size sliding window?").
• Granularity Loss in Video Titles: Individual videos often cover 4–6 distinct sub-problems or variations. YouTube video titles and metadata cannot exhaustively list every problem or concept explained inside the video body.

60+ Hours of DSA Video Content
   |
   | Student solves a practice problem weeks later
   v
Memory Recall Trigger ("I remember seeing this logic before...")
   |
   | Manual Search Failure: Title doesn't list the sub-question
   v
Hours Wasted Scrubbing Through 127+ Video Timelines

The solution is to leverage Artificial Intelligence—specifically Retrieval-Augmented Generation (RAG)—to build a specialized index over the spoken content of the entire video library.

2. System Architecture: The Interactive Video RAG Solution

Instead of manually scrubbing through hours of footage, the user interacts with an AI assistant grounded strictly in the transcript database of the playlist.

System Capabilities

1. Multilingual / Code-Switched Queries: Accepts queries in English, Hindi, or mixed technical phrasing (e.g., "HashMap kab use karna chahiye?" or "What is variable size sliding window?").
2. Context-Aware Answers: Summarizes the specific technique in concise text.
3. Embedded In-Browser Video Player: Extracts the exact YouTube Video ID and start timestamp, rendering the video player directly within the interface at the precise second the topic begins.
4. Out-of-Scope Filtering: If a user asks about a topic not covered in the playlist (e.g., "What is React JS?"), the system explicitly declines to answer based on grounded retrieval constraints rather than hallucinating external knowledge.

3. Quick-Start Guide: Running the Pre-built System

For developers or students who simply want to consume the revision assistant locally without retraining the transcription pipeline from scratch, the pre-computed index can be launched in three shell steps using uv (a fast Python package installer and resolver).

Prerequisites

• Python 3.10+ installed.
• An API key from Groq or Google Gemini for LLM generation.

Execution Workflow

Step 1: Clone Repository & Navigate to Workspace

git clone https://github.com/padhowithpratyush/ai-engineer.git
cd ai-engineer/week3/yt_scraper

Step 2: Sync Dependencies & Set Environment Variables

# Sync Python environment dependencies
uv sync

# Create local environment configuration from template
cp .env.example .env

Open the .env file in your editor and paste your LLM provider API key:

GROQ_API_KEY=gsk_your_actual_groq_api_key_here

Step 3: Load Vector Index & Launch the Local Server

# Load pre-built vector index and transcript metadata
uv run yt_rag load

# Launch local server interface
uv run yt_rag serve

Navigate to http://localhost:8080 or the terminal-provided local URL in Chrome to interact with the assistant.

4. Engineering Mindset: The "What, How, Why" Framework

Creating high-value AI portfolio projects requires moving beyond standard tutorials (e.g., basic library management systems or template CRUD apps) and focusing on real user pain points.

The Three Principles of Applied AI Projects

• WHAT: A custom revision RAG assistant that turns unstructured video playlists into an instant queryable knowledge engine.
• HOW: An automated engineering pipeline using yt-dlp for audio extraction, OpenAI Whisper for local GPU speech-to-text transcription, time-windowed semantic chunking, Qdrant vector storage, and an LLM for final response synthesis.
• WHY: Existing LLM tools (ChatGPT, Claude, Gemini) have no intrinsic visibility into private or specific long-form YouTube video timestamps unless provided via a custom grounding pipeline.

Note: Modern developer productivity relies heavily on AI coding assistants (e.g., Claude, Cursor, ChatGPT) for generating framework boilerplate. True engineering capability lies in pipeline orchestration, data modeling, edge-case handling, and problem framing, rather than memorizing syntax.

5. Data Pipeline Step 1: Audio Extraction via yt-dlp

Video streams contain high-volume visual payload that is unnecessary for semantic text search. Since instructors explain concepts verbally while coding, the information lives primarily in the audio track.

Processing Strategy

1. Pass the target YouTube playlist URL to the pipeline.
2. Use yt-dlp to download only the compressed audio track (.mp3 or .wav) instead of full video files.
3. Skip video re-encoding to save bandwidth and local disk storage.

# Conceptual pipeline for extracting audio tracks using yt-dlp
import yt_dlp

def extract_playlist_audio(playlist_url: str, output_dir: str):
    ydl_opts = {
        'format': 'm4a/bestaudio/best',
        'outtmpl': f'{output_dir}/%(id)s.%(ext)s',
        'postprocessors': [{
            'key': 'FFmpegExtractAudio',
            'preferredcodec': 'mp3',
            'preferredquality': '192',
        }],
    }
    with yt_dlp.YoutubeDL(ydl_opts) as ydl:
        ydl.download([playlist_url])

6. Data Pipeline Step 2: Speech-to-Text via OpenAI Whisper

Once raw audio files are downloaded, they must be transcribed into text while preserving crucial timestamp metadata.

OpenAI Whisper Integration

• Model: OpenAI Whisper (run locally or on cloud GPUs).
• Hardware Requirements: Local transcription of massive playlists requires dedicated GPU acceleration (e.g., NVIDIA RTX GPUs).
• Processing Scale: Transcribing ~68 hours of mixed Hindi/English (Hinglish) audio across 127 videos takes roughly 8 hours of continuous processing on a mid-range consumer GPU (e.g., NVIDIA RTX 3050/4050).

Real-World Engineering Reality: Pipelines operating over multiple hours face real-world failure points (power loss, driver crashes, network drops). Production transcription scripts must be written idempotently—checking if a video transcript already exists locally before invoking GPU execution so processing can seamlessly resume after interruptions.

7. Data Pipeline Step 3: Time-Based Semantic Chunking & Vector Storage

Feeding an entire 30-minute transcript into a single vector embedding destroys spatial precision; searching for a specific sub-problem would return the entire video. Conversely, splitting purely by standard word count breaks sentences mid-explanation.

Time-Windowed Chunking Strategy

Instead of standard paragraph splitting, use a time-based windowing approach:

• Fixed Time Span: Segment the transcript into continuous blocks of 75 seconds.
• Metadata Attachment: Attach explicit metadata to each 75-second chunk:
  • video_id: YouTube unique string.
  • video_title: Title of the lesson.
  • start_timestamp: Beginning time in seconds (e.g., 312).
  • end_timestamp: Ending time in seconds (e.g., 387).

Vector Embedding & Qdrant Upload

1. Each 75-second text chunk is converted into a high-dimensional vector representation using a lightweight embedding model (e.g., all-MiniLM-L6-v2).
2. The resulting vector embeddings and their associated metadata payloads (text, video_id, start_time) are indexed in a Qdrant Vector Database.

# Conceptual schema for vector payload insertion in Qdrant
payload = {
    "text": "For variable size sliding window, we maintain two pointers...",
    "video_id": "abc123XYZ",
    "video_title": "Sliding Window Pattern Part 2",
    "start_seconds": 346,
    "formatted_timestamp": "05:46"
}

8. Complete System Flow Walkthrough

The following vertical diagram illustrates the entire request-response lifecycle from user interaction to player rendering:

9. Architectural Synthesis: Tools & Responsibilities

10. Practical Edge Cases & Debugging Realities

When building and deploying video RAG systems, several edge cases require engineering workarounds:

1. Approximate Timestamp Drift

• Issue: Speech alignment in Whisper or 75-second static window boundaries may sit slightly ahead of or behind the actual code demonstration on screen.
• Mitigation: Instruct the UI component to open the YouTube timestamp 10–15 seconds earlier than the retrieved start segment to account for introductory remarks.

2. Ambiguous Problem Names

• Issue: Questions like "How do I solve problem 3?" are ambiguous across a 127-video playlist.
• Mitigation: Prompt the LLM to request topic clarifications or rank results by matching video titles alongside transcript semantic similarity.

3. Handling Uncovered Topics

• Issue: Users asking general domain questions not present in the playlist (e.g., System Design, React, OS scheduling).
• Mitigation: System prompt strictness must enforce direct refusal when similarity scores fall below a strict distance threshold:

  "This topic was not covered in the provided lecture playlist."

Summary and Key Takeaways

• Targeted Problem Solving: Software tools built to solve actual personal workflows (e.g., pinpointing exact timestamps in a 60-hour course) provide far greater practical value than generic tutorial projects.
• Audio-First Extraction: Processing YouTube video content into searchable text should prioritize extracting compressed audio via yt-dlp to minimize computational overhead.
• Timestamped Transcriptions: OpenAI Whisper enables offline GPU transcription of long-form audio playlists while preserving precise start/end timing data.
• Time-Based Chunking: Chunking transcripts by 75-second continuous intervals retains spatial local context far better than standard paragraph/character splitting for video indexing.
• Grounded Retrieval: Combining Qdrant vector similarity with LLM generation prevents hallucinations and grounds answers directly in original lecture statements.
• Seamless Local Setup: Packaging vector indices and execution steps with uv allows end users to run local RAG revision tools using simple commands (uv sync, uv run yt_rag load, uv run yt_rag serve).