How the Downloader Works
A complete technical look into our pipeline: URL validation, SSRF defense, worker threading, stream multiplexing with FFmpeg, and automated lifecycle cleanup.
End-to-End Download Pipeline
Unlike ad-ridden scraper websites that rely on third-party redirection networks and malicious popups, our web downloader operates as a clean, local-first Python application. Every download progresses through seven distinct stages:
URL & Host Allowlist
The incoming link is parsed using Python's urllib.parse. The hostname is checked against our strict allowlist (youtube.com, youtu.be, m.youtube.com).
SSRF Network Defense
The server resolves the target hostname via DNS and checks the resulting IP against reserved and private subnets to prevent Server-Side Request Forgery attacks.
SQLite State Database
A job record is initialized with state QUEUED in a WAL-mode SQLite database with atomic locking to ensure consistency across concurrent workers.
yt-dlp Extraction
A background worker claims the job and executes yt-dlp to stream video and audio tracks directly from YouTube's CDN with throttling bypasses.
FFmpeg Multiplexing
FFmpeg remuxes the independent high-bitrate video and audio streams into a clean, synced MP4 or transcodes audio to 320 kbps MP3.
UUID File Handling
The output file is saved with a randomized UUID handle (e.g., video_a7b8c9d0-1234.mp4) preventing path traversal attacks and collision risks.
24h Auto-Purge
A recurring background cleanup worker inspects file modification timestamps and permanently purges completed downloads after 24 hours.
Security Architecture & SSRF Protection (E-E-A-T)
Web applications that fetch external URLs requested by users are vulnerable to Server-Side Request Forgery (SSRF) if implemented carelessly. An attacker might provide URLs pointing to internal services like http://127.0.0.1:8080/admin, cloud metadata endpoints like http://169.254.169.254/latest/meta-data/, or private intranet routers.
Our backend enforces multi-layered defensive security before any network request leaves the server:
1. DNS Resolution & IP Subnet Verification
Before allowing yt-dlp to touch a URL, our validate_url() function resolves the target host's IP address and verifies it using Python's ipaddress module:
resolved_ips = socket.getaddrinfo(hostname, None)
for family, stype, proto, canonname, sockaddr in resolved_ips:
ip = ipaddress.ip_address(sockaddr[0])
if ip.is_private or ip.is_loopback or ip.is_link_local or ip.is_reserved:
raise InvalidURLError("URL resolves to a restricted network address.")
This ensures that even if an attacker attempts DNS rebinding tricks or inputs a domain pointing to 127.0.0.1 or 192.168.1.1, the request is immediately aborted with HTTP 400.
2. Hostname Whitelisting
Only authentic YouTube domains are accepted:
www.youtube.comyoutube.comm.youtube.comyoutu.be
Any other domain, IP literal, local file scheme (file:///), or FTP URL is rejected before socket initialization.
3. Sanitized Error Handling
Stack traces, local operating system paths, environment variables, and internal Python file locations are strictly filtered from API responses. If an upstream extraction error occurs, the user receives a clean, actionable error message without exposing server internals.
4. Rate Limiting & Concurrency Control
To prevent denial-of-service scenarios and protect system resources, requests are throttled per IP address, and background worker queues are capped at 50 concurrent items with 3 active worker threads.
Video & Audio Processing Pipeline Flow
| Stage | Component | Function | Latency |
|---|---|---|---|
| Validation | Python urllib + socket |
SSRF check, hostname filter, URL normalization | ~10 – 30 ms |
| Metadata | yt-dlp --skip-download |
Fetch video title, author, thumbnail, format codes | ~500 – 900 ms |
| Queueing | SQLite WAL Engine | Thread-safe job state persistence (QUEUED) | < 5 ms |
| Streaming | yt-dlp HTTP client |
Concurrent chunk download from YouTube CDN | Network Dependent |
| Multiplexing | FFmpeg Subprocess |
Mux video + audio streams into unified MP4 / MP3 | ~1 – 6 seconds |
| Serving | Flask send_file |
Chunked streaming download to client browser | Immediate |
Deep-Dive Engineering Articles
Want to understand the codebase or build a similar architecture yourself? Read our engineering guides: