How do you prevent Server-Side Request Forgery (SSRF) in media downloader web apps? Never pass user-supplied URLs directly to backend network fetchers. Implement strict URL parsing, enforce an allowlist of permitted domain hosts (e.g., youtube.com, youtu.be), resolve destination hostnames to IP addresses using socket.getaddrinfo(), and block all private, loopback (127.0.0.1), link-local (169.254.0.0/16), and cloud metadata IP ranges (169.254.169.254).
- Domain Allowlisting: Strict regex checking of hostname suffixes.
- DNS Resolution Check: Validating resolved IPs against RFC 1918 private ranges.
- Cloud Metadata Shield: Blocking AWS/GCP/Azure instance metadata endpoints.
What Is SSRF and Why Do Media Downloaders Get It?
SSRF occurs when an attacker can cause a server to make HTTP requests to an arbitrary URL of the attacker's choosing. In a typical SSRF attack chain:
- Attacker submits a URL like
http://169.254.169.254/latest/meta-data/iam/security-credentials/ - The server makes an HTTP request to that internal AWS metadata endpoint
- The response is returned to the attacker, revealing AWS IAM credentials
- Attacker uses those credentials to gain unauthorized access to cloud resources
Media downloaders are particularly vulnerable because their core functionality is "accept a URL, fetch content from it." Without careful validation, that functionality becomes an open SSRF vector.
Layer 1: URL Scheme and Host Allowlisting
The first and most important defense is strict allowlisting of permitted URL patterns:
ALLOWED_HOSTS = {
'youtube.com', 'www.youtube.com', 'm.youtube.com',
'youtu.be', 'www.youtube-nocookie.com',
'music.youtube.com',
}
ALLOWED_SCHEMES = {'https', 'http'}
def validate_url(url: str) -> dict:
try:
parsed = urlparse(url)
except Exception:
return {'valid': False, 'error': 'Malformed URL.'}
if parsed.scheme not in ALLOWED_SCHEMES:
return {'valid': False, 'error': f'Scheme "{parsed.scheme}" not allowed.'}
host = parsed.hostname.lower() if parsed.hostname else ''
# Check exact match and subdomains
allowed = any(
host == h or host.endswith(f'.{h}')
for h in ALLOWED_HOSTS
)
if not allowed:
return {'valid': False, 'error': f'Host "{host}" not in allowlist.'}
return {'valid': True, ...}
Layer 2: Post-DNS IP Range Blocking
Host allowlisting alone is insufficient because of DNS rebinding attacks. An attacker could register a domain like youtube-evil.com that resolves to YouTube's hostname at first check, then switches to 192.168.1.1 when yt-dlp makes the actual download request. Or an attacker could register a custom domain that resolves to an internal IP address directly.
The solution is to perform a DNS resolution of the hostname and then validate that the resolved IP address does not fall within any private, loopback, or link-local IP ranges:
import ipaddress, socket
PRIVATE_RANGES = [
ipaddress.ip_network('10.0.0.0/8'),
ipaddress.ip_network('172.16.0.0/12'),
ipaddress.ip_network('192.168.0.0/16'),
ipaddress.ip_network('127.0.0.0/8'),
ipaddress.ip_network('169.254.0.0/16'), # Link-local / AWS metadata
ipaddress.ip_network('::1/128'), # IPv6 loopback
ipaddress.ip_network('fc00::/7'), # IPv6 private
]
def is_safe_ip(hostname: str) -> bool:
try:
resolved_ip = socket.gethostbyname(hostname)
ip = ipaddress.ip_address(resolved_ip)
for private_range in PRIVATE_RANGES:
if ip in private_range:
return False
return True
except socket.gaierror:
return False # DNS resolution failed, treat as unsafe
Layer 3: Error Message Sanitization
SSRF vulnerabilities are often exacerbated by verbose error messages that leak internal network information. If your application returns the raw yt-dlp error string directly to users, it may reveal internal IP addresses, file paths, or service endpoint details that aid an attacker in mapping your internal network.
YT Downloader sanitizes all yt-dlp error messages before returning them to the client, replacing implementation-specific details with user-friendly categorized error codes.
Layer 4: Rate Limiting
Even with URL allowlisting, an attacker could submit massive volumes of YouTube URL requests to probe YouTube's own infrastructure for SSRF-adjacent vulnerabilities or to cause downstream service disruption. Flask-Limiter's per-IP rate limiting (30 requests per minute by default) provides a first line of defense against bulk submission attacks.
Layer 5: X-Robots-Tag on API Endpoints
API endpoints are set with X-Robots-Tag: noindex, nofollow response headers to prevent search engines from crawling and indexing them. This reduces the API's public exposure surface and avoids advertising endpoint patterns to automated vulnerability scanners.
Testing Your SSRF Defenses
To verify your validation is working correctly, test these payloads against your /api/metadata endpoint and confirm they are rejected:
# Should all return validation errors, NOT fetch content:
curl -X POST /api/metadata -d '{"url": "http://169.254.169.254/"}' -H "Content-Type: application/json"
curl -X POST /api/metadata -d '{"url": "http://localhost:5432/"}' -H "Content-Type: application/json"
curl -X POST /api/metadata -d '{"url": "http://192.168.1.1/"}' -H "Content-Type: application/json"
curl -X POST /api/metadata -d '{"url": "file:///etc/passwd"}' -H "Content-Type: application/json"
curl -X POST /api/metadata -d '{"url": "https://evil.example.com/"}' -H "Content-Type: application/json"
Practical Python Defense: Complete SSRF Validator Implementation
Below is a hardened Python utility module illustrating robust URL validation against SSRF vulnerabilities:
import ipaddress
import socket
from urllib.parse import urlparse
ALLOWED_DOMAINS = {'youtube.com', 'youtu.be', 'vimeo.com', 'soundcloud.com'}
def is_safe_url(target_url: str) -> bool:
try:
parsed = urlparse(target_url)
if parsed.scheme not in ('http', 'https'):
return False
hostname = parsed.hostname
if not hostname:
return False
# Verify domain suffix
if not any(hostname == d or hostname.endswith('.' + d) for d in ALLOWED_DOMAINS):
return False
# Resolve all associated IP addresses
addr_info = socket.getaddrinfo(hostname, None)
for _, _, _, _, sockaddr in addr_info:
ip_str = sockaddr[0]
ip_obj = ipaddress.ip_address(ip_str)
# Reject loopback, private, link-local, and reserved addresses
if (ip_obj.is_private or ip_obj.is_loopback or
ip_obj.is_link_local or ip_obj.is_reserved or
ip_str == '169.254.169.254'):
return False
return True
except Exception:
return False
By invoking this check before initiating any yt-dlp metadata extraction or download workflow, your server guarantees that malicious actors cannot weaponize your application to probe your internal network topology or cloud metadata services.