Key takeaways
Three main routes exist: the YouTube Data API v3, the yt-dlp library, and direct calls to YouTube's internal /youtubei/v1/ JSON endpoints.
Public video metadata (title, view count, description, tags) sits in the ytInitialPlayerResponse JSON embedded in the page HTML, reachable with requests and a regex.
yt-dlp pulls metadata, comments, and transcripts with no API key; pass download=False to stay metadata-only.
For steady, large-scale collection, route requests through rotating residential, datacenter, or mobile proxies matched to the data target and volume.
You can scrape public YouTube video data three ways: the official YouTube Data API v3, the yt-dlp library, or YouTube's internal /youtubei/ JSON endpoints called directly with Python. The API is simplest at low volume but capped at 10,000 quota units per day. For steady, large-scale collection of publicly available data, route requests through rotating residential or datacenter proxies for reliable connectivity.
Below you get working Python for each route, plus the part most tutorials skip: which proxy type fits which YouTube data target, and what it costs next to a fixed daily quota.
You can generally collect publicly available YouTube data such as titles, view counts, descriptions, and public comments. US courts have also held that scraping public data is not automatically a computer-fraud violation. Downloading or re-hosting copyrighted videos is separate and governed by copyright law and YouTube’s Terms.
This guide covers public metadata only, such as views, channel names, tags, comments, and captions. Video files are out of scope.
Start with YouTube’s Terms of Service. They restrict automated access, including scrapers, except for qualifying public search engines or with YouTube’s permission. Check the rules for your use case before building. The official API is YouTube’s approved route.
In the US, hiQ v. LinkedIn found that accessing publicly available data likely does not constitute unauthorized access under the CFAA. The ruling is narrow and does not resolve contract or other legal claims.
Privacy matters too. Comment names and channel handles may be personal data under GDPR. Collect only what you need and avoid storing unnecessary PII. For commercial projects, consult counsel.
Teams use Proxy-Cheap for collection of publicly available data in line with applicable requirements. See our data scraping use cases for more.
There are four practical ways to collect public YouTube data:
| Method | Setup | Data | Best for | Proxy fit |
|---|---|---|---|---|
| YouTube Data API v3 | Low | Structured fields, quota-limited | Low-volume data | Usually not needed |
| yt-dlp | Low | Metadata, comments, transcripts | Batch jobs from known URLs | Helpful at scale |
| Direct /youtubei/ endpoints | Medium | Search, channels, videos, comments | Flexible, scaled collection | Recommended |
| Browser automation | High | Anything the page renders | JS-heavy, infinite-scroll pages | Recommended |
YouTube Data API v3: Clean, structured JSON with documented fields, but limited by daily quota.
yt-dlp: Easy to install and handles YouTube internals. Best when you already know the videos or channels to collect.
Direct /youtubei/ endpoints: Use the same JSON endpoints as YouTube’s frontend, giving you more control over pacing and IP rotation. The trade-off is maintenance when response formats change.
Browser automation: A heavier option that renders the page like a user’s browser. Use it when content only appears after scrolling or interaction, since it requires more CPU and bandwidth.
TThe YouTube Data API v3 returns structured data for videos, channels, playlists, comments, and search. A default project gets 10,000 quota units per day for most methods, with videos.list costing 1 unit. Since June 2026, search.list has its own 100-call daily bucket, so search-heavy workflows hit the limit quickly.
Setup is simple: create a Google Cloud project, enable YouTube Data API v3, and create an API key under Credentials. An API key is enough for public data; OAuth 2.0 is only needed for user-authorized actions.
Here’s a videos.list request for multiple IDs:
import requests API_KEY = "YOUR_API_KEY" VIDEO_IDS = ["dQw4w9WgXcQ", "jNQXAC9IVRw"] params = { "part": "snippet,statistics,contentDetails", "id": ",".join(VIDEO_IDS), "key": API_KEY, } resp = requests.get( "https://www.googleapis.com/youtube/v3/videos", params=params, timeout=15, ) resp.raise_for_status() for item in resp.json()["items"]: print( item["snippet"]["title"], "|", item["snippet"]["channelTitle"], "|", item["statistics"].get("viewCount"), "views", )
The call costs 1 quota unit, regardless of whether you request one ID or a batch. According to the official YouTube Data API docs, videos.list, channels.list, commentThreads.list, and playlistItems.list each cost 1 unit. The June 2026 quota change put search.list in a separate 100-call daily bucket. Each results page counts as another call, and quotas reset at midnight Pacific Time.
The API works well for known IDs, but large keyword crawls and comment-heavy jobs can hit limits quickly. Quota increases are possible through Google’s audit process, but aren’t guaranteed.
For jobs like checking 2,000 known video IDs daily, the API is a strong choice: low risk, structured data, and no HTML parsing. If you’re building a service around it, see our guide to working with a proxy API.
YouTube embeds public video metadata in a ytInitialPlayerResponse JSON object inside the page HTML. Fetch the page with requests, locate the object, and parse it with Python’s json module. The videoDetails key contains the title, views, channel, and description, with no API key required.
Watch pages also include ytInitialData, which contains related videos and other page content. This approach only needs requests and Python’s standard library.
import json import re import requests HEADERS = { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) " "AppleWebKit/537.36 (KHTML, like Gecko) " "Chrome/140.0 Safari/537.36", "Accept-Language": "en-US,en;q=0.9", } def get_player_response(video_id, proxies=None): url = f"https://www.youtube.com/watch?v={video_id}" html = requests.get( url, headers=HEADERS, proxies=proxies, timeout=20 ).text match = re.search(r"ytInitialPlayerResponse\s*=\s*", html) if not match: raise ValueError("ytInitialPlayerResponse not found") data, _ = json.JSONDecoder().raw_decode(html, match.end()) return data details = get_player_response("dQw4w9WgXcQ")["videoDetails"] print("Title:", details["title"]) print("Channel:", details["author"], details["channelId"]) print("Views:", details["viewCount"]) print("Length:", details["lengthSeconds"], "seconds") print("Keywords:", details.get("keywords", [])[:5]) print("About:", details["shortDescription"][:200])
videoDetails includes videoId, title, author, channel ID, views, length, keywords, and description. Other page data contains fields such as likes and upload dates. For those, yt-dlp provides a simpler interface.
This approach is much lighter than browser automation because one GET returns the page data. EU visitors may encounter a consent page, so check for the JSON object before parsing.
For higher volumes, proxies can improve reliability. Teams doing market research can use this approach to track views and keywords across thousands of public competitor videos.
yt-dlp extracts YouTube metadata, comments, and transcripts without an API key. Set download=False to get metadata without saving video files. For comments, use getcomments=True. Captions can be handled with yt-dlp or the youtube-transcript-api.
The yt-dlp project is actively maintained. Install it with pip install yt-dlp. Full YouTube support may also require a JavaScript runtime such as Deno or Node.js.
Metadata only:
from yt_dlp import YoutubeDL URL = "https://www.youtube.com/watch?v=dQw4w9WgXcQ" opts = {"quiet": True, "skip_download": True} with YoutubeDL(opts) as ydl: info = ydl.sanitize_info(ydl.extract_info(URL, download=False)) for key in ("id", "title", "channel", "upload_date", "duration", "view_count", "like_count", "comment_count", "tags"): print(f"{key:20} {info.get(key)}")
Comments: Use getcomments=True and limit results with max_comments:
opts = { "quiet": True, "skip_download": True, "getcomments": True, "extractor_args": { "youtube": { "max_comments": ["200", "all", "0"], "comment_sort": ["top"], } }, }
Comment entries include fields such as author, text, likes, timestamp, and parent ID, allowing you to rebuild threads.
Transcripts: yt-dlp supports subtitle downloads through writesubtitles, writeautomaticsub, and language options. Avoid building timedtext URLs manually because YouTube now uses signed parameters and, for some clients, proof-of-origin tokens. For a simpler workflow, use youtube-transcript-api:
from youtube_transcript_api import YouTubeTranscriptApi api = YouTubeTranscriptApi() transcript = api.fetch("dQw4w9WgXcQ", languages=["en", "de"]) print(transcript.language, "| auto-generated:", transcript.is_generated) for snippet in transcript.snippets[:10]: print(f"{snippet.start:7.2f}s {snippet.text}")
Choose yt-dlp for quick batch jobs from known URLs. Use direct endpoints when you need more control at scale.
For long-running bulk pulls, unlimited bandwidth proxies can simplify bandwidth costs.
YouTube search, channel, and Shorts data use the same internal /youtubei/v1/ endpoints called by the site's frontend. Replicate the requests with Python, then use continuation tokens to paginate. The same ytInitialData and ytInitialPlayerResponse parsing works for regular videos and Shorts.
To find the request, open YouTube in Chrome, press F12, then use Network > Fetch/XHR and run a search. Look for a POST to /youtubei/v1/search containing a context object and your query. Right-click it, copy as cURL, and translate it to Python if needed.
The first response includes results and a continuation token. Send that token back for the next page. The example below reads the current client version from the homepage and uses find_key() to search the full response, making it less sensitive to layout changes.
python import re import requests BASE = "https://www.youtube.com" HEADERS = { "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 " "(KHTML, like Gecko) Chrome/140.0 Safari/537.36", "Accept-Language": "en-US,en;q=0.9", } def find_key(obj, key): """Yield every value stored under `key`, at any depth.""" if isinstance(obj, dict): for k, v in obj.items(): if k == key: yield v yield from find_key(v, key) elif isinstance(obj, list): for item in obj: yield from find_key(item, key) def text_of(field): field = field or {} return field.get("simpleText") or "".join(r.get("text", "") for r in field.get("runs", [])) def search_videos(query, max_pages=3, proxies=None): session = requests.Session() session.headers.update(HEADERS) session.proxies.update(proxies or {}) home = session.get(BASE, timeout=20).text version = re.search(r'"INNERTUBE_CONTEXT_CLIENT_VERSION":"([^"]+)"', home) context = {"client": {"clientName": "WEB", "clientVersion": version.group(1) if version else "2.20260708.00.00", "hl": "en", "gl": "US"}} payload = {"context": context, "query": query} for _ in range(max_pages): resp = session.post(f"{BASE}/youtubei/v1/search", params={"prettyPrint": "false"}, json=payload, timeout=20) resp.raise_for_status() data = resp.json() for video in find_key(data, "videoRenderer"): yield { "video_id": video.get("videoId"), "title": text_of(video.get("title")), "channel": text_of(video.get("ownerText")), "views": text_of(video.get("viewCountText")), "published": text_of(video.get("publishedTimeText")), } tokens = [cmd["token"] for cmd in find_key(data, "continuationCommand") if "token" in cmd] if not tokens: break payload = {"context": context, "continuation": tokens[-1]} if __name__ == "__main__": for row in search_videos("python web scraping tutorial", max_pages=2): print(row)
The hl and gl fields control language and country. Change gl to collect search results for another market.
Channels. Channel metadata and video listings use separate requests. The internal API uses POST /youtubei/v1/browse with the channel's browseId, followed by the same continuation loop. For most projects, yt-dlp's flat extraction is simpler:
python from yt_dlp import YoutubeDL CHANNEL = "https://www.youtube.com/@YouTube/videos" # use /shorts for the Shorts tab opts = {"quiet": True, "extract_flat": "in_playlist", "playlistend": 100} with YoutubeDL(opts) as ydl: tab = ydl.extract_info(CHANNEL, download=False) print(tab.get("channel"), tab.get("channel_follower_count")) for entry in tab["entries"]: print(entry["id"], entry.get("title"), entry.get("view_count"))
Shorts. Shorts are regular videos with a vertical player. Their IDs work with standard watch URLs, and the page exposes the same ytInitialPlayerResponse, so get_player_response() from Method 2 works unchanged.
At scale, steady collection often depends on rotating IPs, since hundreds of paginated calls from one address can reduce success rates. See our YouTube data collection guide for common setups. The matrix below maps each job to a proxy type.
The right proxy depends on the target and volume. Rotating residential proxies suit high-volume search and channel crawls that need fresh IPs. Datacenter proxies offer high throughput for public metadata. Rotating mobile proxies fit the most demanding targets, while static residential (ISP) proxies suit steady, session-based collection.
| Data target | Typical scale | Recommended Proxy-Cheap product |
|---|---|---|
| Video metadata (single/known URLs) | Low to medium | Datacenter IPv4 |
| Search + channel crawls | High, rotation-heavy | Rotating residential |
| Comments and transcripts, bulk | High, bandwidth-heavy | Unlimited bandwidth SOCKS5 |
| Steady session-based collection | Medium, sticky | Static residential (ISP) |
| Hardest / most sensitive targets | Any | Rotating mobile |
Rotating residential. Rotating residential proxies provide fresh ISP-assigned IPs across 180+ countries, with country and city targeting. Use sticky sessions when a crawl needs one IP for longer. Pay-as-you-go GB billing suits search and channel crawls.
Datacenter IPv4 and IPv6. Datacenter proxies provide high throughput and low per-request costs for public metadata and comment pulls. They use per-IP pricing, unlimited bandwidth, and IP whitelist authentication.
Rotating mobile. Rotating mobile proxies use real carrier IPs across 3G, 4G, and 5G networks in 100+ countries. They offer high IP trust for demanding collection jobs and are billed per GB.
Static residential (ISP). ISP proxies keep the same residential IP for the subscription, making them suitable for scheduled jobs that revisit the same channels. IP whitelist authentication keeps credentials out of your code.
Unlimited bandwidth SOCKS5. Bulk comment and transcript pulls can move substantial amounts of text, making flat per-IP pricing attractive. The unlimited bandwidth line covers 195+ locations over SOCKS5.
The API quota is free but fixed: once the daily 100-search limit is reached, you wait until midnight Pacific Time. Proxy-Cheap plans are pay-as-you-go with no monthly commitment, so you pay for the GB or IPs you use.
To size a plan, log len(resp.content) for 100 requests and multiply the average by your daily record count. Payload sizes vary between watch pages and JSON endpoints, so measure your own workload first. If you are new to proxy types, see our overview of types of proxies.
Add proxies to a Python scraper by passing a proxies dict to requests, or setting the proxy option in yt-dlp. Rotating residential proxies use username/password authentication, while static residential and datacenter lines also support IP whitelisting. For SOCKS5, use socks5h to resolve DNS through the proxy.
With requests. Use the same proxy URL for both schemes. The example uses the US rotating residential gateway. Every function in this guide accepts this proxies dict.
python import requests PROXY = "http://USERNAME:[email protected]:5959" proxies = {"http": PROXY, "https": PROXY} resp = requests.get("https://www.youtube.com/watch?v=dQw4w9WgXcQ", proxies=proxies, timeout=30) print(resp.status_code, len(resp.text))
With yt-dlp. Set proxy in the options dict. The same value works with the --proxy command-line option.
python from yt_dlp import YoutubeDL opts = { "quiet": True, "skip_download": True, "proxy": "socks5h://USERNAME:PASSWORD@HOST:PORT", "sleep_interval_requests": 1, } with YoutubeDL(opts) as ydl: info = ydl.extract_info("https://www.youtube.com/watch?v=dQw4w9WgXcQ", download=False) print(info["title"], info["view_count"])
For SOCKS5 with requests, install requests[socks] first.
Rotation vs sticky sessions. Rotate per request for independent searches and metadata lookups. Use a sticky session when paginating a search or comment thread so all pages use the same IP and location. Rotating residential sticky sessions last about 30 minutes.
SOCKS5 vs SOCKS5H. SOCKS5 resolves DNS locally; socks5h resolves it through the proxy, keeping DNS consistent with the exit location.
Authentication. Rotating residential proxies use username/password. Static residential proxies and datacenter lines support username/password and IP whitelist authentication. Whitelisting is convenient for servers because credentials stay out of your code.
Pacing. Add 1 to 2 seconds between requests and back off on errors to improve reliability and respect the site.
API and support. Manage proxies programmatically through the Proxy-Cheap API docs. For SOCKS5 setup and whitelist limits, see Proxy-Cheap support.
If you do not want to write code, two routes exist: browser extensions that read visible fields into a spreadsheet, and hosted scraper services that run collection in the cloud. For either route, a managed proxy makes collection more reliable at volume. Proxy-Cheap's browser extensions handle proxy switching without any code.
Browser extensions. Scraper extensions read what is on screen, like titles and view counts on a search page, and export it as CSV. They are fast to set up and fine for a few hundred rows.
Hosted cloud scrapers. You paste URLs or keywords into a web form, and the service runs the job on its servers and returns a dataset. You trade control for convenience, and pricing usually scales per result.
No-code fits small, one-off pulls and non-developers, such as a marketer checking a competitor's last 50 uploads. It fits poorly when you need fine-grained control, or when volume climbs into the tens of thousands of records.
If your extension or hosted tool runs in your browser, you can route it through a proxy with no code at all. The Proxy-Cheap Chrome proxy extension and Firefox proxy add-on are free. Both support centralized proxy management with HTTP/SOCKS5 import. You also get one-click profile switching and WebRTC protection.
