Reddit scraper vs API: scrape Reddit without getting blocked

A Reddit scraper that worked in 2024 will probably fail today. In May 2026 Reddit shut down its unauthenticated .json endpoints and started sending logged-out visitors on old.reddit.com to a login page. It stopped issuing self-service API keys in November 2025. Its User Agreement prohibits scraping without written consent. If you need Reddit data from code in 2026, you have five routes: the .json trick, old.reddit HTML, PRAW with approved OAuth credentials, a headless browser with proxies, or a hosted API. Only the last two scale, and only one of those stays stable when Reddit changes something.

This guide walks through each route and why it gets blocked. It also covers the legal and terms-of-service picture, with sources, and gives working code for the hosted route. For background on Reddit's official API first, read our Reddit API guide.

Why your Reddit scraper gets blocked

Three changes in the last year broke most scripts.

Reddit has turned off unauthenticated JSON. On May 28, 2026 it announced in r/modnews that it was "shutting down unauthenticated .json endpoints", because they "can be used to scrape Reddit without accountability." The same post updated Rule 8 ("don't break the site") to cover "API misuse." Logged-in access was left alone.

Self-service API keys are over. Since November 11, 2025, every new OAuth token needs Reddit's explicit approval. On September 30, 2026, Reddit said it stops accepting new public API requests on October 31, 2026 and will close public access for the remaining apps in March 2027.

Old Reddit and RSS are closing too. The same September update in r/modnews announced it was "limiting access to logged-in mods and recent users" on Old Reddit, and that RSS feeds end on November 13, 2026.

On 2026-10-14 we checked what a plain unauthenticated request gets, once with a python-requests User-Agent and once with a desktop Chrome string. The result was the same both times:

That is one IP on one day, so treat it as a snapshot. It does match what developers report. In r/webscraping, u/Dawlphy had a Node.js script making "maybe 30-60 requests per hour" that suddenly started getting 403 errors, and moving to a mobile hotspot didn't help. In r/ClaudeAI, the author of a Reddit MCP server wrote that requests "come back as a 403 'blocked by network security' page" from "a home residential IP, a VPN, and even a paid residential-proxy scraper."

Route 1: the .json endpoints

Most tutorials teach this trick: append .json to a Reddit URL and you get the page back as JSON, with no key. In April 2026, a r/webscraping poster who wanted "the top 50 posts of about 15 subreddits each month" listed it first among their options (thread).

Since May 2026 it no longer works without authentication. Developers in that thread found requests still went through once a logged-in session was attached, and u/Coding-Doctor-Omar summed up the requirement as "TLS + SESSION COOKIES". Borrowing a browser session and faking a browser's TLS fingerprint to get past Reddit's checks is circumvention. It breaks Reddit's terms and puts the account at risk, so we won't cover how to do it.

The JSON shape is still worth learning, because the OAuth API and most hosted APIs return the same Listing objects, with fullnames like t3_ for posts and t1_ for comments (Reddit API reference).

Route 2: old.reddit HTML

Old Reddit's server-rendered HTML was the classic fallback for a Python reddit web scraping script: one request, BeautifulSoup, done. When Reddit closed JSON access, mods in the announcement thread predicted this exact move. u/Signe_ wrote: "I can already see that scrapers are just going to use old reddit and scrape the html instead."

Reddit closed that route as well. In our test, a logged-out request to old.reddit.com returned a 302 redirect to a login page. That matches Reddit's announcement that Old Reddit is limited to logged-in mods and recent users. In June, one r/ClaudeCowork user reported that "the answer ended up being old.reddit.com." Four months later, that answer sits behind a login wall.

Route 3: PRAW and the official API

With approved credentials, this is the clean route. PRAW, the Python Reddit API Wrapper, handles OAuth, rate limit headers and pagination for you. A read-only client needs three things: a client id, a client secret and a user agent.

import praw

reddit = praw.Reddit(
    client_id="YOUR_CLIENT_ID",
    client_secret="YOUR_CLIENT_SECRET",
    user_agent="script:my-research-app:v1.0 (by /u/your_username)",
)

for post in reddit.subreddit("SaaS").top(time_filter="week", limit=25):
    print(post.score, post.num_comments, post.title)

Getting the credentials is the hard part. The Responsible Builder Policy says you must "request access and get explicit approval before accessing any Reddit data," and developers describe the process as slow or silent. In r/redditdev, u/luitron5 said: "I gave up after 2 months." The window also closes on October 31, 2026, when Reddit stops taking new requests. Commercial use needs a separate contract under the Data API Terms.

Our guide to getting a Reddit API key walks through the request in detail, and the Reddit API in Python compares PRAW with a hosted API line by line.

Route 4: headless browsers and proxies

When the endpoints close, scrapers move to a real browser: Playwright, Selenium or Puppeteer driving Chrome, usually behind rotating proxies. "Just use Python Selenium library. It works," one r/webscraping commenter wrote. Others had tried it. u/MadeByHideoForHideo tried Selenium and "immediately get the captcha page when opening the subreddit." In r/n8n, u/Abhi-Age-2050 said their browser agent got their account "cold banned", which locked them out of Reddit on their phone.

Browser scraping costs more than it looks:

• Every page is a full browser render. A mod in the scraping announcement thread pointed out that Reddit "has to render the entire page every single time," and your scraper has to as well.
• Residential proxies are billed by the gigabyte, and pages with comment trees are heavy.
• Reddit changes its markup and your selectors break. You find out when the job returns nothing.
• Scraping while logged in ties the activity to your account.

For a one-off pull of a few pages you can see while logged in, a browser can still make sense. As a production data pipeline, it's the most fragile option on this list.

Route 5: a hosted Reddit API

A hosted API does the collection for you and returns JSON over a normal REST call. You get one key and pay per request. When Reddit changes something, the provider fixes the scraper. Our roundup of the best Reddit scrapers and scraping APIs compares the options, with prices checked on each provider's own pages.

ChatterAPI is one of them, built for developers and AI agents. It is independent and not affiliated with Reddit. Here is the same job as the PRAW example, the top posts in r/SaaS this week, through GET /v1/reddit/subreddits/{name}/posts:

curl "https://api.chatterapi.com/v1/reddit/subreddits/SaaS/posts?sort=top&time=week" \
  -H "Authorization: Bearer $CHATTERAPI_KEY"

Here is the response, trimmed to two posts. It is a recorded sample from a test key, which returns one real response per endpoint for free:

{
  "data": [
    {
      "id": "1wwnhbt",
      "fullname": "t3_1wwnhbt",
      "title": "I've created Open Source 100% Rust clean-room replacements for 7 of Adobe's apps...",
      "author": "ai_art_is_art",
      "subreddit": "SaaS",
      "score": 1773,
      "numComments": 378,
      "createdAt": "2026-10-03T13:37:37.000Z",
      "url": "https://www.reddit.com/r/SaaS/comments/1wwnhbt/ive_created_open_source_100_rust_cleanroom/"
    },
    {
      "id": "1wvjgm2",
      "fullname": "t3_1wvjgm2",
      "title": "I made $7,802 today",
      "author": "OkPositive9373",
      "subreddit": "SaaS",
      "score": 614,
      "numComments": 276,
      "createdAt": "2026-10-02T03:29:02.000Z",
      "url": "https://www.reddit.com/r/SaaS/comments/1wvjgm2/i_made_7802_today/"
    }
  ],
  "page": { "nextCursor": "AdxXcaGWSY8c…", "hasMore": true },
  "credits": { "charged": 0, "remaining": 719 }
}

Here is the same call in Python, paging until you have 100 posts:

import os
import requests

BASE = "https://api.chatterapi.com/v1"
HEADERS = {"Authorization": f"Bearer {os.environ['CHATTERAPI_KEY']}"}

posts, cursor = [], None
while len(posts) < 100:
    params = {"sort": "top", "time": "week"}
    if cursor:
        params["cursor"] = cursor
    r = requests.get(f"{BASE}/reddit/subreddits/SaaS/posts", headers=HEADERS, params=params, timeout=30)
    r.raise_for_status()
    body = r.json()
    posts += body["data"]
    if not body["page"]["hasMore"]:
        break
    cursor = body["page"]["nextCursor"]

for p in posts:
    print(p["score"], p["numComments"], p["title"])

Each page costs 2 credits on a live key. Failed calls cost nothing. Post comments come from GET /v1/reddit/threads/{id} at 3 credits per page, and keyword search comes from GET /v1/reddit/search. If you want to try it without writing code, the free Reddit scraper tool pulls a single post in the browser, and subreddit posts lists a feed. Agents can use the same endpoints through the hosted MCP server at https://api.chatterapi.com/mcp.

The rules on what you can do with the content are the same with a hosted API. What changes is that someone else maintains the collection.

Is it legal to scrape Reddit?

Short answer: it's against Reddit's terms, and the law is unsettled. Read the primary sources rather than a blog's summary of them, including this one.

• Reddit's User Agreement, effective July 1, 2026, says you may not "access, search, or collect data from the Services by any means (automated or otherwise)" except as permitted, and that "scraping the Services without Reddit's prior written consent is prohibited."
• Reddit's robots.txt disallows every path for every user agent and points to its Public Content Policy.
• Reddit sues scrapers. In October 2025 it sued Perplexity, SerpApi, Oxylabs and AWMProxy, alleging they scraped Reddit content from Google results and got around anti-scraping measures. The defendants deny it.
• On the law itself: Eric Goldman, who teaches internet law, ran a Reddit AMA on web scraping in October 2026. His write-up says a scraping campaign "must survive each and every one of the doctrines" that apply, which include the CFAA and state computer crime laws, trespass to chattels, contract, copyright and privacy. He calls it "an area of doctrinal chaos."

This isn't legal advice. If Reddit data is going into a commercial product, ask a lawyer and read Reddit's Data API Terms before you build.

Even AI coding tools pick up on the terms question. An r/webscraping post titled "Claude Code refusing to build scraper" drew 137 comments. The top reply, with 146 upvotes, was the joke "Great news, we got permission from the website."

The case for building your own Reddit scraper

There is an honest argument for rolling your own, and plenty of experienced scrapers make it.

Small jobs may never hit the limits. For the 15-subreddits-a-month job, u/tendie_bot wrote "you wont even come close to triggering reddit WAF." The author of Reddrix, an open-source scraper posted to r/webscraping in October, said they ran it "using my normal home IP and I never had an issue."

You also control the fields. A scraper reads whatever the page renders, including fields an API might not return.

And it's cheap, at least at first. The code is free, and so is a home connection.

Against that: Reddit has closed three access routes in five months, each closure broke working scripts overnight, and the User Agreement prohibits scraping regardless of volume. One mod in the announcement thread expects scrapers to "switch to even more resource-intensive scraping methods" each time, and that keeps happening. If the data feeds something customers rely on, the 2 a.m. breakage costs more than the code does.

Pick the route that fits the job

For a personal one-off, read the thread in your browser. If you already have Reddit API credentials, keep using PRAW and register your app before January 12, 2027. For anything scheduled, commercial or agent-driven, use a hosted API, so that when Reddit changes something again, someone else handles the fix.

Sign up with Google to get 100 free credits with no card. That is enough for about 50 subreddit pages or 33 full threads with comments. Plans start at $19 a month for 25,000 credits.