Skip to content

Reddit dataset: how to build one from live Reddit data

How to build a Reddit dataset: pick communities and queries, collect posts and comments as JSON, page through results, store the date, and what to check before use.

Piyushh · November 7, 2026 · 2 min read

A Reddit dataset is a table of Reddit posts or comments you collected for a purpose: a model, a study, a market scan. You can download an archive, use a published dataset, or build your own from live data. If you build your own, you choose the exact communities and dates you need.

Three ways to get Reddit data

Plan the collection first

Write down which subreddits you pulled, the search words, the time window, and why you chose them. That list is what makes your dataset defensible. If you plan to train a model on it, check the terms that cover the content and your use of it. This is not legal advice.

Collect posts and page through results

curl "https://api.chatterapi.com/v1/reddit/search?q=data+engineering&time=year" \
  -H "Authorization: Bearer $CHATTERAPI_KEY"

Each response has a data array and a page.nextCursor. Loop until hasMore is false. Save the raw JSON with the date you ran it, since scores and comment counts change over time.

To cover a whole community, call subreddit posts. For comments, call a thread for each post you care about. Data science with the Reddit API covers what to do next.

Turn it into a table

In Python, load the saved responses and run pandas.json_normalize on the data array. Fields such as score, numComments and createdAt become columns. A post can come back from more than one query, so deduplicate on the post id.

What to check before you use it

  • Removed content is missing, which can bias the sample.
  • Scores are a snapshot from the day you collected them.
  • Posts about identifiable people may need an ethics review.

To export a single thread to a file without writing code, use the Reddit comment exporter. Methods notes are covered in the academic research use case.

FAQ

Where can I get a Reddit dataset?
You can build your own from live data, use a public archive, or use a published research dataset. Each has different coverage and different terms.
How do I build a Reddit dataset with an API?
Search or read a subreddit, page through the results with the cursor, save each raw response with its date, and flatten the posts into a table.
Does a dataset from live data include deleted posts?
No. Removed and deleted posts are not returned, so counts can differ from what the thread showed earlier.

More on Reddit API

Try it free for 7 days.

1,000 credits to make your first calls. One key for Reddit and X, over REST or MCP.