Skip to content

Twitter dataset: how to collect and save X posts for analysis

How to build a Twitter dataset from live X data: define the query, collect posts as JSON, page with cursors, save each run with its date, and what to check first.

Piyushh · November 7, 2026 · 1 min read

A Twitter dataset is a set of X posts you collected for analysis: sentiment, topics, a class project. You can use a published dataset or collect your own. If you collect your own, you choose the query and can record the day you ran it.

Collect posts by keyword

curl "https://api.chatterapi.com/v1/x/tweets/search?q=open+source+llm" \
  -H "Authorization: Bearer $CHATTERAPI_KEY"

The response holds a data array of posts, each with its text, author, time and engagement counts, plus a page.nextCursor. Keep paging with the cursor until you have enough rows. For time filters, see search tweets by date.

Save it properly

Store the raw JSON along with the date and the exact query, because a methods section needs both. Flatten with pandas.json_normalize and deduplicate on the post id.

Other sources of posts

For student or research work, see Twitter API for students and for academic research. If you're writing code, see Twitter API in Python.

Limits

Results are whatever X returns on the day you make the call, so deleted posts and protected accounts won't be in them. Check your institution's rules before you collect posts about named people.

FAQ

Can I download a Twitter dataset?
You can use a published dataset or build your own from live X data. Published datasets are fixed and may carry their own terms.
How do I collect tweets for a dataset?
Run a keyword search, page through results with the cursor and save each raw response with its date.
Are deleted tweets included?
No. Deleted posts and protected accounts are not returned.

More on X (Twitter) API

Try it free for 7 days.

1,000 credits to make your first calls. One key for Reddit and X, over REST or MCP.