How to Collect Tweets With Python

To collect tweets with Python, install the apify-client package, run apidojo’s Tweet Scraper V2 with an X search query, and iterate over the run’s dataset. Ten lines of code return posts with text, date, author and engagement at $0.40 per 1,000 tweets, with no X developer account.

This guide builds a complete Python collector: setup, a first run, parsing dates, date-window batching, deduplication and saving to CSV. The actor choice is explained in how to scrape Twitter data in 2026.

What Do You Need to Collect Tweets With Python?

You need Python 3, the apify-client package, and an Apify API token from a paid Apify plan. Tweet Scraper V2 can’t be called through the API on the free plan (Apify). Listed below are the 3 setup steps.

1. Install the client: pip install apify-client pandas.
2. Copy your API token from Apify Console.
3. Store it in an environment variable: export APIFY_TOKEN=your_token.

How Do You Run a First Tweet Collection in Python?

You run a first collection by calling the actor with a searchTerms query and reading items from its default dataset. The field names below come from the actor’s output example (Apify).

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("apidojo/tweet-scraper").call(run_input={
    "searchTerms": ["from:NASA -filter:retweets"],
    "sort": "Latest",
    "maxItems": 500,
})

for tweet in client.dataset(run.default_dataset_id).iterate_items():
    print(tweet["createdAt"], tweet["likeCount"], tweet["text"][:80])

call() starts the run and waits for it to finish. Every X search operator works in the query; see the Twitter advanced search operators reference.

How Do You Parse Tweet Dates in Python?

Tweet Scraper V2 returns createdAt in the format Fri Nov 24 17:49:36 +0000 2023, which datetime.strptime parses with %a %b %d %H:%M:%S %z %Y.

from datetime import datetime

def parse_created_at(value: str) -> datetime:
    return datetime.strptime(value, "%a %b %d %H:%M:%S %z %Y")

print(parse_created_at("Fri Nov 24 17:49:36 +0000 2023"))

How Do You Collect a Large History in Python?

Collect a large history by generating one query per date window and running them in batches of 5, the maximum per Tweet Scraper V2 run. Store posts in a dictionary keyed by id to remove duplicates (Apify).

from datetime import date

def month_windows(start: date, end: date):
    current = start
    while current < end:
        nxt = date(current.year + current.month // 12, current.month % 12 + 1, 1)
        yield current, min(nxt, end)
        current = nxt

queries = [f"from:NASA since:{a} until:{b}" for a, b in month_windows(date(2025, 1, 1), date(2026, 1, 1))]
tweets = {}

for i in range(0, len(queries), 5):
    run = client.actor("apidojo/tweet-scraper").call(run_input={
        "searchTerms": queries[i:i + 5],
        "includeSearchTerms": True,
        "sort": "Latest",
    })
    for item in client.dataset(run.default_dataset_id).iterate_items():
        tweets[item["id"]] = item

print(len(tweets), "unique tweets")

Window sizes are discussed in how to scrape tweets by date range, and deduplication in Twitter scraper pagination.

How Do You Save Tweets to CSV With pandas?

Flatten the posts with pandas.json_normalize, keep the columns you need, and write a CSV. Nested author fields become columns such as author.userName.

import pandas as pd

df = pd.json_normalize(list(tweets.values()))
columns = ["id", "url", "createdAt", "author.userName", "text",
           "likeCount", "retweetCount", "replyCount", "quoteCount"]
df[columns].to_csv("tweets.csv", index=False)

Keep id as text in spreadsheets so the 19-digit IDs aren’t rounded. Other formats are covered in how to export tweets to CSV or JSON.

Which Other apidojo Actors Work From Python?

Every apidojo actor runs from Python the same way; only the actor ID and input change. Listed below are the 4 you are most likely to need.

Task	Actor ID	Key input
Small queries, single posts, threads	apidojo/twitter-scraper-lite	searchTerms, startUrls, conversationIds
Replies to posts	apidojo/twitter-replies-scraper	Post IDs or URLs
Profiles, followers, following	apidojo/twitter-user-scraper	twitterHandles, getFollowers, getFollowing
Profile timelines with replies	apidojo/twitter-profile-scraper	Profiles

Threads in Python are shown in how to scrape a Twitter thread and all its replies. The official X API in Python, with requests and a bearer token, is covered in how to get historical tweets with an API.