How to Scrape Twitter Search Results

To scrape Twitter search results, copy the query you ran on X, or the search results page URL, into apidojo’s Tweet Scraper V2 on Apify, and export the posts it returns. The actor accepts any X advanced search query in searchTerms and any search URL in startUrls, at $0.40 per 1,000 tweets.

X search has no export button, so this is the direct route from a search you already built to a dataset. Every operator you can use is in the Twitter advanced search operators reference. Every other collection task is mapped in how to scrape Twitter (X) data.

How Do You Turn an X Search Into a Dataset?

You turn an X search into a dataset by running the same query through Tweet Scraper V2 and downloading the run’s results. Listed below are the 5 steps.

1. Build the search on X and check that the results look right; the actor fetches what that search shows (Apify).
2. Copy the query from the search bar, or the full URL from the address bar.
3. Paste the query into searchTerms, or the URL into startUrls.
4. Choose sort: Latest for time order, Top for engaged posts, Latest + Top for the most results.
5. Run the actor and export the dataset as JSON, CSV or Excel.

Should You Use a Search Query or a Search URL?

Use searchTerms for queries you write or generate, and startUrls when you already have X search result URLs to paste. Both return the same post data; queries are easier to edit and batch (Apify).

{
  "searchTerms": ["(crm OR pipeline) -salesforce min_faves:10 lang:en"],
  "sort": "Top",
  "maxItems": 2000
}

{
  "startUrls": ["https://x.com/search?q=%23buildinpublic%20min_faves%3A50&f=live"],
  "maxItems": 2000
}

In a search URL, f=live is the Latest tab. If you don’t know the operator syntax, the actor’s Query Wizard builds queries from filters such as author, date range and engagement minimums.

Which Search Filters Can the Actor Apply for You?

Tweet Scraper V2 has input fields for the most common search filters, so you can apply them without writing operators. Listed below are the 9 filter inputs (Apify).

Input	Same as
tweetLanguage	lang:
author	from:
inReplyTo	to:
mentioning	@username
minimumFavorites	min_faves:
minimumRetweets	min_retweets:
minimumReplies	min_replies:
onlyImage, onlyVideo, onlyQuote	filter:images, filter:videos, filter:quote
onlyVerifiedUsers	filter:verified

What Data Does a Scraped Search Result Contain?

Each scraped search result contains the post, its engagement counts and its author. The main fields from the actor’s output example are listed below (Apify).

Field	Contains
id, url	The post ID and link
text, lang	The post text and detected language
createdAt	Post time, for example Fri Nov 24 17:49:36 +0000 2023
likeCount, retweetCount, replyCount, quoteCount, bookmarkCount	Engagement counts
isReply, isRetweet, isQuote	Post type
author	userName, name, followers, following, isVerified, isBlueVerified

Saving these results to a file is covered in how to export tweets to CSV or JSON.

What Limits Apply to Scraping Search Results?

Each Tweet Scraper V2 query must return at least 50 tweets, one run can batch up to 5 queries, and only one run can run at a time. The same query must not be repeated at short intervals (Apify). Small searches go to Twitter Scraper Unlimited, and long periods to how to scrape tweets by date range.