Twitter List Scraper: tweets from any public X List

Two ways in, and one number that matters

A List is named either by its URL or by the ID at the end of that URL, and the two cost the same. What actually shapes the bill is the 50-tweet minimum each list has to clear.

• startUrls

  $0.008per list

  ~20 tweets included

  Paste the List link. Either x.com or twitter.com form works.

  x.com/i/lists/78783491

• listIds

  $0.008per list

  ~20 tweets included

  The digits at the end of that URL. What to store if the list is going into a database.

  78783491

• What the price table does not say

  the 50-tweet minimum

  $0.02the real floor

  Below 50 risks rate-limiting

  Every list has to fetch at least fifty. Twenty are free, so thirty are always billed. The floor is $0.02 a list, not $0.008.

  $0.008 + 30 × $0.0004

• maxItems

  ÷ $0.0004budget to cap

  Set it every time

  The ceiling, per list. Budget divided by $0.0004 is roughly what to put here.

  1000

The listing is candid about the floor and about what to do below it: for jobs smaller than fifty tweets it points you at Twitter Scraper Unlimited instead, which prices single queries rather than lists.

One list, many lists, or the people on them

Three shapes of run cover what this actor is for: read one curated List, sweep a set of them in a single pass, or mine the author objects to get at the members behind the tweets.

{
  "startUrls": ["https://x.com/i/lists/78783491"],
  "maxItems": 500
}

{
  "type": "tweet",
  "id": "1757983181972930764",
  "url": "https://x.com/ESPNStatsInfo/status/1757983181972930764",
  "text": "Anthony Davis and Rui Hachimura are the 1st @Lakers duo…",
  "retweetCount": 58,
  "replyCount": 10,
  "likeCount": 158,
  "quoteCount": 8,
  "bookmarkCount": 7,
  "createdAt": "Thu Feb 15 04:20:26 +0000 2024",
  "lang": "en",
  "isReply": false
}

{
  "listIds": ["78783491", "1657850814910590977"],
  "maxItems": 2000
}

{
  "isRetweet": false,
  "isQuote": false,
  "media": [
    "https://pbs.twimg.com/media/GGWb69-a0AAj1iA.jpg"
  ],
  "author": {
    "userName": "ESPNStatsInfo",
    "followers": 1828908,
    "isVerified": true,
    "isBlueVerified": true
  }
}

{
  "startUrls": ["https://x.com/i/lists/78783491"],
  "maxItems": 1000
}

"author": {
  "type": "user",
  "userName": "ESPNStatsInfo",
  "id": "53120768",
  "name": "ESPN Stats & Info",
  "description": "Our mission is to serve fans…",
  "location": "Bristol, CT",
  "followers": 1828908,
  "following": 61,
  "protected": false,
  "createdAt": "Thu Jul 02 17:03:54 +0000 2009",
  "mediaCount": 46266,
  "statusesCount": 162931,
  "verifiedType": "business"
}

A tweet, and whoever wrote it

One shape. The listing publishes an availability guarantee for fifteen of its fields: fourteen are always present and only media is conditional.

• 6

  The tweet

  All guaranteed present

  • id
  • url, twitterUrl
  • text
  • createdAt
  • lang

• 5

  Engagement

  All guaranteed present

  • likeCount, retweetCount
  • replyCount
  • quoteCount
  • bookmarkCount

• 12

  The author

  Nested on every tweet

  • userName, id, name
  • description, location
  • followers, following
  • statusesCount, mediaCount
  • isVerified, isBlueVerified
  • verifiedType, protected

• 4

  Kind and media

  media is the one conditional field

  • isReply, isRetweet
  • isQuote
  • media[], URLs if present
  • author.profilePicture, coverPicture

The author block is the reason this scraper doubles as member research: every tweet carries twelve fields about whoever posted it, so one list run yields both the content and a profile for each active member without a second call.

Where a run goes wrong

Five rules, and the listing states four of them itself. The fifth is what its own price table leaves out.

• 01

  Fifty tweets minimum, per list

  Anything smaller creates a negative margin and may trip the rate limiter. The listing’s own recommendation for smaller jobs is to use Twitter Scraper Unlimited instead, which bills per query rather than per list.

• 02

  So the floor is $0.02, not $0.008

  Fifty tweets with twenty included leaves thirty billed. The headline $0.008 is the query fee, not what a compliant run can actually cost.

• 03

  Private Lists are unreachable

  Authentication is not supported, so a private List returns an empty dataset rather than an error. Check visibility first when a run comes back with nothing.

• 04

  One or two concurrent runs

  The listing asks for no more than two at a time, with a few minutes between large extractions.

• 05

  customMapFunction must not filter

  Using it to drop records causes automatic blocking, which the listing lists twice, once under troubleshooting and once under what not to do.

Free Apify accounts get 5 runs a month, capped at 10 items each.

Flat per tweet, once you clear the floor

Every list is $0.008 with roughly 20 tweets inside it, then $0.0004 a tweet. Past the 50-tweet minimum the rate never changes, whether the run reads one list or a hundred.

All six figures the listing publishes reproduce exactly from its two charge events, which is rarer than it should be. The first row is this page’s addition: the minimum run, which the price table does not price.

Who reads other people’s Lists?

Five kinds of work treat a curated List as a ready-made panel: lead generation, competitor intelligence, journalism, investor tracking and recruiting. Somebody already did the curation, which is the whole appeal.

• Lead generation

  Get the members, on industry Lists

  Mine the author objects for handles, bios and follower counts from a list someone else already qualified

  $0.40 per 1,000 tweets

• Competitor intelligence

  Sweep several, on a rival’s public Lists

  Read the Lists a competitor curates to see who they consider worth following

  $2.00 per 5 lists

• Journalists

  Read one List, scheduled

  Follow a beat through a hand-picked panel instead of an algorithmic timeline

  $0.02 per check at the minimum

• Investor and VC tracking

  Read one List, on a portfolio List

  Watch a portfolio or sector panel in one feed, with each poster’s reach attached

  $0.20 per 500 tweets

• Recruiting

  Get the members, on conference Lists

  Turn a speaker or attendee List into a sourcing pipeline with bios and locations

  $4.00 per 10,000 tweets

Scripting a List pull

The actor is apidojo/twitter-list-scraper. With four inputs there is little to configure, so the work in code is on the way out: de-duplicating authors, and keeping maxItems above the fifty-tweet floor.

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

# maxItems below 50 per list risks the rate limiter, per the listing.
run = client.actor("apidojo/twitter-list-scraper").call(run_input={
    "listIds": ["78783491", "1657850814910590977"],
    "maxItems": 2000,
})

# Members arrive as tweet authors, so de-duplicate to get the roster.
members = {}
for t in client.dataset(run.default_dataset_id).iterate_items():
    a = t["author"]
    members[a["userName"]] = a

for handle, a in sorted(members.items(), key=lambda kv: -kv[1]["followers"]):
    print(handle, a["followers"], a["isBlueVerified"], a.get("location", ""))

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });

const run = await client.actor('apidojo/twitter-list-scraper').call({
  startUrls: ['https://x.com/i/lists/78783491'],
  maxItems: 1000,
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();

// Who carries the List: share of its tweets, by author.
const byAuthor = Object.groupBy(items, (t) => t.author.userName);
const share = Object.entries(byAuthor)
 .map(([h, ts]) => [h, ts.length, (100 * ts.length) / items.length])
 .sort((a, b) => b[1] - a[1]);
console.log(share.slice(0, 10));

curl -X POST \
  "https://api.apify.com/v2/acts/apidojo~twitter-list-scraper/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"listIds":["78783491"],"maxItems":1000}'

Results land in an Apify dataset, downloadable as CSV or readable over the API. Keep concurrent runs to one or two, which the listing asks for explicitly, and leave a few minutes between large extractions.

Twitter List Scraper FAQ

Where Twitter List Scraper fits