Skip to content
AgentsIndex

Crawl4AI

A crawler for clean Markdown and typed JSON extraction from pages for RAG pipelines.

Best for
Developers who need LLM-ready web data with browser control and self-hosting.
GitHub stars
85.6k

What we checked

Pages last read
Facts checked
5 facts

From 8 sources

AI answers
Too few read

58 buyer-question answers read, November 2026

At a glance

Tags
Open source, Self-hosted

Code

Last push
15 October 2026

For developers

Works with
Claude, Cursor, Windsurf

What is Crawl4AI?

Crawl4AI is an open-source web crawling tool for developers who need LLM-ready data with browser control and self-hosting. It combines Generate Clean Markdown, Structured Extraction, Advanced Browser Control, and Adaptive Crawling, with hooks, proxies, stealth modes, and session reuse for harder pages. It integrates with CSS/XPath and LLM-based extraction paths, and the project is documented around AsyncWebCrawler workflows. The repo shows 65.7k GitHub stars and 6.7k forks.

What it actually does well

Crawl4AI is an async Python crawler (Playwright-based, also usable via a CLI or a Docker/FastAPI server) built specifically for feeding LLMs and RAG pipelines rather than general-purpose scraping. Its markdown generator applies heuristic content filtering (a pruning filter and a BM25 filter keyed to a query) to strip nav/boilerplate before the page ever reaches an LLM, and it ships three extraction paths — CSS/XPath schemas, LLM-based extraction against a Pydantic schema, and a table-specific LLM extractor with automatic chunking for large tables. Version 0.7.0 added 'adaptive crawling', which stops crawling a site once an information-sufficiency threshold is met rather than exhausting a fixed page budget, and 0.8.0 added crash-recoverable deep crawls (BFS/DFS/best-first with resumable state). It needs no API key or account to use at all — everything runs on infrastructure you control.

Adoption and maintenance

The PyPI package (github.com/unclecode/crawl4ai) has been downloaded 18.5 million times all-time and 1.64 million times in the trailing 30 days (pepy.tech/projects/crawl4ai, checked today); the Docker image has 3.21 million pulls (Docker Hub API). Development is active day-to-day, not just at release time, against a steady release cadence (v0.9.0 on 18 June 2026, v0.9.1 on 8 July 2026, v0.9.2 on 15 July 2026). This is a genuinely live, high-traffic project, not an abandoned star-farm.

The Docker API's security history

If you self-host the Docker deployment (rather than the plain pip library), know that it has twice shipped a critical, unauthenticated remote-code-execution flaw with a maximum CVSS score of 10.0. CVE-2026-26216 (disclosed Jan 2026, fixed in v0.8.0) let an unauthenticated caller smuggle Python code into the /crawl endpoint's hooks parameter and run arbitrary system commands via exec(). CVE-2026-57572 (disclosed mid-2026, fixed in v0.9.0) let an unauthenticated caller inject Chromium launch switches through browser_config.extra_args to spawn an attacker-controlled process — a partial denylist fix in v0.8.9 didn't fully close it. Both are documented candidly in the project's own SECURITY.md, which also lists a third fixed deserialization RCE (v0.8.1) and commits to a 48-hour acknowledgment and severity-tiered fix timeline. v0.9.0 (18 June 2026) changed the default posture: the Docker API now requires authentication and binds to loopback unless a token is explicitly configured, rather than shipping open by default. (Sources: github.com/advisories/GHSA-5882-5rx9-xgxp, ionix.io/threat-center/cve-2026-57572, github.com/unclecode/crawl4ai/blob/main/SECURITY.md)

Cost and what's actually priced

The library and Docker image are free under Apache 2.0 — no seat fees, no usage caps, no account required. The project's only current monetization is GitHub Sponsors, with published tiers from $5/month (support the project) to $2,000/month (a 'Data Infrastructure Partner' tier with dedicated support) — these buy sponsor recognition and, at higher tiers, direct maintainer access, not product features (github.com/sponsors/unclecode). A hosted 'Crawl4AI Cloud API' is announced as a closed beta with no public pricing; access is by application through a Google Form, and the vendor's own framing — 'drastically more cost-effective than any of the existing solutions' — is an unverified marketing claim with no numbers behind it yet.

Who's behind it, and what that means for continuity

Crawl4AI is led by Hossein Tohidi ('Unclecode' on GitHub/X), who is also the founder of Kidocode, an ed-tech company that appears in Crawl4AI's own sponsor list as a Gold-tier sponsor (github.com/unclecode/crawl4ai README). We found no evidence of institutional (VC) funding behind Crawl4AI itself, and no public information on team size beyond the maintainer and a small number of named contributors thanked in SECURITY.md. This is, as far as we can establish, a single-founder open-source project rather than a funded company — which is consistent with its extremely active commit history but also means there is no established institutional continuity plan if the maintainer stops.

One thing to check before you rely on their attribution language

The README states that using Crawl4AI 'requires' one of two attribution methods — a badge or a text credit — in your own documentation or product. The Apache 2.0 license itself (github.com/unclecode/crawl4ai/blob/main/LICENSE) only obligates preserving existing copyright and license notices when you redistribute the licensed source; it does not require a public 'powered by' badge on a downstream product built with the library. Teams with legal review processes should treat this as the maintainer's request, not a license term, when deciding whether to comply.

Independent reviews are thin

We could not find neutral, non-vendor-authored comparisons of Crawl4AI. Most comparison content found in search (Apify's blog, Bright Data's blog, Firecrawl's own 'alternatives' page, spider.cloud's benchmark post) is published by a competing scraping vendor and should be read as competitor marketing, not independent testing. We found no rating on Trustpilot, G2, or a comparable independent review site.

Crawl4AI price and terms

Terms

Free plan
YesYes
The terms in full

The Free hosted plan is presented as a standing no-card plan with 1,000 credits per month, 10 requests per minute, and two concurrent requests.

For AI agents

Developer docs

Where these prices come from

crawl4ai.com, checked

All plans and limits

PlanPriceConcurrencyCreditsRate limit
Free$0 a month2 at a time1,000 every month10 a minute requests/minute
Starter$10 a month10 at a time10,000 a month120 a minute requests/minute
Builder$99 a month20 at a time125,000 a month600 a minute requests/minute
Scale$499 a month50 at a time750,000 a month1,500 a minute requests/minute
EnterprisePrice on requestNot statedNot statedNot stated

Read from crawl4ai.com.

What we checked on Crawl4AI

YesPartlyNo

What it costs to startPartly

What the price counts

The hosted offering combines fixed monthly plan charges with included credits and separately priced extra credits.

Money enters through a monthly plan, and extra credits on a plan.

crawl4ai.com/docsChecked

What it costs to startYes

Free to use

A standing hosted Free plan costs nothing, and the self-hosted crawler is also available without payment.

$0 · no card

crawl4ai.comChecked

How you build with itYes

Public API

The hosted service documents an authenticated HTTP interface for crawling, extraction, search, and related operations.

One key, plain JSON, one fast endpoint.

crawl4ai.com/docsChecked

Crawl4AI alternatives

Questions about Crawl4AI

How much does Crawl4AI cost?
Crawl4AI has a free plan. Its paid plans include Starter at $10 a month, Builder at $99 a month and Scale at $499 a month. From crawl4ai.com/#pricing, checked 8 November 2026.
Does Crawl4AI have an API or an MCP server?
Crawl4AI has an API. From crawl4ai.com/docs, checked 29 October 2026. Crawl4AI has an MCP server. We checked api.crawl4ai.com/.well-known/oauth-protected-resource ourselves on 15 November 2026.
Is Crawl4AI open source, and can you host it yourself?
Crawl4AI is open source. We checked github.com/unclecode/crawl4ai/blob/main/LICENSE ourselves on 15 November 2026. You can host Crawl4AI yourself.
What have we checked on Crawl4AI?
5 facts, under these headings: What it costs to start, how you build with it.
How do we check Crawl4AI?
We hold 5 facts on Crawl4AI, quoted from 8 pages. Facts last checked 15 November 2026. Price last checked 8 November 2026. Last change found 15 November 2026.

How we check every listing

Sources and changes

Last full check

Sources on crawl4ai.com6

What changed1

  • What it does

Repository

Licence
Apache-2.0
Stars
85,621
Last push
15 October 2026

Crawl4AI price history

Crawl4AI had a free plan, and paid plans from $10 a month, on all 4 of our reads, from 29 October 2026 to 8 November 2026.

PricePlanFirst seenLast seenReadingsSource
$10 a monthCheapest paid plan29 October 20268 November 20264Pricing page, read 29 October 2026

From our research, read twice since 29 October 2026, and 2 of our own checks since 30 October 2026. 2 more of our checks could not pick this price out of the page and are left out. A date is when we read the page, not when the price changed.

Similar in Agent tools and integrations

See all in Agent tools and integrations

From our research

All research

Is Crawl4AI yours?

Free, and the page then carries the verified owner mark. You confirm it with an email at crawl4ai.com. Claim this listing

Report an error if a fact is wrong.