Firecrawl Release Notes

Today we're launching the Life Sciences category in the Firecrawl Research Index. It covers 41M+ papers across the drug discovery, clinical trial, and biology literature. Your AI agents get citable papers back for a query and can pull the full text on demand.

The whole Research Index is also free to use now. That includes the AI and ML literature it already carried alongside the new Life Sciences papers.

How the Life Sciences category makes research easier

• High-recall retrieval. The index hits 90% recall@10 on our paper-retrieval eval, so your AI agents find more of what matters on the first page of results.
• Millions of life sciences papers. The corpus holds 41M+ papers from authoritative sources and refreshes daily, so your AI agents only cite domain-specific literature.
• Abstract to full text. Your AI agents search abstracts to find the right papers, then pull the full text to verify a claim against the source.
• No DIY stack. Research Index stands in for the source APIs plus the parsing and ranking you would otherwise build yourself.

What you can use it for

• Build and improve life sciences models. Your AI agents call the index to support internal work on chemical compounds and predictive biology.
• Power a research platform's answers. Use the index as the backend for your platform. A user asks a question and the index returns the papers and passages your platform needs to answer it.
• Run targeted academic and clinical research. Point the research harness you already use at the index and your AI agents retrieve life sciences literature scoped to your objective.

Try it today

The Life Sciences category is live now in Firecrawl Research Index through the API at /search/research, plus the CLI, MCP, and SDKs. Every category is free to query.

Original source

The Life Sciences category is now available in Firecrawl Research Index. It covers 41M+ papers across the drug discovery, clinical trial, and biology literature. Your AI agents get citable papers back for a query and can pull the full text on demand. The whole index is also free to use now.

Highlights

• High-recall retrieval. The index hits 90% recall@10 on our paper-retrieval eval, so your AI agents find more of what matters on the first page of results.
• Millions of life sciences papers. The corpus holds 41M+ papers from authoritative sources and refreshes daily, so your AI agents only cite domain-specific literature.
• Abstract to full text. Your AI agents search abstracts to find the right papers, then pull the full text to verify a claim against the source.
• No DIY stack. Research Index stands in for the source APIs plus the parsing and ranking you would otherwise build yourself.
• Free to use. Every category is free, the AI and ML literature it already carried alongside the new Life Sciences papers.
• Available everywhere you build. Query it through the API at /search/research, plus the CLI, MCP, and SDKs.

Read the full blog here.

Original source

Firecrawl pdf-inspector

What is Firecrawl pdf-inspector

pdf-inspector is a from-scratch Rust library for PDFs, with 1.9M views and 8.1k GitHub stars on the repo at time of writing. It reads a PDF's internal structure (font encodings, text operators, image coverage) in milliseconds, without rendering anything, and decides per page whether the content is text-based or needs OCR.

• Text-based pages get native extraction directly, with reading order preserved.
• Scanned or image-heavy pages are flagged with the reason, so a vision pipeline can pick them up.

For a fully text-based PDF, pdf-inspector alone is the entire pipeline. For mixed documents, it's the smart router that decides what actually needs a GPU.

Why it matters on its own

Most PDF pipelines make the same wrong bet: treat every page as if it might be scanned, so route everything through OCR. That's slow, expensive, and often less accurate than the native text that was sitting in the PDF the whole time.

pdf-inspector fixes the routing layer:

• Per-page classification. Analyzes internal structure only. No rendering, no GPU, milliseconds per page.
• Native text extraction. Pulls text directly from text-based pages with reading order intact.
• Clean handoff for the rest. For scanned pages, it hands back the page reference and the reason, so an OCR pipeline can do the heavy lifting only where it's needed.

That routing layer is what makes Fire-PDF, Firecrawl's hosted PDF parsing engine, 3.5x to 5x faster than the previous pipeline: for a 200-page report where 150 pages are pure text, 150 pages skip GPU entirely.

How to use it

Add pdf-inspector to your Rust project directly from the repo. The README covers the classifier and native-extraction APIs, plus the reproducible-results branch if you want to benchmark it against your own PDFs.

Or don't integrate it directly at all: send PDFs to Firecrawl's /parse or /scrape and they go through pdf-inspector (and Fire-PDF for the scanned pages) automatically.

Firecrawl AnyDoc

What is Firecrawl AnyDoc

AnyDoc is the other half of the stack, also a from-scratch Rust library, also markdown out. It converts documents with a single call:

anydoc::to_markdown("file.docx")

It supports 14 formats in one binary:

• docx
• doc
• docm
• xlsx
• xls
• xlsm
• pptx
• ppt
• rtf
• odt
• ods
• odp
• epub
• csv

No API key. No system dependencies. Nothing to install alongside it.

Why it matters

No single existing library reliably covers every common document format. Each one handles a subset, and the formats it doesn't cover become someone else's dependency, with different output, different failure modes, and often much slower conversions. AnyDoc handles all 14 in one dependency-free library.

Who feels this most: the engineer whose users upload "whatever they have": a .docx contract, an .xls export from 2009, a pitch deck, an .epub. Today, getting usable text out of all of it is a plumbing project. AnyDoc is the part that stops being plumbing.

It's also quickly become a dev favorite. Here's Garry Tan on it:

[Embedded Tweet]

Here's Nick, Firecrawl's Co-founder and CTO, on how AnyDoc is different from pdf-inspector:

[Embedded Tweet]

How it compares

We benchmarked AnyDoc against the common alternatives on 94 documents spanning all 14 formats.

• Coverage: 14 of 14. AnyDoc is the only library in the benchmark that parses every format. The nearest alternative, LibreOffice, covers 12 of 14. Every other option covers a subset, which is exactly why teams end up stitching multiple libraries together.
• Speed: 4.6ms median per document. Across all 14 formats, AnyDoc's median conversion time is 4.6ms. The alternatives run between 52ms and 1,130ms, roughly 20x to 245x slower depending on the tool. LibreOffice sits at the slow end of that range.
• Quality: highest overall, with honest caveats. Quality is LLM-judged on completeness, structure, formatting, and cleanliness. AnyDoc scored highest on every format we tested, overall score 80 vs 68 for the next-best option. Two notes worth being upfront about: the corpus is ours, and mammoth scores higher than AnyDoc on completeness alone (95 vs 87) on the single format it supports (docx). If completeness on docx is the only thing you care about, that gap matters. If you care about coverage across every format your users actually upload, AnyDoc wins the aggregate.

How to use it

Add AnyDoc to your Rust project, then call it on any supported document:

use anydoc;
let markdown = anydoc::to_markdown("contract.docx")?;

That's the whole integration. No API key, no external service, no separate binary. JavaScript bindings are in progress; we'll share those when they land.

Or don't integrate anything at all: Firecrawl's /parse and /scrape endpoints already use AnyDoc automatically when they encounter a non-PDF document.

Two libraries, one shape

Same principles across the pair: from-scratch Rust, local execution, no API key, no system dependencies, markdown out. That's not a coincidence. It's the shape developers kept asking for after pdf-inspector shipped, and it's the shape that makes both libraries safe to drop into any pipeline without dragging in a heavy runtime.

The two are deliberately separate repos and deliberately separate products. AnyDoc is not "pdf-inspector grown up." pdf-inspector handles PDFs. AnyDoc handles everything else. One lineage, two Firecrawl products, and together they cover the document formats a real AI pipeline actually sees.

Try the stack

• pdf-inspector (PDFs): github.com/firecrawl/pdf-inspector
• AnyDoc (14 non-PDF formats): github.com/firecrawl/anydoc
• Use both via API: send any document to /parse or /scrape. PDFs go through pdf-inspector (and Fire-PDF); everything else goes through AnyDoc. No configuration.

If you're already stitching four libraries together to cover the document formats your users upload, we'd love to hear how the two of these hold up as a replacement.

Original source

Firecrawl is now available as an official ChatGPT plugin. You can install it directly inside ChatGPT and give it access to live, clean web data in seconds.

Getting started

Install the plugin in three steps:

1. Open ChatGPT
2. Go to the Firecrawl plugin page and click Install
3. Connect your Firecrawl account when prompted

That's it. ChatGPT now has direct access to live web data across every conversation and inside Codex.

What you can do

Once connected, the plugin gives ChatGPT a full web data toolkit:

• Search the web and get back the excerpts that best answer your query, with full page content included in the results.
• Scrape any page or document (PDF, DOCX, and more) and turn it into clean, structured data.
• Crawl entire sites to build datasets and knowledge bases.
• Interact with dynamic pages to reach content that regular scraping misses (logins, forms, pagination, infinite scroll).
• Monitor pages and get alerts whenever something on the web changes. Monitors can post directly to Slack, so the alert lands where your team already works.

You don't need to remember any commands. Just ask in plain language and ChatGPT picks the right Firecrawl tool automatically. The plugin also ships with a set of ready-to-use skills, so you can kick off common tasks (searching the web, scraping a page, crawling a site, running an interaction flow) in one click without writing the prompt from scratch.

Example prompts

Search the web for this week's biggest AI announcements

Scrape https://news.ycombinator.com and list the top 10 stories with links and points

Interact with Campspot.com to find campsites near Yosemite available Aug 15 to 17

Monitor https://openai.com/blog for new posts and send updates to my #product-updates Slack channel

Why this matters

ChatGPT is only as fresh as the web it can reach. Its built-in browsing gives you snippets and summaries, but the moment you need real answers grounded in real sources, you hit walls: shallow search results with no page content, noisy and unstructured pages, JavaScript that renders inconsistently, and messy HTML that ChatGPT can't parse properly. Firecrawl fixes both sides of that problem.

Firecrawl search runs the query, picks the best results, and hands ChatGPT the full page content along with the links. You get the excerpts that answer the question instead of ten titles and a guess. For pages or sites you already have in mind, Firecrawl handles JavaScript, dynamic content, PDFs, and DOCX, and gives ChatGPT clean markdown or structured JSON to work with.

Together, ChatGPT can go from a vague question to a well-sourced answer without you leaving the conversation. Same inside Codex: search for how something works, then pull the exact docs it needs to write and reason about your code.

Example

Search the web for the latest guidance on receiving Stripe events in a webhook endpoint. Pull the relevant Stripe docs and implement it in my app.

No more copy-pasting URLs into separate tools. No more writing custom scrapers. No more prompting around stale search snippets. Just ask ChatGPT to find and pull the data you need, and it works.

Use cases

A few ways developers, marketers, growth engineers, and sales folks are already using the plugin:

• Live research with real sources: Search the web for what's happening right now and get back the actual page content, not just links or stale summaries.
• Answer engine over any topic: Ask a question in plain language, let Firecrawl search and pull the top results, and have ChatGPT synthesize a grounded answer with citations.
• Documentation lookups: Search a topic or crawl a specific docs site, then get answers, comparisons, or ready-to-use code snippets inside your conversation.
• Competitive intelligence: Search for competitor coverage or scrape pricing and feature pages directly as structured data.
• Lead enrichment: Search for a company, then scrape its site for contact info, tech stacks, or product details.
• Change monitoring: Track pricing pages, job boards, or release notes and get notified when something changes.
• Dynamic sites: Use the interact capability to reach content behind logins, forms, or JavaScript-heavy flows.

Try it out

The plugin is live now. Install it from ChatGPT and start pulling live web data into your ChatGPT and Codex workflows.

• Install the plugin
• Read the docs
• Try Firecrawl

We'd love to hear what you build with it.

Original source

Firecrawl /search now returns the excerpts that best answer your query from each result. Agents using it achieve state-of-the-art accuracy on SimpleQA while using 10x fewer tokens than processing full pages.

Highlights

• Custom relevance model — A custom model scores every paragraph, list, and table against your query, returning the most relevant excerpts structured and ready for your agent to use.
• State-of-the-art accuracy on SimpleQA — Agents using Firecrawl /search score 94.7% on SimpleQA, higher than any other provider.
• 10x fewer tokens than full pages — The relevant excerpts returned by /search take up far less of your agent's context window and are cheaper and faster to reason over.
• Nothing to change in your code — The API shape is unchanged, so existing /search calls get better results automatically.

Read the full announcement here.

Original source

Today we're shipping a major upgrade to Firecrawl /search: a new custom relevance model that returns the excerpts that best answer your query from each result. Agents using /search achieve state-of-the-art accuracy on SimpleQA while using 10x fewer tokens than processing full pages.

Agents rely on search tools to find answers on the web, but those answers are often buried in noise. Navigation, boilerplate, and tangents wrap around the excerpt that matters, so an agent processes far more content than it needs to, which is slower, more expensive, and more likely to land on the wrong answer.

Our /search endpoint now runs every result through the new custom relevance model, returning only the excerpts that answer your exact query.

How the improved Firecrawl /search works

Our custom model scores every paragraph, list, and table against your query. It surfaces the most relevant excerpts wherever they appear on the page, then returns them structured and ready for your agent to use.

State-of-the-art accuracy on SimpleQA

SimpleQA is OpenAI's factuality benchmark, measuring whether models correctly answer short, fact-seeking questions. In our eval, agents using Firecrawl /search score 94.7%, higher than with any other provider we tested.

Evaluation methodology

A GPT-5.4 agent with high reasoning effort runs with up to 20 tool calls: search_web, backed by the provider under test, plus web_fetch through that provider's own extract API (Firecrawl Scrape, Parallel Extract, or Exa Contents). Claude native search is Claude Sonnet 4.6 with Anthropic's server-side web search, evaluated as a complete system.

Answers are graded by an LLM judge using GPT-5.4 and the official SimpleQA grader prompt. The full SimpleQA set of 4,326 questions was run across two sessions per provider, with the best observed score selected. As a baseline, GPT-5.4 with no search tools scores 43.8 under the same judge.

10x fewer tokens than full pages

By returning key excerpts instead of full-page content, /search leaves more of an agent's context window available for reasoning. Fewer input tokens also make downstream model calls cheaper and faster.

What you can build with it

• Multi-step agent search: Keep an agent's context lean across long chains of searches by returning only the excerpts that answer each query.
• RAG and LLM grounding: Add relevant excerpts directly to a prompt without separate chunking or reranking.
• Lead enrichment: Find specific facts about a person, company, or domain, including titles, revenue, and hiring signals.
• Research and market intelligence: Pull a specific claim, figure, or quote without processing the rest of the page.

Nothing to change in your code

Existing /search calls return more relevant context automatically, with no changes required. The API shape is unchanged, and each result still includes its title, URL, and description alongside query-relevant Markdown excerpts by default. If you need complete page content, you can still retrieve full-page Markdown in the same call:

from firecrawl import Firecrawl
firecrawl = Firecrawl(api_key = "fc-YOUR-API-KEY")
results = firecrawl.search(
  "what changed in the EU AI Act enforcement timeline",
  limit = 5,
  scrape_options = {"formats": ["markdown"]},
)
for result in results.web:
  print(result.title)
  print(result.url)
  print(result.markdown)

Try it today

The upgraded /search is live for every Firecrawl user, across the API, SDKs, CLI, and MCP.

1. Read the /search documentation
2. Try it in the Playground
3. Point your agent at /search and give it state-of-the-art search accuracy

Try /search in the Playground · Read the docs

Original source

Today we're launching web-scale /monitor

an always-on search that watches the web and pings you or your agent the moment something comes online.

Before, /monitor only worked for single pages or websites. You named the URLs, Firecrawl diffed them on a schedule, and notified you when something changed. Now you can use that same power on the entire web.

Define a query and a goal, and Firecrawl runs the search on your schedule, dedupes results, and alerts only when new, relevant pages appear. Every run delivers what's new and meaningful to your AI agent or app, via webhook or email.

What is web-scale /monitor?

Web-scale /monitor finds new pages across the web as they appear. You define one or more search queries, a recency window, and a plain-language goal. On every check, Firecrawl:

1. Runs the queries against the web
2. Filters to results inside your searchWindow
3. Dedupes by canonical URL against prior checks
4. Judges new results against your goal (when judging is enabled)
5. Pings you or your agent when something worth acting on comes online

Pages and sites vs. the entire web

Same scheduling, goals, judging, and notification channels underneath. If you've used /monitor for page watching, the API shape will feel familiar: web monitors use a search target instead of URLs.

Create your first web monitor

A web monitor is a standard create_monitor call with a type: "search" target:

[Code examples in Python and Node.js omitted for brevity]

Tune recall and precision

Two levers do different jobs:

• queries control recall: what each search pulls in. Cast a wide enough net with relevant terms and variations, then let your goal decide what actually alerts.
• goal controls precision: which retrieved results actually alert.

Good queries read like search terms, not sentences. Use includeDomains and excludeDomains for domain scoping instead of site: operators in the query string.

Get pinged by webhook or email

Point a webhook at your agent and subscribe to monitor.page and monitor.check.completed, the same events page monitors use. When a new matching result comes online, the payload arrives with status: "new" and a judgment explaining why it matters.

Email summaries work the same way: sent only when a check surfaces new or errored results.

What you can build with web-scale /monitor

Use case | Example queries | Goal

Track regulatory and legal changes | FDA approval, clinical trial posting, SEC filing | Alert when a new approval, trial, or filing is published

Monitor competitor developments | "Acme Corp" funding OR acquisition, competitor careers site | Alert on funding rounds, new roles, or product launches

Track breaking news and events | Phase 3 trial results biotech, M&A filing | Alert when trial results or deal filings appear

Replace the DIY stack

Before web-scale /monitor, staying current on the open web meant wiring together a scheduler, a search API, deduplication logic, a relevance filter, and a notification layer, then maintaining all of it as queries and sources shift.

One endpoint replaces that stack. You define the query once; Firecrawl handles the always-on search, dedupe, classification, and delivery.

Pricing: web monitors cost 2 credits per 10 results per check, plus 1 credit per result the judge evaluates when judging is enabled. There is no separate per-monitor fee.

Try it today

Web-scale /monitor is live for all Firecrawl users.

1. Create a monitor in the dashboard
2. Read the Web Monitoring docs
3. Wire the monitor.page webhook into your agent or pipeline

If you're already watching known URLs with /monitor, add a search target alongside them: page monitors for what you know, web monitors for what hasn't been published yet.

Get started with web-scale /monitor · Read the docs

Original source

Web-scale /monitor watches the entire web for you. Define search queries and a goal, and Firecrawl pings you or your agent the moment something new comes online.

Highlights

• Always-on web search — Give /monitor search queries and a goal, and it searches the entire web on your schedule, alerting you when something new matches.
• Tune recall and precision — queries control what each search pulls in, while goal decides which of those results actually alert.
• Webhook or email delivery — Get pinged the moment something new matches, with a plain-English explanation of why it matters.
• One endpoint replaces the DIY stack — Define your query once. Firecrawl handles the search, deduplication, filtering, and alerts.

Read the full documentation here.

Original source

Firecrawl v2.11.0 ships the Firecrawl Research Index, keyless endpoint access, automatic PII redaction, a deterministicJson format, and video discovery on any page - plus much more.

Highlights

• Firecrawl Research Index — Search across 3M+ arXiv papers and the GitHub code behind them (issues, merged PRs, and READMEs, refreshed daily), fetch a paper's details or related work, and check claims against full text. It has state-of-the-art recall on arXivQA, outperforming the next best provider by 18% at comparable cost.
• Keyless access for core endpoints — Use /scrape, /search, /interact, and /parse without an API key from official MCP, CLI, and SDK clients.
• Automatic PII redaction — A new redactPII option strips personal and sensitive data like names, emails, phone numbers, addresses, and secrets out of scraped content before it's returned.
• deterministicJson format — Get structured JSON without running an LLM on every request. Firecrawl generates a reusable extractor for your schema and caches it per site, so repeat scrapes are cheaper and return consistent results.
• Video discovery on any page — The video format now finds videos on any page, not just supported providers like YouTube, returning each video's URL, title, thumbnail, duration, and more.

Read the full changelog here.

Original source

AI/ML research moves fast, and the work that matters is split between new papers and the code that implements them. Most search providers omit or misrank key papers, leaving you to review sources by hand without ever being sure you've caught everything.

Today we're launching Firecrawl Research Index, a specialized index for agents pushing the frontier of AI/ML research. It gives agents the entire AI/ML literature and the code behind it, plus the tools to turn them into answers — agents retrieve the right papers, verify claims against full text, and pull code for implementation autonomously.

On arXivQA, the index has state-of-the-art recall, 18% above the next best provider at similar cost. It also scores 0.750 MRR, meaning the correct paper lands in the top two results.

How the Firecrawl Research Index makes agentic deep research easier

Highest recall, at comparable cost

On arXivQA, the index hits 53.3% recall at $0.32 per task, against 45.4% for the next best provider. Your agent works from a fuller set of papers without paying a premium.

Aemon (YC W26), which builds autonomous AI research engineers, saw the same pattern in their own benchmark of scientific and technical retrieval systems:

"Aemon is building autonomous AI research engineer that solve hard scientific and technical problems. To do that, our systems must continuously learn from the frontier of research—papers, implementations, benchmarks, and technical discussions across the web.
We use Firecrawl Research as part of the retrieval stack behind Aemon. In our internal benchmark of scientific and technical retrieval systems, it delivered the strongest recall of any provider we tested, particularly at deeper search depths where comprehensive coverage is critical. Firecrawl consistently surfaced relevant scientific and technical sources that would otherwise have been missed."
— Ray Xu, Co-Founder, Aemon (YC W26)

Surface the right source first

On arXivQA, the index scores 0.750 MRR, meaning the correct paper lands in the top two results. Higher MRR means fewer wasted tokens before an agent finds what it actually needs.

Search millions of papers alongside their code

The index includes all 3M+ arXiv papers, plus GitHub artifacts from top research repos (issues, merged PRs, READMEs), refreshed daily so agents always stay current.

A complete toolset for research loops

The built-in toolset lets agents run research end-to-end — retrieving the right papers, verifying claims against full text, and pulling code. Agents can go from literature to implementation in one query, with no manual filtering, cross-referencing, or review required.

What you can use it for

• Power your research platform's search: Plug in the index, and your platform ships state-of-the-art search across millions of papers and the code behind them.
• Autonomous research agents: Your agent finds the most relevant papers and code for its problem, follows citations, and verifies against full text before it builds on them. An agent tuning a training run overnight could pull an optimizer from a recent paper and a stability fix from a related GitHub issue, then test both in its next experiment.
• Literature reviews and discovery: Find the relevant work on any topic, including papers published this week. Then start from your best hit and surface its references, citers, and related work, reaching papers a keyword search misses.

Methodology

We benchmarked on roughly 200 queries from alphaXiv's ArXivQA, each labeled with up to 10 ground-truth arXiv IDs. To measure recall, we let Opus 4.8 run each provider through its MCP and SKILL.md, then scored the papers it surfaced against those labels.

Availability

Firecrawl Research Index is available now in the API via /search/research, plus the CLI, MCP, and SDKs. It plugs into any harness you already run, including Codex, Claude Code, and Grok Build.

Get started with Firecrawl Research Index

Original source

Setting up a web data API usually starts with a signup form. Create an account, generate a key, paste it into an env file, write your first line of code. That's friction before you've built anything.

Today we're launching Firecrawl Keyless. Search, scrape, and interact with the web without an API key. Every developer gets 1,000 free credits a month, automatically. Sign up only when you need more.

With Firecrawl Keyless, you can:

• Search the web for live results with full-page content
• Scrape any URL for clean markdown, including JavaScript-heavy pages
• Interact with pages to click, fill forms, paginate, and navigate dynamic sites

Live today across our MCP, CLI, and API.

How Firecrawl Keyless makes building easier

Agents work without setup

For coding agents, this matters even more. Connect Claude Code, Cursor, OpenClaw, Hermes Agent, OpenCode, or any other MCP-compatible host, and the agent starts scraping and searching immediately. No human in the loop to generate a key and paste it into config.

Demos and prototypes that don't break

API keys fail at the worst moment: a hackathon demo, a workshop, a side project picked up months later. Keyless removes that class of failures for projects where 1,000 credits a month is plenty.

Try it today

Firecrawl Keyless is live across every surface:

• MCP - point any MCP-compatible client at https://mcp.firecrawl.dev/v2/mcp
• CLI - run npx firecrawl-cli@latest and start scraping
• API - call the Firecrawl REST endpoints directly, no Authorization header required

You get 1,000 free credits per month, every month. When you outgrow it, sign up and bring your own key.

Get started with Firecrawl Keyless · Read the docs

Original source

Firecrawl Research Index is a specialized index for agents pushing the frontier of AI/ML research. It has state-of-the-art recall on arXivQA, outperforming the next best provider by 18% at comparable cost.

Highlights

• Benchmark-leading performance — On arXivQA, the index hits 53.3% recall versus 45.4% for the next best provider. The index also scores 0.750 MRR, putting the right paper in the top one or two results.
• Search millions of papers alongside their code — The index includes 3M+ arXiv papers, plus GitHub artifacts from top research repos (issues, merged PRs, READMEs), refreshed daily.
• A complete toolset for research loops — Agents can retrieve papers, verify claims against full text, and pull code, going from literature to implementation with no manual review.
• Plugs into any agent harness — Works with Codex, Claude Code, Grok Build, and more.

Read the full documentation here.

Original source

Agents can easily scrape any web page with Firecrawl. But knowing the moment a page changes — and staying focused on the changes that actually matter — is trickier. Traditionally, this required writing a cron job, standing up snapshot storage, diffing old against new, building a webhook layer with retries, and filtering out the noise (ads, timestamps, session tokens).

Today we're launching Firecrawl /monitor — the whole monitoring stack in one endpoint, only pinging your agent when something meaningful changes.

What is Firecrawl Monitoring?

/monitor runs scheduled checks on pages or entire sites, and notifies your agent when something you care about changes. Simply describe what you want to monitor in plain English, and Firecrawl configures the URLs, logic, and schedule for you.

The output is a structured diff: what was added, what was removed, what changed. Your agent only ingests what changes on a page, skipping unchanged content and noisy diffs — using up to 90% fewer LLM tokens.

How does Firecrawl /monitor keep your agents current?

Know the moment a page changes

Set a cadence (every 5 minutes, hourly, daily, or a custom cron schedule) and Firecrawl checks the page for you. When a change is detected, your agent gets a notification. When nothing has changed, nothing is sent. No polling loops, no wasted checks.

Watch only what matters

You can monitor a single page or an entire site, and track the whole page or drill down to specific fields. You can also edit content filters and scrape behavior manually if you need more precise control.

See exactly what's different

Every notification includes a rich diff (what was added, what was removed, what changed), formatted for easy reading by a human or an agent. Each check also gets a permalink so you can share a specific change with a teammate or hand it directly to an agent for further processing.

Deliver changes straight to your agent

Changes fire as signed webhooks with custom headers and per-event subscriptions. Your agent only wakes for the events it cares about. Email delivery works the same way, with the diff already in the message body.

Know your costs before you commit

See the estimated monthly cost before you turn a monitor on. You know exactly what your chosen schedule will cost before you commit to it.

How the /monitor pipeline works

1. Create a monitor for one or more web pages, or an entire site
2. Firecrawl scrapes the pages on your schedule and stores a snapshot
3. On each check, the new version is diffed against the last stored snapshot
4. If something changed, Firecrawl sends a signed webhook or email with the structured diff and a permalink

What you can build with /monitor

Use case What it looks like Triggering agents on change A deep research agent monitoring dozens of sources can generate a fresh summary the moment one of them updates. Keeping RAG pipelines fresh When a watched page changes, Firecrawl pings your pipeline with the diff so you can refresh just that document instead of rescraping everything. Tracking competitors and markets Watch competitor pricing pages, product catalogs, or job listings and know within minutes when something shifts: a price drop, a new SKU, a new role. Monitoring sources you rely on Get updates for new research papers, developer docs, changelogs, financial filings, and government regulations.

Try it today

Firecrawl Monitoring is live for all Firecrawl users. Create your first monitor from the dashboard or via the API.

Get started with Firecrawl Monitoring · Read the docs

Original source

AI apps and agents built on Vercel often need web data: a page to scrape, a search to run, a dynamic site to interact with. Until now, that meant a separate account, separate billing, and manually copying API keys. Firecrawl is now a native integration on the Vercel Marketplace.

What the Vercel + Firecrawl integration sets up

The Vercel Marketplace integration handles Firecrawl account provisioning, API key management, and billing automatically. Here is what happens when you install:

• Provisions a Firecrawl team and API key automatically
• Injects FIRECRAWL_API_KEY directly into your Vercel project's environment variables (no copy-paste)
• Adds Firecrawl to your Vercel invoice, so there is no separate vendor or payment method to manage
• Gives you SSO into the Firecrawl dashboard from Vercel, with no separate login

Plan changes, API key rotation, and uninstall all work through the Marketplace the same way.

How to get started

1. Go to the Vercel Marketplace and find Firecrawl.
2. Pick a plan: Hobby, Starter, Growth, or Scale.
3. Select the Vercel project to attach it to.
4. Done. FIRECRAWL_API_KEY is live in your project environment.

What can you build with Firecrawl on Vercel?

Firecrawl gives your Vercel app three endpoints for pulling data from the web. It covers AI agents that browse the web, RAG pipelines that stay fresh, and SaaS tools that extract structured data:

• search — search the web and get full page content from results in a single call
• scrape — convert any URL into LLM-ready Markdown, structured JSON, or screenshots
• interact — scrape a page, then click, scroll, or fill forms to reach dynamic content

Here is what teams typically build:

Use case Endpoint Description RAG pipelines with fresh web data scrape Pull docs, changelogs, or any public page into LLM-ready Markdown on a schedule. RAG pipelines stay current without manual re-ingestion. Research and deep research search Returns complete page content per result. A deep research agent can search a topic, retrieve full articles, and synthesize across sources in one pass. Stanford's AI Playground processes 800 sources daily this way for real-time LLM grounding. Real-time product and news monitoring search Runs on each request so your app always reflects current data. Surface product launches, competitor updates, or breaking news as they happen. Data extraction for AI-native SaaS scrape + interact scrape returns structured JSON you can validate with Zod or Pydantic. interact handles clicks and scrolls for dynamically loaded content.

Try it today

Firecrawl is live on the Vercel Marketplace now. Install takes under a minute and your API key lands in your project environment automatically.

Install on Vercel Marketplace · Read the docs

Original source

Enter a URL, describe what you want to track, and /monitor notifies your AI agent via webhook the moment pages or sites change. Use up to 90% fewer LLM tokens by only ingesting what actually changes.

Highlights

• Set a goal in plain English — Describe what to watch, like "alert me when the Claude Code docs add new slash commands," and /monitor configures the URLs, schema, and schedule for you.
• Up to 90% fewer LLM tokens — Your agent only ingests what changes on a page, skipping unchanged content and noisy diffs.
• Any cadence, with cost upfront — 5 minutes, hourly, daily, or a custom cron schedule. The estimated monthly cost is shown before you turn a monitor on, so you know what you're committing to.
• Webhook or email delivery — Every change fires a signed webhook with custom headers and per-event subscriptions, or arrives by email with the diff in the body.
• Permalinks for every change — Diffs are first-class objects you can share with a teammate or hand straight to another agent.

Read the full documentation here.

Original source