Firecrawl Release Notes

Setting a fantasy football lineup usually means checking rankings on one site, projections in a different scoring format on another, and the injury report on a third. None of them know who is on your roster, so you line it all up yourself, often right before kickoff.

Today, fantasy football data is live in Firecrawl Alexandria. Your agent can read your league, pull this week's projections from six expert sources in your league's scoring format, check injury reports and player news, and tell you who to start, all through the same Firecrawl connection it already uses for search and scrape.

Firecrawl Alexandria is already live in Claude, ChatGPT, Codex, and other agent tools, so you can ask from wherever you already chat.

Set your fantasy football lineup in ChatGPT or Claude

Either button opens a chat with the prompt ready, and the agent:

1. Gives you the link to add the Firecrawl plugin if it is not connected yet.
2. Asks for your Sleeper username, your ESPN or Yahoo league link, or a screenshot of your roster.
3. Pulls this week's projections for your players from every fantasy source in Firecrawl Alexandria, in your league's scoring format.
4. Tells you who to start and sit in your real lineup slots, with one line on why for each change, then the top three free agents worth adding.

If you would rather not open a chat, the free fantasy lineup page works on its own. Type a Sleeper username, paste an ESPN or Yahoo team page, or drop a screenshot, and it ranks your roster with no account. Your roster stays in your browser.

What's in the Firecrawl Alexandria fantasy pack

The lineup tool merges several kinds of data, and each one is a capability your agent can also call on its own.

Data What your agent gets Weekly projections and rankings Start/sit order and projected points for QB, RB, WR, TE, K, and team defense from six expert sources, in PPR, half-PPR, and standard scoring (not every source covers every position and format). Some sources add floor and ceiling projections, tiers, and stat lines. Projections from betting lines Projected points built from sportsbook player props, with the consensus line behind each number. Season and rest-of-season boards Long-range rankings for trades and waivers, plus dynasty and rookie rankings and average draft position. Injuries and news The weekly injury and practice participation report, and the latest news for a player. League and schedule data Weekly schedules and scores, rosters, standings, and player profiles. Accuracy checks Last week's projected versus actual points, so you can see which projections held up.

The lineup tool takes the projections from all six sources, merges them by player, and shows the consensus next to the range. A narrow range means the sources agree. A wide one means the call is close and the matchup matters more, so the tool grades each opponent by how many fantasy points it has allowed to that position.

Discovering capabilities is free. Every capability shows its price in credits before you call it.

What you can do with Firecrawl Alexandria fantasy data

Each of these is one request to your agent in ChatGPT, Claude, or Codex:

• Settle a start/sit call. Compare two or three players across every source's rank and projection, then check the injury report before you lock it in.
• Work the waiver wire. Paste your league's available players and rank them against rest-of-season projections and this week's injury report.
• Plan a trade. Line up both sides of an offer against season-long projections and each player's remaining schedule.
• Check how projections held up. Pull last week's projected versus actual points to see which calls hit and which missed.
• Prep for next year's draft. Pull average draft position by scoring format and league size, including past seasons.

You do not need to know capability names in advance. Describe the data you want, and Firecrawl Alexandria returns the capabilities that can supply it, ranked, with their inputs and prices attached.

One key for every source

Your agent calls every source with your Firecrawl key and pays in Firecrawl credits at the price listed on each capability. There is no separate key per site and nothing to scrape or parse yourself. Every capability in Firecrawl Alexandria lists its inputs, response shape, and price in the same format, so your agent can work out how to call each source before it spends anything.

What's next

The sports category in Firecrawl Alexandria already reaches past fantasy football, with league data across the NFL, NBA, MLB, NHL, college sports, and soccer, plus prediction markets and ticket prices. New providers join every week across 26 categories.

To use this from your own agent, connect Firecrawl's MCP or install the CLI and skills.

Original source

Your AI agents can now research fantasy sports with Firecrawl Alexandria. Bring fantasy rankings and projections together with player and team stats, sports betting odds and the latest news to make more informed lineup decisions in ChatGPT or Claude.

What you can do

• Research who to start. Compare projections with this week's matchups and injury news before choosing your lineup.
• Look beyond one ranking. Check where fantasy analysis sources agree or disagree and use the wider research to assess a close call.
• Try the fantasy football demo. Paste your roster to compare projections, switch scoring formats and see suggested start/sit changes for your league's lineup slots.

Connect Firecrawl in ChatGPT or Claude and ask your AI agent to use Alexandria for your next lineup decision, or try the fantasy football demo.

Original source

What's in the Firecrawl Alexandria people pack

Enrichment is where agent-built GTM workflows tend to stall. The agent can find a company and read its website, but turning that into a named person with a work email means integrating a data provider, learning its schema, and wiring up its billing before the first record comes back.

Today Apollo, FullEnrich, and Data Legion are live in Firecrawl Alexandria. Your agent can find the people at a company, identify their work emails and phone numbers through official data providers, and enrich the company itself, using the same Firecrawl API it already calls for search and scrape.

Firecrawl Alexandria is already live in Claude, ChatGPT, Codex, and other agent tools, so you can use it from wherever you already work. Ask any of them for the decision makers at a list of accounts and their contact details, and it can go get them.

Each provider returns data that web search cannot reach, structured so your agent can reason over it, through the same connection and with no integration work.

Provider | Use it to | What it adds

Apollo | Build account and lead lists from scratch | Free person search by title and company domain. Work emails with employment history, up to ten people per call. Company funding, headcount, tech stack, job postings, and news.

FullEnrich | Find people by background and reach them directly | Search by skills and past employers, not only current title. Mobile numbers and personal emails alongside work emails. Reverse lookup to identify the person behind an email address.

Data Legion | Fill in records you already have | Match a person from almost any identifier (email, phone, social URL, or name plus company) and a company from its domain, name, or ticker. Base records cover the core profile; premium adds derived insights and confidence scores. Skip contact data when you only need the profile, at a lower price.

Apollo's person search is free. Every other capability shows its price in credits before you call it.
People Data Labs is coming soon to Firecrawl Alexandria, along with more enrichment providers joining the people pack. Try it out here: firecrawl.dev/app/alexandria.

What you can do with Firecrawl Alexandria people enrichment

Each of these is one ask to your agent in Claude, ChatGPT, or Codex, and each composes with the rest of the library:

• Build a target account list. Search Apollo companies by headcount and technology, run the free person search on every domain to find whoever owns the relevant function, and buy work emails only for the shortlist.
• Find people by what they have done. Search FullEnrich by skills and work history rather than job title, then pull work emails only for the profiles worth contacting.
• Find a work contact. Search Apollo or FullEnrich for a person's current title, employer, and work email.
• Route inbound signups. Run FullEnrich reverse email lookup on a new signup address to identify the person and company behind it, enrich the company from its domain, and route the lead before anyone opens it.
• Watch competitors hire. Pull Apollo job postings and company news for a set of rivals each week to spot bursts of new roles or funding announcements.
• Put the filed picture next to the operating one. Pull the legal entity from Companies House or the Delaware registry, both already in Firecrawl Alexandria, then layer Data Legion company enrichment on top.

You do not need to know capability names in advance. Describe the data you want and Firecrawl Alexandria returns the capabilities that can supply it, ranked, with their inputs and prices attached, and discovery itself is free.

One key for every provider

You call every provider with your Firecrawl key and pay in Firecrawl credits at the price listed on each tool. There is no second API key to rotate. Where a provider asks your team to accept its terms first, you do that once in the dashboard.
Because every provider in Firecrawl Alexandria is priced in the same unit and described in the same shape, an agent can compare what two capabilities cost and pick the one the task needs before it spends anything.

Terms and access

Some providers need your team to accept their terms before the first call. An unaccepted call returns THIRD_PARTY_DATA_TERMS_REQUIRED with a requiresAction.url, and an org admin accepts at that URL or in provider settings. Agents need explicit authorization from you before accepting terms on your behalf. Agent runs skip providers your team has not accepted and tell you which ones they skipped.
Enrichment data carries usage restrictions that vary by provider. For example, FullEnrich's terms bar marketing to personal email addresses. Each provider's terms are published at the point of acceptance, so read them before you build on the data.

What's next

New data providers are being added to Firecrawl Alexandria every week, across 20 categories including finance, news, jobs, government spending, competitive web intelligence, and retail pricing. Your agent can pull any of them into the same pipeline, so enrichment sits alongside hiring signals, funding news, and procurement records.
Try it out →
Become a provider →
To use this from your agent, connect Firecrawl's MCP or install the CLI and skills.

Original source

Your agents can now find decision makers, look up work emails, and enrich company data with Alexandria. Apollo, FullEnrich, and Data Legion are live through the same Firecrawl connection your agents already use to search and scrape the web.

Use the people enrichment pack in Claude, ChatGPT, Codex, or your own agent through the Firecrawl API, MCP, or CLI. Describe the data you need and Alexandria finds the available capabilities, with their inputs and prices attached.

What's available

• Apollo: Search for people by title, company domain, and location, then enrich selected profiles with work emails and employment history. Enrich companies with funding, headcount, and technologies, or pull job postings and company news.
• FullEnrich: Search for people by role, employer, location, skills, and work history. Find work emails, look up profiles, identify the person behind an email address, and retrieve company firmographics and technologies.
• Data Legion: Enrich a known person or company at base or premium detail, with contact data available for person enrichment.

What you can do

Build a target account list, find the people who own a relevant function, and look up work emails for your shortlist. Enrich an inbound signup with company context, or combine company enrichment with hiring signals and financial data from other Alexandria providers.

Try asking your agent:

Use Firecrawl Alexandria to find engineering leaders at a company I specify. Show their current roles and the available work email enrichment options with prices before running enrichment.

One connection, clear pricing

Use your Firecrawl account and credits across providers, with no second API key to rotate. Discovery is free, and each tool shows its execution price before you call it. Apollo's person search is free; enrichment and other capabilities use their listed prices.

Some providers require your organization to accept their terms before the first call. If a call returns THIRD_PARTY_DATA_TERMS_REQUIRED , an organization admin can follow the returned requiresAction.url or open provider settings . Agents need your explicit authorization before accepting terms on your behalf. Provider-specific usage restrictions apply; FullEnrich's terms, for example, prohibit marketing to personal email addresses.

Try the people enrichment pack or connect Firecrawl over MCP.

Original source

You can now use Alexandria directly in ChatGPT and Codex through the Firecrawl plugin. Search and scrape the live web, with access to 100+ data providers and Firecrawl's specialized indexes from the same conversation.

What you can do

• Research a company. Bring together its website, SEC EDGAR filings, and financial data from Yahoo Finance in one brief.
• Compare products. Check prices and specs across Amazon, Best Buy, and Newegg alongside lab reviews from RTINGS.
• Work through a bug in Codex. Find relevant GitHub issues, merged pull requests, READMEs, and documentation through the Developer Index, then check the package on npm or PyPI.
• Dig into a research topic. Search scientific paper abstracts through the Research Index to find studies relevant to your question.

These are a few examples of what you can research with the data providers, specialized indexes, and live web available to your agent.
In our internal evaluations, agents using Alexandria scored 21% higher on answer quality than those using built-in web tools. We used the same model and prompts across 845 tasks, with blind AI judging.

Get started

Install the Firecrawl plugin in ChatGPT and Codex and connect your Firecrawl account. Alexandria requires authentication. If you already have the plugin connected, try it in your next conversation:

Use Firecrawl Alexandria to find remote software engineering jobs posted in the past week. Include salaries where available and application links.

Your agent discovers relevant tools, reads their inputs and prices, and calls the ones it needs. Discovery and inspection are free; execution is billed at each tool's listed price. Web search and scraping are billed as normal. Some providers require an organization admin to accept their terms in provider settings first.

Browse Alexandria or read the Alexandria documentation.

Original source

Firecrawl Alexandria is now available inside the ChatGPT plugin

ChatGPT and Codex can reach 100+ data providers and Firecrawl's own indexes, alongside the live web they already search.

What is Firecrawl Alexandria?

Ask ChatGPT about a company's latest quarterly filing today and it searches the web, finds a page discussing the filing, and summarizes that page. You get a secondhand account of the numbers. The filing itself sits in a provider API the model has no way to call.

That limit shapes what agents are good for. They answer well when the open web has already written up a subject, and poorly when the answer turns on a primary source: what a series actually reported last quarter, what a regulator filed, what a dataset records. That information exists in structured form behind provider APIs, and an agent's only route to it has been whatever a blog post happened to quote.

Firecrawl Alexandria gives the agent that source directly, under agreement with the providers who maintain it. One request finds it, shows what it holds and what it costs, and pulls the data.

Without Firecrawl Alexandria vs With Firecrawl Alexandria

• What the agent reads: A page that discusses the data vs The provider's own record
• Coverage: Whatever the open web has written up vs 100+ official providers, plus Firecrawl's Research, Developer, and Government indexes
• Finding a source: Search, then hope a page quoted the number vs One request finds the source and reports what it holds
• Providers: Earn nothing from the answer vs Paid under agreement when your agent pulls their data

Across 845 tasks with the same model, the same prompts and blind AI judging, agents using Firecrawl Alexandria scored 21% higher on answer quality than agents using built-in web tools.

How to use Firecrawl Alexandria in ChatGPT and Codex

Firecrawl Alexandria follows three steps: find a tool, check what it needs, then run it. You ask a question and the plugin walks those steps for you.

Ask for something the open web answers badly:

ChatGPT searches for relevant tools, reads the inputs and price on the one it picks, and calls it. You see what it chose and what the data cost.

The same pattern covers research papers, company filings, government spending records, podcast transcripts, package registries, and real estate. When no provider fits, the plugin falls back to web search and scraping, so a gap in provider coverage doesn't dead-end the question.

You can also ask what exists before committing to anything:

None of that costs anything. ChatGPT comes back with the providers, their capabilities, and their prices, and you decide whether to spend anything.

The Developer Index in Codex

Codex gets the Developer Index, built over public repositories, GitHub issues, merged pull requests, READMEs, and code documentation.

The answer to a question like this usually sits in a README, an issue thread, or a merged fix rather than in a blog post. The Developer Index searches that material directly and hands Codex the matching passages.

Terms and billing

Discovery is free, execution is billed. Asking what data exists, reading a tool's inputs, and checking its price cost nothing. You pay the tool's listed price when it runs.

Some providers need their terms accepted first. Accepting a provider's terms binds your whole organization, so by default an admin accepts them in the dashboard. When a provider needs this, ChatGPT shows you the terms and points to the page.

Getting started

If you already have the official ChatGPT plugin installed, Firecrawl Alexandria is live in your next conversation.

If you don't:

1. Open ChatGPT
2. Install the Firecrawl plugin
3. Connect your Firecrawl account when prompted

Then ask for a quarterly filing and watch which provider it reaches for.

• Browse the provider catalog
• Read the Firecrawl Alexandria docs

Frequently Asked Questions

Do I need a Firecrawl API key to use Firecrawl Alexandria in ChatGPT?

Yes. Install the Firecrawl plugin from the ChatGPT directory and connect your Firecrawl account when prompted. Firecrawl Alexandria is not available to keyless sessions, so the plugin has to be connected to an account before ChatGPT can discover or run any provider tools.

What does Firecrawl Alexandria cost inside ChatGPT?

Finding and inspecting tools is free. You can ask ChatGPT what data exists for a topic, and read each tool's inputs, response shape, and price, without spending anything. You are charged only when ChatGPT runs a tool, at the price listed on that tool. Web search and page scraping are billed as normal.

What happens if a provider requires accepting terms first?

Some providers require your organization to accept their terms before you can pull data. By default, an organization admin accepts them in the Firecrawl dashboard. If your organization has enabled agent acceptance, ChatGPT may accept them after showing the current terms; otherwise it points an admin to the dashboard.

Does this work in Codex as well as ChatGPT?

Yes. The Firecrawl plugin covers both surfaces, so the same provider catalog and the same Developer Index are available in Codex. Codex is where the Developer Index gets the most use, since it answers questions about library behavior from repositories, issues, and merged pull requests.

What is the Developer Index?

It is one of Firecrawl's own indexes, built over public repositories, GitHub issues, merged pull requests, READMEs, and code documentation. It returns the matching passages from that material, which helps when you need to know how a library behaves in practice and the explanation sits in an issue thread rather than the documentation.

Original source

Your agents in Claude and Claude Code can now reach 100+ data providers through Alexandria, alongside Firecrawl's live web search, scraping, and specialized indexes.

Research companies, pull economic data, find scientific papers, or work through a coding question without leaving your conversation. Describe what you need and your agent can discover a source, inspect its inputs and price, and retrieve the data through Firecrawl.

What you can do

• Research a company. Bring together its website, SEC EDGAR filings, and financial data from Yahoo Finance in one brief.
• Explore economic data. Pull inflation, interest rates, and other time series from providers such as FRED.
• Dig into a research topic. Find relevant scientific papers through the Research Index and check supported full-text passages.
• Work through a bug in Claude Code. Search repositories, documentation, GitHub issues, and merged pull requests through the Developer Index.

Get started

Connect Firecrawl in Claude, or install the Firecrawl plugin for Claude Code, and connect your Firecrawl account. Alexandria requires authentication. Then ask:

Use Firecrawl Alexandria to show US CPI, the Fed funds rate, and the 10-year Treasury yield for the past 12 months in one table.

Discovering providers and reading their tool contracts is free. Data calls use Firecrawl credits at the price shown on each tool. Some providers require an organization admin to accept their terms in provider settings before the first call.

Explore Alexandria or connect Firecrawl over MCP.

Original source

Alexandria is the knowledge library for superintelligence. It gives AI agents a common way to discover sources and retrieve information through Firecrawl, bringing together the live web, official data providers, site-specific connectors and our own indexes.

AI agents can search our Research, Developer and Government indexes, query data providers and use specialized tools to reach information beyond individual web pages. We’re continuing to add sources and build more indexes.

Across the verticals we tested, AI agents using Alexandria scored 21% higher on answer quality than those using built-in web tools. We used the same model and prompts across 845 tasks, with blind AI judging.

Our $75M Series B

We raised a $75M Series B led by Smash Capital, with participation by Altos Ventures, Nexus Venture Partners, Y Combinator, Freestyle and Offline Ventures.

We’re investing in search and Alexandria, building deeper indexes and retrieval systems and working directly with more knowledge providers. Our goal is to make useful knowledge easier to reach and worth sharing for the people who create it.

Try Alexandria or read the full announcement.

Original source

We raised a $75M Series B led by Smash Capital, with participation by Altos Ventures, Nexus Venture Partners, Y Combinator, Freestyle, and Offline Ventures.

We’re going to spend a good chunk of it buying knowledge from people, which takes some explaining.

Today we’re introducing Alexandria. It brings official data providers, custom connectors and Firecrawl’s own indexes together with the live web, so your AI agent has one way to find a source, see what it holds and pull from it.

We named it after the ancient Library of Alexandria. If you wanted the world’s knowledge in one place back then, you physically moved it there and hoped nothing happened to the roof. Papyrus doesn’t copy itself.

Today, we can share information without shipping scrolls around, but much of what people know is still out of reach. Someone might spend a lifetime learning something useful without ever writing it down. A publisher might maintain a valuable dataset that your AI agent has no way to access.

We want to make that knowledge available and give the people who contribute it a reason to keep doing so. That means paying them when AI agents use it.

We already do this through agreements with several data providers, including Wikimedia Enterprise. This funding will help us bring that opportunity to more people and organizations, while improving search and expanding the sources AI agents can reach through Alexandria.

That’s how we want to build the library for superintelligence, with the knowledge people have today and the discoveries humans and AI agents make next.

We started with a problem of our own

Before Firecrawl, we built Mendable, an AI chat product for documentation. Mendable worked. What we learned building it was that getting clean, reliable information out of the web was the hardest part of the whole stack, and that other teams building with AI were solving the same problem from scratch.

So we built Firecrawl to handle that work. Give it a URL and it handles everything from crawling and rendering to parsing and cleanup.

We figured a few hundred people had that problem. Over 1.5 million users build with Firecrawl now.

Solve one problem well and you get promoted to the next one, whether you wanted it or not. As our users’ AI agents got more capable, they needed broader coverage, better search and sources that search and scraping alone couldn’t reach.

An AI agent researching a company might start with its website, then need financial data and filings to understand the business. Each source adds useful context, and each comes with its own API, pricing and data formats. Maintaining those integrations takes time away from building.

Leaving a source out can affect the answer. Even the strongest model can’t reason over information it never found.

Before you order the hoodies

So you're building an AI customer-support tool. You've bought the domain. This is the point at which things become, technically, real.

Before you spend six months on it, there's a question worth asking, and it's a slightly uncomfortable one: who else already had this idea?

The answer is never "nobody." The answer is also never the three names you can think of off the top of your head. What you actually want to know is which startups are working on this, who they sell to, and whether the thing you think makes you different is already the headline on somebody else's homepage. (It might be. It usually is. Better to find out now.)

This is the kind of research that sounds easy and isn't, because the information is scattered across a bunch of directories and websites that were each built for humans clicking around, not for an agent doing it in bulk. With Alexandria, your agent can query those directories directly and walk through whole startup batches at once. It can compare what each company actually builds and pull the underlying records, which is how you separate the real competitors from the companies that merely put "AI" in their description because it was 2024 and everyone did it.

Okay. Say you've done that, and you still want to build it. Good. There's one more small detail: somebody has to pay you.

Same agent, same connection. Have it look for companies that match your target customer, then use people enrichment to find the actual humans you'd want to talk to. For support software, that's probably whoever runs customer support. The point is you go from "who am I competing with" to "who might buy this" without rewiring your agent to a new data source every time the question changes. That part is boring. It's also the part that usually eats the week.

None of this gets you out of building something people want. It just means you'll know who else is building it, and who might want it, before you order the hoodies.

More places to look

Alexandria gives AI agents a common way to discover sources, understand what they provide, and retrieve information through the Firecrawl API you already use.

Those sources include the live web, official data providers and Firecrawl's own indexes, alongside custom connectors and workflows. An AI agent can read a webpage or search an index, query a provider or use specialized tools to collect entire datasets.

Our Research Index includes tens of millions of scientific paper abstracts. Our Developer Index spans tens of millions of primary sources across documentation and READMEs, issues and merged pull requests. Our Government Index covers laws, regulations, and ordinances.

Across the verticals we tested, AI agents using Alexandria scored 21% higher on answer quality than those using built-in web tools. We used the same model and prompts across 845 tasks, with blind AI judging.

Making knowledge worth sharing

We already pay official data providers through individual agreements, most notably Wikimedia Enterprise. Millions of requests for Wikipedia data flow through Firecrawl each month. We pay for direct access to that data, supporting Wikipedia while giving our users a better way to retrieve it.

We’re using this funding to help individuals, content creators and organizations earn from what they know through a self-service system we plan to open soon. That includes people whose expertise has never been shared online, and eventually could include AI agents making useful discoveries of their own.

What we’re building next

We’ll keep investing in search and Alexandria by improving access to the live web, building deeper indexes and retrieval systems and connecting more first-party sources.

To everyone who found Firecrawl early, filed an issue or trusted us in production, thank you. Your feedback showed us what to build next.

Alexandria starts today. Give it a company to research, scientific work to compare or a technical question to investigate. Use it on its own or alongside web search and scraping to give your AI better source material.

Connect your AI agent through Firecrawl’s MCP or build with the API.

For the CLI and skills, ask your AI agent to run this command.

npx -y firecrawl-cli@latest init --all --browser

Then restart your AI agent to load the skills.

Try Alexandria →

If you create content, maintain data or have expertise that AI agents should be able to use, we’d like to hear from you.

Become a provider →

We’re building Alexandria so AI agents can find and build on what we know today and what we discover tomorrow, while the people who contribute that knowledge share in the value it creates.

Original source

The Firecrawl Developer Index is now available, a specialized index for coding agents. It covers 70M+ artifacts across READMEs, external documentation, issues, pull requests, and OpenAPI specs, with semantic retrieval and metadata filters. Your AI agents answer questions about code behavior, API contracts, error messages, and known bugs from primary sources instead of general web pages.

Highlights

• Highest recall of any major coding-specific index. Scores 0.63 recall@10 across the 1,179 real developer queries in our open DevDex benchmark, ahead of every other provider we tested.
• 70M+ artifacts. READMEs, docs, issues, PRs, and OpenAPI specs from the most popular public repos and documentation sites, refreshed continuously with most sources updated daily.
• Answers with passages. Every result carries a stable id, a url, and the matched passages in markdown, so your agents act on the answer without a second scrape.
• Rich filters in the API. Scope by result type, repository, documentation source, language, topic, license, minimum stars, and more. These are API-only; agents on the CLI and MCP perform best without them.
• Keyless to start. No API key needed to try it; add one for higher rate limits. A developer search costs 2 credits per 10 results.
• Available everywhere you build. Query it through the API at /search/developer, plus the CLI, MCP, and SDKs, or install the companion skill with npx -y firecrawl-cli@latest setup developer-index.

Read the full blog here.

Original source

Coding agents spend a large share of their tool calls searching for one of three things:

• the repository that implements an idea
• the documentation page that answers a question
• the issue or pull request where a bug was fixed

That context is scattered across GitHub, docs sites, and threads, and the existing options for retrieving it are lexical, incomplete, or both.

Today we're launching the Firecrawl Developer Index, a specialized index for coding agents. It indexes the artifacts agents actually need to write working, current code (READMEs, external documentation, issues, pull requests, and OpenAPI specs) with semantic retrieval and metadata filters, refreshed daily.

Alongside the index, we're releasing DevDex, an open benchmark of 1,179 developer-search queries scored on Recall@10 and MRR@10, so teams can measure how well any retrieval system supports real coding-agent workflows.

Why we built the Firecrawl Developer Index

Coding agents are one of the biggest categories of what customers search and scrape for on Firecrawl. When we dug into what customers were actually retrieving, three patterns kept surfacing:

1. Agentic products (think Lovable, Replit, Bolt) doing backend debugging on behalf of end users.
2. Knowledge-base builders stitching internal and external repos into a single retrieval layer.
3. Frontier labs that need open developer documentation, code, issues, and PRs as training and evaluation data.

They were all working around the same gap. Existing providers weren't designed for a world of agents: search is lexical, not semantic, and getting complete artifacts (a README plus its issues plus its recent PRs) meant stitching together 50+ API calls, or building the whole pipeline from scratch. Even Firecrawl's general search and scrape gave subpar results for this shape of query, because a coding agent doesn't want a web page, it wants an artifact.

The Developer Index is that artifact layer.

What's in the Firecrawl Developer Index

• 70M+ artifacts across READMEs, pull requests, issues, OpenAPI specs, skills, and external documentation, refreshed continuously, with most sources refreshed daily.
• Issues and pull requests from top repositories, with their linked artifacts.
• READMEs from a broad set of public repositories.
• External documentation sources (Stripe, and everything of that shape).
• OpenAPI specs and popular skill repos.
• Metadata on every artifact: stars, licenses, artifact type.

To be clear about what it is not: the Developer Index does not store code, and it is not a general web search endpoint. It is a purpose-built retrieval layer organized around the artifacts coding agents produce and consume.

How the Firecrawl Developer Index works

You send a natural-language question to the Firecrawl Developer Index and get back ranked developer results with the passages that matched, so an agent can act on the answer without a second scrape. There are two ways to reach it:

• Firecrawl /search/developer returns developer sources only, with result-type, repository, and documentation-source filters. This is the surface to reach for when you want ranked developer results with matched passages.
• Firecrawl /search with categories: ["developer"] returns Developer Index results through the standard /search response, in the same shape as ordinary web results.

Every Firecrawl Developer Index result carries a stable id (like issue:owner/repo#123) whose prefix tells you the artifact kind (doc:, issue:, pull_request:, or readme:), a url, and its matched passages in markdown, so tables and code blocks survive. Through the API, filters let you scope by types, repos, sources, language, topic, license, min_stars, and more. You can also set skills: "only" to search indexed agent-skill files. These filters are API-only: on the CLI and MCP, agents perform best without them, so they are intentionally not exposed there.

No Firecrawl API key is needed to get started; add one for higher rate limits. A developer search costs 2 credits per 10 results, rounded up.

The easiest way to give your agent access is Firecrawl's dedicated developer skill, which plugs into the Firecrawl CLI or MCP server:

npx -y firecrawl-cli@latest setup developer-index

For the full parameter reference and response schema, see the Developer Index docs.

What is Firecrawl DevDex?

DevDex is an open benchmark for developer-search retrieval, scoring how well any system returns the right docs, GitHub pages, and Stack Overflow answers for real coding-agent queries. We built it because the standard search benchmarks don't reflect what agents actually look up while writing code.

How does DevDex measure developer search?

To measure whether a specialized artifact index like the Firecrawl Developer Index actually improves agent outcomes, we built DevDex. It's a benchmark of 1,179 developer-search queries across three tracks that mirror how coding agents actually retrieve.

We are releasing part of the dataset and the evaluation harness as an open-source evaluation so that DevDex can serve as a public standard for developer search on the web.

The three tracks:

• Repository discovery. Find the repo that matches a described capability without knowing its name. Example: "a library for incremental PDF parsing."
• Documentation lookup. Find the exact page that answers a how-to. Example: "How do I add Pydantic to my project using uv instead of pip?"
• Issue and PR resolution. Find where a specific bug was discussed and fixed. Example: "scikit-learn LogisticRegression random_state not working."

Each track is scored deterministically on Recall@10 and MRR@10 against fixed gold references, with a memorization check to drop any query the driver model can answer from pretraining. Every provider runs under a matched setup: same driver model (Claude Opus 4.8), one search tool active per run, same harness.

Results

The Firecrawl Developer Index leads overall recall, ahead of general web search and every other developer-search provider.

Recall@10 measures whether the correct artifact appears anywhere in the top ten results. Higher is better. Each overall score is the mean across the three tracks. The Firecrawl Developer Index scores 0.63, ahead of Firecrawl Search (no category) at 0.58, Parallel at 0.57, and Mintlify and Exa at 0.54. Native web search sits at 0.45, and Context7 at 0.17.

Broken out by track, the Firecrawl Developer Index leads issue and PR resolution at 0.66, and is statistically tied with Context7 on documentation lookup (both at 0.47, a 0.006 gap well inside the 95% CI). Context7 is docs-focused, and scores near zero on the other two tracks (0.01 on repository discovery, 0.03 on issues and PRs). On repository discovery the Developer Index posts 0.76, behind Parallel at 0.82 and Firecrawl Search at 0.78.

The gap between the "no tools" control and every other row is the size of the retrieval problem: coding agents can't answer these queries from pretraining alone. The gap between native web search and the specialized indexes is what a purpose-built artifact layer buys you.

Release

We're open-sourcing half of the dataset plus the evaluation harness so any team can reproduce results on their own systems.

• Repo: github.com/firecrawl/benchmark-devdex
• To submit a provider: open a PR against the repo with your results on the public half, and email [email protected] with valid API keys so we can rerun on the held-out half.

What you can build with the Firecrawl Developer Index

• Ship an agentic product that debugs its own code. Wire the Firecrawl Developer Index into your agent's tool loop and let it search issues and PRs the moment it hits an error, instead of guessing.
• Build a developer knowledge base. Use the Firecrawl Developer Index to pull external documentation and repo artifacts alongside your internal sources, without maintaining a GitHub scraper.
• Train or evaluate coding models. Use the Firecrawl Developer Index as a retrieval layer for RAG-style training data pipelines, and use DevDex to score whatever retriever you build.

Try it today

Firecrawl Developer Index is available now in the API, CLI, MCP, and SDKs. It plugs into any harness you already run, including Codex, Claude Code, and Grok Build.

Read the Firecrawl Developer Index docs

Original source

Today we're launching the Life Sciences category in the Firecrawl Research Index. It covers 41M+ papers across the drug discovery, clinical trial, and biology literature. Your AI agents get citable papers back for a query and can pull the full text on demand.

The whole Research Index is also free to use now. That includes the AI and ML literature it already carried alongside the new Life Sciences papers.

How the Life Sciences category makes research easier

• High-recall retrieval. The index hits 90% recall@10 on our paper-retrieval eval, so your AI agents find more of what matters on the first page of results.
• Millions of life sciences papers. The corpus holds 41M+ papers from authoritative sources and refreshes daily, so your AI agents only cite domain-specific literature.
• Abstract to full text. Your AI agents search abstracts to find the right papers, then pull the full text to verify a claim against the source.
• No DIY stack. Research Index stands in for the source APIs plus the parsing and ranking you would otherwise build yourself.

What you can use it for

• Build and improve life sciences models. Your AI agents call the index to support internal work on chemical compounds and predictive biology.
• Power a research platform's answers. Use the index as the backend for your platform. A user asks a question and the index returns the papers and passages your platform needs to answer it.
• Run targeted academic and clinical research. Point the research harness you already use at the index and your AI agents retrieve life sciences literature scoped to your objective.

Try it today

The Life Sciences category is live now in Firecrawl Research Index through the API at /search/research, plus the CLI, MCP, and SDKs. Every category is free to query.

Original source

The Life Sciences category is now available in Firecrawl Research Index. It covers 41M+ papers across the drug discovery, clinical trial, and biology literature. Your AI agents get citable papers back for a query and can pull the full text on demand. The whole index is also free to use now.

Highlights

• High-recall retrieval. The index hits 90% recall@10 on our paper-retrieval eval, so your AI agents find more of what matters on the first page of results.
• Millions of life sciences papers. The corpus holds 41M+ papers from authoritative sources and refreshes daily, so your AI agents only cite domain-specific literature.
• Abstract to full text. Your AI agents search abstracts to find the right papers, then pull the full text to verify a claim against the source.
• No DIY stack. Research Index stands in for the source APIs plus the parsing and ranking you would otherwise build yourself.
• Free to use. Every category is free, the AI and ML literature it already carried alongside the new Life Sciences papers.
• Available everywhere you build. Query it through the API at /search/research, plus the CLI, MCP, and SDKs.

Read the full blog here.

Original source

Firecrawl pdf-inspector

What is Firecrawl pdf-inspector

pdf-inspector is a from-scratch Rust library for PDFs, with 1.9M views and 8.1k GitHub stars on the repo at time of writing. It reads a PDF's internal structure (font encodings, text operators, image coverage) in milliseconds, without rendering anything, and decides per page whether the content is text-based or needs OCR.

• Text-based pages get native extraction directly, with reading order preserved.
• Scanned or image-heavy pages are flagged with the reason, so a vision pipeline can pick them up.

For a fully text-based PDF, pdf-inspector alone is the entire pipeline. For mixed documents, it's the smart router that decides what actually needs a GPU.

Why it matters on its own

Most PDF pipelines make the same wrong bet: treat every page as if it might be scanned, so route everything through OCR. That's slow, expensive, and often less accurate than the native text that was sitting in the PDF the whole time.

pdf-inspector fixes the routing layer:

• Per-page classification. Analyzes internal structure only. No rendering, no GPU, milliseconds per page.
• Native text extraction. Pulls text directly from text-based pages with reading order intact.
• Clean handoff for the rest. For scanned pages, it hands back the page reference and the reason, so an OCR pipeline can do the heavy lifting only where it's needed.

That routing layer is what makes Fire-PDF, Firecrawl's hosted PDF parsing engine, 3.5x to 5x faster than the previous pipeline: for a 200-page report where 150 pages are pure text, 150 pages skip GPU entirely.

How to use it

Add pdf-inspector to your Rust project directly from the repo. The README covers the classifier and native-extraction APIs, plus the reproducible-results branch if you want to benchmark it against your own PDFs.

Or don't integrate it directly at all: send PDFs to Firecrawl's /parse or /scrape and they go through pdf-inspector (and Fire-PDF for the scanned pages) automatically.

Firecrawl AnyDoc

What is Firecrawl AnyDoc

AnyDoc is the other half of the stack, also a from-scratch Rust library, also markdown out. It converts documents with a single call:

anydoc::to_markdown("file.docx")

It supports 14 formats in one binary:

• docx
• doc
• docm
• xlsx
• xls
• xlsm
• pptx
• ppt
• rtf
• odt
• ods
• odp
• epub
• csv

No API key. No system dependencies. Nothing to install alongside it.

Why it matters

No single existing library reliably covers every common document format. Each one handles a subset, and the formats it doesn't cover become someone else's dependency, with different output, different failure modes, and often much slower conversions. AnyDoc handles all 14 in one dependency-free library.

Who feels this most: the engineer whose users upload "whatever they have": a .docx contract, an .xls export from 2009, a pitch deck, an .epub. Today, getting usable text out of all of it is a plumbing project. AnyDoc is the part that stops being plumbing.

It's also quickly become a dev favorite. Here's Garry Tan on it:

[Embedded Tweet]

Here's Nick, Firecrawl's Co-founder and CTO, on how AnyDoc is different from pdf-inspector:

[Embedded Tweet]

How it compares

We benchmarked AnyDoc against the common alternatives on 94 documents spanning all 14 formats.

• Coverage: 14 of 14. AnyDoc is the only library in the benchmark that parses every format. The nearest alternative, LibreOffice, covers 12 of 14. Every other option covers a subset, which is exactly why teams end up stitching multiple libraries together.
• Speed: 4.6ms median per document. Across all 14 formats, AnyDoc's median conversion time is 4.6ms. The alternatives run between 52ms and 1,130ms, roughly 20x to 245x slower depending on the tool. LibreOffice sits at the slow end of that range.
• Quality: highest overall, with honest caveats. Quality is LLM-judged on completeness, structure, formatting, and cleanliness. AnyDoc scored highest on every format we tested, overall score 80 vs 68 for the next-best option. Two notes worth being upfront about: the corpus is ours, and mammoth scores higher than AnyDoc on completeness alone (95 vs 87) on the single format it supports (docx). If completeness on docx is the only thing you care about, that gap matters. If you care about coverage across every format your users actually upload, AnyDoc wins the aggregate.

How to use it

Add AnyDoc to your Rust project, then call it on any supported document:

use anydoc;
let markdown = anydoc::to_markdown("contract.docx")?;

That's the whole integration. No API key, no external service, no separate binary. JavaScript bindings are in progress; we'll share those when they land.

Or don't integrate anything at all: Firecrawl's /parse and /scrape endpoints already use AnyDoc automatically when they encounter a non-PDF document.

Two libraries, one shape

Same principles across the pair: from-scratch Rust, local execution, no API key, no system dependencies, markdown out. That's not a coincidence. It's the shape developers kept asking for after pdf-inspector shipped, and it's the shape that makes both libraries safe to drop into any pipeline without dragging in a heavy runtime.

The two are deliberately separate repos and deliberately separate products. AnyDoc is not "pdf-inspector grown up." pdf-inspector handles PDFs. AnyDoc handles everything else. One lineage, two Firecrawl products, and together they cover the document formats a real AI pipeline actually sees.

Try the stack

• pdf-inspector (PDFs): github.com/firecrawl/pdf-inspector
• AnyDoc (14 non-PDF formats): github.com/firecrawl/anydoc
• Use both via API: send any document to /parse or /scrape. PDFs go through pdf-inspector (and Fire-PDF); everything else goes through AnyDoc. No configuration.

If you're already stitching four libraries together to cover the document formats your users upload, we'd love to hear how the two of these hold up as a replacement.

Original source

Firecrawl is now available as an official ChatGPT plugin. You can install it directly inside ChatGPT and give it access to live, clean web data in seconds.

Getting started

Install the plugin in three steps:

1. Open ChatGPT
2. Go to the Firecrawl plugin page and click Install
3. Connect your Firecrawl account when prompted

That's it. ChatGPT now has direct access to live web data across every conversation and inside Codex.

What you can do

Once connected, the plugin gives ChatGPT a full web data toolkit:

• Search the web and get back the excerpts that best answer your query, with full page content included in the results.
• Scrape any page or document (PDF, DOCX, and more) and turn it into clean, structured data.
• Crawl entire sites to build datasets and knowledge bases.
• Interact with dynamic pages to reach content that regular scraping misses (logins, forms, pagination, infinite scroll).
• Monitor pages and get alerts whenever something on the web changes. Monitors can post directly to Slack, so the alert lands where your team already works.

You don't need to remember any commands. Just ask in plain language and ChatGPT picks the right Firecrawl tool automatically. The plugin also ships with a set of ready-to-use skills, so you can kick off common tasks (searching the web, scraping a page, crawling a site, running an interaction flow) in one click without writing the prompt from scratch.

Example prompts

Search the web for this week's biggest AI announcements

Scrape https://news.ycombinator.com and list the top 10 stories with links and points

Interact with Campspot.com to find campsites near Yosemite available Aug 15 to 17

Monitor https://openai.com/blog for new posts and send updates to my #product-updates Slack channel

Why this matters

ChatGPT is only as fresh as the web it can reach. Its built-in browsing gives you snippets and summaries, but the moment you need real answers grounded in real sources, you hit walls: shallow search results with no page content, noisy and unstructured pages, JavaScript that renders inconsistently, and messy HTML that ChatGPT can't parse properly. Firecrawl fixes both sides of that problem.

Firecrawl search runs the query, picks the best results, and hands ChatGPT the full page content along with the links. You get the excerpts that answer the question instead of ten titles and a guess. For pages or sites you already have in mind, Firecrawl handles JavaScript, dynamic content, PDFs, and DOCX, and gives ChatGPT clean markdown or structured JSON to work with.

Together, ChatGPT can go from a vague question to a well-sourced answer without you leaving the conversation. Same inside Codex: search for how something works, then pull the exact docs it needs to write and reason about your code.

Example

Search the web for the latest guidance on receiving Stripe events in a webhook endpoint. Pull the relevant Stripe docs and implement it in my app.

No more copy-pasting URLs into separate tools. No more writing custom scrapers. No more prompting around stale search snippets. Just ask ChatGPT to find and pull the data you need, and it works.

Use cases

A few ways developers, marketers, growth engineers, and sales folks are already using the plugin:

• Live research with real sources: Search the web for what's happening right now and get back the actual page content, not just links or stale summaries.
• Answer engine over any topic: Ask a question in plain language, let Firecrawl search and pull the top results, and have ChatGPT synthesize a grounded answer with citations.
• Documentation lookups: Search a topic or crawl a specific docs site, then get answers, comparisons, or ready-to-use code snippets inside your conversation.
• Competitive intelligence: Search for competitor coverage or scrape pricing and feature pages directly as structured data.
• Lead enrichment: Search for a company, then scrape its site for contact info, tech stacks, or product details.
• Change monitoring: Track pricing pages, job boards, or release notes and get notified when something changes.
• Dynamic sites: Use the interact capability to reach content behind logins, forms, or JavaScript-heavy flows.

Try it out

The plugin is live now. Install it from ChatGPT and start pulling live web data into your ChatGPT and Codex workflows.

• Install the plugin
• Read the docs
• Try Firecrawl

We'd love to hear what you build with it.

Original source