Benchmarks
Scrape: Novagentum vs Firecrawl
How much of a page's main content does each tool return, and how much clutter comes with it?
We used Firecrawl's own public test set: 1,000 real web pages, each with a sample of its main content and a sample of its clutter, such as menus and footers (firecrawl/scrape-content-dataset-v1, MIT licence). We ran both tools on the same 833 pages on 8 November 2026, with default settings, and scored both with the same code.
Firecrawl wins this test on coverage: 86% against 52%. Novagentum returns cleaner output: 24% of the clutter came through, against 43% for Firecrawl.
Coverage: pages where at least 10% of the main content was found
Higher is better. This is Firecrawl's own definition of coverage.
Show the data
| Item | Value | Note |
|---|---|---|
| Firecrawl | 86% | |
| Novagentum | 52% |
Clutter included: menu, footer and ad text in the output
Lower is better.
Show the data
| Item | Value | Note |
|---|---|---|
| Novagentum | 24% | |
| Firecrawl | 43% |
| Firecrawl | Novagentum | |
|---|---|---|
| Coverage (higher is better) | 86% | 52% |
| Main content found, on average (higher is better) | 71% | 43% |
| Clutter included, on average (lower is better) | 43% | 24% |
| Pages that returned content | 804 of 833 | 529 of 833 |
| Pages skipped because robots.txt disallows them | 0 | 29 |
Why we score lower, and what we are changing
Most of the gap comes from pages where Novagentum returned nothing, not from poor extraction. On the pages where we did return content, our coverage was 82%.
| Problem | Pages affected | What we are doing |
|---|---|---|
| robots.txt blocked by a firewall or unreadable | 142 | Under review. A firewall that blocks robots.txt usually blocks the page too. We will test again from our production servers. |
| Wrong part of the page picked as the main content | 94 | Planned: better main-content detection. |
| Empty result, mostly pages that need JavaScript | 84 | Paid Scrapes render these in a browser; this test used plain HTML. Planned: better detection of these pages. |
| robots.txt disallows the page | 29 | No change. We respect robots.txt on purpose. |
| Page too slow to parse | 22 | Planned: a faster parser for dense pages. |
| No response within 30 seconds | 12 | Scrapes can set a timeout of up to 60 seconds; this test used the default. |
| Page larger than 3 MB | 6 | No change for now. Pages up to 3 MB are converted. |
| Other: redirects to plain HTTP, unsupported files, network errors | 9 | No change. Redirects from HTTPS to plain HTTP are refused on purpose. |
We run this benchmark again after changes to Scrape and update this page with the latest results.
Limits of this test
- We ran Novagentum from a test machine, with plain HTML only, like a free Scrape. Paid Scrapes also render JavaScript pages in a real browser. Firecrawl ran on its own production service.
- Novagentum respects robots.txt. Pages a site does not allow are not scraped, and count as misses here.
- We left out 133 pages that had already failed when Firecrawl built the dataset, 20 pages with no reference text, and 14 PDFs.
- Firecrawl's dataset is from October 2025. Some pages have changed since then, for both tools equally.
- Firecrawl does not publish its scoring code, so our numbers for Firecrawl can differ from the ones it publishes.
Crawl: what each tool can see in a page's HTML
Many website problems are in the HTML code, not in the visible text. We asked each tool for five items on the same pages: the meta description, the canonical link, structured data (JSON-LD), the number of H1 headings and the noindex tag.
HTML items each tool could see, out of 5
Higher is better.
- ChatGPT (web search)
- It found the H1 on both pages tested, with thinking set to medium and to high.
- Claude app (Opus 5.5)
- Read the page's HTML when it could run code. Asked twice to audit the same site, it found different problems each time.
- Claude web fetch tool (Claude Code)
- The tool returns page text, not HTML.
Show the data
| Item | Value | Note |
|---|---|---|
| Novagentum Crawl | 5 of 5 | |
| ChatGPT (web search) | 1 of 5 | It found the H1 on both pages tested, with thinking set to medium and to high. |
| Claude app (Opus 5.5) | 5 of 5 | Read the page's HTML when it could run code. Asked twice to audit the same site, it found different problems each time. |
| Claude web fetch tool (Claude Code) | 0 of 5 | The tool returns page text, not HTML. |
Read the full results: ChatGPT vs Novagentum and Claude vs Novagentum.
How we test
- Same inputs. Every tool gets the same pages, on the same day, with its default settings.
- Same scoring. One piece of code scores every tool. We do not use an AI model to judge results.
- Public method and data. The test set, the scoring code and the result for every page can be downloaded below.
- Every result is dated. We run tests again after product changes and keep the history.
- We publish when we lose. A benchmark that only shows wins is not useful to you.
How the scrape test is scored
Text is compared word by word (character by character for Chinese, Japanese and Korean).
- Main content found: the share of the reference main-content words that appear in the tool's output.
- Coverage: a page counts as covered when at least 10% of its main content was found.
- Clutter included: the share of the reference clutter words (menus, footers, ads) that appear in the output.
A page that fails or returns nothing scores zero.
Download: the result for every page (CSV), the scoring code, the code that runs each tool, how the pages were chosen and notes on the run.
Try it on your own site
Run a Crawl of up to 100 pages, and start free. You will see every finding, the page it affects, and what we found there.