# Scrape content benchmark, 8 November 2026

- Test set: firecrawl/scrape-content-dataset-v1 (MIT licence), https://huggingface.co/datasets/firecrawl/scrape-content-dataset-v1
- Pages scored: 833 of 1,000. Left out: 133 rows the dataset marks as failed, 20 rows without reference text, 14 PDFs (see `benchmarkRows` in dataset.mts.txt).
- Providers, default settings, Markdown output:
  - Firecrawl: /v2/scrape, run on 8 November 2026 from a test machine. Requests the free plan rate-limited were retried, and the waiting time was not counted.
  - Novagentum: the production `scrapePage()` with plain HTML fetching (no JavaScript rendering), run from a test machine. Code as on main on 8 November 2026, which follows redirects to other websites and converts HTML pages up to 3 MB.
- Scoring: score.mts.txt. Every provider is scored by the same code. results.csv has one row per page and provider.
