realvuln.com is hosted on Amazon CloudFront. It measures traffic with Google Analytics and Google Tag Manager.
| Technology | Version | First seen |
|---|---|---|
| CDN & Hosting3 | ||
| – | 7 Nov 2026 | |
| – | 7 Nov 2026 | |
| – | 7 Nov 2026 | |
| Analytics & Tag Management2 | ||
| – | 7 Nov 2026 | |
| Fonts & Icon Fonts1 | ||
| – | 7 Nov 2026 | |
Registration from the registry's RDAP record, the certificate the site serves, and its rank in public traffic lists.
The crew hasn’t read this site’s colors yet. We read a palette when we capture a site’s screenshot, busiest sites first. Meanwhile, browse palettes from other sites.
Counted from every stack change we've recorded: of the sites we watched running one of this site's tools, how many went on to add another. A pattern across sites, not a forecast for this one.
ns-1707.awsdns-21.co.ukns-619.awsdns-13.netns-1401.awsdns-47.orgns-242.awsdns-30.comUser-agent: * Allow: / # Do NOT add "Disallow: /v/" here. The frozen release snapshots under /v/ are # de-indexed by a noindex,follow tag in their own <head> (see release.py). # Blocking the path would stop crawlers from ever reading that tag, so already # indexed snapshots would stay in the results and keep competing with the live # pages — the opposite of the intent. They must remain crawlable to be dropped. Sitemap: https://realvuln.com/sitemap.xml
# RealVuln > An open, reproducible benchmark measuring how well security scanners find real vulnerabilities in real code. Version 3.1: 203 pinned Python repositories, 7,118 hand-labeled vulnerabilities and 280 false-positive traps, 45 scanners compared. Ranked by F3 (recall-weighted). Every ground-truth label, scanner output and scoring script is public so results can be reproduced independently. RealVuln is maintained by John Pellew and Faizan Raza of Kolega.Dev, whose scanner is itself one of the evaluated tools — hence the open audit trail. ## Core pages - [Benchmark overview](https://realvuln.com/): headline leaderboard and what the benchmark measures. - [Interactive dashboard](https://realvuln.com/dashboard.html): full leaderboard, precision-recall, cost-efficiency and per-category breakdowns. - [Methodology](https://realvuln.com/methodology.html): benchmark design, ground-truth labeling, finding-matching algorithm and F3 scoring. - [Dataset](https://realvuln.com/dataset.html): the corpus, ground-truth schema, false-positive traps and framework coverage. - [Findings](https://realvuln.com/findings.html): results and analysis, including the three-tier scanner hierarchy and per-CWE detection. - [Roadmap & contributing](https://realvuln.com/roadmap.html): versioning policy, how to contribute scanners or repositories, and how to cite. - [Paper](https://realvuln.com/paper.html): abstract, headline results, common questions and citation for arXiv:2604.13764. ## Per-scanner results - [Kolega DevSec Core V0.0.1 (Kolega DevSec Platform)](https://realvuln.com/scanners/kolega-devsec-core-v0.0.1.html): per-repository scores, detection breakdown and run conditions. - [Kolega DevSec Max V0.1.0 (Kolega DevSec Platform)](https://realvuln.com/scanners/kolega-devsec-max-v0.1.0.html): per-repository scores, detection breakdown and run conditions. - [Kolega Scan OSS - V2 - 3 Model (Kolega Scan OSS)](https://realvuln.com/scanners/kolega-claude-adaptation.html): per-repository scores, detection breakdown and run conditions. - [Kolega Scan OSS - V1 - 2 Model (Kolega Scan OSS)](https://realvuln.com/scanners/kolega-claude-adaptation-deepseek-only.html): per-repository scores, detection breakdown and run conditions. - [Anthropic Method (DeepSeek V4 Pro) (Bespoke Adaptation)](https://realvuln.com/scanners/kolega-original-claude-adaptation-deepseek-v4-pro.html): per-repository scores, detection breakdown and run conditions. - [Anthropic Method - Sonnet 4.6 (Claude Code)](https://realvuln.com/scanners/kolega-ca-cc-sonnet.html): per-repository scores, detection breakdown and run conditions. - [GPT-5.5 (agentic-v1)](https://realvuln.com/scanners/gpt-5.5-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [GPT-5.6 Luna (codex-cli)](https://realvuln.com/scanners/gpt-5.6-luna-codex-cli.html): per-repository scores, detection breakdown and run conditions. - [GPT-5.6 Terra (codex-cli)](https://realvuln.com/scanners/gpt-5.6-terra-codex-cli.html): per-repository scores, detection breakdown and run conditions. - [GPT-6 Astra (codex-cli)](https://realvuln.com/scanners/gpt-6-astra-codex-cli.html): per-repository scores, detection breakdown and run conditions. - [GPT-5.6 Sol (codex-cli)](https://realvuln.com/scanners/gpt-5.6-sol-codex-cli.html): per-repository scores, detection breakdown and run conditions. - [Daybreak Blue (codex-cli)](https://realvuln.com/scanners/gpt-daybreak-blue-codex-cli.html): per-repository scores, detection breakdown and run conditions. - [Sonnet 5 (Claude Code)](https://realvuln.com/scanners/claude-sonnet-5-cc-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [GLM-5.3 (agentic-v1)](https://realvuln.com/scanners/glm-5.3-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [GLM-5.2 (agentic-v1)](https://realvuln.com/scanners/glm-5.2-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [GLM-5.1 (agentic-v1)](https://realvuln.com/scanners/glm-5.1-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [GLM-5 (agentic-v1)](https://realvuln.com/scanners/glm-5-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [DeepSeek V4.1 Flash (agentic-v1)](https://realvuln.com/scanners/deepseek-v4.1-flash-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [DeepSeek V4 Flash (agentic-v1)](https://realvuln.com/scanners/deepseek-v4-flash-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [DeepSeek V4 Pro (agentic-v1)](https://realvuln.com/scanners/deepseek-v4-pro-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [Kimi K3 (agentic-v1)](https://realvuln.com/scanners/kimi-k3-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [Kimi K2.7 (agentic-v1)](https://realvuln.com/scanners/kimi-k2.7-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [Kimi K2.6 (agentic-v1)](https://realvuln.com/scanners/kimi-k2.6-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [Kimi K2.5 (agentic-v1)](https://realvuln.com/scanners/kimi-k2.5-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [Claude Fable 5 (claude-code-v1)](https://realvuln.com/scanners/claude-fable-5-cc-v1.html): per-repository scores, detection breakdown and run conditions. - [Claude Opus 5 (agentic-v1)](https://realvuln.com/scanners/claude-opus-5-cc-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [Claude Opus 4.8 (agentic-v1)](https://realvuln.com/scanners/claude-opus-4-8-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [Claude Opus 4.7 (agentic-v1)](https://realvuln.com/scanners/claude-opus-4-7-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [Claude Opus 4.6 (agentic-v1)](https://realvuln.com/scanners/claude-opus-4-6-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [Claude Sonnet 4.6 (agentic-v1)](https://realvuln.com/scanners/claude-sonnet-4-6-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [Claude Haiku 4.5 (agentic-v1)](https://realvuln.com/scanners/claude-haiku-4-5-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [Claude Haiku 4.5 (direct-v1)](https://realvuln.com/scanners/claude-haiku-4-5-v1.html): per-repository scores, detection breakdown and run conditions. - [Gemini 3.1 Pro (agentic-v1)](https://realvuln.com/scanners/gemini-3.1-pro-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [Gemini 3.5 Flash (agentic-v1)](https://realvuln.com/scanners/gemini-3.5-flash-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [Grok 4.20 Reasoning (agentic-v1)](https://realvuln.com/scanners/grok-4.20-reasoning-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [Grok 3 (agentic-v1)](https://realvuln.com/scanners/grok-3-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [Minimax M2.7 (agentic-v1)](https://realvuln.com/scanners/minimax-m2.7-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [Qwen 3.5 397B (agentic-v1)](https://realvuln.com/scanners/qwen-3.5-397b-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [Qwen 3.6 35B (agentic-v1)](https://realvuln.com/scanners/qwen3.6-35b-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [Gemma 4 31B (agentic-v1)](https://realvuln.com/scanners/gemma4-31b-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [Ornith 1.0 35B (agentic-v1)](https://realvuln.com/scanners/ornith-q3-agentic-v1.html): per-repository scores, detection breakdown and run conditions. - [Rowan (0.2.0)](https://realvuln.com/scanners/rowan.html): per-repository scores, detection breakdown and run conditions. - [Semgrep (rule-based)](https://realvuln.com/scanners/semgrep.html): per-repository scores, detection breakdown and run conditions. - [Snyk Code (pattern+flow)](https://realvuln.com/scanners/snyk.html): per-repository scores, detection breakdown and run conditions. - [SonarQube (community)](https://realvuln.com/scanners/sonarqube.html): per-repository scores, detection breakdown and run conditions. ## Journal - [RealVuln Journal](https://realvuln.com/journal.html): the narrative dev-log — every release, benchmark run, and methodology change, dated and in order. - [First LLM scanner runs: the agentic models enter the leaderboard](https://realvuln.com/journal/first-llm-scanner-runs.html) - [Frontier and community scanners: Claude Opus 5 and the first external contribution (Rowan)](https://realvuln.com/journal/frontier-and-community-scanners.html) - [Introducing the RealVuln benchmark framework](https://realvuln.com/journal/introducing-the-benchmark.html) - [Local models and GPT-5.6: can open-weight scanners compete?](https://realvuln.com/journal/local-models-and-gpt-5-6.html) - [Non-scoring ground truth: what to do with a finding you cannot grade](https://realvuln.com/journal/non-scoring-ground-truth.html) - [Reproducible by design: prompt versioning and the manifest](https://realvuln.com/journal/reproducible-by-design.html) - [Roster update: GPT-6 Astra, Claude Sonnet 5 and GLM-5.3 join the Overall leaderboard](https://realvuln.com/journal/roster-update-astra-sonnet-5-glm-5-3.html) - [Scaling the roster: Opus 4.7, GLM-5.1, Kimi K2.6, GPT-5.5 and DeepSeek-V4](https://realvuln.com/journal/scaling-the-scanner-roster.html) - [The paper, and why we score F3 (recall weighted nine times over precision)](https://realvuln.com/journal/the-paper-and-the-f3-metric.html) - [Most scanners get worse on TypeScript, and the drop is mostly precision](https://realvuln.com/journal/the-python-to-typescript-drop.html) - [A second language: the TypeScript/JavaScript corpus and per-language leaderboards](https://realvuln.com/journal/the-typescript-corpus.html) - [The v2 corpus: adding 40 LLM-generated applications and measuring authorship](https://realvuln.com/journal/the-v2-corpus-llm-generated-code.html) - [RealVuln 1.0.0: freezing the first release and opening the public site](https://realvuln.com/journal/v1-0-0-release.html) - [RealVuln 2.0.0: the full 66-repo, false-positive-aware release](https://realvuln.com/journal/v2-0-0-release.html) - [RealVuln 2.1.0: the current release and the dataset export](https://realvuln.com/journal/v2-1-0-release.html) - [RealVuln 3.0.0: TypeScript/JavaScript, non-scoring entries, and per-language leaderboards](https://realvuln.com/journal/v3-0-0-release.html) - [RealVuln 3.1.0: DeepSeek V4.1 Flash, and a harness bug that was costing every agentic scanner coverage](https://realvuln.com/journal/v3-1-0-release.html) - [Why we built RealVuln: the false-positive problem no benchmark measured](https://realvuln.com/journal/why-we-built-realvuln.html) ## Optional - [Paper (PDF)](https://realvuln.com/assets/realvuln-paper.pdf): arXiv:2604.13764. - [Source & ground truth](https://github.com/kolega-ai/Real-Vuln-Benchmark): Apache 2.0. - [Archived releases](https://realvuln.com/versions.json): frozen, citable snapshots under /v/<version>/. Cite these for reproducibility; prefer the live pages above for current results.
| – |
| 7 Nov 2026 |