Visual document retrieval indexes a picture of each page instead of the text pulled out of it. On ViDoRe V3, the ACL 2026 benchmark built from roughly 26,000 enterprise pages and 3,099 human-verified queries, visual retrievers beat text pipelines outright. The catch for a Dutch reader sits in the paper's own limitations section: every document in that corpus is French or English, and Dutch is not one of the six query languages.
What ViDoRe V3 actually measured
ViDoRe V3 was built by ILLUIN Technology with contributions from NVIDIA, and it is a deliberate reaction to how soft the earlier benchmarks were. Ten corpora, about 26,000 pages, 3,099 queries, and 12,000 hours of human annotation for query writing, page relevance ranking, bounding boxes and reference answers. Eight datasets are public. Two are held privately by the MTEB team, a nuclear regulatory set and a telecom standards set, specifically so that models cannot be tuned against them.
The public corpora are the kind of documents that actually cause trouble: US and French annual reports, EU HR reports, French government energy reports, FDA reports, USAF technical orders, computer science textbooks and French physics lectures. Text, tables, charts, infographics and photographs, on pages laid out by people who were not thinking about your retrieval pipeline.
Three findings carry: visual retrievers outperform text ones, late-interaction models and text reranking both improve results substantially, and giving the generating model visual or hybrid context produces better answers than text alone. The best average on the English splits is 0.656 NDCG@10, from nemo-colembed-3b. For scale, colqwen2.5 sits at 0.592 and colpali-v1.3 at 0.530 on the same splits.
Read that top score honestly. The best available model, on a benchmark its authors designed to be fair rather than flattering, lands around two thirds. This is a direction that works, not a solved problem.
Why does embedding the page beat extracting the text?
ColPali, published at ICLR 2025, is the architecture the whole family descends from. A page screenshot goes into a vision language model as a 32 by 32 grid, so 1,024 image patches, plus six instruction tokens. Each of those 1,030 tokens is projected into a 128-dimensional vector. A query is tokenised into the same space, and scoring uses MaxSim, the late-interaction mechanism inherited from ColBERT: for every query token, take its best match across all page patches, then sum. Vespa's engineering write-up on scaling this to billions of pages is the clearest public description of the mechanics.
The reason it wins is what it skips. A conventional pipeline has to decide where a table's cells begin and end, what the reading order is, whether the stamp in the corner is content or noise, and how to chunk what comes out. Every one of those decisions is a chance to lose information before retrieval has even started. A page image contains none of those decisions. It is why the gain concentrates exactly where layout carries meaning: a werkbon with signature blocks and handwritten hours, a pakbon whose meaning lives in a table, a keuringsrapport where the photograph is the finding.
This is also the honest boundary with our earlier work on BM25 and vector search over text-and-table documents. That post was about what happens after extraction goes wrong. Visual retrieval sidesteps the extraction step entirely, which is a different and stronger claim.
What happens when the query is not in English?
Here is the part nobody writing about ViDoRe V3 in English has any reason to mention. All 3,099 queries are provided in six languages: English, French, Spanish, German, Italian and Portuguese. Dutch is not among them. The corpora are narrower still. The authors state the limitation plainly: the benchmark is currently limited to French and English documents, because resource constraints prevented broader coverage.
The paper does measure what cross-language costs. For the strongest models, translated queries cost 3 to 5 NDCG@10 points against English-only. The best multilingual average is 0.576, from jinav4, versus 0.656 for the best English score. Those two numbers are not the same measurement, and it is worth keeping them apart: the 3 to 5 points is what translating the query costs a given model, while the wider gap from 0.656 to 0.576 also folds in the French-language corpora, which are harder splits in their own right.
The number that should change an SME's plan is not that one. It is what the same table does to the small models. ColModernVBERT drops from 0.507 on English to 0.245 multilingual. ColSmol-256M drops from 0.464 to 0.214. The large models give up a few points. The small models give up roughly half their score. And the small models are precisely the ones a 20 to 50 person firm can afford to self-host on its own hardware.
So for a Dutch or Flemish document set you do not have a number. You have a plausible direction, an unmeasured language, and a documented pattern of the affordable end of the model range degrading hardest when the language shifts.
Build the only benchmark that applies to you
The fix is unglamorous and takes an afternoon.
- Take 300 pages out of your own archive, the real ones, including the crooked scans.
- Collect 40 to 60 questions from the people who actually ask them. The planner, the werkvoorbereider, the person who answers the phone when a customer asks what was replaced in 2023.
- Have a human mark which page or pages answer each question. This is the expensive part and there is no way around it.
- Measure recall@5 and recall@10 for two candidates: your current search, and a visual retriever. Then look at the failures one by one.
One warning, taken straight from why ViDoRe V3 cost 12,000 human hours: do not generate the questions from the pages with a language model. Synthetic queries derived from a page tend to reuse its vocabulary, which flatters every retriever you test and tells you nothing. The earlier ViDoRe versions leaned on synthetic generation and the V3 authors treated fixing that as a core contribution.
If the result is good, a Dutch benchmark is a community effort rather than a research project. Korean already has one: KoViDoRe was built independently in the same shape and is now mirrored into MTEB. As of this writing there is no Dutch equivalent, which is a gap somebody in the Netherlands or Flanders could close in a quarter.
Retrieval is not extraction
This distinction gets collapsed in almost every vendor deck, and collapsing it is how projects end up delivering the wrong thing.
ViDoRe measures whether the right page comes back. Posting a werkbon into AFAS needs typed fields: hours, material lines, a project code, a signature that was or was not given. That is extraction, and a visual retriever does not do it. It hands you a page.
The architecture that actually works is both, in order. Visual retrieval narrows thousands of pages to the handful that matter, then a vision language model or a structured extractor reads those few pages properly. Vespa frames it the same way, and for the same reason: running a frontier model over every page in the archive for every question is not a pipeline, it is a bill.
Which also means that if your whole use case is getting an invoice total into the ledger, none of this is your problem. That is a straight run at invoice processing and an integration into Exact, AFAS or e-Boekhouden, and it does not need a page encoder.
What does a page-image index cost to store?
Storage was the standard objection to this architecture, and it stopped being a good one some time ago. Take a round 100,000 pages, roughly ten years of paperwork for a 30-person installatiebedrijf, and assume ColPali-style encoding at 1,030 vectors of 128 dimensions per page.
- At float32: 1,030 x 128 x 4 bytes is about 527 KB per page, so around 53 GB for the archive.
- Binary quantised: Vespa packs each 128-dimensional vector into 128 bits, so 16 bytes per patch and roughly 16 KB per page. That is about 1.6 GB, a 32x saving.
Vespa also measured what that costs in quality rather than asserting it was free, in engineering work published back in September 2024. On DocVQA, nDCG@5 went from 52.4 with float vectors to 49.5 with binary, and back to 51.6 once a float reranking pass was added over the binary candidates. Their Hamming-based MaxSim ran about 3.5 times faster than the float dot product version in the same test.
Two 2026 results push harder. The Visual RAG Toolkit, a February 2026 preprint submitted to the SIGIR demonstrations track rather than a peer-reviewed result, applies training-free spatial pooling to cut stored vectors per page from thousands to dozens for candidate generation, then reranks exactly with the full multi-vector embeddings, reporting roughly 4x throughput while largely preserving NDCG and recall at 5 and 10 on ViDoRe V2. NeoMME, released under Apache 2.0 at the end of August 2026, reports hierarchical token pooling and asymmetric quantisation compressing its own late-interaction embeddings by 255x while keeping over 95 percent of baseline nDCG@10, with a 260M-parameter retriever scoring 0.523 on ViDoRe V3.
The remaining cost is not the index. It is the GPU on the ingest side, because every page has to be encoded once.
So should a Dutch SME switch its document pipeline?
Move if all three hold: your source documents are scans or PDFs where layout carries meaning, the questions people ask are extractive or numerical rather than open-ended, and you have somewhere to run a page encoder. The query-type table in ViDoRe V3 is worth taking seriously here. Extractive queries score 0.668 and numerical 0.633, while open-ended land at 0.438 and multi-hop at 0.515. Most questions an SME asks of its own archive are extractive. That is the half of the distribution where this works.
Wait if your source is already structured. If the invoice arrives as a Peppol document or the data can be exported from the ERP, you are about to solve a problem you do not have. Wait too if what you need is a synthesis across fifty documents, because that is the 0.438 column.
And in every case, measure it in Dutch before you commit. That is not caution for its own sake. It is the one variable the published benchmark does not cover, and it is the variable your users live in. We wrote separately about what to measure in a pilot and about what this class of work costs to build; the short version is that the eval set is the least expensive part of the project and the only part that tells you whether to continue.
Last updated 11 September 2026. ViDoRe V3 results are from the published benchmark and the MTEB leaderboard; ColPali code is at illuin-tech/colpali.
