Benchmark · last verified live run October 13, 2026
Every gate, including the ones we fail.
We test Verifex through its live screening API on names chosen before the test: real listed names, the spellings that hide them, other alphabets, and clean names that must not alert. We publish the rules, the confidence intervals and the failures. Sweden's financial regulator found no established benchmark index for how well these tools work. This is not one: it is our own test of our own engine, published with its rules and its failures.
What this measures
The engine as deployed on October 13, 2026.
Pre-registered gates
The rules were fixed before the test ran.
Each gate was written and approved before a single name was sent. An answer the engine could not give conclusively counts against it, not for it. Of 7 gates, none passed on this run.
G1
FailEvery listed name, exactly as published, alerts
Rule: 0 misses in at least 2,995 cases (recall at least 99.9%, one-sided 95%)
3,050 of 3,053 alerted and none was served clear. 3 came back unresolved, which the rule counts against: exact lower bound 99.746%, below 99.9%.
G2
FailEvery must-alert variation alerts
Rule: 0 misses in 300 for each must cell, plus cross-list spellings and Cyrillic and Arabic script
All 8 variation cells, cross-list spellings and both scripts fail. Worst: legal-form variants 54.3%, one-letter typos 77.7%.
G3
FailClean names rarely alert
Rule: false-alert rate upper 95% bound at most 10 per 10,000 easy clean names
213 false alerts in 9,202 resolved clean names: 2.3%, upper bound 264 per 10,000. Every company name ran but only part of the person names, which alert more often, so this understates the full set; inconclusive under G7.
G4
Not evaluablePolitical exposure is kept apart from sanctions
Rule: 0 PEP-only critical; 0 PEP-and-sanctioned below critical
Suite F did not run in this test (quota). In the previous run (October 8) it failed: no response in either run was served as critical, so a person who is both PEP and sanctioned returned high.
G5
Not measuredSame answer under concurrent load
Rule: 0 must-band decisions differ between concurrency 1 and 3
No concurrency records in this run.
G6
Not measuredThe regression suite behaves as before
Rule: Suite H (239 earlier cases) answers as it did
Suite H is not in this run's case file, so this gate could not be evaluated.
G7
InconclusiveFew enough answers are unresolved to read a result
Rule: at most 10% unresolved per suite, else the suite is inconclusive
Variations 13.5%, scripts 42.4% (mostly Chinese and Korean, which are not screened yet), clean names 32.8%.
- screens answered
- 28,420
- of 37,543 pre-registered cases; 582 more refused for quota
- answered “screening unavailable”
- 1,057
- counted as unresolved, never as clear
- listed names alerted, as published (OFAC and UN)
- 99.9%
- 95% CI 99.71%–99.97%
- must-alert spellings alerted, pooled
- 87.8%
- 95% CI 86.5%–89.1%
How listed names hide
Found, by the way the name was changed.
Every case is a real listed name: as published, changed in one documented way, spelled as another official list spells it, or written in its own script. Switch the band to see what the policy requires. An answer the engine could not give conclusively counts against the cell.
An unresolved answer counts against the cell, as pre-registered.
A miss here fails the release (policy §2).
- Listed names, exactly as published99.9%99.71–99.97%
- Case, punctuation or spacing changed99.0%97.1–99.66%
- Accents added or removed97.7%95.3–98.9%
- Company legal form added, dropped or swapped54.3%48.7–59.9%
- Name particle joined or split (al-, bin, von)89.7%85.7–92.6%
- Name parts in a different order99.0%97.1–99.66%
- Official transliteration (BGN/PCGN)92.0%88.4–94.6%
- Passport transliteration (ICAO 9303)93.3%89.9–95.6%
- One-letter typo (names of 8+ letters)77.7%72.6–82.0%
- Spelling another official list uses for the same subject99.3%98.2–99.72%
- Original script: Cyrillic91.9%88.2–94.5%
- Original script: Arabic97.3%94.8–98.6%
Show the data
| Variation | Band | Cases | Alerted | Not found | Unresolved | Alerted % | 95% CI |
|---|---|---|---|---|---|---|---|
| Listed names, exactly as published | must | 3053 | 3050 | 0 | 3 | 99.9% | 99.71–99.97% |
| Case, punctuation or spacing changed | must | 300 | 297 | 1 | 2 | 99.0% | 97.1–99.66% |
| Accents added or removed | must | 300 | 293 | 0 | 7 | 97.7% | 95.3–98.9% |
| Company legal form added, dropped or swapped | must | 300 | 163 | 46 | 91 | 54.3% | 48.7–59.9% |
| Name particle joined or split (al-, bin, von) | must | 300 | 269 | 13 | 18 | 89.7% | 85.7–92.6% |
| Name parts in a different order | must | 300 | 297 | 0 | 3 | 99.0% | 97.1–99.66% |
| Official transliteration (BGN/PCGN) | must | 300 | 276 | 3 | 21 | 92.0% | 88.4–94.6% |
| Passport transliteration (ICAO 9303) | must | 300 | 280 | 4 | 16 | 93.3% | 89.9–95.6% |
| One-letter typo (names of 8+ letters) | must | 300 | 233 | 25 | 42 | 77.7% | 72.6–82.0% |
| Spelling another official list uses for the same subject | must | 561 | 557 | 0 | 4 | 99.3% | 98.2–99.72% |
| Original script: Cyrillic | must | 296 | 272 | 0 | 24 | 91.9% | 88.2–94.5% |
| Original script: Arabic | must | 297 | 289 | 0 | 8 | 97.3% | 94.8–98.6% |
Context
What regulators measured at banks.
Supervisors run the same kind of test inside banks. Their files, lists and measures differ from ours, so read this as a guide, not a ranking.
| Who measured | Names as listed | Altered names | Scope |
|---|---|---|---|
| Verifex, October 13, 2026 | 99.9% | 77.2% all variations87.8% of the must-alert ones | Our own test through the live API: 3,053 listed names (OFAC and UN), 4,500 altered spellingsMeasure: share of test names that alerted, unresolved counted against |
| Finansinspektionen (Sweden), 2024 | 86.3% average99.3% median | 63.9% average73.4% median, range 5.0%–96.1% | 19 banks, 21 customer-screening systems, tested by an external firm on UN and EU list namesMeasure: share of test names caught by customer screening, averaged across banks |
The UK's Financial Conduct Authority (UK) reported in May 2026 that 90% of alerts raised in its testing on exact names, and 75% on variants, identified the right sanctioned party. That measures whether an alert names the right party, which this run did not score, so it is not in the table.
Other alphabets
Names in the script the list publishes.
A name the engine cannot screen yet is answered HTTP 503, screening unavailable, and marked review, never clear: route it to a person. Armenian, Georgian, Chinese (Han), Korean (Hangul), Japanese, Thai names were in that state on this run; we show them as not screened, not as found.
Cyrillic
296 cases · gated
91.9%
ScreenedArabic
297 cases · gated
97.3%
ScreenedArmenian
2 cases
—
Not screened yet: returned as reviewGeorgian
1 case
—
Not screened yet: returned as reviewGreek
169 cases
58.6%
Partly screenedChinese (Han)
300 cases
—
Not screened yet: returned as reviewKorean (Hangul)
92 cases
—
Not screened yet: returned as reviewHebrew
35 cases
100.0%
ScreenedJapanese
8 cases
—
Not screened yet: returned as reviewThai
11 cases
—
Not screened yet: returned as review
Show the data
| Script | Cases | Alerted | Unresolved | Not found | Alerted % | 95% CI |
|---|---|---|---|---|---|---|
| Cyrillic | 296 | 272 | 24 | 0 | 91.9% | 88.2–94.5% |
| Arabic | 297 | 289 | 8 | 0 | 97.3% | 94.8–98.6% |
| Armenian | 2 | 0 | 2 | 0 | 0.0% | 0.0–65.8% |
| Georgian | 1 | 0 | 1 | 0 | 0.0% | 0.0–79.3% |
| Greek | 169 | 99 | 70 | 0 | 58.6% | 51.0–65.7% |
| Chinese (Han) | 300 | 0 | 300 | 0 | 0.0% | 0.0–1.3% |
| Korean (Hangul) | 92 | 0 | 92 | 0 | 0.0% | 0.0–4.0% |
| Hebrew | 35 | 35 | 0 | 0 | 100.0% | 90.1–100.0% |
| Japanese | 8 | 0 | 8 | 0 | 0.0% | 0.0–32.4% |
| Thai | 11 | 0 | 11 | 0 | 0.0% | 0.0–25.9% |
Clean names and workload
What false alerts cost a compliance team.
Of 13,700 clean names screened, 9,202 received a conclusive answer and 213 of those alerted: 2.31% (95% CI 2.03%–2.64%), against a target of 0.1%. Another 32.8% of the names screened came back unresolved, which makes this result inconclusive under the benchmark's own rule, and the person names that alert most often ran only in part. The simulator shows what such rates mean at your volume.
An illustration, not a prediction. Rates come from the October 13, 2026 measurement of the engine as deployed then. The clean-name rate is inconclusive under the benchmark's own rule (too many unresolved answers), understates the full clean set (person names ran only in part), and its clean names were checked against four lists. Ranges carry each rate's 95% interval.
- Listed names screened
- 10
- 1 in 10,000 of 100,000
- Listed names that alert
- 10
- 99.9% measured
- Listed names served as clear or PEP-only
- 0
- 0.00% measured
- False alerts on clean names
- 1,361–1,776
- 1.6% of clean names screened
- Unresolved: a person must look
- 32,829
- 0 listed, 32,829 clean; never served as clear
- Of every 100 alerts, real listed parties
- about 0.6
- precision at this rate
Precision = recall × p ÷ (recall × p + false-alert rate × (1 − p)), where p is the share of screened names that are listed and the false-alert rate is per clean name screened. When real matches are rare, almost every alert is a false one, even with a low false-alert rate: that is why the false-alert rate decides the workload.
Show the data
| Listed names | Share of alerts that are real |
|---|---|
| 1 in 100 | 39.36% |
| 1 in 300 | 17.69% |
| 1 in 1,000 | 6.04% |
| 1 in 3,000 | 2.10% |
| 1 in 10,000 | 0.64% |
| 1 in 30,000 | 0.21% |
| 1 in 100,000 | 0.06% |
Methodology
How the test is built.
Cases are drawn by fixed seeds from the published lists and from independent clean populations, frozen and fingerprinted, then sent once each through the same API a customer calls.
- 01
Pre-register
Questions, suites, the alert policy and every gate are written and approved before the first call.
- 02
Freeze
The case file is fixed and fingerprinted, so any later change to it is visible.
- 03
Serve
Each case goes once through the public screening API, in the account's real plan.
- 04
Score and bound
Counts per cell, Wilson 95% intervals, exact one-sided bounds for the gates, and unresolved counted against.
Self-administered, and labelled that way
Verifex designed and ran this test. Its results were re-derived from the raw served log by a separate AI agent with no build context, which is a check, not independent certification. The study explains how a bank's validator can rerun it.
Limitations
What these numbers do not prove.
- 01
This measures the engine as deployed on October 13, 2026. Changes deployed after the run (from October 17, 2026) and changes prepared on October 22 (legal forms, alias typos, identifier search, a review floor for sanctions candidates, input fail-safes) are not measured here; the next live run measures them.
- 02
The run stopped after 29,002 of 37,543 cases when the test account's quota ran out; the last 582 were refused (HTTP 402), so 28,420 were answered, 1,057 of them screening unavailable. Suite F (PEP separation), Suite G (123 cases) and the 2,000 hard clean names never ran.
- 03
Listed names come from two lists: OFAC (3,016) and UN (384); the listed-name suite has no EU or UK name.
- 04
The clean names that ran are all 10,000 company names (GLEIF) but only 3,700 scored person names (ParaNames). Person names false-alert far more often (11.99% against 1.57% of resolved names), so the clean-name rate understates the full set. On the 13,700 names this run and the October 8, 2026 run both decided, false alerts rose from 197 to 213.
- 05
An answer of political exposure only ("match") on a clean name is not counted as a false alert, as the benchmark's scorer specifies. Its alert policy read literally would count the 134 such answers: 347 of 9,267, 3.7%.
- 06
Self-administered and not independent. The same team built the engine and the test, and some of the October fixes were tuned on these same test cases, so the next run on them is a measurement of production, not proof that the fixes generalise. The exposure is recorded in the benchmark's contamination log.
- 07
Clean names were checked against OFAC, UN, EU and UK only, while the test account's plan screened 41 lists; some alerts counted as false may be genuine namesakes on another list.
- 08
546 answers were served from the API's cache rather than computed for this run.
- 09
The pre-registration's run-identity section (its own freeze hash, plan, cache bypass) was never filled in. The rules themselves were committed on October 7, 2026, six days before the run.
- 10
Mostly name-only queries. Real traffic often carries a date of birth or country, which this test rarely does.
- 11
An alert is counted when it is raised, not yet when it names the right listed subject; that check comes with the next run.
- 12
Results describe these cases and this configuration. A different mix of names or lists can move them.
Correction · October 8, 2026
A change we called an improvement also under-called real matches.
What we published
Weak resemblance resolves to a watch signal.
Presented as less analyst noise: near-miss names would stop surfacing at severe levels.
What it also did
Some genuine sanctions matches were served as low.
A fuzzy or phonetic match on a designated party could be returned at a low risk level, without the review flag. Corrected on October 8, 2026 for matches at or above 75% confidence; matches between 50% and 75% are still being addressed.
Earlier study · September 7, 2026
Tested on the aliases OFAC publishes.
A separate alias-lookup study on a local OFAC-only database, before the October engine changes. It is not a measurement of the live API, and it found the gap between made-up test names and real aliases that shaped the benchmark above.
Read the alias studyRetrieval experiment · rejected
We built FAISS. The evidence said no.
Semantic vector retrieval was tested against the same corpus and turned off. It added a measurable precision cost without a recall improvement distinguishable from zero. It remains disabled.
+4.4pp
Precision cost
More severe false positives
+0.1pp
Recall change
Confidence intervals overlap
Off
Production state
Rejected for this corpus
Historical archiveEarlier measurements, kept for continuityIt is not the current production result. Each describes an earlier engine.+
Production Engine vNext, measured August 12, 2026
300 / 300
Sanctions positives actionable
97.33%
Exact-name recall
0 / 150
PEP-only at sanctions grade
May 2026 Engine v3 synthetic benchmark
99.68%
F1
99.36%
Recall
100%
Precision
500
Synthetic cases
Reproduce it
Check our numbers.
The case file is fixed and fingerprinted (SHA-256 110825465fe0d838…), and every rate here comes with the per-cell counts behind it, published as data with the study, and its interval is recomputed from them. A bank validator can ask us to rerun the test on their own account; publishing the full case files waits on a licence check of the list-derived names.