TPRM Benchmark · SecurityScorecard
Help build the world’s first TPRM benchmark.
A founding research effort to measure whether AI can investigate third-party risk, resist pressure, and justify decisions from evidence.
Prelaunch research cohort · release 2026.11 · 120 admitted tasks · anonymous provisional scores published
Why this benchmark exists
AI systems are entering vendor-risk workflows before the field has a shared way to test them. TPRM Benchmark gives researchers and practitioners a frozen, inspectable instrument for measuring operational performance and robustness.
Two questions, one evaluation
- TPRM Performance
- Can the system investigate vendor evidence and reach the correct risk decision?
- Model Robustness
- Can it preserve evidence integrity, resist misleading claims, and reason through security failures?
How participation works
- Join the founding research cohort through Driftnet.
- Run a private evaluation with your provider, model, or agent.
- Help validate the instrument before results become a public ranking.
Provisional results
The leaderboard publishes measured scores and ranks for ten anonymous model classes. Direct model identities remain withheld until external validation and cohort review are complete.
TPRM GYM / IMPROVE AND RETEST
Don’t stop at a score.
Test a better strategy.
A benchmark reveals where an AI struggles. The TPRM Gym uses those findings to develop strategy guidance, then tests whether the same model can solve more tasks on separate evidence.
Same model. A revised strategy.
Improvement has to be measured.
Driftnet sign-in required · Your provider account
More TPRM tasks solved.
in TPRM Performance
Two more tasks solved. More evidence needed.
This experiment did not meet the Gym’s promotion threshold (paired p = 0.50). The strategy was not promoted. A higher score in one small test is a reason to investigate further, not a guarantee of improvement.
20 performance + 4 security
- 01Learn from the evaluation
Identify where the model’s investigation can improve.
- 02Develop a strategy
Add reusable guidance to the same model’s instructions.
- 03Put the change to the test
Compare both strategies on the same separate validation tasks.
See the comparison, scope and limitations
| Measurement | Fresh baseline | With strategy | Change |
|---|---|---|---|
| TPRM Performance | 7/20 · 35.0% | 9/20 · 45.0% | 10.0 pp |
| Security subset | 4/4 · 100.0% | 4/4 · 100.0% | 0.0 pp |
| All validation tasks | 11/24 · 45.8% | 13/24 · 54.2% | 8.3 pp |
What was compared: the same model and configuration, once with its starting instructions and once with the proposed strategy, on identical validation tasks. unguided mode; provider default reasoning; 4,096 maximum output tokens per call.
Why this is not the original leaderboard score: the source evaluation covered 120 tasks. Only 15 of these 24 validation task IDs overlap with that run, and the underlying datasets differ. The Gym remeasured a baseline on the fresh validation dataset to make a paired comparison possible.
Evidence strength: 2 tasks were solved only with the strategy; 0 were solved only by the baseline. Exact two-sided paired McNemar p = 0.50. This single, limited experiment does not establish repeatability or a general improvement for the model.
Scope: no model-security tasks were included. The four security tasks do not represent the full Model Robustness dimension. These results are separate from the 120-task leaderboard; publishing an improved leaderboard entry would require a complete, independently reviewed confirmation evaluation.
Open scrutiny, protected holdout
Methods, release identity, aggregate coverage, and representative examples are public. Scored answers and evaluation scaffolding remain protected so the benchmark can test capability rather than memorization.
Where the research goes next
A proposed threat-informed TPRM track will study prioritization and reverse blast radius across critical supplier relationships. It is a research direction, not part of the current admitted release.