Standard benchmarks for language models use binary accuracy metrics that classify each answer as correct or incorrect. Real university exams, however, often employ non-additive scoring schemes that weight partial knowledge differently. A study published on arXiv on August 18, 2026, quantifies this discrepancy for the first time using Vietnam's 2025 high school graduation exam and shows: Eight tested language models are overestimated by standard metrics by 0.020 to 0.159 points per question.
Convex scoring scheme penalizes partial knowledge more heavily
The 2025 reform of the Vietnamese National High School Graduation Examination introduced a convex point scheme in Part II. Candidates evaluate four true/false statements per question. The number of correct statements yields zero, 0.10, 0.25, 0.50, or 1.00 points. Three correctly identified statements earn 0.50 points, not the 0.75 points that a proportional metric would award. Part II accounts for 4.00 out of 10.00 total points.
The authors developed the THPT-Ladder Benchmark, which precisely maps the ministry's scoring practice. The benchmark comprises 632 items from 21 official exams across eleven subject areas. Since the Vietnamese Ministry publishes results from over one million candidates, language models can be directly placed within the human cohort.
Point discrepancy shifts model by 13 percentile points
The official rubric awarded 0.020 to 0.159 fewer points per Part II question on the eight tested models than proportional credit would. These differences have significant consequences: The model Qwen3.5-27B achieved a point shortfall of 0.042 points in the 2025 history exam. This shifted the model from the 90th to the 77th percentile among 481,293 candidates.
For Claude Sonnet 5, different error distributions at the same accuracy level resulted in scores between 0.869 and 0.932 points per question—a difference of 0.063 points (7.3 percent variation). Official marks depend on how correct statements are grouped. Model accuracy alone is not predictive of actual scoring under the convex scheme.
Systematic error source in evaluation
The study demonstrates a systematic error source: Simply replacing complex scoring schemes with binary accuracy metrics overestimates reported scores. This is particularly problematic for exams where partial credit schemes are non-linear. The discrepancy is not randomly distributed but depends on error patterns.
The authors conclude: To correctly evaluate AI models on real exams, actual scoring schemes must be implemented. Standard accuracy metrics can misrepresent a model's ranking by up to 13 percentile points, thereby underestimating the systemic problem of non-proportional partial credits.