Round-Trip Translation Reveals What Frontier Multilingual Benchmarks Miss

This project identifies a critical 'blind spot' in current LLM evaluation: the conflation of reasoning (logic/math) with linguistic proficiency. By using Round-Trip Translation (RTT) as a diagnostic tool, it proves that 'thinking' models (like O1 or R1) can achieve high scores on multilingual benchmarks without actually understanding the target language, simply by brute-forcing the underlying logic. While the insight is high-value, the project's defensibility is low (score: 3) because it is a research-oriented reference implementation rather than a productized platform. The code is likely a set of evaluation scripts that can be easily replicated or integrated into existing suites like LM Evaluation Harness or HELM. Frontier labs are at 'medium' risk because they will likely absorb these findings to refine their internal evaluation pipelines, potentially making this specific standalone tool redundant. The quantitative signals (0 stars, 3 days old) reflect its status as a brand-new academic release. Its primary value is as a methodology to expose 'benchmark gaming' in non-English languages.