MMLU-CF: A Contamination-free Multi-task Language Understanding Benchmark

MMLU-CF addresses a critical pain point in the LLM industry: the 'leaking' of benchmark questions into the training data of frontier models. While theoretically valuable, the project suffers from a lack of adoption, evidenced by 0 stars despite being over a year old. The 11 forks suggest some academic interest, but it lacks the community momentum required to become an industry standard like the original MMLU or newer alternatives like GPQA or LiveBench. Frontier labs (OpenAI, Anthropic) have already pivoted toward private, held-out evaluation sets or dynamic benchmarks that update frequently to prevent contamination, making a static 'decontaminated' version of an old benchmark less relevant. Its defensibility is low because the methodology for decontamination is reproducible and its value is entirely dependent on widespread adoption, which has not materialized. Competitors include the original MMLU, Big-Bench, and automated evaluation frameworks like LMSYS Chatbot Arena which bypass static contamination issues entirely.