Rethinking Evaluation in the Era of Time Series Foundation Models: (Un)known Information Leakage Challenges

The project addresses a critical but niche bottleneck in the deployment of Time Series Foundation Models (TSFMs): the 'leakage' problem where models appear to perform well due to seeing test data during pre-training. With 0 stars but 4 forks, it currently functions primarily as an academic artifact rather than a community-driven tool. Its defensibility is low because the techniques for detecting data contamination (like cross-dataset hash matching or temporal window auditing) are standard ML hygiene, though applying them specifically to the TSFM 'zero-shot' claim is a valuable contribution. Frontier labs (Google with TimesFM, Amazon with Chronos, Salesforce with MOIRAI) are the primary entities whose models need this auditing; while they may build internal versions, an independent evaluation standard is necessary for the ecosystem. The project's value lies in its methodology, but without a 'pip installable' suite or integration into major benchmarking platforms like Hugging Face or GluonTS, it remains a reference point rather than a moat-building infrastructure. The risk of displacement is high if larger benchmarking leaderboards (e.g., PapersWithCode) implement automated leakage detection.