Collected sources and patterns will appear here. Add from search, explore, or the patterns library.
DatasetSpec -> List<EvaluationInstance>
Sample a deterministic, capped subset of evaluation instances from a dataset spec to minimize testing costs.
Problem it solves
Running whole standard benchmarks over paid external APIs during prototyping is prohibitively expensive.
Consumes
Emits
The real projects this mechanism was found in. Attribution is the point — this is how the best teams actually do it.