Joint Accuracy and Latency Profiling

external call

EvaluationBenchmark -> Tuple<AccuracyMetrics, LatencyMetrics>

Measure TTFT, TPOT, and general system throughput metrics inline while executing correctness benchmarks.

Problem it solves

Evaluating inference performance and task correctness separately requires redundant model runs and resource usage.

Consumes

EvaluationBenchmark

Emits

AccuracyMetricsLatencyMetrics

Distilled from 1 source

The real projects this mechanism was found in. Attribution is the point — this is how the best teams actually do it.