Jake-yutong/HEPTA

HEPTA is in its absolute infancy (0 days old, 1 star, 0 forks), representing a prototype-level academic or personal research project. While specialized benchmarks for HCI education are relatively niche, the project currently lacks the 'data gravity' or community adoption required to become a standard. Benchmarks derive their value from network effects—the more labs that cite a score, the more valuable the benchmark becomes. Technically, it likely follows standard evaluation patterns (prompting an LLM with a dataset of HCI questions and grading the response), which is a commodity pattern. Frontier labs like OpenAI or Google are unlikely to build an 'HCI-specific' educator benchmark themselves, but they are building general-purpose evaluation frameworks (like OpenAI Evals or Vertex AI Gen AI Evaluation) that make domain-specific benchmarks like this easy to ingest or replace. The primary risk is displacement by more established academic benchmarks or broader educational evaluation suites (like MMLU or GSM8K) if they expand their taxonomy. For HEPTA to succeed, it would need to release a high-quality, human-validated dataset that is difficult to replicate, which is not yet evident from its current trajectory.