Evaluating Students' Open-ended Written Responses with LLMs: Using the RAG Framework for GPT-3.5, GPT-4, Claude-3, and Mistral-Large

This is an academic paper with no repository stars, forks, or active development. It presents a comparative evaluation of existing LLM models (GPT-3.5, GPT-4, Claude-3, Mistral-Large) applied to student assessment using standard RAG techniques. While the application domain (automated educational assessment) is timely and relevant, the technical contribution is minimal—it applies commodity LLMs with a well-established pattern (RAG) to a specific use case. No novel architecture, algorithm, or dataset appears to be presented. DEFENSIBILITY: Score of 2 reflects that this is a research paper demonstrating applicability, not a deployable product or defensible platform. There are no users, no adoption, no moat. Any educator or institution could reproduce this work by directly calling the same LLM APIs with similar prompting strategies. PLATFORM DOMINATION RISK (high): OpenAI, Anthropic, Google, and Meta are all actively building native educational assessment features into their platforms. GPT-4, Claude, and Google's educational initiatives already support grading workflows. These vendors could trivially incorporate this exact capability (multi-model comparison + RAG for grading) as a bundled feature within 6 months, without needing this paper's code. MARKET CONSOLIDATION RISK (medium): EdTech incumbents (Blackboard, Canvas, Turnitin, Gradescope) and emerging AI grading platforms (e.g., Turnitin's integrated LLM features) are moving into this space. They have existing customer relationships, compliance certifications, and distribution. A well-funded EdTech player could absorb or outbuild this capability. However, the market is still fragmented enough that acquisition of a proven research team could be attractive. DISPLACEMENT HORIZON (6 months): Platforms are already launching LLM-based grading (OpenAI's educational API partnerships, Gradescope's AI integration, Turnitin's new features). The specific RAG + multi-model comparison approach is not defensible—it's a straightforward application of existing LLM capabilities. A competitive product could launch in weeks, not months. IMPLEMENTATION DEPTH: Marked as 'reference_implementation' because this is published academic work. Code may accompany the paper on arXiv or a supplementary repository, but it is not a production system. It validates the concept but lacks hardening for real-world deployment (error handling, scalability, privacy, data governance, bias mitigation). NOVELTY: Incremental. The paper applies known LLM models and standard RAG techniques to a known problem domain (student assessment). There is no breakthrough in model architecture, no novel evaluation metric, and no unique dataset that would constitute a technical moat. It is a solid empirical study but not a defensible innovation.