inference-gateway/inference-gateway

The project addresses the 'LLM fragmentation' problem by providing a proxy layer. However, this is one of the most crowded niches in the current AI ecosystem. With ~100 stars and zero current velocity after over a year, it is significantly trailing behind dominant open-source competitors like LiteLLM (15k+ stars) and OneAPI. Technically, it offers standard gateway features (retries, fallbacks, load balancing) which are now considered commodity capabilities. The defensibility is low because there is no proprietary logic or network effect; the value of a gateway is almost entirely dependent on the breadth of its provider integrations and the size of its community for maintaining those integrations as APIs change. Frontier labs and major cloud providers (AWS Bedrock, Azure AI Foundry, Cloudflare AI Gateway) already offer superior, managed versions of this functionality, leaving very little room for a standalone, low-traction open-source gateway to survive.