Transformers are Bayesian Networks

This project is a high-concept theoretical paper bridging Deep Learning (Transformers) and Probabilistic Graphical Models (PGMs). Its core value lies in providing a 'white-box' explanation for Transformer behavior via Loopy Belief Propagation (BP). From a competitive standpoint, the defensibility is extremely low (2/10) because it is a theoretical discovery rather than a proprietary software artifact; once the paper is public, the 'moat' evaporates as the community absorbs the knowledge. The quantitative signals (0 stars, 1 fork) indicate this is either brand new or has not yet gained traction in the broader AI engineering community. Frontier risk is low because labs like OpenAI and Anthropic are more focused on empirical scaling and safety alignment than formal Bayesian proofs of existing architectures, though they may use these insights for mechanistic interpretability. The primary risk to this work is displacement by a more general theory (e.g., proving the same for Softmax-based Transformers, which are more common than the Sigmoid-based ones studied here). This is a 'foundational insight' project that could eventually inform new architectures ('Bayesian Transformers') that offer better uncertainty quantification or sample efficiency, but as it stands, it is an academic contribution with no immediate commercial moat.