Why and When Visual Token Pruning Fails? A Study on Relevant Visual Information Shift in MLLMs Decoding

This project identifies a critical flaw in current Multimodal Large Language Model (MLLM) optimization: visual tokens pruned during the initial pre-fill stage may become relevant again during the autoregressive decoding stage (RVIS). While the discovery is intellectually significant, the project's defensibility is low (3/10) because it is primarily an academic contribution. The 'moat' consists of the specific algorithmic fix for RVIS, which can be easily replicated by infrastructure providers once the paper is read. With 5 forks but 0 stars in just 2 days, there is immediate 'peer' interest (likely other researchers), but no broad adoption. Frontier labs (OpenAI, Google) are the primary competitors here; they are aggressively optimizing token counts for vision-language models to reduce costs. If RVIS is a valid bottleneck, these labs will integrate dynamic pruning or 'token retrieval' mechanisms directly into their proprietary inference engines, likely making this standalone research obsolete within a 6-month horizon. It competes with existing pruning methods like FastV and LLaVA-PruMerge, positioning itself as a more 'reasoning-aware' alternative.