interleaved multimodal token sequence assembly

transform

List<TextEmbedding> + List<ProjectedSensorEmbedding> -> Tensor<Sequence>

Interleave discrete text token embeddings with projected vision and continuous-state embeddings into a unified sequence for LLM inference.

Problem it solves

Decoder-only LLMs require visual, state, and text data to be contextualized as a single sequential stream.

Consumes

List<Embedding<Text>>List<Embedding<Sensor>>

Emits

Tensor<MultimodalSequence>

Distilled from 1 source

The real projects this mechanism was found in. Attribution is the point — this is how the best teams actually do it.