Sequence, Language, and Multimodal Learning

Part III

Sequences introduce memory; attention makes relationships directly accessible; multimodal learning asks different representations to share meaning.

Part II focused on spatial structure. This Part turns to ordered data and context. Chapter 7 develops recurrent networks, LSTMs, GRUs, and the information bottlenecks that motivated attention. Chapter 8 examines the Transformer and the query–key–value mechanism that replaced recurrence with direct token relationships. Chapter 9 connects Transformer architecture to pre-training, transfer, and modern language-model families. Chapter 10 brings vision and language together through contrastive learning, shared embedding spaces, and multimodal fusion.

Across these chapters, the central question evolves from how a system remembers a sequence to how it represents relationships within and across modalities. The emphasis remains architectural: each design choice is connected to the limitation it was intended to solve.