Part 2 traces the evolution of transformer architecture from vanilla transformers to modern LLMs.
This explains query, key, value attention, autoregressive generation, KV Cache reuse, GPU memory growth, and MQA, GQA, MLA reduce LLM serving costs.
Transformers
Find Out Whether Your Model Runs on AWS Trainium
A torch-neuronx Space lists Transformers models tested on AWS Trainium, showing PASS results, cores, time, memory, scripts, and steps to rerun tests.