Transformer
A revolutionary neural network architecture that processes entire sequences of data simultaneously using "attention" mechanisms, serving as the foundation for modern LLMs and video models.
What it is
Introduced by Google in 2017, the Transformer architecture replaced older sequential models. It uses a "self-attention" mechanism to look at every word in a sentence (or patch in an image) at the same time, learning the complex relationships and context between them. It is the underlying technology powering everything from GPT-4 to Sora.
When you would use it
You reference transformers when discussing the fundamental technological breakthrough that enabled the current generation of highly capable AI models.
Common operations
- Not applicable.
Related terms
Where this is taught
No learning path uses this term yet. Browse the Learning Atlas for guided sequences through related ideas.