I think language models are on of the coolest inventions of the century.

Most sci-fi doesnt even dream of AI being easy to talk to and aligned with humans best interests.

First we convert a sentence to an embedding, which is a list of numbers.

Attention mechanisms let the model focus on different parts of a sentence, or even previous sentences, when deciding what to say next.

Attention is all you need