LLMs are trained to predict which text is most likely, given the text they have seen so far; in other words, they are trained to extrapolate text. By training LLMs on very large and diverse datasets, they are able to extrapolate text they haven't seen before - they generalize.

Some of the first practical applications of LLMs use text extrapolation directly; for example as a super-powered autocomplete for code.

def sum_list(numbers):
    """
    Sum the numbers in the list
    """
 
   return sum(numbers)

LLMs can also be trained to follow instructions. Using their ability to extrapolate text, and by training on text datasets that include instructions and their corresponding responses, LLMs can learn to extrapolate the most likely result of following an instruction. Due to their ability to generalize from diverse training data, LLMs can learn to follow a diverse range of instructions and complete a wide variety tasks.

For example, this lets us use an instruction to convert code from one programming language to another:

Input to LLM

Output from LLM

The ability of LLMs to follow instructions, despite just extrapolating from inputs, is what makes it possible to use LLMs as a programming primitive. It allows us to use LLMs not just as text generators, but as information processors.

Notebooks

The following notebooks demonstrate how to translate code using LLMs from OpenAI, Anthropic, and Google. These links open in Google Colab, where you can run the code yourself.

Read next → AI is just another API

If AI Dev Explainer inspires you to build something cool → put it on Github, send us a link and we will send you this sticker pack.

Get stickers