How Do Large Language Models Work?

Introduction

Large language models (LLMs) are a type of artificial intelligence (AI) that are trained on massive amounts of text data. This training allows LLMs to learn the statistical relationships between words and phrases. Once trained, LLMs can be used to generate text, translate languages, write different kinds of creative content, and answer your questions in an informative way.

In this tutorial, we'll not only dive into how they work, but also build one up from scratch. This is the best way to illustrate the behind-the-scenes operations that lead to magical results.

How do LLMs work?

Large language models (LLMs) work by using a technique called deep learning. Deep learning is a type of machine learning that uses artificial neural networks to learn from data. Artificial neural networks are inspired by the structure and function of the human brain. They are made up of layers of interconnected nodes, and each node performs a simple mathematical operation.

To train an LLM, a large dataset of text is fed into the neural network. The network then learns to predict the next word in the sequence, given the previous words. This is done by minimizing the loss function, which is a measure of how different the predicted word is from the actual word.

The loss function is typically calculated using the cross-entropy loss function, which is defined as follows:

cross_entropy(p, q) = -sum(p_i * log(q_i))

where:

• p is the probability distribution of the predicted words
• q is the probability distribution of the actual words

The cross-entropy loss function is minimized using an optimization algorithm, such as gradient descent. Gradient descent works by iteratively updating the parameters of the neural network in order to reduce the loss function.

Once the neural network is trained, it can be used to generate text by starting with a seed phrase and then predicting the next word in the sequence until the desired length of text is reached.

Mathematical details

The mathematical details of how LLMs work are complex, but we will provide a brief overview of some of the key concepts:

One of the key concepts in LLMs is the use of word embeddings. Word embeddings are vectors that represent words in a high-dimensional space. The goal of word embeddings is to learn representations of words that capture their semantic and syntactic relationships.

Another key concept in LLMs is the use of recurrent neural networks (RNNs). RNNs are a type of neural network that are well-suited for processing sequential data, such as text. RNNs work by maintaining a state vector that captures the information from the previous inputs.

To generate text, an LLM typically uses the following steps:

1. Encode the seed phrase into a word embedding vector.
2. Pass the word embedding vector to the RNN.
3. The RNN will predict the next word in the sequence.
4. The predicted word is encoded into a word embedding vector.
5. Steps 2-4 are repeated until the desired length of text is reached.

This is a simplified overview of how LLMs work. The actual implementation of LLMs can be much more complex and involve the use of additional techniques, such as attention and transformers.

Before moving on, lock in the core loop: an LLM is trained by repeatedly guessing the next token, measuring how wrong it was, and nudging its parameters to reduce that error.

In the visual, notice how the same path is used again and again: text → tokens → probabilities → loss → parameter update. Generation later reuses the prediction part, but without the training update.

Before moving on, lock in the core loop: an LLM repeatedly turns text into numbers, predicts a probability distribution for the next token, compares that prediction to the real next token, then adjusts its weights.

In the visual, follow one training example from left to right. The important idea is not “memorize sentences” — it is “reduce prediction error across many examples.”

LLMs are a type of artificial intelligence that are trained on massive amounts of text data.

Different LLM architectures

One of the most popular LLM architectures is the transformer architecture. Transformers were first introduced in the paper "Attention is All You Need" by Vaswani et al. (2017). Transformers have a number of advantages over previous LLM architectures, including:

• Parallel processing: Transformers can process input and output sequences in parallel, which makes them much faster than previous LLM architectures.
• Self-attention: Transformers use a technique called self-attention, which allows them to learn long-range dependencies in text. This makes transformers well-suited for tasks such as machine translation and text summarization.

Other popular LLM architectures include:

• Recurrent neural networks (RNNs): RNNs are a type of neural network that are well-suited for processing sequential data, such as text. RNNs work by maintaining a state vector that captures the information from the previous inputs.
• Convolutional neural networks (CNNs): CNNs are a type of neural network that are well-suited for processing spatial data, such as images. CNNs can also be used to process text by converting the text into a sequence of vectors.

Self-attention intuition

The key idea behind transformers is self-attention. Instead of reading a sentence strictly from left to right, the model compares each token with the other tokens around it and decides which ones are most useful.

For example, in the sentence:

The dog chased the ball because it was excited.

The word it should pay more attention to dog than to ball. Self-attention gives the model a way to learn that relationship from data.

A simplified version looks like this:

1. Convert each word into a vector.
2. Compare vectors using dot products.
3. Turn the scores into probabilities with softmax.
4. Use those probabilities to create a weighted summary of the important words.

Here is a tiny NumPy example of the scoring step:

```python
import numpy as np

# Three toy word vectors: dog, ball, excited
vectors = np.array([
    [1.0, 0.2],  # dog
    [0.1, 1.0],  # ball
    [0.9, 0.3]   # excited
])

# Let the token "excited" attend to all tokens
query = vectors[2]

scores = vectors @ query
weights = np.exp(scores) / np.exp(scores).sum()

print(weights)
```

The largest weight tells us which token this query is paying the most attention to.

__________ is the name of the technique used to fine-tune a pre-trained LLM on a specific task.

Building a simple LLM in Python

To build a simple LLM in Python, we will use the following libraries:

• TensorFlow
• spaCy
• NLTK

Prerequisites

You will need to have Python 3 installed on your system. You can also install the necessary libraries using the following command:

pip install tensorflow spacy nltk

Importing the necessary libraries

First, we need to import the necessary libraries:

import tensorflow as tf
import spacy
import nltk

Loading and preprocessing the training data

Next, we need to load and preprocess the training data. We can use the nltk.corpus.gutenberg corpus to load a collection of English books. We can then use the spaCy library to preprocess the text, such as tokenizing it and removing stop words.

# Load the training data
gutenberg_corpus = nltk.corpus.gutenberg.raw(fileids=['austen-emma.txt', 'austen-persuasion.txt'])

# Preprocess the text
nlp = spacy.load('en_core_web_sm')
preprocessed_text = [nlp(doc.strip()) for doc in gutenberg_corpus.split('\n')]

Defining the LLM architecture

We will use a simple recurrent neural network (RNN) architecture for our LLM. The RNN will be trained to predict the next word in a sequence, given the previous words in the sequence.

class LLM(tf.keras.Model):
    def __init__(self, vocabulary_size, embedding_dim, hidden_dim):
        super(LLM, self).__init__()

        # Embedding layer
        self.embedding_layer = tf.keras.layers.Embedding(vocabulary_size, embedding_dim)

        # RNN layer
        self.rnn_layer = tf.keras.layers.LSTM(hidden_dim)

        # Dense layer
        self.dense_layer = tf.keras.layers.Dense(vocabulary_size)

    def call(self, inputs):
        embeddings = self.embedding_layer(inputs)
        rnn_output = self.rnn_layer(embeddings)
        predictions = self.dense_layer(rnn_output)

        return predictions

To use a transformer architecture instead of RNN in the Python example, we can use the following code:

import tensorflow as tf
from transformers import TFLongformerModel

class LLM(tf.keras.Model):
    def __init__(self, vocabulary_size, embedding_dim, hidden_dim):
        super(LLM, self).__init__()

        # Embedding layer
        self.embedding_layer = tf.keras.layers.Embedding(vocabulary_size, embedding_dim)

        # Transformer encoder
        self.transformer_encoder = TFLongformerModel.from_pretrained('allenai/longformer-base-4096')

        # Dense layer
        self.dense_layer = tf.keras.layers.Dense(vocabulary_size)

    def call(self, inputs):
        embeddings = self.embedding_layer(inputs)
        transformer_output = self.transformer_encoder(embeddings)
        predictions = self.dense_layer(transformer_output[0])

        return predictions

# Create a transformer LLM
transformer_model = LLM(len(vocabulary), 128, 256)

# Compile the model
transformer_model.compile(loss='categorical_crossentropy', optimizer='adam')

The main difference between the RNN and transformer models is that the transformer model uses a transformer encoder instead of an RNN layer. The transformer encoder is able to learn long-range dependencies in text, which can improve the performance of the model on tasks such as machine translation and text summarization.

Another difference is that the transformer model uses a different embedding layer than the RNN model. The transformer model uses a positional encoding embedding layer, which allows the model to encode the position of each word in the sequence. This can be helpful for tasks such as question answering, where the model needs to understand the context of the question in order to answer it accurately.

Overall, the transformer model is a more powerful and versatile model than the RNN model. However, it is also more computationally expensive to train.

The Python example has several moving parts, but they form one simple pipeline. Each library has a job: NLTK loads text, spaCy prepares tokens, and TensorFlow defines and trains the neural network.

Use this visual as a map while reading the code: if a line feels abstract, ask which pipeline stage it belongs to.

The Python example has several moving parts, so read it as a pipeline rather than isolated snippets.

Notice the handoff between stages: raw text becomes tokens, tokens become integer sequences, sequences pass through embedding and recurrent layers, and the model outputs a next-word prediction.

Training the LLM

To train the LLM, we will use the following steps:

1. Create a training dataset by converting the preprocessed text to sequences of words.
2. Compile the LLM model with the appropriate loss function and optimizer.
3. Train the LLM model on the training dataset.

Here's the RNN version:

# Create a training dataset
training_dataset = tf.data.Dataset.from_tensor_slices(preprocessed_text)
training_dataset = training_dataset.batch(64)

# Compile the LLM model
model = LLM(len(vocabulary), 128, 256)
model.compile(loss='categorical_crossentropy', optimizer='adam')

# Train the LLM model
model.fit(training_dataset, epochs=10)

Evaluating the LLM

To evaluate the LLM, we will use the following steps:

1. Generate text from the LLM model.
2. Compare the generated text to the original text.

# Generate text from the LLM model
generated_text = model.predict(tf.constant([vocabulary['the']], dtype=tf.int32))
generated_text = vocabulary[tf.argmax(generated_text, axis=1)[0]]

# Compare the generated text to the original text
original_text = preprocessed_text[0][0].text

print('Generated text:', generated_text)
print('Original text:', original_text)

Which is not a type of language model architecture?

Transfer learning

Transfer learning is a technique where a pre-trained model is used as a starting point for a new task. This can be useful for tasks where there is not enough data to train a model from scratch.

To use transfer learning for LLMs, you can fine-tune a pre-trained LLM on your specific task. Fine-tuning involves updating the parameters of the pre-trained LLM to improve its performance on the new task.

Transfer learning can be a very effective way to train LLMs, as it can save you a significant amount of time and resources.

Which statement best describes fine-tuning an LLM?

Fine-tuning

Fine-tuning is the process of updating the parameters of a pre-trained model to improve its performance on a specific task. It can be done for a variety of tasks, such as machine translation, text summarization, and question answering.

To fine-tune an LLM, you need to provide it with a dataset of labeled examples for the specific task. The LLM will then learn to update its parameters in order to minimize the loss function on the labeled data.

Fine-tuning can be a very effective way to improve the performance of an LLM on a specific task. However, it is important to note that fine-tuning can also lead to overfitting, which is when the LLM learns the training data too well and is unable to generalize to new data.

Fine-tuning is powerful because it starts from a model that already understands general language patterns. The risk is that the model can adapt too tightly to the small task dataset.

In the visual, follow the path from pretrained knowledge to task performance, and watch the guardrails that prevent overfitting: validation data, smaller learning rates, and early stopping.

Transfer learning and fine-tuning are easiest to separate by scale.

Pretraining teaches broad language patterns from massive unlabeled text. Fine-tuning starts from that model and makes smaller, task-specific updates using labeled examples — while watching for overfitting.

Attached is the final code to play around with:

import tensorflow as tf
import spacy
import nltk
from transformers import TFLongformerModel

# Load the training data
gutenberg_corpus = nltk.corpus.gutenberg.raw(fileids=['austen-emma.txt', 'austen-persuasion.txt'])

# Preprocess the text
nlp = spacy.load('en_core_web_sm')
preprocessed_text = [nlp(doc.strip()) for doc in gutenberg_corpus.split('\n')]

# Create a vocabulary
vocabulary = {}
for doc in preprocessed_text:
    for token in doc:
        if token.text not in vocabulary:
            vocabulary[token.text] = len(vocabulary)

# Create a training dataset
training_dataset = tf.data.Dataset.from_tensor_slices(preprocessed_text)
training_dataset = training_dataset.batch(64)

# Define the transformer LLM model
class LLM(tf.keras.Model):
    def __init__(self, vocabulary_size, embedding_dim, hidden_dim):
        super(LLM, self).__init__()

        # Embedding layer
        self.embedding_layer = tf.keras.layers.Embedding(vocabulary_size, embedding_dim)

        # Transformer encoder
        self.transformer_encoder = TFLongformerModel.from_pretrained('allenai/longformer-base-4096')

        # Dense layer
        self.dense_layer = tf.keras.layers.Dense(vocabulary_size)

    def call(self, inputs):
        embeddings = self.embedding_layer(inputs)
        transformer_output = self.transformer_encoder(embeddings)
        predictions = self.dense_layer(transformer_output[0])

        return predictions

# Create the model
model = LLM(len(vocabulary), 128, 256)

# Compile the model
model.compile(loss='categorical_crossentropy', optimizer='adam')

# Train the model
model.fit(training_dataset, epochs=10)

# Generate text from the model
generated_text = model.predict(tf.constant([vocabulary['the']], dtype=tf.int32))
generated_text = vocabulary[tf.argmax(generated_text, axis=1)[0]]

# Print the generated text
print('Generated text:', generated_text)

To run this code, you will need to have Python 3 and TensorFlow installed. You can install TensorFlow with the following command:

pip install tensorflow

Once you have TensorFlow installed, you can run the code by saving it as a Python file (e.g. llm.py) and running the following command:

python llm.py

This will train the LLM model on the Gutenberg corpus and generate some text from the model.

In this tutorial, we learned about the basics of large language models (LLMs) and how to build a simple LLM in Python. We also discussed different LLM architectures, transfer learning, and fine-tuning.

LLMs are a powerful new technology with a wide range of potential applications. However, it is important to be aware of the limitations of LLMs and to use them responsibly.

One Pager Cheat Sheet

• This tutorial focuses on Large Language Models (LLMs), a type of AI that's trained on vast text data to learn the statistical relationships between words and phrases, enabling them to generate text, translate languages, and answer questions informatively.
• Large language models (LLMs) operate using deep learning, specifically utilizing artificial neural networks to learn from data with a core task of predicting subsequent words in a text sequence, employing word embeddings and recurrent neural networks (RNNs) to account for semantic and syntactical relationships among words and process sequential data respectively.
• - LLM training is a repeated next-token prediction loop: predict, compute loss, update weights.
• - LLM training is a repeated loop: tokens → model → next-token probabilities → loss → weight update.
• Large Language Models (LLMs) are a type of artificial intelligence that utilize machine learning and deep learning with artificial neural networks to train on massive amounts of text data. Key technical elements such as word embeddings and recurrent neural networks are used to learn from this data, enabling LLMs to generate its own text using a seed phrase.
• The transformer architecture is a popular LLM (Language model learning) architecture due to its parallel processing and self-attention capabilities, while other popular LLM architectures include Recurrent neural networks (RNNs) for sequential data processing and Convolutional neural networks (CNNs) for spatial data processing.
• Transfer learning is a machine learning strategy where a model, such as a pre-trained Language Model (LLM), is initially trained on a large benchmark dataset to recognize general patterns and then further fine-tuned using task-specific data, leveraging the model's pre-existing knowledge to improve the performance of specific tasks such as text classification or sentiment analysis.
• The tutorial provides steps to build a simple Language Model (LLM) in Python using TensorFlow, spaCy and NLTK libraries, covering aspects like library importation, loading and preprocessing the training data using the Gutenberg corpus, and defining the LLM architecture using either Recurrent Neural Network (RNN) or transformer architecture, with the latter being more powerful but computationally expensive.
• - The simple Python LLM pipeline is: load corpus → preprocess tokens → build model → train next-token predictor → generate text.
• - A simple language model pipeline is: corpus → preprocessing → vocabulary/sequences → model layers → next-token output.
• To train the LLM, one needs to create a training dataset from preprocessed text in sequence of words, compile the LLM model with suitable loss function and optimizer, and then train it using the dataset, while evaluating the LLM involves generating text from the LLM model and comparing it with the original text; these steps are implemented with tf.data.Dataset.from_tensor_slices, model.compile, and model.fit for training, and model.predict for evaluation.
• The concept of modern versatile synapses is not a recognized language model architecture in machine learning or deep learning, unlike familiar architectures like Recursive Neural Network (RNN), Long Short-Term Memory (LSTM), Gated Recurrent Unit (GRU), Transformer Models (BERT, GPT-2/3, T5), and Convolutional Neural Networks (CNN).
• Transfer learning is a technique where a pre-trained model is fine-tuned for a new task, especially useful in training LLMs to save time and resources.
• Fine-tuning is the process of updating the parameters of a pre-trained model to enhance its performance on a specific task, but it can potentially lead to overfitting where the model cannot generalize to new data.
• - Fine-tuning adapts a pretrained LLM to a task, but validation checks are needed to avoid overfitting.
• - Pretraining builds general language ability; fine-tuning adapts that ability to a specific labeled task.
• The document contains the final code to build and train a Large Language Model (LLM) using Python, TensorFlow, and other libraries on the Gutenberg corpus, and also discusses basic LLM architectures, transfer learning, and fine-tuning.