What Is a Large Language Model (LLM)?

How an LLM generates text

When you send a prompt, your text is first split into tokens, which are whole words, pieces of words, or punctuation. The LLM then calculates the probability of every possible next token, picks one, adds it to the text, and repeats. A full answer is built one token at a time, which is why many assistants show responses appearing word by word.

Because the model usually samples from those probabilities rather than always taking the single most likely token, the same prompt can produce different answers. Some tools expose a setting, often called temperature, that controls how much variety goes into those choices.

The model is not retrieving a stored answer from a database. It generates new text based on patterns it learned during training. That is why it can write fluent, original sentences, and also why fluent text is not a guarantee of correct text.

Everything the model can consider at one time (your instructions, the conversation so far, and any documents you paste in) has to fit inside its context window. Anything outside that limit is invisible to it.

How an LLM is trained

Pretraining comes first. The developer feeds the model an enormous collection of text and adjusts a very large number of internal settings, called parameters, until it gets good at predicting the next token. This stage is where the model picks up grammar, general knowledge, and writing styles. It is also why its built-in knowledge stops at a training cutoff.

Further training comes next. A model fresh from pretraining is good at continuing text, but not reliably at following instructions. Developers train it further on examples of helpful responses, often guided by human feedback on which answers are better, so it follows instructions, holds a conversation, and declines certain requests.

Training a model from scratch is expensive and is done by its developer. Some providers also let businesses customize an existing model with additional training on their own examples (called fine-tuning), and some vendors sell models adapted to one field, such as law or finance, as domain-specific LLMs. Most everyday users never retrain anything.

Once trained, the model’s parameters stay fixed while it is in use, a stage called inference, so it does not learn from your conversation.

Strengths and limits to know

Because developers can build LLMs into their own software through an API, the same model may sit behind a chat assistant, an email tool’s writing feature, and a help desk’s suggested replies. In each case, the model brings the same strengths and limits.

LLMs are one kind of generative AI, focused on language, though many can also accept images or audio (often called multimodal models).

LLMs are strong at language-heavy work: drafting, summarizing, rewriting for a new audience, translating, pulling key details from messy notes, classifying requests, and answering questions about material you provide. Their main limits:

• Knowledge cutoff. For recent or company-specific facts, use an app that connects the model to web search or your own documents, for example through retrieval-augmented generation (RAG).
• Hallucinations. It can hallucinate, confidently stating wrong facts, figures, or citations. Verify anything that matters.
• Limited context. Long files may need splitting, and details buried deep in a long input are easier to miss.
• Precise math. Exact calculations can be unreliable unless the assistant uses a calculator or code tool.
• Bias and inconsistency. Output can reflect gaps in the training data, and wording shifts between runs, so give a template when you need a consistent format.

Match the task to these strengths: rely on an LLM for language work, and pair it with search, your documents, or a calculator when facts and figures must be exact.

Further reading

• Claude 101 WorkshopThis 90-minute, beginner-friendly, hands-on workshop for groups covers Claude, an assistant built on LLMs, including custom instructions, memory, Projects, and styles.