Context Window
AI Fundamentals
A context window is the maximum amount of content, measured in tokens, that an AI model can consider at once, including its own response.
AI Fundamentals
What Are Tokens in AI?
Tokens are the small chunks of text that a language model reads and writes: whole words, parts of words, or punctuation marks. Before a model processes your prompt, a tokenizer splits the text into tokens and turns each one into a number. Tokens are the unit used to measure input and output, so they set how much text fits in a request, how long a response can be, and, for most pay-as-you-go language model services, what usage costs.
Also called token, AI tokens, LLM tokens
Example
A localization agency is quoting a job to translate a 40,000-word English software manual into Japanese and Spanish. It plans to use an AI model through an API (a direct software connection to the model) for first drafts, with human linguists editing every page.
Rather than pricing from word count alone, the project manager drafts a sample chapter in each language and checks it with the provider’s token-counting tool. The counts differ noticeably between Japanese and Spanish, so she estimates each language separately. She also counts the style guide and terminology list sent with every request, since those input tokens are billed each time. She splits the manual into sections small enough that each translated section stays within the model’s output limit. Her quote lists estimated tokens per language and prices human review separately.
How tokenization works
Before a language model can read your prompt, a component called a tokenizer breaks the text into tokens drawn from a fixed vocabulary. Common words are often a single token. Longer or rarer words get split into pieces, so a word like “tokenization” might become “token” and “ization.” Punctuation marks, digits, and spaces count too, either as their own tokens or as part of one.
From then on, the model works with numeric token IDs rather than the letters you typed. Most language models generate their response one token at a time, predicting each next token from everything that came before it.
A widely published rule of thumb for English is that one token is about four characters, or roughly three-quarters of a word, so 100 tokens is around 75 words. Treat that as an estimate only. Different models use different tokenizers, so the same text can produce different counts depending on the model.
Why tokens matter for your business
Tokens are the unit behind three practical things you will run into when you use AI at work: capacity, cost, and response length.
- How much fits. Every model has a context window, a maximum number of tokens it can consider at once. Your instructions, attached documents, earlier messages, and the model’s reply all count toward it.
- What it costs. When you use a model through an API, providers typically bill by the number of tokens sent and received, with input and output priced separately. How those rates work, and how to estimate a bill, is covered under cost per token.
- How long and how fast. Many tools let you cap the number of output tokens, which limits response length. Because most models generate output one token at a time, longer responses generally take longer to finish, and very long inputs can also add delay before the response starts.
Even on flat-rate chat subscriptions, tokens sit underneath. Long conversations and large files use more of a model’s capacity and can run into usage limits sooner.
What makes token counts go up
Token counts often surprise people because they don’t track neatly with word counts. Common reasons they go up:
- Languages other than English. Many tokenizers are most efficient with English, so other languages, especially those written in non-Latin scripts, can need more tokens to say the same thing.
- Code and data files. Symbols, indentation, and unusual names can break into many small tokens.
- Conversation history. In many chat setups, earlier messages are sent back to the model with each new turn, so input tokens grow as a conversation gets longer.
- Instructions you don’t see. Background instructions set by the app or developer (often called a system prompt), descriptions of tools the model can use, and documents pulled in to answer your question all count, even when they stay behind the scenes.
- Images and audio. Multimodal models convert images and audio into tokens too, and larger images or longer recordings typically use more of them.
- Reasoning. Some models work through a problem before answering, and those extra reasoning tokens are often counted as output even if you don’t see all of them.
What tokens are not
Tokens and embeddings are related but not the same thing. A token is a unit of input, such as a piece of text, and its ID is simply a position in the model’s vocabulary with no meaning of its own. Inside the model, each token ID is converted into a list of numbers learned during training, sometimes called a token embedding. The embeddings used for search and retrieval are related but usually represent a whole sentence, passage, or document rather than a single token.
The word “token” also shows up elsewhere in tech with unrelated meanings. An API token or access token is a credential that lets software sign in to a service, and crypto tokens are digital assets. When people talk about tokens in the context of AI models, they mean the units a model reads and writes, as described on this page.
Further reading
- AI ImplementationHelpful if you plan to build AI into your processes and systems, where token use becomes a practical design and budget question.
- AI Training & SpeakingUseful if you want your team to build a shared, practical understanding of how AI tools work.
Beyond the definitions
Put AI to work for your business.
Go beyond the terminology. Explore practical AI training, workflows, and tools from SN3 Group.