What Is a Context Window?

How a context window works

Every time you send a message, the AI tool packages up everything the model needs to see and sends it as one request. The context window is the limit on how big that package can be. It is measured in tokens, the small chunks of text that models read and write, rather than in words or pages.

Several things share that space:

• Instructions, including the system prompt the app adds behind the scenes
• The conversation so far
• Attached files, pasted text, and images (in models that accept them)
• Descriptions of any connected tools, plus the results of searches or tool use
• The response the model is writing, including any reasoning steps in models that “think” before answering

Because it is one shared budget, a very large attachment leaves less room for everything else, including the answer. Many providers also set a separate, smaller cap on how long a single response can be. Window sizes vary widely by model and have grown a great deal over time, from around a thousand tokens in some early models to hundreds of thousands, or even millions, in some newer ones. The app you use may set its own limit too, so check its documentation.

Think of the context window as the model’s working memory for a single request: nothing carries over to the next request unless the app sends it back in.

What happens when you reach the limit

When a request would exceed the context window, apps handle it in different ways. Some show an error or refuse a file that is too large. Some quietly drop the oldest messages in a conversation. Others cut off part of a document, search a large file and pass along only the passages that look relevant, or automatically summarize earlier parts of the chat to make room. You are not always told which of these happened.

In practice, this shows up in a few ways:

• Long chats drift. Instructions you gave at the start, such as a preferred format or a rule to follow, may stop being applied, either because they were dropped to make room or because the model gives them less weight as the conversation grows.
• Big documents get partial treatment. The model can only reason about the parts it actually received.
• Cost and speed change. Many tools resend the conversation with every message, so longer chats use more tokens per reply. That can mean slower responses and, where pricing is based on tokens, higher costs.

Even within the limit, models do not always use a long context evenly, and details in the middle of a very long input can be overlooked. When information is missing, a model may still give a confident answer, which is one way an AI hallucination can happen.

Context window vs. training data vs. memory

These three are often confused, but they work at different levels.

• Training data is the large body of text a large language model learned from before it was released. It shapes the model’s general knowledge and has a cutoff date. A normal conversation does not change what the model already learned, although some providers may use conversations to train future versions, depending on your plan and settings.
• The context window is what the model can see right now, for this request. Anything you paste in, upload, or type lives here temporarily.
• Memory features are tools some AI assistants add on top of the model. They save details from earlier conversations, such as your role or preferences, or search past chats, and place relevant pieces into the context window of a new chat.

The practical takeaway: a model does not “remember” a document you shared last week unless the app stores it and brings it back into the context window. And because saved memories are placed into the window, they take up some of the same space as everything else.

How to work within the limits

A few habits can help you get more reliable results when work is long or document-heavy:

• Start a new chat when one runs long. Ask for a short summary of decisions so far, then paste it into a fresh conversation. For an unrelated task, just open a new chat.
• Split large documents. Break a long report or contract into parts, work through each one, then bring the results together.
• Keep key instructions visible. Restate important rules when they matter, rather than relying on something said many messages ago.
• Send only what is relevant. Trim boilerplate, duplicates, and unrelated attachments.
• Retrieve instead of pasting everything. Retrieval-augmented generation (RAG) searches a larger collection of documents and places only the passages it ranks as most relevant into the context window, so it is efficient but can still miss something.

A larger context window helps when a task needs all the material at once, such as comparing two long reports for contradictions or tracing a theme across a long policy manual. But bigger is not automatically better: filling a large window can cost more and can be slower, and, as noted above, the model may not weigh every detail equally.

Further reading

• Claude 101 WorkshopA beginner-friendly, hands-on session covering Claude features such as memory and Projects, useful if you want practice managing what an assistant works from.