What changes when frontier models work with enormous amounts of information
This part examines what changes inside the frontier capability itself: very large working context, variable reasoning effort, multimodal problem solving, and AI participation in research.
A customer sends a large project: requirements, meeting notes, architecture diagrams, support tickets, source code, spreadsheets and previous decisions. The AI reads all of it. Later, someone asks about one small decision buried deep in that material, and the answer sounds confident but is wrong.
If the model can hold so much information, why can this still happen?
A Short Reminder: What Context Means
Context is the information available to the model while it is working on the current request. It can contain instructions, conversation history, documents, code, tool results and other task information. The context window is the maximum working space available for that information during a single model inference.
A large context window means the model can work with a large amount of information at once. It does not mean the model permanently remembers every fact or will use every detail correctly. Keeping those two ideas apart is the whole subject of this chapter.
Why One Million Tokens Changes the Problem
OpenAI currently documents GPT-6 Astra with a 1,050,000-token context window and a maximum output of 128,000 tokens. That is large enough to change how applications can be designed. Instead of breaking every long document or codebase into small pieces immediately, a system can sometimes give the model a much larger working set.
But a larger working set creates a new engineering problem. The system is no longer only trying to fit the information; it must make sure the model can find and use the important part at the right time.
Capacity Is Not Attention
Imagine a model receives hundreds of documents, and the answer to the user's question is present somewhere inside them. The information being present does not guarantee that the model will treat it as important. Relevant evidence may be surrounded by large amounts of unrelated material, earlier instructions may compete with newer information, and similar passages may point in different directions.
The engineering problem changes from "Can the information fit?" to "Can the system keep the right information useful?"
Why Retrieval Still Matters
A very large context window does not remove the need for retrieval. Retrieval means keeping information outside the active model request and bringing in the pieces that are relevant to the current problem.
If a user asks why an authentication design was rejected, the application can search decision records, meeting notes and architecture documents, then bring the strongest evidence into the active context. That is often more reliable than asking the model to search mentally through every document that happens to fit inside a huge window.
Long Tasks Need More Than Context
The problem becomes harder when the model is not answering one question but working for hours. A long-running agent may read files, call tools, edit code, run tests, receive new instructions, wait for an external system, recover from an error and continue. If every event is simply appended forever, the working context becomes increasingly expensive and noisy.
This is where state becomes important. State is the durable record of where the task stands: what has been completed, what changed, what is waiting, what failed, and what the next step needs to know. Context is what the model sees now; state is what the system keeps.
Compaction: Keeping the Meaning Without Keeping Everything
OpenAI documents compaction support for Astra. Compaction reduces older working material into a smaller representation so a long task can continue without carrying every earlier detail in full.
That sounds simple, but it creates a trade-off. Compression saves space by removing detail, and a fact that appeared unimportant at step 20 may become important at step 80. A robust system therefore needs a way to return to original sources when a compacted summary is not enough.
Prompt Caching and Persisted Reasoning
Astra also supports prompt caching and persisted reasoning, which solve different parts of the long-task problem. Prompt caching avoids repeatedly processing an unchanged prefix of a large request. Persisted reasoning allows useful reasoning state to continue across related work.
Neither should be treated as magical memory. They are engineering mechanisms that reduce repeated work and help continuity.
Where Claude Fable 5.1 Enters the Picture
Anthropic describes Claude Fable 5.1 as its most advanced model for coding and knowledge work, designed for demanding and long-running tasks. For this book, the interesting question is not whether Fable has a bigger number in one specification. It is how the Claude system keeps a difficult task coherent while it spans many steps, tools and possibly several applications.
Anthropic does not publicly disclose every internal mechanism behind that continuity. Where the design is not documented, this book says so rather than filling the gap with assumptions.
How a Long-Context Agent Actually Works
Put the pieces together and a modern long-context agent is a loop rather than a single act of reading:
1. Load the current task state.
2. Retrieve the information needed for the next decision.
3. Assemble the active context.
4. Run model inference.
5. Produce an answer or choose a tool action.
6. Execute the action outside the model.
7. Validate what happened.
8. Update durable state.
9. Build the next context and continue.
Reliable long-running work does not depend on the model remembering the whole project by itself. The surrounding system continually rebuilds the information the model needs. This loop is a system-level conceptual pattern, not a description of any vendor's hidden implementation.
Where It Can Still Fail
Relevant information is present but underused. The model sees the evidence but gives more weight to other material.
Retrieval brings back the wrong evidence. The correct source exists, but the surrounding system selects weaker or unrelated information.
Compaction removes a detail too early. A summary loses something that later becomes important.
Task state becomes stale. The stored record says one thing while the external system has already changed.
The model simply reasons incorrectly. Better context reduces one class of failure; it does not remove ordinary model error.
The move to very large context windows is important, but the deeper change is architectural. Frontier AI systems are becoming combinations of model inference, retrieval, caching, state, compaction, tools and orchestration. A million-token model can read much more, but a well-designed system must still decide what matters, what should survive, what can be compressed, and what must be fetched again.
Large context is capacity. Reliable continuity is a system-design problem.
Three Things to Remember, One Thing to Do
1. Large context is not the same as permanent or perfect memory.
2. Long-running AI needs state, retrieval and compaction in addition to context.
3. When an AI appears to forget, the failure may be in the model or in the system around it.
One thing to do. Take one long-document task you already give to AI. Before pasting everything in, write down which facts must be in the active context, which should be retrieved only when needed, and which belong in a durable record outside the conversation. Then run it both ways and compare the answers.
How much do you need? Architect: Use. Everyone else: Know.