Input limits and output limits are different

A model has more than one size limit. Your document can fit with room to spare and the reply can still stop short, because the space for reading and the space for writing are measured separately.

OpenAI's help page says so directly. The window caps what a model can work with in a request, and models also have an output limit.¹

The three limits

The window. This is the total space, covering what you send plus what the model writes. See what is a context window.

The input limit. If the input alone is bigger than the window, Anthropic's API returns an error that reads "prompt is too long".²

The output limit. This caps how much the model will write in one reply. Anthropic says a single request to its 1M-token models can generate up to 128k output tokens.² Checked on 10 November 2026.

OpenAI's page lists these under separate names, and says the output setting is called max_completion_tokens for one of its interfaces and max_output_tokens for another.¹

Why a reply gets cut off

Suppose your document fits. Several things can still end the reply early.

1. The reply reached its own cap. Models have an output limit, and the tools that call them can use an output-token setting too.¹
2. Input and reply together filled the window. Anthropic explains that on its newer models the request is accepted when input tokens plus the maximum output setting exceed the window. If generation then reaches the window limit, it stops.² A big document leaves less room for the answer.
3. Hidden thinking used up part of the budget. Some models reason before they answer. OpenAI says to allow space for reasoning tokens as well as the visible answer.¹ Anthropic says thinking tokens are part of the output setting and billed as output.²

That last point surprises people. OpenAI notes that a short visible answer can use more tokens than its displayed text suggests.¹

A worked picture

Say a model has a window and a separate output cap. You send a long report that fills most of the window. The room left for the reply is small, whatever the cap says. The reply is bound by whichever limit it hits first.

I haven't used real numbers here on purpose. Limits vary by model and change over time, so use the figures on the maker's own page for the model you use.

What you can do

• Ask for shorter pieces. Request a summary of one section at a time, then join the pieces.
• Trim the input. OpenAI's page suggests dividing large inputs into smaller parts or removing repeated context.¹
• Look for a stop reason. If a developer built your tool, ask them to check it. Anthropic's API reports a stop reason when generation reaches the window limit.²
• Check the output setting. If your business uses a tool built by someone else, ask them whether a cap has been set.

Chat apps can behave differently from the programming interfaces these pages describe. The maker's help centre for your app is the place to confirm.

If the reply stops mid-sentence, you can ask the assistant to continue. Check the result carefully, because a continued answer may not match the first half perfectly.

A bigger document isn't a reason to relax the usual rule. Customer records, passwords and card numbers stay out of the prompt unless your employer's policy permits them.

To plan a prompt's size before sending it, read how to count tokens.