Skip to content
titah
Sign in

Using Titah

Context & compaction

Why compaction is about correctness rather than tidiness, what exactly gets dropped, and every knob that changes it.

Titah compacts automatically once the context approaches the model’s window, and /compact runs the same thing on demand.

/compact                        # summarise everything but the last turns
/compact the database schema    # same, but keep that material in full detail

Why this exists at all

When history exceeds the window, providers do not reject the request — they truncate the oldest part, and the model then answers confidently about decisions it can no longer see. ollama truncates at num_ctx (4096 by default) without a single warning.

That is a correctness problem, not a tidiness one.

Declaring the window

Automatic compaction needs to know how large the model’s window is, and nothing is guessed:

"provider": {
  "ollama": { "models": { "qwen3:14b": { "contextWindow": 32768 } } }
}

Without it, automatic compaction is off for that model. titah doctor lists every model missing one, and the TUI says so once per session, quietly. That notice is information, not a failure: the turn runs normally, and /compact still works either way.

Tuning

"compaction": {
  "auto": true,        // turn the whole thing off with false
  "reserved": 8192,    // tokens held back for the next answer and the summary itself
  "tailTurns": 2,      // recent turns kept verbatim, never summarised
  "prune": true        // drop old tool output first — free, and usually enough
}

reserved is a headroom, not a hard token count. It covers two things, both absolute rather than proportional to the conversation: the next response, and the summarisation call itself. It does not cover the growth of the next step — one more tool result. That is budgeted separately and automatically, from the largest tool result seen so far in the running turn, and forgotten as soon as the turn ends.

The trigger is therefore:

contextWindow - reserved - (largest tool result this turn)

with both subtractions capped at a quarter of the window each.

Without the cap on reserved, the 8192 default would equal the whole window on a common 8k local model, push the threshold to zero, and fire compaction on every single turn regardless of how little context was in use. The cap on the growth margin exists for the mirror-image reason: a single result larger than a quarter of the budget will not fit after any compaction, so reserving room for it would move the overflow rather than prevent it.

titah doctor reports every model where the reserved cap is biting, and the value actually in effect.

What compaction does, precisely

  • Only what is sent to the model shrinks. The transcript on screen and in SQLite is untouched — scroll up and the real conversation is still there.
  • The last few turns (tailTurns) are kept verbatim, so the reply right after compacting does not lose the detail you just typed.
  • The cut always lands on a user message. Cutting between a tool call and its result would leave an orphaned result that providers reject.
  • Compacting again re-summarises the previous summary rather than stacking a new one on top, so the summary cannot grow without bound.
  • Old tool output is dropped first (prune), because it is the bulk of an agentic turn and costs nothing to discard — the model can re-read a file. Only if that is not enough is a summary written.
  • This happens mid-turn too, not just between turns. One long turn reading thirty files is the case that overflows most often, and there is no user message in the middle of it where a between-turns check could fire.
  • Mid-turn, the messages kept verbatim are bounded by size, not only by count: at most a quarter of the available budget, and at least one message whatever its size. Counting messages alone bounds nothing when one read of a 22 KB file is a message — measured, that single case sent 2.4× the window for an entire turn.
  • If pruning outside the tail and summarising are both still not enough, old tool output inside the tail is pruned too, as a last resort. Pruning never removes a message, so nothing is orphaned. That last resort is measured against the window itself, not the threshold: reserved is headroom for the answer, not a wall, and a result that still fits is delivered rather than thrown away. A 22 KB file on an 8k window reaches the model; a 30 KB one cannot, and is replaced by a marker saying so.
  • The trigger counts what has already arrived. A tool result that landed after the last measurement is part of the next request whether or not the provider has counted it yet, so it is added before comparing against the threshold. Without that, a single result larger than the growth margin slipped in unnoticed and the next request was already over the window — measured at 110% of an 8k window with a 30 KB read.
  • Sub-agent results are exempt from ordinary pruning. The prune marker tells the model to re-run the tool, which is right for read and wrong for task: recovering a sub-agent’s answer costs another full nested turn. Summarisation handles them instead — lossy, but not destructive. In the last-resort tail prune nothing is exempt, because a silently truncated request is worse still, but there the marker says what was lost and what it costs to get back.

“Was pruning enough?”

That question is answered by measuring the request that is about to be sent — the messages plus the system prompt — not by subtracting an estimate from a number the provider reported for a different request one step ago.

That arithmetic could not be right, and it was wrong in the direction that costs money: measured, at a 28 KB result on an 8k window, the request that would actually go out was 490 tokens — 6% of the window — while the summariser fired on 29 of 30 steps. One ruler now answers both this and “will it fit”.

The measurement counts the system prompt and the prompt you just typed, both of which ride the request without being in the stored history. What it cannot count is the tool schemas, and a byte-per-token ratio is an estimate: token-dense content (code, CJK, base64) can make the real count a third higher than measured. reserved is the headroom that absorbs that, and the trigger — which reads the provider’s own number — fires again the next step.

The summariser

The summary is written by smallModel when one is configured, falling back to the turn’s model. /compact makes the same choice, so a session’s automatic and manual summaries are never written by two different models.

The summariser’s own prompt is bounded by the window of the model that writes it. A transcript larger than that window is summarised in chunks — each chunk small enough to fit, then the chunk summaries summarised in turn. Without this the prompt was unbounded: measured at 78,964 tokens against a smallModel declaring 4,096 — 19.3× — and providers do not reject that, they truncate it.

When smallModel declares no window, the bound falls back to the turn model’s — the same thing titah doctor tells you it will do. Only when nothing at all is declared is chunking off, which is the same rule the rest of compaction follows.

Your focus text (/compact <focus>) is counted against that budget and clamped to a quarter of it, because it rides every chunk.

Failure handling

  • If any chunk comes back empty — a provider error, or Esc — the whole summarisation is abandoned and nothing is saved. A summary missing one chunk, stored as though complete, is exactly the failure this feature exists to prevent.
  • A failed compaction never fails the turn. That step’s compaction is simply skipped and the turn continues with whatever context it already had.
  • Compaction is cancellable. Because it runs unbidden, Esc reaches the summariser itself — a smallModel that hangs ends the turn instead of holding the session open.

Never invent

The summariser is instructed, above everything else, never to invent: identifiers are copied verbatim, and anything it cannot confirm is recorded as unresolved. A summary that drifts is worse than no summary, because it reads as an agreed record that the model has no way to check.

What compaction cannot reach

plan and memory are stored outside model_message, so compaction cannot touch either.

They differ on exactly one axis — how long the fact stays true. plan is intent for the work in progress and dies with the session; memory is a fact about the project that is still true tomorrow, in a different session. See Tools.