Context Engineering

Deliberate structuring of instructions, examples, retrieved material, and working memory inside the context window for reliable model behaviour.

Context engineering packs the right information — system instructions, few-shot examples, tool documentation, retrieved knowledge, and intermediate agent state — into the finite context window at useful density. In Huyen's usage, the prompt is the whole model input; context is the enabling information inside it (in-context learning). Operationally this is the feedforward pass: retrieve, snippetize, score, and assemble under prompt element importance.

Place critical material with lost in the middle in mind: edges beat the middle. Prefer context grounding over hoping parametric memory is enough. Placement affects more than passive recall, too: a warning appended to the tail of a tool observation can go unacted-on for the same reason — see next-turn salience promotion for surfacing it at the point where the model actually decides what to do next instead. For LLM agents, tool descriptions, ReAct traces, and reflection compete with task content — over-large tool inventories starve the budget. Treat every token of format scaffolding as a cost that must earn its keep, because quality falls with length well before the window is full (context rot). For agents, the practical toolkit reduces to four moves: write, select, compress, isolate. When the context itself should improve from the system's own execution feedback rather than being hand-tuned, see context adaptation.

For conversational task agents, assemble conversational agent context deliberately: phase-relevant tools only, elastic conversation artifacts, and prior-history extent that matches topic continuity rather than unbounded retention. Long tool trajectories also benefit from collapsed observations and an agent episode prompt stack so system, demo, instance, and next-step layers compete predictably for tokens. When the task outlives any fixed window, prefer OS-inspired agent memory (virtual context / paging) over hoping a larger maximum length will use mid-prompt material well.

Sources

  • AI Engineering (Chip Huyen), ch. 5
  • AI Engineering (Chip Huyen), ch. 6
  • Prompt Engineering for LLMs (Berryman), ch. 8
  • SWE-agent (Yang et al.), pp. 31–45
  • MemGPT (Packer et al.), §1
  • ACE (Zhang et al.), §2.1