LLM Application Loop

An LLM application is a transformation layer converting a user's problem into a document the model completes, then converting the completion back.

An LLM application is a transformation layer between the user's problem domain and the model's text domain — and a transformation layer with a purpose: solving problems. The model itself does only one thing, complete documents, but this affords great flexibility: chat is completing a transcript document, and tool execution is completing a specialized transcript that includes function-calling syntax.

The application runs this transformation as a loop: transform the user's problem into a document the model completes (see the feedforward pass for how), then transform the completion back into a solution in the user's domain — prose, parsed fields, function calling, or UI/TTS events. Loops vary in cardinality:

  • Single iteration — e.g. converting a bulleted list into prose; the job is done once the model responds, with no state retained between runs.
  • Multiple iterations in a row — e.g. a chat assistant, where each turn feeds into the next.
  • Iterative, stateful, and shape-shifting — e.g. a travel-planning app that moves through brainstorming, then booking, then reminders and tips as the problem itself evolves over the interaction.

Problem difficulty scales with the medium, the level of abstraction, context needs, and statefulness. Prompt engineering's crux at every point in the loop is satisfying four criteria at once: train-data familiarity (the Little Red Riding Hood principle), relevant information without oversaturation, a cue that leads to a solution rather than mere elaboration (prompt transition), and a natural stop (chat end-of-turn or completion stop sequences). On the way back out, parsing depends on completion preamble types and completion boundary markers so the harness can extract the answer without paying for surrounding fluff.

This framing is the ground floor under everything else in agent control flow and conversational agent context, both of which describe richer shapes this same fundamental transform-complete- transform loop can take. Complexity then grows along state, retrieval-augmented generation, reasoning depth, and tools — the same axes traced by progressive agent architecture and prompt engineering sophistication levels.

Sources

  • Prompt Engineering for LLMs (John Berryman), ch. 4