Strong-to-Weak Prompt Optimization

Optimize prompts for a cheaper inference model with feedback from a stronger model so you pay a fixed optimization cost and keep inference cheap.

When prompt optimization tooling can use a separate “gradient” or critique model, run the optimizer/feedback engine on a stronger model and the forward reasoner on the weaker/cheaper model you will deploy. The TextGrad reasoning experiments optimized a gpt-3.5-turbo system prompt with gpt-4o feedback over small train/val/test splits (Object Counting, Word Sorting, GSM8k), keeping the prompt only when validation improved after each minibatch step. Concrete splits used there: BBH Word Sorting / Object Counting at 50/100/100, GSM8k (DSPy) at 200/300/1319. Score Object Counting and GSM8k with gold-standard matching on the last numerical value; use a structured LLM judge (binary 1/0 inside tagged fields) for Word Sorting where exact string match is brittle.

This is a harness cost trade: spend tokens once during offline search, then serve the cheaper model with an improved instruction (often still zero-shot — see instruction vs demonstration optimization). DSPy teacher-program distillation makes the same trade a different way — a stronger compiled program's execution traces become bootstrapped demonstrations or finetuning data for the cheap program, instead of a stronger model's textual criticism rewriting a prompt. Gate acceptance with offline prompt evaluation or per-task offline harness tests so optimizer noise does not ship. Pair with harness drift awareness: the optimized prompt is a versioned artifact tied to a specific inference model.

---

Sources

• TextGrad (Yuksekgonul et al.), ch. 3 §3.3; Appendix E