Optimizer In-Context Examples

Show the textual optimizer worked examples of desired variable properties so rewrites match target style, format, or quality traits.

Besides gradients and constraints, TextGrad can condition textual gradient descent on in-context examples of good (or illustrative) variable values: “base on the following examples when modifying the {role_description}” with an <EXAMPLES> block. Authors claim this helps the optimizer implement desired properties of the optimized variable.

Use sparingly: examples compete for context with textual optimizer momentum and batch feedback. Prefer short exemplars that show format and reasoning habits under optimization variable role description, not full task solutions that invite copying. Validate that examples do not leak eval items under eval observation anti-leakage when used offline.

---

Sources

• TextGrad (Yuksekgonul et al.), ch. 7