Cross-Task Few-Shot CoT Transfer

Few-shot CoT exemplars from another domain help mainly when answer format matches; mismatched formats collapse gains — often worse than zero-shot CoT.

Few-shot chain-of-thought prompting is sensitive to how well exemplars match the task and answer format. Cross- domain exemplars that keep the same multiple-choice framing can still lift accuracy substantially over plain zero-shot, but transferring to a different answer type (e.g. CommonsenseQA → MultiArith) helps far less. In Kojima et al., both cross-task Few-shot-CoT variants still underperformed zero-shot chain of thought on MultiArith — underscoring the engineering cost of curated exemplars.

This aligns with the broader in-context learning pitfall that models often latch onto repeated format more than task semantics. When building exemplar packs, match output schema first; if you cannot maintain task-matched demos, prefer a robust reasoning trigger over mismatched CoT shots, and verify with offline prompt evaluation.

---

Sources

• Large Language Models are Zero-Shot Reasoners (Kojima et al.), §3 Table 5