How-toSelected

Together AI Shares Cost-Ordered Playbook for Fixing Model Evals

Why we picked it

The ordering matters because most teams jump straight to fine-tuning when a model fails an eval, spending compute on problems a system prompt or sampling parameter change would solve. Teams can treat the four levers as a debug checklist before budgeting a training run. Anyone running open-weight models in production, especially smaller teams without dedicated ML infrastructure, gets the most value from starting at the cheap end.

Together AI posted guidance on X outlining the order in which teams should try to fix an open model that underperforms on their evaluation. The recommended sequence starts with the cheapest interventions: rewrite the system prompt, then tune inference settings, then improve the context and harness around the model.

Body

Only after those three levers have been tested does Together recommend changing the model weights through fine-tuning. The core principle is that fine-tuning should come after optimising the system around the model, since most eval misses can be resolved with prompt or inference-level changes at a fraction of the cost and effort of weight updates.

For teams deploying open-weight models, this ordering offers a practical checklist: exhaust prompt and harness improvements before committing to training runs, which require datasets, compute budget and revalidation. The advice applies broadly to developers benchmarking open models from labs such as DeepSeek, Qwen or Llama against their own workloads.

Original · Together AI

when an open model misses your eval, start with the cheapest levers

rewrite the system prompt → tune inference settings → improve context + harness → change the model weights

fine-tuning comes after you’ve tested the system around the model https://t.co/5kBpklR0TU

Industry Coordinates & Source Verification

This report is categorized under AITOP's "How-to" desk, focusing on US frontier AI labs and Silicon Valley technology leaders. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include open-source models, fine-tuning, prompt engineering, Together.

Source attribution: curated from Together AI (@togethercompute).