AITOPAI热报
How-toOfficialSelected

AWS shows three ways to train Qwen3 on Trainium with torchtitan

官方一手AWS Machine Learning Blog@Josh Longenecker
Blog/Media· 4 sources
Read the original
Why we picked it

The notable point is that one config setup now covers three different training backends on Trainium, so teams can switch between them without rewriting launch scripts or breaking checkpoint compatibility. Engineers choosing between NVIDIA GPUs and AWS silicon can use this to evaluate Trainium for fine-tuning open-source models like Qwen3. The guide also shows how NKI MoE kernels fit into the same workflow.

AWS published a guide for training Qwen3 on a single trn2.48xlarge Trainium instance using torchtitan. It covers three approaches: the Hugging Face Transformers v5 modeling backend, torchtitan's native Qwen3 implementation, and NKI mixture-of-experts kernels. All three paths share the same torchrun launcher and config structure. Each path supports loading and saving published Hugging Face safetensors checkpoints.

Image · AWS Machine Learning Blog
Original · AWS Machine Learning Blog

Train Qwen3 on AWS Trainium with torchtitan three ways on a single trn2.48xlarge: the Hugging Face Transformers v5 modeling backend, torchtitan's native Qwen3, and NKI mixture-of-experts kernels. The same torchrun launcher and config structure work unchanged across all three paths, including loading and saving published Hugging Face safetensors.

#Qwen3#AWS Trainium#torchtitan#fine-tuning#Hugging Face

Keep reading

Recent stories with the same tag or section
PapersBuildersSelected13:32
Meta and Michigan State paper finds LLM-built reasoning graphs unreliable

The non-obvious point is that most reasoning-graph pipelines assume the LLM's decomposition is correct without ever verifying it, so confidence scores inherit hidden graph errors. Builders of agent workflows and RAG-style decomposition systems should add graph-level validation before trusting computed confidence. The Qwen3-32B versus Qwen3-8B result is a concrete warning that scaling a model does not guarantee better sub-question answering.

#Qwen3#Meta#reasoning models#AI safety
rohanpaul_ai@rohanpaul_aiSource
PapersBuildersSelected00:20
Tencent paper details recursive self-improvement for terminal agents

The reward-design audit is the most transferable part: dense progress rewards let failed runs harvest half the reward, so teams training RL agents should measure how much reward failure earns before trusting their curves. RL and agent-training engineers can apply the strict-success-versus-partial-credit split to their own reward pipelines. Compared with generic terminal-agent RL reports, this one pairs benchmark gains with a concrete failure-mode analysis.

#Tencent#Qwen3#Terminal-Bench#PPO
elvis@omarsar0Source
How-toBuildersSelected21:27
Reasoning From Scratch round 8 covers distillation with Qwen3-0.6B

Distilling reasoning traces from a larger teacher into a 0.6B-parameter model shows practitioners a far cheaper path than RL training runs, with MATH-500 as the benchmark to verify the gains. Builders training small models on limited compute can reuse the standalone training script and data-preparation pipeline directly. The soft vs. hard and on-policy vs. off-policy distillation framing is a useful mental model when choosing a distillation approach.

#Qwen3#DeepSeek R1#distillation#reasoning models
官方账号Sebastian Raschka@rasbtSource
ModelsBuildersMulti-sourceSelected15:31
Hojo-ASR-Multi-V1 posts 3.54 average WER across 5 languages

Replacing a dedicated speech decoder with a general-purpose 4B LLM means recognition inherits stronger word-context modelling, which helps resolve acoustically ambiguous words — an architectural shift other ASR builders may follow. Developers building multilingual voice features can run it locally today under Apache 2.0, inspecting or fine-tuning it without API costs, and benchmark it against Whisper-class models on their own audio.

#Hojo-ASR-Multi-V1#Qwen3-4B#ASR#open-source models
Event
rohanpaul_ai@rohanpaul_ai3 sourcesSource
PapersBuildersSelected12:49
Behavioral Self-Rewarding Lifts Qwen3-14B on SWE-bench

The method requires no new environment interactions, so teams can layer it onto existing reward setups without extra rollout cost. Researchers and engineers training coding agents on sparse test-based rewards get a concrete fix for zero-variance groups, with the bounded-reward design showing how to keep self-judgement from overriding ground-truth correctness. It compares against standard GRPO-style binary reward training on Qwen3-14B.

#Qwen3#SWE-bench#Microsoft Research#reinforcement learning
DAIR.AI@dair_aiSource
PapersBuildersSelected09:22
Primus Projection Cuts Distributed LLM Training Tuning to Single-Node Simulation

Teams burning GPU-hours on OOM crashes and suboptimal parallelism recipes can instead score candidate configurations in seconds on one machine, then commit only to the promising ones. The paper's validated error rates at production scale — including on DeepSeek V3 and Qwen3 — make the projections credible for real deployments. Engineers planning training runs on 256 to 1,024 GPU clusters, or on hardware not yet available, get a practical tuning workflow they can adopt directly.

#Primus Projection#distributed training#DeepSeek#Qwen3
arXiv: DeepSeek@Anshu Raina et al. (10)Source