How-to7月15日 09:52

用Qwen3量化模型Ternary-bonsai-27b-mlx-2bit本地部署体验

berryxia@berryxia

Post

View the post

Why we picked it

给想低成本跑大模型的开发者参考：用Qwen3量化版，M1 Pro就能流畅做本地任务。

作者基于Qwen3模型进行2bit量化，得到Ternary-bonsai-27b-mlx-2bit模型。在M1 Pro 32G笔记本上通过LM Studio加载并开启本地服务器，直接接入业务产品。

Body

回答速度较快，多模态功能表现良好，无大问题。

Original · berryxia

这个模型也是通过Qwen3模型打底做的量化，Ternary-bonsai-27b-mlx-2bit，已经接在了我们公司自己的业务中去做本地的任务，没有太大问题，还是多模态很舒服。

我是load在LM Studio 中，然后开启local Server后，直接到我的产品中配置这个本地地址就行了。

我的这个笔记本使用的M1Pro 32G 回答速度也挺快的，感觉还不错啊！我再体验体验😄

Industry Coordinates & Source Verification

This report is categorized under AITOP's "How-to" desk, focusing on China's AI industry and domestic foundation model ecosystem. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Qwen3, Ternary-bonsai, LM Studio, local deployment.

Source attribution: curated from berryxia (@berryxia).

#Qwen3 #Ternary-bonsai #LM Studio #local deployment #quantized models

Keep reading

Recent stories with the same tag or section

Papers Builders12:00

Chinese contentCAPS：从日志 bandit 反馈学习校准决策头

训练分类头时只有“选了但对不对”这种反馈怎么办？这篇论文给出 CAPS 方法，比只学确认样本强不少，还把代码和适配器全开源了。做 RLHF 之外反馈学习或部署校准的工程师值得仔细看实验设计。

#CAPS #Qwen3 #LoRA #calibration

arXiv cs.LG@Ye YuanSource

Papers Builders11:52

Chinese contentSimRec 用强化学习训练推荐智能体的完整交互策略

推荐系统开发者值得看看这篇：浙大团队开源了 SimRec，用 GRPO 端到端训练会搜目录、会追问用户的推荐智能体。4B 的 Qwen3 打赢了更大规模的免训练方案，代码可复现，做电商搜索推荐的团队可以直接拿来参考。

#SimRec #Qwen3 #GRPO #recommendation systems

arXiv cs.AI@Yaxin Zhang et al. (9)Source

Papers Builders10:50

Chinese content研究重新审视“RL 泛化、SFT 记忆”：SFT 数据质量与规模是关键

搞训练的朋友别再默认 SFT 天生不如 RL 了。这篇论文用 Qwen 系列实验证明，只要数据混合得当、rollout 预算对齐，SFT 泛化能追平甚至反超 RL。做后训练、纠结 SFT 数据配比的人值得细看。

#SFT #RL #Qwen2.5 #Qwen3

Event

arXiv cs.LG@Yunlong Hou et al. (6)Source

Papers BuildersSelected00:31

Recursive Self-Improvement Applied to Frontier Pre-Training Run

The non-obvious takeaway is that RSI may pay off more in pre-training headroom than in post-training tricks, since a barely fine-tuned base beat fully post-trained rivals. Researchers running large-scale training pipelines and labs weighing agent-assisted data curation should study the loop decomposition and shared context store design. The model being open-weight and sized at 30B MoE makes it directly comparable for local deployment against Qwen3-30B-A3B and Nemotron-3.5-Lightning-30B-A3B.

#RSI #801-30B-A3B #Qwen3 #open-source models

DAIR.AI@dair_aiSource

Products BuildersSelected01:31

ExecuTorch WebGPU runs PyTorch models in the browser

Removing the conversion step means the standard PyTorch export path now covers browser deployment, which collapses a tooling gap for teams shipping web-based AI demos and privacy-sensitive inference. Developers building in-browser assistants, image editors or audio tools can test it today via the WebGPU Playground. It gives web apps an alternative to server-side APIs from OpenAI or Google, with no data leaving the tab.

#PyTorch #ExecuTorch #WebGPU #Llama 3.2

Official accountPyTorch@PyTorchSource

Papers BuildersMulti-source13:18

Chinese contentarXiv 论文：小模型按步动态升精度解码，Qwen3-0.6B 在 GSM8K 上验证

这篇论文尝试让小模型平时用 4-bit 算，只在不确定的步骤回退 8-bit，代价测算得很细。诚实地说，实机数据还没和全精度拉开显著差距，但门控捕获率和字节成本分析对做端侧推理量化的人很有参考价值，适合 Qwen3 用户和量化工程方向的研究者看。

#Qwen3 #quantization #GSM8K #inference optimization

Event

arXiv cs.AI@Mohammed Nayeem Khan et al. (3)Source