Unsloth Runs Open-Source Laya Decision Model Locally on 4GB RAM
The practical catch is that Laya's confidence formula differs from Jev's, so existing Jev confidence thresholds cannot be carried over — Unsloth advises writing rules on probabilities instead, such as triggering when technical probability exceeds 0.8. Teams running decision routing on constrained hardware or air-gapped environments get a drop-in local option with documented limits: 64 questions per request, 255 options per choice, and label truncation past roughly 20 described options. Named alternatives to compare against are the Jev API itself, since the migration path is designed to be nearly transparent.
Unsloth AI has released local deployment support for the open-source Laya decision model, requiring a minimum of 4GB RAM.
Three variants are available: Multilingual (678 MB, 4GB RAM, 100+ languages, 1024 token context), English (846 MB, 5GB RAM, 512 token context), and Typed Decisions (846 MB, 5GB RAM, tuned for invoices, security incidents, customer service and agent traces).
After a 10–20 second first-load, the model responds sub-second on CPU, with optional GPU acceleration that falls back to CPU automatically when VRAM is insufficient.
Laya is served through a Jev-compatible API via Unsloth Desktop or a one-click script, and migration from Jev requires only installing typesafe-sdk and pointing TYPESAFE_BASE_URL to localhost:8888.
Unsloth 让开源决策模型 Laya 在本地部署，最低仅需 4GB 内存，全面支持 CPU、Mac、Windows、Linux 及 GPU 环境，Jev API 无缝替换 @UnslothAI 文档地址 unsloth.ai/docs/models/de… 部署与性能特征 1. Multilingual（默认）- 678 MB 体积、4 GB 内存，100+ 语言，1024 token 上下文 2. English - 846 MB 体积、5 GB 内存，更大的 encoder，512 token 上下文 3. Typed Decisions - 846 MB 体积、5 GB 内存，针对发票、安全事件、客服、agent 轨迹优化 性能表现很适合生产环境：首次请求加载模型需 10–20 秒，之后 CPU 上亚秒级响应；GPU 是可选加速，且显存不足时自动回落到 CPU 继续服务。安装走 Unsloth Desktop 或一键脚本，在 Settings → API → Decision API 里开启即可。 文档中最重要的工程建议 用 probabilities 做决策阈值，不要用 confidence。 例如规则应写成“当 technical 概率 > 0.8 时触发”，而不是“confidence > 某值”。这条建议出现两次，并且直接关联 Jev 迁移的头号坑：Laya 的 confidence 计算公式与 Jev 不同，在 Jev 上调好的 confidence 阈值不能直接搬过来。 其他硬性限制：单请求最多 64 个问题、每个 choice 最多 255 个选项、每个 score 最多 10 档；choice 的各选项共享固定 token 预算，带描述的选项超过约 20 个后标签会被截断，实际使用中选项数量要控制在这个范围内。 从 Jev 迁移 只需装 typesafe-sdk 并把 TYPESAFE_BASE_URL 指向 http://localhost:8888，SDK 默认的 jev-latest 会映射到你在设置里选的模型。除上述 confidence 公式差异外，唯一另一处行为差异就是 choice 选项的 token 预算截断。迁移成本设计得很低。 Unsloth AI @UnslothAI You can now run Laya Decision models locally on just 4GB RAM! 🔥 Works on CPU, Mac, Windows, Linux and GPU setups. Serve Laya through a Jev-compatible API via Unsloth Desktop. GitHub: github.com/unslothai/unsl… P Guide: unsloth.ai/docs/models/de… T 🔗 View Quoted Tweet
This report is categorized under AITOP's "Products" desk. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows. Key entities and technical topics under continuous tracking include Unsloth, Laya, Jev, local deployment.
Source attribution: curated from shao__meng (@shao__meng).
