Lawyer runs DeepSeek-V4.1-Flash on 12 used Nvidia V100 GPUs
12 张二手英伟达 V100:律师自建 AI 集群跑 DeepSeek,最高每秒 184 个词元
Second-hand datacentre GPUs plus aggressive expert quantisation can still beat newer consumer units like the DGX Spark on throughput per dollar, but only at the cost of 2,100W power draw and noise that rules out desk-side use. Builders weighing local inference for large MoE models should compare this V100 route against renting GPU capacity, especially where electricity is costly. The published per-context benchmark table is directly reusable for planning similar PP/TP cluster builds.
A Reddit user, a lawyer posting as u/TumbleweedNew6515, disclosed on 20 Nov 2025 a self-built AI server made of 12 second-hand Nvidia Tesla V100 SXM2 32GB cards, purchased for about US$9,450 (roughly RMB 63,239). The cluster runs DeepSeek-V4.1-Flash locally with 4-bit NVFP4 quantisation and reached a peak of 184 tokens per second on a single code-generation request. The setup uses a 3-stage pipeline parallel (PP3) × 4-way tensor parallel (TP4) layout, with each of the model's 15,360 experts compressed to roughly 15,360 parameters so the expert shards fit across the 12 cards.
Benchmark tables posted by the builder show prefill speeds of roughly 1,567 tokens/s at 1K context rising to about 4,700 tokens/s at 64K–128K, first-token latency of 0.7s cold start (0.2s with cached documents) at 1K context, and about 58s cold start at 256K context. Under multi-stream workloads the cluster hit 344–443 tokens/s across three streams, though the builder noted it is not a batch-decoding engine. Context was configured at 256K.
The builder reports the setup outperforms published 4x DGX Spark combinations by about 15%–45% on code generation in single-stream scenarios, at roughly half the cost of a 3-unit DGX Spark bundle. The trade-offs are up to about 2,100W of power draw and high noise, making it unsuitable for ordinary office environments.
12 张二手英伟达 V100:律师自建 AI 集群跑 DeepSeek,最高每秒 184 个词元
IT之家 11 月 21 日消息,律师 u/TumbleweedNew6515 昨日(11 月 20 日)在 Reddit 社区发帖,斥资 9,450 美元 (IT之家注:现汇率约合 63,239 元人民币) 使用 12 张二手英伟达 V100 显卡搭建私人 AI 服务器,本地运行 DeepSeek-V4.1-Flash,集群最高吞吐量达到每秒 184 个词元。 在硬件部分,该律师使用 12 张英伟达 NVIDIA Tesla V100 SXM2 32GB 搭建,通过 NVLink 与 PCIe 组成可协同推理的本地 AI 集群,整套硬件采购价约 9,450 美元 (现汇率约合 63,239 元人民币) 。 这套系统采用 3 个流水线阶段(PP3)× 每阶段 4 卡张量并行(TP4)的结构,系统运行的是 DeepSeek-V4.1-Flash,并采用 4-bit NVFP4 量化。发帖者称,每个专家被压缩至约 15,360 个参数,使 12 张 V100 能分别承载模型中的专家分片。 配置 3× DGX Spark(TP3、SGLang、MXFP4 专家) 4× DGX Spark(TP4;三堆叠) 12× V100(本次配置) 单请求,代码 未发布 125(SGLang)/ 138(TensorFold 4.0 bpw)/ 136(vLLM + DSpark) 153–184 单请求,普通文本 38–51 79–88 83–95 预填充,长提示词 约 2,000 3,100–3,300(TensorFold)/ 4,900–5,000(SGLang,真实文本)/ 8,500(vLLM + DSpark,32K–128K) 4,500–4,700 多请求 4 个流合计 85 32 个流时 290(TensorFold)/ 16 个流时 340–570(SGLang,从散文到结构化) 3 个流时 344–443;它不是批量解码引擎 上下文 配置为 256K 1M 配置为 524K 性能方面,该网友的测试结果如下: 上下文长度 首个 token(冷启动) 预填充速度(tok/s) 代码 法律 普通文本 三个代码请求合计 首个 token(文档已缓存) 1K 0.7 秒 1,567 184 102 86 443 0.2 秒 8K 2.5 秒 3,267 176 93 90 421 0.2 秒 32K 7.3 秒 4,478 176 92 95 404 0.3 秒 64K 13.9 秒 4,697 157 93 83 364 0.3 秒 128K 28.1 秒 4,670 157 93 86 360 0.4 秒 256K 58.0 秒 4,518 153 92 85 344 0.6 秒 这位用户称,在单请求、单流场景下,该服务器优于已公开的 4 台英伟达 DGX Spark 组合,代码生成速度快约 15% 至 45%,普通文本生成速度也更高。 整套系统成本约 9,450 美元 (现汇率约合 63,239 元人民币) ,约为三台 DGX Spark 组合价格的一半;不过,其代价是功耗最高可达约 2,100W,且噪音较大,并不适合普通办公环境。 参考 Lawyer with too many GPUs: DeepSeek-V4.1-Flash on 12x V100, all 15,360 experts in NVFP4 on the cards. 157-184 tok/s on one code stream, faster than any published 4x DGX Spark stack and double 3 Sparks, for half the price of 3 Sparks. 128K file read in 28 s.
This report is categorized under AITOP's "Models" desk, focusing on China's AI industry and domestic foundation model ecosystem. Within our cross-regional analytical framework, it directly maps to AI developer toolchains, coding agents, and engineering workflows and US-China foundation model benchmarks and commercialization comparison. Key entities and technical topics under continuous tracking include DeepSeek, NVIDIA V100, DGX Spark, local deployment.
Source attribution: curated from IT之家.