Home AI lab setup with Qwen 3.8 and DeepSeek v4
This detailed guide is useful for anyone looking to build a local AI infrastructure, showing how to combine different models and hardware for specific tasks.
A user describes their home lab setup for local AI experimentation, including a Framework Desktop with an eGPU running Qwen 3.8 27B for fast token processing, two DGX Spark servers running DeepSeek v4 Flash 0731 for batch processing, and a Raspberry Pi 5 for monitoring. The system uses a homegrown routing plugin to decide between local and cloud models.
current homelab setup for local AI experimentation: - hermes box hosted on a Framework Desktop Mainboard AI Max+ 395 - 5090 eGPU running Qwen 3.8 27B for fast tok/s LLM use - sometimes 150+ tok/s - 2x DGX Spark: running Deepseek v4 Flash 0731 - better but slower model - pi 5 for monitoring - Mac mini as a dev box - use Herdr and ohmypi/codex/claude depending on the use case - housed in a 10" DeskPi mini rack (mostly) Hermes is defaulted to local AI but with a homegrown routing plugin hitting a small low TTFT model (Arch-Router) to decide whether to go local or upgrade to cloud/frontier. Trying to get to 100% local over time, but right now probably more like 60-70% The Sparks are for batch processing background runs (all the cron jobs, longer dev builds, etc) do you need all of this? Absolutely not lol. I started with the mac mini and couldn't help myself but to add over time! Also I regret getting the eGPU so I wouldn't recommend that to anyone 💬 18 🔄 6 ❤️ 45 👀 5394 📊 25 ⚡
