General Compute pairs GPUs with Positron chips for split inference
Specialising silicon per inference phase sidesteps the compromise a single chip type must make, and avoiding HBM removes exposure to the tightest supply constraint in AI hardware. Teams running high-volume generation workloads can compare this two-chip endpoint against all-GPU serving on cost and tokens-per-second. It also signals a broader shift toward heterogeneous inference stacks that infrastructure buyers should track.
General Compute is running prefill on GPUs and handing token-by-token decode to Positron AI's memory-first hardware, sold to customers as a single endpoint. The split reflects the workload physics: prefill is a large parallel compute job, while decode is dominated by memory bandwidth when pulling weights and cache for each new token. Positron's chips target the decode phase without using HBM, the memory type GPUs compete for. Customers get one contract and one set of SLAs, with chip assignment handled by General Compute.
Splitting prefill and decode across different silicon is one of the most sensible ideas in inference right now.
@general_compute is running prompt reads on GPUs and handing token-by-token generation to @positron_ai’s memory-first systems, then selling both as one endpoint.
i.e. run prefill on GPUs and decode on Positron hardware, giving customers one endpoint for a request that crosses 2 chip types.
These chips are built to write AI answers faster and cheaper by focusing on memory speed (the real bottleneck when AI generates text) instead of raw processing power.
The split follows the physics. Prefill is a large parallel compute job. Decode is mostly the chip waiting to pull weights and cache out of memory for every new token. A single chip has to compromise on at least one of those.
Positron builds for the second job, and builds it without relying on HBM, the memory GPUs are all competing for.
From the customer’s side it stays one contract and one set of SLAs. Deciding which chip handles which phase is General Compute’s problem.
