Nvidia said on Monday, August twenty fourth, 2026 that Groq 3 LPX has entered full production, the first shipping product to come out of its twenty billion dollar purchase of Groq's assets last December. Announced at Hot Chips 2026, LPX is a dedicated decode accelerator rather than a general purpose GPU, co designed with the Vera Rubin NVL72 rack and sold alongside it rather than instead of it. Nvidia quotes 3,400 output tokens per second on a hundred thousand token context, four times the nearest alternative platform, and up to thirty five times the inference throughput per megawatt of Blackwell NVL72 on trillion parameter models.
The short answer
Nvidia announced on August 24, 2026 at Hot Chips that Groq 3 LPX has entered full production. It is a dedicated decode accelerator co designed with the Vera Rubin NVL72 rack, and the first shipping product from Nvidia's roughly twenty billion dollar acquisition of Groq's assets in December 2025. Nvidia quotes 3,400 output tokens per second on a hundred thousand token context, four times the nearest alternative platform, and up to 35 times the inference throughput per megawatt of Blackwell NVL72 on trillion parameter models. Nebius is the first named adopter.
Nvidia spent twenty billion dollars on Groq in December and the obvious question since then has been what, exactly, it bought. This week we found out: a decode tier, shipping in volume, sitting next to the GPUs rather than replacing them.
Two phases, one chip, and the mismatch in between
Serving a large language model is not one workload. It is two, and they want opposite things from hardware.
Prefill is the phase that reads your prompt. It is highly parallel and compute bound, which is exactly the shape a GPU was built for. Decode is the phase that produces the answer, one token at a time, each token depending on the last. It is serial by nature and spends most of its time moving weights and cached attention state through memory rather than doing arithmetic. On a GPU, decode leaves an enormous amount of compute idle while the memory system does all the work.
That mismatch is the entire commercial case for a separate decode part, and it is why the industry has been steadily disaggregating inference rather than buying more of the same. We have seen the same logic play out in AMD and Cerebras splitting prefill and decode across different silicon, and in Cerebras putting three wafers in a single rack.
Groq's original architecture attacked decode by keeping weights in on chip SRAM instead of streaming them from external memory, trading capacity for bandwidth and predictability. That is the property Nvidia acquired, and Groq 3 LPX is what it looks like as a product with Nvidia's rack scale engineering wrapped around it.
What Nvidia published
The headline throughput figure is 3,400 output tokens per second on a hundred thousand token context, measured on Gemma 4 31B, which Nvidia says is four times the nearest alternative platform. The long context part matters more than the raw rate. Short prompts have never been the problem; the cost and latency of serving agents that carry large working contexts is.
Physically, an LPX rack holds 256 accelerators connected by direct chip to chip links rather than through a switched fabric. It is co designed with Vera Rubin NVL72 and sold beside it. The framing throughout is heterogeneous rather than substitutive: GPUs keep the prefill and training work, LPX takes decode.
The efficiency claim is the one that will be quoted most and deserves the most care. Nvidia says up to 35 times the inference throughput per megawatt of Blackwell NVL72 on trillion parameter models. That is a specialist beating a generalist at the generalist's least favourite task, which is roughly what you would expect a specialist to do. It is not a claim that everything runs 35 times better.
Per megawatt is the number that matters now
There is a reason the efficiency figure led the announcement rather than the throughput one.
Power, not silicon, is the binding constraint on new AI capacity. Grid interconnection queues are the long pole in most builds, Microsoft has described a gap between the chips it owns and the power to run them, and European operators are siting facilities 175 kilometres from the hubs they would rather be in because that is where the power is. In that world, a part that delivers more tokens per watt is not an efficiency nicety. It is capacity that does not require a substation.
That also explains the timing. Nvidia has been warning of AI server price rises of around 15 percent for early 2027 on memory costs. A decode accelerator built around on chip SRAM is, among other things, a part whose cost structure depends less on the HBM market than a GPU's does.
The networking half of the announcement
LPX arrived with fabric news attached, which is consistent with everything else happening this month.
Nvidia detailed Spectrum-X Multiplane, which it says scales to 512,000 GPUs without requiring a third network tier, delivers up to 1,600 gigabits per second per GPU through ConnectX-9 SuperNICs, and offers around 1.6 times the AI networking performance of standard Ethernet. The resilience figures are the interesting ones: roughly 90 percent of bandwidth retained when a plane fails, and hardware fault recovery about 11 times faster.
The plane based structure and the emphasis on surviving a failed plane rather than preventing failure will look familiar to anyone who read Meta's MetaRoCE publication the same week. Two very different organisations arrived at the same conclusion within a day of each other: at this scale you design for the fabric being degraded, not for it being perfect.
Nvidia also announced Vera CPUs for the orchestration and tool use side of agentic work, and named adopters: Nebius first on LPX, CoreWeave on Spectrum-X Multiplane.
What to do with this
If you buy racks, the question is whether your inference mix is decode heavy enough at long context to justify a second kind of silicon in the room. Heterogeneous racks are more work to schedule and operate than homogeneous ones, and that cost is real.
If you rent inference, which is most people, the thing to watch is published price per million output tokens at long context. That is the specific number this hardware exists to move. Vendor efficiency claims settle into reality on price lists eventually, and price lists are harder to construct a favourable baseline for than benchmarks are.
Sources and further reading
- With Groq 3 LPX in Full Production, NVIDIA Extends Vera Rubin Inference for Agents, NVIDIA blog, August 24, 2026
- Nvidia's dedicated inference accelerator Groq 3 LPX enters full production, SiliconANGLE, August 24, 2026
- Inside NVIDIA Groq 3 LPX, NVIDIA Technical Blog
- NVIDIA Groq 3 LPU Accelerators for Heterogeneous AI Compute at Hot Chips 2026, ServeTheHome
Frequently asked questions
What is an LPX and why is it not just another GPU?
It is a decode accelerator, purpose built for the token by token generation phase of inference rather than for training or for the prefill phase that chews through a prompt. Those two phases have genuinely different shapes. Prefill is compute bound and parallel, so a GPU suits it. Decode produces one token at a time and spends most of its life waiting on memory, so the bottleneck is bandwidth and latency rather than raw arithmetic. Groq's original insight, which Nvidia bought, was to build for that second shape specifically. LPX is sold as a companion to Vera Rubin NVL72, not a replacement: the rack still needs GPUs, it just stops asking them to do a job they were not shaped for.
Where does the Groq name come from in an Nvidia product?
Nvidia acquired Groq's assets for roughly twenty billion dollars in December 2025. Groq was an inference chip startup built around a deterministic architecture that kept model weights in on chip SRAM rather than streaming them from external memory, which is what let it post the low latency numbers it was known for. Nvidia kept the name on the product line. Groq 3 LPX, announced at Hot Chips 2026 and now in full production, is the first shipping result of that purchase, which is a fast turnaround for a silicon acquisition and suggests the design was already well advanced when the deal closed.
How seriously should I take the 35x per megawatt claim?
Treat it as a vendor figure with a specific and favourable comparison baked in. It is throughput per megawatt, on trillion parameter models, against Blackwell NVL72, which means it is comparing a specialised decode part against a general purpose training and inference GPU doing decode work. That comparison flatters the specialist by construction. The number is still meaningful, because power is the actual constraint in most new data centre builds, and a decode tier that costs a fraction of the watts is worth real money. But it describes one workload shape on one comparison, not a thirty five fold improvement in anything you happen to run.
What does the rack actually look like?
Nvidia describes 256 accelerators per LPX rack, connected by direct chip to chip links rather than through a switched fabric, and co designed with the Vera Rubin NVL72 platform it sits beside. Alongside the LPX announcement, Nvidia detailed Spectrum-X Multiplane, which it says scales to 512,000 GPUs without needing a third network tier, offers up to 1,600 gigabits per second per GPU through ConnectX-9 SuperNICs, retains around 90 percent of bandwidth if a plane fails, and recovers from hardware faults roughly 11 times faster. Nebius is named as the first adopter of LPX and CoreWeave as a Spectrum-X Multiplane deployer.
Does this change anything if I am renting inference rather than buying racks?
Probably yes, on price and on latency, but not immediately and not uniformly. A dedicated decode tier is an argument about cost per token at long context, which is exactly where inference bills have been painful. If the per megawatt claims hold up even partially, that pressure shows up as cheaper long context serving from the providers who deploy it first. The practical move is not to change anything today, it is to keep watching published price per million tokens at long context specifically, because that is the number this hardware is designed to move and the one that will show whether it did.