What changed
On August 24, 2026, NVIDIA announced that Groq 3 LPX, the inference rack integrated with the Vera Rubin platform, is in full production. Artificial Analysis was given access to an NVIDIA-hosted private deployment serving Gemma 4 31B and measured median output speed of 3,431 tokens per second with 100K input context and about 3,382 tokens per second at 10K context across 50 sequential single-concurrency requests. NVIDIA also reports a 4,767-token/s median on the open SPEED-Bench coding set. Nebius is the first announced AI cloud adopter, while Groq says it will also deploy LPX with Vera Rubin NVL72 in its inference cloud.
Why it matters
Agentic workloads repeatedly decode after accumulating large histories, codebases and tool outputs. A serving system that maintains very high per-user generation speed at 100K context could materially shorten long multi-step tasks even if model intelligence is unchanged. The result also shows NVIDIA incorporating Groq’s SRAM-heavy LPU design directly into the Vera Rubin serving architecture rather than relying on GPUs alone. Builders should not translate 3,431 tokens per second directly into expected API performance, however: the benchmark used a private demonstration endpoint, one request at a time, and does not include tool latency, shared-cloud contention, pricing or full workflow completion time.
The hardware has crossed into production
NVIDIA says Groq 3 LPX is now in full production as the interactive-inference extension to Vera Rubin. One rack combines 256 LPU accelerators with 128 GB of SRAM, 12 TB of DDR5 memory, roughly 40 PB/s of SRAM bandwidth and high-speed links to Vera Rubin NVL72. The architectural aim is to use Rubin GPUs for memory-rich and broad compute workloads while LPX provides deterministic low-latency token generation.
The long-context result is the strongest practical signal
Artificial Analysis ran its standard 1K, 10K and 100K input-length serving prompts against an NVIDIA-hosted Groq 3 LPX deployment of Gemma 4 31B. It measured about 3,400 output tokens per second at both 10K and 100K, indicating little decode-speed degradation across that tested context increase. NVIDIA’s technical write-up reports 3,431 tokens/s at 100K versus 870 tokens/s for the fastest public endpoint in the comparison set.
The benchmark is useful but not a public-cloud apples-to-apples test
The LPX endpoint was a private demonstration system and the headline result came from 50 sequential requests at concurrency one. Public serverless providers in the comparison operate shared production systems with different load, queuing and economics. The result therefore demonstrates impressive single-user decode capability, not a proven four-times advantage in multi-tenant cost or throughput.
Agent workflows still contain non-inference latency
At 3,431 output tokens/s, generating 5,000 tokens takes roughly 1.5 seconds once decoding begins, compared with 50 seconds at 100 tokens/s. But real agents also run tools, read and write files, wait on APIs, retrieve data and perform multiple model calls. Builders should benchmark completed tasks and sequential-loop latency rather than using raw tokens per second as the sole performance metric.
Availability and pricing are the next builder-facing questions
Nebius is the first announced AI-cloud adopter, and Groq says it will deploy LPX into its own inference cloud with Dell. NVIDIA has not yet published a simple public per-token price or a broadly accessible LPX endpoint. The architecture becomes materially more relevant to application builders when public services expose it with clear prices, concurrency behavior and service guarantees.