# NVIDIA Groq 3 LPX enters full production with 3,431-token/s long-context inference NVIDIA’s Groq 3 LPX rack is now in full production and has posted a 3,431 output-token/s third-party measurement on Gemma 4 31B with 100K input context, giving agent builders a concrete signal that low-latency decode can remain fast deep into long-context sessions. Groq 3 LPX is moving from architecture announcement to manufactured infrastructure. Artificial Analysis measured about 3,400 output tokens/s at both 10K and 100K context on an NVIDIA-hosted private endpoint, but the single-concurrency benchmark does not yet establish public-cloud price, multi-tenant throughput or end-to-end agent speed. - Status: Active - Published: 2026-08-27T08:56:33+12:00 - Updated: 2026-08-27T08:56:33+12:00 - Categories: Artificial Intelligence, Cloud & Infrastructure, Compute & AI Infrastructure, Inference & APIs - Tags: AI accelerators, Groq 3 LPX, inference infrastructure, low-latency inference, NVIDIA - Canonical HTML: https://beyondthe.news/dossiers/nvidia-groq-3-lpx-full-production-long-context-inference ## What changed On August 24, 2026, NVIDIA announced that Groq 3 LPX, the inference rack integrated with the Vera Rubin platform, is in full production. Artificial Analysis was given access to an NVIDIA-hosted private deployment serving Gemma 4 31B and measured median output speed of 3,431 tokens per second with 100K input context and about 3,382 tokens per second at 10K context across 50 sequential single-concurrency requests. NVIDIA also reports a 4,767-token/s median on the open SPEED-Bench coding set. Nebius is the first announced AI cloud adopter, while Groq says it will also deploy LPX with Vera Rubin NVL72 in its inference cloud. ## Why it matters Agentic workloads repeatedly decode after accumulating large histories, codebases and tool outputs. A serving system that maintains very high per-user generation speed at 100K context could materially shorten long multi-step tasks even if model intelligence is unchanged. The result also shows NVIDIA incorporating Groq’s SRAM-heavy LPU design directly into the Vera Rubin serving architecture rather than relying on GPUs alone. Builders should not translate 3,431 tokens per second directly into expected API performance, however: the benchmark used a private demonstration endpoint, one request at a time, and does not include tool latency, shared-cloud contention, pricing or full workflow completion time. ## The hardware has crossed into production NVIDIA says Groq 3 LPX is now in full production as the interactive-inference extension to Vera Rubin. One rack combines 256 LPU accelerators with 128 GB of SRAM, 12 TB of DDR5 memory, roughly 40 PB/s of SRAM bandwidth and high-speed links to Vera Rubin NVL72. The architectural aim is to use Rubin GPUs for memory-rich and broad compute workloads while LPX provides deterministic low-latency token generation. ## The long-context result is the strongest practical signal Artificial Analysis ran its standard 1K, 10K and 100K input-length serving prompts against an NVIDIA-hosted Groq 3 LPX deployment of Gemma 4 31B. It measured about 3,400 output tokens per second at both 10K and 100K, indicating little decode-speed degradation across that tested context increase. NVIDIA’s technical write-up reports 3,431 tokens/s at 100K versus 870 tokens/s for the fastest public endpoint in the comparison set. ## The benchmark is useful but not a public-cloud apples-to-apples test The LPX endpoint was a private demonstration system and the headline result came from 50 sequential requests at concurrency one. Public serverless providers in the comparison operate shared production systems with different load, queuing and economics. The result therefore demonstrates impressive single-user decode capability, not a proven four-times advantage in multi-tenant cost or throughput. ## Agent workflows still contain non-inference latency At 3,431 output tokens/s, generating 5,000 tokens takes roughly 1.5 seconds once decoding begins, compared with 50 seconds at 100 tokens/s. But real agents also run tools, read and write files, wait on APIs, retrieve data and perform multiple model calls. Builders should benchmark completed tasks and sequential-loop latency rather than using raw tokens per second as the sole performance metric. ## Availability and pricing are the next builder-facing questions Nebius is the first announced AI-cloud adopter, and Groq says it will deploy LPX into its own inference cloud with Dell. NVIDIA has not yet published a simple public per-token price or a broadly accessible LPX endpoint. The architecture becomes materially more relevant to application builders when public services expose it with clear prices, concurrency behavior and service guarantees. ## Key details - NVIDIA announced Groq 3 LPX was in full production on August 24, 2026. - Artificial Analysis measured a median 3,431 output tokens/s on Gemma 4 31B with 100K input context on an NVIDIA-hosted private demonstration endpoint. - The 10K-context measurement was about 3,382 output tokens/s. - Artificial Analysis used 50 sequential requests at single concurrency for the reported test. - NVIDIA reports a 4,767-token/s median on SPEED-Bench coding tasks using the same model/system. - Each LPX rack contains 256 LPU accelerators and NVIDIA lists 128 GB aggregate SRAM plus 12 TB DDR5 memory. - Nebius is the first announced AI-cloud adopter; Groq has also announced plans to deploy LPX with Vera Rubin NVL72. ## Builder takeaways - Treat the 3,431-token/s figure as a low-latency hardware signal, not an expected public API speed until a provider publishes comparable production measurements. - For coding and research agents, benchmark task completion at long context specifically; decode performance that stays flat near 100K can matter more than short-prompt headline speed. - Measure tool and network waits separately from model generation so you know whether faster inference would actually improve your workflow. - When LPX-backed endpoints become public, compare price per completed task, concurrency, time-to-first-token and long-context throughput rather than tokens per second alone. - Infrastructure buyers should test the GPU/LPU division of work on representative models and batch sizes before assuming the architecture is universally better than GPU-only serving. ## What to watch - Public LPX-backed API availability from Nebius, Groq or other providers and the first transparent pricing. - Independent multi-concurrency and multi-tenant measurements rather than single-request private-endpoint tests. - End-to-end coding-agent and research-agent benchmarks that include tool calls and long histories. - Power consumption and throughput-per-watt under real production loads. - How NVIDIA divides prefill, attention, feed-forward and decode work between Rubin GPUs and LPX for larger models. - Whether the measured long-context speed holds for multi-hundred-thousand-token and trillion-parameter workloads. ## Uncertainties - The Artificial Analysis measurement used a private NVIDIA demonstration endpoint rather than a public production service. - Single-concurrency measurements do not establish shared-service throughput, queueing behavior or per-token economics. - NVIDIA’s broader throughput-per-megawatt and revenue-per-watt claims are projections or vendor claims and should be validated independently. - No simple public LPX API price or broad service-level commitment was available at research time. - Raw decode speed does not include prompt processing, external tools, networking or agent orchestration overhead. ## Sources - [NVIDIA Groq 3 LPX Now in Full Production With World-Class Speed for Agentic AI](https://nvidianews.nvidia.com/news/nvidia-groq-3-lpx-now-in-full-production-with-world-class-speed-for-agentic-ai) — NVIDIA · primary announcement · 2026-08-24T00:00:00+12:00. Primary announcement for production status, Vera Rubin integration, adoption and headline performance. - [How NVIDIA Groq 3 LPX Unlocks Ultrafast Interactivity at Long Context on NVIDIA Vera Rubin](https://developer.nvidia.com/blog/how-nvidia-groq-3-lpx-unlocks-ultrafast-interactivity-at-long-context-on-nvidia-vera-rubin/) — NVIDIA Technical Blog · primary technical analysis · 2026-08-24T00:00:00+12:00. Detailed architecture and benchmark conditions, including the Artificial Analysis 3,431-token/s result and SPEED-Bench measurement. - [NVIDIA Groq 3 LPX](https://www.nvidia.com/en-us/data-center/lpx/) — NVIDIA · primary product documentation. Current rack architecture, memory, bandwidth and Vera Rubin integration details. - [Hot Chips 2026: Nvidia presents Groq 3 LPX architecture and unveils its first third-party inference benchmark](https://www.tomshardware.com/tech-industry/semiconductors/nvidia-presents-groq-3-lpx-architecture-and-unveils-its-first-third-party-inference-benchmark) — Tom's Hardware · specialist independent reporting · 2026-08-27T00:00:00+12:00. Highlights the private single-concurrency benchmark conditions and why they are not directly comparable to shared public endpoints.