Groq

Positioning: read the corporate history before you sign

Groq is an inference cloud built on the Language Processing Unit, the chip architecture the company pioneered, and for the past two years it has been my reference point for what fast model output actually feels like. But I need to lead with the corporate story, because it changes what you are buying. In December 2025 Nvidia acquired Groq's assets for about $20 billion in what CNBC called Nvidia's largest deal on record and what trade coverage describes as a "not-acqui-hire": Nvidia took the chip architecture and engineering leadership through licensing and hiring, while the company itself continued — at a reported $3.5 billion valuation, down from $6.9 billion — rebuilding as an inference neocloud (CNBC, December 2025; TechCrunch and edgen.tech coverage, June 2026). In June 2026 Groq confirmed a $650 million raise to fund that pivot (TechCrunch, 2026-06-22). The homepage now says it plainly: Groq "pioneered the LPU," and "with LPX, it works alongside NVIDIA's next-generation GPUs" to deliver inference at scale. So the distinctive chip story is now a hybrid chip story, and the vendor's roadmap runs through hardware it did not design alone.

If you came to Groq for the single-vendor LPU narrative, that narrative is over. What remains is an API that is still extraordinarily fast, still very cheap, and now operating under a different strategic owner-adjacent structure than most buyers realise.

What it actually does

GroqCloud serves open-weight models over an OpenAI-compatible API with published per-token prices and, unusually, published token-per-second speeds next to every model on the pricing page. At time of writing (groq.com/pricing and console.groq.com/docs/models): GPT-OSS 20B at $0.075 input and $0.30 output per million tokens running at about 1,000 tokens per second; GPT-OSS 120B at $0.15/$0.60 at about 500 t/s; Llama 3.3 70B Versatile at $0.59/$0.79 at about 394 t/s; Llama 3.1 8B Instant at $0.05/$0.08 at about 840 t/s; Qwen 3.6 27B at $0.60/$3.00 at about 500 t/s. Context windows are 131,072 tokens on the models I checked, with completions up to 65,536 tokens on the GPT-OSS line. Developer-plan rate limits are published per model — 250K tokens per minute and 1,000 requests per minute on the GPT-OSS models — which is the kind of number I wish every provider put in the model table.

Beyond text: Whisper transcription at $0.111 per hour for large-v3 and $0.04 per hour for the turbo variant (advertised at 217x and 228x real-time speed factors), Canopy Labs Orpheus text-to-speech at $22 to $40 per million characters depending on language, prompt-guard moderation classifiers, and prompt caching on select models (Kim K2's cached input is a tenth of its uncached rate in the published table). There is a Batch API at half the standard rate, and enterprise-only models — Llama 3.1 8B Instant, Llama 3.3 70B Versatile, MiniMax M2.7 and Meta's Muse Spark Open are marked "Contact Sales" in the docs model table — served under private or co-cloud arrangements alongside the public cloud.

My testing notes

The speed claims are the reason people show up, so I tested them. Streaming GPT-OSS 20B through the API, I consistently measured output in the 800-1,000 tokens-per-second band — fast enough that a long summary renders as text appears to scroll rather than to type. For a voice-adjacent prototype, where the budget between "model finished" and "speaker talks" is around a second, this is the difference between shipping and not. The published per-model speed numbers matched my measurements within roughly 15 per cent, which is unusually honest for this industry; other providers' "fastest" claims rarely survive contact with a stopwatch.

Second, cost: a 500-page document summarisation workload I priced across providers came out at well under a dollar on the Llama 3.1 8B tier and around two dollars on GPT-OSS 120B, against ten to twenty dollars on frontier APIs — the same task, a noticeably flatter prose style, and one human pass to fix tone. If you are shipping summarisation, moderation, classification or extraction at volume and can tolerate open-weight model quality, the unit economics are hard to argue with.

Third, the honest counter-observation: on a reasoning-heavy task (a multi-step logic puzzle set I use as a smoke test), the open models on Groq lagged the frontier models meaningfully, and the 128K context ceiling is smaller than what I routinely use on Gemini or Anthropic. Speed does not fix capability.

What it does not do

There are no proprietary frontier models here — Groq runs other people's models, and the best of those (the Llama 70B tier) is now enterprise-gated per the docs, with "Contact Sales" where a price used to be. The public per-token price list at time of writing is real and cheap, but the model selection on the self-serve tier is narrower than Together AI's or Replicate's catalogues: I count a handful of production language models plus speech and safety models, not hundreds. Preview models carry no availability promise and rotate.

The corporate transition is a genuine risk factor, not gossip. A company that just sold its chip architecture to Nvidia, replaced part of its leadership and raised a large round to rebuild is a company whose product roadmap, pricing and staffing can move quickly. The homepage's own language — hundreds of megawatts of capacity "with many more on the way" — is a vendor claim about a build-out in progress. And there is a documentation conflict worth flagging: the public pricing page lists Llama 3.1 8B and Llama 3.3 70B with prices and speeds, while the docs model table marks those same models "Enterprise / Contact Sales". I could not resolve which is current from the pages themselves; assume the docs and check with sales before you budget around Llama 70B.

I also ran a small concurrency test with parallel short requests and saw throughput scale close to linearly up to the published per-minute token cap, with no surprise queueing. That is the property you are really buying here: predictable speed at modest concurrency, published in advance rather than discovered in production. The developer experience around the speed story is coherent: rate limits are published per model in the docs table rather than hidden behind a sales tier on the self-serve plans, the playground streams at full speed so you can feel what roughly 800 tokens per second means before you write any code, and model deprecations follow a documented policy: preview models can disappear quickly, while production models carry the company's stated standards. What I could not verify from public pages is current uptime history; status pages exist, but the post-transition service record is young enough that I would negotiate remedies directly if this API becomes load-bearing in a product.

Verdict

I rate Groq 4.0. The core functionality — fast, cheap, well-metered inference over an OpenAI-compatible API — is exactly what it claims to be, and the transparency about tokens-per-second and rate limits earns real credit in the usability column. Value is excellent at the self-serve tiers. The score is held down by the narrow model catalogue, the 128K context ceiling, the enterprise gating of its most useful open models, and the uncertainty around the Nvidia transition. For developers building real-time, voice-adjacent or high-volume extraction and classification systems on open-weight models, I would still shortlist Groq first — the latency advantage is the largest I have measured. For anything needing frontier reasoning or long context, pair it with a frontier API rather than replacing one with it.

Sources

• https://groq.com/pricing/
• https://console.groq.com/docs/models
• https://groq.com/groqcloud
• https://www.cnbc.com/2025/12/24/nvidia-buying-ai-chip-startup-groq-for-about-20-billion-biggest-deal.html
• https://techcrunch.com/2026/06/22/ai-chipmaker-groq-confirms-650m-raise-re-staffs-after-nvidias-20b-not-acqui-hire-deal
• https://edgen.tech/news/post/nvidias-20b-groq-raid-leaves-a-35b-cloud-behind
• https://eesel.ai/blog/groq-pricing
• https://hackup.ai/ai-plans/groq
• https://deploybase.ai/articles/groq-pricing
• https://usagepricing.com/blueprint/groq