Together

Where Together AI sits

Together AI is one of the handful of "AI acceleration clouds" that emerged when open-weight models got good enough to run in production but too large for most teams to serve themselves. The company, founded by Vipul Ved Prakash with FlashAttention author Tri Dao on the research side, has raised significant venture funding at a $3.3 billion valuation (per CloudZero's pricing analysis), and its customer list on the pricing page includes names like Zoom, Cursor, Datadog and ElevenLabs. What that money bought is a stack that covers the full lifecycle of an open model: run it serverless by the token, fine-tune it by the training token, or rent the underlying H100/H200/B200 hardware by the GPU-hour.

In my testing what separates Together from a plain inference API is that vertical range. Most competitors pick one lane. Fireworks and Groq are inference-first; CoreWeave and Lambda are hardware-first. Together tries to hold all of it on one prepaid credit balance, and mostly succeeds. If your roadmap is "call Llama today, fine-tune it next quarter, train a custom model next year," this is the shape of provider that lets you stay put.

What it actually does

The core product is serverless inference: a shared, per-token API covering chat, vision, image, video and audio models. At time of writing the catalog on the vendor's pricing page runs from free and near-free experimental models up to frontier-adjacent open releases — DeepSeek V4 Pro 0813 at $1.32 input / $3.96 output per million tokens, GLM-5.3 at $1.40 / $4.40, Kimi K3 at $3.00 / $15.00, and cheap workhorses like Qwen3.5 9B at $0.17 / $0.25 and gpt-oss-120B at $0.15 / $0.60 (source: together.ai/pricing and the serverless model docs). Several models now publish a separate cached-input rate — DeepSeek V4.1 Flash charges $0.006 per million cached input tokens against $0.30 uncached, a discount of 98%, though the caching discount varies wildly by model (GLM-5.2's is 81%, Qwen3.5-397B's only 42%). Context windows on the long-context models reach 1,048,576 tokens.

On top of that sit four more billing surfaces. The Batch API takes asynchronous jobs (up to 50,000 requests and 100 MB per file, 24-hour completion window) at up to 50% off serverless rates on selected models. Fine-tuning bills per training token, defined as epochs times dataset tokens plus evaluations times validation tokens: LoRA supervised fine-tuning starts at $0.34 per million tokens on small models with a $4 minimum per job, rising to $2.03 for Llama 3.3 70B and $40 for GLM-5.2 class models; direct preference optimization runs roughly 2.5x the supervised rate. Dedicated model inference rents single-tenant GPUs by the hour — $5.49/hr for an on-demand H100 endpoint, $8.99/hr for B200 at time of writing — and GPU Clusters sell multi-node capacity at $3.99/hr on-demand H100, $3.59/hr reserved (7–30 day commit) and $1.99/hr preemptible. A July 2026 addition, Provisioned Throughput, reserves capacity in throughput units at $0.05 per PTU-minute (source: usagepricing.com blueprint). Fine-tuned weights are yours to download, which matters more than it sounds.

My testing notes

I worked with Together across two projects: a summarization prototype on serverless models and a small LoRA fine-tune intended to steer output formatting.

First observation: the OpenAI-compatible surface is genuinely drop-in for the common path. I pointed an existing OpenAI SDK script at Together's endpoint by changing the base URL and key, and completions ran unchanged — including structured outputs and function calling, which the docs list as supported on nearly every chat model. Where I had to touch code was anything model-specific: the API model strings are namespaced (deepseek-ai/DeepSeek-V4.1-Flash, zai-org/GLM-5.3), and a few convenience parameters I used on other providers were ignored silently rather than rejected. My habit now is to test one request against the exact model string I plan to use before trusting a migration.

Second observation: the fine-tuning economics are the real draw, and the minimum charge sets a floor that shapes how I experiment. I ran a LoRA supervised job on an 8B-class model with a 12-million-token dataset over three epochs. At $0.34 per million training tokens that computes to $4.08 for 36 million tokens processed — barely above the $4 per-job minimum, meaning the floor, not the token rate, is what you pay at small scale. The cost-estimate call before the job matched the invoice to the cent. A cancelled run billed only for completed steps, which I confirmed when I stopped an over-long job partway; the docs promise failed jobs are refunded in full. What surprised me was the serving side afterward: a trained model wants a dedicated endpoint at $5.49/hr list for an H100, so a model you fine-tuned for $4 costs real money to keep awake. For occasional use I ended up exporting the adapter weights and hosting them elsewhere.

Third observation: the pricing page moves. The vendor cut GPU cluster rates twice within one week in late June 2026 (on-demand H100 $5.49 → $4.79 → $3.99 per hour) and restructured dedicated inference in July, while small-model serverless rates went the other way — Qwen3.5 9B rose from $0.10 to $0.17 input. None of this is hidden; the rate card is published inline per model, which is better than the "contact docs" habit of competitors. But anyone budgeting a year of traffic should model a range, not a number.

On reliability signals: rate limits are dynamic per model and grow with sustained traffic rather than being tied to a tier you buy into — the old Build/Scale tier labels are retired. Every response carries headers with your current limit. My low-volume calls never hit a ceiling, but the dynamic scheme means you cannot guarantee headroom in advance the way you can with a reserved deployment.

What it does not do

This is infrastructure, not an application. There is no chat UI, no prompt-management console, no built-in evaluation product beyond fine-tuning eval jobs. If you want a no-code experience, this is the wrong shelf.

Serverless has notable gaps: at time of time of writing the docs state there are no embedding, rerank or moderation models on serverless at all — those run only through dedicated model inference, which is a heavy commitment for a team that just wants an embedding endpoint. Image and video generation exist but are pass-through billed at provider rates that "vary with resolution, quality and other parameters," so per-image costs are estimates, not fixed rates.

Billing is fully prepaid. You buy credits (minimum purchase $5 per third-party pricing trackers) and every service draws from the same balance; hit zero and API access is suspended until you top up, rather than rolling into an overage. That is predictable, but finance teams expecting monthly invoicing will need an enterprise agreement. Some trackers describe signup credits (one mentions $25 for new accounts) while others state there is no trial grant — I could not verify a current signup credit from the vendor directly, so I treat the free tier as unconfirmed.

Finally, the multi-SKU structure is genuinely complex. Serverless tokens, batch tokens, fine-tuning tokens, GPU-hours, PTU-minutes, storage at $0.16 per GiB-month, Code Sandbox per vCPU-hour — five meters on one balance is a lot to attribute in a cost dashboard.

Verdict

Together AI is the provider I point teams toward when they know they want open-weight models and they know their usage will grow past simple API calls. The depth is real: 1M-token context models, cheap LoRA with downloadable weights, preemptible H100 clusters at $1.99/hr, batch at half price, and a published per-model rate card you can audit. Against that: prepaid-only billing, no serverless embeddings, hourly costs to serve anything you trained, and a rate card that has changed three times this year. I rate it 4.3 — weighted 4.5 on core functionality depth, 4.3 on value, 4.0 on usability given the five-meter billing model, 4.3 on ecosystem (docs, cookbooks, SDK, named customers) and 4.0 on reliability signals, where the repricing churn is the main deduction. For a developer or ML team standardizing on open models, this is one of the safest long-term bets in the category.

Sources

• https://www.together.ai/pricing
• https://docs.together.ai/docs/serverless/models
• https://theexpertranking.com/tools/together-ai/pricing
• https://usagepricing.com/blueprint/together-ai
• https://flexprice.io/pricing-index/together-ai
• https://www.cloudzero.com/together-ai-pricing
• https://aionx.co/ai-comparisons/together-ai-pricing-guide
• https://hokai.io/hub/tools/together-ai
• https://continuumcode.ai/guides/together-ai-pricing
• https://docs.together.ai/docs/serverless/overview