Together AI sells predictability, then has to build it

A reader asking a newspaper a question is not buying a GPU. The reader wants a useful answer while the question still matters. The newspaper, meanwhile, needs a service it can afford to keep running. Together AI occupies the space between those two expectations: it sells the work of turning available models and computing capacity into a dependable product.

Its Washington Post case study gives that proposition a concrete shape. Together reports 1.79 billion monthly input tokens, thousands of daily queries and roughly two-second model responses for Ask The Post AI. The deployment combines dedicated and serverless inference, uses Llama and Mistral models, and has fixed monthly pricing. The amount of that bill is not disclosed. These are vendor-published customer results, not an independently reproduced benchmark. Customer case study.

The Post's own announcement dates the launch to November 7, 2024. The Post’s launch announcement.

The attraction is understandable: keep more control over the model without taking on the entire job of serving it. But removing one dependency creates another. Together has to earn its place through engineering, capacity and service, while charging enough to cover them. Its new compiler and its much larger infrastructure commitments show both sides of that bargain.

A financing round is not an income statement

On July 1, Together announced an $800 million Series C. The release carried by lead investor Aramco Ventures specified an $8.3 billion post-money valuation, thousands of paying customers and annual bookings that had crossed $1.15 billion in the previous quarter. It also described an ambition to increase the infrastructure footprint roughly fiftyfold over five years. Those are company and investor disclosures, with financial interests attached. Financing release.

Bookings deserve their own label. They are not interchangeable with recognized revenue, annual recurring revenue or cash collected. The release does not provide enough detail about contract duration, cancellation rights or the calculation to turn its headline into any of those measures. Nor does a valuation tell readers whether selling inference is already profitable. The business can be expanding rapidly while those questions remain open.

CEO Vipul Ved Prakash's accompanying essay added another distinction: the company said it had secured commitments for more than 500 megawatts of compute capacity, to be capitalized independently by new investors. That is a statement about future supply and financing arrangements, not a claim that 500 megawatts were already serving customers. July 1 company essay.

Separate financing may reduce the equity Together itself must devote to physical infrastructure. It does not, by itself, disclose who bears an idle machine's cost. A buyer can rent instead of own and still have a substantial obligation to pay. The missing information is the connection between customer commitments, supplier commitments and the dates on which each becomes binding. That connection matters more to the economics than the label attached to the cloud.

The capacity arrives on a different clock

IBM supplied a more specific example on August 11. The companies announced a multi-year $240 million agreement for an NVIDIA HGX B300 cluster on IBM Cloud, with availability expected in the first quarter of 2027. Together would use it for open-model inference. IBM also identified Spectrum-X networking. As of this article's November 9 observation date, this is announced future capacity, not an operational cluster verified by this reporting. IBM announcement.

The announcement does not specify the agreement's exact length, payment schedule, minimum draw or termination provisions. Dividing $240 million by an assumed number of years would manufacture an annual cost. Treating the amount as immediate expenditure, or as revenue already earned by either party, would be equally premature. A large agreement is evidence of commercial intent; its cash consequences depend on terms the release does not show.

The strategic logic is nevertheless clear. Customer-facing inference needs supply that can be counted on, and reserved supply needs customers who will use it. If those commitments line up, a specialist can combine purchasing scale with software efficiency. If customer demand changes faster than the capacity arrangement, the same structure can leave expensive resources attached to yesterday's forecast. This is a conditional business risk, not evidence that Together has excess capacity today.

There is also a timing problem distinct from demand. A customer can begin paying for an application before a planned cluster is delivered, or need more capacity while an expansion is still under construction. A credible growth story therefore needs an operating bridge between the existing fleet and the future one. An infrastructure announcement explains where additional supply is intended to come from. It does not show that bridge's cost or spare capacity.

Software has to pay its way

Together's answer is not simply to acquire more GPUs. On November 4, it announced Together Compiler as part of a broader effort to adapt inference software to changing models, hardware and traffic. The company says AI agents search for faster implementations and that the compiler already supports some production serverless endpoints. Customer access was described as a limited beta; self-service optimization of dedicated deployments remained forthcoming. November 4 announcement.

The underlying engineering idea has a more restrained precedent. A May 27, 2025 Hazy Research post examined a single-sequence Llama-3.2-1B workload on an H100. Combining operations into one larger GPU program reduced gaps between operations; the researchers reported using 78% of memory bandwidth and beating the tested systems by more than 1.5 times. That was a narrow, disclosed experiment, not a universal inference speedup. Research and test scope.

The commercial interpretation is simple: more completed work from the same hardware can create room for a lower customer price, a better provider margin, or both. But the benefit must survive the actual workload. A fast result on a small model with one user does not establish the cost of serving many users, longer prompts or a different model. Efficiency is valuable precisely because it has to be achieved in specific circumstances.

Together's current product page says candidate optimizations are checked for speed and correctness before production use. It reports up to 2.6 times the tokens per GPU at matched per-user speed in a set of Blackwell tests, following a 24-hour compilation using two nodes per model. The page also says supported serverless customers receive compiled kernels without a price change. These are the vendor's measurements and product claims. Compiler product details.

That last detail makes the business question sharper. If the customer pays the same token price and Together needs less hardware time, the provider could retain some savings. Competition could instead push the selling price down. Maintaining an advantage then requires repeated improvements, not a single published result. The research team is potentially part of the production cost structure as well as the sales story, and automation is an attempt to make that expertise serve more workloads.

Predictability has a utilization bill

Together's public menu makes the software premium visible without revealing its margin. On November 9, an H100 was listed at $5.49 per GPU-hour for dedicated inference and $3.99 for on-demand GPU clusters. These are different products, not interchangeable quotes. The $1.50 difference cannot be called profit: serving software, support, configuration and underlying supply economics all matter. Observed pricing.

For reserved token capacity, the documentation describes provisioned throughput units, or PTUs, at $0.05 per unit per minute. Input, output and cached tokens draw on capacity at different, model-specific rates. Unused units do not roll over or receive a refund. Commitments are tied to a model or model family; changing models requires discussion with sales. Provisioned throughput documentation.

At that published rate, one continuously reserved PTU over exactly thirty days costs $2,160 before any additional charges; ten cost $21,600. This is arithmetic, not a customer quote or a statement that one unit can run a particular application. If the same reservation serves half as much useful work, its cost per completed unit of work doubles. A predictable bill removes surprise from spending, not from demand.

The service promise also needs careful reading. Marketing uses a 99% uptime shorthand, but the documentation defines reliability as at least 99% of eligible requests completing successfully each month without a Together-caused error. Requests outside purchased capacity and stated limits are excluded. Overflow may use shared serverless capacity at standard rates, without the PTU guarantee. A request-success measure is not a promise about clock-time availability. Product overview; Detailed terms.

This is the practical bargain beneath the fixed bill: the customer buys a defined amount of preparedness, not an unlimited safety net. Steady traffic can make the commitment attractive. Sharp peaks can require more reserved capacity or expose overflow to a different service level. Together benefits when reservations and available machines can be planned together. The buyer benefits when the purchased shape resembles the workload. Neither side wins automatically merely because billing is predictable.

Model choice is not a finished exit plan

The Washington Post deployment is useful because it shows an actual application rather than a hypothetical model migration. It does not establish the Post's complete unit economics. The public case study gives neither the total monthly charge nor the output-token volume and workload distribution needed to reproduce the bill. Its response-time result also does not disclose a percentile or a complete end-to-end measurement method.

For any customer, retaining model access and retaining a cheap, fast deployment are different things. Moving a model to another provider may be possible while still requiring integration work, performance tuning and new capacity commitments. The value of a specialist is partly the work it saves. That same saved work can become work the customer must recreate when leaving. This is an operational switching cost, even without an exclusive model contract.

Together therefore has a plausible position between self-operated infrastructure and a closed model API. It can sell a managed service while leaving more model choice with the customer. The durable advantage would be making that freedom economical in practice: serving the chosen model efficiently, adapting when it changes, and keeping capacity available without pricing away the benefit.

What would make the case stronger

The next revealing disclosures would connect bookings to recognized revenue and cash, distinguish committed capacity from live supply, and show the cost of keeping customer promises. Research results should be followed by production economics on comparable workloads. The November compiler announcement makes the software argument more concrete; the IBM agreement makes the supply ambition more concrete. Neither, alone, proves the margin between them.

For the newspaper's reader, all of this remains backstage. A useful answer arrives, or it does not. Together's opportunity is to make that ordinary experience easier for customers to deliver at scale. Its test is to turn the engineering and capacity required for predictability into a business that is predictable enough itself.