When the AI feature eats the margin: gross margin for a SaaS that calls a model

A SaaS product used to cost almost nothing to serve one more customer. The servers, the database and the support desk were shared, so gross margin sat high and stayed there. Add a feature that calls a language model and that changes: every time a customer presses the button, you pay the model vendor. The cost is small per use, but it grows with use, and use is not spread evenly.

This article works through one small vendor's numbers: what the AI feature costs per customer, why a tenth of the customers can take most of the margin, and what four levers (prompt caching, a cheaper model for most requests, caps and pricing) do to it. Then how investors read AI gross margin in 2026, because it is now one of the first numbers they ask about.

What one use costs

Model vendors price by the token, roughly three quarters of a word, with output tokens costing about five times input tokens. List prices per million tokens, read on 15 October 2026:

- Anthropic. Claude Sonnet 5.5 is $2 input and $10 output; Claude Haiku 5.5 is $0.10 and $0.50 for prompts up to 100,000 tokens; Claude Opus 5.5 is $4 and $20. Reading from the prompt cache costs $0.10 per million on Sonnet 5.5, a twentieth of the normal input price, and batch processing (results within hours, not seconds) is half price (Claude, Pricing).
- OpenAI. gpt-5.4-mini is $0.75 input, $0.075 cached input and $4.50 output; gpt-5.4-nano is $0.20, $0.02 and $1.25; the batch tier is also half price (OpenAI API, Pricing).

Take a typical feature: the customer asks the product to draft something (a reply, a summary, a report section). Each request sends about 6,000 tokens (instructions, examples and the customer's own data) and gets back about 800. On Sonnet 5.5 that is 6,000 × $2 + 800 × $10 per million, or 2 cents a use. Nobody notices 2 cents. The question is how many uses each customer makes.

A worked example: 100 customers at $30 a month

An illustrative vendor sells one plan at $30 a month with the AI feature included and unlimited. Hosting, support and payment fees come to $3 a customer. It has 100 customers, and their use of the feature falls into three bands:

The numbers are ours, but the shape is the usual one. Light users are almost free. The average customer is fine. Ten heavy users, who might be the ones running the feature from a script or pasting in a whole mailbox every morning, cost twice what they pay and take 72% of the AI bill. Gross margin falls from 90% to 62% and the average hides why.

Agent features make the spread wider. An agent that plans, calls tools, checks its work and retries uses many requests per task; ICONIQ, the growth investor, puts multi-step agent workflows at "5 to 30 times as many tokens as a simple chat exchange" (ICONIQ, AI Economics is a Design Problem). Its 2026 report has a builder whose workflow, budgeted at ten cents a run, went past $1.50 as the agents retried and corrected themselves (ICONIQ, 2026 State of AI).

Lever 1: prompt caching

Most of those 6,000 input tokens are the same for every request: the instructions, the examples, the format. Model vendors let you mark that shared beginning of the prompt as cacheable. The first request writes it to the cache at a small premium (1.25 times the input price for a five-minute cache on Anthropic), and requests that start with the same text within the window read it at a fraction of the price (Claude Docs, Prompt caching). Because the instructions are the same across customers, a product with steady traffic keeps the cache warm all day.

If 4,000 of the 6,000 tokens are the shared part, a use on Sonnet 5.5 drops from 2.0 to about 1.24 cents. The AI bill goes from $828 to $513 and gross margin from 62% to 73%. The work is ordering the prompt so the fixed part comes first and the customer's data last; nothing changes for the customer. Heavy users still lose money, though: $37 of cost against $30 of price.

Lever 2: a cheaper model for most requests

Not every request needs the best model. Short drafts, classification, extracting fields and summaries of short texts are often done well by a small model at a twentieth of the price or less. The common pattern is routing: send each request to the cheapest model that does that kind of task well enough, and send the hard ones up. ICONIQ's advice is to "use the lowest-cost model that reliably clears the quality bar for a task, and escalate only when the added capability is worth more than the added cost". In its 2026 survey of about 300 software companies, two thirds reported better cost per query, helped by "managing inference costs, improved model routing strategies" and growth.

In the example, sending 70% of requests to Haiku 5.5 (with caching) and 30% to Sonnet 5.5 brings an average use to about 0.42 cents. The AI bill falls to $173 and gross margin to 84%. Even the heavy band is now profitable, at about 48%. The cost is testing: you have to show that the small model's output is good enough for the requests you route to it, on your own examples. The method in fifty examples and a spreadsheet works for this.

Running an open-weight model on your own servers is the next step down in cost per token for very high volume, with its own fixed costs; we went through when it pays in open-weight models in a small company.

Lever 3: caps and fair use

"Unlimited" is a promise to your heaviest user. A cap does not have to be tight to work: in the example, including 1,000 uses a month covers every light and normal customer many times over and touches only the heavy band. What happens at the cap matters more than the number. Choices, in order of how much they annoy people: a warning at 80%, a slower or smaller model after the cap, a top-up pack, or a hard stop until next month. Write it in the plan description before you need it, not in an email to the one customer who hit it.

ICONIQ makes a fair point about caps: "A cap can prevent a runaway bill; it cannot determine which workflows deserve investment." A cap protects the margin from accidents and scripts. It does not tell you whether the heavy users are your best customers, which they often are.

Lever 4: charge for the heavy use

If the heavy users get real value, charge for it instead of capping it. With the 1,000 included uses and $15 per extra 1,000, each heavy customer in the example pays $60 a month instead of $30, which roughly covers their AI cost before any caching and leaves a healthy margin once caching is on. Revenue goes from $3,000 to $3,300 with no change for 90% of customers. A higher plan with a bigger allowance does the same thing with less arithmetic for the customer.

This is where much of software pricing is going. ICONIQ's 2026 survey found consumption-based pricing used by 42% of AI product builders, up from 35% six months earlier, and outcome-based pricing at 23%, up from 18%; subscriptions are still the most common. Which unit to charge for is its own decision; we compared the options in per seat, per use or per result.

How investors read AI gross margin

Gross margin has always been one of the first things a software investor checks, because high gross margin is what makes software worth more per dollar of revenue than services. AI has moved the reference points:

- Bessemer split fast-growing AI companies into two groups in its State of AI 2025 report. "Supernovas", which grow extremely fast, average about 25% gross margin, often negative. "Shooting Stars", which grow more like the best SaaS companies, average about 60%, "slightly lower than SaaS peers due to faster growth and modest model-related costs" (Bessemer, The State of AI 2025).
- ICONIQ's survey of about 300 software executives building AI products found average AI product gross margin of 45% in 2025, with 53% projected for 2026 and 59% for 2027 (ICONIQ, 2026 State of AI).

So a margin well below classic SaaS is not a reason to walk away in 2026, for an AI-first product. What investors look for is the direction and the explanation: margin rising as model prices fall and as you route, cache and price better; cost per customer known by band, not only on average; and no customer group that loses money without a plan for it. For a small vendor adding an AI feature to an existing product, the bar is higher: buyers and investors will compare you with your own margin before the feature, and will want to see that the feature earns its cost.

A monthly check for a small vendor

None of this needs a finance team or a data warehouse; the model vendor's usage export and your customer list are enough. The vendor in the example went from 62% to 84% without raising a single price, by ordering its prompts and sending easy requests to a small model. The heavy customers who were losing money became the most profitable ones once they paid for what they used.

Sources

Claude, Pricing (Sonnet 5.5, Haiku 5.5 and Opus 5.5 token prices, cache read prices, batch at 50%); Claude Docs, Prompt caching (cache writes at 1.25 times input for five minutes, reads at a fraction of input); OpenAI API, Pricing (gpt-5.4-mini and gpt-5.4-nano prices, batch at 50%); ICONIQ, 2026 State of AI: The Builder's Economy (July 2026; about 300 executives; AI product gross margin 45% in 2025, 53% and 59% projected; pricing models; routing; the ten-cent workflow); ICONIQ, AI Economics is a Design Problem (13 August 2026; 5 to 30 times the tokens; routing advice; caps); Bessemer Venture Partners, The State of AI 2025 (Supernovas about 25%, Shooting Stars about 60% gross margin). Pages read on 15 October 2026. The vendor, its customers, usage bands and the token counts per use are our own illustration, not measurements.