Open-weight models in a small company: when running your own beats the API bill

Sooner or later someone in a small company looks at the monthly bill from an AI provider, then at a list of models that can be downloaded for nothing, and asks the obvious question: why are we renting this?

It is a fair question and it has a number for an answer. This article works the number out from price pages as they stood on 12 October 2026, for a firm of five to fifty people. The result may surprise you. On cost alone, running your own model wins much later than most people expect. Running your own model for other reasons is a different matter, and we get to those.

First, what "open-weight" means

A language model is, in the end, a very large file of numbers called weights. With a closed model you never see the file: you send text to the company that owns it and get text back. With an open-weight model the file is published and you can run it on any machine big enough.

Open-weight is not the same as open source. You get the finished model, not the data or the recipe that made it, and each one comes with its own licence. OpenAI's gpt-oss models are under Apache 2.0, about as permissive as licences get (model card). Others ship under terms their makers wrote themselves, with conditions on use. Read the licence before you build a product on a model, the same way you would for any library.

Four places a model can run

The usual framing is "API or self-host". There are really four choices, and the one in the middle is the one people forget.

1. A closed model, hosted by its maker. You pay per token, which is roughly three quarters of a word. Text you send is input, text you get back is output, and output costs about five times as much.

2. An open-weight model, hosted by someone else. Companies such as Together AI run the downloadable models on their own hardware and sell them by the token, exactly like the closed ones. Same model you could run yourself, none of the hardware.

3. An open-weight model on a GPU you rent by the hour. Now you pay for the machine whether it is busy or not, and you run the software on it.

4. An open-weight model on hardware you own. A card in a workstation under a desk, or a server in a cupboard.

Here is what each costs. To compare them we need one figure per option, so we assume a job that reads three tokens for every one it writes, which is typical of summarising, extracting and answering questions over documents.

The rows, in order: OpenAI's gpt-6-astra, gpt-6.1-sol and gpt-6-luna (its largest rival lists mid and small tiers at the same $2 / $10 and $0.10 / $0.50); gpt-oss-120b on Together AI; one Nvidia H100 on Lambda, running all 730 hours of a month; and an RTX 5090 card at the late-July street price quoted by Hatchworks. Links are at the end. Prices move every few months, so treat the table as a method and put in today's figures when you read this.

Two things stand out before any arithmetic. The small closed models are now cheaper per token than a hosted open-weight model of decent size. And the last two rows have no per-token price at all. That is the whole story of self-hosting in one line: you stop paying for use and start paying for capacity.

The break-even, worked through

A rented H100 costs $2,402 a month and does not care how much you send it. So divide.

The $750 is the low end of what Hatchworks estimates for upkeep: ten to twenty hours a month of updates, monitoring and the evening when it stops answering (Hatchworks). In a company of twelve that time belongs to your best developer, so it is not really the cheap end of anything.

Now put a real company next to those figures. Say thirty people each have fifty exchanges a day with an assistant, about 2,000 tokens apiece, over 21 working days. That is 63 million tokens a month. At mid-tier prices it costs $252. On a small model, about $13. The rented GPU would cost more than nine times the first figure and sit idle nearly all the time.

So ordinary office use is nowhere near the line, and it is not close. A firm reaches 600 million tokens a month only when software, not people, is doing the reading: a pipeline that goes through every incoming document, a product feature used by thousands of customers, coding agents running all day.

Why "how busy" decides everything

One benchmark quoted by Hatchworks has a 70-billion-parameter model on a single H100 producing about 2,400 tokens a second when a hundred requests arrive at once. Flat out for a month, that is over six billion tokens of output. At that rate the rented card writes text for about $0.38 per million tokens, which beats every closed model on the list.

But nobody runs flat out. An office works about 160 of the 730 hours in a month, and inside those hours requests come in bursts. Fill the same card to a tenth of its capacity and each million tokens of output costs $3.81. At two percent, $19. The price of self-hosting is set almost entirely by how much of the machine you leave empty. A hosting company fills its cards with thousands of customers. You have one.

The discounts on the other side

The comparison above uses list prices, and heavy users do not pay list. Work that can wait a day is half price: OpenAI's batch rates are half its standard ones. Text the model has already seen in the last few minutes is billed at a fraction: $0.10 per million for gpt-6.1-sol against $2 fresh. An agent that re-reads the same project files on every step is mostly paying that lower rate. Before you conclude the API is expensive, check that you are using it the cheap way.

When running your own does win

The data is not allowed to leave

This is the strongest reason, and it has nothing to do with the table. If a client contract, a regulator or your own judgement says the documents stay on your machines, the price of the alternative is irrelevant.

It is worth being exact about what the hosted providers actually do, because the fear is often vaguer than the facts. OpenAI's documentation says data sent to its API "is not used to train or improve OpenAI models" unless you opt in, and that abuse-monitoring logs are "retained for up to 30 days". Keeping nothing at all is possible, but it "requires prior approval", and so does storing data in a chosen region, which adds 10% to the price of recent models (OpenAI). For plenty of small firms that is enough. For a law practice, a clinic or anyone holding another company's source code, thirty days on someone else's disk may be thirty days too many.

A steady job that keeps the machine full

Overnight transcription. Classifying every support message. Reading two hundred thousand contracts. Work that arrives in a queue can be fed to a GPU at whatever pace keeps it busy, which is exactly the condition under which the capacity price turns into a good per-token price.

Even here, do the sum both ways. One and a half billion tokens a month at mid-tier list price is $6,000, and the rented card with upkeep is $3,152, so the card wins. With the batch discount the API bill is $3,000 and it is a draw. And the same open-weight model, hosted by someone else and billed by the token, is under $400. For a queue of that size the cheapest answer is usually to rent the model, not the machine.

Hardware you already own, and a narrow task

The sums change when the capacity is free. A workstation with a good graphics card that is idle from six in the evening, or a retired gaming PC, costs only electricity to run. The smaller gpt-oss model is built to run "within 16GB of memory", which is a mid-range card. Models of that size sort, tag, extract fields and tidy text perfectly well. They are not the model you want writing a client proposal.

Buying hardware for the purpose is harder to justify. A $4,100 card, in a machine that brings the total to perhaps $6,000, replaces $252 a month of mid-tier API use for our thirty-person firm. That is two years to pay back, before power and before anyone's time, and against a small hosted model it never pays back.

You need the model to stay the same

Hosted models are retired. OpenAI promises "at least 6 months" of notice for a generally available model and as little as two weeks for a preview one, and its list shows a model released in August 2025 due to be switched off in December 2026 (OpenAI). If you have tuned prompts, tests and a customer-facing feature around one model's behaviour, each retirement is a small migration project you did not schedule. A file on your own disk does not change until you change it. For a regulated process that has to give the same answer next year, that is worth money.

What you take on

Upkeep. Drivers, serving software, model updates, a queue, logs, a restart at two in the morning. Somebody owns it.

Security. You now run a server that holds your most sensitive documents and answers questions about them. It needs the same care as your database: no open ports, access per person, patches.

Quality. The best open-weight models are good and the best closed ones are still ahead on the hardest work: long chains of reasoning, large codebases, awkward judgement calls. Do not trust anyone's leaderboard on this, including ours. Take fifty real examples of your own task, run them through both, and have the person who normally does the work mark the results blind.

The second machine. One GPU is one point of failure. If the work matters, you want a spare or a fallback to an API, and the fallback means the data-never-leaves rule needs an exception written down in advance.

A short way to decide

Under about 100 million tokens a month, people typing: use a hosted API. Spend the saved effort on choosing a smaller model for the easy jobs.

Cost is the complaint: before any hardware, move bulk work to a small model or a hosted open-weight one, turn on batch pricing and caching. This usually cuts the bill by more than self-hosting would.

Data must stay inside: start with one owned machine and one narrow job. Measure how full it runs for a month before buying a second.

A product feature with thousands of users: price it per token on someone else's hosting until your own monthly volume, not a forecast of it, passes the break-even you worked out. Then rent by the hour.

If you sell software, the model bill comes straight out of gross margin, which investors now read more closely than headcount. We covered that in small teams, big revenue per head. And if the real question is whether to keep paying for a tool or build your own around a model, that is a separate sum, in buy, build or prompt.

Sources

Prices read on 12 October 2026: OpenAI, API pricing (model prices, batch rates, cached input) and Deprecations; Together AI, Pricing (gpt-oss-120b per token); Lambda, Pricing (H100 per hour). OpenAI, Your data for training, retention and residency. gpt-oss-120b model card for the licence and memory needs. Hatchworks, How to deploy an LLM (updated 1 September 2026) for the upkeep estimate, the graphics card price and the throughput benchmark it quotes from Spheron. The break-even tables, the thirty-person example and the cost at each level of use are our own arithmetic from those figures; the 3:1 mix of input to output and the $6,000 workstation are assumptions, stated where used.