the real cost of llm use is already hitting some large companies like a truck. and i expect individual developers will soon start to feel the costs too.

i have been talking about the end of the subsidies and how to prepare for the actual costs of llms with my peers for a while now. and at the information and data team at cejam, i have helped us prepare for it in three major ways: implementing a governance gateway for cost control, providing training on llm usage, and deploying cheaper models.

the end of the subsidies

current llm subscription models were heavily subsidised. be it for market capture or gathering more training data, those subsidies are starting to run out.

anthropic is changing the way they charge users and github moved to usage-based billing. the truth is the subsidized subscription model was unsustainable long-term, and the folks at the major ai companies most certainly knew that from the get-go.

while before, some companies had leaderboards for token use, now there is talk of a tokenpocalipse, and people are scrambling to reduce token usage¹.

bifrost as an llm gateway

i have been using microsoft foundry for a while now to deploy llms for my team. the trouble with foundry is that it exposes a single project endpoint and token that are used for consuming llms from the api.

this is no good for a team with many members, where many people might be using the same llm at once. how do you keep track of costs and use? how do you control leaks and exploits? microsoft offers their own api gateway solution, but the cheapest subscription is $150, and i knew i could find a better value proposition.

while looking for alternatives, i found litellm, which seems to be turning into a sort of industry standard. but the recent supply chain incident turned me away from it. so i decided to go with bifrost, a newer llm gateway written in go to be performant and lite.

with bifrost, we generate keys for team members and projects, setting budget limits for varying time frames. it also gives full governance over requests, letting me see exactly what is being sent to the llms.

llms agents are not magic (though they sometimes feel like they are)

i have found that a basic understanding of context windows, token caching, and how llm input and reasoning pricing² works goes a long way in helping users make better use of llm agents. when users understand how context accumulation affects cost and performance, they naturally adopt better practices like using fresh sessions for new tasks.

deploying cheaper models

open source models have been getting really exceptional in their cost-benefit. deepseek v4 pro is really cheap for what it can do.

i am a strong believer that we will see more and more that llm performance improvements have to do with technique than model size: appropriate context handling, atomic tasks, agent management.

we are already seeing that with cybersecurity. smaller models can perform as well as mythos when given the proper harness, for a fraction of the cost.

the future

frontier models like claude sonnet can be 5x the base price of open weight models like deepseek v4 pro. for many production use cases, this cost difference makes smaller models with good harnesses a more practical choice.

i believe the future of llm agent implementation in production will favor targeted, atomic tasks using smaller models with strong tooling and constrained contexts.