Somewhere out there, a developer is teaching Claude to talk like a caveman. This is a real thing. A viral Claude Code skill strips the articles, pleasantries, and filler out of the model’s responses, so a sentence like “the reason your component keeps re-rendering is likely that you are creating a new object reference on each render cycle” collapses into “new object ref each render, re-render, wrap in useMemo.” The reported saving is up to 75 percent fewer output tokens, with the technical content fully intact. Brain still big, as the skill’s author puts it. Mouth small.
It is the most primitive cost optimization imaginable, and the fact that people reach for it tells you exactly where AI operations are heading. Developers are willing to make their assistant grunt like a Neanderthal to trim a few thousand tokens off a session. Something changed to make that feel worth doing.
For the past two years, most of us have been running AI workloads on someone else’s dime. Flat-rate subscription plans made generative AI feel free at the point of use. Fire off as many prompts as you want, let your coding agent churn through refactors overnight, and the bill stays the same.
That era is ending. LLM providers are increasingly pushing heavy workloads toward usage-based pricing, credits, stricter rate limits, or token-metered APIs, and the reason is simple: the plans were too cheap. The compute behind a heavy agentic workload costs real money, and providers can no longer subsidize power users with the margins of light users. Rate limits are tightening, “unlimited” tiers are disappearing, and token-based billing is becoming the norm.
If you have ever watched an AWS bill balloon because nobody rightsized their EC2 fleet, you know exactly what happens next. The token is the new gigabyte, and we are about to relive the FinOps learning curve, this time for AI.
That is why we need FinAIOps: FinOps for AI systems, where LLMOps meets cost discipline in production. It is the practice of running AI workloads with the same operational and cost rigor we apply to everything else in the cloud.
Lessons from building agents in production
At Cloudar we have been building and running agentic AI workloads in production. Doing that teaches you very quickly where tokens go to die. These are the practices that made the biggest difference.
1. Use smaller models for simpler tasks
Not every task needs your most capable model. Agent routing, classification, and simple extraction run perfectly well on smaller, cheaper models. Reserve the frontier model for the reasoning-heavy steps. The per-token price difference between a frontier model and a small model on Bedrock is easily an order of magnitude, so a lightweight router that dispatches work to the right specialist pays for itself immediately.
This is where Amazon Bedrock shines: switching between model families and sizes is a configuration change, not a re-architecture. You can experiment with the cost and quality trade-off per task and measure the result.
2. Gatekeep the agent: deterministic first
The cheapest LLM call is the one you never make. If a task can be solved deterministically, solve it deterministically. Fetching a metric, checking a threshold, parsing a known log format: none of that needs an agent. Put a gate in front of your agent that handles the predictable cases with plain code and only escalates genuine ambiguity to the model.
3. Keep prompts compact
Prompts are tokens, and tokens are money. Every instruction, every example, every “please be helpful and thorough” costs you on every single invocation. Ruthlessly edit your system prompts. Say what you need, cut what you do not, and test whether shorter prompts degrade quality. Usually they do not.
4. Limit the tools your MCP servers expose
This one is underestimated. Every tool definition you expose to an agent is injected into the context on every call. It might feel convenient to give your agent the full toolbox, but a wall of tool schemas eats tokens before the agent has done any actual work.
Only expose the tools the agent genuinely needs. And when you do not know upfront which tools an agent will use, for example with open-ended investigative workloads, take an iterative approach: expose everything, have the agent write a short analysis after each run, and after a set period feed those results to an advanced model to analyze which tools earn their place and which can be dropped. Treat your tool catalog like an IAM policy: least privilege, reviewed regularly.
5. Cap the loop
Agentic workloads can run away. An agent stuck in a retry loop is the token equivalent of a Lambda retry storm. Set a maximum number of steps and tool calls per run, and a hard token budget per invocation. Fail loud, not expensive.
6. Mind your outputs and your history
For many frontier models, output tokens are priced significantly higher than input tokens. Ask for structured output instead of prose, set max_tokens deliberately, and instruct the model to be terse. And do not drag the full conversation history through every turn: summarize or window older context, because in long-running agents the context snowballs and you pay for all of it on every call.
7. Use prompt caching
On Bedrock, prompt caching lets you cache the static parts of your context, such as system prompts and tool definitions, and pay a fraction of the input price on subsequent calls. If you have a large tool catalog you cannot trim further, caching softens the blow considerably. Combined with point 4, this is one of the highest-impact optimizations available today.
8. Batch what is not urgent
Amazon Bedrock batch inference runs asynchronous workloads at a 50 percent discount compared to on-demand pricing. You submit a JSONL file with your prompts, Bedrock processes them asynchronously, and the results land in S3, typically within 24 hours. Periodic analysis jobs are a textbook example: nobody is waiting on the result, so there is no reason to pay real-time prices for it.
Bedrock also offers a Flex service tier for supported models, trading latency for lower cost. Unlike batch, Flex uses the regular invocation API: you add a service tier parameter to your call and accept increased latency in exchange for the lower rate. Availability and discount levels depend on the model and tier, so check the current pricing page before assuming the same economics as batch. It is a good fit for background agent runs that are real-time in shape but not in urgency.
The practices above, at a glance:
| Optimization | Saves | Risk |
|---|---|---|
| Smaller model routing | High | Quality regression |
| Deterministic gate | Very high | Missed ambiguity |
| Prompt trimming | Medium | Lost instructions |
| Tool pruning | High | Agent can’t act |
| Loop caps | Very high | Incomplete runs |
| Prompt caching | High | Cache eligibility limits |
| Batch inference | High | Latency |
Measure it or it did not happen
Optimization without measurement is guesswork. Two AWS capabilities matter here.
Amazon CloudWatch gives you deep visibility into every dimension of your Bedrock usage. Bedrock publishes runtime metrics to the AWS/Bedrock namespace, with ModelId among the available dimensions. Which dimensions apply, and whether ModelId alone is enough, depends on the model, Region, service tier, and whether you invoke through an inference profile, so treat any dashboard as a starting point rather than a template that works unchanged everywhere. The ones to watch:
Invocations: how often each model is calledInputTokenCountandOutputTokenCount: where your money actually goesInvocationLatency: the quality side of the trade-offInvocationThrottlesandInvocationClientErrors: the signals that an agent is misbehaving
Build a dashboard on these, and set an alarm on token consumption. For example, a CloudWatch alarm on the hourly Sum of OutputTokenCount per model catches a runaway agent loop within the hour instead of on next month’s invoice:
aws cloudwatch put-metric-alarm \
--alarm-name bedrock-output-token-spike \
--namespace AWS/Bedrock \
--metric-name OutputTokenCount \
--dimensions Name=ModelId,Value=<your-bedrock-model-id-or-inference-profile> \
--statistic Sum --period 3600 \
--threshold 5000000 \
--comparison-operator GreaterThanThreshold \
--evaluation-periods 1 \
--alarm-actions arn:aws:sns:eu-west-1:123456789012:ops-alerts
Replace the model ID with the exact model or inference profile dimension used in your account, since Bedrock model IDs are versioned and vary by Region and provider. Enable model invocation logging as well, so you can see the actual prompts behind an anomaly. Treat anomalous spend like any other operational incident.
Bedrock application inference profiles let you tag usage per agent, per workflow, or per customer. You create a profile that wraps a foundation model, attach cost allocation tags, and invoke through the profile ARN instead of the model ID:
aws bedrock create-inference-profile \
--inference-profile-name customer-a-investigator \
--model-source copyFrom=arn:aws:bedrock:eu-west-1::foundation-model/anthropic.claude-sonnet-4-5 \
--tags key=customer,value=customer-a key=agent,value=investigator
Those tags flow through to Cost Explorer and CloudWatch, turning “the AI bill is high” into “agent X on task Y for customer Z is driving the cost.” For a managed service provider, this is essential: it enables proper chargeback and shows customers exactly what their AI workloads cost.
Finally, track the metric that actually matters: cost per outcome. Tokens per completed task beats tokens per month. An agent that uses three times the tokens but delivers twice the results is the cheaper agent. That is the KPI that makes FinAIOps a business discipline instead of a savings exercise.
The bill is coming. Be ready.
The shift from plans to tokens is not a pricing tweak, it is a forcing function. Teams that treat AI as free will get the same surprise bills we saw in the early cloud days. Teams that apply FinAIOps discipline, right-sized models, gated invocations, lean prompts, curated tool catalogs, caching, batching, and real cost attribution, will run AI workloads that are both powerful and predictable.
We learned the hard way with EC2 that capacity discipline pays off. Let us not learn it the hard way again with tokens.
