How much does Enterprise AI actually cost?
Topic
AI Transformation
Kind
News
Author
Mario Grunitz
Read time
3 min read
One of the least useful questions in enterprise AI is:
How much does the model cost per million tokens?
It matters.
But it tells you surprisingly little about how much your AI system costs.
A cheap model inside an inefficient workflow can be expensive.
An expensive model completing valuable work in one attempt can be cheap.
The metric enterprises eventually need is closer to:
Cost per successful task.
Start with the task
Suppose an AI agent prepares a sales account review.
To complete one review, it might:
Retrieve CRM data.
Search previous correspondence.
Query financial information.
Call an LLM.
Execute an analysis.
Call the LLM again.
Generate a report.
Store the result.
The model API is one cost inside that sequence.
Model inference
Inference costs depend on:
- model
- input volume
- output volume
- context size
- caching
- number of calls
- retries
Agentic workflows can multiply these costs because one user request may trigger many model interactions.
A €0.03 call repeated twenty times is no longer a €0.03 task.
Retrieval
Knowledge systems have their own economics.
Documents need processing.
Embeddings may need generation.
Indexes need storage.
Search infrastructure needs running.
Information needs refreshing.
At enterprise scale, retrieval becomes a service with operational cost.
Compute
Agents increasingly execute work.
Python.
Browsers.
Containers.
Sandboxes.
Data processing.
Image generation.
Model inference.
Each action consumes compute.
Long-running autonomous workflows can consume considerably more infrastructure than simple chat interactions.
Integration
Connecting enterprise systems is often a larger implementation cost than model usage.
ERP.
CRM.
Databases.
Identity.
Document systems.
APIs.
Legacy applications.
The AI needs reliable access to the organisation around it.
That engineering work is part of the AI business case.
Evaluation
Production AI needs measurement.
Test datasets.
Automated evaluations.
Human reviews.
Monitoring.
Regression testing.
Red-team testing.
These activities cost money.
They also prevent much larger costs caused by unreliable systems.
Human escalation
Human-in-the-loop systems contain a human cost.
Suppose an agent completes 90% of a process automatically but escalates 10%.
That 10% should be included in the economics.
The important calculation is the cost of the complete operating model.
Failure
Failures are expensive.
A failed task may trigger:
- retries
- larger models
- human review
- customer support
- correction
- repeated execution
This is why model accuracy and cost cannot be evaluated independently.
A cheaper model with a high failure rate can produce a more expensive system.
The right equation
A simplified enterprise calculation might look like:
AI task cost = inference + retrieval + compute + infrastructure + human intervention + expected failure cost
Then compare that with:
Value per successful task.
That value might come from:
Time saved.
Revenue generated.
Errors prevented.
Faster response.
Higher conversion.
Reduced outsourcing.
Better employee capacity.
Measure cost per successful task
Imagine two systems.
System A costs €0.08 per attempt and succeeds 70% of the time.
System B costs €0.14 and succeeds 98% of the time.
The cheaper inference is not automatically the cheaper business process.
Now add human correction and the difference can grow considerably.
This is why we prefer task economics over token economics.
Small models can change the equation
High-volume predictable tasks may benefit from smaller models.
Classification.
Routing.
Extraction.
Entity recognition.
Simple transformation.
If an SLM performs adequately, running those tasks locally or on cheaper infrastructure can materially reduce operating cost.
Save frontier models for tasks that require frontier capabilities.
Normal software can change it even more
Sometimes an LLM should not be called.
If a database can calculate something, let the database calculate it.
If deterministic software can validate something, use deterministic software.
If an answer can be safely cached, reuse it.
AI architecture becomes more economical when teams stop trying to make AI perform every operation.
Include the cost of dependency
There is another cost that rarely appears in the spreadsheet.
Switching.
If prompts, workflows, memory, data and business logic become deeply tied to one provider, moving later becomes expensive.
That is architectural debt.
It should be considered when evaluating apparently cheap platforms.
Build the business case around work
Do not begin with:
How many tokens will we use?
Begin with:
How much work is being performed today?
What does that work cost?
How long does it take?
What is the error rate?
What could AI change?
What percentage can realistically be automated or accelerated?
Then design the system.
The objective is not cheap AI.
It is economically useful AI.





Fresh insights, straight to your inbox.
Engineering breakdowns, sovereign-AI perspectives and practical guides from the people building enterprise AI. One email when there is something worth your time.