How much does Enterprise AI actually cost?

Topic

AI Transformation

Kind

News

Author

Mario Grunitz

Read time

3 min read

/

/

How much does Enterprise AI actually cost?

/

/

How much does Enterprise AI actually cost?

One of the least useful questions in enterprise AI is:

How much does the model cost per million tokens?

It matters.

But it tells you surprisingly little about how much your AI system costs.

A cheap model inside an inefficient workflow can be expensive.

An expensive model completing valuable work in one attempt can be cheap.

The metric enterprises eventually need is closer to:

Cost per successful task.

Start with the task

Suppose an AI agent prepares a sales account review.

To complete one review, it might:

Retrieve CRM data.

Search previous correspondence.

Query financial information.

Call an LLM.

Execute an analysis.

Call the LLM again.

Generate a report.

Store the result.

The model API is one cost inside that sequence.

Model inference

Inference costs depend on:

  • model
  • input volume
  • output volume
  • context size
  • caching
  • number of calls
  • retries

Agentic workflows can multiply these costs because one user request may trigger many model interactions.

A €0.03 call repeated twenty times is no longer a €0.03 task.

Retrieval

Knowledge systems have their own economics.

Documents need processing.

Embeddings may need generation.

Indexes need storage.

Search infrastructure needs running.

Information needs refreshing.

At enterprise scale, retrieval becomes a service with operational cost.

Compute

Agents increasingly execute work.

Python.

Browsers.

Containers.

Sandboxes.

Data processing.

Image generation.

Model inference.

Each action consumes compute.

Long-running autonomous workflows can consume considerably more infrastructure than simple chat interactions.

Integration

Connecting enterprise systems is often a larger implementation cost than model usage.

ERP.

CRM.

Databases.

Identity.

Document systems.

APIs.

Legacy applications.

The AI needs reliable access to the organisation around it.

That engineering work is part of the AI business case.

Evaluation

Production AI needs measurement.

Test datasets.

Automated evaluations.

Human reviews.

Monitoring.

Regression testing.

Red-team testing.

These activities cost money.

They also prevent much larger costs caused by unreliable systems.

Human escalation

Human-in-the-loop systems contain a human cost.

Suppose an agent completes 90% of a process automatically but escalates 10%.

That 10% should be included in the economics.

The important calculation is the cost of the complete operating model.

Failure

Failures are expensive.

A failed task may trigger:

  • retries
  • larger models
  • human review
  • customer support
  • correction
  • repeated execution

This is why model accuracy and cost cannot be evaluated independently.

A cheaper model with a high failure rate can produce a more expensive system.

The right equation

A simplified enterprise calculation might look like:

AI task cost = inference + retrieval + compute + infrastructure + human intervention + expected failure cost

Then compare that with:

Value per successful task.

That value might come from:

Time saved.

Revenue generated.

Errors prevented.

Faster response.

Higher conversion.

Reduced outsourcing.

Better employee capacity.

Measure cost per successful task

Imagine two systems.

System A costs €0.08 per attempt and succeeds 70% of the time.

System B costs €0.14 and succeeds 98% of the time.

The cheaper inference is not automatically the cheaper business process.

Now add human correction and the difference can grow considerably.

This is why we prefer task economics over token economics.

Small models can change the equation

High-volume predictable tasks may benefit from smaller models.

Classification.

Routing.

Extraction.

Entity recognition.

Simple transformation.

If an SLM performs adequately, running those tasks locally or on cheaper infrastructure can materially reduce operating cost.

Save frontier models for tasks that require frontier capabilities.

Normal software can change it even more

Sometimes an LLM should not be called.

If a database can calculate something, let the database calculate it.

If deterministic software can validate something, use deterministic software.

If an answer can be safely cached, reuse it.

AI architecture becomes more economical when teams stop trying to make AI perform every operation.

Include the cost of dependency

There is another cost that rarely appears in the spreadsheet.

Switching.

If prompts, workflows, memory, data and business logic become deeply tied to one provider, moving later becomes expensive.

That is architectural debt.

It should be considered when evaluating apparently cheap platforms.

Build the business case around work

Do not begin with:

How many tokens will we use?

Begin with:

How much work is being performed today?

What does that work cost?

How long does it take?

What is the error rate?

What could AI change?

What percentage can realistically be automated or accelerated?

Then design the system.

The objective is not cheap AI.

It is economically useful AI.

Avatar-1
Avatar-2
Avatar-4
Avatar-4
Avatar-3

Fresh insights, straight to your inbox.

Engineering breakdowns, sovereign-AI perspectives and practical guides from the people building enterprise AI. One email when there is something worth your time.