Your Enterprise AI probably does not need a frontier model for every task

Topic

Sovereign AI

Kind

Perspective

Author

Dmitry Ermakov

Read time

5 min read

/

/

Your Enterprise AI probably does not need a frontier model for every task

/

/

Your Enterprise AI probably does not need a frontier model for every task

Enterprise AI has inherited a strange assumption:

More intelligence must be better.

If a 400-billion-parameter model can solve a task, why would you use something smaller?

Because enterprises do not buy intelligence in the abstract.

They buy outcomes.

And once AI systems start processing thousands or millions of requests, the economics change quickly.

The question becomes:

What is the smallest, fastest and most controllable model capable of completing this particular task reliably?

That question leads to a very different AI architecture.

One enterprise does not have one AI problem

Consider a company building an internal AI system.

Employees want to:

  • search documents
  • classify incoming requests
  • extract information from invoices
  • query operational data
  • summarise contracts
  • generate proposals
  • analyse unusual cases
  • create reports
  • trigger workflows

These tasks have radically different reasoning requirements.

Yet many early AI architectures send almost everything to the same large model.

It works.

It is also the AI equivalent of using a Formula 1 car to deliver the post.

Start with the workload

We prefer to classify enterprise AI workloads before choosing models.

A task might require:

Classification

Determine what kind of request has arrived.

Extraction

Turn unstructured content into structured data.

Retrieval

Find relevant information.

Transformation

Rewrite, translate or restructure existing information.

Reasoning

Interpret several sources and reach a conclusion.

Generation

Create new content based on instructions and context.

Execution

Choose and perform actions across other systems.

Each category has different requirements for intelligence, latency, context, privacy and cost.

Model selection should follow those requirements.

Where Small Language Models become interesting

Small Language Models, or SLMs, can be useful for constrained and repetitive workloads.

They may be deployed locally, in a private cloud or within controlled infrastructure.

For the right task, that creates several potential advantages.

Lower inference cost

Smaller models generally require fewer computational resources.

At sufficient volume, the difference can materially affect operating costs.

Lower latency

Running a smaller model close to the application can reduce network and inference overhead.

That matters when AI becomes part of interactive software rather than a separate chatbot.

Greater control

Local or privately deployed models can give organisations more control over where processing happens and how information flows through the system.

Specialisation

A model does not need broad knowledge of philosophy, poetry and astrophysics if its job is to classify service tickets into 23 predefined categories.

Narrow tasks create opportunities for narrow intelligence.

This does not mean replacing frontier models

Large models remain extremely useful.

We use them.

The mistake is treating model selection as a binary choice.

A modern AI system can route work dynamically.

Imagine an enterprise request arriving:

"Show me the customers in Germany whose orders declined more than 20% this quarter and draft an account review for the five largest."

The system might use one model to interpret intent.

A controlled query service retrieves the relevant data.

Code performs the calculation.

A larger reasoning model identifies meaningful patterns.

Another model generates the final account summaries.

The interface presents the result.

Several forms of intelligence may contribute to one user interaction.

The employee sees one system.

Model routing becomes an economic layer

This creates an interesting engineering problem.

For every task, an orchestration layer can consider:

  • required capability
  • sensitivity of the data
  • latency requirement
  • context length
  • expected output quality
  • inference cost
  • geographic requirements
  • model availability

The system can then select the appropriate model.

We call this an intelligence layer rather than a model integration.

The distinction matters.

If your application is designed around one API, the model defines the architecture.

If your application is designed around capabilities, the architecture chooses the model.

A simple enterprise model hierarchy

In practice, an organisation might create several model tiers.

Tier 1: deterministic software

Use normal software when normal software solves the problem.

Calculations, validations, database operations and fixed business rules often do not need an LLM.

Tier 2: small local models

Use for high-volume, predictable tasks where speed, privacy or cost matters.

Tier 3: specialised models

Use domain-specific models for tasks where specialisation improves performance.

Tier 4: frontier models

Use when the task requires broader reasoning, difficult interpretation, long context or sophisticated generation.

This prevents an expensive model from becoming the default answer to every technical problem.

The biggest saving may come from avoiding inference

There is another step that gets less attention.

Sometimes the cheapest AI request is the one you do not make.

Enterprise systems can cache suitable results, reuse embeddings, retrieve deterministic information, precompute common outputs and execute normal software functions.

Agents can also learn that some requests do not require reasoning at all.

An effective AI architecture therefore optimises the complete execution path, rather than simply negotiating a lower token price.

This is part of the thinking behind SINAS

We built SINAS around the assumption that the model layer will keep changing.

Models will become smaller.

Models will become cheaper.

New open models will appear.

European providers will improve.

Specialised models will outperform general models on particular tasks.

Frontier models will continue moving forward.

An enterprise architecture should benefit from all of those developments without being rebuilt each time.

SINAS separates orchestration and execution from the underlying models so that different workloads can use different intelligence.

This also makes local deployment much more practical.

An organisation can keep certain workloads inside controlled infrastructure while selectively using external models where their capabilities justify it.

Benchmark your own workload

There is no universal percentage by which SLMs reduce enterprise AI costs.

Claims such as "85% cheaper" are meaningless without knowing the workload, model, infrastructure, token volumes and quality threshold.

A useful benchmark compares actual tasks.

Take 500 or 1,000 representative examples.

Run them against several model configurations.

Measure:

  • task success rate
  • output quality
  • latency
  • infrastructure cost
  • token cost
  • failure rate
  • escalation rate

Then calculate cost per successful task.

That number is far more useful than cost per million tokens.

Enterprise AI will become heterogeneous

The future enterprise AI stack is unlikely to contain one model.

It will contain many.

Some will run in hyperscale clouds.

Some will run on European infrastructure.

Some may run inside the enterprise itself.

Some will be enormous.

Some will be surprisingly small.

And some tasks will still be handled by ordinary software because ordinary software is the right tool.

The competitive advantage will come from knowing when to use which.

That is where AI engineering becomes more interesting than simply connecting another API.

Avatar-1
Avatar-2
Avatar-4
Avatar-4
Avatar-3

Fresh insights, straight to your inbox.

Engineering breakdowns, sovereign-AI perspectives and practical guides from the people building enterprise AI. One email when there is something worth your time.