RAG Is not an Enterprise AI architecture
Topic
Enterprise Intelligence
Kind
Perspective
Author
Vladyslava Mylohradova
Read time
4 min read
Retrieval-Augmented Generation solved one of the first major problems with enterprise generative AI.
Models know a lot.
They do not know your company.
RAG gave applications a practical way to retrieve relevant company information and provide it to a language model before generating an answer.
It remains useful.
But somewhere along the way, RAG started being treated as an enterprise AI architecture.
It isn't.
It is one technique inside one.
What RAG does well
The basic pattern is straightforward.
A user asks a question.
The application searches a knowledge source.
Relevant information is retrieved.
That information is added to the model's context.
The model generates an answer.
This works particularly well for relatively static knowledge.
Policies.
Documentation.
Manuals.
Research.
Product information.
Internal knowledge bases.
The model gets access to information it was never trained on without requiring the organisation to retrain the model.
Enterprise knowledge is not one knowledge base
Now ask:
Which customers in Germany have overdue invoices, declining revenue and an open support escalation?
The answer may require:
CRM data.
ERP data.
Financial records.
Support systems.
Identity permissions.
Current transaction data.
A vector database containing embedded documents cannot reliably represent all of that.
Some information needs retrieval.
Some needs SQL.
Some needs APIs.
Some needs calculations.
Some needs another enterprise system.
The AI layer needs to know which is which.
Retrieval can make information stale
A RAG system often creates another representation of company information.
Documents are chunked.
Embeddings are generated.
Information is indexed.
The source changes.
Now the index needs to change too.
For many knowledge use cases this is completely manageable.
For transactional information it can become unnecessary duplication.
If inventory changes every minute, ask the inventory system.
If a customer's payment status changed ten seconds ago, ask the financial system.
AI does not need to memorise the enterprise.
It needs to know how to reach it.
Agents need tools
This is where tool-based architectures become useful.
An AI system can have several capabilities.
- search_documents()
- get_customer()
- query_orders()
- calculate_margin()
- get_inventory()
- find_support_cases()
The model or orchestration layer determines which capabilities are required.
RAG becomes one tool among many.
Different knowledge needs different access
A useful enterprise architecture can distinguish between:
Unstructured knowledge
Documents, reports, manuals and policies.
Often suited to retrieval.
Structured data
Customers, transactions, products and financial information.
Often better accessed through APIs or controlled database queries.
Real-time state
Inventory, system status, prices and operational events.
Best queried directly.
Calculated information
Margins, trends, forecasts and aggregations.
Often better computed than generated.
External information
Potentially retrieved through controlled web or third-party services.
Treating all five as documents inside a vector database removes useful structure from the information.
RAG does not solve permissions
Retrieval also needs access control.
A document may exist in the index.
That does not mean every employee should retrieve it.
Permissions should travel through the retrieval process.
Who is asking?
Which sources can they access?
Which chunks are eligible?
Can the resulting information be sent to this model?
The retrieval layer is part of enterprise security.
RAG does not solve execution
There is another limitation.
Retrieval helps an AI know something.
Enterprises increasingly want AI to do something.
Create a report.
Update a record.
Run an analysis.
Prepare an order.
Execute code.
Start a workflow.
RAG has little to say about those problems.
Execution requires tools, permissions, orchestration, validation and often human approval.
RAG does not solve memory
Retrieving documents is also different from maintaining workflow state.
An agent processing a task over several hours may need to know:
What has already happened?
Which tools were called?
Which approvals were received?
What remains unfinished?
That is operational memory.
It requires different architecture from semantic document retrieval.
Build a knowledge architecture instead
Instead of asking:
How do we put our company into RAG?
Ask:
How should AI access each type of company information?
For each source, determine:
Where does the authoritative information live?
How current must it be?
Who can access it?
How should it be retrieved?
Should the model see the raw data?
Can normal software calculate the answer?
Does the information need to leave its source at all?
Those questions produce a much stronger architecture.
Where SINAS fits
SINAS treats retrieval as one capability available to agents.
An agent can retrieve documents when retrieval is appropriate.
It can query structured systems when structured access is better.
It can execute controlled tools.
It can call models.
It can request human approval.
The orchestration layer coordinates these capabilities.
This is important because enterprise intelligence does not live in a vector database.
It lives across the organisation.
RAG helped AI read company documents.
The next architecture needs to help AI work with the company itself.





Fresh insights, straight to your inbox.
Engineering breakdowns, sovereign-AI perspectives and practical guides from the people building enterprise AI. One email when there is something worth your time.