Once an AI prototype becomes a real system, an interesting thing happens.
The model stops being the architecture.
It becomes one component inside a larger system of data, retrieval, tools, policies, evaluation, observability and operational controls.
That distinction matters because teams can spend enormous amounts of time choosing models while under-investing in everything that determines whether the resulting system is actually useful in production.
| Experience · Applications · Agents | ||
| Orchestration | Context & Retrieval | Tools & Integrations |
| Model layer · LLMs · embeddings · specialised models | ||
| Evaluation | Observability | Security & Governance |
The model is only one layer
A useful way to think about an AI application is as a set of layers, each with a different responsibility.
| Layer | Primary responsibility |
|---|---|
| Experience | How users interact with the capability. |
| Orchestration | How the workflow moves between models, tools and application logic. |
| Context | What information the system is allowed to use. |
| Model | Generation, classification, reasoning or transformation. |
| Tools | How the system interacts with external services. |
| Policy | What the system is allowed to do. |
| Evaluation | How we know whether it works. |
| Observability | How we understand behaviour in operation. |
The exact implementation will vary, but the responsibilities are useful because they stop everything being pushed into a prompt.
Experience is not just a chat box
The interface should reflect the job the AI is doing.
Sometimes conversation is exactly right. Sometimes it is not.
An AI system reviewing a pull request might work better inside the development workflow. An operations agent may need a queue of proposed actions. A customer service assistant may need structured fields alongside generated text.
The interface should make the system’s state and authority visible.
If an action is waiting for approval, the user should know. If the system is uncertain, that should be distinguishable from a completed decision. If an agent has taken an action, the resulting state should be visible.
Orchestration is where the workflow lives
As soon as an AI application uses multiple steps, orchestration becomes important.
A workflow might look like:
- Receive a request.
- Authenticate the user.
- Retrieve relevant information.
- Ask a model to interpret the request.
- Validate the proposed result.
- Call a tool.
- Check the outcome.
- Return the result or escalate.
Putting all of that into one enormous prompt makes the system difficult to reason about.
Explicit orchestration makes the workflow inspectable. It also gives engineers places to insert validation, retries, policy checks and observability.
Context is a product decision
Retrieval is often described as a technical problem: put documents into a vector database and retrieve the relevant chunks.
But the deeper question is:
What information should this system be allowed to use when making this decision?
That is partly an architectural question and partly a product and governance question.
Context can come from documents, databases, APIs, conversation history or live operational systems. Each source has different freshness, authority and access characteristics.
A production system should therefore distinguish between information that is merely available and information that is authoritative for the decision being made.
Don’t ask the model to enforce every rule
One of the recurring mistakes in AI application design is putting deterministic business logic into natural-language instructions.
If a rule can be expressed precisely in software, it often should be.
The model can interpret an ambiguous request. A policy engine can determine whether the resulting action is permitted.
The model can extract the relevant amount from a document. Application logic can check whether that amount is within an authorised limit.
The model can propose a database change. The application can validate the schema and permissions before executing it.
This division of responsibility makes the system easier to test and easier to govern.
Tools turn language into side effects
Tool use is where AI applications become operationally interesting.
A model generating text has limited direct impact. An agent with access to APIs can change the world.
That means tools deserve the same engineering attention as any other production interface.
For each tool, define:
- what it does;
- which inputs it accepts;
- which identities can invoke it;
- what permissions it requires;
- what side effects it creates;
- how success is confirmed;
- how failure is handled;
- whether the operation is reversible.
Tool definitions should be deliberately narrow. “Manage the customer account” is a much harder capability to constrain than “update the customer’s support status”.
Evaluation should sit beside development
Evaluation is not a final gate that happens before launch.
It should be part of the development loop.
Every meaningful change can potentially affect behaviour:
- a different model;
- a prompt change;
- a retrieval change;
- a new tool;
- a policy change;
- a different chunking strategy;
- a new piece of context.
The evaluation harness gives the team a way to see whether those changes improve the system or simply move errors around.
This is one of the strongest arguments for treating AI development as engineering rather than prompt experimentation.
Observability needs two dimensions
AI systems need conventional technical observability: latency, errors, throughput and availability.
They also need behavioural observability.
You want to know things such as:
- which types of requests are failing;
- which tools are being selected;
- how often humans intervene;
- where the system is escalating;
- which evaluation cases are regressing;
- how model and retrieval changes affect outcomes.
Without behavioural signals, an AI system can be technically healthy while becoming less useful.
Cost belongs in the architecture
AI cost is not simply an infrastructure bill.
It can influence architectural choices.
A workflow that calls a large model five times, performs several retrieval operations and invokes multiple tools may produce an excellent result while being economically unsuitable at scale.
That does not necessarily mean using a cheaper model everywhere.
It may mean changing the workflow:
- use deterministic logic before invoking a model;
- route simple cases to smaller models;
- retrieve only the context that is needed;
- cache stable results;
- reduce unnecessary agent loops;
- use expensive reasoning only where it changes the outcome.
The architecture should make those choices visible rather than hiding them inside a single abstraction.
Security becomes part of the AI design
AI systems introduce familiar security problems in unfamiliar places.
Prompt injection, data leakage, excessive tool permissions and untrusted retrieved content can all become pathways to unwanted behaviour.
The response is not simply “write a better system prompt”.
Use conventional security boundaries:
- least-privilege credentials;
- explicit authorisation;
- input and output validation;
- network and service boundaries;
- secret management;
- audit trails;
- approval for consequential actions.
Assume that model instructions can be confused. Design the surrounding system so that confusion does not automatically become a security incident.
Choose the smallest architecture that proves the point
There is a danger in the other direction too.
Once teams discover the growing AI engineering stack, it is tempting to build everything at once: vector databases, agent frameworks, evaluation platforms, orchestration engines and elaborate observability pipelines.
That can turn a small experiment into a platform project before the underlying use case has been validated.
A better approach is to add architecture in response to a real requirement.
Start with the narrowest workflow that can demonstrate value. Add evaluation when you need repeatability. Add retrieval when the task needs external context. Add tools when the workflow needs actions. Add stronger policy controls when the consequences require them.
The architecture should grow with the evidence.
A useful architecture review
For any proposed AI capability, I would walk through the system with eight questions:
- Experience: What is the user actually trying to accomplish?
- Workflow: What steps are required to achieve it?
- Context: Which information is authoritative?
- Model: Which parts genuinely require generative or probabilistic behaviour?
- Tools: What external actions are necessary?
- Policy: Which actions are allowed, conditional, approval-required or prohibited?
- Evaluation: How will we know a change improved the system?
- Operations: How will we detect, diagnose and recover from failures?
If those questions have clear answers, the system is becoming an engineered product rather than an AI feature wrapped around a prompt.
The model is becoming a commodity component
Models will continue to improve, prices will change and new providers will appear.
That makes it increasingly important not to build the entire application around assumptions about one particular model.
The durable engineering value is often elsewhere: the workflow, the data, the evaluation set, the policy layer, the integrations and the operational knowledge accumulated from real usage.
A strong AI architecture should therefore make model changes possible without requiring the entire product to be redesigned.
AI engineering is systems engineering
The most interesting AI applications are not simply applications with an LLM inside them.
They are systems in which probabilistic components are deliberately combined with deterministic software, data, policies and human judgement.
The model is important.
But the engineering challenge is everything around it.
Build the smallest system that can prove the value. Measure its behaviour. Put explicit boundaries around its authority. Then add complexity only when the evidence says you need it.
That is how an AI prototype becomes an engineered system.