Case Studies

Building AI that keeps working when the internet does not

Running a language model locally on compact hardware for crisis-response teams

Sector

Humanitarian technology and crisis response

Client

Crisis-response technology venture

Challenge

Give field teams access to AI when internet and cloud services are unavailable

Solution

Fully self-contained AI assistant running locally on compact, low-power hardware

/

/

Building AI that keeps working when the internet does not

/

/

Building AI that keeps working when the internet does not

01

The challenge

Cloud AI stops working when the connection disappears.

Many AI products assume three things:

  • Reliable internet.

  • Available cloud infrastructure.

  • Plenty of computing power.

Crisis-response teams cannot always rely on any of them.

Field teams may work in areas affected by:

  • Natural disasters

  • Infrastructure failure

  • Conflict

  • Remote geography

  • Network outages

  • Limited power

When communication infrastructure fails, cloud-based tools become unavailable at exactly the moment teams may need information most.

The challenge was to prove that a useful language model could run entirely locally on compact hardware.

02

What needed to work

The prototype had to operate as a self-contained system.

No external internet connection.

No cloud inference.

No dedicated data centre.

No complicated setup for the person using it.

A responder with a phone, tablet or laptop should be able to connect directly to the device and begin using the AI assistant.

That meant solving several problems together:

  • Local AI inference

  • Compact hardware

  • Local networking

  • Browser access

  • Model optimisation

  • Power efficiency

  • Simple field interaction

03

What we built

A portable offline AI node.

WeAreBrain built the first operational offline AI prototype for the crisis-response venture.

At its core is an Orange Pi 5 Max, a compact single-board computer using the RK3588 processor and its integrated neural processing unit.

The language model runs directly on the device.

The hardware creates its own Wi-Fi network.

Users connect with an ordinary phone, tablet or laptop.

A browser-based interface gives them access to the AI assistant.

Everything happens locally.

04

The architecture

One compact device. No cloud dependency.

The prototype brings the full interaction onto one local system.

Hardware

Orange Pi 5 Max

AI processing

RK3588 NPU

Operating system

Armbian Linux

Model runtime

rkllm

Language model

Qwen2.5-3B-Instruct

Interface

Open WebUI

Connectivity

Local Wi-Fi access point

Storage

Local model and operational storage

The device therefore provides its own compute, model, network and interface.

1. Local AI inference

Run the model where the user is.

The language model executes directly on the device rather than sending prompts to an external API.

We selected Qwen2.5-3B-Instruct, optimised for the available hardware.

The model was chosen to balance:

  • Response quality

  • Memory requirements

  • Inference speed

  • Hardware limitations

The prototype demonstrates that useful language-model inference does not always require large cloud infrastructure.

2. Hardware-aware optimisation

Running a language model on compact hardware creates different engineering constraints from running it in the cloud.

Memory is limited.

Compute is limited.

Power availability may be limited.

The model and runtime therefore need to match the device.

The prototype uses the integrated RK3588 NPU and the rkllm runtime to make local inference practical on the available hardware.

This turns model choice into an architecture decision rather than a simple API selection.

3. Its own local network

The device becomes the infrastructure.

The Orange Pi is configured as a Wi-Fi access point.

Nearby devices can connect directly to it.

That means a responder can use:

  • A smartphone

  • A tablet

  • A laptop

without needing a router, mobile network or external internet connection.

The AI node brings its own local network with it.

4. Automatic access

Make the field experience simple.

Technical capability is not enough in an emergency environment.

People need to reach the tool quickly.

A captive portal automatically directs connected users to the browser-based AI interface.

The user experience becomes:

01 Connect

Join the local Wi-Fi network.

02 Open

The interface becomes available in the browser.

03 Ask

Interact with the AI assistant.

No specialised client software is required on the user's device.

5. Fully disconnected operation

Once deployed, the core interaction does not depend on an external service.

The model runs locally.

The interface runs locally.

The network runs locally.

Queries stay within the local environment.

This creates a very different infrastructure model from typical cloud AI.

05

What this proves

Useful AI can move closer to the edge.

The prototype confirmed several important technical assumptions.

Language models can run on compact hardware

The selected model runs on the RK3588 NPU.

Local inference is practical

AI queries can be handled without sending requests to a cloud service.

The entire experience can be self-contained

Compute, network, model and interface can operate as one local unit.

Ordinary devices can access it

Users can interact through a browser on phones, tablets and laptops.

Offline AI can be designed for constrained environments

The architecture can support scenarios where normal cloud assumptions do not apply.

06

Why local AI matters

Offline operation is useful far beyond crisis response.

Crisis environments make the requirement obvious.

But the same architecture raises interesting possibilities for other situations where connectivity, privacy or infrastructure is constrained.

Examples include:

  • Remote industrial sites

  • Ships and offshore environments

  • Field engineering

  • Defence and civil protection

  • Remote healthcare

  • Factories

  • Secure facilities

  • Research stations

  • Temporary operations

  • Sensitive enterprise environments

Local inference can also be useful where sending information to external infrastructure is undesirable.

07

Cloud, private cloud or edge

The right deployment depends on the job.

We do not believe every AI system should run locally.

Cloud models are often the right choice.

They provide access to powerful models, scalable infrastructure and rapid development.

Local AI becomes interesting when requirements change.

Cloud

Best where connectivity and scale matter most.

Private infrastructure

Useful where organisations need stronger control over data and deployment.

Edge and offline

Useful where connectivity cannot be assumed or processing needs to happen close to the user.

Good AI architecture starts by understanding those constraints.

08

Small models can solve useful problems

The AI industry often focuses on increasingly large models.

Edge deployment asks a different question:

What is the smallest model that can do the job well enough?

For a field system, a smaller model can bring advantages:

  • Lower compute requirements

  • Lower power requirements

  • Faster local responses

  • Simpler deployment

  • No network latency

  • Greater operational independence

The model does not need to know everything.

It needs to be useful for the task it was deployed to support.

09

The next step

From technical prototype to field-ready system.

The first prototype establishes technical feasibility.

A production crisis-response system would require additional work around areas such as:

  • Curated emergency knowledge

  • Local document retrieval

  • Multilingual support

  • Field testing

  • Power management

  • Hardware protection

  • Device management

  • Security

  • Updates

  • User permissions

  • Evaluation

  • Operational protocols

These are the steps that turn a working prototype into dependable field infrastructure.

10

The result

A working AI assistant without internet or cloud services.

The project demonstrated that a language model can operate locally on compact hardware and remain accessible through an ordinary browser.

Fully local inference

Queries can be processed without calling external AI services.

Self-contained networking

The device creates the network users connect to.

Compact hardware

The complete prototype runs on a single-board computer.

Simple access

Users interact through a familiar browser interface.

Foundation for field pilots

The prototype creates a technical base for testing offline AI in real operational environments.

11

Why this matters

AI infrastructure is becoming more flexible.

For the first generation of generative AI products, the standard architecture was simple:

  • User

  • โ†’ Internet

  • โ†’ API

  • โ†’ Large cloud model

That is no longer the only option.

Models can increasingly run:

  • In private clouds

  • On enterprise hardware

  • On workstations

  • At the edge

  • On compact devices

That gives organisations more choices around:

  • Connectivity

  • Latency

  • Privacy

  • Cost

  • Resilience

  • Control

This prototype explores the far edge of that spectrum.

AI services used

Data Infrastructure

Selecting and configuring the model, runtime, hardware and local infrastructure needed to make offline inference possible.

Software Engineering

Building the network, runtime, browser interface and end-to-end system around the model.

Avatar-1
Avatar-2
Avatar-4
Avatar-4
Avatar-3

Need AI somewhere the cloud cannot reach?

AI architecture does not have to start with an external API. We design systems around the environment they need to operate in, from cloud infrastructure to private deployments and offline edge AI.