NOMDAD
Discuss a project
§ LOCAL LLMs

When your data cannot leave the building

6 min read ← All entries

Health records, case files, personnel data. Sometimes the constraint is not a preference — it is the law. Local models make that workable.

01

Private by architecture, not by promise

A contractual guarantee that a provider will not train on your data is a promise you have to audit. A model running on hardware you control is a property of the network diagram. For categories like health data under Article 9 GDPR, the second is a materially easier conversation with a supervisory authority than the first.

02

The quality gap narrowed

Open-weight models will not beat the largest hosted ones on open-ended reasoning. For the tasks most organisations actually need — extract these fields, classify this document, answer from these twelve pages — they are close enough that the difference is no longer the deciding factor. Retrieval quality moves the result far more than model size does.

03

Every answer needs a source

In a regulated setting an unsourced answer is not an answer. Every output should carry the passage it came from, so a human can check it in seconds rather than trusting it. This is also the cheapest safety mechanism available: people catch a wrong citation immediately, and they never catch a confident paragraph.

04

What it costs to run

One workstation-class GPU handles a department. The real cost is not the card, it is operations: updates, monitoring, someone who owns it when it stops. Compare against per-token pricing at your actual volume before deciding, and count the staff time honestly on both sides.

05

A sane starting point

Pick one document set and one question people ask of it repeatedly. Build retrieval over that set, put a local model behind it, and measure against answers you already know are correct. You will learn more in two weeks than in six months of vendor evaluation.

◆

Working on something like this?

Tell us what you are building. We will tell you honestly whether we are the right studio for it.

Related service: Local LLMs

→

Next entry

Why your AI pilot never shipped

Start a project → ← All entries