When your data cannot leave the building
6 min read ← All entries
Health records, case files, personnel data. Sometimes the constraint is not a preference — it is the law. Local models make that workable.
Private by architecture, not by promise
A contractual guarantee that a provider will not train on your data is a promise you have to audit. A model running on hardware you control is a property of the network diagram. For categories like health data under Article 9 GDPR, the second is a materially easier conversation with a supervisory authority than the first.
The quality gap narrowed
Open-weight models will not beat the largest hosted ones on open-ended reasoning. For the tasks most organisations actually need — extract these fields, classify this document, answer from these twelve pages — they are close enough that the difference is no longer the deciding factor. Retrieval quality moves the result far more than model size does.
Every answer needs a source
In a regulated setting an unsourced answer is not an answer. Every output should carry the passage it came from, so a human can check it in seconds rather than trusting it. This is also the cheapest safety mechanism available: people catch a wrong citation immediately, and they never catch a confident paragraph.
What it costs to run
One workstation-class GPU handles a department. The real cost is not the card, it is operations: updates, monitoring, someone who owns it when it stops. Compare against per-token pricing at your actual volume before deciding, and count the staff time honestly on both sides.
A sane starting point
Pick one document set and one question people ask of it repeatedly. Build retrieval over that set, put a local model behind it, and measure against answers you already know are correct. You will learn more in two weeks than in six months of vendor evaluation.
Working on something like this?
Tell us what you are building. We will tell you honestly whether we are the right studio for it.
Related service: Local LLMs