Learn
Do local LLMs have guardrails?
Partly – and not the part most teams assume. An open-weight model you pull with Ollama carries refusal training in its weights, so it declines some harmful requests on its own. What it does not carry is a runtime: no moderation service between the model and your application, no policy to enforce, no record of what was asked. Ollama, LM Studio, llama.cpp and vLLM are model runners – none of them ships a content filter of its own.
A hosted API gives you three layers of protection. Self-hosting keeps exactly one. That gap is fillable – the labs that publish the models also publish separate guard models, openly and free – but nothing installs them for you, and what a production application needs reaches well past content classification.
Three layers of safety, and the one that survives
When you call a hosted model, three independent mechanisms stand between a user and a bad outcome. Only the first of them is inside the model, which is why downloading the weights feels like it should be equivalent and is not.
In the weights
Refusal training from the lab’s alignment work. This travels with the file – it is the layer you keep. It is also the layer a fine-tune, a system prompt, or a weight edit can remove.
On the wire
The provider’s own moderation, running on their side before and after your call. Self-hosting deletes this entirely. No local runner replaces it, and no setting turns it back on.
Around the account
Rate limits, abuse detection, retention policy, and the provider’s audit logs. Once the model is yours, all of it is yours – including the obligation to prove what happened.
What the weights actually carry: four examples
Refusal behaviour varies enormously between checkpoints, and the variance is not marketing – it is documented by the labs themselves. Four concrete cases show the range you are choosing between when you pick a local model.
The pattern worth noticing: every lab that takes safety seriously ships its safety as a separate model. That is the clearest available admission that the answering model’s own weights were never meant to be the enforcement boundary.
Llama (Meta)
Safety-tuned, and shipped alongside dedicated guard models – Llama Guard and Prompt Guard – which Meta positions as the filter for inputs and outputs. The guard is a separate download you must run yourself.
Mistral 7B Instruct
The model card states plainly that it has no moderation mechanisms, and invites the community to help add guardrails for deployments needing moderated output. Unusually honest, and unchanged across v0.1 to v0.3.
Gemma (Google)
Safety-tuned, with ShieldGemma published separately as the content classifier. As with Llama, the safety model is a distinct artifact on a distinct release track.
DeepSeek R1
Cisco researchers ran 50 HarmBench prompts against it and reported a 100% attack success rate – not one prompt refused. Their reading: cost-efficient reasoning training came at the expense of safety training.
What Ollama and LM Studio actually ship
The runner is the layer people most often mistake for a safety layer, because it is the thing with the interface. It is not one. Neither Ollama nor LM Studio inspects a prompt, scores a response, or blocks anything – they load a checkpoint, apply its chat template, and stream tokens back. llama.cpp and vLLM are the same story with fewer buttons. Whatever refusal you observe came from the weights.
What they do give you is a system prompt slot – Ollama’s SYSTEM instruction in a Modelfile, LM Studio’s per-model preset. Useful for tone and task framing, and worth setting. But a system prompt is an instruction to a probabilistic model, not an enforcement point: it sits in the same context window as the user’s text, competing for attention with it, which is the entire reason prompt injection works.
The gap that surprises teams is access control. Ollama binds to localhost by default and ships no authentication at all – there is an open feature request for a “secure mode” with auth. Point __OLLAMA_HOST__ at 0.0.0.0 to reach it from another machine and the API is simply open, including the endpoints that pull and create models. Cisco researchers scanning Shodan found 1,139 exposed Ollama instances, 214 of them serving live models, all answering prompts without credentials. An earlier path-traversal flaw in the pull endpoint (CVE-2024-37032) reached remote code execution. LM Studio also binds locally by default and does ship optional API tokens you can switch on in server settings – off unless you enable them.
Both tools do distribute the guard models: llama-guard3, shieldgemma and granite3-guardian are all one pull command away in Ollama’s own library. None of them runs unless you build the plumbing that calls it. The safety is in the catalogue, not in the path your request takes.
Content filtering
None, in any of them. No moderation endpoint, no policy file, no blocklist. The model refuses what its training makes it refuse, and the runner passes the result through untouched.
Authentication
Ollama has no auth mechanism; LM Studio has opt-in tokens. Both listen on localhost until you change it – and changing it is exactly what teams do when they move the model to a shared box.
Audit trail
Logs tell you a request happened, not what was in it, whether it was risky, or what the model answered. Nothing is retained for review by default, which is a problem the first time somebody asks what an agent did.
The guard models you can add yourself
The missing layer is genuinely available offline, and mostly from the same labs and the same catalogues. These are ordinary open-weight models: you run one next to your answering model, feed it the prompt and the response, and get back a safe/unsafe verdict with categories.
The cost is architectural, not financial. Each check is a second model call on your own hardware, and you have to write the code that calls it, decides what a verdict means, and records the outcome. A classifier that returns “unsafe” to nobody is not a guardrail.
Llama Guard 4 (12B)
Meta’s current guard – text and image, multilingual, folding in what were two separate Llama Guard 3 models. Classifies against 14 hazard categories, S1 to S14.
Prompt Guard 2 (22M / 86M)
A jailbreak and injection detector, not a content filter. Tiny enough to run on CPU in front of every request, which is exactly how it is meant to be used.
ShieldGemma (2B / 9B / 27B)
Google’s classifier over four harm categories – sexually explicit, dangerous content, hate, harassment. ShieldGemma 2 rebuilds it on Gemma 3 for image safety.
Qwen3Guard (0.6B / 4B / 8B)
Three-tier verdicts – safe, controversial, unsafe – across 119 languages. The Stream variant classifies token by token, so it can intervene mid-generation.
Granite Guardian 3.x (2B / 8B)
IBM’s Apache-2.0 guard. Beyond harm categories it scores RAG groundedness, which puts it closer to a hallucination check than the others get.
WildGuard
An open classifier that answers three questions at once: was the request harmful, did the model refuse, was the response harmful. The refusal signal is what makes it good at scoring safe refusals.
Uncensored and abliterated models
A large share of models on public hubs have had their refusal behaviour deliberately removed. The dominant technique, abliteration, computes the direction in activation space that separates harmful from harmless prompts and projects it out of the weights – no retraining, no fine-tuning dataset, no jailbreak prompt. Research from 2024 framed refusal as a single direction; later work argues it is more than one, and separates detecting harm from executing the refusal.
This matters even if you would never run such a model on purpose. It means a checkpoint’s safety behaviour cannot be inferred from its family name: a file whose name still says “llama-3-8b-instruct” may have had its refusals ablated by a third party. Pin what you serve by digest, and test the artifact you actually loaded rather than the model card you read.
There are legitimate reasons these exist – red-teaming, safety research, and domains where refusal training over-blocks constantly, like offensive-security tooling and clinical text. The conclusion is not that abliteration is illegitimate. It is that refusal in the weights is a preference, not a control: reversible by anyone with the file, which is everyone.
The checks a guard model still does not give you
Content classification answers one question – is this text harmful. Most of what goes wrong with a local deployment is not harmful text. It is a confidently wrong figure in a summary, JSON that breaks the parser at 2 a.m., an agent calling a tool it should not have, a run that loops until the GPU hours are gone, and no trail that explains any of it afterwards.
Running the model locally solves data residency, which is usually why teams self-host in the first place. It does nothing for correctness and nothing for agency – if anything it raises the stakes, because the checkpoint answering your questions has had less safety work than the hosted model it replaced, and there is now no provider between you and the output.
That is the layer in/guard/out runs: grounding checks that catch altered figures, format enforcement, content safety backed by these same local shields, and agent controls over tools, permissions and budgets – all on the wire, so the checks are identical whether the model is OpenAI, Claude, or LM Studio on localhost.
Frequently asked questions
Do local LLMs have content filters?
Not as a runtime layer. The weights carry refusal training, which is why a local model declines some requests, but Ollama, LM Studio and llama.cpp add no filtering of their own – there is no moderation endpoint, no policy, and no log unless you build one.
Does Ollama censor responses?
Ollama itself censors nothing. Any refusal you see comes from the model’s weights, and it varies by checkpoint – some models on the registry are explicitly uncensored builds with their refusal behaviour removed.
Is a local LLM safer than an API because the data stays on my machine?
Privacy is not safety. Self-hosting solves data residency and removes a third party from the path. It does not make the answers correct, the JSON valid, or an agent’s tool calls appropriate – and it removes the provider’s moderation layer you were previously getting for free.
Which local guardrail model should I run?
Pair a content classifier with an injection detector – they solve different problems and a safety model is not an injection detector. Llama Guard 4, ShieldGemma, Qwen3Guard and Granite Guardian cover content; Prompt Guard 2 covers injection at a fraction of the size.
Does Ollama have authentication?
No. It binds to localhost by default and ships no auth mechanism, so exposing it on a network exposes the API – including model pull and create. Cisco researchers found over a thousand unauthenticated Ollama servers reachable from the internet. LM Studio offers optional API tokens; they are off unless you enable them.
Can a system prompt act as a guardrail?
Only as a preference. A Modelfile SYSTEM instruction or an LM Studio preset shares the context window with the user’s text and competes with it for attention, which is why prompt injection works at all. Use it to set behaviour; do not use it to enforce anything that matters.
Are abliterated models safe to use in production?
They have had their refusal behaviour removed by design, so anything you need enforced has to be enforced outside the model. That is true of every model to some degree, but with an abliterated checkpoint there is no fallback if your external layer misses.
Keep reading
See the checks run on your own traffic.
One base-URL change puts every request behind the guardrail pipeline. We’re running a limited demo – sign up and we’ll get you in as soon as we can.