Aligning the model was never going to govern it

Aligning the Model: Beyond Governance and Safety

Skip to content

Navigation

• News
• Events
  • TNW Conference
    • June 19 & 20, 2025
    • All events
• Newsletters
• Partner with us
• Jobs
• Contact

Latest

• Deep tech
• Sustainability
• Ecosystems
• Data and security
• Fintech and ecommerce
• Future of work

Conference Media Hub

• More
• Startups and technology
• Investors and funding
• Government and policy
• Corporates and innovation
• Podcast
• Artificial Intelligence

---

Aligning the model was never going to govern it

July 30, 2026 – 6:47 am

Most AI roadmaps rest on a quiet bet: that the labs will eventually ship a model safe and aligned enough to simply trust in production. Better training, better guardrails, one more version, and the thing behaves.

Here is the flaw in that bet. Even a perfectly aligned model cannot tell you who used it, on what data, under whose policy, or hand you a record you could show an auditor. Those are not facts about how the model behaves; they are facts about how it was deployed, and they live entirely outside the weights. A better model answers a different question than the one regulators, auditors, and security teams are actually asking, and the gap between those two questions is where the real risk now sits.

Security engineering named this trap fifty years ago. A 1972 US Air Force study defined the reference monitor, the component that decides whether an action is allowed, and set three conditions for trusting it: it must be tamperproof, invoked on every access, and small enough to be completely verified.

The State of EU Tech

The 💜 of EU tech. The latest rumblings from the EU tech scene, a story from our wise ol’ founder Boris, and some questionable AI art. It’s free, every week, in your inbox. Sign up now!

Governance vs. Safety

The confusion hides in one word. Safety and alignment ask whether a model tends to behave well—that is, a disposition that lives in the weights, and the labs have gotten genuinely good at shaping it. Governance, however, asks something different: who used which model, on what data, under whose policy, and with what auditable record. That is a property of a specific deployment; training can shape the first but cannot, by construction, supply the second. Treating them as one problem is the category error underneath most AI risk conversations.

The Growing Problem

This is not a niche concern for the labs; it lands on everyone who deploys AI, and it is expanding for two reasons:

1. Models are gaining agency: When a model only produced text, a bad interaction meant a bad answer. Now models take actions through tools and protocols like MCP, and every tool call is a fresh decision about who is acting, on what data, under whose authority. Those decisions multiply with every task and every user. OWASP’s 2026 tracking finds that most of the agentic projects it follows are coding agents, and that prompt injection maps to six of the ten risks on its agentic top-ten list.

2. Accountability is now legal: California’s SB 53 and the European Union’s rules for general-purpose AI put obligations on the labs but also place separate duties on deployers, and data-protection law already holds organizations responsible for how personal data is used. When a regulator asks what your AI did, “we used an aligned model” is not an answer, and a vendor’s safety report will not tell them who inside your company sent which data to which model last Tuesday.

Public benchmarks like StrongREJECT, HarmBench, and AgentHarm grade the model on generic prompts, not you.