← All posts

What If AI Never Needed to Be Trusted?

The problem may not be that AI is unreliable, but that we keep giving unreliable intelligence authority.

By Yacine Kellib

There is something strange happening in AI.

Every few months, the models become more capable.

They reason better.

They write better code.

They understand more context.

They use tools.

They browse.

They remember.

They operate computers.

They can run workflows for hours without human intervention.

And our response is predictable:

Give them more access.

Then we build another layer to make sure they don't misuse it.

Guardrails.

Permissions.

Classifiers.

Human approvals.

Runtime monitoring.

Identity.

Sandboxing.

Kill switches.

More policy.

More detection.

More controls.

It all makes sense.

But perhaps we are starting from the wrong question.


What if the model doesn't need to be trusted?

Imagine an employee sitting in front of a banking system.

The employee says:

"I want to transfer €500,000."

That's an intention.

It is not authority.

The employee might need approval.

They might have a spending limit.

The beneficiary might need to be verified.

The transaction might need to comply with company policy.

Perhaps another person needs to authorize it.

Nobody finds this strange.

We have spent decades designing systems around this distinction.

Knowing what should happen is not the same thing as being allowed to make it happen.

Yet with AI, we are increasingly collapsing these two things.

The model reasons about an action.

The agent decides to perform it.

The agent calls the API.

The system changes.

The world changes.

Intelligence has quietly become authority.

Why?


The LLM is not the problem

Let's make one thing clear.

This isn't an argument that LLMs are bad.

They aren't.

They are extraordinary machines for producing intelligence.

They can interpret.

They can summarize.

They can reason.

They can explore possibilities.

They can create plans.

They can discover patterns humans might miss.

They can generate software.

They can propose decisions.

Their weakness is not that they are unintelligent.

Their weakness is that they are probabilistic.

They can be wrong.

They can misunderstand.

They can hallucinate.

They can be manipulated.

They can be influenced by malicious information.

They can encounter situations they were never designed for.

And this remains true even as models become dramatically better.

So perhaps the answer isn't:

Make the model perfect.

Perhaps the answer is:

Stop requiring perfection before allowing usefulness.


The industry is already discovering this

This isn't a secret.

Anthropic has increasingly focused on containment because model-level safeguards have a non-zero failure rate (Anthropic, May 2026).

OpenAI is investing heavily in sandboxing, permissions, network controls, identity and agent security (OpenAI, Agent approvals & security).

NIST is working on agent identity and authorization (NIST NCCoE).

And NVIDIA has gone particularly far in articulating the architectural distinction: higher layers can propose actions while lower layers make authoritative decisions (NVIDIA).

The industry is moving toward a realization:

The model cannot be the entire security boundary.

That's important.

But I think there is another step.


Separate the intention from the authority

This is the idea behind ZIFFER.

An AI agent should be able to say:

"I intend to transfer €500,000."

It should not be able to say:

"Therefore I am authorized to transfer €500,000."

Those are two completely different statements.

The first belongs to intelligence.

The second belongs to authority.

So why do we put them in the same system?


A very simple architecture

Think about this:

AI

What should happen?

INTENT

What exactly is being proposed?

AUTHORITY

Is this specific thing allowed?

EXECUTION

Make exactly that authorized thing happen.

That's it.

The AI remains powerful.

But it doesn't own the authority.


And suddenly, something interesting happens

Suppose the AI is manipulated.

An attacker injects instructions into a webpage.

The model reads them.

The model reasons incorrectly.

The model produces:

"Export all customer records."

In a conventional agent architecture, we now hope that one of the many defenses catches the problem.

Maybe the model recognizes the attack.

Maybe the prompt-injection detector catches it.

Maybe the policy engine blocks it.

Maybe the runtime monitor detects suspicious behavior.

Maybe a human sees the approval request.

Maybe the kill switch gets activated.

That's a lot of maybes.

ZIFFER asks a simpler question:

Does the agent have the authority to export the records?

If not, the malicious intention remains just that.

An intention.

It doesn't become an action.


This changes the security equation

We have traditionally tried to make:

P(error)

as small as possible.

That's obviously valuable.

But it may never become zero.

ZIFFER adds another objective:

P(error) should not automatically imply P(unauthorized consequence).

That is a different engineering philosophy.

We don't need to prove that the intelligence will never make a mistake.

We need to ensure that a mistake cannot automatically acquire authority.


But wait.

What about the human?

Surely we can just ask the human.

Every time the agent wants to do something important:

Approve?

Problem solved.

Except...

What happens when the agent asks 500 times?

Or 5,000?

Or 50,000?

This isn't hypothetical.

Anthropic reported that users approve approximately 93% of Claude Code permission prompts (Anthropic, March 2026).

And Anthropic explicitly identified the resulting problem:

approval fatigue.

The human is still there.

The approval button still exists.

The security control technically still works.

But something has changed.

The human is no longer meaningfully evaluating every decision.


The approval button can become a rubber stamp

This creates an uncomfortable paradox.

We introduce human approval because AI cannot be trusted.

Then we give the AI enough autonomy to generate so many approval requests that the human stops carefully reviewing them.

The security mechanism itself begins to degrade.

The more autonomous the system becomes,

the more approvals it generates.

The more approvals it generates,

the less attention each approval receives.

The less attention each approval receives,

the less meaningful human authorization becomes.

And eventually:

human-in-the-loop

can become

human-near-the-loop.

The human is present.

But the human is no longer really the control.


Authority Fatigue

This deserves a name.

Authority Fatigue.

Not because humans are careless.

Because humans are human.

Attention is finite.

Repetition changes behavior.

When 99 requests look harmless, the 100th one gets the same treatment.

The attacker doesn't necessarily need to defeat the authorization mechanism.

They can exploit the fact that the authorization mechanism depends on human attention.

This is an interesting gap in the way we normally think about agent security.

MITRE ATLAS already addresses related concepts such as excessive agency and user execution.

But the specific degradation of authorization caused by repeated autonomous approval requests deserves to be treated as a distinct failure mode.


So perhaps we are asking humans the wrong question

Instead of:

"Should I approve this action?"

over and over again,

perhaps the better question is:

"What authority should this agent have?"

That's a fundamentally different conversation.

Imagine telling an agent:

You may read customer records for this purpose.

You may update customer addresses.

You may issue refunds up to €500.

You may not access payment credentials.

You may not export customer data.

Transactions above €10,000 require additional authorization.

The human establishes the boundary.

The system enforces it.

The agent operates inside it.

Now the human doesn't have to become a button.

The human defines authority.


More autonomy without more authority

This is where the idea gets interesting.

The conventional trajectory looks like this:

More capable AI

More useful agent

More tools

More permissions

More autonomy

ZIFFER proposes:

More capable AI

More useful intelligence

Better intentions

Independent authorization

Controlled execution

The model becomes smarter.

The authority doesn't automatically grow with it.

That means we can potentially have:

More autonomy without more inherent authority.


This is not about making AI weak

Quite the opposite.

Imagine a future model that is dramatically more capable than today's systems.

It can reason over millions of documents.

It can design software.

It can coordinate hundreds of agents.

It can discover strategies humans didn't anticipate.

Why would we want to make that intelligence weaker?

We shouldn't.

We should make the consequences controllable.

Let the intelligence explore.

Let it propose.

Let it reason.

Let it surprise us.

But when the proposal reaches the boundary between software and reality:

authority takes over.


The biological lesson

This is where biology gives us an interesting model.

A cell doesn't assume every molecule entering it is trustworthy.

It has a membrane.

It has receptors.

It controls what crosses the boundary.

It separates compartments.

It regulates resources.

It contains damage.

The cell doesn't need to understand the moral character of every molecule.

It controls what that molecule can cause.

Perhaps autonomous software should work the same way.

We don't need every signal entering an AI system to be trustworthy.

We need to ensure that:

A signal can influence intent without automatically acquiring authority.

That is a very different way of thinking about AI security.


Trust is not the right primitive

This may be the deeper idea.

We keep asking:

Can we trust the AI?

Maybe the better question is:

Why does the AI need to be trusted with authority?

An operating system doesn't need to trust every application.

A banking system doesn't need to trust every employee.

A network doesn't need to trust every packet.

A secure system establishes boundaries.

AI should be no different.


Intelligence can be uncertain

Authority cannot.

The model might say:

"I think this is the right action."

Fine.

The model might be wrong.

Also fine.

The authorization system asks:

"Is this action permitted?"

That decision must be governed by something other than the model's confidence.

Because confidence is not authority.

Reasoning is not authority.

Capability is not authority.

Identity is not authority.

Intent is not authority.


The four things we should stop conflating

INTELLIGENCE

What the system can understand and reason about.

INTENT

What the system proposes should happen.

AUTHORITY

What the system is permitted to cause.

EXECUTION

What actually happens.

These are four different properties.

Why should they live in one place?


A new way to consume AI

This may ultimately be bigger than AI security.

It changes the way we consume AI.

Today we increasingly say:

"Let the AI do it."

ZIFFER suggests:

"Let the AI think about it."

Then:

Let the system decide whether it is allowed.

This doesn't reduce the usefulness of AI.

It changes the relationship between intelligence and consequence.

AI becomes a source of intelligence.

Authority becomes infrastructure.

Execution becomes controlled.


What happens to trust?

Trust doesn't disappear.

It moves.

Instead of asking:

"Do I trust this model?"

we ask:

"Do I trust the authority system that controls what this model can cause?"

That's a much better question.

Because models can be replaced.

A model can be upgraded.

A model can be fine-tuned.

A model can be compromised.

A model can be retired.

The authority architecture can remain.


This could change how enterprises adopt AI

Imagine an enterprise that wants to use five different models.

OpenAI.

Anthropic.

Mistral.

An open-source model.

A specialized internal model.

Today, each model and agent framework may introduce different assumptions about permissions, tools, memory and execution.

What if the enterprise instead had a common authority layer?

Then the model becomes replaceable.

The intelligence becomes modular.

The authority remains organizational.

The enterprise could say:

We don't need to trust every model equally.

We need to know:

Whatever model we deploy, it cannot exceed the authority boundary.

That's a very different enterprise AI strategy.


The real architectural boundary

Perhaps the most important security boundary in an agentic system isn't:

Model ↔ Model

or

User ↔ Agent

or even:

Agent ↔ Tool

It may be:

Intent ↔ Authority

That is the point where possibility becomes consequence.

Before the boundary:

Thinking.

After the boundary:

Power.

And those two things should not be casually coupled.


ZIFFER

This is what ZIFFER proposes.

Not another attempt to make the model perfect.

Not another prompt-injection detector.

Not another AI firewall.

Not another monitoring dashboard.

Not another permission popup.

A foundational separation:

INTELLIGENCE

INTENT

AUTHORITY

EXECUTION

The agent proposes.

The authority layer decides.

The executor verifies.

The system remembers.


The question for the industry

The AI industry is spending extraordinary amounts of money making models more capable.

It should.

The world will benefit from better intelligence.

But perhaps we should stop assuming that:

More intelligence must eventually mean more authority.

Perhaps the two should evolve independently.

Perhaps the safest autonomous system isn't the one in which we finally trust the AI enough to give it control.

Perhaps it is the one in which:

The AI never needed control to begin with.


ZIFFER Doctrine

Intelligence proposes.

Intent describes.

Authority decides.

Execution verifies.

Evidence remembers.

AI can be autonomous without being sovereign.


One final question

If we already know that AI can be wrong...

If we already know it can be manipulated...

If we already know that no guardrail is perfect...

If we already know humans cannot meaningfully approve an unlimited number of actions...

Why are we still trying to make the intelligence trustworthy enough to possess authority?

Maybe the next generation of AI architecture should ask a different question:

How powerful can intelligence become if we never make it the authority?

That is the question ZIFFER is asking.

Sources

← All posts