Context aware human approval for AI assistants

Sat, 25 Apr 2026 09:19:02 +0200

In order to do something meaningful, an agent needs to interact with the world using other systems. If an generative artificial intelligence (GenAI) is given the option to do so, that is called “using a tool” or more technical: tool calling or function calling. For this some control is needed to ensure the tool is used when and how it was intended. In combination with a restrictive environment you can get a good fundamental level of protection.

TL;DR

AI applications ask for permissions. Over time those grants accumulate towards execution patterns that are frequently used. The fact that a granted permission was good once does not necessarily make it acceptable next time. If permissions are given with context, time constraint and impact limit, the intention is much clearer. In the Relagent permission model a grant does not only have a session or global scope but also a maximum sensitivity limit. Since privacy is a primary concern for this agent, tying the grant to the potential negative outcome of loosing personal data gives the user a realistic change to consider the trade offs.

A huge part why AI agents are are so useful, is their access to tools: Their connection to the world. As most tools, their use can be leading to something good or bad. The issue is not the tool or the agent alone, it’s how both work together.

Why a sandbox is not enough

A fundamental security principle to keep systems safe is to only give access that is needed (see Principle of least privilege). For example does a web research require internet access, but should not be able to use your local camera. A climate control system in contrast should not be allowed to freely access the internet. Set’s say you isolate your agent and tools good enough so that they can do what they are supposed to do, and not more. Which is a hard enough task in itself. You still can not predict what information will go through the allowed channels. Does private information leak to web form your agent tries to fill out for you? Does some malicious text found on the internet end up in the email the agent is composing for you? A restrictive box around your agents does not know, it operates on the wrong layer.

What do you sign off with a stamp of approval?

An agent asking you to allow executing a specific tool or command is a very common picture. And for you it is not too difficult to choose yes or no if you followed the conversation that lead to that approval question. You might ask yourself something like:

Does the action make sense?
Does it seem safe?
Is it reasonable? And often you can almost intuitively answer yes and approve the action.

To avoid slowing down productivity usually there is the option to approve now and not ask again. Then the approval remains valid for a longer time, maybe even during future sessions. But:

Is your given approval still that simple if you don’t know the context? If you have no idea what the agent tries to accomplish?

The agent does not care, your approval remains valid, your intentions are no longer relevant.

Security questions are annoying, they need attention, they slow you down. Still that should not push you to avoiding them completely.

Taking the risk and context into consideration

I see several concrete aspects that would make a given approval more specific and less unpredictable.

Binding to a context: A chat session, a topic, branch or project. That is commonly already the case.
Constraining time: The user might be confident for the actions that are foreseeable right now. But not forever.
Impact limit: Adding a constraint like a “only if …”. For example only if the monthly costs are within 10,- or as long as no personal data is shared.

With such attributes in place a user has a better way formalizing the intent. For example the agent then can use the web as long as it does not share sensitive data. Or it could access memory, but only related to a specific project.

That is the condensed data model used in Relagent Engine:

class Grant(BaseModel, frozen=True):
    approval_type: ApprovalType = ApprovalType.OutgoingData
    component: str | None = "web_search"
    allowed_parameters: dict[str, str | int | bool] = {}
    wildcard_parameter: str | None = None
    max_sensitivity: SensitivityLevel = SensitivityLevel.OpenInformation,
    expires_at: datetime | None = None

In the user interface it looks like this:

You still need isolation on system level

Given you get a really good permission setup that limits the tool usage exactly to what you intent to do. And you manage to separate data source so that trusted and unknown origins do not mix. Which is a non-trivial problem. There still are a lot of unplanned things that can happen outside the how and when tools are called. Your tools might do more than they advertise (hidden features, supply chain attacks). Functionality can change over time (updates, cloud service). Software has bugs (that won’t change any time soon). And of course someone might put effort into manipulating your system by feeding it malicious data. Having guards that prevent your system from doing actions you never intended it to do, might be the last defense to stop such bad actions.

permissions on Venkado Blog

Context aware human approval for AI assistants

What do you sign off with a stamp of approval?

Taking the risk and context into consideration