Who runs the OAuth flow decides your network topology

An agent that answers questions over governed data has a problem a chatbot doesn’t. Query the warehouse as a service account and it sees everything that account sees. A row policy attached to a person stops applying the moment a robot does the reading. So every query has to run as the actual person asking. Then the warehouse’s own RBAC and policy engine decide what comes back.

Which means a per-user OAuth token has to reach the data tool. That sounds solved until you look at the pieces. An agent on a managed runtime has no interactive way to run an OAuth flow. The enterprise IdP is an on-prem PingFederate that predates all of this. And the whole thing lives inside a VPC Service Controls perimeter whose purpose is to stop traffic leaving.

This post is about the parts of that which are in nobody’s tutorial. Which process performs the token-code exchange decides your network topology. Tool discovery runs at a phase where the per-user token has not been plumbed anywhere the listing request can see it, and the agent ends up with zero tools. One configuration field decides the audience of the minted token, which decides how many hops the design survives. And a set of failures authenticate perfectly and return nothing at all.

The stack here is Google’s Agent Development Kit on Vertex AI Agent Engine, fronted by Gemini Enterprise, talking to a Snowflake MCP server. The shape transfers to any managed platform that owns OAuth on your behalf.

Who performs the exchange decides your topology

There are two places the OAuth authorization-code exchange can happen. The agent framework can drive it, holding the client secret and calling the IdP’s token endpoint. Or the platform in front of the agent can own the whole lifecycle: run the flow, hold the secret, manage refresh, and inject the per-user token into the agent’s invocation state.

Framework-driven looks simpler in a diagram. But in a regulated network it is much harder, and the reason is the perimeter. Google’s Agent Engine documentation is explicit that a Private Service Connect interface is required to get any internet access under VPC-SC. Even then the VPC provides no outbound path to the public internet by default. You configure an egress path deliberately, usually a proxy VM with an RFC 1918 address.

So an agent that performs the token-code exchange needs to reach the IdP’s token endpoint, outside the perimeter. That hole has to be requested, justified, scoped to a host, reviewed, and then maintained by someone after you leave. It exists because the OAuth client landed in the wrong process.

If the platform performs the exchange, it is already inside the perimeter. The agent never contacts the IdP. Its only outbound need is the private path to the data source you needed regardless.

flowchart LR
  U[User consents in platform] --> GE{{Platform owns OAuth:<br/>code exchange, secret, refresh}}
  GE -->|per-user access token<br/>injected into invocation state| AE[Agent runtime]
  AE --> T[Function tool reads token]
  T -->|Bearer token over PrivateLink| MCP[MCP server at data source]
  MCP --> DB[(Warehouse RBAC +<br/>row/column policy)]
  GE -.->|inside the perimeter,<br/>no egress hole needed| P[/VPC-SC perimeter/]
  style GE fill:#0f766e22,stroke:#0f766e,stroke-width:2px
  style T fill:#0f766e22,stroke:#0f766e,stroke-width:2px

The agent becomes a thin identity-passing bridge. Its configuration contains no client id, no client secret and no OAuth endpoint. That absence is the tell that you split the system correctly. The only auth value it needs is the identifier of the authorization resource whose token it should look for.

Client secrets leak, expire without warning, and have to be rotated by someone with permissions you won’t have once the engagement ends. Holding none of them is a smaller thing to hand over.

The tool-listing trap

Now the failure that costs the most time. It presents as the model being stupid instead of as an auth problem. Agent frameworks discover MCP tools by opening a session to the MCP server and listing what it offers. In ADK that is McpToolset, and it calls get_tools() during setup.

Here is the part I got wrong the first time. I assumed the per-user token simply did not exist yet at listing time. It does. In current ADK, get_tools() runs inside the invocation, from _preprocess_async, and it is handed a ReadonlyContext built from the invocation context. That context’s state is readable, a MappingProxyType over the session state. By the time preprocessing runs the platform has already injected the token there.

What went wrong in ADK 1.x is narrower. The MCP client session that McpToolset opened to list tools did not carry that token in its headers. The listing request went out unauthenticated, the server returned 401, and the toolset came back with an empty list. The wrong mental model sends you debugging the wrong layer.

Then the agent, holding zero tools but still asked a data question, does what models do with an impossible instruction. It invents plausible tool names and calls them. The symptom is hallucinated tool calls. The cause is a 401 two layers away. Nothing in the trace prints “401” anywhere near the model’s confused output.

Current ADK gives you a direct fix for the discovery session. McpToolset now takes a header_provider, a callable handed the same ReadonlyContext, whose return value is merged into the headers before list_tools runs. Read the token out of readonly_context.state. Set Authorization on the listing request. Discovery authenticates. If your server enforces tenant isolation at discovery, that is the mechanism you want.

For per-user pass-through I still wouldn’t list tools dynamically. A phase mismatch remains that no header hook removes. Discovery answers “what tools exist” once, early, but per-user auth answers “who is asking” late, per invocation. Binding the inventory to one user’s token makes the tool list a function of who is in the loop.

Declare one function tool statically instead. Listing it requires no session and no auth. The tool reads the token and opens its own authenticated MCP session at execution time, when the token is unambiguously present.

from google.adk.agents import LlmAgent as _LlmAgent
from google.adk.tools import ToolContext
from mcp import ClientSession
from mcp.client.streamable_http import streamablehttp_client

# A deployed runtime can be older than your local ADK and reference LlmAgent
# fields your locally built object never set. Subclassing with the newer
# defaults keeps an older deployed runtime from tripping over the gap.
class LlmAgent(_LlmAgent):
    mode: str | None = None
    wait_for_output: bool = False
    parallel_worker: bool | None = None

root_agent = LlmAgent(
    name="data_agent",
    model=GEMINI_MODEL,
    instruction=(
        "You are a data analyst. DESCRIBE a table before writing "
        "column-specific SQL, fully-qualify names, run one statement per call, "
        "and never ask the user for a token."
    ),
    tools=[run_sql],   # one static function tool; no McpToolset, no discovery
)

Keep one test as the regression, because it encodes the lesson in a single assertion. The agent must expose its tool before anyone has authorized. An empty tool inventory for an unauthenticated caller means you have rebuilt the trap.

Where the token actually lands

The platform injects the token into invocation state under a key derived from the authorization resource’s id. That key is not stable across versions of the runtime and the platform. I have watched the same token arrive under the bare id, under a temp:-prefixed variant, and under a decorated key that merely contains the id. The token is present. Your lookup misses it. The user gets asked to consent again by a system that already holds their consent. So the lookup tries the shapes it has seen, in order, then falls back to a scan. I never did work out what decides which shape you get.

AUTH_ID = os.environ["AUTH_RESOURCE_ID"]   # the only auth value the agent holds

def _read_injected_token(state) -> str | None:
    """Find the platform-injected per-user token in invocation state."""
    if not AUTH_ID or state is None:
        return None
    for key in (AUTH_ID, f"temp:{AUTH_ID}"):       # bare key, then temp: variant
        val = state.get(key)
        if isinstance(val, str) and val:
            return val
    # last resort: a contains-match scan. ADK's State has get(), __getitem__(),
    # __contains__() and to_dict(), but no items()/keys()/__iter__(). Iterating
    # it directly raises AttributeError; to_dict() is how you enumerate it.
    for key, val in state.to_dict().items():
        if AUTH_ID in str(key) and isinstance(val, str) and val:
            return val
    return None

That to_dict() is the detail the API only teaches by use. An earlier version of this function looped over state.items(). It reads fine and it passes review. But it raises AttributeError the first time the decorated-key branch runs, which is exactly the branch you wrote it for. State is not a dict and not a Mapping. It exposes reads and a to_dict(). Returning None instead of raising is deliberate: an absent token is a normal state with a friendly answer.

The wrapper that never raises

The tool opens a fresh authenticated MCP session per call and runs one statement. A couple of claims about it are load-bearing, and both have to be visible in the code or they are assertions.

def _err(msg: str) -> dict:
    return {"status": "error", "error_message": msg}

def _concat_text(content) -> str:
    return "".join(b.text for b in content if getattr(b, "type", "") == "text")

async def run_sql(sql: str, tool_context: ToolContext) -> dict:
    """Run one SQL statement as the invoking user. Never raises: every failure
    comes back as a structured error the model can read and act on."""
    if not sql or not sql.strip():
        return _err("sql must be a non-empty statement.")
    if not MCP_URL:
        return _err("data endpoint is not configured.")
    token = _read_injected_token(getattr(tool_context, "state", None))
    if not token:
        return _err("No authorized session. Ask the user to authorize.")

    headers = {"Authorization": f"Bearer {token}"}
    try:
        async with streamablehttp_client(url=MCP_URL, headers=headers,
                                         timeout=MCP_TIMEOUT,
                                         sse_read_timeout=SSE_READ_TIMEOUT) as (r, w, _):
            async with ClientSession(r, w) as session:
                await session.initialize()
                result = await session.call_tool(name=REMOTE_SQL_TOOL,
                                                  arguments={"sql": sql})
    except Exception as exc:            # transport, auth, protocol: keep the turn alive
        return _err(str(exc)[:300])

    # A tool-level failure is a successful CallToolResult with isError=True.
    # Skip this and a policy or auth denial reaches the model as {"status":"ok"}.
    if result.isError:
        return _err(_concat_text(result.content)[:300])
    return {"status": "ok", "result": _concat_text(result.content)}

The first claim is that it never raises. Empty input, missing config, a missing token, and any transport or auth exception all come back as a structured error instead of an exception that kills the turn. The broad except is a deliberate choice for turn survival. But it does hide bugs, and that is a cost I pay knowingly.

The second claim is the result.isError check, and it is the one people leave out. The MCP SDK returns a tool-level failure as a successful CallToolResult with isError=True, not as a raised exception. A downstream authorization failure, a policy denial and a bad statement rejected by the server all arrive as a result object your code will happily read as success. The model is then told the query was fine and narrates past a permission error. In mcp==1.28.1, the version bundled with ADK 1.x, the field is CallToolResult.isError: bool = False.

That line does something else worth noticing. _concat_text(result.content) is text the far side wrote: a policy denial, a masking message, a column name somebody chose years ago. It comes back as a tool result the model reads and acts on. So the error path is an input channel into the model’s context and not only a diagnostic. The 300-character truncation is a blast radius as much as it is log hygiene.

Opening a fresh session per call, with no pooling, is the last deliberate choice. Under pass-through every call carries a different user’s token. A pooled authenticated session would be a correctness bug waiting for your second concurrent user. It also decides where a fan-out costs you. Parallel tool calls in one invocation reuse the single injected token, so the IdP sees one flow however wide the fan-out gets, and the cost lands on session setup.

Three failures that authenticate successfully

The failures worth writing down are the ones where every component reports success and the user gets nothing.

The consent popup that closes before a token exists

Users reported that the consent window flashed and vanished. Nothing surfaced in the agent, because nothing had reached the agent.

The authorization resource registered with the platform carried a bare authorizationUri, the authorization endpoint with no query string. The server-side OAuth config has no separate fields for client_id, scope or response_type. Whatever you put in that URI is the entire authorization request. PingFederate received a request with no client_id and returned invalid_request. The popup closed on the error page faster than anyone could read it. The fix is a fully parameterised URI:

https://<idp-host>/as/authorization.oauth2
  ?client_id=<client-id>
  &redirect_uri=<platform-callback>
  &response_type=code
  &scope=SESSION:ROLE-ANY
  &access_type=offline
  &prompt=consent

Two operational notes that cost an afternoon each. The authorization resource is create-only via POST, so re-running your registration returns a 409 saying it already exists, and you need a PATCH or a delete-and-recreate. And the redirect URI has to be registered on the IdP client as well; a mismatch there fails in the same silent way, with the popup gone before the error is legible.

Authenticated, authorised, and zero rows

The better failure. A token minted correctly, accepted by the warehouse, session established, query runs, returns nothing. No error. An empty result.

The reflex is to blame the role. The scope in that authorization request, SESSION:ROLE-ANY, resolves to the user’s default role, and Snowflake requires that the user have a default role for it to resolve. The roleless case is a real trap. I also described it wrong the first time. A user with no default role doesn’t get an empty result. The session falls back to PUBLIC, PUBLIC has no grant on the table, and Snowflake answers 002003, “object does not exist or not authorized,” refusing to confirm the object even exists to someone not allowed to see it.

An empty result set with no error is a different signature. It is a row access policy that evaluated to false for every row under this user’s role. The user can see the table and the query is valid. But the policy engine filters the rows away one predicate at a time. That is the expected shape of “allowed to run the query, allowed to see none of these particular rows.” It is indistinguishable from an empty table unless you already know a policy is in force.

When permissions behave impossibly, the first thing I run now is CURRENT_ACCOUNT(). More than once the data lived on a different warehouse account from the developer’s home account and roles were being checked on the wrong one.

The token the tenant stopped accepting

Same code, same config, worked yesterday, 401 today. The warehouse rejected the external OAuth token with an audience and key-validation error.

The isolation is the useful part. It is a general method for deciding whether a failure is even yours. A token issued for one account, presented to a different account, was rejected. A personal access token against that same account succeeded, which cleared the network and the account. The same external OAuth token against its home account succeeded, which cleared the token’s shape and signature. And the previous day’s runtime logs showed the identical path returning 200 six times, which made it a regression.

Cheap observations, and together they localise the fault to external-OAuth validation on one tenant. That isn’t your bug and no agent change fixes it. Being able to show that quickly is most of what the ladder is for.

Two things shorten that ladder next time. Snowflake validates the audience against the account URL plus whatever sits in EXTERNAL_OAUTH_AUDIENCE_LIST, so a token is bound to one account by construction and a question spanning two accounts is two consents. And SYSTEM$VERIFY_EXTERNAL_OAUTH_TOKEN takes the access token as an argument and says whether validation passed and, if not, why. It is a SQL function, so you need a session on the account whose validation you are questioning, which is why the personal-access-token rung comes first.

Audience decides how many hops you get

That registered authorizationUri is the only lever you have on the token’s audience, and the audience bounds how far this design travels. RFC 9068 says that when a request carries no resource parameter, the authorization server falls back to a default resource indicator for aud and should infer it from scope. The authorization resource has fields for the client id, the secret, the authorization URI and the token URI. There is nowhere to put resource on the token request; the platform builds that leg. So the audience is a property of the IdP client registration and a scope string, fixed by someone who was not thinking about which tool the model would pick.

The hop count runs into that. The MCP authorization spec requires a server to reject tokens not issued for it, and forbids a server from forwarding the token it received to an upstream API. What happens here is a bearer pass-through. It is legal because the MCP server and the warehouse are the same audience: aud is the account, and the account is what the server is.

Stand a separate MCP deployment in front of the warehouse and a conforming server has to refuse the token the platform just injected. That means a second exchange whose target audience is named in the request. At two hops you can name it up front. At three, the middle hop has to mint, so it needs a client, a secret and a path to the IdP, and the perimeter argument from the top of this post lands on a component you never planned to own.

Debugging a flow you cannot click through

The awkward part of platform-managed OAuth is that the failing step is a browser popup, and you can’t drive a browser popup from a test harness. That was exactly the step that was broken. So the interesting failures were upstream of anything the agent could log.

What worked was a headless harness that reproduces the platform’s chain: the authorization request, the code, the token exchange, then the MCP call. Authenticating over Kerberos/SPNEGO as the developer’s own domain principal removes the browser. One parameter can then be varied at a time against a real IdP. The harness stands in for the platform, so it holds the client secret the agent never sees.

import requests
from requests_negotiate_sspi import HttpNegotiateAuth   # Windows SSPI, no browser
from urllib.parse import urlparse, parse_qs

def fetch_code(authorization_uri: str) -> str:
    """Drive the authorization endpoint headlessly as the current principal.
    A bare endpoint with no client_id returns 400 right here."""
    r = requests.get(authorization_uri, auth=HttpNegotiateAuth(),
                     allow_redirects=False)
    r.raise_for_status()
    return parse_qs(urlparse(r.headers["location"]).query)["code"][0]

def exchange_code(code: str) -> dict:
    r = requests.post(f"https://{IDP_HOST}/as/token.oauth2",
                      data={"grant_type": "authorization_code", "code": code,
                            "redirect_uri": REDIRECT_URI},
                      auth=(CLIENT_ID, CLIENT_SECRET))   # client_secret_basic
    r.raise_for_status()
    return r.json()   # {"access_token": "...", "expires_in": 1799, ...}

With that, bisecting is mechanical. Token exchange against /as/token.oauth2 with client_secret_basic succeeded on its own, the streamable-HTTP MCP transport negotiated and ran on its own, and PrivateLink reachability resolved on its own. Each step passing in isolation localised the defect to the authorization request, the one step the popup was hiding. Then the controlled comparison, changing only the URL:

One variable, two outcomes. The final end-to-end proof still ran on an injected token, because the popup cannot be driven headlessly. Login, token exchange and role resolution were each verified independently. The remaining seam is understood instead of assumed. Say that out loud when you hand the result to someone else.

The instruction is the only guardrail

The agent applies no SQL allow-listing of its own, which I will defend in a moment. Its one lever on behaviour is the instruction, and one line of it earns its place. Requiring the model to DESCRIBE a table before writing column-specific SQL cut tool round-trips across an eleven-question stress set from 41 to 27, and eliminated the failed column-name guesses.

Those are totals from one stress set on one schema. I wouldn’t carry the ratio to a different warehouse without re-running it.

The identity angle is easy to miss. Every call runs as the user, so the schema the model can DESCRIBE is already the user’s governed view. Forcing it to look before it writes means it discovers the columns that person can see. The alternative is guessing at columns their role may not include and generating a query that fails for a reason the model then has to unwind. The guardrail buys cheaper round-trips and it also keeps the model inside the caller’s real grants.

Version pinning is not paranoia here

A deployment detail that reads as fussy until it takes a day from you. The deploy step rebuilds the deployed requirements from the project’s declared dependencies, and the managed runtime resolves them fresh at build time. A >= floor means “whatever is newest when the platform happens to build.” That can be a different program from the one you tested.

A minor-version bump of the agent framework broke the deployed agent with 'OAuth2Auth' object has no attribute 'prompt' at query time, not at deploy time. Everything built, everything registered, but it failed on the first real question. Pinning every dependency with == is what makes the deployed engine the artifact you verified. The LlmAgent subclass from earlier has the same root cause. The thing that runs your agent isn’t the thing you built it against, and the gap resolves at their build time.

Carrying the audit trail

Passing identity through solves access control. It doesn’t by itself produce an audit trail. The record of which agent acted for whom has to be carried deliberately at each hop or it is gone. Pass-through does fix the worst version at the data layer: under a shared service account every query in the warehouse’s history logs as that account, whereas under pass-through the warehouse logs the query as the person, because the person’s token ran it.

RFC 8693 draws a line those logs cannot. Section 1.1 defines impersonation as principal A being given B’s rights and being indistinguishable from B in that context, and delegation as A keeping its own identity while acting for B. Those read like two settings on a dial, but they are two different kinds of record. Judged by that sentence, what pass-through buys is impersonation: the query history has one identity column and the person’s token filled it. That is the price of the row policy working, and it is worth saying to whoever signs off on the audit story.

The act claim is the delegation form. It names the current actor while sub stays the human, and nesting expresses a chain. Section 4.1 has the line that separates having read the RFC from having implemented it: the consumer of a token “MUST only consider the token’s top-level claims and the party identified as the current actor by the act claim,” and “prior actors identified by any nested act claims are informational only and are not to be considered in access control decisions.” The nested chain is an audit record and nothing else.

The same section rules out something people assume is in there. Non-identity claims such as exp, nbf and aud are not meaningful inside an act claim and are not used, so an actor entry says who acted and never says when that authority began, when it ended, or which resource it was good for. Those facts live in your own logs, which makes the delegation record and the access record two records in two systems.

In a heterogeneous federation, not every authority emits act at all. Microsoft Entra records the acting application in azp or appid, its own convention, so a hop through it flattens the chain in the middle and the orchestrator drops out of the record. Whichever component owns delegation has to hold the actor list out of band and construct the chain itself.

Platforms will offer to keep this record for you. Google’s Agent Identity documentation promises logs showing both the agent’s and the user’s identity when an agent acts on someone’s behalf. That record lives in the platform; the query lives in the warehouse’s history under the person’s name; nothing carries a key across the two. Reconstructing which agent caused which query is a timestamp join unless the hop that holds both facts writes one down before it spends the token. That hop is your tool, the only place where the actor and the statement are in the same process at the same time.

Tokens expire; agent tasks do not care

One piece of arithmetic decides whether long tasks work at all, and it is a min. An exchanged or derived token inherits its lifetime from what it was derived from. A sane authorization server will not mint a credential that outlives the authority behind it. The token I was handed came back with expires_in=1799, about thirty minutes. If the credential behind it carries a shorter lifetime by on-prem policy, a nominally sixty-minute downstream token is fiction; you get the shorter number.

RFC 8693 explains why you cannot repair this at the point of exchange. Section 2.2.1 says a refresh token will typically not be issued when the exchange trades one temporary credential for another. The exchanged token is a leaf. Renewal has to run against the user’s session at the root, so a long task re-pays for one exchange and not the whole chain.

Inside the agent there is a harder boundary. The token arrives as a string copied into invocation state, a value and not a reference to whatever the platform’s refresh loop is doing. Re-reading state on every call, which is what the lookup above does, picks up whatever this invocation was handed. But it cannot produce a refresh in the middle of one. The refresh boundary is the invocation boundary. A plan that has to outlive the token’s remaining life has to be cut into more invocations, which is a task-decomposition decision usually made by someone who believes they are tuning latency.

When the credential runs out mid-plan, the wrapper hands the model No authorized session as a tool result. The model keeps planning, generally by re-issuing the statement it just tried. The replay costs nothing because every statement here is a read. The same wrapper in front of a tool that writes would replay a committed side effect and call it error handling, so read-only is load-bearing for this design. Fresh sessions help again: expiry lands between statements, never inside one.

Where the authorization boundary belongs

A design note that reads as a security hole until you follow it through. The agent applies no SQL allow-listing. It runs whatever statement the model produces.

That is deliberate. Authorization is enforced at the warehouse by the per-user token’s role and the policy engine. A second allow-list in the agent would be a weaker copy of a policy that already exists somewhere authoritative. The failure mode of a divergent copy is that it drifts and starts permitting things the real policy forbids. The agent is not the access-control boundary and shouldn’t pretend to be one.

Which puts the whole weight on which role the token carries, settled at registration time. The scope lives in the query string of a create-only authorization resource. It is one string for every user and every question the agent will ever be asked. SESSION:ROLE-ANY gets registered because it works without knowing anyone’s roles. The narrow alternative, SESSION:ROLE:<role>, costs an authorization resource and a consent screen per role, and a user shown three consent screens has been trained to click through all of them. Consent granularity is bounded by the number of authorization resources you will register, not by the number of things the agent does.

An account-level parameter sits underneath. EXTERNAL_OAUTH_ANY_ROLE_MODE defaults to DISABLE, which blocks USE ROLE inside the session; ENABLE permits it, and ENABLE_FOR_PRIVILEGE permits it for holders of USE_ANY_ROLE. With SESSION:ROLE-ANY and ENABLE set together, the role a statement runs under is chosen by whoever writes the SQL, and under an agent that is the model. Ask which mode the account is in before you tell anyone the token pins the role. Fresh sessions bound it either way, since a USE ROLE the model emits dies with the statement that carried it.

So a prompt injection telling the agent to read a table it should not cannot escalate a privilege the agent never held. The model may well write the query; the warehouse evaluates it under that user’s role and returns what that user was always allowed to see. The ceiling is the caller’s grants, which is not the same as the task’s intent, and the gap between those two is exactly the width of the scope you registered.

When not to do any of this

If a single service identity is acceptable, because the data is shared and non-sensitive and everyone who can reach the agent can already see all of it, use one. Dynamic MCP tool discovery works fine when auth is ambient, and none of the machinery above earns its cost. And if the platform in front of your agent does not manage OAuth for you, the client secret, the token endpoint, the perimeter egress path and the refresh handling all become yours, which is a larger system than the one described here.

The thing I would press on anyone starting: decide which process owns the OAuth lifecycle before you draw the network diagram. That choice determines whether your agent needs a hole in the perimeter, whether anyone has to rotate a secret after you leave, and whether your tools exist at listing time.