Early-bird pricing (ends Dec 25), locked in while you stay subscribed: Go $10/mo (normally $20), Plus with Opus 5.5 just $30/mo (normally $50)Subscribe →

Gemini API Tool Retrieval: When It Saves Tokens (2026)

With Tool Retrieval in the Gemini API, agents carry hundreds of tool names but load full parameter schemas only for the tools each task needs. The catch: each lookup runs another model pass, so workloads that search every turn can pay more, not less.

The feature launched in preview on November 19, 2026, inside the Interactions API only, per Google's announcement. It adds a tool_search tool plus a defer_loading: true flag. Deferred tools expose name and description while parameter JSON stays out of context. There is no separate fee; standard input, cached-input, and output rates apply.

Why 200 tools used to cost 164,000 tokens before a single call

Schema size, not tool count alone, determines prompt cost, so large catalogs can dominate the prompt.

An independent technical breakdown of Google's docs relays the illustrative gap: roughly 164,000 input tokens without deferral versus about 23,650 with Tool Retrieval for a 200-tool library where only a few tools run per request. Round-number illustration from that analysis, not a benchmark — the point is you stop paying for the schemas you never touch.

The second problem is accuracy. Dozens of similar schemas competing in context give the model more ways to pick wrong. Google frames Tool Retrieval as help for both token load and selection accuracy. No accuracy number is published, so treat the accuracy half as a hypothesis to test on your own tasks.

How Gemini Tool Retrieval runs: tool_search and defer_loading

Setup is two abbreviated declaration fragments — a tool_search entry of {"type": "tool_search"} plus "defer_loading": true on each deferred tool — with the full request shape shown in the docs breakdown. The model still sees each deferred tool's name and description. Per the defer_loading explainer, when short_description is set the model sees that; otherwise it sees the first sentence of the full description.

Server-side flow, all in one request:

  1. Model emits tool_search_call, for example requesting issue_refund.
  2. API returns tool_search_result with the full schema.
  3. Model emits the real function_call (Google's refund example uses order_id: "A-1042" and amount: 40 with model gemini-4-hydrogen).
  4. Your code executes and returns function_result.

Log everything in interaction.steps. When the wrong tool loads, check the trace first for vague names or descriptions.

Server-side vs client-side: one request vs one round trip

ModeWho searchesIndex to buildExtra round tripsBest fit
Server-side (default)GeminiNone — "There's no index to build or host."None, same requestLarge, mostly static libraries; whole MCP servers
Client-side ("execution": "client")Your appYour registry or embedding indexOne per searchPer-user permissions, private registries, tools added mid-conversation

Server-side retrieval needs no hosted index and completes in the same response; setting defer_loading: true on an mcp_server tool defers every tool that server exposes, per the docs breakdown.

Client-side uses tool_search with "execution": "client" (example name search_tools with a required query string). Your app returns matches in a function_result, then follows with client.interactions.create carrying previous_interaction_id. Per the same breakdown, only one tool_search per request, and the two modes cannot mix.

Choose client-side when authorization is per-user or integrations can appear mid-conversation: a server-side search over one shared catalog does not by itself apply per-user permission filtering, per the cited analysis. Per that breakdown, client-side is also the mode that supports tools appearing mid-conversation without declaring everything up front.

Gemini Tool Retrieval billing math: 26 tokens now, another full pass later

Tool Retrieval has no separate charge, per Google's announcement — "Tool Retrieval has no separate charge." Four metered costs remain, per the token-economics breakdown:

  • tool_search adds a small fixed input-token block to every request.
  • The cited analysis estimates roughly 26 tokens per deferred name-and-description line.
  • Each server-side search runs another model pass over current context, billed via total_tool_use_tokens at roughly the prompt's input volume minus cache hits, per that analysis.
  • Each loaded schema stays billed as input for the rest of the conversation and cannot be unloaded.
Workload (illustrative estimates from cited analysis)Without retrievalWith retrievalVerdict
200 tools, few used per task~164,000 tokens~23,650 tokensSaves
10 tools, 30k context, search almost every turn~68,000 tokens~92,830 tokensCosts more
Illustrative input token comparison for Gemini Tool Retrieval workloads

Google's guidance in the same announcement: "Measure it against your own traffic."

"In server-side mode... every server-side search adds another model pass over the current context... So a big library where each task uses a few tools can benefit. A small library with a long conversation and frequent searches can cost more." — @tekroch

Deferral can stabilize the cached prefix but may also drop a prompt below the cache-size threshold; track cache-hit rate alongside input tokens, per the cited analysis.

When to defer and when to leave it loaded

For small libraries, benchmark static loading first; search frequency and schema size matter more than any tool-count cutoff. For large catalogs with 1–3 tools per task, defer the tail server-side and keep frequently used tools undeferred (tools used on most requests). For per-user or mid-chat tools, use client-side and budget one round trip per lookup.

The cited implementation notes list three failure modes to wire around:

  1. Do not defer a tool named in system instructions or another tool's output; the model may call it without searching and the next turn can fail.
  2. Deferred functions still need complete parameters in the request; name-plus-description alone can return an empty, uncallable schema.
  3. In stateless apps, echo tool_search_call, tool_search_result, signatures, and the full tools array unchanged every turn.

Description quality decides retrieval: the model picks from names and descriptions before seeing any schema. "Search customer invoices by date range" retrieves; "Invoice helper" collides. Deferral is not a security boundary — deferred MCP names and descriptions stay visible as prompt input, and loaded tools run with your granted permissions, per the security analysis.

What preview does not give you

Preview only, Interactions API, v1beta surface. No generateContent support; defer_loading on an unsupported model returns HTTP 400, per the docs breakdown.

Model coverage is currently narrower than the headline: the announcement says Gemini 3.6 and later including all Gemini 4 models, while the cited breakdown reports the live docs currently list gemini-4-hydrogen, gemini-4-argon, gemini-3.8-flash, and gemini-3.6-flash. Confirm your model in the docs before assuming siblings work. No GA date is announced. Anthropic shipped the same defer_loading idea as a beta in November 2025; Google's deltas are server-side MCP deferral and client-side mode, per launch coverage.

Your library shapeMode to pickFirst action
Small, stableStatic loadingMeasure schema tokens before adding complexity
50–200 tools, few used per taskServer-side deferralDefer the tail, keep hot tools loaded
Per-user or mid-chat toolsClient-side retrievalBuild permission-aware search
Every tool used every turnStatic loadingSearch overhead likely exceeds savings

Ship one long-tail slice first, then compare input tokens, total_tool_use_tokens, latency, and task success with and without deferral on identical tasks.

FAQ

Does Tool Retrieval work with generateContent?

No. Interactions API on v1beta only.

Which models support it?

Announced as Gemini 3.6 and later including Gemini 4; the cited breakdown reports live docs currently narrow that to models including gemini-4-hydrogen and gemini-3.6-flash. Check the docs for your exact model.

Can it raise my bill?

Yes. The cited illustrative 10-tool workload rises from ~68,000 to ~92,830 tokens when searches run nearly every turn.