With Tool Retrieval in the Gemini API, agents carry hundreds of tool names but load full parameter schemas only for the tools each task needs. The catch: each lookup runs another model pass, so workloads that search every turn can pay more, not less.
The feature launched in preview on November 19, 2026, inside the Interactions API only, per Google's announcement. It adds a tool_search tool plus a defer_loading: true flag. Deferred tools expose name and description while parameter JSON stays out of context. There is no separate fee; standard input, cached-input, and output rates apply.
Why 200 tools used to cost 164,000 tokens before a single call
Schema size, not tool count alone, determines prompt cost, so large catalogs can dominate the prompt.
An independent technical breakdown of Google's docs relays the illustrative gap: roughly 164,000 input tokens without deferral versus about 23,650 with Tool Retrieval for a 200-tool library where only a few tools run per request. Round-number illustration from that analysis, not a benchmark — the point is you stop paying for the schemas you never touch.
The second problem is accuracy. Dozens of similar schemas competing in context give the model more ways to pick wrong. Google frames Tool Retrieval as help for both token load and selection accuracy. No accuracy number is published, so treat the accuracy half as a hypothesis to test on your own tasks.
How Gemini Tool Retrieval runs: tool_search and defer_loading
Setup is two abbreviated declaration fragments — a tool_search entry of {"type": "tool_search"} plus "defer_loading": true on each deferred tool — with the full request shape shown in the docs breakdown. The model still sees each deferred tool's name and description. Per the defer_loading explainer, when short_description is set the model sees that; otherwise it sees the first sentence of the full description.
Server-side flow, all in one request:
- Model emits
tool_search_call, for example requestingissue_refund. - API returns
tool_search_resultwith the full schema. - Model emits the real
function_call(Google's refund example usesorder_id: "A-1042"andamount: 40with modelgemini-4-hydrogen). - Your code executes and returns
function_result.
Log everything in interaction.steps. When the wrong tool loads, check the trace first for vague names or descriptions.
Server-side vs client-side: one request vs one round trip
| Mode | Who searches | Index to build | Extra round trips | Best fit |
|---|---|---|---|---|
| Server-side (default) | Gemini | None — "There's no index to build or host." | None, same request | Large, mostly static libraries; whole MCP servers |
Client-side ("execution": "client") | Your app | Your registry or embedding index | One per search | Per-user permissions, private registries, tools added mid-conversation |
Server-side retrieval needs no hosted index and completes in the same response; setting defer_loading: true on an mcp_server tool defers every tool that server exposes, per the docs breakdown.
Client-side uses tool_search with "execution": "client" (example name search_tools with a required query string). Your app returns matches in a function_result, then follows with client.interactions.create carrying previous_interaction_id. Per the same breakdown, only one tool_search per request, and the two modes cannot mix.
Choose client-side when authorization is per-user or integrations can appear mid-conversation: a server-side search over one shared catalog does not by itself apply per-user permission filtering, per the cited analysis. Per that breakdown, client-side is also the mode that supports tools appearing mid-conversation without declaring everything up front.
Gemini Tool Retrieval billing math: 26 tokens now, another full pass later
Tool Retrieval has no separate charge, per Google's announcement — "Tool Retrieval has no separate charge." Four metered costs remain, per the token-economics breakdown:
tool_searchadds a small fixed input-token block to every request.- The cited analysis estimates roughly 26 tokens per deferred name-and-description line.
- Each server-side search runs another model pass over current context, billed via
total_tool_use_tokensat roughly the prompt's input volume minus cache hits, per that analysis. - Each loaded schema stays billed as input for the rest of the conversation and cannot be unloaded.
| Workload (illustrative estimates from cited analysis) | Without retrieval | With retrieval | Verdict |
|---|---|---|---|
| 200 tools, few used per task | ~164,000 tokens | ~23,650 tokens | Saves |
| 10 tools, 30k context, search almost every turn | ~68,000 tokens | ~92,830 tokens | Costs more |
Google's guidance in the same announcement: "Measure it against your own traffic."
"In server-side mode... every server-side search adds another model pass over the current context... So a big library where each task uses a few tools can benefit. A small library with a long conversation and frequent searches can cost more." — @tekroch
Deferral can stabilize the cached prefix but may also drop a prompt below the cache-size threshold; track cache-hit rate alongside input tokens, per the cited analysis.
When to defer and when to leave it loaded
For small libraries, benchmark static loading first; search frequency and schema size matter more than any tool-count cutoff. For large catalogs with 1–3 tools per task, defer the tail server-side and keep frequently used tools undeferred (tools used on most requests). For per-user or mid-chat tools, use client-side and budget one round trip per lookup.
The cited implementation notes list three failure modes to wire around:
- Do not defer a tool named in system instructions or another tool's output; the model may call it without searching and the next turn can fail.
- Deferred functions still need complete
parametersin the request; name-plus-description alone can return an empty, uncallable schema. - In stateless apps, echo
tool_search_call,tool_search_result, signatures, and the fulltoolsarray unchanged every turn.
Description quality decides retrieval: the model picks from names and descriptions before seeing any schema. "Search customer invoices by date range" retrieves; "Invoice helper" collides. Deferral is not a security boundary — deferred MCP names and descriptions stay visible as prompt input, and loaded tools run with your granted permissions, per the security analysis.
What preview does not give you
Preview only, Interactions API, v1beta surface. No generateContent support; defer_loading on an unsupported model returns HTTP 400, per the docs breakdown.
Model coverage is currently narrower than the headline: the announcement says Gemini 3.6 and later including all Gemini 4 models, while the cited breakdown reports the live docs currently list gemini-4-hydrogen, gemini-4-argon, gemini-3.8-flash, and gemini-3.6-flash. Confirm your model in the docs before assuming siblings work. No GA date is announced. Anthropic shipped the same defer_loading idea as a beta in November 2025; Google's deltas are server-side MCP deferral and client-side mode, per launch coverage.
| Your library shape | Mode to pick | First action |
|---|---|---|
| Small, stable | Static loading | Measure schema tokens before adding complexity |
| 50–200 tools, few used per task | Server-side deferral | Defer the tail, keep hot tools loaded |
| Per-user or mid-chat tools | Client-side retrieval | Build permission-aware search |
| Every tool used every turn | Static loading | Search overhead likely exceeds savings |
Ship one long-tail slice first, then compare input tokens, total_tool_use_tokens, latency, and task success with and without deferral on identical tasks.
FAQ
Does Tool Retrieval work with generateContent?
No. Interactions API on v1beta only.
Which models support it?
Announced as Gemini 3.6 and later including Gemini 4; the cited breakdown reports live docs currently narrow that to models including gemini-4-hydrogen and gemini-3.6-flash. Check the docs for your exact model.
Can it raise my bill?
Yes. The cited illustrative 10-tool workload rises from ~68,000 to ~92,830 tokens when searches run nearly every turn.