Gemini Agentic Video Understanding API Guide

A two-hour lecture and a 20-second product clip should not be inspected the same way. Gemini’s new agentic video understanding mode is the better starting point for long-form or targeted-moment questions, but Google’s “up to” savings are benchmark claims, not a guaranteed 66% cut on every API bill.

Pick the processing mode from the task, not the video length

Gemini agentic video understanding fits questions that require finding or revisiting particular moments. Static processing remains the simpler choice when a short clip must return quickly or every sampled interval needs consistent coverage.

Google names four launch use cases: sub-second moment retrieval, long-form “needle-in-a-haystack” search, anomaly detection, and counting actions or objects. The official announcement describes examples ranging from 10-minute how-to guides to 90-minute lectures and multi-hour recordings.

No published duration cutoff exists: a five-minute video with one hidden event can justify agentic processing, while a longer uniform recording may not.

One feature, two API request shapes

“Agentic” is not one universal JSON field. Google’s launch example uses the Interactions API and puts processing: "agentic" inside the video input; the GenerateContent path exposes media processing on the media part.

Interactions API: set processing on the video input

Google’s launch post shows this Python pattern for a YouTube video:

from google import genai

client = genai.Client()

interaction = client.interactions.create(
    model="gemini-3.7-flash",
    input=[
        {
            "type": "video",
            "uri": "https://youtu.be/7Z5Vy9JBANs",
            "processing": "agentic"
        },
        {
            "type": "text",
            "text": "What are the 3 most important announcements in this keynote?",
        },
    ],
)

print(interaction.output_text)

Place the setting on the video object, not an unrelated top-level field.

GenerateContent: set media processing on the media part

The Gemini video-understanding documentation documents the GenerateContent form separately. In that request shape, the media input uses media_processing: "AGENTIC"; JavaScript uses the corresponding camelCase property, mediaProcessing.

A minimal Python shape looks like this:

from google import genai
from google.genai import types

client = genai.Client()
video_uri = "YOUR_VIDEO_FILE_URI"

response = client.models.generate_content(
    model="gemini-3.7-flash",
    contents=[
        types.Part.from_uri(
            file_uri=video_uri,
            mime_type="video/mp4",
            media_processing="AGENTIC",
        ),
        "Find the first product announcement and return its timestamp.",
    ],
)

print(response.text)

For a local video, replace YOUR_VIDEO_FILE_URI with the URI returned by the Gemini file-upload flow and wait until the file is ready before sending the media part. Use the current official video-understanding guide to confirm SDK details for your installed client version.

Do not interchange snippets when migrating from client.interactions.create to generateContent.

What agentic video understanding actually does

Static video processing samples a video at a fixed frame rate. Google says the default is 1 frame per second, with the rate adjustable through the API, and supplies the sampled frames for the model to analyze.

The agentic workflow lets Gemini search and scan the timeline, choose a useful speed, and draw on visual frames, audio, or transcripts instead of loading the entire recording uniformly:

1. Request the segment or modality relevant to the question.
2. Inspect the returned frames, audio, or transcript.
3. Revisit a narrower interval when more temporal precision is needed.

Static sampling gives developers a straightforward coverage rule. Agentic processing gives the model more responsibility for deciding where to look, so production systems should validate timestamps and counts against representative footage.

The savings claim is not a 66% discount

Google reports the following results across standard video-analysis benchmarks, with the largest gains emphasized for long-form video:

Agentic video understanding uses standard Gemini API token pricing and has no additional feature fee, according to Google’s launch announcement. Actual cost still depends on the model, the content retrieved, the request, and the output. For broader token and model pricing context, see the Gemini 3.7 Flash API pricing guide.

For repeated questions about the same recording, cache the video context where the API workflow supports it. Google Cloud’s practical video-analysis notes identify context caching as a key cost lever for repeated analysis.

Availability and rollout boundaries

Agentic video understanding is a processing capability across three Flash-family models, not a separately announced video model. Google says it launched on September 1, 2026 for these models and surfaces:

The planned consumer and YouTube rollouts are not proof of current availability for every account.

A four-run production test before switching the default

The launch announcement provides capability claims, but it does not tell you how the mode behaves on your particular videos. Use a small evaluation set before replacing an existing static workflow.

1. Timestamp retrieval: Include clips with scene cuts, brief appearances, and one event whose exact start and end time matter. Compare the returned timestamps with a human-checked reference.
2. Fast-action counting: Use footage with repeated movements or objects crossing the frame. Check the total count and whether the answer explains the counting window.
3. Audio and transcript grounding: Ask questions where the spoken words and visible action differ. Verify that the answer does not silently substitute one modality for the other.
4. Operational behavior: Run the same files in both modes and log input/output tokens, wall-clock latency, retries, empty answers, missed events, and the static fallback result. Keep the fallback when selective retrieval misses evidence or adds unacceptable delay.

Do not create a hard “use agentic above X minutes” rule from the launch post. The published material supports task-based selection, not a universal duration threshold.

Gemini agentic video understanding API FAQ

Is agentic video understanding a new Gemini model?

No. Google presents it as a video-understanding capability available across Gemini 3.7 Flash, Gemini 3.6 Flash, and Gemini 3.5 Flash-Lite.

Can I use an uploaded video instead of a YouTube URL?

Yes. Google lists uploaded video files and YouTube videos as supported API inputs; for a local file, upload it first and pass the returned file URI in the media part after processing is ready.

Does agentic video processing cost extra?

Google says there is no additional feature fee. Normal Gemini API token pricing applies, and the final bill depends on the retrieved content and the rest of the request.

Run a representative set through both modes, then retain agentic processing as the default only where its measured evidence quality and token cost fit the application’s target.