Iris Search Agent Review: Start With Mini, Not Pro

Iris looks like a strong open-weight search agent, but the decisive number is not 88.6. It is the 17.5-point BrowseComp swing produced by changing Iris-mini's context handling. Start with Iris-mini and the official harness; reserve Iris-pro for teams that already operate multi-node inference.

The Iris decision in one table

The practical Iris Search Agent recommendation is simple: evaluate Iris-mini first. Iris-pro reports better scores, but its official launch example requires far more parallel infrastructure, while neither checkpoint has a hosted Hugging Face inference provider as of September 14, 2026.

Sources: official Iris-mini model card, Iris-pro model card, and Iris repository.

The weights, model cards, and Iris-Harness evaluation code are public. The repository still says the data-construction and training pipelines are “coming soon,” so Iris is open weight with an open evaluation harness, not yet a fully released training recipe.

The benchmark headline changes when the harness changes

Iris benchmark scores cannot be read as properties of the checkpoints alone. AllSpark explicitly states that a published number belongs to the agent and its harness, and the official ablations show why.

The official Iris GitHub README defines discard-all as resetting to the original question after the running context crosses a threshold. retry restarts an episode that ended without a parseable answer while carrying a short summary of what was ruled out.

For Iris-mini, discard-all raises BrowseComp from 64.7 to 82.2—17.5 points without changing the checkpoint. For Iris-pro, the same comparison rises from 72.6 to 88.6, a 16-point gain. A deployment that omits the official conversation format, reset policy, search tools, or answer parser is not reproducing the advertised system.

The main cross-model tables also require caution. AllSpark reports that baseline values came from each project's public report and may use different context-management configurations. Iris-mini leads its listed size class on BrowseComp, BrowseComp-ZH, and HLE, but XYZ-Aquila-mini remains ahead on DeepSearchQA, 89.5 versus 86.9. Iris-pro leads or ties the listed roughly 400B systems across all four columns, though its DeepSearchQA margin over XYZ-Aquila-pro is only 0.4 points.

Iris-mini or Iris-pro: choose by deployment burden

Iris-mini is the sensible first checkpoint because active parameters do not erase total-model storage and memory demands. The MoE router activates about 3B of 35B parameters per step, but the complete checkpoint still has to be stored and served.

The official Iris-mini command uses four-way tensor parallelism and a 262,144-token context:

python -m sglang.launch_server \
  --model-path AllSpark-Research/Iris-mini \
  --served-model-name Iris-mini \
  --port 21234 --tp-size 4 \
  --context-length 262144 \
  --reasoning-parser qwen3 \
  --tool-call-parser qwen3_coder

Iris-pro's documented configuration uses --tp-size 8 --ep-size 8. Its official model card says the 397B checkpoint requires multiple nodes or a large single node. That is a deployment warning, not a minor configuration difference.

The larger model gains 6.4 points over mini on BrowseComp and 6.0 on DeepSearchQA under discard-all, but only 0.3 on BrowseComp-ZH. Teams should not accept the infrastructure jump merely because “pro” is larger. The gain must matter on the workload being evaluated.

Neither model card publishes measured latency, throughput, VRAM usage, or operating cost. Those missing values prevent a responsible cost-per-answer comparison. Run a bounded pilot rather than converting parameter counts into invented GPU estimates.

A reproducible first Iris evaluation

A first Iris evaluation should test the full agent path, not send trivia to a chat-completions endpoint. The official harness includes the agent loop, search and scrape tools, context strategies, benchmark adapters, and graders.

1. Serve Iris-mini with the official parsers. Use the SGLang command above and confirm that the OpenAI-compatible endpoint is reachable at port 21234.
2. Install Iris-Harness. The repository uses uv to create the environment.
3. Prepare the benchmark data. Run the included preparation script rather than assembling an ad hoc copy.
4. Start with a narrow benchmark slice. The official example selects browsecomp:0:1 and a 131,072-token discard threshold.
5. Record the complete configuration. Keep the model revision, tool backend, threshold, parser versions, and retry policy beside every result.

git clone https://github.com/AllSpark-Research/Iris.git
cd Iris/Iris-Harness
uv sync
uv run python data/prepare_data.py

bash scripts/run_eval.sh \
  --base-url http://127.0.0.1:21234/v1 \
  --llm-config iris-mini \
  --benchmarks "browsecomp:0:1" \
  --context-discard-threshold 131072

Iris is trained to use the OpenAI function-calling interface and wrap final answers in \boxed{}. The harness also carries prior reasoning into later turns. Replacing that protocol with a generic chat template can break tool use or grading even when ordinary text generation appears normal.

Use three acceptance checks for a pilot:

• Answer quality: Does the agent recover the required evidence, not merely guess the final entity?
• Search economy: How many search and scrape calls are consumed per accepted answer?
• Recovery behavior: When context is discarded or an episode is retried, does the agent avoid repeating the same failed path?

The official release provides no production SLA and no hosted endpoint. A successful benchmark reproduction establishes that the system runs under a known harness; it does not establish reliability on unrestricted web research.

What is released—and what still blocks a full reproduction

The current Iris release is substantial but incomplete. Developers can download both checkpoints without an approval gate, use them commercially under Apache 2.0, inspect the benchmark tables, and run Iris-Harness against an OpenAI-compatible endpoint.

The paper describes reverse-constructing multi-hop questions from hyperlink graphs, filtering trajectories at whole-run and individual-turn levels, and alternating supervised fine-tuning with live-search reinforcement learning. However, the repository's release status means outside teams can reproduce inference and evaluation before they can reproduce training end to end.

Early community discussion reflects that distinction. A technical-news account summarized the release as follows:

“Weights + eval code are public. Training data and recipe come later.” — @Chinazhidx

That is the right level of confidence. Iris is formally released, not a rumor, but its strongest claims still need external runs under disclosed harness settings. The Hugging Face pages showed 335 monthly downloads for Iris-mini and 685 for Iris-pro when checked on September 14, 2026; download counts are activity signals, not validation.

Iris Search Agent FAQ

Is Iris a model or a complete search product?

Iris is an open-weight model family plus an evaluation harness. The release does not include a hosted consumer search application or managed API.

Can Iris run locally?

Yes, but “local” does not mean laptop-friendly. The official Iris-mini example uses four-way tensor parallelism; Iris-pro is documented for eight-way tensor and eight-way expert parallelism.

What does 35B-A3B mean?

Iris-mini has about 35B total parameters and 3B active parameters per inference step. Sparse activation reduces compute relative to activating every parameter, but the full checkpoint still affects storage and serving memory.

Is Iris fully open source?

The checkpoints and harness are public under permissive terms, but the repository still lists the data-construction and training pipelines as forthcoming. “Open weight with open evaluation code” is the most precise current description.

Does Iris have an API?

The models can be served behind an OpenAI-compatible endpoint using SGLang or vLLM. No hosted Hugging Face inference provider or official managed Iris API is listed on either model card.

Which Iris model should I use?

Use Iris-mini for evaluation unless a measured workload requires Iris-pro's extra benchmark performance and the team already has suitable multi-node or large-node infrastructure.

The next action: pilot mini with a fixed harness

Download Iris-mini, preserve the official tool and parser protocol, and benchmark a small set of questions that resemble the intended workload. Fix the context policy before comparing results, because changing discard-all or retry can move the score more than changing models.

Move to Iris-pro only if the pilot identifies a quality gap worth its deployment burden. The unresolved trade-off is straightforward: Iris offers unusually strong reported open-weight search performance, but reproducible training details and production cost evidence are not available yet.