DeepInfer H100 Support: What the Official List Means

The short answer for H100 and RTX teams

DeepInfer H100 support is not officially announced or documented. RTX support is not officially documented either. The current release targets DeepSeek-V4.1-Flash and lists NVIDIA SM100 hardware with CUDA 13, or Ascend 950 with CANN 9.2. The repository also requires Linux, GCC 13 or newer, and Python 3.12 (official DeepInfer repository).

“Not listed” is a support verdict for procurement and production planning, not proof that an experimental patch can never compile.

What DeepInfer officially supports today

The support boundary is narrower than “NVIDIA GPU” or “DeepSeek model.” It is a specific accelerator generation, software stack, and model path.

DeepSeek’s deployment Q&A reports benchmark topologies of 4 SM100 GPUs for prefill and 64 SM100 GPUs for decode in a scale-up domain. These are SM100 measurements, not H100 capacity estimates.

Why SM100 does not automatically include H100 or RTX

The official DeepInfer list does not name H100, SM90, or an RTX SKU. CUDA compatibility alone does not establish DeepInfer compatibility.

The repository integrates architecture-sensitive components such as FlashMLA, DeepGEMM, DeepEP, and TileLang operations (project documentation). A recognized GPU target can still fail at runtime, produce incorrect results, or lose the performance that motivated the migration.

That is why three claims must stay separate:

1. The code compiles for an architecture.
2. The model runs with correct outputs and stable distributed behavior.
3. The deployment is validated at useful latency and throughput.

Only the first claim can be established by a successful build. DeepInfer’s public materials do not provide an H100 or RTX benchmark, an official SM90 build recipe, or a validated consumer-RTX configuration.

Migration paths for existing infrastructure

If your cluster is H100 or another Hopper system

Do not replace an operating H100 serving stack solely to test DeepInfer. Treat the current path as an experimental porting project, not a supported deployment.

A sensible migration plan is:

1. Keep the production model server unchanged.
2. Create an isolated branch with pinned DeepInfer, CUDA, driver, compiler, and model versions.
3. Inventory architecture-specific dependencies before compiling.
4. Run numerical comparison tests against a known-good runtime.
5. Measure prefill latency, decode latency, concurrency, memory use, and failure recovery.
6. Promote the port only if it improves the workload that matters to your team.

The practical alternative is to evaluate a serving runtime that explicitly documents your H100 configuration, such as vLLM, SGLang, or TensorRT-LLM. Select among them based on the exact DeepSeek checkpoint, quantization, API features, and distributed topology you need—not on DeepInfer’s SM100 numbers.

If your team has RTX workstations or RTX servers

The official DeepInfer list names no RTX SKU. Its documented benchmark topology targets large-scale inference, so a single RTX card or small RTX pool should not be treated as a low-cost substitute for the 4-GPU prefill or 64-GPU decode configurations.

Use a general local inference runtime if the objective is development, evaluation, or a smaller self-hosted endpoint. Keep DeepInfer on a watch list for a future architecture-specific port, but do not budget around an unmerged issue or a forum claim.

If you can access SM100 or Ascend 950

This is the only path that matches the current official matrix. Even here, DeepInfer is not a complete online API service: the official deployment notes describe missing production components such as persistent KV cache, prefill/decode orchestration, and replica load balancing.

That makes DeepInfer a better fit for teams that want to study or benchmark the engine, have the required accelerator topology, and are prepared to build the surrounding platform. It is not automatically the best choice for a multi-model API gateway.

Deployment constraints beyond the GPU

Hardware support is only one migration gate. The public release is described as offline inference, not a finished OpenAI-compatible server. The missing pieces affect architecture decisions:

The separate deepseek-recipe project translates Messages, Chat Completions, and Responses formats, but it does not execute inference or provide the complete production service.

A verification checklist before changing hardware

Before approving a DeepInfer migration, require evidence for each item:

1. Official architecture and build support: Is the exact GPU listed, and can a clean machine reproduce the pinned environment?
2. Kernel and correctness coverage: Do attention, GEMM, and expert-parallel kernels produce matching outputs, structured responses, and long-context behavior?
3. Distributed performance: Does the intended GPU count and interconnect topology remain stable at your concurrency and memory targets?
4. Service completeness: Where do KV cache, routing, authentication, HTTP transport, and replica balancing come from?

If the answer to the first question is “no,” label the project an experiment. If the answer to the service questions is “external,” include that engineering work in the migration cost.

DeepInfer H100 support FAQ

Can DeepInfer run on an NVIDIA H100?

H100 support is not officially documented. The current NVIDIA support entry is SM100. An experimental port may be possible, but there is no official H100 build recipe or validated H100 benchmark in the cited materials.

Does DeepInfer support RTX 4090, 5090, or other RTX cards?

RTX support is not officially listed. The cited official materials provide no validated RTX configuration. Treat RTX deployment as unsupported until the upstream project documents the architecture and validates its kernels.

What should an H100 team use instead?

Keep or evaluate a runtime with explicit H100 support, then compare it on your own checkpoint and traffic pattern. vLLM, SGLang, and TensorRT-LLM are practical starting points, while DeepInfer is best treated as a future porting target or a reason to evaluate SM100 infrastructure.

Treat an H100 or RTX port as experimental until the exact architecture, kernels, service layer, and workload results are documented.