Skip to content

GPU compatibility matrix

Which GPUs CodeNavi runs on, and how to check yours before install.

The image pins a single torch wheel at build time; that wheel must contain CUDA kernels for your GPU's compute capability, or the embedder cannot run on that card.1

The standard build: cu128 / torch 2.7.1 as the universal wheel.2 Its arch_list is sm_75 … sm_120, so ONE image covers Turing (2018) → Blackwell (2025) — empirically verified running fp16/bf16 matmul + SDPA on a real RTX 2080 Ti (sm_75) on 2026-07-03. Build it with --gpu-arch universal (alias of blackwell). Pre-Turing NVIDIA — Volta, Pascal, Maxwell — is not supported (CN-568, 2026-08-31): NVIDIA's R580 driver branch was the last to carry those cards, the 590+ branches dropped them, and Windows loads a single driver for every NVIDIA card in the system. Use --gpu-arch cpu on such hosts.

The embedder also fails loud2 at model load if the baked wheel has no kernels for the attached GPU: a clear RuntimeError naming the GPU's sm_XY, the baked arch_list, and the --gpu-arch fix — instead of the old silent 0-results. Override with CODENAVI_SKIP_GPU_ARCH_CHECK=1.

--gpu-arch Compute cap Example cards Wheel index torch package
universal (recommended default) sm_75 – sm_120 Any Turing→Blackwell: RTX 20/30/40/50-series, T4, A100, L4, L40S, A6000, RTX PRO Blackwell cu128 torch==2.7.1
blackwell sm_100 / sm_120 RTX 50-series (5070/5080/5090), RTX PRO 4500 / 6000 Blackwell, B100/B200 cu128 torch==2.7.1
hopper sm_90 H100, H200 cu124 torch==2.6.0
ada sm_89 RTX 4090 / 4080 / 4070, L40, L4 cu124 torch==2.6.0
ampere sm_80 / sm_86 A100, RTX 3090 / 3080, A6000, RTX A4000 cu124 torch==2.6.0
turing sm_75 RTX 2080 Ti, T4, Quadro RTX cu124 torch==2.6.0
rocm (AMD) gfx900–gfx1201 see AMD section below rocm7.2 torch==2.12.1
cpu — CPU-only smoke build (very slow indexing) cpu torch==2.6.0

Driver floor. cu128 wheels require an NVIDIA driver ≥ 570.65 (the CUDA 12.8 runtime) on the customer host; the older cu124 wheels run from ~525. A card that passes the arch check but sits on too-old a driver fails at CUDA init, not at load.

AMD GPUs (--gpu-arch rocm)

One wheel covers the whole supported AMD range4 — the official pytorch.org rocm7.2 wheels bundle the complete ROCm userland, so the standard image base works and the customer host needs only the in-tree amdgpu kernel driver (no ROCm install, no container toolkit). Validated end-to-end on an RX 9070 (gfx1201): full-reindex parity vs a CUDA build at min-cos 0.999988.

torch==2.12.1+rocm7.2 gfx targets:

Family gfx targets Example cards
Instinct CDNA gfx908, gfx90a, gfx942, gfx950 MI100, MI210/MI250, MI300A/X, MI350
RDNA2 gfx1030 RX 6800/6900 XT
RDNA3 gfx1100 / 1101 / 1102, gfx1150 / 1151 RX 7900 XTX/XT, RX 7800/7700, 780M/890M APUs
RDNA4 gfx1200 / 1201 RX 9060/9070 (XT)

Not covered: RDNA1 (gfx101x — RX 5000 series) and Vega consumer variants outside gfx900/906. The arch gate23 refuses those loudly at model load (exact gfx match — HIP has no CUDA-style minor-version compat).

Multi-GPU hosts: CodeNavi runs on exactly one card; pick it with CODENAVI_GPU_DEVICE=<index> in .env (both vendors — since v1.3.21). A second install on the same machine can take another card with a different index. On Windows (Docker Desktop) the index counts as nvidia-smi -L prints it and works from package 1.3.43 on — earlier Windows packages ignored the setting there (Docker Desktop exposes every adapter through one device, so only the CUDA layer can select). On podman + NVIDIA the index has to be present in /etc/cdi/nvidia.yaml, which is a static snapshot of the cards seen when it was generated — a GPU added later is invisible until sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml is re-run (docs/podman.md). Docker enumerates cards live and needs no such step.

AMD-specific install facts (all encoded in docker-compose.rocm.yml + setup.sh): - Passthrough is two plain device nodes: --device /dev/kfd --device /dev/dri. - security_opt: label=disable is REQUIRED on SELinux-enforcing hosts (Fedora/RHEL). - group_add needs the HOST's render/video GIDs (setup.sh writes RENDER_GID/VIDEO_GID). - ROCm images bake PYTORCH_CUDA_ALLOC_CONF=expandable_segments:False — the CUDA default crashes the HIP allocator (never set it True on AMD). - Detection: a host with /dev/kfd + a /dev/dri/renderD* node and no NVIDIA GPU is treated as an AMD host by setup.sh (from-source and airgap paths).

Which arch does your card need?

Run, on the target host: 

nvidia-smi --query-gpu=name,compute_cap --format=csv,noheader
Map the compute cap via the table above. The standard package ships the universal wheel. A compute cap below 7.5 (pre-Turing NVIDIA — Volta, Pascal, Maxwell) is not supported; install CPU-only instead. For mixed fleets, per-arch images are available — one per compute cap; check each host's card and request the matching build.

For AMD hosts (no nvidia-smi): 

ls /dev/kfd /dev/dri/renderD* && lspci | grep -iE 'vga|display' | grep -i amd
If both exist, you need the ROCm build (--gpu-arch rocm) — check the card against the AMD gfx table above (RDNA1/RX 5000 is not supported).

Verification

Smoke-test on the target arch after install: 

docker exec <container> python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_capability(0))"
./codenavi search "any reasonable query" --limit 3   # must return > 0 results
A 0 results outcome on a known-good codebase usually means the wheel is wrong for the arch — request the build matching your card's compute cap.

  1. Vendor change reference CN-251. ↩

  2. Vendor change reference CN-310. ↩↩↩

  3. Vendor change reference CN-372. ↩

  4. Vendor change reference CN-380. ↩