Skip to content

Providers

CodeNavi has two model slots, and they are not symmetric.

  • Embedding — the model that turns your code into vectors at index time and your queries into vectors at search time. This is Buric-E4B, CodeNavi's own code-specialized embedder, baked into the image and run on your GPU. It is local by default and deliberately not something you swap — see why.
  • LLM (chat) — the generative model behind the web UI's Chat screen and POST /chat. Search and every MCP tool work without it. It is pluggable: on-box (Ollama, LM Studio, vLLM, AnythingLLM) or cloud (Claude, OpenAI-compatible, Gemini, OpenRouter).

The line that matters: search never sends code anywhere. Only chat with a cloud LLM provider sends retrieved chunks off the box, and the UI says so next to the provider picker.

LLM providers

Variable Default Description
LLM_PROVIDER ollama LLM provider — on-box: ollama, lmstudio, vllm, anythingllm; cloud: claude, openai, gemini, openrouter. Each provider also reads its own model/URL/key vars (e.g. CLAUDE_MODEL, GEMINI_API_KEY, VLLM_URL) — see .env.example for the full list.
LLM_MODEL llama3.1 Model name (provider-specific, e.g. claude-sonnet-4-6, gpt-4o)
OLLAMA_URL http://host.docker.internal:11434 Ollama base URL (for LLM_PROVIDER=ollama)
ANTHROPIC_API_KEY (empty) API key for LLM_PROVIDER=claude
OPENAI_API_KEY (empty) API key for LLM_PROVIDER=openai or openrouter
OPENAI_BASE_URL (empty) OpenAI-compatible endpoint for the openai provider — set https://api.deepseek.com (DeepSeek), a Groq/Mistral/Together URL, etc.; OPENAI_API_KEY then holds that provider's key. Empty = api.openai.com. Also editable on the web UI Settings screen.1
LLM_TEMPERATURE (empty) Chat sampling temperature, 0–2, every provider (Claude caps at 1). Empty = the provider's own default. Also editable on the web UI Settings screen.2

Provider-specific keys (CLAUDE_MODEL, GEMINI_API_KEY, VLLM_URL, LMSTUDIO_URL, OPENROUTER_MODEL, …) are listed in the .env.example shipped in your package.

Chat requires a working LLM. The simplest options:

Local (Ollama): 

# Install Ollama from https://ollama.com, then:
ollama pull llama3.1
# In .env:
LLM_PROVIDER=ollama
LLM_MODEL=llama3.1

Cloud (Claude): 

# In .env:
LLM_PROVIDER=claude
LLM_MODEL=claude-sonnet-4-6
ANTHROPIC_API_KEY=your-key-here

After changing the provider or key, recreate the container. LLM settings — and the API key especially, which is environment-only (never read from settings.json or the web UI) — are read once at startup: 

docker compose up -d --force-recreate
Editing .env or switching provider in the chat UI alone won't reload the key; a stale/empty key surfaces as /chat → 500: Could not resolve authentication method.

Small-VRAM GPUs (≤ ~12 GB) + a cloud provider: chat generation is remote, so a co-resident local Ollama only competes for the embedder's VRAM and can trigger CUDA out of memory mid-chat. When using Claude/OpenAI/Gemini on such a card, stop/disable Ollama (sudo systemctl stop ollama) or set OLLAMA_KEEP_ALIVE=0. See CODENAVI_VRAM_BUDGET_GIB in Configuration.

The llm section of /health shows the current status: - "status": "ok" — LLM reachable and responding - "status": "unreachable" — provider not reachable (search still works) - "status": "error" — provider reachable but returned an error

Picking the model from the web UI

The Chat screen's rail has a provider dropdown and a provider-aware model box. Self-hosted backends (ollama, lmstudio, AnythingLLM, vLLM) and Gemini list their models live from the backend; claude / openai / openrouter show a curated, editable list, and the field is always free text so a newer model can be typed. The choice is saved to that provider's own model key (LLM_MODEL for ollama/lmstudio, CLAUDE_MODEL / OPENAI_MODEL / GEMINI_MODEL / … otherwise). API keys and base URLs are not editable in the UI — they live in .env. Details in the web UI guide.

OpenAI-compatible endpoints

The openai provider is OpenAI-compatible: set OPENAI_BASE_URL to DeepSeek, Groq, Mistral, Together or any compatible API and put that provider's key in OPENAI_API_KEY. The chat rail discloses the custom endpoint so it is visible that chat is not talking to api.openai.com.1

Embedding providers

Variable Default Description
CODE_EMBEDDING_PROVIDER local Provider for code embeddings: local, ollama, openai
CONTENT_EMBEDDING_PROVIDER local Provider for doc/web embeddings: local, ollama, openai

local is the shipped default and the only option with the published search quality. Changing the provider invalidates every vector (full reindex of every codebase), and openai POSTs every code chunk to a remote API — so the server refuses it, on save and before any reindex, unless CODENAVI_ALLOW_REMOTE_CODE_EMBEDDER=true is set in .env. Generic embedder APIs measured ~30 quality points behind Buric-E4B on code retrieval; if your host has no GPU, use a cloud reindex or a vendor-assisted remote embedder (contact support) instead of a generic API.

  1. Vendor change reference CN-535. ↩↩

  2. Vendor change reference CN-537. ↩