Skip to content

Pre-build a CodeNavi index in the cloud

For users who want to serve CodeNavi on a Mac (or any non-GPU host) at full retrieval quality (parity with our reference GPU box). The index itself is portable — build it once on a cheap cloud GPU, ship the LanceDB files to your Mac, then run CodeNavi locally against them.

TL;DR: spin up a cheap Linux + NVIDIA GPU spot instance for ~10–30 min, run setup.sh && reindex.sh --full, tar czf index.tgz index/, scp the tarball home, terminate the instance. Total cost: typically $0.10–$0.30. The instance never sees your local search traffic — it only builds the embedding index.

When to use this

  • You're on a Mac (Apple Silicon or Intel) — Docker on Mac runs queries at full quality but can't index the embedder is CPU-only inside the Linux VM (install-mac.md)
  • You're on a Windows / Linux laptop without an NVIDIA GPU
  • Your dev box has an NVIDIA GPU but you'd rather not block it on a 6–30 min reindex
  • You want to schedule a nightly reindex without leaving a workstation on

If you have a Linux GPU host already, just run reindex there and scp the result — see setup.md. This guide is for everyone else.

What you'll have at the end

  • A directory index/lancedb/ containing the LanceDB tables (typically 100 MB–2 GB depending on codebase size)
  • Drop it into your local ~/cn-deploy/index/, set CODEBASE_CONFIG, start CodeNavi, and queries serve at full quality

The index is environment-independent. Built on Linux + CUDA, runs fine on Mac + CPU — validated end-to-end on a real Mac.1

Provider options at a glance

Provider Instance GPU Hourly Reindex cost (~10 min) Notes
AWS EC2 spot g4dn.xlarge T4 16 GB ~$0.16–0.20 ~$0.05 Cheapest mainstream cloud; spot can be reclaimed mid-build
AWS EC2 on-demand g4dn.xlarge T4 16 GB ~$0.526 ~$0.10 No reclamation risk
AWS EC2 spot g5.xlarge A10G 24 GB ~$0.30–0.40 ~$0.07 Faster, ~2× T4 throughput
GCP spot n1-standard-4 + T4 T4 16 GB ~$0.35 ~$0.07
AWS on-demand — verified g6.xlarge L4 24 GB see calculator — Full install measured: reindex 3,243 files in 1420 s — Cloud GPU VM
GCP on-demand — verified g2-standard-4 L4 24 GB see calculator — Same, 1444 s (Docker) / 1453 s (Podman, Rocky 9) — Cloud GPU VM
Azure spot Standard_NC4as_T4_v3 T4 16 GB ~$0.20–0.30 ~$0.05
Hetzner dedicated GEX44 RTX 4000 16 GB ~€0.50/hr (no spot) ~$0.10 EU; no spot pricing but no setup tax either
Lambda Labs on-demand gpu_1x_a10 A10 24 GB ~$0.50–0.75 ~$0.10 Simplest UX of the bunch
Vast.ai — verified datacenter varies A100, L40S, A6000, 4090, RTX PRO Blackwell ~$0.20–0.60 ~$0.07 Supported for a full CodeNavi deployment under EULA Addendum A — see rented-gpu.md. Turing→Blackwell OK; avoid pre-Turing (Volta/Pascal).
Vast.ai — community / spot varies 3090 / 4090 etc. ~$0.10–0.30 ~$0.05 Cheapest, but unverified hosts — not suited for proprietary code; out of scope for Addendum A

Recommendation: start with AWS g4dn.xlarge spot if you already have an AWS account. For everything else equal, Lambda Labs has the simplest setup. On Vast.ai, use only verified-datacenter offers — these are now a supported CodeNavi deployment tier under EULA Addendum A (see rented-gpu.md); avoid community/spot hosts for proprietary code, as they run on hardware you don't control with unclear data-handling guarantees.

Walkthrough — AWS g4dn.xlarge spot

Assumes you have AWS CLI configured and an SSH key pair in your target region.

1. Launch the instance

# Pick a Deep Learning AMI (Ubuntu 22.04 + NVIDIA driver + Docker pre-installed)
AMI=$(aws ec2 describe-images \
  --owners amazon \
  --filters "Name=name,Values=Deep Learning Base OSS Nvidia Driver GPU AMI (Ubuntu 22.04)*" \
  --query 'Images | sort_by(@, &CreationDate) | [-1].ImageId' --output text)

# Request a spot instance
aws ec2 request-spot-instances \
  --instance-count 1 \
  --type one-time \
  --launch-specification "{
    \"ImageId\": \"$AMI\",
    \"InstanceType\": \"g4dn.xlarge\",
    \"KeyName\": \"YOUR_KEY\",
    \"SecurityGroupIds\": [\"YOUR_SG\"],
    \"BlockDeviceMappings\": [{
      \"DeviceName\": \"/dev/sda1\",
      \"Ebs\": {\"VolumeSize\": 100, \"VolumeType\": \"gp3\"}
    }]
  }"

100 GB of EBS gp3 covers Docker images (the ~8 GB baked Buric-E4B embedder included) + your codebase + LanceDB output. Cheap (~$0.012/hr while the instance lives).

2. SSH in and install CodeNavi

ssh -i ~/.ssh/YOUR_KEY ubuntu@<instance-public-ip>

# Copy your CodeNavi install bundle up from your machine (there is no public
# git distribution — you received a signed release package)
rsync -a ~/codenavi/ ubuntu@<instance-public-ip>:~/codenavi/   # run from your laptop, or scp the release tarball and unpack it
cd ~/codenavi

# Run setup — auto-detects NVIDIA, picks GPU compose path
./setup.sh

# Verify Docker can see the GPU
docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

3. Upload your codebase

# From your laptop
rsync -az --exclude=node_modules --exclude=vendor --exclude=.git \
  ~/myapp/ ubuntu@<instance>:/home/ubuntu/codebase/

# Or have the instance clone it directly if it's a git repo
ssh ubuntu@<instance> 'git clone --depth 1 https://your-git/myapp /home/ubuntu/codebase'

Excluding node_modules, vendor, and other generated directories is critical — they bloat the index without adding signal. CodeNavi's default EXCLUDE_DIRS already filters most, but keeping the upload tight saves time.

4. Configure and reindex

ssh ubuntu@<instance>
cd ~/codenavi
# Point CODEBASE_PATH at the codebase you uploaded
sed -i 's|CODEBASE_PATH=.*|CODEBASE_PATH=/home/ubuntu/codebase|' deploy/.env

# First run starts the container — the Buric-E4B embedder is baked in; only the ~80 MB reranker downloads
TRANSFORMERS_OFFLINE=0 docker compose -f deploy/docker-compose.yml up -d
docker logs -f codenavi   # wait for "Reranker warmed up"

# Trigger the full reindex
./reindex.sh --full

# Wait until "Reindex complete" (typically 6–30 min for 30k–100k chunks on T4)
docker logs -f codenavi

5. Download the index and tear down

# On the instance
docker compose -f deploy/docker-compose.yml down
tar czf ~/index.tgz -C deploy index lancedb data/aliases_*.json 2>/dev/null
ls -lh ~/index.tgz   # typically 100 MB – 2 GB

# From your laptop
scp ubuntu@<instance>:~/index.tgz ~/Downloads/

# Terminate the spot instance (saves money)
aws ec2 terminate-instances --instance-ids <i-xxxxxxxxxxxx>

Other providers — short notes

GCP: verified image families and host steps are on Cloud GPU VM — note the Deep Learning VM image ships no Docker. Use a recent Deep Learning VM Image with NVIDIA driver ≥ 570.65 — the cu128 GA image's floor; the older common-cu124-debian-12 ships a ~550 driver and fails CUDA init. Spot/preemptible VM, attach a T4 or newer. Same workflow as AWS; replace aws ec2 with gcloud compute instances create.

Azure: pick the NCasT4_v3 or NC_T4_v3 family. Azure's "Spot" is now called "Spot VM" in the portal. Use the Ubuntu Data Science VM image to skip CUDA install.

Hetzner: provision a GEX44 (RTX 4000) dedicated server from https://hetzner.com/dedicated-rootserver. No spot pricing but no per-hour setup either — flat €0.50/hr. EU jurisdiction may matter for GDPR-conscious shops.

Lambda Labs: easiest path of all — sign up at lambdalabs.com, click "Launch instance", pick gpu_1x_a10. They give you ssh ubuntu@… immediately and the OS is preloaded with NVIDIA drivers + Docker. Same setup.sh && reindex.sh --full workflow.

Then: drop the index on your Mac

# On your Mac (or wherever CodeNavi will run)
mkdir -p ~/cn-deploy
tar xzf ~/Downloads/index.tgz -C ~/cn-deploy
ls ~/cn-deploy/lancedb   # should show *_chunks.lance directories per codebase

# Configure deploy/.env so CodeNavi sees the index
cat >> ~/cn-deploy/.env <<EOF
CODEBASE_CONFIG={"myapp":"/codebase"}
CODEBASE_PATH=/path/to/your/myapp
EOF

# Start CodeNavi (Mac: see install-mac.md for colima/Apple Container/native)
cd ~/codenavi
./mac/run-apple-container.sh up      # or your usual `docker compose up -d`
curl -s http://localhost:8765/health | python3 -m json.tool
# Expected: tables show non-zero chunks, status=ok

Privacy notes

Code chunks do leave your network during the cloud reindex — they're processed on the cloud GPU host. Mitigations:

  • Use your own cloud account — the instance, the storage, and the snapshot all live in your VPC. Customer-managed cloud is not third-party processing for most legal frameworks.
  • Use spot/ephemeral instances — terminate immediately after the index downloads. There's no persistent state on the cloud side.
  • On marketplace GPUs, verified-datacenter only. Vast.ai / RunPod verified-datacenter offers are a supported CodeNavi tier under EULA Addendum A (rented-gpu.md); their community / peer-to-peer offers run on hardware you don't control — avoid those for proprietary code.
  • Skip secrets in your codebase — CodeNavi's security scanner masks secrets and PII automatically before they're embedded (14 secret-detection + 6 PII rules among its 44 total — security model), but the rsync to the cloud instance happens before the scanner runs. Consider running git secrets --scan first if you don't already gate that in your repo.

For shops where code cannot leave the network at all: the only options are (a) a local NVIDIA GPU box (cheapest path long-term), or (b) accept the 16-point quality regression of native Mac install (details).2

Reindexing on a schedule

If your codebase changes frequently and you want the index fresh, two patterns:

  1. Periodic full reindex: cron job on the cloud instance that pulls latest, reindexes, uploads to S3, and your Mac downloads + reloads on a hook. Costs ~$0.30/run × frequency.

  2. Incremental reindex: keep the cloud instance running (~$10/month for spot T4). When code changes, rsync the diff, run ./reindex.sh (incremental — typically 5 sec), download just the updated *.lance files. Cheaper if you reindex multiple times per day.

Most prospects start with option 1 — once a week is plenty for stable codebases.

Reporting issues

Cloud-specific install issues should include: provider, instance type, GPU model (nvidia-smi -L), docker compose logs codenavi 2>&1 | tail -50, and the output of ./reindex.sh --full from the failing run.

  1. Vendor change reference CN-71. ↩

  2. Vendor change reference CN-90. ↩