Supported languages and file types¶
What the indexer can process, per extension — and, just as important, what it cannot. A field report noted this page didn't exist and that its absence turned a 30-second lookup into an afternoon of forensics; here it is.
Three layers decide whether a file enters the index:
INCLUDE_EXTENSIONS(data/settings.json, merged additively with the built-in defaults) — which files the scan discovers.- The security scanner's allowlist — the hard ingestion gate. An extension
you add to
INCLUDE_EXTENSIONSthat the scanner doesn't support is skipped and the server warns at startup and reindex (validate_extension_config). - The chunker routing — decides how a file is split: AST, a dedicated chunker, or the line-based fallback.
Everything the scan skips is counted in the skip ledger — see the
Skipped: line in the reindex output and the skipped-by-reason table in
index_stats. A zero-result search cannot prove code absent in a file type
listed as skipped there; use grep_search (live filesystem) for those.
AST-chunked (tree-sitter — functions/classes/methods as units)¶
| Language | Extensions | Notes |
|---|---|---|
| PHP | .php |
|
| JavaScript | .js .jsx .mjs .cjs |
.jsx parses via the stock JS grammar |
| TypeScript | .ts .mts .cts .tsx |
.tsx uses a dedicated TSX grammar; HOC-wrapped components (forwardRef/memo) extract as named units |
| Python | .py |
|
| Bash | .sh |
|
| Go | .go |
|
| Java | .java |
|
| Ruby | .rb |
|
| Rust | .rs |
|
| C# | .cs |
|
| Fortran | .f90 .f95 .f03 .f08 .f .for |
|
| Perl/CGI | .pl .pm .t .psgi .cgi |
tree-sitter first (vendored wheel), regex + brace-balanced chunker as fallback |
| Swift | .swift |
|
| Kotlin | .kt .kts |
|
| Scala | .scala .sc |
Dedicated non-AST chunkers¶
| Type | Extensions | Chunking |
|---|---|---|
| Smarty templates | .tpl |
section boundaries |
| HTML | .html .htm |
landmark boundaries — <form> <script> <table> <section> <nav> <header> <footer>, <div> with an id/class, and ``-style comments; shares the Smarty chunker |
| SQL | .sql |
statement boundaries |
| Markdown / MDX | .md .mdx |
header sections (MDX's JSX blocks ride along as text) |
| Plain text / reST | .txt .rst |
paragraph blocks |
| External documents | .pdf .docx |
via DOCS_DIR ingestion (reindex_docs), not the code scan |
How HTML is scanned.
.html/.htmcarry thehtmllanguage, which the scanner already treats as prose — so the secret, PII and doc-scoped prompt-injection rules all apply to markup. The handful of code-scoped rules (the PHP/JS dangerous-function checks) deliberately do not run inside an inline<script>; put executable code in a.jsfile if you want those.
Line-based fallback (indexed and searchable, no structural units)¶
.css .json .ini .conf .yml .yaml .tmpl (Template Toolkit)
.ps1 .psm1 .psd1 (PowerShell)
PowerShell (CN-639). Indexed since 1.3.30, with the line-based chunker — no PowerShell grammar is vendored, so scripts split on line blocks rather than into
functionunits, the same treatment SQL and YAML get. Content search,file_searchby name and the secret/PII rules all work;find_callersand the other call-graph tools do not (see below). Existing installs need a full reindex to pick up.ps1files that were skipped before.
Not indexed¶
- Anything not listed above — e.g.
.vue,.svelte,.stub,.svg,.xml, images/fonts/binaries, and extensionless files without a shebang (Dockerfile,Makefile,LICENSE) — the indexer is extension-gated. The skip ledger names these per repo so the gap is visible, not silent. Exception (CN-551): an extensionless file whose first line is a recognized shebang (sh/bash/zsh/dash/ksh,python,perl,php,node,ruby) is indexed as that language —bin/deploywith#!/usr/bin/env bashflows through the same pipeline as a.shfile..svgis a deliberate omission, not an oversight: it is generated coordinate data (d="M12 2L2 7..."), a typical icon set is hundreds of files with almost nothing to embed, and indexing it dilutes vector results. The files stay visible tolist_directory, which reads the live filesystem. - Always blocked (secret-bearing types, never indexed regardless of
config):
.env.pem.key.p12.pfx.htpasswd.ovpn.ppk. - Oversized files — over
MAX_FILE_SIZE(default 250 KB) are skipped and counted in the ledger.
Call-graph coverage is narrower than index coverage¶
find_callers / impact_analysis / diff_impact walk a symbol graph that
only covers the AST languages above (minus Perl, phase-1). Files that are
indexed but have no extractor (SQL, templates, docs, YAML) are invisible to
the graph — the impact tools state this per response ("Graph coverage" line)
and refuse to assess targets whose file type isn't indexed at all.
Coverage also varies within a supported language. A call site is only attributed to a caller when the graph can name the function containing it, and two different things leave it unattributed:
- Top-level code. A call in a legacy PHP script, a shell script, or an
anonymous
describe(...)/it(...)test callback has no enclosing named function. Nothing is missing — there is no name to attribute it to. - A definition shape the extractor doesn't name. Declarations, class
methods and assignment-shaped definitions (
const f = () => {},exports.f = function () {}— how most modern TypeScript and every React component is written) are all named; a shape no extractor recognises loses every call in its body. Functions declared as properties of an object literal (const api = { save: () => {} }) are one such shape today.
Both leave the graph thinner than it looks, so the "Graph coverage" line
reports callers unnamed for N% of <lang> call sites once a language passes
60% of its call sites, and a reindex logs the same as a warning. Healthy
values are single digits to low tens; a large repository sitting far above
that is telling you impact_analysis walks a fraction of the real graph.
An upgrade that changes extraction reparses the call graph once, by itself, on the next reindex. Only the call graph is rebuilt — the index and the embeddings are untouched, so this costs seconds, not a reindex.