feat(gleam): add full Gleam language support and Erlang FFI resolution - #1546
feat(gleam): add full Gleam language support and Erlang FFI resolution#1546zhoumao wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
CodeGraph review
Overall risk: 🟠 Medium — Gleam support lands by changing shared extraction and resolution hooks with many upstream dependents, and several of those hot paths still show NO TEST REACHES THIS.
Worth double-checking
- Shared function extraction path
- Gleam import and external-name resolution boundaries
- FFI metadata persistence across upgrade and reindex flows
What to look for in each
- Shared function extraction path —
extractFunctionnow callsthis.extractor.afterExtractFunction?.(node, funcNode, this.makeExtractorContext())in the generic path before body traversal. Because that path is shared by many languages and the graph flagsLanguageExtractor,extractFunction, andisExportedas widely depended on with no direct test reach, scrutinize whether existing extractors still emit identical nodes and edges outside Gleam. - Gleam import and external-name resolution boundaries —
resolveViaImportnow resolves Gleamimportsreferences directly to file nodes, whileisBuiltInOrExternalnow short-circuitsgleam/*imports and prelude names as external. Scrutinize same-name collision cases where a project symbol resembles a compiler-provided module or prelude constructor, because these branches decide whether resolution stops early or falls through to normal matching. - FFI metadata persistence across upgrade and reindex flows — Schema version 10 adds
unresolved_refs.metadata,rowToUnresolvedReferenceround-trips it, andresurrectRefFromDroppedEdgeonly reconstructs metadata whenffi,targetLanguage === 'erlang',module,function, and numericarityare all present. Scrutinize upgrade, sync, and edge-drop/rebuild paths to confirm older indexes and reindexed projects retain enough metadata to relink foreign-function calls.
Business rules — 2 honored · 10 not applicable
| Status | Rule | Note |
|---|---|---|
| ✔ Honored | The graph model | The schema and query changes extend unresolved_refs with nullable metadata and preserve its role as the staging table for post-extraction resolution. |
| ✔ Honored | Changelog and release contract | CHANGELOG.md gains a user-facing Gleam feature entry under the unreleased feature section, and package.json is not version-bumped. |
| — Not applicable | Impact radius (blast radius) | The diff does not modify getImpactRadius or its traversal semantics; the blast radius here is consumer data, not changed implementation. |
| — Not applicable | Risk policy: surface, don't score | No getNodeMetrics or engine-side risk-scoring logic is changed in the provided diff. |
| — Not applicable | Test selection (affected tests) | The affected-test CLI behavior in src/bin/codegraph.ts is untouched; this PR only adds tests and underlying extraction/resolution behavior. |
| — Not applicable | Surfaces | The existing CLI/MCP/library surfaces remain the same; this PR changes what those surfaces can index and resolve, not which surfaces exist. |
| — Not applicable | CodeGraph business rules | The README language-support updates do not modify the separate business-rules source of truth described by this rule. |
| — Not applicable | Installer contract | Installer behavior and __tests__/installer-targets.test.ts are untouched by this change. |
| — Not applicable | Agent interface rules | No MCP initialize behavior or agent-facing interface rules are modified in the provided diff. |
| — Not applicable | Server instructions are the single source of truth | The change does not edit src/mcp/server-instructions.ts or duplicate agent guidance into other files. |
| — Not applicable | Call budget by repo size | No explore-call budgeting code in src/mcp/tools.ts is changed. |
| — Not applicable | Source strings must exclude interpolated template literals | Although src/extraction/index.ts changes, the provided source slice shows deleted-file/edge resurrection work, not changes to extractSourceStrings or interpolated-template indexing. |
Full assessment
This PR adds full Gleam support across indexing, import resolution, constructor/type handling, and Erlang FFI linking, and it extends unresolved_refs so structured FFI metadata survives persistence and migration. The implementation is not isolated to a new parser file: it also changes shared extraction and resolution paths such as LanguageExtractor, extractFunction, import resolution, and built-in/external filtering. I do not see a concrete source-level defect in the provided slices, but those shared paths sit under broad blast radius and several are explicitly marked as having no direct test reach, so the merge should focus on regression containment rather than on a single obvious bug.
QA checklist — 4 things to verify in the running product
- cli — Create a tiny Gleam project with two modules where one imports and calls a public function from the other, run
codegraph init, then inspect callers/callees for that function. (Before this PR,.gleamfiles were not part of CodeGraph's supported-language flow, so this cross-file call graph should only appear with the new change.) - cli — Create a Gleam project that imports a compiler-provided module function while the repo also defines a same-named local helper, then inspect the caller's outgoing links. (This PR adds explicit filtering for Gleam standard-library and prelude references; with the change, the compiler-provided call should no longer falsely bind to a same-named project symbol.)
- cli — Create a Gleam wrapper annotated to call an Erlang function, add the matching Erlang module/function with the same arity, run
codegraph init, and inspect the wrapper's outgoing call edge. (The new behavior persists FFI metadata and resolves by module, function, and arity; without this PR, that cross-language edge would not resolve.) - cli — In a git-tracked Gleam project, index once, delete a tracked
.gleamfile, rerun full indexing, and then list/query that path to confirm it is gone and the run reports no file error. (This PR adds a regression test for tracked-but-deleted Gleam files during full indexing; the observable difference is clean removal instead of stale file data or an indexing error.)
Blast radius: 219 files affected beyond the diff · 1161 symbols · 133 test files selected
Tests to run:
__tests__/adaptive-explore-sizing.test.ts__tests__/android-res-exclusion.test.ts__tests__/arkts-resolution.test.ts__tests__/batched-ref-cleanup.test.ts__tests__/c-fnptr-kernel-sweep.test.ts__tests__/c-fnptr-synthesizer.test.ts__tests__/celery-dispatch-synthesizer.test.ts__tests__/cfml-inheritance-resolution.test.ts__tests__/cfml-receiver-inference.test.ts__tests__/cli-affected-paths.test.ts__tests__/cli-no-color.test.ts__tests__/cli-node-command.test.ts__tests__/cli-query-command.test.ts__tests__/closure-collection-synthesizer.test.ts__tests__/concurrent-locking.test.ts__tests__/config-secret-redaction.test.ts__tests__/context-ranking.test.ts__tests__/context.test.ts__tests__/cooperative-yield.test.ts__tests__/daemon-bind-failure.test.ts
Add tree-sitter-backed extraction for Gleam modules, symbols, imports, calls, constructors, and type references, with package-aware src/test resolution. Resolve explicit Erlang FFI targets by module, function, and arity, persisting foreign-reference metadata and extracting Erlang function arities. Handle opaque constructor visibility, prelude and external terminals, _build and deleted-file indexing, extraction invalidation, and regression coverage.
a7e2c47 to
7d29502
Compare
There was a problem hiding this comment.
CodeGraph review
Overall risk: 🟠 Medium — Gleam support lands by touching shared extractor and import-resolution paths, and several of those core helpers still show no direct test reach.
Worth double-checking
- Shared import and symbol resolution
- Database upgrade and retry paths
- Gleam external and foreign-function classification
What to look for in each
- Shared import and symbol resolution — Scrutinize the new Gleam branches in the shared resolver, especially the
gleamentry inEXTENSION_RESOLUTION, the Gleam file-edge branch inresolveViaImport, and the memoized export walk infindExportedSymbol; these sit on common resolution infrastructure, and the blast radius flagsresolveViaImportandfindExportedSymbolas core helpers with no direct test reach. - Database upgrade and retry paths — Check the version-10 migration plus all unresolved-ref readers/writers together:
CURRENT_SCHEMA_VERSIONmoved to 10, migration 10 conditionally addsmetadata TEXT, and every unresolved-ref fetch now round-tripsmetadatathroughrowToUnresolvedReference; the important thing to verify is that upgraded databases and failed-ref retry flows preserve that metadata instead of silently dropping it. - Gleam external and foreign-function classification — Review the interaction between Gleam builtin/external filtering and FFI resolution:
isBuiltInOrExternalnow short-circuits Gleam stdlib/prelude refs,gleamErlangFfiMetadataaccepts only structured Erlang-call metadata, andresolveGleamErlangFfiresolves only when exactly one Erlang function carries the matchingerlang-arity:Ndecorator.
Full assessment
This PR adds first-class Gleam indexing, import resolution, constructor/type handling, Erlang FFI linking, and persistence of unresolved-reference metadata for that new resolution path. The schema also advances to version 10 so older databases can store the new metadata column. I do not see a concrete defect in the provided source slices, but the change reaches shared extraction and resolution code paths, so regression risk comes from breadth rather than an obvious break.
QA checklist — 4 things to verify in the running product
- cli — Index a tiny Gleam package with one module importing another, then use the CLI to inspect the imported function and a custom-type constructor from the caller file. (Before this PR,
.gleamfiles were not supported end to end, so the new language should now produce searchable symbols and cross-file call/reference links that would have been missing.) - cli — Create a repo with a real
src/*.gleamfile and a generated_build/default/.../*.gleamfile, run a full index, and confirm only the real source shows up in file/symbol results. (This PR adds default exclusion for Gleam build output; reverting it would cause generated_buildfiles to be indexed.) - cli — Track a Gleam source file in git, index it, delete the file, run a full index again, and confirm the deleted file disappears from results without the run reporting an extraction error for that path. (The PR adds explicit coverage for tracked-but-deleted Gleam files during full indexing, so this behavior should differ from pre-change failure or stale-file retention.)
- cli — Index a small project containing a Gleam external declaration and a matching Erlang module, then inspect the caller/callee or explore output; repeat once with the Erlang function using the wrong arity and confirm the cross-language link disappears. (This PR introduces structured unresolved-ref metadata plus arity-based Erlang FFI resolution, so only the matching-arity case should now link across languages.)
Blast radius: 228 files affected beyond the diff · 1182 symbols · 141 test files selected
Tests to run:
__tests__/adaptive-explore-sizing.test.ts__tests__/android-res-exclusion.test.ts__tests__/arkts-resolution.test.ts__tests__/batched-ref-cleanup.test.ts__tests__/c-fnptr-kernel-sweep.test.ts__tests__/c-fnptr-synthesizer.test.ts__tests__/celery-dispatch-synthesizer.test.ts__tests__/cfml-inheritance-resolution.test.ts__tests__/cfml-receiver-inference.test.ts__tests__/cli-affected-paths.test.ts__tests__/cli-no-color.test.ts__tests__/cli-node-command.test.ts__tests__/cli-query-command.test.ts__tests__/closure-collection-synthesizer.test.ts__tests__/concurrent-locking.test.ts__tests__/config-secret-redaction.test.ts__tests__/context-ranking.test.ts__tests__/context.test.ts__tests__/cooperative-yield.test.ts__tests__/daemon-bind-failure.test.ts
Summary
.gleamfiles, including functions, external functions, constants, custom types, constructors, type aliases, imports, calls, and type references.src/andtest/, including aliases, selective imports, namespace calls, constructor visibility, opaque types, prelude terminals, and_buildexclusion.Extraction and Graph Validation on Public GitHub Repositories
9aa6f14c9732bastate=complete, zero pending references, and zero pending changes.valkyrie: 464 of 900 call edges are cross-file.yamleam: 599 of 830 call edges are cross-file.valkyrie'smonotonic_now/0resolves from Gleam tosrc/valkyrie_ffi.erlwith persisted Erlang FFI metadata.Counts are whole-index totals, not only
.gleamfiles. Validation used the local CodeGraph build from this PR: CodeGraph 1.5.0, extraction version 26, Node 24.18.0. PASS indicates a complete index with no pending references or changes; the repositories' own test suites were not executed.Automated Verification
npm run buildpasses under Node 24.18.0.git diff --checkpasses.Authorship
This PR was created with the help of OpenAI Codex. Please review the implementation and validation results before merging.