Skip to content

Make AST & Symbol immutable to allow for multi-file support#842

Open
huydo862003 wants to merge 8 commits intomasterfrom
refactor/ast-immutable
Open

Make AST & Symbol immutable to allow for multi-file support#842
huydo862003 wants to merge 8 commits intomasterfrom
refactor/ast-immutable

Conversation

@huydo862003
Copy link
Contributor

@huydo862003 huydo862003 commented Mar 17, 2026

Doc: Link

Problem

Context: Currently, the AST & Symbol is mutable:

  • The Validator sets SyntaxNode.parent and SyntaxNode.symbol.
  • The Binder sets SyntaxNode.referees and NodeSymbol.references.

Problems:

  • Parsing is entirely local: That is, parsing only requires 1 file at a time & theoretically, the parsed AST can be reused if the file does not change.
  • Validation is entirely local: It just validates the structure of the AST and registers symbols into the symbol table, it doesn’t require information from other files.
  • Binding are global: It requires many files to resolve symbols, etc.

=> When a file changes, its binding phase reruns. Binding writes two things that span across files:

  1. Symbols onto AST nodes — including nodes belonging to other files
  2. References onto NodeSymbols — the symbol objects themselves are mutated

This means both ASTs and NodeSymbols become stale across the entire project, so everything must be discarded and re-parsed & re-validated from scratch — even files that didn't change.

The fix: make both ASTs and NodeSymbols immutable after parsing and validation, so a re-bind doesn't corrupt other files' data and unchanged files can be safely reused.

Solution:

  • Move symbols outside the AST.
  • The AST is immutable after parsing.

⇒ Allow us to reuse most ASTs if only 1 file changes.

Solution: Move symbol/referee mappings out of the AST so it becomes immutable after parsing -> ASTs can be reused across files when only one file changes.

Changes:

  • Remove symbol and referee from SyntaxNode; introduce NodeToSymbolMap and NodeToRefereeMap on Analyzer
  • Remove references from NodeSymbol; introduce NodeToReferencesMap on Analyzer
  • Add bind method to Analyzer; move parent-node assignment to the parser (after gatherInvalid)
  • Update binder, validator, interpreter, and LSP services to use the new maps

@huydo862003 huydo862003 marked this pull request as draft March 17, 2026 03:49
@huydo862003 huydo862003 force-pushed the refactor/ast-immutable branch from ff5272c to 1627dcb Compare March 17, 2026 03:53
@huydo862003 huydo862003 reopened this Mar 17, 2026
@huydo862003 huydo862003 force-pushed the refactor/ast-immutable branch 5 times, most recently from 42b4ee9 to 7b72e6e Compare March 17, 2026 07:45
@huydo862003 huydo862003 force-pushed the refactor/ast-immutable branch from 7b72e6e to 3e3e473 Compare March 17, 2026 08:05
@huydo862003 huydo862003 marked this pull request as ready for review March 17, 2026 10:16
@huydo862003 huydo862003 force-pushed the refactor/ast-immutable branch from b4e7f6a to 632bc32 Compare March 17, 2026 10:24
@huydo862003 huydo862003 changed the title Make AST immutable to allow for multi-file support Make AST & Symbol immutable to allow for multi-file support Mar 20, 2026
@github-actions
Copy link

Coverage Report

Commit: 8986732

Overall Coverage

Metric Coverage
Lines ✅ 84.18% (5450/6474)
Statements ✅ 83.49% (5815/6965)
Functions ✅ 89.44% (1271/1421)
Branches ⚠️ 75.21% (3010/4002)

Package Coverage

Package Lines Statements Functions Branches
@dbml/dbml-cli ✅ 100.00% ✅ 100.00% ✅ 100.00% ✅ 100.00%
@dbml/dbml-connector ⚠️ 64.62% ⚠️ 64.36% ⚠️ 59.53% ⚠️ 59.09%
@dbml/dbml-core N/A N/A N/A N/A
@dbml/dbml-parse ✅ 87.52% ✅ 86.73% ✅ 93.58% ⚠️ 77.68%

⚠️ Coverage Warnings

The following packages have coverage below 80%:

  • @dbml/dbml-connector: 64.62% line coverage

Files with Coverage Below 80%

@dbml/dbml-connector

9 file(s) below 80% coverage
File Lines Statements Functions Branches
src/connectors/bigquery/index.ts 0.00% 0.00% 0.00% 0.00%
src/utils/credential-loader.ts 0.00% 0.00% 0.00% 0.00%
src/utils/helpers.ts 0.00% 0.00% 0.00% 0.00%
src/connectors/snowflake/index.ts 10.56% 10.31% 0.00% 0.00%
src/utils/parseSchema.ts 46.15% 42.85% 28.57% 27.27%
src/connectors/connector.ts 66.66% 66.66% 100.00% 57.14%
src/connectors/oracle/tables.ts 71.25% 67.39% 100.00% 56.96%
src/connectors/oracle/index.ts 80.00% 80.76% 100.00% 25.00%
src/connectors/oracle/utils.ts 85.71% 85.71% 100.00% 71.42%

@dbml/dbml-parse

40 file(s) below 80% coverage
File Lines Statements Functions Branches
src/compiler/queries/container/scope.ts 0.00% 0.00% 0.00% 0.00%
src/services/diagnostics/provider.ts 0.00% 0.00% 0.00% 0.00%
src/core/interpreter/elementInterpreter/project.ts 51.42% 51.42% 100.00% 36.36%
src/core/interpreter/records/utils/data/sqlTypes.ts 56.25% 58.82% 75.00% 46.55%
src/core/analyzer/symbol/utils.ts 56.52% 56.52% 100.00% 50.00%
src/core/analyzer/analyzer.ts 61.53% 60.00% 57.14% 0.00%
src/core/analyzer/binder/elementBinder/note.ts 62.50% 64.70% 83.33% 50.00%
src/compiler/queries/utils.ts 64.83% 65.95% 88.88% 47.43%
src/core/interpreter/records/utils/data/values.ts 65.13% 57.14% 72.72% 50.37%
src/compiler/queries/token.ts 66.66% 66.66% 66.66% 100.00%
src/compiler/queries/parse.ts 70.00% 70.00% 66.66% 100.00%
src/core/parser/utils.ts 71.71% 72.43% 100.00% 78.98%
src/core/analyzer/binder/elementBinder/enum.ts 72.22% 73.68% 100.00% 62.50%
src/core/analyzer/validator/elementValidators/indexes.ts 73.33% 74.07% 90.90% 57.69%
src/core/analyzer/validator/elementValidators/note.ts 74.50% 72.72% 76.92% 68.75%
src/core/analyzer/binder/elementBinder/ref.ts 75.75% 77.14% 90.90% 70.00%
src/core/analyzer/symbol/symbolIndex.ts 76.92% 77.50% 92.85% 50.00%
src/core/analyzer/binder/elementBinder/project.ts 77.77% 78.94% 100.00% 50.00%
src/core/analyzer/validator/elementValidators/project.ts 78.12% 78.78% 100.00% 56.25%
src/core/analyzer/utils.ts 78.50% 77.06% 88.88% 77.55%
src/core/utils.ts 78.57% 77.41% 80.00% 60.71%
src/core/analyzer/validator/elementValidators/records.ts 79.45% 80.00% 93.75% 74.19%
src/core/analyzer/binder/elementBinder/indexes.ts 79.54% 78.26% 90.90% 68.18%
src/core/analyzer/validator/elementValidators/checks.ts 79.62% 81.03% 93.75% 71.87%
src/core/analyzer/binder/elementBinder/tableGroup.ts 80.00% 80.64% 100.00% 50.00%
src/core/interpreter/records/utils/constraints/pk.ts 82.00% 80.00% 92.30% 54.54%
src/services/suggestions/utils.ts 82.35% 76.78% 92.85% 70.37%
src/core/analyzer/binder/elementBinder/records.ts 82.97% 83.50% 93.75% 71.42%
src/compiler/queries/container/token.ts 83.33% 85.71% 100.00% 75.00%
src/core/analyzer/validator/elementValidators/tablePartial.ts 83.52% 80.82% 87.23% 64.28%
src/core/analyzer/binder/elementBinder/tablePartial.ts 86.00% 86.53% 100.00% 69.23%
src/core/parser/parser.ts 87.08% 87.23% 100.00% 79.25%
src/services/suggestions/recordRowSnippet.ts 89.28% 85.29% 100.00% 78.00%
src/compiler/queries/symbol.ts 92.00% 92.59% 100.00% 78.57%
__tests__/utils/compiler.ts 92.92% 92.75% 100.00% 70.32%
__tests__/utils/testHelpers.ts 93.54% 94.28% 75.00% 88.09%
src/core/interpreter/elementInterpreter/sticky_note.ts 95.00% 95.23% 100.00% 66.66%
src/core/interpreter/records/utils/constraints/unique.ts 96.15% 93.93% 100.00% 66.66%
src/core/analyzer/symbol/symbolTable.ts 100.00% 100.00% 100.00% 75.00%
src/services/definition/provider.ts 100.00% 94.44% 100.00% 70.00%

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant