Skip to content

refactor: remove unnecessary usage of lru#3204

Open
julienrbrt wants to merge 15 commits intomainfrom
julien/cache
Open

refactor: remove unnecessary usage of lru#3204
julienrbrt wants to merge 15 commits intomainfrom
julien/cache

Conversation

@julienrbrt
Copy link
Member

@julienrbrt julienrbrt commented Mar 26, 2026

Overview

My previous PR got merged with a test commit (longer version of this). This should have been merged instead.
Same behavior.

Summary by CodeRabbit

  • Chores
    • Optimized data-availability start-height determination during sync: now derived from genesis/state and only adjusted when peer syncing is active.
    • Added a hidden configurable override to force the DA retrieval start height (logs previous and overridden values).
  • Tests
    • Updated configuration tests to expect the additional hidden option.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Mar 26, 2026

📝 Walkthrough

Walkthrough

Reworks how the DA retriever start height is determined during syncer initialization: derive baseline from genesis+persisted state, only elevate it from cache when P2P syncing is active (headerStore present and ahead), and allow an explicit override via config.DA.StartHeight.

Changes

Cohort / File(s) Summary
Syncer DA initialization
block/internal/syncing/syncer.go
Replace cache-based DA height derivation from last block with baseline-from-genesis+state; use s.cache.DaHeight() only when P2P syncing is active (s.headerStore != nil && s.headerStore.Height() > state.LastBlockHeight); add config override s.config.DA.StartHeight and log the override.
DA config flag
pkg/config/config.go
Add new DA config field StartHeight uint64 and register hidden CLI flag --evnode.da.start_height wired to it; exclude the field from YAML/mapstructure decoding; minor help-string tweak for DB flag.
Flag count test update
pkg/config/config_test.go
Increment expected flag count from 77 to 78 to account for the added hidden flag.

Sequence Diagram(s)

sequenceDiagram
    participant Syncer
    participant State
    participant Cache
    participant HeaderStore
    participant Config

    Syncer->>State: read persisted state (LastBlockHeight, DAHeight, genesis)
    Syncer->>Config: read DA.StartHeight
    alt Config.StartHeight > 0
        Syncer->>Syncer: set daHeight = Config.StartHeight
    else
        Syncer->>Syncer: baseline daHeight = genesis.DAStartHeight
        alt state.DAHeight > genesis.DAStartHeight
            Syncer->>Syncer: baseline daHeight = state.DAHeight - 1
        end
        HeaderStore->>Syncer: provide headerStore (may be nil)
        alt headerStore exists and HeaderStore.Height() > state.LastBlockHeight
            Syncer->>Cache: request cached DA height (s.cache.DaHeight())
            Cache-->>Syncer: cachedDaHeight
            Syncer->>Syncer: daHeight = max(baseline, cachedDaHeight)
        else
            Syncer->>Syncer: keep baseline daHeight
        end
    end
    Syncer->>Syncer: store s.daRetrieverHeight = daHeight
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

Suggested reviewers

  • tac0turtle

Poem

🐰 I nibble on heights from genesis and past,
I peek at the cache only when P2P's fast,
A flag tells me exactly where to start,
I hop, I store, I do my small part. 🥕✨

🚥 Pre-merge checks | ❌ 3

❌ Failed checks (2 warnings, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ⚠️ Warning The PR title 'refactor: remove unnecessary usage of lru' does not match the actual changes, which modify DA retriever height initialization logic and add a DA start height configuration flag. Update the title to accurately reflect the main changes, such as 'refactor(syncer): simplify DA retriever height initialization with config override' or similar.
Description check ❓ Inconclusive The description is minimal and lacks technical detail about the actual changes made. It only mentions replacing a test commit without explaining the purpose, rationale, or specifics of the refactoring. Expand the description to include details on why DA retriever height logic was simplified, how the config override works, and the rationale for this change.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch julien/cache

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Contributor

github-actions bot commented Mar 26, 2026

The latest Buf updates on your PR. Results from workflow CI / buf-check (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed⏩ skipped✅ passed✅ passedMar 27, 2026, 2:33 PM

@claude
Copy link
Contributor

claude bot commented Mar 26, 2026

Claude Code is working…

I'll analyze this and get back to you.

View job run

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@block/internal/syncing/syncer.go`:
- Around line 359-362: The daHeight computation uses min(state.DAHeight-1, 0)
which always yields 0; change it to compute a safe candidate from state.DAHeight
(use candidate = state.DAHeight-1 when state.DAHeight > 0, else candidate = 0)
and then set daHeight = max(s.genesis.DAStartHeight, candidate). Keep the
subsequent conditional that bumps daHeight to max(daHeight, s.cache.DaHeight())
when s.headerStore != nil && s.headerStore.Height() > state.LastBlockHeight;
update the code around s.genesis.DAStartHeight, state.DAHeight, s.headerStore,
s.cache.DaHeight(), and state.LastBlockHeight accordingly.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 08094f15-06b3-4f6a-bf87-9ba1939c5d75

📥 Commits

Reviewing files that changed from the base of the PR and between 162cda6 and 240a0dd.

📒 Files selected for processing (1)
  • block/internal/syncing/syncer.go

@codecov
Copy link

codecov bot commented Mar 26, 2026

Codecov Report

❌ Patch coverage is 42.85714% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 61.41%. Comparing base (162cda6) to head (df4d8a3).

Files with missing lines Patch % Lines
block/internal/syncing/syncer.go 27.27% 6 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3204      +/-   ##
==========================================
- Coverage   61.42%   61.41%   -0.02%     
==========================================
  Files         120      120              
  Lines       12449    12456       +7     
==========================================
+ Hits         7647     7650       +3     
- Misses       3942     3946       +4     
  Partials      860      860              
Flag Coverage Δ
combined 61.41% <42.85%> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
block/internal/syncing/syncer.go (1)

359-362: ⚠️ Potential issue | 🔴 Critical

Bug: min(state.DAHeight-1, 0) always evaluates to 0 for uint64.

This is a critical logic error that was flagged in a previous review but remains unfixed:

  • When state.DAHeight > 0: e.g., if state.DAHeight = 5, then min(4, 0) = 0
  • When state.DAHeight = 0: underflows to MaxUint64, then min(MaxUint64, 0) = 0

The result is that daHeight always equals s.genesis.DAStartHeight, completely ignoring the persisted state.DAHeight. This contradicts the comment at line 356-358 which states the intent is to use the maximum of genesis, state, and cache DA heights.

🐛 Proposed fix
 	// Set DA height to the maximum of the genesis start height, the state's DA height, and the cached DA height.
 	// The cache's DaHeight() is initialized from store metadata, so it's always correct even after cache clear.
 	// Only use cache.DaHeight() when P2P is actively syncing (headerStore has higher height than current state).
-	daHeight := max(s.genesis.DAStartHeight, min(state.DAHeight-1, 0))
+	daHeight := s.genesis.DAStartHeight
+	if state.DAHeight > 0 {
+		// Use state.DAHeight - 1 to re-fetch the last DA height in case it wasn't fully processed
+		daHeight = max(daHeight, state.DAHeight-1)
+	}
 	if s.headerStore != nil && s.headerStore.Height() > state.LastBlockHeight {
 		daHeight = max(daHeight, s.cache.DaHeight())
 	}
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@block/internal/syncing/syncer.go` around lines 359 - 362, The computation for
daHeight incorrectly uses min(state.DAHeight-1, 0) on a uint64 (always 0);
instead compute a safe state candidate by checking if state.DAHeight > 0 then
use state.DAHeight-1 else use 0, then set daHeight =
max(s.genesis.DAStartHeight, stateCandidate) and finally, if s.headerStore !=
nil && s.headerStore.Height() > state.LastBlockHeight keep the existing behavior
of daHeight = max(daHeight, s.cache.DaHeight()); update the logic around the
daHeight variable and reference s.genesis.DAStartHeight, state.DAHeight,
s.cache.DaHeight(), s.headerStore.Height(), and state.LastBlockHeight when
making the change.
🧹 Nitpick comments (1)
block/internal/syncing/syncer.go (1)

357-358: Comment may be misleading about cache behavior after clear.

The comment states "The cache's DaHeight() is initialized from store metadata, so it's always correct even after cache clear." However, based on the relevant code snippets:

  1. RestoreFromStore() (context snippet 1) only runs once at initialization
  2. maxDAHeight (context snippet 3) is updated via setMaxDAHeight() only when items are added
  3. After a cache clear during syncing, maxDAHeight can regress if no new items with higher DA heights are subsequently added

This means s.cache.DaHeight() may return a lower value than expected after a cache clear, which could cause re-fetching of already-processed DA heights. Consider clarifying the comment or verifying the intended behavior.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@block/internal/syncing/syncer.go` around lines 357 - 358, The comment
incorrectly guarantees cache.DaHeight() stays correct after a clear; because
RestoreFromStore() runs only at init and maxDAHeight is updated only via
setMaxDAHeight() when items are added, clearing the cache can leave maxDAHeight
too low and cause re-fetches. Fix by either updating the comment to state that
DaHeight may regress after a cache clear unless restored from persistent
metadata, or change the logic: on cache.Clear()/during syncing ensure
RestoreFromStore() or a persistent max DA readback is invoked (or retain/restore
maxDAHeight from store metadata) so that s.cache.DaHeight() reflects the true
maximum; reference RestoreFromStore, setMaxDAHeight, maxDAHeight and
cache.DaHeight in your change.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@pkg/config/config.go`:
- Line 570: Fix the typo in the flag help string passed to cmd.Flags().String
for FlagDBPath in config.go: change the duplicated wording "path for for node
database" to a correct phrase such as "path for node database" (or "path to node
database"); ensure the call creating the flag remains
cmd.Flags().String(FlagDBPath, def.DBPath, "<corrected help text>").

---

Duplicate comments:
In `@block/internal/syncing/syncer.go`:
- Around line 359-362: The computation for daHeight incorrectly uses
min(state.DAHeight-1, 0) on a uint64 (always 0); instead compute a safe state
candidate by checking if state.DAHeight > 0 then use state.DAHeight-1 else use
0, then set daHeight = max(s.genesis.DAStartHeight, stateCandidate) and finally,
if s.headerStore != nil && s.headerStore.Height() > state.LastBlockHeight keep
the existing behavior of daHeight = max(daHeight, s.cache.DaHeight()); update
the logic around the daHeight variable and reference s.genesis.DAStartHeight,
state.DAHeight, s.cache.DaHeight(), s.headerStore.Height(), and
state.LastBlockHeight when making the change.

---

Nitpick comments:
In `@block/internal/syncing/syncer.go`:
- Around line 357-358: The comment incorrectly guarantees cache.DaHeight() stays
correct after a clear; because RestoreFromStore() runs only at init and
maxDAHeight is updated only via setMaxDAHeight() when items are added, clearing
the cache can leave maxDAHeight too low and cause re-fetches. Fix by either
updating the comment to state that DaHeight may regress after a cache clear
unless restored from persistent metadata, or change the logic: on
cache.Clear()/during syncing ensure RestoreFromStore() or a persistent max DA
readback is invoked (or retain/restore maxDAHeight from store metadata) so that
s.cache.DaHeight() reflects the true maximum; reference RestoreFromStore,
setMaxDAHeight, maxDAHeight and cache.DaHeight in your change.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 7b99f109-c982-4c0c-a05c-190556abf106

📥 Commits

Reviewing files that changed from the base of the PR and between 240a0dd and b580bc5.

📒 Files selected for processing (3)
  • block/internal/syncing/syncer.go
  • pkg/config/config.go
  • pkg/config/config_test.go
✅ Files skipped from review due to trivial changes (1)
  • pkg/config/config_test.go

@julienrbrt julienrbrt requested a review from tac0turtle March 26, 2026 22:10
@julienrbrt julienrbrt enabled auto-merge March 26, 2026 22:10
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
pkg/config/config.go (1)

570-570: ⚠️ Potential issue | 🟡 Minor

Fix the duplicated word in the DB path help text.

--help currently prints path for for node database.

📝 Suggested fix
-	cmd.Flags().String(FlagDBPath, def.DBPath, "path for for node database")
+	cmd.Flags().String(FlagDBPath, def.DBPath, "path to node database")
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/config/config.go` at line 570, The help text for the DB path flag
contains a duplicated word; update the call to cmd.Flags().String(FlagDBPath,
def.DBPath, ...) to remove the extra "for" so the description reads "path for
node database" (or a clearer alternative like "path to node database"); edit the
string literal passed to cmd.Flags().String where FlagDBPath and def.DBPath are
used to fix the typo.
🧹 Nitpick comments (1)
block/internal/syncing/syncer.go (1)

367-374: Treat the manual DA start-height as an unsafe override and validate it.

A mistyped value here can move the follower before s.genesis.DAStartHeight or beyond the recovered watermark and silently skip historical DA heights. Please reject impossible values, and at least emit a warning when operators opt into an unsafe jump.

As per coding guidelines "Validate all inputs from external sources in Go code".

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@block/internal/syncing/syncer.go` around lines 367 - 374, The manual DA
start-height override (s.config.DA.StartHeight) must be validated: ensure
startHeight >= s.genesis.DAStartHeight and startHeight <= daHeight (the
recovered watermark) and reject (return an error) if out of that range; if the
value is in-range but not equal to the current daHeight emit a warning via
s.logger.Warn() that this is an unsafe jump/override; keep the existing info log
when accepted but do not silently accept impossible values—use s.logger to warn
and return a clear error when validation fails.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@block/internal/syncing/syncer.go`:
- Around line 356-365: The current logic uses s.cache.DaHeight() (a global max)
to bump daHeight when headerStore is ahead, which can skip DA ranges; instead,
derive the bump from the per-block included-by entry for state.LastBlockHeight
(the persisted per-block inclusion height written by the submitter) or leave
daHeight unchanged if that per-block entry is missing. Concretely: replace the
branch that uses s.cache.DaHeight() with a lookup of the included-height for
state.LastBlockHeight (or call the cache/manager API that returns the
included-by height for a specific block) and max that value into daHeight only
when that per-block value exists; otherwise do not use the cache-wide
s.cache.DaHeight(). Ensure you still respect s.genesis.DAStartHeight and the
state.DAHeight-1 logic.

---

Duplicate comments:
In `@pkg/config/config.go`:
- Line 570: The help text for the DB path flag contains a duplicated word;
update the call to cmd.Flags().String(FlagDBPath, def.DBPath, ...) to remove the
extra "for" so the description reads "path for node database" (or a clearer
alternative like "path to node database"); edit the string literal passed to
cmd.Flags().String where FlagDBPath and def.DBPath are used to fix the typo.

---

Nitpick comments:
In `@block/internal/syncing/syncer.go`:
- Around line 367-374: The manual DA start-height override
(s.config.DA.StartHeight) must be validated: ensure startHeight >=
s.genesis.DAStartHeight and startHeight <= daHeight (the recovered watermark)
and reject (return an error) if out of that range; if the value is in-range but
not equal to the current daHeight emit a warning via s.logger.Warn() that this
is an unsafe jump/override; keep the existing info log when accepted but do not
silently accept impossible values—use s.logger to warn and return a clear error
when validation fails.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 63b1d6c5-1133-4102-8ea6-6061279613ef

📥 Commits

Reviewing files that changed from the base of the PR and between b580bc5 and df4d8a3.

📒 Files selected for processing (2)
  • block/internal/syncing/syncer.go
  • pkg/config/config.go

Comment on lines 356 to 365
// Set DA height to the maximum of the genesis start height, the state's DA height, and the cached DA height.
// Use the DA height from the last executed block instead of the maximum from all blocks,
// because P2P-fetched heights may be lost on restart.
daHeight := max(s.genesis.DAStartHeight, min(state.DAHeight-1, 0))
if state.LastBlockHeight > 0 {
if lastHeaderDA, ok := s.cache.GetHeaderDAIncludedByHeight(state.LastBlockHeight); ok {
daHeight = max(daHeight, lastHeaderDA)
}
if lastDataDA, ok := s.cache.GetDataDAIncludedByHeight(state.LastBlockHeight); ok {
daHeight = max(daHeight, lastDataDA)
}
// The cache's DaHeight() is initialized from store metadata, so it's always correct even after cache clear.
// Only use cache.DaHeight() when P2P is actively syncing (headerStore has higher height than current state).
daHeight := s.genesis.DAStartHeight
if state.DAHeight > s.genesis.DAStartHeight {
daHeight = max(daHeight, state.DAHeight-1)
}
if s.headerStore != nil && s.headerStore.Height() > state.LastBlockHeight {
daHeight = max(daHeight, s.cache.DaHeight())
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Resume from the last applied block's DA height, not the cache-wide max.

block/internal/cache/manager.go:174-176 defines DaHeight() as the highest DA height ever seen across the caches, while block/internal/submitting/submitter.go:440-458 still persists per-block inclusion heights. Using the global max here can jump the retriever past untouched DA ranges whenever later submissions are already in cache metadata, and those skipped heights are no longer covered by the sequential catchup path when P2P priority hints are dropped. Please derive this bump from the included-by-height entries for state.LastBlockHeight, or leave the recovered watermark unchanged when they are missing.

🛠️ Safer resume logic
-	if s.headerStore != nil && s.headerStore.Height() > state.LastBlockHeight {
-		daHeight = max(daHeight, s.cache.DaHeight())
-	}
+	if s.headerStore != nil && s.headerStore.Height() > state.LastBlockHeight {
+		if headerDAHeight, ok := s.cache.GetHeaderDAIncludedByHeight(state.LastBlockHeight); ok {
+			daHeight = max(daHeight, headerDAHeight)
+		}
+		if dataDAHeight, ok := s.cache.GetDataDAIncludedByHeight(state.LastBlockHeight); ok {
+			daHeight = max(daHeight, dataDAHeight)
+		}
+	}
Based on learnings "DA priority heights (queued via `QueuePriorityHeight` in `block/internal/syncing/da_follower.go`) are untrusted, best-effort optimizations sourced from P2P hints. Dropping a hint on a transient fetch failure is intentional — the sequential catchup loop in `da.Subscriber` will cover every height eventually."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@block/internal/syncing/syncer.go` around lines 356 - 365, The current logic
uses s.cache.DaHeight() (a global max) to bump daHeight when headerStore is
ahead, which can skip DA ranges; instead, derive the bump from the per-block
included-by entry for state.LastBlockHeight (the persisted per-block inclusion
height written by the submitter) or leave daHeight unchanged if that per-block
entry is missing. Concretely: replace the branch that uses s.cache.DaHeight()
with a lookup of the included-height for state.LastBlockHeight (or call the
cache/manager API that returns the included-by height for a specific block) and
max that value into daHeight only when that per-block value exists; otherwise do
not use the cache-wide s.cache.DaHeight(). Ensure you still respect
s.genesis.DAStartHeight and the state.DAHeight-1 logic.

@julienrbrt julienrbrt changed the title chore(syncer): update last commit refactor: remove unnecessary usage of lru Mar 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant