Skip to content

Conversation

@niran
Copy link
Contributor

@niran niran commented Jan 25, 2026

Overview

Add background state trie cache warming that continuously warms caches during block building by streaming per-transaction state updates over a channel — matching reth's state root task pattern for future swapability.

Problem Statement

When the builder runs with disable_state_root: true, witness generation for the state trie uses cold caches, causing slow performance. This can contribute to missed blocks during high-load periods.

Solution

Stream per-transaction EvmState diffs to a background StateTrieWarmerTask that incrementally accumulates HashedPostState and continuously computes state roots to warm OS/DB caches. The channel-based design mirrors reth's parallel state root system (MultiProofMessage / StateHookSender) so the warming task can be swapped for the real state root task later without changing execution code.

Architecture

Execution: evm.transact() → EvmState
    ↓ (StateTrieHook — mpsc channel)
StateTrieWarmerTask: receives StateUpdate(EvmState)
    → evm_state_to_hashed_post_state() → HashedPostState
    → accumulates incrementally per-tx
    → debounced state_root_with_updates() (10ms after first tx)
    → re-schedules if new txs arrive during computation
    ↓ (on Drop)
StateTrieHook sends FinishedStateUpdates → final warming → task exits

Key types (mirrors reth's state root task interface)

  • StateTrieMessageStateUpdate(EvmState) | FinishedStateUpdates (mirrors MultiProofMessage)
  • StateTrieHook — sender wrapper with auto-finish on Drop (mirrors StateHookSender)
  • StateTrieWarmerTask — background task on spawn_blocking with continuous warming algorithm
  • evm_state_to_hashed_post_state() — converts per-tx EVM state to hashed form (copied from reth multiproof.rs)

Continuous warming algorithm

  1. Block on recv() for first StateUpdate, accumulate into HashedPostState
  2. Debounce: recv_timeout(10ms) to drain more updates
  3. Compute state_root_with_updates(accumulated.clone()) to warm caches
  4. try_recv() non-blocking drain of messages queued during computation
  5. If new messages arrived → go to 3 (skip debounce, immediate re-computation)
  6. If no new messages → go to 1 (back to blocking wait)
  7. FinishedStateUpdates at any step → final warming if needed, then exit

Lifecycle

  • Block start: create channel, spawn task with provider
  • During execution: hook sends state after each evm.transact() (before commit)
  • Block end: hook dropped → FinishedStateUpdates → task runs final warming → exits

Configuration

--flashblocks.enable-state-trie-warming  # (default: false)
FLASHBLOCKS_ENABLE_STATE_TRIE_WARMING=true

Metrics

  • base_builder_state_trie_warming_started_count — warming computations started
  • base_builder_state_trie_warming_completed_count — warming computations completed
  • base_builder_state_trie_warming_duration — duration histogram
  • base_builder_state_trie_warming_error_count — error count

Test Plan

  • Unit tests for StateTrieHook (noop, drop semantics)
  • Unit tests for StateTrieWarmerTask (completion, channel close, no-update exit, accumulation)
  • Deploy to Sepolia Alpha with FLASHBLOCKS_ENABLE_STATE_TRIE_WARMING=true
  • Validate warming metrics are recorded
  • Measure witness generation latency with warming enabled vs disabled
  • Verify no regression in flashblock build times

Files Changed

  • bin/builder/src/cli.rs — CLI flag
  • crates/builder/core/src/flashblocks/state_trie_warmer.rsStateTrieMessage, StateTrieHook, StateTrieWarmerTask, evm_state_to_hashed_post_state(), tests
  • crates/builder/core/src/flashblocks/context.rsstate_hook parameter, send_state_update() calls after each evm.transact()
  • crates/builder/core/src/flashblocks/payload.rs — channel+task creation, hook threading, removed per-flashblock start_warming calls
  • crates/builder/core/src/flashblocks/config.rsenable_state_trie_warming config field
  • crates/builder/core/src/flashblocks/mod.rs — module exports
  • crates/builder/core/src/metrics.rs — warming metrics

@cb-heimdall
Copy link
Collaborator

cb-heimdall commented Jan 25, 2026

🟡 Heimdall Review Status

Requirement Status More Info
Reviews 🟡 0/1
Denominator calculation
Show calculation
1 if user is bot 0
1 if user is external 0
2 if repo is sensitive 0
From .codeflow.yml 1
Additional review requirements
Show calculation
Max 0
0
From CODEOWNERS 0
Global minimum 0
Max 1
1
1 if commit is unverified 0
Sum 1

@github-actions
Copy link
Contributor

github-actions bot commented Feb 9, 2026

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@github-actions github-actions bot added the Stale label Feb 9, 2026
@niran niran force-pushed the builder-new-payload branch from 64c7022 to 2064044 Compare February 9, 2026 19:18
@niran niran added this to the v0.5.0 milestone Feb 9, 2026
@github-actions github-actions bot removed the Stale label Feb 10, 2026
Add background state trie cache warming feature that calculates state
roots to pre-warm state trie caches before witness generation. This
addresses performance issues when state root calculation is disabled.

When the builder runs with `disable_state_root: true`, witness generation
for the state trie uses cold caches, causing slow performance. By
proactively calculating state roots in the background (even though the
result isn't used), we warm these caches. The warming process runs
asynchronously after each flashblock, and the warm caches remain available
for fast witness generation.

- **Background warming**: State root calculation runs in background threads
  after each flashblock publishes, never blocking the main build pipeline
- **Smart throttling**: Only one warming task runs at a time using atomic
  flags; additional attempts are skipped and tracked in metrics
- **Non-interruptible**: State root calculation runs to completion once
  started, but doesn't block new FCU arrivals
- **Comprehensive metrics**: Full observability with start/complete/skip/error
  counters and duration histograms

New CLI flag and environment variable:
- `--flashblocks.enable-state-trie-warming` (default: false)
- `FLASHBLOCKS_ENABLE_STATE_TRIE_WARMING`

- **Cold cache state root**: ~500-2000ms (disk I/O intensive)
- **Warm cache state root**: ~50-200ms (CPU-bound)
- **Net improvement**: 5-10x faster witness generation
- **Use case**: Helps avoid missed blocks during high-load periods

1. Added CLI flag to FlashblocksArgs
2. Added config field to FlashblocksConfig
3. Created StateTrieWarmer module with background task spawning
4. Added 5 new metrics for observability
5. Integrated warming after each flashblock publishes
6. Warming uses spawn_blocking for CPU-intensive work

This is a DRAFT implementation to illustrate potential performance
improvements. It has NOT been tested in production and requires
thorough testing and validation before deployment.

The primary goal is to demonstrate a possible solution for avoiding
missed blocks by pre-warming state trie caches.
…teTrieWarmerTask

Replace the batch-oriented StateTrieWarmer with a channel-based
StateTrieWarmerTask that receives per-transaction EvmState updates,
matching reth's state root task pattern for future swapability.

- StateTrieMessage enum with StateUpdate/FinishedStateUpdates variants
- StateTrieHook sender wrapper with auto-finish on Drop
- StateTrieWarmerTask with continuous warming: debounced 10ms after
  first tx, re-schedules if txs arrive during computation
- evm_state_to_hashed_post_state() helper copied from reth multiproof
- Single channel+task per block instead of per-flashblock warming calls
- State updates sent after each evm.transact() before commit
@niran niran force-pushed the builder-new-payload branch from 2064044 to 53cf1c8 Compare February 12, 2026 05:21
@niran niran changed the title feat(flashblocks): add state trie cache warming for witness generation feat(builder): state trie cache warming with channel-based streaming Feb 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants