feat: optional throttling when reading blocks from the chain #2966
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Alenar
temporarily deployed
to
testing-preview
GitHub Actions
Inactive
There was a problem hiding this comment.
Pull request overview
This PR adds optional throttling functionality to the CardanoBlockScanner and BlockStreamer to reduce CPU load during block import operations. The throttling ensures a minimum interval between consecutive block polls, helping to limit resource consumption on aggregator and signer infrastructure.
Changes:
- Implemented
BlockStreamerWithThrottlingdecorator to throttle block polling operations - Added
cardano_transactions_block_streamer_throttling_intervalconfiguration parameter with 400ms default - Configured throttling in both signer and aggregator dependency injection with the new parameter
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
internal/cardano-node/mithril-cardano-node-chain/src/chain_scanner/block_streamer_with_thottling.rs |
New throttling decorator implementation with comprehensive tests |
internal/cardano-node/mithril-cardano-node-chain/src/chain_scanner/block_scanner.rs |
Added throttling interval support and conditional wrapper logic |
internal/cardano-node/mithril-cardano-node-chain/src/chain_scanner/mod.rs |
Registered new throttling module |
internal/cardano-node/mithril-cardano-node-chain/Cargo.toml |
Added tokio test-util feature for paused time testing |
mithril-signer/src/configuration.rs |
Added throttling interval configuration field with 400ms default |
mithril-signer/src/dependency_injection/builder.rs |
Applied throttling interval to block scanner |
mithril-aggregator/src/configuration.rs |
Added throttling interval configuration field with 400ms default |
mithril-aggregator/src/dependency_injection/builder/enablers/cardano_node.rs |
Applied throttling interval to block scanner |
mithril-test-lab/mithril-end-to-end/src/mithril/signer.rs |
Configured 30ms throttling for faster end-to-end tests |
mithril-test-lab/mithril-end-to-end/src/mithril/aggregator.rs |
Configured 30ms throttling for faster end-to-end tests |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
internal/cardano-node/mithril-cardano-node-chain/src/chain_scanner/mod.rs
Outdated
Show resolved
Hide resolved
internal/cardano-node/mithril-cardano-node-chain/src/chain_scanner/block_scanner.rs
Outdated
Show resolved
Hide resolved
Alenar
force-pushed
the
djo/2909/chain_data_preloader_throttling
branch
from
c71c4aa to
e28e426
Compare
Alenar
temporarily deployed
to
testing-preview
GitHub Actions
Inactive
jpraynaud
approved these changes
Jan 30, 2026
internal/cardano-node/mithril-cardano-node-chain/src/chain_scanner/block_scanner.rs
Outdated
Show resolved
Hide resolved
Alenar
force-pushed
the
djo/2909/chain_data_preloader_throttling
branch
from
e28e426 to
f9aeb37
Compare
Alenar
temporarily deployed
to
testing-preview
GitHub Actions
Inactive
…igner With a default configuration value of `400ms`
Use an interval of `30ms` to keep the throttler enabled but without slowing the scenarios.
Alenar
force-pushed
the
djo/2909/chain_data_preloader_throttling
branch
from
f9aeb37 to
1f3fd68
Compare
Alenar
temporarily deployed
to
testing-preview
GitHub Actions
Inactive
* mithril-cardano-node-chain from `0.1.16` to `0.1.17` * mithril-aggregator from `0.8.16` to `0.8.17` * mithril-signer from `0.3.11` to `0.3.12` * mithril-end-to-end from `0.4.116` to `0.4.117`
Alenar
temporarily deployed
to
testing-preview
GitHub Actions
Inactive
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Content
This PR add an optional throttling ability to the
CardanoBlockScannerandBlockStreamerused to poll blocks from the chain. The goal is to reduce the aggregator and signer load on the servers when running the import process (which take more than 60% of the CPU time today).It works by configuring an interval in milliseconds, the throttler then ensure that each calls that polls an group of blocks from the
BlockStreamertake at least the given time, if the calls was slower than the interval it execute immediately.It's configured with a default value of
400msin the signer and aggregator, and setting the configuration value to0disable the throttler entirely.Pre-submit checklist
Issue(s)
Closes #2909