Skip to content

Commit

Permalink
Add committer from-block configuration option (#104)
Browse files Browse the repository at this point in the history
### TL;DR

Added a new configuration option for the committer to specify the starting block.

### What changed?

- Introduced a new `fromBlock` parameter for the committer in the configuration.
- Updated the README to document the new `committer-from-block` option.
- Added a new command-line flag `--committer-from-block` with a default value of 0.
- Modified the `CommitterConfig` struct to include the `FromBlock` field.
- Updated the `NewCommitter` function to use the new `FromBlock` value from the committer config instead of the poller config.

### How to test?

1. Run the application with the new `--committer-from-block` flag:
   ```
   ./app --committer-from-block 20000000
   ```
2. Alternatively, set the `COMMITTER_FROMBLOCK` environment variable:
   ```
   export COMMITTER_FROMBLOCK=20000000
   ./app
   ```
3. Or update the YAML configuration file:
   ```yaml
   committer:
     fromBlock: 20000000
   ```
4. Verify that the committer starts processing blocks from the specified block number.

### Why make this change?

This change allows users to specify a custom starting block for the committer, providing more flexibility in data processing. It's particularly useful for scenarios where users want to start committing from a specific block height, rather than always starting from the genesis block or the current latest block.
  • Loading branch information
iuwqyir authored Oct 16, 2024
2 parents 15948f0 + 1d6d598 commit 0f4dd40
Show file tree
Hide file tree
Showing 4 changed files with 15 additions and 1 deletion.
11 changes: 11 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -331,6 +331,17 @@ committer:
blocksPerCommit: 1000
```

#### Committer From Block
From which block to start committing. Default is `0`.

cmd: `--committer-from-block`
env: `COMMITTER_FROMBLOCK`
yaml:
```yaml
committer:
fromBlock: 20000000
```

#### Reorg Handler
Whether to enable the reorg handler. Default is `true`.

Expand Down
2 changes: 2 additions & 0 deletions cmd/root.go
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@ func init() {
rootCmd.PersistentFlags().Bool("committer-enabled", true, "Toggle committer")
rootCmd.PersistentFlags().Int("committer-blocks-per-commit", 10, "How many blocks to commit each interval")
rootCmd.PersistentFlags().Int("committer-interval", 1000, "How often to commit blocks in milliseconds")
rootCmd.PersistentFlags().Int("committer-from-block", 0, "From which block to start committing")
rootCmd.PersistentFlags().Bool("reorgHandler-enabled", true, "Toggle reorg handler")
rootCmd.PersistentFlags().Int("reorgHandler-interval", 1000, "How often to run reorg handler in milliseconds")
rootCmd.PersistentFlags().Int("reorgHandler-blocks-per-scan", 100, "How many blocks to scan for reorgs")
Expand Down Expand Up @@ -109,6 +110,7 @@ func init() {
viper.BindPFlag("committer.enabled", rootCmd.PersistentFlags().Lookup("committer-enabled"))
viper.BindPFlag("committer.blocksPerCommit", rootCmd.PersistentFlags().Lookup("committer-blocks-per-commit"))
viper.BindPFlag("committer.interval", rootCmd.PersistentFlags().Lookup("committer-interval"))
viper.BindPFlag("committer.fromBlock", rootCmd.PersistentFlags().Lookup("committer-from-block"))
viper.BindPFlag("reorgHandler.enabled", rootCmd.PersistentFlags().Lookup("reorgHandler-enabled"))
viper.BindPFlag("reorgHandler.interval", rootCmd.PersistentFlags().Lookup("reorgHandler-interval"))
viper.BindPFlag("reorgHandler.blocksPerScan", rootCmd.PersistentFlags().Lookup("reorgHandler-blocks-per-scan"))
Expand Down
1 change: 1 addition & 0 deletions configs/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ type CommitterConfig struct {
Enabled bool `mapstructure:"enabled"`
Interval int `mapstructure:"interval"`
BlocksPerCommit int `mapstructure:"blocksPerCommit"`
FromBlock int `mapstructure:"fromBlock"`
}

type ReorgHandlerConfig struct {
Expand Down
2 changes: 1 addition & 1 deletion internal/orchestrator/committer.go
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ func NewCommitter(rpc rpc.IRPCClient, storage storage.IStorage) *Committer {
triggerIntervalMs: triggerInterval,
blocksPerCommit: blocksPerCommit,
storage: storage,
pollFromBlock: big.NewInt(int64(config.Cfg.Poller.FromBlock)),
pollFromBlock: big.NewInt(int64(config.Cfg.Committer.FromBlock)),
rpc: rpc,
}
}
Expand Down

0 comments on commit 0f4dd40

Please sign in to comment.