Skip to content

Commit

Permalink
Merge branch 'main' into zkbesu
Browse files Browse the repository at this point in the history
# Conflicts:
#	.github/workflows/acceptance-tests.yml
#	.github/workflows/artifacts.yml
#	.github/workflows/codeql.yml
#	.github/workflows/docker.yml
#	.github/workflows/integration-tests.yml
#	.github/workflows/pre-review.yml
#	.github/workflows/reference-tests.yml
#	.github/workflows/release.yml
#	.github/workflows/sonarcloud.yml
#	besu/src/main/java/org/hyperledger/besu/services/TransactionSimulationServiceImpl.java
#	ethereum/core/src/main/java/org/hyperledger/besu/ethereum/transaction/CallParameter.java
  • Loading branch information
fab-10 committed Mar 13, 2024
2 parents 349fd98 + 4cc6b74 commit acfe742
Show file tree
Hide file tree
Showing 13 changed files with 221 additions and 48 deletions.
64 changes: 64 additions & 0 deletions .github/workflows/develop.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@

name: develop pre-release

on:
push:
branches:
- main
env:
GRADLE_OPTS: "-Dorg.gradle.parallel=true -Dorg.gradle.caching=true"

jobs:
artifacts:
runs-on: ubuntu-22.04
permissions:
contents: write
steps:
- name: checkout
uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11
- name: Set up JDK 17
uses: actions/setup-java@387ac29b308b003ca37ba93a6cab5eb57c8f5f93
with:
distribution: 'temurin'
java-version: '17'
- name: setup gradle
uses: gradle/actions/setup-gradle@9e899d11ad247ec76be7a60bc1cf9d3abbb9e7f1
with:
cache-disabled: true
- name: assemble release
run:
./gradlew assemble
- name: upload tarball
uses: actions/upload-artifact@5d5d22a31266ced268874388b861e4b58bb5c2f3
with:
path: 'build/distributions/besu*.tar.gz'
name: besu-develop.tar.gz
compression-level: 0
- name: hashes
id: hashes
run: |
cd build/distributions
echo "zipSha=$(shasum -a 256 besu*.zip)" >> $GITHUB_OUTPUT
echo "tarSha=$(shasum -a 256 besu*.tar.gz)" >> $GITHUB_OUTPUT
- name: upload zipfile
uses: actions/upload-artifact@5d5d22a31266ced268874388b861e4b58bb5c2f3
with:
path: 'build/distributions/besu*.zip'
name: besu-develop.zip
compression-level: 0
- name: Upload Release assets
uses: softprops/action-gh-release@de2c0eb89ae2a093876385947365aca7b0e5f844
with:
prerelease: true
name: develop
tag_name: develop
fail_on_unmatched_files: true
append_body: false
files: |
build/distributions/besu*.tar.gz
build/distributions/besu*.zip
body: |
This is an automated, bleeding edge build from the tip of ${{ github.ref_name }}. No promises. YOLO.
${{steps.hashes.outputs.tarSha}}
${{steps.hashes.outputs.zipSha}}
15 changes: 10 additions & 5 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@

### Breaking Changes
- RocksDB database metadata format has changed to be more expressive, the migration of an existing metadata file to the new format is automatic at startup. Before performing a downgrade to a previous version it is mandatory to revert to the original format using the subcommand `besu --data-path=/path/to/besu/datadir storage revert-metadata v2-to-v1`.
- PoA networks won't start with SNAP or CHECKPOINT sync (previously Besu would start with this config but quietly fail to sync, so it's now more obvious that it won't work) [#6625](https://github.com/hyperledger/besu/pull/6625)
- BFT networks won't start with SNAP or CHECKPOINT sync (previously Besu would start with this config but quietly fail to sync, so it's now more obvious that it won't work) [#6625](https://github.com/hyperledger/besu/pull/6625), [#6667](https://github.com/hyperledger/besu/pull/6667)

### Upcoming Breaking Changes

Expand All @@ -18,18 +18,18 @@
- Make layered txpool aware of minGasPrice and minPriorityFeePerGas dynamic options [#6611](https://github.com/hyperledger/besu/pull/6611)
- Update commons-compress to 1.26.0 [#6648](https://github.com/hyperledger/besu/pull/6648)
- Update Vert.x to 4.5.4 [#6666](https://github.com/hyperledger/besu/pull/6666)
- Add blob transaction support to `eth_call` [#6661](https://github.com/hyperledger/besu/pull/6661)
- Add blobs to `eth_feeHistory` [#6679](https://github.com/hyperledger/besu/pull/6679)
- Refactor and extend `TransactionPoolValidatorService` [#6636](https://github.com/hyperledger/besu/pull/6636)
- Transaction call object to accept both `input` and `data` field simultaneously if they are set to equal values [#6702](https://github.com/hyperledger/besu/pull/6702)
- Introduce `TransactionSimulationService` [#6686](https://github.com/hyperledger/besu/pull/6686)

### Bug fixes
- Fix txpool dump/restore race condition [#6665](https://github.com/hyperledger/besu/pull/6665)
- Make block transaction selection max time aware of PoA transitions [#6676](https://github.com/hyperledger/besu/pull/6676)
- Don't enable the BFT mining coordinator when running sub commands such as `blocks export` [#6675](https://github.com/hyperledger/besu/pull/6675)

### Download Links

## 24.2.0-SNAPSHOT
## 24.3.0

### Breaking Changes
- SNAP - Snap sync is now the default for named networks [#6530](https://github.com/hyperledger/besu/pull/6530)
Expand All @@ -46,7 +46,7 @@
- Release docker images now provided at ghcr.io instead of dockerhub

### Deprecations
- X_SNAP and X_CHECKPOINT are marked for deprecation and will be removed in 24.4.0 in favor of SNAP and CHECKPOINT [#6405](https://github.com/hyperledger/besu/pull/6405)
- X_SNAP and X_CHECKPOINT are marked for deprecation and will be removed in 24.6.0 in favor of SNAP and CHECKPOINT [#6405](https://github.com/hyperledger/besu/pull/6405)
- `--Xp2p-peer-lower-bound` is deprecated. [#6501](https://github.com/hyperledger/besu/pull/6501)

### Upcoming Breaking Changes
Expand All @@ -73,6 +73,8 @@
- More accurate column size `storage rocksdb usage` subcommand [#6540](https://github.com/hyperledger/besu/pull/6540)
- Adds `storage rocksdb x-stats` subcommand [#6540](https://github.com/hyperledger/besu/pull/6540)
- New `eth_blobBaseFee`JSON-RPC method [#6581](https://github.com/hyperledger/besu/pull/6581)
- Add blob transaction support to `eth_call` [#6661](https://github.com/hyperledger/besu/pull/6661)
- Add blobs to `eth_feeHistory` [#6679](https://github.com/hyperledger/besu/pull/6679)
- Upgrade reference tests to version 13.1 [#6574](https://github.com/hyperledger/besu/pull/6574)
- Extend `BesuConfiguration` service [#6584](https://github.com/hyperledger/besu/pull/6584)
- Add `ethereum_min_gas_price` and `ethereum_min_priority_fee` metrics to track runtime values of `min-gas-price` and `min-priority-fee` [#6587](https://github.com/hyperledger/besu/pull/6587)
Expand All @@ -86,6 +88,9 @@
- Fix traces so that call gas costing in traces matches other clients traces [#6525](https://github.com/hyperledger/besu/pull/6525)

### Download Links
https://github.com/hyperledger/besu/releases/tag/24.3.0
https://github.com/hyperledger/besu/releases/download/24.3.0/besu-24.3.0.tar.gz / sha256 8037ce51bb5bb396d29717a812ea7ff577b0d6aa341d67d1e5b77cbc55b15f84
https://github.com/hyperledger/besu/releases/download/24.3.0/besu-24.3.0.zip / sha256 41ea2ca734a3b377f43ee178166b5b809827084789378dbbe4e5b52bbd8e0674

## 24.1.2

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,10 @@ public void setUp() throws Exception {
strictNode = besu.createArchiveNode("strictNode", configureNode((true)));
miningNode = besu.createMinerNode("strictMiningNode", configureNode((true)));
cluster.start(lenientNode, strictNode, miningNode);
// verify all nodes are done syncing so the tx pool will be enabled
lenientNode.verify(eth.syncingStatus(false));
strictNode.verify(eth.syncingStatus(false));
miningNode.verify(eth.syncingStatus(false));
}

@Test
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,9 @@ public void setUp() throws Exception {
accountOne = accounts.createAccount("account-one");
minerWebSocket = new WebSocket(vertx, minerNode.getConfiguration());
archiveWebSocket = new WebSocket(vertx, archiveNode.getConfiguration());
// verify all nodes are done syncing so the tx pool will be enabled
archiveNode.verify(eth.syncingStatus(false));
minerNode.verify(eth.syncingStatus(false));
}

@AfterEach
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ public class TransactionSimulationServiceImpl implements TransactionSimulationSe
public TransactionSimulationServiceImpl() {}

/**
* Configure the plugin
* Configure the service
*
* @param blockchain the blockchain
* @param transactionSimulator transaction simulator
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -15,31 +15,26 @@
*/
package org.hyperledger.besu.ethereum.chain;

import static org.hyperledger.besu.ethereum.chain.BadBlockReason.DESCENDS_FROM_BAD_BLOCK;
import static org.hyperledger.besu.ethereum.chain.BadBlockReason.SPEC_VALIDATION_FAILURE;

import org.hyperledger.besu.ethereum.core.Block;

import com.google.common.base.MoreObjects;

public class BadBlockCause {
public enum BadBlockReason {
// Standard spec-related validation failures
SPEC_VALIDATION_FAILURE,
// When an unexpected exception occurs during block processing
EXCEPTIONAL_BLOCK_PROCESSING,
// This block is bad because it descends from a bad block
DESCENDS_FROM_BAD_BLOCK,
}

private final BadBlockReason reason;
private final String description;

public static BadBlockCause fromBadAncestorBlock(final Block badAncestor) {
final String description =
String.format("Descends from bad block %s", badAncestor.toLogString());
return new BadBlockCause(BadBlockReason.DESCENDS_FROM_BAD_BLOCK, description);
return new BadBlockCause(DESCENDS_FROM_BAD_BLOCK, description);
}

public static BadBlockCause fromValidationFailure(final String failureMessage) {
return new BadBlockCause(BadBlockReason.SPEC_VALIDATION_FAILURE, failureMessage);
return new BadBlockCause(SPEC_VALIDATION_FAILURE, failureMessage);
}

private BadBlockCause(final BadBlockReason reason, final String description) {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
/*
* Copyright Hyperledger Besu Contributors.
*
* Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on
* an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the
* specific language governing permissions and limitations under the License.
*
* SPDX-License-Identifier: Apache-2.0
*/
package org.hyperledger.besu.ethereum.chain;

public enum BadBlockReason {
// Standard spec-related validation failures
SPEC_VALIDATION_FAILURE,
// This block is bad because it descends from a bad block
DESCENDS_FROM_BAD_BLOCK,
}
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,9 @@

import org.hyperledger.besu.datatypes.AccessListEntry;
import org.hyperledger.besu.datatypes.Address;
import org.hyperledger.besu.datatypes.Transaction;
import org.hyperledger.besu.datatypes.VersionedHash;
import org.hyperledger.besu.datatypes.Wei;
import org.hyperledger.besu.ethereum.core.Transaction;

import java.util.List;
import java.util.Objects;
Expand Down Expand Up @@ -209,21 +209,40 @@ public String toString() {
+ ", gasLimit="
+ gasLimit
+ ", maxPriorityFeePerGas="
+ maxPriorityFeePerGas
+ maxPriorityFeePerGas.map(Wei::toHumanReadableString).orElse("N/A")
+ ", maxFeePerGas="
+ maxFeePerGas
+ maxFeePerGas.map(Wei::toHumanReadableString).orElse("N/A")
+ ", maxFeePerBlobGas="
+ maxFeePerBlobGas.map(Wei::toHumanReadableString).orElse("N/A")
+ ", gasPrice="
+ gasPrice
+ (gasPrice != null ? gasPrice.toHumanReadableString() : "N/A")
+ ", value="
+ value
+ (value != null ? value.toHumanReadableString() : "N/A")
+ ", payloadSize="
+ (payload == null ? "null" : payload.size())
+ (payload != null ? payload.size() : "null")
+ ", accessListSize="
+ accessList.map(List::size)
+ ", blobVersionedHashesSize="
+ blobVersionedHashes.map(List::size)
+ '}';
}

public static CallParameter fromTransaction(final Transaction tx) {
return new CallParameter(
tx.getSender(),
tx.getTo().orElseGet(() -> null),
tx.getGasLimit(),
tx.getGasPrice().orElse(Wei.ZERO),
tx.getMaxPriorityFeePerGas(),
tx.getMaxFeePerGas(),
tx.getValue(),
tx.getPayload(),
tx.getAccessList(),
tx.getMaxFeePerBlobGas(),
tx.getVersionedHashes());
}

public static CallParameter fromTransaction(final org.hyperledger.besu.datatypes.Transaction tx) {
return new CallParameter(
tx.getSender(),
tx.getTo().orElse(null),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -86,12 +86,20 @@ public CompletableFuture<Boolean> validatePeer(
}
final List<BlockHeader> headers = res.getResult();
if (headers.size() == 0) {
// If no headers are returned, fail
LOG.debug(
"Peer {} is invalid because required block ({}) is unavailable.",
ethPeer,
blockNumber);
return false;
if (blockIsRequired()) {
// If no headers are returned, fail
LOG.debug(
"Peer {} is invalid because required block ({}) is unavailable.",
ethPeer,
blockNumber);
return false;
} else {
LOG.debug(
"Peer {} deemed valid because unavailable block ({}) is not required.",
ethPeer,
blockNumber);
return true;
}
}
final BlockHeader header = headers.get(0);
return validateBlockHeader(ethPeer, header);
Expand All @@ -105,6 +113,10 @@ public boolean canBeValidated(final EthPeer ethPeer) {
return ethPeer.chainState().getEstimatedHeight() >= (blockNumber + chainHeightEstimationBuffer);
}

protected boolean blockIsRequired() {
return true;
}

@Override
public Duration nextValidationCheckTimeout(final EthPeer ethPeer) {
if (!ethPeer.chainState().hasEstimatedHeight()) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -49,4 +49,13 @@ boolean validateBlockHeader(final EthPeer ethPeer, final BlockHeader header) {
}
return validDaoBlock;
}

/**
* In order to support chain history pruning, clients do not need to have the dao fork block to be
* deemed valid.
*/
@Override
protected boolean blockIsRequired() {
return false;
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -64,12 +64,11 @@
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.Semaphore;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicReference;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
import java.util.function.Supplier;
import java.util.stream.Collectors;
import java.util.stream.Stream;
Expand Down Expand Up @@ -664,7 +663,7 @@ private void onAdded(final Transaction transaction) {
}

class SaveRestoreManager {
private final Lock diskAccessLock = new ReentrantLock();
private final Semaphore diskAccessLock = new Semaphore(1, true);
private final AtomicReference<CompletableFuture<Void>> writeInProgress =
new AtomicReference<>(CompletableFuture.completedFuture(null));
private final AtomicReference<CompletableFuture<Void>> readInProgress =
Expand All @@ -685,25 +684,20 @@ private CompletableFuture<Void> serializeAndDedupOperation(
final AtomicReference<CompletableFuture<Void>> operationInProgress) {
if (configuration.getEnableSaveRestore()) {
try {
if (diskAccessLock.tryLock(1, TimeUnit.MINUTES)) {
try {
if (!operationInProgress.get().isDone()) {
isCancelled.set(true);
try {
operationInProgress.get().get();
} catch (ExecutionException ee) {
// nothing to do
}
if (diskAccessLock.tryAcquire(1, TimeUnit.MINUTES)) {
if (!operationInProgress.get().isDone()) {
isCancelled.set(true);
try {
operationInProgress.get().get();
} catch (ExecutionException ee) {
// nothing to do
}

isCancelled.set(false);
operationInProgress.set(CompletableFuture.runAsync(operation));
return operationInProgress.get();
} catch (InterruptedException ie) {
isCancelled.set(false);
} finally {
diskAccessLock.unlock();
}

isCancelled.set(false);
operationInProgress.set(
CompletableFuture.runAsync(operation).thenRun(diskAccessLock::release));
return operationInProgress.get();
} else {
CompletableFuture.failedFuture(
new TimeoutException("Timeout waiting for disk access lock"));
Expand Down
Loading

0 comments on commit acfe742

Please sign in to comment.