Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: adapt StatsSigningTestingTool to work with Bytes wrapper for transactions #17144

Open
wants to merge 31 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
63edb17
refactor: consumer for system transactions
mustafauzunn Dec 19, 2024
50669f7
refactor: adapt StatsSigningTestingTool to work with Bytes wrapper fo…
IvanKavaldzhiev Dec 20, 2024
fd7222c
Merge remote-tracking branch 'origin/develop' into 16871-adapt-StatsS…
IvanKavaldzhiev Dec 20, 2024
a7d6e22
style: restore headers
IvanKavaldzhiev Dec 20, 2024
b4dcec7
Merge remote-tracking branch 'origin/16703-refactor-consumer-system-t…
IvanKavaldzhiev Dec 20, 2024
8478fd9
Merge branch 'develop' into 16703-refactor-consumer-system-transactions
mustafauzunn Dec 20, 2024
2c6d151
Merge remote-tracking branch 'origin/16703-refactor-consumer-system-t…
IvanKavaldzhiev Dec 20, 2024
b67ca58
Merge remote-tracking branch 'origin/develop' into 16871-adapt-StatsS…
IvanKavaldzhiev Dec 20, 2024
2145caa
nit: resolve PR comments
IvanKavaldzhiev Dec 20, 2024
9315b1c
nit: resolve PR comments
IvanKavaldzhiev Dec 20, 2024
c26e80d
Merge remote-tracking branch 'origin/main' into 16871-adapt-StatsSign…
IvanKavaldzhiev Dec 26, 2024
a3d2c8b
nit: resolve PR comment and change system transaction differentiation
IvanKavaldzhiev Dec 27, 2024
7520540
nit: fix build
IvanKavaldzhiev Dec 27, 2024
c82d849
nit: resolve PR comment
IvanKavaldzhiev Dec 27, 2024
dcb186e
Merge remote-tracking branch 'origin/main' into 16871-adapt-StatsSign…
IvanKavaldzhiev Dec 27, 2024
3d56d76
Merge branch 'main' into 16871-adapt-StatsSigningTestingTool-to-work-…
rbarkerSL Jan 2, 2025
4e9ef8b
Merge remote-tracking branch 'origin/main' into 16871-adapt-StatsSign…
IvanKavaldzhiev Jan 6, 2025
0cc6405
Merge remote-tracking branch 'origin/16871-adapt-StatsSigningTestingT…
IvanKavaldzhiev Jan 6, 2025
f3fd9b8
Merge remote-tracking branch 'origin/main' into 16871-adapt-StatsSign…
IvanKavaldzhiev Jan 7, 2025
09bba4d
refactor: consume converted system transaction in callback instead of…
IvanKavaldzhiev Jan 7, 2025
734ed9a
Merge remote-tracking branch 'origin/main' into 16871-adapt-StatsSign…
IvanKavaldzhiev Jan 7, 2025
90f12bc
style: spotless apply and remove unnecessary check
IvanKavaldzhiev Jan 7, 2025
c91aa49
style: spotless apply
IvanKavaldzhiev Jan 7, 2025
9877ca8
Merge remote-tracking branch 'origin/main' into 16871-adapt-StatsSign…
IvanKavaldzhiev Jan 8, 2025
e8e5eae
nit: add final declarations and comments for elaborating business logic
IvanKavaldzhiev Jan 8, 2025
24c549a
Merge remote-tracking branch 'origin/main' into 16871-adapt-StatsSign…
IvanKavaldzhiev Jan 9, 2025
8a809cf
refactor: simplify logic for distinguishing between system and applic…
IvanKavaldzhiev Jan 10, 2025
fd85b62
Merge remote-tracking branch 'origin/main' into 16871-adapt-StatsSign…
IvanKavaldzhiev Jan 10, 2025
2ab240e
nit: resolve PR comment
IvanKavaldzhiev Jan 10, 2025
289fe7c
nit: fix PR comment
IvanKavaldzhiev Jan 10, 2025
32c9803
nit: resolve PR comment
IvanKavaldzhiev Jan 10, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -1,8 +1,30 @@
// SPDX-License-Identifier: Apache-2.0
/*
* Copyright (C) 2024-2025 Hedera Hashgraph, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

plugins { id("org.hiero.gradle.module.application") }

// Remove the following line to enable all 'javac' lint checks that we have turned on by default
// and then fix the reported issues.
tasks.withType<JavaCompile>().configureEach { options.compilerArgs.add("-Xlint:-cast") }

application.mainClass = "com.swirlds.demo.stats.signing.StatsSigningTestingToolMain"

testModuleInfo {
requires("org.assertj.core")
requires("org.junit.jupiter.api")
requires("org.mockito")
requires("org.junit.jupiter.params")
}
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@
import static com.swirlds.platform.test.fixtures.state.FakeStateLifecycles.FAKE_MERKLE_STATE_LIFECYCLES;
import static com.swirlds.platform.test.fixtures.state.FakeStateLifecycles.registerMerkleStateRootClassIds;

import com.hedera.hapi.platform.event.StateSignatureTransaction;
import com.hedera.pbj.runtime.io.buffer.Bytes;
import com.swirlds.common.constructable.ClassConstructorPair;
import com.swirlds.common.constructable.ConstructableRegistry;
import com.swirlds.common.constructable.ConstructableRegistryException;
Expand All @@ -52,6 +54,7 @@
import com.swirlds.platform.system.Platform;
import com.swirlds.platform.system.SwirldMain;
import edu.umd.cs.findbugs.annotations.NonNull;
import java.nio.ByteBuffer;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;

Expand All @@ -63,23 +66,25 @@
public class StatsSigningTestingToolMain implements SwirldMain {
// the first four come from the parameters in the config.txt file

public static final String STATE_SIGNATURE_MARKER = "STATE_SIGNATURE_MARKER";
private static final Logger logger = LogManager.getLogger(StatsSigningTestingToolMain.class);

static {
try {
logger.info(STARTUP.getMarker(), "Registering StatsSigningTestingToolState with ConstructableRegistry");
ConstructableRegistry constructableRegistry = ConstructableRegistry.getInstance();
final ConstructableRegistry constructableRegistry = ConstructableRegistry.getInstance();
constructableRegistry.registerConstructable(
new ClassConstructorPair(StatsSigningTestingToolState.class, () -> {
StatsSigningTestingToolState statsSigningTestingToolState = new StatsSigningTestingToolState(
FAKE_MERKLE_STATE_LIFECYCLES,
version -> new BasicSoftwareVersion(version.major()),
() -> null);
final StatsSigningTestingToolState statsSigningTestingToolState =
new StatsSigningTestingToolState(
FAKE_MERKLE_STATE_LIFECYCLES,
version -> new BasicSoftwareVersion(version.major()),
() -> null);

Check warning on line 82 in platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolMain.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolMain.java#L81-L82

Added lines #L81 - L82 were not covered by tests
return statsSigningTestingToolState;
}));
registerMerkleStateRootClassIds();
logger.info(STARTUP.getMarker(), "StatsSigningTestingToolState is registered with ConstructableRegistry");
} catch (ConstructableRegistryException e) {
} catch (final ConstructableRegistryException e) {

Check warning on line 87 in platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolMain.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolMain.java#L87

Added line #L87 was not covered by tests
logger.error(STARTUP.getMarker(), "Failed to register StatsSigningTestingToolState", e);
throw new RuntimeException(e);
}
Expand Down Expand Up @@ -257,7 +262,7 @@

if (transPerSecToCreate > -1) { // if not unlimited (-1 means unlimited)
// ramp up the TPS to the expected value
long elapsedTime = now / MILLISECONDS_TO_NANOSECONDS - rampUpStartTimeMilliSeconds;
final long elapsedTime = now / MILLISECONDS_TO_NANOSECONDS - rampUpStartTimeMilliSeconds;

Check warning on line 265 in platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolMain.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolMain.java#L265

Added line #L265 was not covered by tests
double rampUpTPS = 0;
if (elapsedTime < TPS_RAMP_UP_WINDOW_MILLISECONDS) {
rampUpTPS = expectedTPS * elapsedTime / ((double) (TPS_RAMP_UP_WINDOW_MILLISECONDS));
Expand Down Expand Up @@ -313,4 +318,21 @@
public BasicSoftwareVersion getSoftwareVersion() {
return softwareVersion;
}

@Override
@NonNull
public Bytes encodeSystemTransaction(@NonNull final StateSignatureTransaction transaction) {
final String stateSignatureMarker = STATE_SIGNATURE_MARKER;
final int markerSize = stateSignatureMarker.length();
final byte[] parsedStateSignatureTransaction =
StateSignatureTransaction.PROTOBUF.toBytes(transaction).toByteArray();
final int bufferCapacity = Integer.BYTES + markerSize + parsedStateSignatureTransaction.length;

final ByteBuffer buffer = ByteBuffer.allocate(bufferCapacity);
buffer.putInt(markerSize);
buffer.put(stateSignatureMarker.getBytes());
buffer.put(parsedStateSignatureTransaction);

return Bytes.wrap(buffer.array());
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why put a string as a marker? a byte would be enough.
also, you can use var out = new WritableStreamingData(new ByteArrayOutputStream)) to write the marker first and then the PBJ record with StateSignatureTransaction.PROTOBUF.write(transaction, out)

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

}
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@

import com.hedera.hapi.node.base.SemanticVersion;
import com.hedera.hapi.platform.event.StateSignatureTransaction;
import com.hedera.pbj.runtime.ParseException;
import com.hedera.pbj.runtime.io.buffer.Bytes;
import com.swirlds.common.constructable.ConstructableIgnored;
import com.swirlds.common.crypto.CryptographyHolder;
import com.swirlds.common.crypto.TransactionSignature;
Expand All @@ -46,6 +48,7 @@
import com.swirlds.platform.system.transaction.ConsensusTransaction;
import com.swirlds.platform.system.transaction.Transaction;
import edu.umd.cs.findbugs.annotations.NonNull;
import java.nio.ByteBuffer;
import java.util.Arrays;
import java.util.List;
import java.util.Objects;
Expand Down Expand Up @@ -115,13 +118,29 @@
@Override
public void preHandle(
@NonNull final Event event,
@NonNull final Consumer<ScopedSystemTransaction<StateSignatureTransaction>> stateSignatureTransaction) {
@NonNull
final Consumer<ScopedSystemTransaction<StateSignatureTransaction>>
stateSignatureTransactionCallback) {
final SttTransactionPool sttTransactionPool = transactionPoolSupplier.get();
if (sttTransactionPool != null) {
event.forEachTransaction(transaction -> {
// We are not interested in pre-handling any system transactions, as they are
// specific for the platform only.We also don't want to consume deprecated
// EventTransaction.STATE_SIGNATURE_TRANSACTION system transactions in the
// callback,since it's intended to be used only for the new form of encoded system
// transactions in Bytes.Thus, we can directly skip the current
// iteration, if it processes a deprecated system transaction with the
// EventTransaction.STATE_SIGNATURE_TRANSACTION type.
if (transaction.isSystem()) {
return;
}

// We should consume in the callback the new form of system transactions in Bytes
if (areTransactionBytesSystemOnes(transaction)) {
consumeSystemTransaction(transaction, event, stateSignatureTransactionCallback);
return;
}

final TransactionSignature transactionSignature =
sttTransactionPool.expandSignatures(transaction.getApplicationTransaction());
if (transactionSignature != null) {
Expand All @@ -139,15 +158,33 @@
public void handleConsensusRound(
@NonNull final Round round,
@NonNull final PlatformStateModifier platformState,
@NonNull final Consumer<ScopedSystemTransaction<StateSignatureTransaction>> stateSignatureTransaction) {
@NonNull
final Consumer<ScopedSystemTransaction<StateSignatureTransaction>>
stateSignatureTransactionCallback) {
throwIfImmutable();
round.forEachTransaction(this::handleTransaction);

round.forEachEventTransaction((event, transaction) -> {
// We are not interested in handling any system transactions, as they are
// specific for the platform only.We also don't want to consume deprecated
// EventTransaction.STATE_SIGNATURE_TRANSACTION system transactions in the
// callback,since it's intended to be used only for the new form of encoded system
// transactions in Bytes.Thus, we can directly skip the current
// iteration, if it processes a deprecated system transaction with the
// EventTransaction.STATE_SIGNATURE_TRANSACTION type.
if (transaction.isSystem()) {
return;
}

// We should consume in the callback the new form of system transactions in Bytes
if (areTransactionBytesSystemOnes(transaction)) {
consumeSystemTransaction(transaction, event, stateSignatureTransactionCallback);
} else {
handleTransaction(transaction);
}
});
}

private void handleTransaction(final ConsensusTransaction trans) {
if (trans.isSystem()) {
return;
}
final TransactionSignature s = trans.getMetadata();

if (s != null && validateSignature(s, trans) && s.getSignatureStatus() != VerificationStatus.VALID) {
Expand Down Expand Up @@ -175,6 +212,62 @@
maybeDelay();
}

/**
* Checks if the transaction bytes are system ones.
*
* @param transaction the transaction to check
* @return true if the transaction bytes are system ones, false otherwise
*/
private boolean areTransactionBytesSystemOnes(@NonNull final Transaction transaction) {
final var transactionBytes = transaction.getApplicationTransaction();

if (transactionBytes.length() == 0) {
return false;

Check warning on line 225 in platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolState.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolState.java#L225

Added line #L225 was not covered by tests
}

final ByteBuffer wrapper = ByteBuffer.wrap(transactionBytes.toByteArray());
final var markerSize = wrapper.getInt();

final var marker = new byte[markerSize];
wrapper.get(marker);

return StatsSigningTestingToolMain.STATE_SIGNATURE_MARKER.equals(new String(marker));
}

private void consumeSystemTransaction(
@NonNull final Transaction transaction,
@NonNull final Event event,
@NonNull
final Consumer<ScopedSystemTransaction<StateSignatureTransaction>>
stateSignatureTransactionCallback) {
try {
final var stateSignatureTransaction = StateSignatureTransaction.PROTOBUF.parse(
stripSystemTransactionBytes(transaction.getApplicationTransaction()));
stateSignatureTransactionCallback.accept(new ScopedSystemTransaction<>(
event.getCreatorId(), event.getSoftwareVersion(), stateSignatureTransaction));
} catch (final ParseException e) {
logger.error("Failed to parse StateSignatureTransaction", e);

Check warning on line 249 in platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolState.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/StatsSigningTestingToolState.java#L248-L249

Added lines #L248 - L249 were not covered by tests
}
}

private Bytes stripSystemTransactionBytes(final Bytes transactionBytes) {
final byte[] transactionBytesArray = transactionBytes.toByteArray();
final ByteBuffer wrapper = ByteBuffer.wrap(transactionBytesArray);
final int transactionSize = transactionBytesArray.length;

// Get the size of the marker string
final int markerSize = wrapper.getInt();

// Get the marker itself
final var marker = new byte[markerSize];
wrapper.get(marker);

// Get the StateSignatureTransaction we are interested in
final var stateSignatureTransaction = new byte[transactionSize - (markerSize + Integer.BYTES)];
wrapper.get(stateSignatureTransaction);
return Bytes.wrap(stateSignatureTransaction);
}

private void maybeDelay() {
if (SYNTHETIC_HANDLE_TIME) {
final long start = System.nanoTime();
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
/*
* Copyright (C) 2022-2024 Hedera Hashgraph, LLC
* Copyright (C) 2022-2025 Hedera Hashgraph, LLC
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
Expand Down Expand Up @@ -212,4 +212,8 @@
}
}
}

public int getTransactionSize() {
return transactionSize;

Check warning on line 217 in platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/SttTransactionPool.java

View check run for this annotation

Codecov / codecov/patch

platform-sdk/platform-apps/tests/StatsSigningTestingTool/src/main/java/com/swirlds/demo/stats/signing/SttTransactionPool.java#L217

Added line #L217 was not covered by tests
}
}
Loading