[ZCash ] Implement shard tree and shard storage #26477

cypt4 · 2024-11-11T17:24:48Z

Resolves brave/brave-browser#39314

Crate pr : #26966

Submitter Checklist:

I confirm that no security/privacy review is needed and no other type of reviews are needed, or that I have requested them
There is a ticket for my issue
Used Github auto-closing keywords in the PR description above
Wrote a good PR/commit description
Squashed any review feedback or "fixup" commits before merge, so that history is a record of what happened in the repo, not your PR
Added appropriate labels (QA/Yes or QA/No; release-notes/include or release-notes/exclude; OS/...) to the associated issue
Checked the PR locally:
- npm run test -- brave_browser_tests, npm run test -- brave_unit_tests wiki
- npm run presubmit wiki, npm run gn_check, npm run tslint
Ran git rebase master (if needed)

Reviewer Checklist:

A security review is not needed, or a link to one is included in the PR description
New files have MPL-2.0 license header
Adequate test coverage exists to prevent regressions
Major classes, functions and non-trivial code blocks are well-commented
Changes in component dependencies are properly reflected in gn
Code follows the style guide
Test plan is specified in PR before merging

After-merge Checklist:

The associated issue milestone is set to the smallest version that the
changes has landed on
All relevant documentation has been updated, for instance:

Test Plan:

bridiver · 2024-11-13T17:47:58Z

components/brave_wallet/browser/BUILD.gn

@@ -312,10 +312,14 @@ static_library("browser") {

  if (enable_orchard) {
    sources += [
+      "zcash/orchard_shard_tree_delegate_impl.cc",


these really need to move into their own BUILD.gn file in zcash directory

meaning all of the files here, not just the new ones

bridiver · 2024-11-13T17:56:14Z

components/brave_wallet/browser/zcash/rust/BUILD.gn

@@ -57,6 +70,17 @@ source_set("orchard_impl") {
  ]
 }

+source_set("shard_store") {
+  visibility = [ ":*" ]
+  sources = [ "cxx/src/shard_store.h" ]


we don't want to use cxx directories anymore and it's not clear to me why this would go somewhere other than the existing ochrard_impl target. In a separate PR we should move all the files in orchard_impl into an internal directory

Also generally speaking you shouldn't create a target in one directory that includes files in a subdirectory, especially if those are the only files it includes.

it's not clear to me why this would go somewhere other than the existing ochrard_impl target.

There will be cycle dependency otherwise rust_lib depends on shard_store.h and orchard_impl depends on rust_lib

I would use qr_code_generator as an example to follow for this which would probably mean shard_store.h moves into this directory.

it's a bit confusing that this directory is called rust and also contains cpp files, but we should probably rename it in a separate PR

Moved to the zcash/rust directory

components/brave_wallet/browser/zcash/rust/orchard_shard_tree_impl.cc

components/brave_wallet/browser/zcash/rust/cxx/src/shard_store.h

bridiver · 2024-11-13T18:16:00Z

components/brave_wallet/common/zcash_utils.h

+// Note witness is a Merkle path in the Orchard commitment tree from the
+// note to the tree root according some selected anchor(selected right border in
+// the commitment tree).
+struct OrchardNoteWitness {


This entire file is a problem, we should not just be dumping a bunch of structs in here like this. You don't necessarily need a file for each, but they should be organized in some sensible way because this file is a mess

it seems like these probably belong with https://github.com/brave/brave-core/pull/26477/files#r1840956570

bridiver · 2024-11-13T18:16:11Z

components/brave_wallet/common/zcash_utils.h

+  std::vector<uint8_t> frontier;
+};
+
+class OrchardShardTreeDelegate {


This definitely should go in its own file

Moved to separate file

components/brave_wallet/common/zcash_utils.h

components/brave_wallet/browser/zcash/rust/lib.rs

bridiver

we'll follow-up as discussed in DM

Resolves brave/brave-browser#39314 Rename ShardStoreContext Switch to using class instead of functions in lib.rs Get rid of clone in impl_result::unwrap Review fix #1 Review fix #2 Review fix#3 Use result wrappers for cpp->rust callbacks Add macro for rust result wrappers Get rid of OrchardShardTreeDelegateImpl Rename ShardTreeDelegate Merge OrchardShardTreeManager to ZCashOrchardSyncState Remove OrchardShardTreeDelegate Review fix Review fix Rename ZCashOrchardStorage Extract CxxOrchardShardTreeDelegate to the separate file Rename CXX generated structs Remove some rust-wrapper impl files Rename rust wrapper types Fix presubmit Rename rust types Review fix Review fix Review fix Rust-related types renaming Add more comments Review fix

github-actions · 2024-12-13T14:44:06Z

[puLL-Merge] - brave/brave-core@26477

Description

This PR makes significant changes to the Brave Wallet's ZCash implementation, particularly focusing on the Orchard protocol. It introduces new structures and classes for handling Orchard-specific operations, refactors existing code, and improves the overall architecture of the ZCash wallet component.

Possible Issues

The extensive refactoring may introduce subtle bugs that are not immediately apparent.
Changes to core data structures like OrchardTreeState and OrchardCheckpoint might require updates in other parts of the codebase not visible in this diff.
The introduction of new synchronization mechanisms (e.g., OrchardSyncState) may lead to potential race conditions or deadlocks if not properly managed.

Security Hotspots

Changes to the ZCashOrchardStorage class, now renamed to OrchardStorage, handle sensitive user data. Ensure proper data sanitization and encryption are maintained.
The new OrchardSyncState class manages the synchronization state, which could potentially expose user transaction history if not properly secured.
Modifications to the OrchardBlockScanner affect how blocks are scanned and processed, which is critical for maintaining an accurate view of the user's balance and transaction history.

Changes

Changes

components/brave_wallet/browser/BUILD.gn:
- Removed zcash_orchard_storage files
- Added dependency on new orchard_storage internal component
components/brave_wallet/browser/internal/BUILD.gn:
- Added new orchard_sync_state files
- Introduced test support for Orchard-related components
components/brave_wallet/browser/internal/hd_key_zip32.cc and .h:
- Renamed ExtendedSpendingKey to OrchardExtendedSpendingKey
- Updated method signatures and implementations accordingly
components/brave_wallet/browser/internal/orchard_block_scanner.cc and .h:
- Significant refactoring of the OrchardBlockScanner class
- Introduced new Result struct with additional fields
- Modified ScanBlocks method to take OrchardTreeState instead of known notes
components/brave_wallet/browser/internal/orchard_storage/:
- New directory with implementation of OrchardStorage and related types
- Introduces OrchardSyncState for managing synchronization state
components/brave_wallet/browser/zcash/rust/:
- Extensive changes to Rust bindings and implementations
- Renamed and refactored various Orchard-related structures and functions
components/brave_wallet/browser/zcash/zcash_shield_sync_service.cc and .h:
- Updated to use new OrchardSyncState instead of ZCashOrchardStorage
- Modified method signatures and implementations to work with new structures
components/brave_wallet/browser/zcash/zcash_wallet_service.cc and .h:
- Updated to use OrchardSyncState instead of ZCashOrchardStorage
- Modified method signatures and implementations accordingly
components/brave_wallet/common/zcash_utils.cc and .h:
- Removed several Orchard-related structures (moved to other files)
- Updated remaining structures and utility functions

sequenceDiagram
    participant User
    participant ZCashWalletService
    participant ZCashShieldSyncService
    participant OrchardSyncState
    participant OrchardBlockScanner
    participant OrchardStorage

    User->>ZCashWalletService: Request balance
    ZCashWalletService->>ZCashShieldSyncService: Start syncing
    ZCashShieldSyncService->>OrchardSyncState: GetAccountMeta
    OrchardSyncState->>OrchardStorage: Fetch account data
    OrchardStorage-->>OrchardSyncState: Return account data
    OrchardSyncState-->>ZCashShieldSyncService: Return account meta
    ZCashShieldSyncService->>OrchardBlockScanner: ScanBlocks
    OrchardBlockScanner->>OrchardSyncState: ApplyScanResults
    OrchardSyncState->>OrchardStorage: Update notes and state
    OrchardStorage-->>OrchardSyncState: Confirm update
    OrchardSyncState-->>ZCashShieldSyncService: Return updated state
    ZCashShieldSyncService-->>ZCashWalletService: Return balance
    ZCashWalletService-->>User: Display balance

components/brave_wallet/browser/zcash/rust/lib.rs

brave-builds · 2024-12-16T00:05:30Z

Released in v1.75.101

cypt4 requested a review from a team as a code owner November 11, 2024 17:24

github-actions bot assigned cypt4 Nov 11, 2024

github-actions bot added feature/web3/wallet feature/web3/wallet/core labels Nov 11, 2024

cypt4 force-pushed the shard_crate_impl branch from 9192b28 to a3ccf60 Compare November 11, 2024 18:53

cypt4 requested review from fmarier, a team and bridiver as code owners November 11, 2024 18:53

cypt4 force-pushed the shard_tree_impl_1 branch 3 times, most recently from 2eb3bc5 to 2d67e9d Compare November 11, 2024 20:18

cypt4 changed the title ~~Implelent shard tree and shard storage~~ [ZCash ] Implement shard tree and shard storage Nov 11, 2024

cypt4 force-pushed the shard_crate_impl branch from a3ccf60 to 284506d Compare November 12, 2024 16:52

cypt4 force-pushed the shard_tree_impl_1 branch from 2d67e9d to 6f8de17 Compare November 13, 2024 15:32

bridiver reviewed Nov 13, 2024

View reviewed changes

components/brave_wallet/browser/zcash/rust/orchard_shard_tree_impl.cc Outdated Show resolved Hide resolved

bridiver reviewed Nov 13, 2024

View reviewed changes

components/brave_wallet/browser/zcash/rust/cxx/src/shard_store.h Outdated Show resolved Hide resolved

bridiver reviewed Nov 13, 2024

View reviewed changes

components/brave_wallet/common/zcash_utils.h Outdated Show resolved Hide resolved

cypt4 mentioned this pull request Nov 13, 2024

[ZCash] Upgrade ZCashOrchardStorage for shard tree purposes #26534

Merged

24 tasks

antonok-edm reviewed Nov 15, 2024

View reviewed changes

components/brave_wallet/common/zcash_utils.h Outdated Show resolved Hide resolved

components/brave_wallet/browser/zcash/rust/lib.rs Outdated Show resolved Hide resolved

cypt4 force-pushed the shard_tree_impl_1 branch from 3964309 to 9e7ed49 Compare November 18, 2024 19:54

cypt4 requested review from bsclifton, a team and deeppandya as code owners November 18, 2024 19:54

cypt4 mentioned this pull request Dec 12, 2024

[ZCash] Review zcash/rust classes to ensure adequate test coverage brave/brave-browser#42867

Open

6 tasks

bridiver approved these changes Dec 12, 2024

View reviewed changes

cypt4 force-pushed the shard_tree_impl_1 branch from 638fc9f to 3be9e86 Compare December 13, 2024 13:21

cypt4 requested review from AlexeyBarabash, samartnik, SergeyZhukovsky and a team as code owners December 13, 2024 13:21

cypt4 changed the base branch from shard_crate_impl to master December 13, 2024 13:22

cypt4 removed request for AlexeyBarabash, samartnik, SergeyZhukovsky and a team December 13, 2024 13:22

github-actions bot added the puLL-Merge label Dec 13, 2024

github-actions bot reviewed Dec 13, 2024

View reviewed changes

github-actions bot added the needs-security-review label Dec 13, 2024

github-actions bot assigned kdenhartog and thypon Dec 13, 2024

Fix build

6617615

cypt4 force-pushed the shard_tree_impl_1 branch from 78df8ea to 6617615 Compare December 13, 2024 15:40

cypt4 merged commit f6393b6 into master Dec 13, 2024
20 checks passed

cypt4 deleted the shard_tree_impl_1 branch December 13, 2024 19:24

github-actions bot added this to the 1.75.x - Nightly milestone Dec 13, 2024

kdenhartog removed the needs-security-review label Dec 16, 2024

kdenhartog unassigned thypon and kdenhartog Dec 16, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ZCash ] Implement shard tree and shard storage #26477

[ZCash ] Implement shard tree and shard storage #26477

cypt4 commented Nov 11, 2024 •

edited

Loading

bridiver Nov 13, 2024

bridiver Nov 13, 2024

bridiver Nov 13, 2024

cypt4 Nov 15, 2024

bridiver Nov 20, 2024

bridiver Nov 20, 2024

cypt4 Nov 28, 2024

bridiver Nov 13, 2024 •

edited

Loading

bridiver Nov 13, 2024

cypt4 Nov 21, 2024

bridiver Nov 13, 2024

cypt4 Nov 20, 2024

bridiver left a comment

github-actions bot commented Dec 13, 2024

Changes

brave-builds commented Dec 16, 2024

[ZCash ] Implement shard tree and shard storage #26477

[ZCash ] Implement shard tree and shard storage #26477

Conversation

cypt4 commented Nov 11, 2024 • edited Loading

Submitter Checklist:

Reviewer Checklist:

After-merge Checklist:

Test Plan:

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bridiver Nov 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bridiver left a comment

Choose a reason for hiding this comment

github-actions bot commented Dec 13, 2024

Description

Possible Issues

Security Hotspots

Changes

brave-builds commented Dec 16, 2024

cypt4 commented Nov 11, 2024 •

edited

Loading

bridiver Nov 13, 2024 •

edited

Loading