Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

examples: verified CAR file fetcher #577

Merged
merged 2 commits into from
May 24, 2024
Merged

examples: verified CAR file fetcher #577

merged 2 commits into from
May 24, 2024

Conversation

hacdias
Copy link
Member

@hacdias hacdias commented Feb 12, 2024

This adds a simple example on how you can use Boxo to verifiably download a file from a trustless gateway. It does not cover the case where we have directories. I think having this, albeit not supporting directories, is better than nothing. With time, we should convert this into its own library module.

Based on #594 to reuse some code.

@hacdias hacdias self-assigned this Feb 12, 2024
@hacdias hacdias force-pushed the example-verified-fetch branch 4 times, most recently from 4f713b7 to 1111b82 Compare March 8, 2024 12:53
@hacdias hacdias changed the title wip: verified fetch examples: verified file fetch Mar 8, 2024
Copy link

codecov bot commented Mar 8, 2024

Codecov Report

Attention: Patch coverage is 36.53846% with 33 lines in your changes are missing coverage. Please review.

Project coverage is 60.11%. Comparing base (09b0013) to head (d3384f3).
Report is 1 commits behind head on main.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #577      +/-   ##
==========================================
- Coverage   60.52%   60.11%   -0.42%     
==========================================
  Files         238      236       -2     
  Lines       29483    23543    -5940     
==========================================
- Hits        17844    14152    -3692     
+ Misses      10022     7755    -2267     
- Partials     1617     1636      +19     
Files Coverage Δ
examples/car-file-fetcher/main.go 36.53% <36.53%> (ø)

... and 196 files with indirect coverage changes

@hacdias hacdias force-pushed the example-verified-fetch branch 2 times, most recently from 7c9f147 to 8e89cad Compare March 8, 2024 13:18
@hacdias hacdias marked this pull request as ready for review March 8, 2024 13:21
@hacdias hacdias requested a review from a team as a code owner March 8, 2024 13:21
@hacdias hacdias requested a review from lidel March 8, 2024 13:21
@hacdias hacdias force-pushed the example-verified-fetch branch from 8e89cad to 879dc05 Compare March 19, 2024 14:21
@hacdias hacdias requested a review from aschmahmann March 22, 2024 13:58
examples/verified-fetch/fetcher.go Outdated Show resolved Hide resolved
examples/verified-fetch/fetcher.go Outdated Show resolved Hide resolved
examples/verified-fetch/fetcher.go Outdated Show resolved Hide resolved
examples/verified-fetch/fetcher.go Outdated Show resolved Hide resolved
@hacdias hacdias force-pushed the example-verified-fetch branch from 879dc05 to e7f12c8 Compare April 2, 2024 12:16
@hacdias hacdias changed the base branch from main to gateway-remote-blocks April 2, 2024 12:16
@hacdias hacdias force-pushed the gateway-remote-blocks branch from 2036531 to 4bef702 Compare April 9, 2024 13:39
Base automatically changed from gateway-remote-blocks to main April 10, 2024 08:19
@hacdias hacdias force-pushed the example-verified-fetch branch 2 times, most recently from 9c31fee to af564b9 Compare April 15, 2024 15:30
@hacdias hacdias marked this pull request as draft April 15, 2024 15:30
@hacdias hacdias force-pushed the example-verified-fetch branch from af564b9 to 21682c3 Compare May 6, 2024 12:28
@hacdias hacdias marked this pull request as ready for review May 6, 2024 12:28
@hacdias hacdias force-pushed the example-verified-fetch branch from 21682c3 to 5dd05ed Compare May 7, 2024 10:55
@lidel lidel force-pushed the example-verified-fetch branch from 6c759ea to d5d4571 Compare May 22, 2024 17:05
@lidel lidel changed the title examples: verified file fetch examples: verified CAR file fetcher May 22, 2024
@hacdias hacdias force-pushed the example-verified-fetch branch from d5d4571 to 140158c Compare May 23, 2024 07:49
@hacdias
Copy link
Member Author

hacdias commented May 23, 2024

@lidel @aschmahmann I'm mesmerised by how short this code snippet is, and it does more than what we had before. It is stupidly simple and works for both regular files and directories. We already had all the necessary pieces in Boxo. No need to be meddling with go-car for this example.

@hacdias hacdias force-pushed the example-verified-fetch branch 2 times, most recently from 1d8cb34 to 720a4a8 Compare May 23, 2024 08:35
@hacdias hacdias requested a review from aschmahmann May 23, 2024 09:06
Copy link
Member

@lidel lidel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @hacdias, hiding all complexity in behind boxo APIs is really nice!

Comment on lines +75 to +85
// Fetch the file or directory from the gateway. Since we're using a remote CAR
// backend gateway, this call will internally fetch a CAR file from the remote
// gateway and ensure that all blocks are present and verified.
_, file, err := backend.GetAll(context.Background(), imPath)
if err != nil {
return err
}
defer file.Close()

// Write the returned UnixFS file or directory to the file system.
return files.WriteTo(file, outputPath)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💭 sweet!

@hacdias hacdias force-pushed the example-verified-fetch branch from 5fbb2d1 to d3384f3 Compare May 24, 2024 06:17
@hacdias hacdias merged commit 1bcd545 into main May 24, 2024
12 checks passed
@hacdias hacdias deleted the example-verified-fetch branch May 24, 2024 06:19
wenyue pushed a commit to wenyue/boxo that referenced this pull request Oct 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
No open projects
Status: 🔎 In Review
Development

Successfully merging this pull request may close these issues.

3 participants