Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: summarizing how network upgrades impact Lotus and its key dependencies #12481

Merged
merged 12 commits into from
Sep 30, 2024

Conversation

BigLep
Copy link
Member

@BigLep BigLep commented Sep 17, 2024

This is an attempt to summarize the relationships between Lotus and its key dependencies that are affected during a network upgrade. Some of this came out of discussing version schemes in filecoin-project/filecoin-ffi#468.

Checklist

Before you mark the PR ready for review, please make sure that:

LOTUS_RELEASE_FLOW.md Outdated Show resolved Hide resolved
documentation/misc/Building_a_network_skeleton.md Outdated Show resolved Hide resolved
documentation/misc/Building_a_network_skeleton.md Outdated Show resolved Hide resolved
documentation/misc/Building_a_network_skeleton.md Outdated Show resolved Hide resolved
@BigLep
Copy link
Member Author

BigLep commented Sep 19, 2024

@Stebalien or @rvagg : @rjan90 and I were speaking and we didn't have full clarity on the filecoin-ffi → go-state-types dependency. I see filecoin-ffi has a go.mod dependency on it, but the update dependency flow that @rjan90 uses doesn't involve making a go-state-type update to filecoin-ffi. Could you please shed some light so we could get this documented?

@rvagg
Copy link
Member

rvagg commented Sep 26, 2024

filecoin-ffi uses the following types from go-state-types:

abi.ActorID
abi.ChainEpoch
abi.DealID
abi.InteractiveSealRandomness
abi.PaddedPieceSize
abi.PieceInfo
abi.PoStRandomness
abi.RegisteredAggregationProof
abi.RegisteredAggregationProof_*
abi.RegisteredPoStProof
abi.RegisteredPoStProof_*
abi.RegisteredSealProof
abi.RegisteredSealProof_*
abi.RegisteredUpdateProof
abi.RegisteredUpdateProof_*
abi.SealRandomness
abi.SectorID
abi.SectorNumber
abi.TokenAmount
abi.UnpaddedPieceSize
proof.AggregateSealVerifyProofAndInfos
proof.copy
proof.Destroy
proof.PoStProof
proof.registered_proof
proof.ReplicaUpdateInfo
proof.SealVerifyInfo
proof.SectorInfo
proof.WindowPoStVerifyInfo
proof.WinningPoStVerifyInfo
big.Int
big.NewFromGo
big.NewInt
big.NewIntUnsigned
big.Zero
network.Version

Most of the time we don't break these, so it's going to typically be safe to not upgrade filecoin-ffi's version of gst. It was important last time because we introduced a new proof type.

Probably it's a good idea to get it updated at least when we have a final go-state-types cut, but for most upgrades it may not be critical to put it in the workflow, maybe we can reduce some complexity by leaving it out until the very end when we get a final non-dev, non-rc tag. But where we are introducing new basic types, such as new proof variants, then we'll need to get it updated during the whole dev & rc cycle.

@BigLep
Copy link
Member Author

BigLep commented Sep 27, 2024

Thanks @rvagg. I'll update docs accordingly.
For my own edification, what did you do to get the list of go-state-types used in filecoin-ffi? I assume gogrep, but I was struggling as a newbie to get it to work. I wanted to make sure I was even using the right tool...

@rvagg
Copy link
Member

rvagg commented Sep 27, 2024

nothing so elegant tbh, I figured out all of the imports with a basic git grep for go-state-types and then for each I did something like git grep -E '\Wabi\.\w+' *.go */*.go | sed -E 's/^.*(abi\.\w+).*$/\1/g' | sort | uniq, a cleverer Gopher might do something more sophisticated

BigLep added a commit that referenced this pull request Sep 27, 2024
@BigLep
Copy link
Member Author

BigLep commented Sep 27, 2024

@rjan90 : I think this is good to go. I added back the ffi dependency on go-state-types.

I also did a followup PR (#12524 ) currently targeting this PR but it could switch to master after this PR merges. It includes more discussion on the ffi dependency on go-state-types plus other things I was clueing into from thinking about this more and reviewing what was done for nv24 skeleton. This also triggered some updates to https://docs.google.com/document/d/1KKJj2COb0vIqAQh-4F7fflJjxiTgGrU9oT9B461nsgg/edit which I @mentioned you on.

@BigLep BigLep merged commit 6ccb1f1 into master Sep 30, 2024
82 checks passed
@BigLep BigLep deleted the biglep/document-lotus-dependency-versions branch September 30, 2024 14:58
BigLep added a commit that referenced this pull request Sep 30, 2024
* docs: summarizing how network upgrades impact Lotus and its key dependencies

* Formatting updates

* formatting

* added diagram

* Moved diagram up

* Update LOTUS_RELEASE_FLOW.md

Co-authored-by: Phi-rjan <[email protected]>

* Update documentation/misc/Building_a_network_skeleton.md

Co-authored-by: Phi-rjan <[email protected]>

* Update documentation/misc/Building_a_network_skeleton.md

Co-authored-by: Phi-rjan <[email protected]>

* Update documentation/misc/Building_a_network_skeleton.md

Co-authored-by: Phi-rjan <[email protected]>

* Adding rust-filecoin-proofs-api and removing go-state-types dependency from ffi

* typos

* Added go-state-types back in as a dependency of ffi.

* docs: update network skeleton docs based on nv24 learnings

This includes some followups/learnings #12481.

* cleanups

* formatting

* Update documentation/misc/Building_a_network_skeleton.md

---------

Co-authored-by: Phi-rjan <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: ☑️ Done (Archive)
Development

Successfully merging this pull request may close these issues.

3 participants