Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Vyper support] Add VyperCheckedContract in lib-sourcify and the new endpoint /verify/vyper in server #1760

Open
wants to merge 15 commits into
base: vyper-verification-main
Choose a base branch
from

Conversation

marcocastignoli
Copy link
Member

@marcocastignoli marcocastignoli commented Nov 28, 2024

This PR currently:

  • Implements VyperCheckedContract and SolidityCheckedContract. VyperCheckedContract generates a metadata file for Vyper contracts.
  • Implements useVyperCompiler, a function similar to useCompiler which handles dynamic download for Vyper versions (need some more work for the platform architectures), and vyper compilation.
  • verifyDeployed now takes as parameter CheckedContract, an abstract class extended by both VyperCheckedContract and SolidityCheckedContract. It skips metadata-related functions for VyperCheckedContract.
  • Stores match in all StorageServices (I still haven't checked everything in detail)

TODO:

  • - Review what's stored in the database
  • - Create a common folder for the compilers code used by both lib-sourcify and server (idk if it fits in this new feature, imo yes because it's a lot of duplicated code now that we have also vyper stuff)
  • - Rewrite platform architecture management (vyper versions for x86, arm, ...)
  • - Add documentation
  • - Handle vyper transformations (I still have to understand what we can do about it)
  • - Add more tests, and finalize lib-sourcify test
  • - Finalize /verify/vyper API errors + openapi responses
  • - Documentation, updating readme files

@manuelwedler manuelwedler self-assigned this Nov 29, 2024
Copy link
Collaborator

@manuelwedler manuelwedler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found it quite impressive how minimal the changes to lib-sourcify are for this feature.

Some general feedback:

  • Why do we need a metadata file mock for VyperCheckContract? I guess this is due to the verifyDeployed method which expects a metadata. This could be documented better in some way. Is it possible to get rid of it when we refactor the verification flow?
  • lib-sourcify README needs to be updated due to renaming of CheckedContract.

I also tried to verify a vyper contract on a local server instance. Unfortunately, it didn't work due to Vyper path not found. The server logs show that vyper was downloaded. But I cannot find the binary on my machine.

2024-12-03T11:19:24.889Z [debug] [Server] Downloaded vyper - version=0.4.0 | vyperPath=/home/manuel/.vyper-bin/vyper.0.4.0.linux | platform=linux - [traceId=db3e02b8-8701-47ad-9deb-c84b6af76241]
2024-12-03T11:19:24.889Z [info] [Server] safeHandler - errorMessage=Vyper path not found | errorStack=Error: Vyper path not found
    at useVyperCompiler (/home/manuel/Projects/sourcify/sourcify/services/server/dist/server/services/compiler/local/vyperCompiler.js:46:15)

packages/lib-sourcify/src/lib/AbstractCheckedContract.ts Outdated Show resolved Hide resolved
packages/lib-sourcify/src/lib/validation.ts Show resolved Hide resolved
packages/lib-sourcify/src/lib/IVyperCompiler.ts Outdated Show resolved Hide resolved
packages/lib-sourcify/src/lib/VyperCheckedContract.ts Outdated Show resolved Hide resolved
Comment on lines +33 to +34
const contractPath = req.body.contractPath;
const contractName = req.body.contractName;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wondered why we need both here, contractPath and contractName.
I found this in the Vyper documentation on the compiler output: The contract name will always be the file name without a suffix

Also I think we don't validate the input in any way. The user could provide any arbitrary name if we don't check that it is also what the compiler returns.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also saw that, but I was scared of finding a programmatic way to always get rid of the suffix... We always had problems dealing with filepaths and filenames. Do you think we can safely assume to remove everything after the last dot?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to check if the Vyper compiler accepts any arbitrary file suffix or only .vy files.

I am just afraid here that the contractName is just accepted from the user as input and never checked against the compiler output. So the user could bring any arbitrary name into the database.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't we just remove the contractName input field and make use of the output of the recompilation? the output should include the correct name

@@ -70,6 +72,11 @@ if (config.get("lambdaCompiler.enabled")) {

export const solc = selectedSolidityCompiler;

logger.info("Using local vyper compiler");
const vyperRepoPath =
(config.get("vyperRepo") as string) || path.join("/tmp", "vyper-repo");
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs to be documented in the server's README.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see this fixed yet

@@ -1052,4 +1052,66 @@ describe("/", function () {
"perfect",
);
});

it("should verify a vyper contract resulting in a partial match", async () => {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For consistency, I would prefer to move this test into a new test file vyper.stateless.spec.ts.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But all the tests regarding APIs for stateless verification are there... At this point we can split also all the other tests into their own file for consistency

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What I meant is that verify.stateless.spec.ts would test anything on the /verify endpoint, and vyper.stateless.spec.ts would test anything on the /very/vyper endpoint.

I think this makes sense since we have verify.stateless.handlers.ts and vyper.stateless.handlers.ts.

How would you split it?

@marcocastignoli
Copy link
Member Author

Why do we need a metadata file mock for VyperCheckContract? I guess this is due to the verifyDeployed method which expects a metadata. This could be documented better in some way. Is it possible to get rid of it when we refactor the verification flow?

The main reason why we still need the metadata here is because we want to store it in RepositoryV1/V2 to align the repository fs to solidity contracts. I think @kuzdogan had this idea first.

@marcocastignoli
Copy link
Member Author

I also tried to verify a vyper contract on a local server instance. Unfortunately, it didn't work due to Vyper path not found. The server logs show that vyper was downloaded. But I cannot find the binary on my machine.

That's probably related to the fact that I still have to work on rewriting platform architecture management (vyper versions for x86, arm, ...)

@kuzdogan
Copy link
Member

kuzdogan commented Dec 3, 2024

Why do we need a metadata file mock for VyperCheckContract? I guess this is due to the verifyDeployed method which expects a metadata. This could be documented better in some way. Is it possible to get rid of it when we refactor the verification flow?

The main reason why we still need the metadata here is because we want to store it in RepositoryV1/V2 to align the repository fs to solidity contracts. I think @kuzdogan had this idea first.

We discussed this during the call but to recap, the reason is all of the APIv1 is based on the existence of the metadata.json. Some examples:

So it's needed for backwards compatiility

Copy link
Collaborator

@manuelwedler manuelwedler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The additions until now look good. There are still some things missing from above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Sprint - Needs Review
Development

Successfully merging this pull request may close these issues.

3 participants