feat: Add persistence to Hardhat node and deployments #669
Assignees
Labels
enhancement
New feature or request
Comments
Merged
4 tasks
mpaya5
added a commit
that referenced
this issue
Dec 4, 2024
- Add volume mappings for deployments, artifacts and cache - Implement manual deployment file saving - Update hardhat configuration for proper deployment paths This change ensures that deployed contracts persist between container restarts by saving deployment data to mounted volumes. Note: Block persistence still requires a different solution (Ganache/Geth/Besu) as Hardhat Network doesn't support state persistence. Related issue: #669
6 tasks
6 tasks
6 tasks
This was referenced Dec 5, 2024
4 tasks
mpaya5
added a commit
that referenced
this issue
Dec 16, 2024
* feat: add hardhat node and web3 package - Add Hardhat node service with Docker configuration - Configure hardhat.config.js with local network settings - Add 20 test accounts with 10,000 ETH each - Add web3 Python package to backend requirements - Update .gitignore for Hardhat artifacts and cache * feat: add contract and try script * feat: setup contract compilation and artifact handling - Add contract compilation setup with hardhat - Configure project structure for smart contracts - Fix file path for GhostContract.json artifact - Add necessary dependencies for contract compilation - Setup directory structure for contracts and artifacts * feat: creating hardhat transactions in consensus mechanism - Moved web3 python package into backend requirements file - Added access to compiled hardhat contract in jsonrpc service - Added database migration file so that a transaction has the ghost contract address of hardhat network - When a genlayer contract is deployed then a hardhat contract is deployed, both are linked in the CurrentState table - When a genlayer write method is executed then the new transaction gets the hardhat contract from the CurrentState table - When a genlayer transaction changes from status then a rollup transaction is created on the hardhat network - todo: remove rollup transaction table, put hardhat port in env, link genlayer account to hardhat account, check for out of gas, remove prints * feat: put hardhat port in env * feat: remove rollup transactions database table * feat: free transactions on hardhat, one hardhat account * test: add hardhat test with code from the transaction_processor * fix: resolve pre-commit error * test: moved test to integration tests, upgrade web3 version in requirements as it gave an import error * feat(hardhat): add genlayer-consensus contracts and setup compilation - Add genlayer-consensus smart contracts to hardhat/contracts directory - Update hardhat.config.js to enable new code generator (viaIR: true) - Add @openzeppelin/contracts and @openzeppelin/contracts-upgradeable dependencies - Configure hardhat Docker container for contract compilation - Verify hardhat node functionality in genlayer-studio This commit sets up the smart contract development environment with the necessary dependencies and configurations to compile and deploy genlayer-consensus contracts. * feat: deleted amm_adaptive.py - Deleted amm_adaptive.py because is not using the new syntax I will upload the new amm_adaptive.py whenevir will be ready * fix: contract interaction - Fixed contract deployment state updates for frontend synchronization - Improved contract method interactions and state management - Ensured proper state updates after contract value changes * fix: pre-commit errors solved * fix: solved black pre-commit issue * feat(hardhat): add deployment persistence to hardhat node - Add volume mappings for deployments, artifacts and cache - Implement manual deployment file saving - Update hardhat configuration for proper deployment paths This change ensures that deployed contracts persist between container restarts by saving deployment data to mounted volumes. Note: Block persistence still requires a different solution (Ganache/Geth/Besu) as Hardhat Network doesn't support state persistence. Related issue: #669 * refactor(hardhat): integrate start script into Dockerfile - Removed start.sh script - Incorporated start script directly into Dockerfile for streamlined execution * fix(db): remove duplicate ghost_contract_address assignment Removes redundant assignment of ghost_contract_address in transaction_from_dict function to improve code clarity and prevent potential inconsistencies * fix(security): address SonarCloud warnings and improve test coverage - Ensure non-root user is used in Dockerfile to enhance security - Adjust permissions to avoid overly permissive access (777) on /app directories - Add tests to cover deployment scripts, ensuring proper functionality and coverage * commit: Add contract initialization and consensus service integration - Updated deploy_contracts.js to handle contract initialization after deployment - Added initial ConsensusService integration into server.py - Enhanced Dockerfile.hardhat with proper permissions and chainId handling - Started implementation of consensus_service.py with basic contract interactions Note: consensus_service.py still needs additional work for complete contract initialization and interaction methods. * feat(deploy): integrate contract deployment and backend service Integrate complete contract deployment persistence in Hardhat and create ConsensusService for contract interaction. - Add deployFixture to Hardhat deployment script to handle contract initialization - Create ConsensusService class to interact with deployed contracts from backend - Add deploy_fixture method to ConsensusService as fallback initialization option - Ensure consistent contract initialization between Hardhat and backend - Load contract artifacts and addresses in ConsensusService from deployment files The deployFixture now handles all contract initialization during deployment, while ConsensusService provides a Python interface to interact with the deployed contracts. The deploy_fixture method in ConsensusService mirrors the Hardhat initialization logic but is kept as a fallback option. * feat(hardhat): implement contract deployment flow and deployment backup system - Add consensus contract deployment flow with proper initialization sequence - Implement contract backup system for deployment persistence - Add deployment verification to prevent unnecessary redeployments - Create copy-deployments script for managing contract backups - Configure hardhat to handle contract compilation and deployment * feat(hardhat): implement contract deployment flow and deployment backup system - Add consensus contract deployment flow with proper initialization sequence - Implement contract backup system for deployment persistence - Add deployment verification to prevent unnecessary redeployments - Create copy-deployments script for managing contract backups - Configure hardhat to handle contract compilation and deployment * refactor: remove ghost_contract_address redundant references - Remove ghost_contract_address from transactions_processor to avoid duplication - Remove unused ghost_contract_address from types.py * refactor: replace hardhat-deploy with hardhat-ignition - Migrate deployment process from hardhat-deploy to hardhat-ignition - Create DeployFixture module following TransactionFlow deployment order - Update deployment tests to ensure initialization and contract connecton as TransactionFlow * refactor: remove deployFixture from consensus_service.py - Remove unused deployFixture function from consensus_service.py - Deployment now handled entirely through hardhat-ignition * refactor(consensus): clean up unused imports and methods in ConsensusService - Remove unused eth_account Account import - Remove unused _send_transaction method * refactor(consensus): clean up unused imports and methods in ConsensusService - Remove unused eth_account Account import - Remove unused _send_transaction method --------- Co-authored-by: kstroobants <[email protected]>
cristiam86
pushed a commit
that referenced
this issue
Dec 16, 2024
* feat: add hardhat node and web3 package - Add Hardhat node service with Docker configuration - Configure hardhat.config.js with local network settings - Add 20 test accounts with 10,000 ETH each - Add web3 Python package to backend requirements - Update .gitignore for Hardhat artifacts and cache * feat: add contract and try script * feat: setup contract compilation and artifact handling - Add contract compilation setup with hardhat - Configure project structure for smart contracts - Fix file path for GhostContract.json artifact - Add necessary dependencies for contract compilation - Setup directory structure for contracts and artifacts * feat: creating hardhat transactions in consensus mechanism - Moved web3 python package into backend requirements file - Added access to compiled hardhat contract in jsonrpc service - Added database migration file so that a transaction has the ghost contract address of hardhat network - When a genlayer contract is deployed then a hardhat contract is deployed, both are linked in the CurrentState table - When a genlayer write method is executed then the new transaction gets the hardhat contract from the CurrentState table - When a genlayer transaction changes from status then a rollup transaction is created on the hardhat network - todo: remove rollup transaction table, put hardhat port in env, link genlayer account to hardhat account, check for out of gas, remove prints * feat: put hardhat port in env * feat: remove rollup transactions database table * feat: free transactions on hardhat, one hardhat account * test: add hardhat test with code from the transaction_processor * fix: resolve pre-commit error * test: moved test to integration tests, upgrade web3 version in requirements as it gave an import error * feat(hardhat): add genlayer-consensus contracts and setup compilation - Add genlayer-consensus smart contracts to hardhat/contracts directory - Update hardhat.config.js to enable new code generator (viaIR: true) - Add @openzeppelin/contracts and @openzeppelin/contracts-upgradeable dependencies - Configure hardhat Docker container for contract compilation - Verify hardhat node functionality in genlayer-studio This commit sets up the smart contract development environment with the necessary dependencies and configurations to compile and deploy genlayer-consensus contracts. * feat: deleted amm_adaptive.py - Deleted amm_adaptive.py because is not using the new syntax I will upload the new amm_adaptive.py whenevir will be ready * fix: contract interaction - Fixed contract deployment state updates for frontend synchronization - Improved contract method interactions and state management - Ensured proper state updates after contract value changes * fix: pre-commit errors solved * fix: solved black pre-commit issue * feat(hardhat): add deployment persistence to hardhat node - Add volume mappings for deployments, artifacts and cache - Implement manual deployment file saving - Update hardhat configuration for proper deployment paths This change ensures that deployed contracts persist between container restarts by saving deployment data to mounted volumes. Note: Block persistence still requires a different solution (Ganache/Geth/Besu) as Hardhat Network doesn't support state persistence. Related issue: #669 * refactor(hardhat): integrate start script into Dockerfile - Removed start.sh script - Incorporated start script directly into Dockerfile for streamlined execution * fix(db): remove duplicate ghost_contract_address assignment Removes redundant assignment of ghost_contract_address in transaction_from_dict function to improve code clarity and prevent potential inconsistencies * fix(security): address SonarCloud warnings and improve test coverage - Ensure non-root user is used in Dockerfile to enhance security - Adjust permissions to avoid overly permissive access (777) on /app directories - Add tests to cover deployment scripts, ensuring proper functionality and coverage * commit: Add contract initialization and consensus service integration - Updated deploy_contracts.js to handle contract initialization after deployment - Added initial ConsensusService integration into server.py - Enhanced Dockerfile.hardhat with proper permissions and chainId handling - Started implementation of consensus_service.py with basic contract interactions Note: consensus_service.py still needs additional work for complete contract initialization and interaction methods. * feat(deploy): integrate contract deployment and backend service Integrate complete contract deployment persistence in Hardhat and create ConsensusService for contract interaction. - Add deployFixture to Hardhat deployment script to handle contract initialization - Create ConsensusService class to interact with deployed contracts from backend - Add deploy_fixture method to ConsensusService as fallback initialization option - Ensure consistent contract initialization between Hardhat and backend - Load contract artifacts and addresses in ConsensusService from deployment files The deployFixture now handles all contract initialization during deployment, while ConsensusService provides a Python interface to interact with the deployed contracts. The deploy_fixture method in ConsensusService mirrors the Hardhat initialization logic but is kept as a fallback option. * feat(hardhat): implement contract deployment flow and deployment backup system - Add consensus contract deployment flow with proper initialization sequence - Implement contract backup system for deployment persistence - Add deployment verification to prevent unnecessary redeployments - Create copy-deployments script for managing contract backups - Configure hardhat to handle contract compilation and deployment * feat(hardhat): implement contract deployment flow and deployment backup system - Add consensus contract deployment flow with proper initialization sequence - Implement contract backup system for deployment persistence - Add deployment verification to prevent unnecessary redeployments - Create copy-deployments script for managing contract backups - Configure hardhat to handle contract compilation and deployment * refactor: remove ghost_contract_address redundant references - Remove ghost_contract_address from transactions_processor to avoid duplication - Remove unused ghost_contract_address from types.py * refactor: replace hardhat-deploy with hardhat-ignition - Migrate deployment process from hardhat-deploy to hardhat-ignition - Create DeployFixture module following TransactionFlow deployment order - Update deployment tests to ensure initialization and contract connecton as TransactionFlow * refactor: remove deployFixture from consensus_service.py - Remove unused deployFixture function from consensus_service.py - Deployment now handled entirely through hardhat-ignition * feat(rpc): add eth_getContract endpoint to retrieve deployed contract addresses - Add new RPC endpoint to fetch contract information by name - Return contract address for deployed contracts - Register endpoint in RPC server initialization * refactor(consensus): remove unused methods from ConsensusService - Remove _send_transaction helper method as it's not being used - Remove get_accounts method as it's no longer needes * feat(rpc): enhance eth_getContract endpoint with ABI and message handling - Add ABI to contract endpoint response for better contract interaction - Integrate MessageHandler into ConsensusService for improved logging * feat(hardhat): add bytecode to contract deployment data - Include bytecode in saved deployment JSON files - Included bytecode in the response for the eth_getContract endpoint * resolve: changed the endpoint name: eth_getContract -> sim_getContract * refactor: delete msg_handler in ConsensusService * refactor: delete msg_handler in ConsensusService * refactor: delete msg_handler in ConsensusService * resolve: changed the endpoint name: eth_getContract -> sim_getContract --------- Co-authored-by: kstroobants <[email protected]>
cristiam86
added a commit
that referenced
this issue
Dec 20, 2024
…ts (#762) * feat: add hardhat node and web3 package - Add Hardhat node service with Docker configuration - Configure hardhat.config.js with local network settings - Add 20 test accounts with 10,000 ETH each - Add web3 Python package to backend requirements - Update .gitignore for Hardhat artifacts and cache * feat: add contract and try script * feat: setup contract compilation and artifact handling - Add contract compilation setup with hardhat - Configure project structure for smart contracts - Fix file path for GhostContract.json artifact - Add necessary dependencies for contract compilation - Setup directory structure for contracts and artifacts * feat: creating hardhat transactions in consensus mechanism - Moved web3 python package into backend requirements file - Added access to compiled hardhat contract in jsonrpc service - Added database migration file so that a transaction has the ghost contract address of hardhat network - When a genlayer contract is deployed then a hardhat contract is deployed, both are linked in the CurrentState table - When a genlayer write method is executed then the new transaction gets the hardhat contract from the CurrentState table - When a genlayer transaction changes from status then a rollup transaction is created on the hardhat network - todo: remove rollup transaction table, put hardhat port in env, link genlayer account to hardhat account, check for out of gas, remove prints * feat: put hardhat port in env * feat: remove rollup transactions database table * feat: free transactions on hardhat, one hardhat account * test: add hardhat test with code from the transaction_processor * fix: resolve pre-commit error * test: moved test to integration tests, upgrade web3 version in requirements as it gave an import error * feat(hardhat): add genlayer-consensus contracts and setup compilation - Add genlayer-consensus smart contracts to hardhat/contracts directory - Update hardhat.config.js to enable new code generator (viaIR: true) - Add @openzeppelin/contracts and @openzeppelin/contracts-upgradeable dependencies - Configure hardhat Docker container for contract compilation - Verify hardhat node functionality in genlayer-studio This commit sets up the smart contract development environment with the necessary dependencies and configurations to compile and deploy genlayer-consensus contracts. * feat: deleted amm_adaptive.py - Deleted amm_adaptive.py because is not using the new syntax I will upload the new amm_adaptive.py whenevir will be ready * fix: contract interaction - Fixed contract deployment state updates for frontend synchronization - Improved contract method interactions and state management - Ensured proper state updates after contract value changes * fix: pre-commit errors solved * fix: solved black pre-commit issue * feat(hardhat): add deployment persistence to hardhat node - Add volume mappings for deployments, artifacts and cache - Implement manual deployment file saving - Update hardhat configuration for proper deployment paths This change ensures that deployed contracts persist between container restarts by saving deployment data to mounted volumes. Note: Block persistence still requires a different solution (Ganache/Geth/Besu) as Hardhat Network doesn't support state persistence. Related issue: #669 * refactor(hardhat): integrate start script into Dockerfile - Removed start.sh script - Incorporated start script directly into Dockerfile for streamlined execution * fix(db): remove duplicate ghost_contract_address assignment Removes redundant assignment of ghost_contract_address in transaction_from_dict function to improve code clarity and prevent potential inconsistencies * fix(security): address SonarCloud warnings and improve test coverage - Ensure non-root user is used in Dockerfile to enhance security - Adjust permissions to avoid overly permissive access (777) on /app directories - Add tests to cover deployment scripts, ensuring proper functionality and coverage * commit: Add contract initialization and consensus service integration - Updated deploy_contracts.js to handle contract initialization after deployment - Added initial ConsensusService integration into server.py - Enhanced Dockerfile.hardhat with proper permissions and chainId handling - Started implementation of consensus_service.py with basic contract interactions Note: consensus_service.py still needs additional work for complete contract initialization and interaction methods. * feat(deploy): integrate contract deployment and backend service Integrate complete contract deployment persistence in Hardhat and create ConsensusService for contract interaction. - Add deployFixture to Hardhat deployment script to handle contract initialization - Create ConsensusService class to interact with deployed contracts from backend - Add deploy_fixture method to ConsensusService as fallback initialization option - Ensure consistent contract initialization between Hardhat and backend - Load contract artifacts and addresses in ConsensusService from deployment files The deployFixture now handles all contract initialization during deployment, while ConsensusService provides a Python interface to interact with the deployed contracts. The deploy_fixture method in ConsensusService mirrors the Hardhat initialization logic but is kept as a fallback option. * feat(hardhat): implement contract deployment flow and deployment backup system - Add consensus contract deployment flow with proper initialization sequence - Implement contract backup system for deployment persistence - Add deployment verification to prevent unnecessary redeployments - Create copy-deployments script for managing contract backups - Configure hardhat to handle contract compilation and deployment * feat(hardhat): implement contract deployment flow and deployment backup system - Add consensus contract deployment flow with proper initialization sequence - Implement contract backup system for deployment persistence - Add deployment verification to prevent unnecessary redeployments - Create copy-deployments script for managing contract backups - Configure hardhat to handle contract compilation and deployment * refactor: remove ghost_contract_address redundant references - Remove ghost_contract_address from transactions_processor to avoid duplication - Remove unused ghost_contract_address from types.py * refactor: replace hardhat-deploy with hardhat-ignition - Migrate deployment process from hardhat-deploy to hardhat-ignition - Create DeployFixture module following TransactionFlow deployment order - Update deployment tests to ensure initialization and contract connecton as TransactionFlow * refactor: remove deployFixture from consensus_service.py - Remove unused deployFixture function from consensus_service.py - Deployment now handled entirely through hardhat-ignition * feat(rpc): add eth_getContract endpoint to retrieve deployed contract addresses - Add new RPC endpoint to fetch contract information by name - Return contract address for deployed contracts - Register endpoint in RPC server initialization * refactor(consensus): remove unused methods from ConsensusService - Remove _send_transaction helper method as it's not being used - Remove get_accounts method as it's no longer needes * feat(rpc): enhance eth_getContract endpoint with ABI and message handling - Add ABI to contract endpoint response for better contract interaction - Integrate MessageHandler into ConsensusService for improved logging * feat(hardhat): add bytecode to contract deployment data - Include bytecode in saved deployment JSON files - Included bytecode in the response for the eth_getContract endpoint * refactor: delete msg_handler in ConsensusService * refactor: delete msg_handler in ConsensusService * refactor: delete msg_handler in ConsensusService * feat: enable transaction submission to node in transactions_processor - Uncomment transaction submission code in transactions_processor - Add consensus_service to fetch GhostContract ABI and bytecode * feat(consensus): Integrate create_rollup_transactions in consensus execution - Added code to call the create_rollup_transactions method from the TransactionsProcessor. - This integration allows for sending transactions to the Hardhat node during consensus executins. * fix: used pre-commit * fix: encapsule send_raw_transactions into a try-except block * feat(consensus): implement transaction flow script and update contracts - Update consensus contracts to latest version - Update deployFixture to match TransactionFlow - Modify test suite for new contract flow - Update hardhat container permissions - Add consensus-flow.js for transaction simulation * refactor: deleted changes that is not needed for that PR * refactor: deleted changes that is not needed for that PR * refactor: added more time in the healthcheck * refactor: returned back the Docker and DockerCompose stable version * hore(docker): adjust healthcheck timing parameters * refactor: add more time in the Hardhat healthcheck * fix: changes healtcheck parameters, change Dockerfile permissons * fix(ci): adjust hardhat healthcheck timing parameters - Increase interval from 40s to 60s to allow more time between checks - Extend timeout from 15s to 30s for slower environments - Reduce retries from 20 to 5 but with longer intervals - Increase start period from 40s to 60s for initial container setup * fix: change healthcheck params hardhat test --------- Co-authored-by: kstroobants <[email protected]> Co-authored-by: Cristiam Da Silva <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Current Behavior
Currently, when the Hardhat container is stopped and restarted:
Proposed Solution
We need to implement persistence for both the blockchain state and contract deployments:
Short-term solution
/app/deploymentsto a named volume/app/artifactsto a named volume/app/cacheto a named volumeLong-term solution
Consider implementing one of these alternatives for full blockchain state persistence:
Ganache Integration
Full Ethereum Client
Note: We attempted to implement block persistence using Hardhat's Ignition feature, but it proved unfeasible due to limitations in the Hardhat Network implementation. This led to the consideration of alternative solutions like Ganache or full Ethereum clients (Geth/Besu) for proper blockchain state persistence.
Implementation Details
docker-compose.ymland 'Dockerfile.hardhat`Additional Considerations
Related Issues
Breaking Changes
/label enhancement
/label blockchain
/label persistence
The text was updated successfully, but these errors were encountered: