Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gp/travis ci improvement #615

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Gp/travis ci improvement #615

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
bf64a94
Higher degree of parallelization for Travis CI
JackPiri Oct 9, 2023
6c4ab12
Minor improvements for sync functions
JackPiri Oct 9, 2023
9d10ded
Changed PoW parameters for Regtest
JackPiri Oct 9, 2023
262b039
Bug fixes
JackPiri Oct 10, 2023
8db6fb7
Added script for updating Python tests estimated execution times
JackPiri Oct 11, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 69 additions & 19 deletions .travis.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ os: linux
- DOCKER_IS_DEB=true
- MAKEFLAGS="--enable-lcov"
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-split=5:1 --coverage"
- TEST_ARGS="rpc --rpc-extended --rpc-split=6:1 --coverage"
- &docker_amd64_linux_ubuntu_jammy_rpc-tests_2
env:
- DOCKER_ARCH=amd64
Expand All @@ -78,7 +78,7 @@ os: linux
- DOCKER_IS_DEB=true
- MAKEFLAGS="--enable-lcov"
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-split=5:2 --coverage"
- TEST_ARGS="rpc --rpc-extended --rpc-split=6:2 --coverage"
- &docker_amd64_linux_ubuntu_jammy_rpc-tests_3
env:
- DOCKER_ARCH=amd64
Expand All @@ -87,7 +87,7 @@ os: linux
- DOCKER_IS_DEB=true
- MAKEFLAGS="--enable-lcov"
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-split=5:3 --coverage"
- TEST_ARGS="rpc --rpc-extended --rpc-split=6:3 --coverage"
- &docker_amd64_linux_ubuntu_jammy_rpc-tests_4
env:
- DOCKER_ARCH=amd64
Expand All @@ -96,7 +96,7 @@ os: linux
- DOCKER_IS_DEB=true
- MAKEFLAGS="--enable-lcov"
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-split=5:4 --coverage"
- TEST_ARGS="rpc --rpc-extended --rpc-split=6:4 --coverage"
- &docker_amd64_linux_ubuntu_jammy_rpc-tests_5
env:
- DOCKER_ARCH=amd64
Expand All @@ -105,7 +105,16 @@ os: linux
- DOCKER_IS_DEB=true
- MAKEFLAGS="--enable-lcov"
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-split=5:5 --coverage"
- TEST_ARGS="rpc --rpc-extended --rpc-split=6:5 --coverage"
- &docker_amd64_linux_ubuntu_jammy_rpc-tests_6
env:
- DOCKER_ARCH=amd64
- DOCKER_TARGET_OS=linux
- DOCKER_FROM=ubuntu_jammy
- DOCKER_IS_DEB=true
- MAKEFLAGS="--enable-lcov"
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-split=6:6 --coverage"
- &docker_amd64_linux_ubuntu_jammy_test-fetch-params
env:
- DOCKER_ARCH=amd64
Expand All @@ -129,31 +138,47 @@ os: linux
- DOCKER_FROM=ubuntu_focal
- DOCKER_IS_DEB=true
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-split=4:1"
- TEST_ARGS="rpc --rpc-extended --rpc-split=6:1"
- &docker_amd64_linux_ubuntu_focal_rpc-tests_2
env:
- DOCKER_ARCH=amd64
- DOCKER_TARGET_OS=linux
- DOCKER_FROM=ubuntu_focal
- DOCKER_IS_DEB=true
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-split=4:2"
- TEST_ARGS="rpc --rpc-extended --rpc-split=6:2"
- &docker_amd64_linux_ubuntu_focal_rpc-tests_3
env:
- DOCKER_ARCH=amd64
- DOCKER_TARGET_OS=linux
- DOCKER_FROM=ubuntu_focal
- DOCKER_IS_DEB=true
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-split=4:3"
- TEST_ARGS="rpc --rpc-extended --rpc-split=6:3"
- &docker_amd64_linux_ubuntu_focal_rpc-tests_4
env:
- DOCKER_ARCH=amd64
- DOCKER_TARGET_OS=linux
- DOCKER_FROM=ubuntu_focal
- DOCKER_IS_DEB=true
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-split=4:4"
- TEST_ARGS="rpc --rpc-extended --rpc-split=6:4"
- &docker_amd64_linux_ubuntu_focal_rpc-tests_5
env:
- DOCKER_ARCH=amd64
- DOCKER_TARGET_OS=linux
- DOCKER_FROM=ubuntu_focal
- DOCKER_IS_DEB=true
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-split=6:5"
- &docker_amd64_linux_ubuntu_focal_rpc-tests_6
env:
- DOCKER_ARCH=amd64
- DOCKER_TARGET_OS=linux
- DOCKER_FROM=ubuntu_focal
- DOCKER_IS_DEB=true
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-split=6:6"
- &docker_amd64_linux_ubuntu_focal_test-fetch-params
env:
- DOCKER_ARCH=amd64
Expand All @@ -172,54 +197,64 @@ os: linux
- &amd64_osx_xcode14_2_rpc-tests_1
env:
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=10:1 --rpc-macrebalance"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=12:1 --rpc-macrebalance"
- MAKEFLAGS="--disable-libs --legacy-cpu"
- &amd64_osx_xcode14_2_rpc-tests_2
env:
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=10:2 --rpc-macrebalance"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=12:2 --rpc-macrebalance"
- MAKEFLAGS="--disable-libs --legacy-cpu"
- &amd64_osx_xcode14_2_rpc-tests_3
env:
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=10:3 --rpc-macrebalance"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=12:3 --rpc-macrebalance"
- MAKEFLAGS="--disable-libs --legacy-cpu"
- &amd64_osx_xcode14_2_rpc-tests_4
env:
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=10:4 --rpc-macrebalance"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=12:4 --rpc-macrebalance"
- MAKEFLAGS="--disable-libs --legacy-cpu"
- &amd64_osx_xcode14_2_rpc-tests_5
env:
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=10:5 --rpc-macrebalance"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=12:5 --rpc-macrebalance"
- MAKEFLAGS="--disable-libs --legacy-cpu"
- &amd64_osx_xcode14_2_rpc-tests_6
env:
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=10:6 --rpc-macrebalance"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=12:6 --rpc-macrebalance"
- MAKEFLAGS="--disable-libs --legacy-cpu"
- &amd64_osx_xcode14_2_rpc-tests_7
env:
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=10:7 --rpc-macrebalance"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=12:7 --rpc-macrebalance"
- MAKEFLAGS="--disable-libs --legacy-cpu"
- &amd64_osx_xcode14_2_rpc-tests_8
env:
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=10:8 --rpc-macrebalance"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=12:8 --rpc-macrebalance"
- MAKEFLAGS="--disable-libs --legacy-cpu"
- &amd64_osx_xcode14_2_rpc-tests_9
env:
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=10:9 --rpc-macrebalance"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=12:9 --rpc-macrebalance"
- MAKEFLAGS="--disable-libs --legacy-cpu"
- &amd64_osx_xcode14_2_rpc-tests_10
env:
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=10:10 --rpc-macrebalance"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=12:10 --rpc-macrebalance"
- MAKEFLAGS="--disable-libs --legacy-cpu"
- &amd64_osx_xcode14_2_rpc-tests_11
env:
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=12:11 --rpc-macrebalance"
- MAKEFLAGS="--disable-libs --legacy-cpu"
- &amd64_osx_xcode14_2_rpc-tests_12
env:
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-extended --rpc-exclude=rpcbind_test.py,invalidblockrequest.py,invalidblockposthalving.py,p2p-acceptblock.py,wallet_mergetoaddress_2.py --rpc-split=12:12 --rpc-macrebalance"
- MAKEFLAGS="--disable-libs --legacy-cpu"
- &amd64_osx_xcode14_2_rpc-tests_13
env:
- TEST_CMD="./qa/zcash/full_test_suite.py"
- TEST_ARGS="rpc --rpc-runonly=wallet_mergetoaddress_2.py"
Expand Down Expand Up @@ -376,6 +411,9 @@ jobs:
- stage: Test
<<: *test_docker_amd64_x-large
<<: *docker_amd64_linux_ubuntu_jammy_rpc-tests_5
- stage: Test
<<: *test_docker_amd64_x-large
<<: *docker_amd64_linux_ubuntu_jammy_rpc-tests_6
- stage: Test
<<: *test_docker_amd64_x-large
<<: *docker_amd64_linux_ubuntu_jammy_clang-tidy
Expand All @@ -397,6 +435,12 @@ jobs:
- stage: Test
<<: *test_docker_amd64_x-large
<<: *docker_amd64_linux_ubuntu_focal_rpc-tests_4
- stage: Test
<<: *test_docker_amd64_x-large
<<: *docker_amd64_linux_ubuntu_focal_rpc-tests_5
- stage: Test
<<: *test_docker_amd64_x-large
<<: *docker_amd64_linux_ubuntu_focal_rpc-tests_6
- stage: Test
<<: *test_docker_amd64
<<: *docker_amd64_linux_ubuntu_focal_test-fetch-params
Expand Down Expand Up @@ -437,6 +481,12 @@ jobs:
- stage: Test
<<: *test_osx_xcode14_2
<<: *amd64_osx_xcode14_2_rpc-tests_11
- stage: Test
<<: *test_osx_xcode14_2
<<: *amd64_osx_xcode14_2_rpc-tests_12
- stage: Test
<<: *test_osx_xcode14_2
<<: *amd64_osx_xcode14_2_rpc-tests_13
- stage: Finalize_coverage
<<: *finalize_docker_amd64_x-large
<<: *docker_amd64_linux_ubuntu_jammy
2 changes: 1 addition & 1 deletion contrib/ci-horizen/scripts/common/setup_environment.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ if [ "${TRAVIS_OS_NAME}" = "osx" ]; then
NEED_B2_CREDS="true"
fi
if [ "${TRAVIS_BUILD_STAGE_NAME}" = "Test" ]; then
export PIP3_INSTALL="${PIP3_INSTALL} pycryptodome pyzmq websocket-client requests b2sdk"
export PIP3_INSTALL="${PIP3_INSTALL} pycryptodome pyzmq websocket-client b2sdk"
export B2_DL_DECOMPRESS_FOLDER="${TRAVIS_BUILD_DIR}"
# shellcheck disable=SC2001,SC2155
export B2_DL_FILENAME="${TRAVIS_CPU_ARCH}-${TRAVIS_OS_NAME}-${TRAVIS_OSX_IMAGE}-${TRAVIS_BUILD_ID}-${TRAVIS_COMMIT}$(sed 's/ //g' <<< "${MAKEFLAGS:-}").tar.gz"
Expand Down
22 changes: 20 additions & 2 deletions contrib/devtools/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ Contents
===========
This directory contains tools for developers working on this repository.


github-merge.sh
==================

Expand Down Expand Up @@ -36,6 +37,7 @@ Configuring the github-merge tool for the bitcoin repository is done in the foll
git config githubmerge.testcmd "make -j4 check" (adapt to whatever you want to use for testing)
git config --global user.signingkey mykeyid (if you want to GPG sign)


fix-copyright-headers.py
===========================

Expand All @@ -49,6 +51,7 @@ For example a file changed in 2014 (with 2014 being the current year):
would be changed to:
```// Copyright (c) 2009-2014 The Bitcoin Core developers```


symbol-check.py
==================

Expand All @@ -69,6 +72,7 @@ If there are 'unsupported' symbols, the return value will be 1 a list like this
.../64/test_bitcoin: symbol std::out_of_range::~out_of_range() from unsupported version GLIBCXX_3.4.15
.../64/test_bitcoin: symbol _ZNSt8__detail15_List_nod from unsupported version GLIBCXX_3.4.15


update-translations.py
=======================

Expand All @@ -81,12 +85,14 @@ It will do the following automatically:

See doc/translation-process.md for more information.


gen-manpages.sh
===============

A small script to automatically create manpages in ../../doc/man by running the release binaries with the -help option.
This requires help2man which can be found at: https://www.gnu.org/software/help2man/


git-subtree-check.sh
====================

Expand All @@ -101,7 +107,8 @@ maintained:
Usage: git-subtree-check.sh DIR COMMIT
COMMIT may be omitted, in which case HEAD is used.

release_management/release_preparation

release_management/(release_preparation)
==================

Inside this folder the following are provided:
Expand All @@ -128,7 +135,8 @@ After script completion the user is required to:
- open a PR merging release preparation into release branch and wait for approval
- once the release preparation branch is merged into the release branch, create annotated tag (vX.Y.Z)

release_management/release_backport

release_management/(release_backport)
==================

Inside this folder the following are provided:
Expand All @@ -153,3 +161,13 @@ After script completion the user is required to:
- check no error code is returned by the script
- push from release backport local branch to remote
- open a PR merging release backport into `main` branch and wait for approval


tests-execution-times-update.py
==================

A script interacting with Travis CI API, its purpose is to update estimated execution times for Python regression tests
based on measured execution times of recent Travis CI runs. The script accepts `travis_token`, `builds_quantity` and
`rpc_tests_sh_file_path` parameters representing respectively the token for authenticating to Travis CI API, the quantity
of builds to consider when computing average estimated execution times and the path to the `rpc-tests.sh` file to update.
The script prints average estimated execution times and consequently updates the file `rpc-tests.sh`.
Loading