Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix github actions build failure #1054

Merged
merged 170 commits into from
Aug 17, 2022
Merged
Show file tree
Hide file tree
Changes from 17 commits
Commits
Show all changes
170 commits
Select commit Hold shift + click to select a range
9b3bd26
test
wind57 Jul 26, 2022
59c5995
add timeout
wind57 Jul 26, 2022
73281a2
trigger
wind57 Jul 26, 2022
266890d
add timeout to 60 min
wind57 Jul 26, 2022
67b6a34
test
wind57 Jul 26, 2022
7a15fd9
test
wind57 Jul 26, 2022
e5d45f8
test
wind57 Jul 26, 2022
fe8b32d
fix what tests each instance is running
wind57 Jul 27, 2022
e19a0c4
github fix for abstract tests
wind57 Jul 27, 2022
9d37ad5
remove file
wind57 Jul 27, 2022
e98200f
removed echo
wind57 Jul 27, 2022
230f675
test circleci
wind57 Jul 27, 2022
6f6e90d
escape
wind57 Jul 27, 2022
8f69377
single?
wind57 Jul 27, 2022
00f0842
test
wind57 Jul 27, 2022
1697e0b
bigger timeout
wind57 Jul 27, 2022
b4de760
add a bit more debugging
wind57 Jul 27, 2022
426ed3f
fix circleci
wind57 Jul 27, 2022
ae804f6
split in half
wind57 Jul 28, 2022
be919bb
test
wind57 Jul 28, 2022
bc58f7e
test
wind57 Jul 28, 2022
b697a05
test
wind57 Jul 28, 2022
3a91408
test
wind57 Jul 28, 2022
11ab4d5
test
wind57 Jul 28, 2022
c6fb425
test
wind57 Jul 28, 2022
30b7fc0
test
wind57 Jul 28, 2022
0b6b4c3
test
wind57 Jul 28, 2022
2564b30
test
wind57 Jul 28, 2022
b089632
test
wind57 Jul 28, 2022
343d2d3
test
wind57 Jul 28, 2022
537db4c
test
wind57 Jul 28, 2022
855dfaa
test
wind57 Jul 28, 2022
717e671
test
wind57 Jul 28, 2022
c2a0aa0
test
wind57 Jul 28, 2022
34e2018
test
wind57 Jul 28, 2022
6664c0f
test
wind57 Jul 28, 2022
b3f2aa4
test
wind57 Jul 28, 2022
dbcf157
test
wind57 Jul 28, 2022
a31d7cb
test
wind57 Jul 28, 2022
68692c9
test
wind57 Jul 28, 2022
f04ee49
test
wind57 Jul 28, 2022
65a734f
test
wind57 Jul 28, 2022
4b93e0e
test
wind57 Jul 28, 2022
66f1021
test
wind57 Jul 28, 2022
584c10e
test
wind57 Jul 28, 2022
d55f66d
test
wind57 Jul 28, 2022
6b284c4
test
wind57 Jul 28, 2022
34995d0
test
wind57 Jul 28, 2022
f9b079b
test
wind57 Jul 28, 2022
4eabc42
test
wind57 Jul 28, 2022
8cce300
test
wind57 Jul 28, 2022
9f8fa55
test
wind57 Jul 28, 2022
9494794
test
wind57 Jul 28, 2022
d6a32e9
test
wind57 Jul 28, 2022
d5ead6a
test
wind57 Jul 28, 2022
8b14f4e
test
wind57 Jul 28, 2022
3f038bf
test
wind57 Jul 28, 2022
8d77355
test
wind57 Jul 28, 2022
531e180
test
wind57 Jul 28, 2022
eb50a09
test
wind57 Jul 28, 2022
23c6a26
circleci should work now
wind57 Jul 28, 2022
00d6c3b
circleci should work now
wind57 Jul 28, 2022
0c72878
drop splitting as it seems un-needed
wind57 Jul 28, 2022
49bd794
remove file when done
wind57 Jul 28, 2022
c3777b6
test
wind57 Jul 29, 2022
1a9e163
test
wind57 Jul 29, 2022
d25d209
test
wind57 Jul 29, 2022
3c41edb
test
wind57 Jul 29, 2022
0e63735
test
wind57 Jul 29, 2022
502d59a
test
wind57 Jul 29, 2022
10fbd06
test
wind57 Jul 29, 2022
696e9a5
test
wind57 Jul 29, 2022
0e6bf04
test
wind57 Jul 29, 2022
3134798
test
wind57 Jul 29, 2022
98dbbd1
test
wind57 Jul 29, 2022
06ba485
test
wind57 Jul 29, 2022
33ff2ed
test
wind57 Jul 29, 2022
fdcda5e
test
wind57 Jul 29, 2022
8e0e323
test
wind57 Jul 29, 2022
19e8602
test
wind57 Jul 29, 2022
fe92872
test
wind57 Jul 29, 2022
4e293a7
test
wind57 Jul 29, 2022
823dd3e
test
wind57 Jul 29, 2022
2c8ee76
test
wind57 Jul 29, 2022
41f356d
test
wind57 Jul 29, 2022
8bed5c3
test
wind57 Jul 29, 2022
ed2bf4b
test
wind57 Jul 29, 2022
878d76a
trigger
wind57 Jul 29, 2022
9a470c7
trigger
wind57 Jul 30, 2022
1fe5b15
test
wind57 Jul 30, 2022
81b49a8
test
wind57 Jul 30, 2022
46117cf
test
wind57 Jul 30, 2022
0e538d7
test
wind57 Jul 30, 2022
c579d48
test
wind57 Jul 30, 2022
4c5d89b
test
wind57 Jul 30, 2022
a139085
test
wind57 Jul 30, 2022
7c223ae
test
wind57 Jul 30, 2022
9f7be4b
test
wind57 Jul 30, 2022
45ebe0f
test
wind57 Jul 30, 2022
2943c9d
test
wind57 Jul 30, 2022
79bcb89
test
wind57 Jul 30, 2022
f2795b5
test
wind57 Jul 30, 2022
c733c24
test
wind57 Jul 30, 2022
640f1bf
test
wind57 Jul 30, 2022
0419451
test
wind57 Jul 30, 2022
45624ab
test
wind57 Jul 30, 2022
a7cc9d7
test
wind57 Jul 30, 2022
619e74c
test
wind57 Jul 30, 2022
020f117
test
wind57 Jul 30, 2022
f715e97
test
wind57 Jul 30, 2022
a9fe5ad
test
wind57 Jul 30, 2022
0e27c68
test
wind57 Jul 30, 2022
22bbd24
test
wind57 Jul 30, 2022
795f0a5
test
wind57 Jul 31, 2022
24419d0
trigger
wind57 Jul 31, 2022
1db6394
test
wind57 Jul 31, 2022
76873ba
test
wind57 Jul 31, 2022
f921c73
test
wind57 Jul 31, 2022
94468af
test
wind57 Aug 1, 2022
8d49413
test
wind57 Aug 1, 2022
6e37088
test
wind57 Aug 1, 2022
a536330
test
wind57 Aug 1, 2022
aa7c81d
test
wind57 Aug 1, 2022
8ee5b3a
test
wind57 Aug 1, 2022
ecbe35f
test
wind57 Aug 1, 2022
b068344
test
wind57 Aug 1, 2022
40d72df
test
wind57 Aug 1, 2022
111ac26
test
wind57 Aug 1, 2022
305aecd
test
wind57 Aug 1, 2022
5c57db4
test
wind57 Aug 1, 2022
e5923a8
test
wind57 Aug 1, 2022
b16152e
test
wind57 Aug 1, 2022
cfffc6d
test
wind57 Aug 1, 2022
0ac03f0
test
wind57 Aug 1, 2022
9923b01
test
wind57 Aug 1, 2022
fb81fee
test
wind57 Aug 1, 2022
765b77c
test
wind57 Aug 1, 2022
609ffa3
trigger
wind57 Aug 1, 2022
2be3a26
test
wind57 Aug 1, 2022
c6c9489
test
wind57 Aug 1, 2022
ca3a1c4
test
wind57 Aug 1, 2022
8d6dba5
test
wind57 Aug 1, 2022
d158e88
Merge branch 'main' into fix-build-on-github
wind57 Aug 1, 2022
35f8fbc
added README
wind57 Aug 3, 2022
04c04f9
refactor load images
wind57 Aug 4, 2022
aad6c37
enable reuse
wind57 Aug 4, 2022
a7a37ad
increaase timeout
wind57 Aug 4, 2022
1407fc5
increaase timeout
wind57 Aug 4, 2022
2ed5072
test
wind57 Aug 4, 2022
0f4c477
test
wind57 Aug 4, 2022
6b6fe6f
test
wind57 Aug 4, 2022
dcde155
test
wind57 Aug 4, 2022
2a1ef4a
no return
wind57 Aug 4, 2022
b507c5e
test
wind57 Aug 4, 2022
fdf148a
test
wind57 Aug 4, 2022
de31f65
test
wind57 Aug 4, 2022
3a3522d
test
wind57 Aug 4, 2022
3d52745
test
wind57 Aug 4, 2022
249ed84
test
wind57 Aug 4, 2022
df27866
test
wind57 Aug 4, 2022
a201e93
test
wind57 Aug 4, 2022
9340505
test
wind57 Aug 4, 2022
cc6bde5
test
wind57 Aug 4, 2022
0032a14
test
wind57 Aug 4, 2022
6dc5c42
test
wind57 Aug 4, 2022
bebdb7c
test
wind57 Aug 4, 2022
cad4be0
disable test
wind57 Aug 6, 2022
39c3c68
pom
wind57 Aug 16, 2022
28f0618
Merge branch 'main' into fix-build-on-github
wind57 Aug 17, 2022
1479efd
enable test back
wind57 Aug 17, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
70 changes: 64 additions & 6 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,11 @@ jobs:
# needed for .withReuse(true) to work
echo "testcontainers.reuse.enable=true" > ~/.testcontainers.properties
- checkout
- restore_cache:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

restore cache for tests

keys:
- spring-cloud-kubernetes-{{ .Branch }}-{{ checksum "pom.xml" }}
- spring-cloud-kubernetes-{{ .Branch }}
- spring-cloud-kubernetes
- attach_workspace:
at: /tmp/docker
- run:
Expand All @@ -57,6 +62,8 @@ jobs:
name: Run regular tests
command: |

set -x

###########################################################################################################################
################################################# Build test support dependency ###########################################
cd spring-cloud-kubernetes-test-support
Expand All @@ -65,14 +72,65 @@ jobs:

###########################################################################################################################
##################################################### Split and run tests #################################################
CLASSNAMES=$(circleci tests glob "**/src/test/**/**.java" | grep -v 'Fabric8IstioIT' \
| xargs grep -l '@Test' \
| sed 's/.*src.test.java.//g' | sed 's@/@.@g' \
| sed 's/.\{5\}$//' \
| circleci tests split --split-by=timings)

# - find all tests
# - exclude Fabric8IstionIT
# - only take classes that have @Test inside them
# - ignore the ones that have 'abstract class'. we do this because otherwise we would pass
# to -DtestsToRun an abstract class, and it will not run anything.
# - drop the "begining" xxx/src/test/java
# - replace / with .
# - drop last ".java"
# - replace newline with space

PLAIN_TEST_CLASSNAMES=($(find . -name '*.java' \
| grep 'src/test/java' \
| grep -v 'Fabric8IstioIT' \
| xargs grep -l '@Test' \
| xargs grep -L 'abstract class' \
| sed 's/.*src.test.java.//g' \
| sed 's@/@.@g' \
| sed 's/.\{5\}$//')

# classes that have @Test and are abstract, for example: "LabeledSecretWithPrefixTests"
# - exclude Fabric8IstionIT
# - only take classes that have @Test inside them
# - only take classes that are abstract
# - drop everything up until the last "/"
# - drop ".java"

ABSTRACT_TEST_CLASSNAMES_COMMAND="find . -name '*.java' \
| grep 'src/test/java' \
| grep -v 'Fabric8IstioIT' \
| xargs grep -l '@Test' \
| xargs grep -l 'abstract class' \
| sed 's/.*\///g' \
| sed 's/.java//g'"

# find classes that extend abstract test classes
DERIVED_FROM_ABSTRACT_CLASSES_COMMAND="find . -name '*.java' \
| grep 'src/test/java' \
| grep -v 'Fabric8IstioIT' \
| xargs grep -l 'extends replace_me ' \
| sed 's/.*src.test.java.//g' \
| sed 's@/@.@g' \
| sed 's/.\{5\}$//'"

while read class_name; do
replaced=$(echo ${DERIVED_FROM_ABSTRACT_CLASSES_COMMAND/replace_me/"$class_name"})
result=($(eval $replaced))
PLAIN_TEST_CLASSNAMES+=(${result[@]})
done < <(eval $ABSTRACT_TEST_CLASSNAMES_COMMAND)

IFS=$'\n'
PLAIN_TEST_CLASSNAMES=$(sort \<<< "${PLAIN_TEST_CLASSNAMES[*]}")
unset IFS

CLASSNAMES=$(echo ${PLAIN_TEST_CLASSNAMES[@]} | circleci tests split --split-by=timings)
echo $CLASSNAMES
TEST_ARG=$(echo $CLASSNAMES | sed 's/ /,/g')
echo $TEST_ARG

./mvnw -s .settings.xml -DfailIfNoTests=false -DtestsToRun=$TEST_ARG -e clean org.jacoco:jacoco-maven-plugin:prepare-agent install \
-U -P sonar -nsu --batch-mode -Dmaven.test.redirectTestOutputToFile=true \
-Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn
Expand Down Expand Up @@ -110,7 +168,7 @@ jobs:
- run:
name: dependencies
command: |
./mvnw -s .settings.xml -U dependency:resolve-plugins dependency:go-offline -B -Dservice.occurence=${_SERVICE_OCCURENCE} || true
./mvnw -s .settings.xml -U dependency:resolve-plugins dependency:go-offline -B || true
- run:
name: build
command: |
Expand Down
96 changes: 67 additions & 29 deletions .github/workflows/maven.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,6 @@ jobs:
shell: bash
run: echo "##[set-output name=branch;]$(echo ${GITHUB_HEAD_REF})"

# if this is the first build, this is a NOOP and every dependency is downloaded.
# if this is != first build, cache is restored. We then issue mvn clean install with "-U",
# and thus update any third party snapshots; also we build the project, thus update
# any inter-module snapshot dependencies.
# only after that is the cache saved, which is what we want. Any subsequent restores as such
# will have the latest versions of snapshots available.
- name: Cache local Maven repository
id: cache-maven-repo
uses: actions/cache@v2
Expand Down Expand Up @@ -89,6 +83,7 @@ jobs:
test:
needs: build
runs-on: ubuntu-latest
timeout-minutes: 60

strategy:
fail-fast: true
Expand Down Expand Up @@ -118,26 +113,63 @@ jobs:
CURRENT_INDEX: ${{ matrix.current_index }}
NUMBER_OF_JOBS: ${{ matrix.number_of_jobs }}
run: |
# find all tests
# exclude Fabric8IstionIT
# only take classes that have @Test inside them
# drop the "begining" xxx/src/test/java
# replace / with .
# drop last ".java"
# replace newline with space
# sort

CLASSNAMES=($(find . -name '*.java' \
| grep 'src/test/java' \
| grep -v 'Fabric8IstioIT' \
| xargs grep -l '@Test' \
| sed 's/.*src.test.java.//g' \
| sed 's@/@.@g' \
| sed 's/.\{5\}$//' \
| sort \
| tr '\n' ' '))

number_of_tests=${#CLASSNAMES[@]}
# - find all tests
# - exclude Fabric8IstionIT
# - only take classes that have @Test inside them
# - ignore the ones that have 'abstract class'. we do this because otherwise we would pass
Copy link
Contributor Author

@wind57 wind57 Jul 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this small comment explains the problem, I noticed this by accident and could reproduce locally too... fixed via this logic.

# to -DtestsToRun an abstract class, and it will not run anything.
# - drop the "begining" xxx/src/test/java
# - replace / with .
# - drop last ".java"
# - replace newline with space
# - replace '\n' with ' '

PLAIN_TEST_CLASSNAMES=($(find . -name '*.java' \
| grep 'src/test/java' \
| grep -v 'Fabric8IstioIT' \
| xargs grep -l '@Test' \
| xargs grep -L 'abstract class' \
| sed 's/.*src.test.java.//g' \
| sed 's@/@.@g' \
| sed 's/.\{5\}$//' \
| tr '\n' ' '))

# classes that have @Test and are abstract, for example: "LabeledSecretWithPrefixTests"
# - exclude Fabric8IstionIT
# - only take classes that have @Test inside them
# - only take classes that are abstract
# - drop everything up until the last "/"
# - drop ".java"

ABSTRACT_TEST_CLASSNAMES_COMMAND="find . -name '*.java' \
| grep 'src/test/java' \
| grep -v 'Fabric8IstioIT' \
| xargs grep -l '@Test' \
| xargs grep -l 'abstract class' \
| sed 's/.*\///g' \
| sed 's/.java//g'"

# find classes that extend abstract test classes
DERIVED_FROM_ABSTRACT_CLASSES_COMMAND="find . -name '*.java' \
| grep 'src/test/java' \
| grep -v 'Fabric8IstioIT' \
| xargs grep -l 'extends replace_me ' \
| sed 's/.*src.test.java.//g' \
| sed 's@/@.@g' \
| sed 's/.\{5\}$//' \
| tr '\n' ' '"

while read class_name; do
replaced=$(echo ${DERIVED_FROM_ABSTRACT_CLASSES_COMMAND/replace_me/"$class_name"})
result=($(eval $replaced))
PLAIN_TEST_CLASSNAMES+=(${result[@]})
done < <(eval $ABSTRACT_TEST_CLASSNAMES_COMMAND)

IFS=$'\n'
PLAIN_TEST_CLASSNAMES=$(sort <<< "${PLAIN_TEST_CLASSNAMES[*]}")
unset IFS

number_of_tests=${#PLAIN_TEST_CLASSNAMES[@]}
number_of_jobs=${NUMBER_OF_JOBS}
current_index=${CURRENT_INDEX}

Expand Down Expand Up @@ -188,11 +220,17 @@ jobs:

diff=$((right_bound - left_bound))
sliced_array=("${CLASSNAMES[@]:$left_bound:$diff}")
TEST_ARG=$(echo $sliced_array | sed 's/ /,/g')
TEST_ARG=$(echo ${sliced_array[@]} | sed 's/ /,/g')
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had a bug here, I was only running the first test from a sliced_array, fixed.


./mvnw -s .settings.xml -DfailIfNoTests=false -DtestsToRun=$TEST_ARG -e clean org.jacoco:jacoco-maven-plugin:prepare-agent install \
echo "will run tests : ${TEST_ARG[@]}"

./mvnw -s .settings.xml -DtestsToRun=${TEST_ARG[@]} -e clean org.jacoco:jacoco-maven-plugin:prepare-agent install \
-U -P sonar -nsu --batch-mode -Dmaven.test.redirectTestOutputToFile=true \
-Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn
-Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=warn \
-Dhttp.keepAlive=false \
-Dmaven.wagon.http.pool=false \
-Dmaven.wagon.http.retryHandler.class=standard \
-Dmaven.wagon.http.retryHandler.count=3

- name: Publish Test Report
uses: mikepenz/action-junit-report@v2
Expand Down

This file was deleted.

Loading