Unable to run the provided test command

[WoutV]

When executing the following command

cover-agent \                                                                        
  --source-file-path "app/translation/utils/utils.py" \
  --test-file-path "tests/translation/test_translation.py" \
  --project-root "$PWD" \
  --code-coverage-report-path "coverage.xml" \
  --test-command "pytest --cov=. --cov-report=xml --cov-report=term" \
  --test-command-dir "$PWD" \
  --coverage-type "cobertura" \
  --desired-coverage 70 \
  --max-iterations 10 \
--model "azure/gpt-4o"

I am met with the output below:

AssertionError: Fatal: Error running test command. Are you sure the command is correct? "pytest --cov=. --cov-report=xml --cov-report=term"
Stderr: 
Command timed out

[EmbeddedDevops1]

Hi. Did you try the example project run (in the top level README) to make sure everything is set up correctly?

Either way it looks like we need to capture more debug info regardless. Thanks for filing the issue.

[jeff-knurek]

we're running into a similar problem with mvn. The command is:

cover-agent   
	--source-file-path="/home/Repo/payment/payment-batch/src/main/java/com/payment/batch/common/JobConverter.java"   
	--test-file-path="/home/Repo/payment/payment-batch/src/test/java/com/payment/batch/common/JobConverterTest.java"   
	--code-coverage-report-path="/home/Repo/payment/payment-batch/target/site/jacoco"   
	--test-command="mvn -Dtest=com/payment/batch/common/** test -f pom.xml"   
	--test-command-dir="/home/Repo/payment/payment-batch"   
	--coverage-type="jacoco"   
	--desired-coverage=70   
	--max-iterations=1

and the error is

AssertionError: Fatal: Error running test command. Are you sure the command is correct? "mvn -Dtest=com/payment/batch/common/** test -f pom.xml"
Stderr: 
Command timed out

I added some debug messages into the run_coverage function in cover_agent/UnitTestValidator.py around

Runner.run_command(
            command=self.test_command, cwd=self.test_command_dir
        )

to see that the command and the command dir were correct. Going to try adding different exception handling in Runner.py to maybe get a solution 🤷‍♂️

---

we were successful in running the gradle example

cover-agent   
--source-file-path="src/main/java/com/davidparry/cover/SimpleMathOperations.java"   
--test-file-path="src/test/groovy/com/davidparry/cover/SimpleMathOperationsSpec.groovy"   
--code-coverage-report-path="build/reports/jacoco/test/jacocoTestReport.csv"   
--test-command="./gradlew clean test jacocoTestReport"   
--test-command-dir=$(pwd)   
--coverage-type="jacoco"   
--desired-coverage=70   
--max-iterations=1

[EmbeddedDevops1]

@jeff-knurek et all, sorry for the delay in getting back to you.

Are you currently running off main? The assertion error should be giving you more information than you provided (in cover_agent/UnitTestValidator.py):

assert (
            exit_code == 0
        ), f'Fatal: Error running test command. Are you sure the command is correct? "{self.test_command}"\nExit code {exit_code}. \nStdout: \n{stdout} \nStderr: \n{stderr}'

Sometimes an error will occur but it's not getting piped out to stderr. This can happen with Pytest and some other testing frameworks.

What do you see in your stdout?