Cannot run all unit tests and check code coverage of a component

[ThibFrgsGmz]

Hi, I am experimenting with the code coverage feature but I can't get it to work.

I followed the detailed procedure here:

From c8672e2, I went to the SignalGen component directory and ran

1. fprime-util generate --ut to build the unit test.
2. fprime-util check --all --coverage to run all unit tests and check the code coverage.

But I get the following error:

(.venv) ThibFrgsGmz@ubuntu:~/Documents/fprime/Ref/SignalGen$ fprime-util generate --all --coverage
[ERROR] Unknown arguments: --all, --coverage
usage: fprime-util [-h] {build,impl,check,generate,purge,fpp-locs,fpp-depends,fpp-check,hash-to-file,info,new} ...

F prime helper application.

optional arguments:
  -h, --help            show this help message and exit

subcommands:
  F prime utility command line. Please run one of the commands. For help, run a command with the --help flag.

  {build,impl,check,generate,purge,fpp-locs,fpp-depends,fpp-check,hash-to-file,info,new}
    build               Build components, ports, and deployments in the specified directory
    impl                Generate implementation template files in the specified directory
    check               Run unit tests in the specified directory
    generate            Generate a build cache directory. Defaults to generating a release build cache
    purge               Purge build cache directories
    fpp-locs            Regenerates the FPP locations file and prints the location
    fpp-depends         Regenerates the build cache and prints located fpp dependencies
    fpp-check           Runs fpp-check utility
    hash-to-file        Converts F prime build hash to filename.
    info                Gets fprime-util contextual information.
    new                 Generate a new component

If I run the command fprime-util check --coverage I have the following error message:

(.venv) ThibFrgsGmz@ubuntu:~/Documents/fprime/Ref/SignalGen$ fprime-util check --coverage
make: *** No rule to make target 'Ref_SignalGen_coverage'.  Stop.
[CMAKE] CMake failed to detect target, attempting CMake cache refresh and retry
make: *** No rule to make target 'Ref_SignalGen_coverage'.  Stop.
[ERROR] CMake erred with return code 2

I don't see a coverage control in the CI so I'm a little stuck. Could you guide me please?

[LeStarch]

This issue comes from an old fprime-tools version running against F´ devel where we have changed code coverage. This is a bug. It should be fixed with #1491 as that fixes a newer version of tools.

[LeStarch]

I just confirmed with merging the above PR into devel, coverage works:

cd Ref/SignalGen
fprime-util check --coverage # Produces "Ref/SignalGen/coverage" folder w/ SignalGen UT coverage analysis

cd Ref
fprime-util check --coverage # Produces "Ref/coverage" folder w/ deployment UT coverage analysis

cd Ref
fprime-util check --coverage --all # Produces "Ref/coverage" folder w/ all UT coverage analysis

Note: open /coverage.html for the coverage report.

[ThibFrgsGmz]

Awesome, I cloned the actual head including your PR and that fixed the issue, thanks!

[LeStarch]

Great. That PR is merged now too!

[ThibFrgsGmz]

@LeStarch well, maybe there is a problem with a file in OSAL: Pthreads/Queue.cpp.

fprime/Os/Pthreads/Queue.cpp

Line 29 in f6462e4

class QueueHandle {

I run fprime-util check --coverage like you inside the deployment folder and I get the following error:

      Start 98: App_SignalGen_ut_exe
36/37 Test #98: App_SignalGen_ut_exe .............   Passed    0.01 sec
      Start 99: App_RegisterAccess_ut_exe
37/37 Test #99: App_RegisterAccess_ut_exe ........   Passed    0.00 sec

100% tests passed, 0 tests failed out of 37

Total Test time (real) =  12.22 sec
[100%] Built target App_check
[INFO] Making temporary directory: /home/ThibFrgsGmz/Documents/fprime-codelab/{{AUTOCODE}}
[INFO] Copying AC files into temporary directory
(WARNING) Unrecognized GCOV output for /home/ThibFrgsGmz/Documents/fprime-codelab/fprime/Os/Pthreads/Queue.cpp
	  branch  3 taken -20 (fallthrough)
	This is indicative of a gcov output parse error.
	Please report this to the gcovr developers
	at <https://github.com/gcovr/gcovr/issues>.
(WARNING) Exception during parsing:
	UnknownLineType: branch  3 taken -20 (fallthrough)
(ERROR) Exiting because of parse errors.
	You can run gcovr with --gcov-ignore-parse-errors
	to continue anyway.
Traceback (most recent call last):
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/workers.py", line 77, in worker
    work(*args, **kwargs)
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov.py", line 317, in process_datafile
    done = run_gcov_and_process_files(
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov.py", line 490, in run_gcov_and_process_files
    process_gcov_data(fname, covdata, abs_filename, options)
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov.py", line 145, in process_gcov_data
    coverage = parse_coverage(
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov_parser.py", line 370, in parse_coverage
    _report_lines_with_errors(lines_with_errors, context)
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov_parser.py", line 509, in _report_lines_with_errors
    raise errors[0]  # guaranteed to have at least one exception
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov_parser.py", line 322, in parse_coverage
    tokenized_lines.append((_parse_line(raw_line), raw_line))
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov_parser.py", line 777, in _parse_line
    raise UnknownLineType(line)
gcovr.gcov_parser.UnknownLineType: branch  3 taken -20 (fallthrough)
Traceback (most recent call last):
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/workers.py", line 77, in worker
    work(*args, **kwargs)
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov.py", line 317, in process_datafile
    done = run_gcov_and_process_files(
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov.py", line 490, in run_gcov_and_process_files
    process_gcov_data(fname, covdata, abs_filename, options)
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov.py", line 145, in process_gcov_data
    coverage = parse_coverage(
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov_parser.py", line 370, in parse_coverage
    _report_lines_with_errors(lines_with_errors, context)
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov_parser.py", line 509, in _report_lines_with_errors
    raise errors[0]  # guaranteed to have at least one exception
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov_parser.py", line 322, in parse_coverage
    tokenized_lines.append((_parse_line(raw_line), raw_line))
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov_parser.py", line 777, in _parse_line
    raise UnknownLineType(line)
gcovr.gcov_parser.UnknownLineType: branch  3 taken -20 (fallthrough)
(ERROR) Uncaught EXCEPTION
Traceback (most recent call last):
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/bin/gcovr", line 8, in <module>
    sys.exit(main())
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/__main__.py", line 308, in main
    collect_coverage_from_gcov(covdata, options)
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/__main__.py", line 367, in collect_coverage_from_gcov
    contexts = pool.wait()
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/workers.py", line 170, in __exit__
    self.wait()
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/workers.py", line 161, in wait
    raise self.exceptions[0][1]
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/__main__.py", line 367, in collect_coverage_from_gcov
    contexts = pool.wait()
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/workers.py", line 161, in wait
    raise self.exceptions[0][1]
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/workers.py", line 77, in worker
    work(*args, **kwargs)
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov.py", line 317, in process_datafile
    done = run_gcov_and_process_files(
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov.py", line 490, in run_gcov_and_process_files
    process_gcov_data(fname, covdata, abs_filename, options)
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov.py", line 145, in process_gcov_data
    coverage = parse_coverage(
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov_parser.py", line 370, in parse_coverage
    _report_lines_with_errors(lines_with_errors, context)
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov_parser.py", line 509, in _report_lines_with_errors
    raise errors[0]  # guaranteed to have at least one exception
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov_parser.py", line 322, in parse_coverage
    tokenized_lines.append((_parse_line(raw_line), raw_line))
  File "/home/ThibFrgsGmz/Documents/tmp/__venv__/lib/python3.8/site-packages/gcovr/gcov_parser.py", line 777, in _parse_line
    raise UnknownLineType(line)
gcovr.gcov_parser.UnknownLineType: branch  3 taken -20 (fallthrough)
[INFO] Removing temporary directory: /home/ThibFrgsGmz/Documents/fprime-codelab/{{AUTOCODE}}

I didn't really look into it. Maybe it's my setup, where F´ is a sub-module of my project.