Add support for CMake

[NidasioAlberto]

Miosix CMake support

A brief history

Support for CMake was first proposed in #3 by @mscuttari (2019), then further developed by @damianoamatruda at Skyward (2022), and now finalized to be merged into the kernel.

Skyward's use case is very specific because in their two main repos they maintain a lot of executables targetting many different boards (in Boardcore there is a test executable for each sensor's driver and in On-Board Software there is an executable for each board on the rocket). For this reason, they basically created a CMake library for each board in order to define multiple executables targeting different boards in a single CMakeFile.txt.

Although the structure used by Skyward has worked well, to officially introduce CMake into the kernel we would like to follow as closely as possible the current Make based system. For this reason, Skyward's CMake was improved upon and now resembles much more a standard CMake based project.

Quick start

Move to you executable directory. We'll take miosix/_examples/blinking_led as an example.

Build your code with:

cmake -Bbuild -DCMAKE_TOOLCHAIN_FILE=../../cmake/Toolchains/gcc.cmake
cmake --build build --target blinking_led

And flash the binary on you board with:

cmake --build build --target blinking_led_program

In depth description

The proposed CMake base build system was developed targeting these features:

• Follow the current Make based build system structure
• Provide a modern and standard build system based on CMake
• Simplify project configuration for first time users by hiding Miosix specific configuration steps
• Build system level support for flashing binaries, like make program
• Possibility to use a custom miosix_settings.h outside of kernel director
• Possibility to support new boards outside of the kernel directory

The build system is contained in the miosix folder and composed of:

• CMakeLists.txt: Includes specific architecture/board files, defines common kernel source files, include directories and flags, and runs the kernel_global_objects.pl script.
• arch/CMakeLists.txt: Contains the list of supported boards, include the selected board's files.
• arch/<family>/CMakeLists.txt: Sets flags (e.g. -mcpu, -mfpu, ...) and source files common to all chips in the family.
• arch/<family>/<board>/CMakeLists.txt: Configures chip/board specific things like the linker script and clock frequency.

To simplify CMake files, common functionalities were moved into function which can be found in miosix/cmake. In particular miosix_link_target(TARGET) links a CMake target against the kernel, automatically configuring link libraries, binary file creation and the <TARGET>_program target which can be used to flash the board.

For the user to link with the kernel, a few lines of CMake are necessary:

set(MIOSIX_OPT_BOARD stm32f429zi_stm32f4discovery CACHE STRING "Target board")
add_subdirectory(../.. miosix EXCLUDE_FROM_ALL)
include(LinkTarget)

add_executable(blinking_led simple.cpp)
miosix_link_target(blinking_led)

Processes

Support for easily create processes was also added, along with a simple example (miosix/_examples/processes).

The idea boils down to create the usual target for your kernel, one target for each process and then a target to build the combined RomFS image.

# Kernel level program
add_executable(main main.cpp)
miosix_link_target(main)

# Processes
miosix_add_process(process1 process1.cpp)
miosix_add_process(process2 process2.cpp)

# RomFS image
miosix_add_romfs_image(
    IMAGE_NAME image
    DIR_NAME bin
    KERNEL main
    PROCESSES process1 process2
)

This will define two additional targets: image to build the RomFS image, and image_program to flash the binary on the board.

Simple tutorial

After cloning Miosix you will find many examples in the miosix/_examples folder. We'll take blinking_led as an example.

The CMakeFile.txt very simple, here a quick overview:

cmake_minimum_required(VERSION 3.16)
project(BlinkingLED C CXX ASM)

These are common to any CMake project. They specify the minimum required version of CMake (3.16 is shipped with Ubuntu 20.04, so you should be fine) and define a new project called "BlinkingLED".

add_compile_definitions($<$<COMPILE_LANGUAGE:C,CXX>:PARSING_FROM_IDE>)

This tells the compiler to define PARSING_FROM_IDE so that the check in miosix_settings.h gets bypassed.

set(MIOSIX_OPT_BOARD stm32f429zi_stm32f4discovery CACHE STRING "Target board")

This tells the build system which board you are using. You can find all the supported boards in the miosix/arch folders or by defining the MIOSIX_PRINT_BOARD_LIST option (e.g. by running cmake -Bbuild -DMIOSIX_PRINT_BOARD_LIST=True).

add_subdirectory(../.. miosix EXCLUDE_FROM_ALL)
include(LinkTarget)

These includes the Miosix CMake project and provide the miosix_link_target function.

add_executable(blinking_led simple.cpp)
miosix_link_target(blinking_led)

These create a new target named blinking_led with the source file simple.cpp and links the miosix library.

You can now build your code on the board with these two commands:

cmake -Bbuild -DCMAKE_TOOLCHAIN_FILE=../../cmake/Toolchains/gcc.cmake
cmake --build build --target blinking_led

And if you also want to flash the code automatically on the board, you can run:

cmake --build build --target blinking_led_program

TODOs

• Review MxGui build system
• Review TsCpp build system
• Properly handle MxGui examples (how do we build them?)
• Test debugger setup

Related pull requests:

• Add support for CMake and standardized project structure tscpp#1
• Add support for CMake mxgui#3

[NidasioAlberto]

During refactoring I've also tried to add support for CMake install feature. It can be done but the CMake code becomes messy and you can only install one build configuration (Release/Debug) at a time. I don't think this feature is needed, just add the kernel as a submodule

[danielecattaneo]

I've started looking at the changes... I see the miosix_np_2 directory still survives, even though it can be removed at this point; CMake already provides support for IDEs built-in, so keeping the old NetBeans-specific files doesn't make much sense.

Another thing I would change is to move the targets for the examples to the _examples directory, and split them into one for each sample instead of having them all together. Same thing holds for the testsuite. I would keep the current structure where at the top-level of the repo there is just a very simple template.

I'm also not sure about renaming all the boards now, as we are thinking of splitting the concept of chips and boards so that may require further change in the future. But let's keep that for now.

[danielecattaneo]

Also, I wouldn't worry about supporting installs for now, I don't see many use-cases for it.

[NidasioAlberto]

I'd really love to remove the miosix_np_2 directory, I thought it was not in the scope of this Pull Request and left it there. In my opinion we should remove miosix_np_2 and restructure the repo in a more standard way:

• The content of the miosix directory should be moved to the root
• Directories starting with an underscore should have the underscore removed (e.g. _examples to examples)
• Headers and sources should be moved into include and src, as standard practice

We could think about these ideas when we'll address Split board e chip. After merging this I could try out some of them. Here is a good reference about how to structure C++ project I've read some times ago.

Creating one CMakeLists.txt for each example (and testsuite) I think is a good idea. In the upcoming days I'll do it.

I'm not a big fan of the top-level main.cpp, I would make it one of the examples. With the new CMake build system, going forward the workflow should be this:

• If you are developing an application with Miosix as your OS (like Skyward or an AOS project like what I did last year), you should add the kernel repo as a submodule (like in the example here)
• If you are working on a kernel feature you should develop an example (or better a test), this way you are also encouraged to provide an example with your new feature. Or you could do the same as above and just commit the changes in the submodule

About renaming some of the boards (basically I didn't like repeating stm32fx in boards like stm32f407vg_stm32f4discovery) I've left that as its own commit, so we can leave it out if we don't like it. I think it is irrelevant wrt future restructuring.

[NidasioAlberto]

I've split up the targets into multiple CMakeLists.txt, one for example/test. The major downside is that this ways you cannot build all of these targets in one go.

This could be very useful to verify that all code compiles when accepting new patches. At Skyward we have a pipeline running on every commit that notifies you via mail if your changes made some of the targets fail to compile.

One single CMakeLists.txt makes it also much faster to jump from one example/test to the next.

Now you need to manually build every example/test or write an additional script to do it for you (this is not a suggestion to do so 😅). A compromise could be to have a single CMakeList.txt for all the examples and another one for all the tests.

We could also move the tests from the _tools directory to a new _tests directory, now in _tools there are also other things like the compilers and bootloader.

[danielecattaneo]

I tried to build Miosix with the current state of the Cmake build system, and I found some issues.

• The toolchain file is never referenced, making the build fail by default.
• The convention for CMake projects is to have the cmake directory added to the module path, but this is not done.
• The file names for the modules in the cmake directory use a snake_case naming convention while CMake prefers the UpperCamelCase convention.
• The board_options.cmake files are inside the miosix/config/arch/ directory. This doesn't make much sense to me, as they also declare which files are part of the BSP of each board. These files should be in miosix/arch/. If we want to keep a user-configurable .cmake files in miosix/config/arch, they should be included from the board_options.cmake in miosix/arch.
• The declaration of the list of Miosix source code files is done in an include and not in the main CMakeLists file, which makes them hard to find and modify.
• Same as above but for the board option files. In my opinion the discovery of the board_options.cmake files should be done in a CMakeLists.txt inside miosix/arch. With the location change of the board_options.cmake file, this would make the hierarchy of the include operations match the hierarchy of the build tree.
• The toolchain file declarares the CMAKE_SYSTEM_NAME to be Linux, but this is of course incorrect. If we specify Generic as the system name, CMake sets the object file extension to .o, which is not what we want. Even when using the Linux setting the final .elf doesn't have any extension and this is not what we want either. We should add a platform file for Miosix by creating a Platform/Miosix.cmake file in the module path: at that point we can specify Miosix as the CMAKE_SYSTEM_NAME.
• The LINK_GROUP:RESCAN generator expression in miosix_link_target.cmake doesn't work for me, it complains that Feature 'RESCAN', specified through generator-expression '$<LINK_GROUP>' to link target 'main', is not supported for the 'CXX' link language.. This error doesn't make any sense to me, and I couldn't figure out (yet) the reason why it pops up. A workaround is to remove the generator expression and add start/end group manually:

    # Link libraries
    target_link_libraries(${TARGET} PRIVATE
        $<TARGET_OBJECTS:Miosix::boot-${OPT_BOARD}>
        -Wl,--start-group
        Miosix::miosix-${OPT_BOARD} stdc++ c m gcc atomic
        -Wl,--end-group
    )

• The minimum required version for CMake is currently too high at 3.25. Consider that Ubuntu 22.04 still ships 3.22, and in general it's not reasonable to force users to very recent versions of a dependency which updates itself so often as CMake does (on average a new version of CMake is released 3 times per year). Apart from the LINK_GROUP generator expression that doesn't work I don't think there are other things that require a recent CMake version. I think a good target CMake version is 3.16, as that's the version shipped with Ubuntu 20.04 LTS, the oldest currently supported Ubuntu release.
• It seems to me that the sheer number of targets is breaking the Makefile generator's percentage progress reporting: it goes from 0% to 50% to 100% with nothing in between. Maybe we should either generate board targets dynamically -- only when they are requested -- or follow Federico's suggestion to only supporting the build of one board at a time.
• With the current implementation of the build system, the list of boards is not very discoverable. I propose adding an option() that, when set to YES, enables printing out the list of boards supported.

The list is a bit long... but it really didn't work out of the box.

[danielecattaneo]

I've split up the targets into multiple CMakeLists.txt, one for example/test. The major downside is that this ways you cannot build all of these targets in one go.

This could be very useful to verify that all code compiles when accepting new patches. At Skyward we have a pipeline running on every commit that notifies you via mail if your changes made some of the targets fail to compile.

One single CMakeLists.txt makes it also much faster to jump from one example/test to the next.

Now you need to manually build every example/test or write an additional script to do it for you (this is not a suggestion to do so 😅). A compromise could be to have a single CMakeList.txt for all the examples and another one for all the tests.

Declaring the examples all together in the same file is not the only way to build them all. One could create a top-level CMakeLists.txt that contains one add_subdirectory per every example, or write a temporary one-liner shell script that does the build thingamajig once for every target :)

The fact that you want to build all the examples to check if the build system works is not the rule but the exception. A normal user typically wants to do one of these things:

• look at the example code without building it
• copy-paste the example as a template
• build one single example that is interesting to him

Having all the examples declared together makes all these things more difficult for no reason.

[NidasioAlberto]

The toolchain file is never referenced, making the build fail by default.

The toolchain file is not referenced because it is supposed to be specified during the build system generation. I usually run:

cmake .. --toolchain=../miosix/cmake/toolchain.cmake -DCMAKE_BUILD_TYPE=Release
make

The convention for CMake projects is to have the cmake directory added to the module path, but this is not done.

Thanks for pointing it out, I've read most of the CMake documentation but missed it.

The declaration of the list of Miosix source code files is done in an include and not in the main CMakeLists file, which makes them hard to find and modify.

Same as above but for the board option files. In my opinion the discovery of the board_options.cmake files should be done in a CMakeLists.txt inside miosix/arch. With the location change of the board_options.cmake file, this would make the hierarchy of the include operations match the hierarchy of the build tree.

The files kernel_sources.cmake and boards.cmake I think are very straight forward, but there is no problem in placing the two lists directly in the CMakeLists.txt. The limit is that we must include() the board_options.cmake in the miosix_add_board_libraries function because they all define the same variables. This would change if we define a single library target as discussed with Federico.

The toolchain file declarares the CMAKE_SYSTEM_NAME to be Linux, but this is of course incorrect. If we specify Generic as the system name, CMake sets the object file extension to .o, which is not what we want. Even when using the Linux setting the final .elf doesn't have any extension and this is not what we want either. We should add a platform file for Miosix by creating a Platform/Miosix.cmake file in the module path: at that point we can specify Miosix as the CMAKE_SYSTEM_NAME.

Other than the CMAKE_SYSTEM_NAME being Miosix and the object file extensions, what should we specify in a custom platform file? Which elf extension do we want? (I guess .a for libraries and .o for objects like in the current build system)

The minimum required version for CMake is currently too high at 3.25

I definitely need to lower the required version. I was already planning to do so, I'll install the 3.16 version and continue testing with that. As you said, other than the LINK_GROUP there should be no dependency on newer feature.

It seems to me that the sheer number of targets is breaking the Makefile generator's percentage progress reporting: it goes from 0% to 50% to 100% with nothing in between. Maybe we should either generate board targets dynamically -- only when they are requested -- or follow Federico's suggestion to only supporting the build of one board at a time.

I'll try to support one board at a time to see also if this behavior changes.

In general I'll continue working on what you pointed out, thanks!

[NidasioAlberto]

I should have addressed all of the things you pointed out.

The toolchain file is never referenced, making the build fail by default.

We can specify the toolchain file in the CMakeLists using set(CMAKE_TOOLCHAIN_FILE ...) before calling project(). For now I still set CMAKE_TOOLCHAIN_FILE in the call to cmake.

To build the main entrypoint in the root:

mkdir build && cd build
cmake .. \
    -DCMAKE_TOOLCHAIN_FILE=../miosix/cmake/Toolchains/gcc-9.2.0.cmake \
    -DCMAKE_BUILD_TYPE=Debug \
    -DMIOSIX_OPT_BOARD=stm32f407vg_stm32f4discovery
make

The convention for CMake projects is to have the cmake directory added to the module path, but this is not done.

The file names for the modules in the cmake directory use a snake_case naming convention while CMake prefers the UpperCamelCase convention.

The toolchain file declarares the CMAKE_SYSTEM_NAME to be Linux, but this is of course incorrect. If we specify Generic as the system name, CMake sets the object file extension to .o, which is not what we want. Even when using the Linux setting the final .elf doesn't have any extension and this is not what we want either. We should add a platform file for Miosix by creating a Platform/Miosix.cmake file in the module path: at that point we can specify Miosix as the CMAKE_SYSTEM_NAME.

Now the files follows the CMake naming conventions. I've also created a simple platform file that should set the correct file extensions. This allows to set Miosix as CMAKE_SYSTEM_NAME but requires the path Platform/Miosix.cmake to be in the module path. For this reason the toolchain file adds to CMAKE_MODULE_PATH to path to the miosix/cmake directory.

One thing to note is that the object files still have the original file extension (stage_1_boot.cpp becomes stage_1_boot.cpp.o). CMake does this to handle the case where there are different files with the same name but with different extension (e.g. foo.c and foo.cpp). This can be bypassed with set(CMAKE_<LANG>_OUTPUT_EXTENSION_REPLACE 1) but this variable is not documented, so I assume it could change in the future. I think it is better to keep the original file extension.

Another thing to not is that set(CMAKE_ASM_OUTPUT_EXTENSION .o) does not seems to work and I've not found out why yet.

The board_options.cmake files are inside the miosix/config/arch/ directory. This doesn't make much sense to me, as they also declare which files are part of the BSP of each board. These files should be in miosix/arch/. If we want to keep a user-configurable .cmake files in miosix/config/arch, they should be included from the board_options.cmake in miosix/arch.

Since all these options where defined in miosix/config/Makefile.inc we had left them all in config. Now the content of board_options.cmake files have been moved in the miosix/arch directory and split into multiple CMakeLists.txt. This follows the double switch construct in miosix/config/Makefile.inc.

• miosix/arch/CMakeLists.txt
• miosix/arch/<arch_name>/CMakeLists.txt
• miosix/arch/<arch_name>/<board_name>/CMakeLists.txt

Other notes

I've made two big changes with respect to the previous version:

1. I've followed @fedetft's suggestion to support one board at a time (like the current build system). The Skyward's use case of multiple targets with different boards can still be supported by creating a CMakeLists.txt for each target and by adapting the already existing script. Unfortunately you cannot create a top-level CMakeLists.txt that contains one add_subdirectory for every target because you would redefine the miosix library already defined in previous files, but this is acceptable
2. All the options configurable in the build system are now CMake cache entries that can easily be configured also with tools like cmake-gui and ccmake

Now the new build system is much more similar the the current one. For now I've only added support for the stm32f407vg_stm32f4discovery and stm32f429zi_stm32f4discovery, I'll add all the others later on.

I still need to review MxGui, but it should not be too difficult.