Overview
How to Clone
How to Build
How to Run
How to Debug
Unit Test Validation
Important Notes
Samples
C/C++ Code Coverage using GCOV
Assembly Language Code Coverage
pinkySim is an ARMv6-M instruction set simulator. It was developed to facilitate the use of Test Driven Development, TDD, on future projects where the code under test will be written in assembly language. With this goal in mind it has these features:
- Supports the core ARMv6-M instruction set needed to run unit tests on the developer's host machine as part of the automated cross-compile/build process.
- pinkSim itself contains a healthy suite of unit tests:
- Contains >450 unit tests that run as part of the automated build process.
- Covers 100% of the code in pinkySim.c.
- A good proportion of the unit tests have been deployed and validated on actual ARM hardware.
- Supports arm-none-eabi-gdb debugging of Thumb code running in the simulator.
- Ability to perform code coverage analysis. This will allow for the determination of the level of code coverage obtained on assembly language code by its corresponding unit tests.
This project uses submodules (ie. MRI). Cloning requires a few more steps to get all of the necessary code.
git clone --recursive [email protected]:adamgreen/pinkySim.git
- or -
git clone [email protected]:adamgreen/pinkySim.git cd pinkSim git submodule init git submodule update cd mri git submodule init git submodule update
- but, actually on OSX you may need to... (TODO: This is hackish and wrong, but maybe unique to my system...) -
git clone https://github.com/greggersaurus/pinkySim.git cd pinkySim git clone https://github.com/adamgreen/mri.git cd mri git clone https://github.com/adamgreen/CppUTest.git
pinkySim uses a single non-recursive makefile at the root of the project to build everything. It supports these top level targets:
- all: This builds the pinkySim code, builds the unit tests, executes the unit tests, and reports the test results. This is the default target if no other is provided to make.
- clean: Cleans up all ouptut files from any previous builds. This forces everything to be built on the next make run.
- gcov: Like the all target, this builds all of the pinkySim code and runs the unit tests but it also instruments the binaries with code coverage tracking and then reports the code coverage obtained from executing the unit tests.
- test: This builds and runs some very special tests which aren't required for the main pinkySim funtionality. There is a section below related to the unit test validation suite run by specifying this target.
Example:
make all - Build pinkySim.
The makefile is constructed in such a way that all dependencies for any single target within the build system are known by GNU Make, including header file dependencies. This means that the user can specify a specific binary target on the make command line and it should successfully build it and all of its dependencies.
Examples:
make lib/libpinkysim.a - Build the main pinkySim library.
make LIBPINKYSIM_tests - Build the main pinkySim library, unit tests, and all required dependencies.
make RUN_LIBPINKYSIM_TESTS - This does what the previous example does but also runs the unit tests.
make GCOV_LIBPINKYSIM - This is similar to the previous example except that it builds binaries which include code coverage instrumentation and then reports the resulting code coverage results after running the unit tests.
Example:
make pinkySim - Build only pinkySim (i.e. no tests).
TODO: Another instance of a possible issue on my build system. Tests fail to build, but this makes sure that at least we can an executable we can use.
The user should feel free to use the -j or --jobs command line flag with GNU Make to let it perform the required build steps in parallel to decrease the build time. It can be hard to read the output when using this feature since the output from the parallel executing commands can be interleaved in a random fashion. If this becomes an issue when attempting to track down a build issue, then just drop the -j flag while debugging.
Example:
make -j4 all - Tells GNU Make that it can have a maximum of 4 commands running at the same time while it is building the default all target.
Note: Multiple targets can be specified on the make command line in combination with this parallel build flag except for the clean target which shouldn't be combined with any others.
All intermediate binaries (*.o and *.a) for code coverage runs are found in the gcov/obj and gcov/lib directories. If you want to examine the line by line coverage results for a library like libpinkysim, you would find it in the gcov/LIBPINKYSIM_tests/ directory.
Usage:
pinkySim [--ram baseAddress size] [--flash baseAddress size] [--gdbPort tcpPortNumber] [--breakOnStart] [--codecov application.elf resultsDirectory] [--restrict sourcePathPrefix] [--logExe chipType] imageFilename.bin [args]
--ram is used to specify an address range that should be treated as read-write. More than one of these can be specified on the command line to create multiple read-write memory regions.
--flash is used to specify and address range that should be treated as read-only. More than one of these can be specified on the command line to create multiple read-only memory regions.
--gdbPort can be used to override the default TCP/IP port of 3333 for listening to GDB connections.
--breakOnStart can be used to have the simulator halt at the beginning of the reset handler and wait for GDB to connect.
--codecov can be used to specify that machine code level code coverage results should be generated based on this simulation. The application.elf argument specifies the .ELF file containing the symbols for the binary being simulated. The resultsDirectory indicates in which directory the code coverage result files should be placed. The results include a summary.txt and a file for each source file providing details on which lines were executed and which were not, similar to GCOV.
--restrict options can be used to specify if the code coverage results generated by the --codecov option should be restricted to source files which have the specified sourcePathPrefix. More than one of these options can be specified on the command line.
--logExe option where a log of the form exeLog_*.csv will be created that logs each instruction (where '*' is filled in with a timestamp of when the first instruction or data access occurred). This option will also result in the creation of a file of the form exeLog_*.c, which attempts to create a naive C style representation of what the simulated code has been doing. This presents the simulation results in a different format that may be easier to conceptualize for some users. The logging feature is intended for studying executions as well as for defining valid instructions and data entries for executables that have had their symbol tables stripped (i.e. raw binary format). Currently only chipType of LPC11U37 is supported.
imageFilename.bin is the required name of the image to be loaded into memory starting at address 0x00000000. By default a read-only memory region is created starting at address 0x00000000 and extends large enough to contain the whole image file. A read-write section will be created based on the initial stack pointer found in the first word of the image file. This section will start at the nearest 256MB page below this initial SP value and extend to the address just below this initial SP value. This behaviour can be overridden by specifying --ram and --flash options on the command line. Execution will start at the address found in the second word of this image.
[args] are optional arguments to be passed into program running under simulation.
Examples:
pinkySim samples/CommandLine_Sample.bin arg1 arg2 - Simulates the CommandLine sample, passing in two arguments: arg1 and arg2.
pinkySim --breakOnStart samples/FileTest_Sample.bin - Simulates the FileTest sample, halts at the beginning of Reset_Handler() and waits for GDB to be connected.
../pinkySim --codecov armv6m/FileTest_Sample.elf results --restrict FileTest/ --restrict libstartup/ FileTest_Sample.bin - Simulates the FileTest samples and then outputs the code coverage results to the results/ directory. The simulation must be run from the same directory as the build since the source file paths in the .ELF symbols are relative to that location.
pinkySim has the ability to act as a GNU Debugger (GDB) remote target server. By default pinkySim will listen on TCP/IP port 3333 for connections from GDB but the developer can override via the use of the --gdbPort command line option. Once GDB connects to pinkySim, it can debug ARMv6-M executables running in the simulator just like JTAG debugging on real hardware. This includes debugging features like:
- hardware breakpoints (PC memory is the only limit to number supported)
- data watchpoints (PC memory is the only limit to number supported)
- single stepping
- halting of running/hung applications
- accessing variables sitting in simulated memory
- stdout/stderr/stdin redirection to/from the GDB console once attached
- works with free GNU Tools for ARM Embedded Processors
When launching a binary to run under the simulator, the --breakOnStart command line option can be specified to have the simulator halt execution at the very beginning of the Reset_Handler and wait for GDB to be attached. This allows the developer to setup breakpoints and watchpoints before global constructors and main() are executed. Here is an example of running pinkySim with the --breakOnStart option and the console output which results indicating that GDB needs to be attached:
/depots/pinkysim$ pinkySim --breakOnStart samples/FileTest_Sample.bin Waiting for GDB to connect...
Now the developer can attach GDB to pinkySim by issuing a target remote localhost:3333 command as seen in this example debug session:
/depots/pinkySim/samples$ arm-none-eabi-gdb armv6m/FileTest_Sample.elf
GNU gdb (GNU Tools for ARM Embedded Processors) 7.6.0.20140228-cvs
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "--host=x86_64-apple-darwin --target=arm-none-eabi".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
Reading symbols from /depots/pinkySim/samples/obj/FileTest_Sample.elf...done.
(gdb) target remote localhost:3333
Remote debugging using localhost:3333
Reset_Handler (argc=1, argv=0x1fffffdc) at libstartup/Reset.c:43
43 {
(gdb) list
38 const uint32_t isr_vector[2] = { (uint32_t)__StackTop, (uint32_t)Reset_Handler };
39
40
41 /* This function is the first thing executed after reset. It sets up the C runtime and then calls main(). */
42 void Reset_Handler(int argc, const char** argv)
43 {
44 int dataSize = (int)__data_end__ - (int)__data_start__;
45 int bssSize = (int)__bss_end__ - (int)__bss_start__;
46
47 memcpy(__data_start__, __etext, dataSize);
(gdb) p argv[0]
$1 = 0x1fffffc0 "samples/FileTest_Sample.bin"
(gdb) break main
Breakpoint 1 at 0xbc: file FileTest/main.c, line 19.
(gdb) c
Continuing.
Note: automatically using hardware breakpoints for read-only addresses.
Breakpoint 1, main () at FileTest/main.c:19
19 {
(gdb) list
14 #include <stdio.h>
15 #include <string.h>
16
17
18 int main(void)
19 {
20 static const char Filename[] = "foo.bar";
21 unsigned char TestBuffer[256];
22 unsigned char ReadBuffer[256];
23 size_t i;
(gdb) As pinkySim was developed, corresponding unit tests were written as well. However one major concern was that misinterpretations of the ARMv6-M instruction set description would find their way into both the unit tests and the actual simulator code. To help reduce the chance of this occurring, the thunk2real subproject was created. This subproject uses a test mock in the place of pinkySim which redirects the process of single stepping to actual ARM hardware. This is accomplished by connecting the test mock to an an actual ARM Cortex-M3 device (mbed-LPC1768) via the GDB remote serial protocol. To run this suite you must:
- Be running the build process from a Macintosh computer running OS X with Xcode installed.
- Build and deploy the realarm firmware to an mbed-LPC1768 device.
- This mbed device must first be running the 21664 interface firmware.
- Setup the GCC4MBED project as a sibling to your pinkySim repository clone.
- Build and deploy by running make deploy from the realarm directory.
- Reboot the mbed device.
- Make sure that the mbed device is connected to your Macintosh before running the next step.
- Build the tests. This can be done by going to the root pinkySim directory on your machine and running make test. This will not only build the tests but it will also detect your connected mbed device and automatically run the tests on it.
Due to the intended use of this simulator, there are several ARMV6-M architectural features which aren't implemented:
- Not cycle accurate: The simulator just executes instructions as it encounters them in the code stream and has no knowledge of clock cycles.
- No memory mapped peripherals: The simulator just issues read/writes to memory. It doesn't attempt to map special memory regions to peripheral registers. This includes the ARMV6-M specified peripherals such as the NVIC.
- Doesn't support exception/interrupts: If the simulator encounters an instruction which would throw an exception, it just returns a non-zero return value from the pinkySimStep() routine. As the simulator doesn't attempt to provide any implementation of peripherals, it also doesn't implement support for interrupts that might occur from such peripherals.
- Only runs in Privileged Thread mode: The simulator doesn't support the switching of the processor into Unprivileged mode. Since exceptions aren't supported, the processor can never enter Handler mode either.
- Main SP only: The simulator doesn't support switching to the process stack pointer.
The pinkySim project contains a few samples that can be built and executed on the simulator.
| Sample | Description |
|---|---|
| CommandLine | Has a main() function which dumps the argv command line arguments passed into the sample via the pinkySim command line. Upon exit it returns argc, the command argument count, and this will be returned from pinkySim to the launching shell. |
| StdIO | Writes text to stdout and reads input from stdin. These reads and writes will be redirected to the GDB console if it is connected to pinkySim. Otherwise it uses the pinkySim console. |
| FileTest | Performs file operations such as fopen(), fwrite(), fread(), etc. |
The samples require that the GNU Tools for ARM Embedded Processors C/C++ toolchain be installed and added to your PATH. To build the samples, run the following commands from the Terminal / Command Prompt in the root directory of the pinkySim repository:
cd samples make all
The resulting binaries for simulation will be found in the samples directory. The binaries with the _gcov.bin suffix contain GCC instrumentation to record C/C++ code coverage results upon exit. The ELF, map, and disassembly for each sample can be found in the samples/armv6m directory (samples/gcov for code coverage versions.)
Key files used by the samples:
| pinkySim.ld | Directs the linker to place code in a 256MB FLASH region starting at address 0x00000000 and variables in a 256MB (minus 32 bytes) RAM region staring at 0x10000000. It also places the isr_vector array from libstartup/Reset.c at the beginning of FLASH where the simulator expects to find the initial stack pointer and reset address. |
| libstartup/Reset.c | Has the isr_vector array which contains the initial stack pointer value (pointing to the top of RAM) and the Reset_Handler function pointer. It also contains the Reset_Handler() code which initializes the C runtime and then transfers control to the sample's main(). |
| libstartup/NewlibRetarget.c | Contains the sys call implementations required by the newlib C standard library. These sys calls are called from newlib when it needs to perform operations such as opening, reading, writing, and closing files on the target system. |
| libstartup/SemihostThunks.s | These ARMV6-M assembly language thunks provide routines that can be called from the NewlibRetarget.c sys calls to transfer execution to pinkySim so that they can be performed on the host system. A BKPT (breakpoint) instruction with a unique constant is used to signal pinkySim that a specific operation is being requested. The include/NewlibSemihost.h header defines the BKPT operation constants supported by pinkySim. These thunks also take care of calculating the length of string arguments and setting errno upon return if an error was encountered by the host as it attempted to execute the sys call on behalf of the simulated program. |
It is possible for GCC to instrument the binaries it produces with code coverage instrumentation by adding the -fprofile-arcs -ftest-coverage flags to the compiler command line. Linking with libgcov.a by adding -lgcov to the linker command line is also required. ARMv6-M binaries with this instrumentation will successfully execute and log their code coverage results under pinkySim.
After performing a build in the /samples directory of the pinkySim project you will be have two .bin files for each sample, the one with the _gcov.bin suffix will include the instrumentation. Running these binaries under pinkySim will generate .gcda coverage results files in the gcov/obj directories next to the corresponding object files. The run-gcov.sh script takes the name of a sample and extracts the code coverage information from these result files. What follows is an example of building the samples, running the FileTest sample and then extracting the human readable results:
/depots/pinkySim/samples$ make Compiling libstartup/NewlibRetarget.c Compiling libstartup/Reset.c Assembling libstartup/SemihostThunks.S Building armv6m/lib/libstartup.a Compiling libstartup/NewlibRetarget.c Compiling libstartup/Reset.c Assembling libstartup/SemihostThunks.S Building gcov/lib/libstartup.a Compiling FileTest/main.c Building armv6m/FileTest_Sample.elf Extracting disassembly to armv6m/FileTest_Sample.disasm Extracting FileTest_Sample.bin Compiling FileTest/main.c Building gcov/FileTest_Sample.elf Extracting disassembly to gcov/FileTest_Sample.disasm Extracting FileTest_Sample_gcov.bin Compiling CommandLine/main.c Building armv6m/CommandLine_Sample.elf Extracting disassembly to armv6m/CommandLine_Sample.disasm Extracting CommandLine_Sample.bin Compiling CommandLine/main.c Building gcov/CommandLine_Sample.elf Extracting disassembly to gcov/CommandLine_Sample.disasm Extracting CommandLine_Sample_gcov.bin Compiling StdIO/main.c Building armv6m/StdIO_Sample.elf Extracting disassembly to armv6m/StdIO_Sample.disasm Extracting StdIO_Sample.bin Compiling StdIO/main.c Building gcov/StdIO_Sample.elf Extracting disassembly to gcov/StdIO_Sample.disasm Extracting StdIO_Sample_gcov.bin /depots/pinkySim/samples$ ../pinkySim FileTest_Sample_gcov.bin Hello World Test completed /depots/pinkySim/samples$ ls -l gcov/obj/FileTest/ total 144 -rw-r--r-- 1 adamgreen wheel 42 May 25 22:41 main.d -rw-r--r-- 1 adamgreen wheel 476 May 25 22:42 main.gcda -rw-r--r-- 1 adamgreen wheel 3352 May 25 22:41 main.gcno -rw-r--r-- 1 adamgreen wheel 59064 May 25 22:41 main.o /depots/pinkySim/samples$ run-gcov.sh FileTest gcov/obj/libstartup/SemihostThunks.gcno:cannot open notes file 40.00% of 10 Creating 'Reset.c 60.00% of 45 Creating 'main.c 62.26% of 53 Creating 'NewlibRetarget.c Detailed code coverage results can be found in gcov/FileTest /depots/pinkySim/samples$ ls -l gcov/FileTest total 48 -rw-r--r-- 1 adamgreen wheel 111 May 25 22:42 FileTest_output.txt -rw-r--r-- 1 adamgreen wheel 6532 May 25 22:42 NewlibRetarget.c.gcov -rw-r--r-- 1 adamgreen wheel 3191 May 25 22:42 Reset.c.gcov -rw-r--r-- 1 adamgreen wheel 4835 May 25 22:42 main.c.gcov
The *.c.gcov files under the gcov/FileTest/ directory contain the source code with the left hand column used to indicate how many times each line was executed, flagging lines not covered with #####.
While the GCOV tool only works for C/C++ code, pinkySim is able to monitor and record all memory accesses so that it can provide detailed reports at the end of a run indicating which machine code instructions were executed and which were not. To complete this analysis it just needs to be given the location of the .ELF file with symbols for the binary being simulated and a directory location for where it should place the code coverage results.
What follows is an example of running the FileTest sample under pinkySim and then having it extract the code coverage results:
/depots/pinkySim/samples$ ../pinkySim --codecov armv6m/FileTest_Sample.elf results --restrict FileTest/ --restrict libstartup/ FileTest_Sample.bin Hello World Test completed Code coverage results can be found in results. /depots/pinkySim/samples$ ls -l results/ total 64 -rw-r--r-- 1 adamgreen wheel 5640 Jul 29 07:54 NewlibRetarget.c.cov -rw-r--r-- 1 adamgreen wheel 2709 Jul 29 07:54 Reset.c.cov -rw-r--r-- 1 adamgreen wheel 6312 Jul 29 07:54 SemihostThunks.S.cov -rw-r--r-- 1 adamgreen wheel 4112 Jul 29 07:54 main.c.cov -rw-r--r-- 1 adamgreen wheel 127 Jul 29 07:54 summary.txt /depots/pinkySim/samples$ cat results/summary.txt 61.90% FileTest/main.c 89.13% libstartup/NewlibRetarget.c 100.00% libstartup/Reset.c 73.08% libstartup/SemihostThunks.S
The *.cov files under the gcov/FileTest/ directory contain the source code with the left hand column used to indicate how many times each line was executed, flagging lines not covered with #####.