The goal of our project is to build a tool that can automatically analyze Peripheral Interupts for ARM embedded system.
The Econotag is an open source and exceptionally simple example of an embedded system which makes it easier for us to testing our approach.
Symbolic Execution(angr)
We start our project based on Isomano project, which used angr to implement a guided symbolic execution on econotag firmware. In this project, we optimized the program and made it automatic.
To make angr works on the econotag firmware we dumped, we did several hooks for those instructions that angr doesn’t support. Besides that, angr cannot to add constraints when it meets some arm branch case, to make our test case work, we also modified and patched the binary (econotag_angr_solve/econotag_new).
Input: python auto_map_handler.py econotag_new econotag_memdump.bin
Output: Found the following interupt number and handler mappings: 0x3f35: 3 0x1001d: 6, 0, 9, 10, 2, 4, 5, 8 0x4029d1: 7 0x3275: 1
After making success on the econotag system, we moved forward to a more complicated embedded device: raspberry pi 3. Unlike the previous test case, we are trying to test on an actual device (Raspberry Pi 3, Model B, 1GB RAM).
We first cross compile and build kernel for raspberry pi 3:
pi@raspberrypi:~ $ uname -ar
Linux raspberrypi 4.9.11-v7+ #1 SMP Tue Feb 21 16:51:15 EST 2017 armv7l GNU/Linux
The next step was trying to dump the memory out of raspberry pi, we tried to use LiME to extract the memory, but it didn’t work out. Then we turned to another way: since we know we can recompile the kernel, we can just re-enable access to /dev/mem. The way to do this is before building the kernel, do make menuconfig and set the CONFIG_STRICT_DEVMEM option to "no". Then we were able to just use dd on /dev/mem to create the memory dump.
Address translation
The memory dump we had was all based on physical address. To make angr work on the mem dump we just had, we need to figure out a way to let angr understand virtual address translation, and we did that using python (vtop_mem.py). After running our script on the original memory dump, now we have an address-translated version of memory dump.
Angr It!
Like I mentioned before, angr met a lot of incompatible bugs while trying to map the econotag interrupt handler, while econotag was an exceptionally simple example of an embedded system. In the raspberry pi case, it wasn’t easy to get angr work as well. We did cpsr_hook to handle incompatible instructions in angr, e.g. msr, mrs, etc. We did find a correct path when we were trying to symbolic execute from address 0xffff0018, which is the virtual address of vector_irq.
The path:
dereference handle_arch_irq
bcm2836_arm_irqchip_handle_irq : 80101464.
handle_domain_irq.
__handle_domain_irq : 8016f340.
generic_handle_irq : 8016ed34.
radix_tree_lookup : 0x8044bfa8.
__radix_tree_lookup : 0x8044bed0.
etc.
After poking around with angr for a couple of weeks, we found the raspberry memory may be too complicate for angr to solve, and there are too many bugs(e.g. Incompatible instructions, symbolic read issues, etc.) we need to fix to make angr work. We decided to try the fuzzing way using qemu/panda。
- Load memory and CPU state in a blank machine definition.
- Trigger interrupt using
cpu_interrupt(cpu, CPU_INTERRUPT_HARD). - Return random values from memory mapped I/O devices.
- Trace what code gets executed as a result.
- Repeat until no new code traced.
Because of the time is limited (graduation), I have to stop at step 2.
Load memory and CPU state
Because we want qemu load our memory file, we decided to use a customized machine definition: Rehosting machine.
Dumping the CPU state was a tricky part: To dump the CPU state(basically the value that each register holds), we build our own kernel module to do that. Check kernel_module_build/ for details.
You can just grab the kernel build module tool kit here, and replace the rpi/LiME/src folder with kernel_module_build folder in the repo.
export PATH=${PATH}:$(PWD)/rpi/tools/arm-bcm2708/gcc-linaro-arm-linux- gnueabihf-raspbian-x64/bin
cd path_to_rpi/rpi/LiME/src
./mcmd.sh # will compile main.c and create an kernel module
Then on raspberry pi:
sudo dmesg -C # clear kernel message
sudo insmod lime.ko
dmesg # now we have the cpu state
NOTE: using show_regs wasn't the best way of dumping the cpu state, because we may be missing lots of values of cpu states such as ttbr, banked registers values, etc. In our test case, the sp register value wasn't quite right because show_regs didn't load irq_svc mode stack address; and we did a hacky way to work around that.
Trigger interrupt using and trace the execution
After acquiring the cpu state, then we load them in, and trace the block execution.
arm-softmmu/qemu-system-arm -machine rehosting,mem-map="MEM 0x00000000-0x3b000000" -panda loadcpu -d in_asm -D qemu.log # will save trace to qemu.log.
In our project, we used show_regs to dump the current cpu_state, which may miss lots of useful registers, hence one of our future work is to figure out a better way of dumping cpu state, one possible approach is to take a snap shot using qemu/raspberry pi machine definition.
Because the limited time, we didn't get to move forward to next step on our fuzzing approach, since we have already successfully loaded the cpu state and make qemu run on the correct path, we should move forward on our fuzzing approach in the future.
Symbolic execution is 1337 (when it works), while angr is still on developing stage, in the future we probably can help fixing some angr issues and make it work on our arm architecture.
If you are intereted in our project, or want to help extending it, feel free to contact: