README.md

TinyEngine: Codebase of Demo Tutorial for Inference (Visual Wake Words)

This is the official demo tutorial for deploying a visual wake words (VWW) inference model on STM32F746G-DISCO discovery boards by exploiting TinyEngine.

Codebase Structure

(The codebase structure after completing the codebase construction process, which is shown below in Detailed Instruction)

.
├── ...
├── Debug                                  # Debug file folder.
│   ├── TinyEngine_vww_tutorial.elf        # The ELF file to be run on MCUs.
│   └── ...
├── Drivers                                # Driver file folder.
├── Inc                                    # Header file folder.
├── Src                                    # Source file folder.
│   ├── main.cpp                           # Main source file.
│   ├── TinyEngine                         # TinyEngine folder.
│   │   ├── codegen                        # Code generation folder.
│   │   │   ├── Include
│   │   │   │   ├── genModel.h             # Header file of memory allocation.
│   │   │   │   └── ...
│   │   │   └── Source
│   │   │       ├── genModel.c             # Codegen of computational graph.
│   │   │       └── ...
│   │   ├── include
│   │   └── src/kernels
│   │       ├── fp_requantize_op           # Operators with floating-point requantization.
│   │       ├── fp_backward_op             # FP32 Operators for backward propagation.
│   │       └── int_forward_op             # INT Operators for forward propagation.
│   └── ...
└── ...

Equipments

1. STM32F746G-DISCO discovery board
2. Arducam Shield Mini 2MP Plus (Optional)
3. Male to female jumper wires (x8) (Optional)

Overview of Usage

1. Download and install STM32CubeIDE version 1.5.0.
2. Download and import this project into your STM32CubeIDE.
3. Connect your Arducam to the board with jumper wires. (Optional)
4. Compile and flash program into your STM32F746G-DISCO discovery board.
5. Done! In the demo, the LCD screen on your STM32F746G-DISCO discovery board should display person detection results (person/no person) and frames per second (FPS).

Detailed Instruction

0. Prepare an STM32F746G-DISCO discovery board (and an Arducam, if applicable).
1. Download STM32CubeIDE, an C/C++ development platform with peripheral configuration, code generation, code compilation, and debug features for STM32 microcontrollers and microprocessors.

• Please download STM32CubeIDE version 1.5.0. [Link]
• Please refer to “UM2563 STM32CubeIDE installation guide” and “UM2553 STM32CubeIDE quick start guide” for more detailed installation and user guides. [Link]

2. Prepare the codebase.

• First, please follow the instruction in Setup for Users to setup your environment.
• Copy the tutorial/inference folder and rename it as TinyEngine_vww_tutorial (for use in STM32CubeIDE later).

cp -r ./tutorial/inference ./tutorial/TinyEngine_vww_tutorial

• Setup PYTHONPATH, and run the codegen example of VWW:

export PYTHONPATH=${PYTHONPATH}:$(pwd)
python examples/vww.py  # To apply patch-based inference, please use `example/vww_patchbased.py`

• Move the new generated codegen folder to the following path:

mkdir ./tutorial/TinyEngine_vww_tutorial/Src/TinyEngine
mv codegen ./tutorial/TinyEngine_vww_tutorial/Src/TinyEngine

• Copy and paste the TinyEngine folder to the following path:

cp -r ./TinyEngine/include ./tutorial/TinyEngine_vww_tutorial/Src/TinyEngine
cp -r ./TinyEngine/src ./tutorial/TinyEngine_vww_tutorial/Src/TinyEngine

• Copy and paste the required Arm files to the correct path by using the following shell script:

bash import_arm_inference.sh

3. Setup STM32CubeIDE for compilation and run.

• Import the TinyEngine_vww_tutorial codebase into your STM32CubeIDE by: [File] -> [Import…] -> [General] -> [Existing Projects into Workspace] (Import the entire TinyEngine_vww_tutorial folder).

• After the import, TinyEngine_vww_tutorial should be shown in Project Explorer of your STM32CubeIDE as the example figure below:

• Open TinyEngine_vww_tutorial/Src/main.cpp.

  • If using an Arducam, please set UseCamera macro to 1 (Line 32), as shown in the figure below:
  
  • If not using an Arducam, please set UseCamera macro to 0 (Line 32) and set NoCamera_Person macro to 0 or 1 (Line 33), as shown in the figure below:
  

• Verify you have the correct compilation settings. (The default settings should be correct, but please follow the following steps to make sure that.):

  • Set the include paths of GCC compiler by [Project] -> [Properties] -> [C/C++ Build] -> [Settings] -> [Tool Settings] -> [MCU GCC Compiler] -> [Include paths] as the figure below:
  
  • Set the optimization level of GCC compiler to -Ofast by [Project] -> [Properties] -> [C/C++ Build] -> [Settings] -> [Tool Settings] -> [MCU GCC Compiler] -> [Optimization] as the figure below:
  
  • Set the include paths of G++ compiler by [Project] -> [Properties] -> [C/C++ Build] -> [Settings] -> [Tool Settings] -> [MCU G++ Compiler] -> [Include paths] as the figure below:
  
  • Set the optimization level of G++ compiler to -Ofast by [Project] -> [Properties] -> [C/C++ Build] -> [Settings] -> [Tool Settings] -> [MCU G++ Compiler] -> [Optimization] as the figure below:
  

• Click [Project] -> [Build Project] to build/compile the program and generate the binary executable files.

• Set the run/debug configurations by [Run] -> [Run Configurations…] -> [STM32 Cortex-M C/C++ Application] -> [TinyEngine_vww_tutorial Debug] -> [C/C++ Application] -> [Browse…]:

  • Point to the correct elf file (file path: Debug/TinyEngine_vww_tutorial.elf) to correctly run the program, as shown in the figure below:

4. Setup your STM32F746G-DISCO discovery board to connect the Arducam to the board and also establish the USB connection with the board.

• (Optional) Connect your Arducam to the board with jumper wires according to the following PIN connection:

  • SPI: MOSI->PB15(D11), MISO->PB14(D12), SCK->PI_1(D13), CS(NSS)->PI_0(D5), VCC-> 3.3V, GND->GND
  • I2C: SCL->PB8(D15). SDA->PB9(D14)
  

                  (Top view)                                       (Side view)
  

• Establish the USB connection with the STM32F746G-DISCO discovery board.

5. Now, let’s run the demo.

• Click [Run] -> [Run] to execute the binary executable file on your board.
• If the system requires updating the ST-LINK firmware, please first click “OK”:

• Click “Open in update mode”:

• Click “Upgrade”:

• Click [Run] -> [Run] in the STM32CubeIDE again.

6. If you successfully run the demo, the LCD screen on your STM32F746G-DISCO discovery board should display person detection results (person/no person) and frames per second (FPS), as shown in the example figure below:

• With Arducam:

                 (Person)                                       (No Person)

• Without Arducam:

                 (Person)                                       (No Person)

Limitation

• Only tested the demo on STM32CubeIDE version 1.5.0.