Our application is derived from the TensorFlow Lite Object Detection Android Demo. We add a data logger and support for game controllers to collect datasets with the robot. Currently, we record readings from following sensors: camera, gyroscope, accelerometer, magnetometer, ambient light sensor, and barometer. Using the Android API, we are able to obtain the following sensor readings: RGB images, angular speed, linear acceleration, gravity, magnetic field strength, light intensity, atmospheric pressure, latitude, longitude, altitude, bearing, and speed. In addition to the phone sensors, we also record body sensor readings (wheel odometry, obstacle distance and battery voltage), which are transmitted via the serial link. Lastly, we record control commands received from a connected game controller, if present. We also integrate several neural networks for person following and autonomous navigation.
- Setup Android Studio
- Android device and Android development environment with minimum API 21
- Android Studio 3.2 or later
- Open Android Studio and select Open an existing Android Studio project.
- Select the OpenBot/android directory and click OK.
- Confirm Gradle Sync if neccessary.
- Connect your Android device and make sure USB Debugging in the developer options is enabled. Depending on your development environment further steps might be necessary.
- Click the Run button (the green arrow) or select Run > Run 'android' from the top menu. You may need to rebuild the project using Build > Rebuild Project.
- If it asks you to use Instant Run, click Proceed Without Instant Run.
The TensorFlow Lite Object Detection Android Demo was used as starting point to integrate TFLite models and obtain the camera feed. The main activity is the NetworkActivity which runs the main thread. It inherits from the CameraActivity which manages the camera and UI. The SensorService reads all other phone sensors and logs them. The env folder contains utility classes such as the GameController interface and an AudioPlayer for the audible feedback. The tflite folder contains the model definitions for the Autopilot and Detector networks.
Coming soon...
