Short video


Drone racing is a recreational sport in which the goal is to pass through a sequence of gates in a minimum amount of time, while avoiding collisions. In autonomous drone racing, one must accomplish this task by flying fully autonomously in an unknown environment by relying only on computer vision methods for detecting the target gates. Due to the challenges such as background objects and varying lighting conditions, traditional object detection algorithms based on colour or geometry tend to fail. Convolutional neural networks offer impressive advances in computer vision, but require an immense amount of data to learn. Collecting this data is a tedious process because the drone has to be flown manually, and the data collected can suffer from sensor failures. In this work, a semi-synthetic dataset generation method is proposed, using a combination of real background images and randomised 3D renders of the gates, to provide a limitless amount of training samples that do not suffer from those drawbacks. Using the detection results, a line-of-sight guidance algorithm is used to cross the gates. In several experimental real-time tests, the proposed framework successfully demonstrates fast and reliable detection and navigation.

Figure 5: Semi-synthetic image generated from annotated background picture. A 3D render of a matched virtual scene is created with OpenGL, where the camera pose matches the one of the real camera used to take the background image. By using random picks from a dataset of base images annotated with their camera pose and by randomly positioning the desired meshes in the scene, an infinite amount of hybrid images can be produced.


Morales, Théo et al. “Image Generation for Efficient Neural Network Training in Autonomous Drone Racing.” 2020 International Joint Conference on Neural Networks (IJCNN) (2020): 1-8.