Remixing the Real World

Remixing the real world: How to use synthetic images and videos to train more robust deep learning models

Speaker:Dr. Ivan Nikolov, Aalborg University, Denmark

As more and more data is needed for training robust and highly performant computer vision models, researchers are hit with problems around data gathering, analysis, and annotation. These problems can balloon the monetary and time budgets of projects and make the introduction of deep learning models for different tasks harder. This is where synthetic data can become extremely useful, as developers can directly create all the needed scenarios and gather all the required annotations even before the real scenarios have been finalized. This talk aims to show how game engines like Unity can be used to create digital twins or augment synthetic elements in existing datasets. Contrary to other popular ways of generating synthetic data through 3D modelling programs, using game engines helps lower the knowledge barrier for creating data and scenarios, as well as the time it takes to generate them. We will discuss examples of synthetic data solutions, useful tools for digitizing real-world objects for the creation of digital twins, and open-source applications for easily finding additional assets for synthetic environments.

Ivan Nikolov currently works as an assistant professor at Aalborg University, Denmark. His work focuses on combining computer graphics, human-computer interaction research, and VR/AR with deep learning and computer vision methodologies. He is interested in synthetic data generation for training deep learning models for surveillance, animal habitat observation, surface inspection, and data visualization. Combining the possibilities of modern game engines with the need for deep learning for more varied data for training and testing has been the main area of work by Ivan.

Moderator: Dr Nadine Aburumman (Lecturer in Computer Science, Brunel University London)