University of York and dock10 collaborate on a research project using AI to improve realistic lighting in virtual studio productions. dock10 has embarked on a research initiative in collaboration with the University of York. This joint venture aims to tackle a significant challenge in virtual studio productions: the realistic simulation of light interactions using artificial intelligence (AI).

The project, funded by an embedded R&D grant from XR Network+ and led by XR Stories at the University of York, seeks to establish a strong connection between academia and the broadcast industry. The primary focus is on driving innovation in virtual production technologies, particularly in complex scenarios such as live sports broadcasts, game shows, and music performances that rely on multi-camera green screen studios.

In the realm of multi-camera setups, a persistent challenge is the absence of natural light interactions, necessitating post-production efforts to introduce shadows or reflections for actors placed within computer-generated environments. While LED volume techniques have been successful in achieving these effects in single-camera setups, the complexity of multi-camera environments demands simultaneous multi-angle captures, rendering current solutions less practical.

The collaboration between dock10’s virtual studio experts and the AI research scholars at the University of York aims to pioneer AI-driven tools specifically designed for real-time applications in virtual studio settings. The ultimate objective is to develop AI-powered compositing solutions capable of seamlessly integrating realistic lighting interactions, thereby transforming live or ‘as live’ broadcasts.