AUGMENTED REALITY IN TV

A more educational, visual and spectacular television.

By CARLOS MEDINA
Audiovisual Technology Expert and Advisor

 

My first experience in television was in my fourth year of Image and Sound university studies in a public channel and a great program. It was a contest-show that would take place on a large set full of props with great detail and spectacular scenery. You could really see the material side, everything was tangible.

Surely, if augmented reality had existed at that time, it would have been an essential technology in the dynamics of this contest. Nowadays, television sets are almost empty of scenographic elements, only equipped with LED screens and infinite backgrounds where the computer-generated scenery is embedded or directly into the broadcast master video signal.

Augmented Reality (AR) is making TV more TV-like than ever before. The word “television” is a hybrid of the Greek words τῆλε (tēle, “far away”) and the Latin visiōnem (accusative of visiō “vision”). In other words, augmented reality is a great commitment towards the visual element.

Writer L. Frank Baum (Chittenango, New York, United States, 1856 – Los Angeles, California, United States, 1919) commented in one of his novels on the idea of “electronic lenses/screens that superimpose information on living beings”. He called them “Character Markers.”

In 1990, a former Boeing engineer, Thomas P. Caudell, coined the term “Augmented Reality” (AR) with propriety.

This reflection on AR is present in the very origin of the invention of television, when in 1926 the engineer John Logie Baird managed to transmit the first images on the screen of a TV set. With this, we do not want to disparage sound, but it was not until 1930 when the first simultaneous transmission of audio and image in black and white was achieved.

Visual content, from almost the beginning, has been adding techniques, technology and processes that have resulted in an increased value of image content both at an informative level and in the ability to attract the viewer’s gaze: from the use of transparencies, models, filters in front of the camera, to more sophisticated solutions such as working with layers, inserting logos/signs (DSK / USK), including visual post-production up to the ability to embed some images within others (such as chroma Key or Luma Key).

So, what does Augmented Reality (AR) mean in the TV Broadcast environment today? The answer is a concrete one: the integration of visual elements in real time, mixing the real world captured by the cameras with computer-generated information/images (2D/3D), thus creating a perfectly synchronized hyper/ultra-realistic whole.

In this sense, AR is the best example of the good work that professionals in the broadcast environment and those in the IT/video games environment have done both throughout the entire workflow and in the resulting image.

 

Three elements are necessary to make Augmented Reality technology possible:

  • A capture device such as a broadcast camera or even a mobile phone, which is what offers us the real part of this process, also called reality.
  • Hardware and software. This will offer the virtual part (graphic content, backgrounds, computer-generated images…) that is added to the real part. These are the so-called graphics engines.
  • A viewing device. From augmented reality glasses, to adapted contact lenses, without a doubt to the televisions we have in our homes or the screens of smartphones/tablets. With these devices, we are offered the resulting image (real part + virtual part).

Therefore, one of the most important aspects for Augmented Reality to become a success among viewers and an increasingly common and profitable production process among television networks is the issue of graphics engines.

These graphics engines result from the solutions that have been implemented in the field of video games such as Unity or Unreal. For example, Unreal Engine’s success in the TV industry is due to the fact that it enables real-time processing, video input, output, keying, blending, compositing, and rendering. And we insist on the fact that it is an essential condition that the whole process is done live and direct between what happens on the TV set and what the viewer sees.

What is augmented reality (AR) for Unreal Engine? According to their own website: “Simply put, it is a technology that superimposes computer-generated images on the world around us. Let’s think of it as a tool that literally augments or overlays reality with digital graphics.”

The company Brainstorm has been gathering experience in real-time AR for broadcasting since 1995. Its current range blends in perfectly to create amazing Augmented Reality, combining the features of virtual studios and high-end 3D graphics. Its InfinitySet solution includes unique features designed to enhance content creation and delivery and can effortlessly integrate Aston graphics as Augmented Reality objects within the scene, with a photorealistic rendering quality through the use of Unreal Engine.

 

Augmented reality is a mix between virtual and real that must work under several conditions that are inherent to the broadcast world:

  • Work live, generating broadcast master signal with AR.
  • Hyper-realistic results for shapes, textures, volumes and finishes.
  • Tracking technology both in regard to positions between cameras, physical space and virtual content; and relating to people and presenters in movements and travel between the real and the virtual. What are also known as artificial vision techniques, currently feature active sensors based on structured light, SLAM tracking or 3D tracking.
  • Direct interaction between what happens in the real world and what is generated virtually.
  • Realistic lighting, reflections, shadows matching real and virtual.
  • Generation of 360º spaces and images.
  • Facilitating hybrid TV production.
  • Automation and integration into the data flow without time delays or lack of synchronicity.

Alongside traditional television media, new resources and technologies have been added such as virtual scenarios, virtual reality, augmented reality, mixed reality, immersive reality, expanded reality, hybrid reality and/or the implementation of artificial intelligence (AI) in content production.

In the case at hand, Augmented Reality (AR) is in the implementation and experimentation stage in many sectors such as leisure, education, entertainment, tourism and also among TV professionals. But the first results that we are getting in TV programs allow us to announce that this technology is going to turn the television medium into a programming offering featuring more educational, visual and spectacular content.

Distortion Studios o
Disguise unveils nex