Real-time HDR to SDR conversion
By Varun Patel, Product Manager LYNX Technik and Lucien Lenzen with the Hochschule RheinMain University of Applied Sciences in Germany.
To address the increasing consumer demand for exceptional quality content, many media professionals are implementing HDR solutions into their workflows, utilizing image sensors and cameras that inherently support wider density and color ranges. HDR (High Dynamic Range) provides viewers with enhanced contrast and increased brightness and more vibrant color, resulting in a far superior image when compared to current standard HD and 4K UHD signals.
Whether content from HDR cameras needs to be processed for delivery to HDR equipped displays, or whether HDR content needs to be converted to SDR to provide content to the vast installed base of standard SDR displays in a media facility, the demand for HDR is clearly there.
One of the main difficulties involved in progressing from standard dynamic range (SDR) to high dynamic range (HDR) television is the need to provide good downward compatibility with the huge number of SDR television receivers already installed and in current use. The new generation of broadcast cameras can provide a much wider scene contrast range than conventional displays and it is obvious that converters are needed to handle the different formats.
In cooperation with the University of Applied Science in Wiesbaden, Germany, LYNX Technik AG developed a set of HDR to SDR conversion solutions for its greenMachine® platform which down convert high dynamic range (HDR) images to standard dynamic range (SDR) images.
To understand the difference between a classic HDR down converter and the enhanced solutions that LYNX Technik has introduced, we should take a look at how contrast compression is performed.
Many camera operators use a knee function to incorporate luminance information which would normally be outside the SDR range. Traditional HDR down converters work in more or less the same way. However, the capabilities of such systems are very limited compared to the wide range of luminance information. Normally they use a fixed curve that doesn’t take much image content into account. Even if it does though, using a flat gradation to cope with a high contrast range will lead to a flat-looking image at some point. This is why the results of some more sophisticated approaches which use metadata to tweak the curve based on the image content are still not necessarily convincing. Moreover, metadata is hard to handle in a live broadcasting environment.
The problem of dealing with a higher contrast range at acquisition rather than at distribution may be a new one for the broadcast industry but it is an existing one for film-makers and computer graphics artists. Film has always captured more information, so manual grading always has been and still is performed by a colorist. The problems with grading though are that it doesn’t work for live broadcasting and it can be expensive. Meanwhile, computer graphics artists use various procedures based on tone mapping. These also have some drawbacks for live broadcasting: They can be computationally intensive for video applications and because they are basically designed for still images, problems arise from the temporal domain, leading to flicker artefacts.
HDR Evie™, powered by LYNX Technik’s greenMachine® platform was developed to enable live automatic contrast compression avoiding the artefacts inherent in classic tone mapping techniques. Statistical analysis is performed on every image to preserve as much scene contrast range as is necessary but at the same time as little as possible. For example, it is not expedient to set a very dark exposure to prevent some small irrelevant highlights from clipping while the main part of the image is underexposed. Based on models of the human visual system, especially around the perception and preferences relating to contrast, the algorithm calculates the amount of clipping, or rather contrast compression, required.
HDR Evie is an automatic process which analyses the incoming HDR image on a frame by frame basis. Changes in luma will also affect the color impression. That is why the chroma is automatically adjusted to the new luma. Temporal filtering is performed to avoid flicker.
HDR Evie provides more than simply better image quality; it also affects the way in which content is produced. By optimizing the lighting balance and exposure, aperture setting becomes less crucial. As long as the information in the captured scene is not absolutely crushed or burnt out, HDR Evie will try to map it into the final contrast range. As a result, controlling aperture is much less critical on the production side. In a typical televised football game, for instance, some areas of the image may be obscured by roof shadows while others are bathed in overly bright sunlight. The aperture needs to be constantly ridden but it’s likely that the overall image will nevertheless be fairly poor. HDR Evie can enable both a constant f-stop number and a balanced image as long as there is no clipping at the camera output. Sensors that can deal with a higher dynamic range therefore work better. Of course, the f-stop still has to be adjusted if the lighting condition changes substantially; on a very good HDR camera sensor, a single cloud would not affect the result, however an overcast sky would require a new setting. Production flexibility can be increased by reducing the need to light a scene.
HDR does not provide a total solution; sometimes even making things worse. This is why there is also a need for HDR-to-HDR conversion. This means not only changing the HDR signal type. For example: from Perceptual Quantizer (PQ) to Hybrid-Log-Gamma (HLG) or vice versa, but rather balancing the image as in the SDR case. HDR at acquisition doesn’t always work well on the intended HDR display. From evaluations with normal TV viewers, it can be shown that more local contrast and homogenization is also preferred for HDR signals. This is why HDR Evie can also be used to enhance HDR processes such as HLG. Because of the good downward compatibility of the HLG curve, the resulting (processed) HLG works very well on both SDR and HDR displays.
Since HDR Evie is designed to work best with natural images, it can produce unintended changes (e.g. in exposure) when processing synthetic images such as a color bar. Another limitation are the results from the temporal filtering. To avoid pumping, the adaption speed is restricted. On a crash zoom or very fast pan it is not possible to guarantee an ideal exposure all the time – but, on the other hand, a wrong f-number would be worse. Moreover, the image analysis needs one full frame before it can start its processing, leading to a delay of one video frame.
HDR is still a challenging topic. Higher dynamic range at capture does not solve the problem of a non-homogeneous lighting condition within a scene and can even create new problems. HDR Evie is designed to produce better images easily and economically, while optimizing lighting balance and exposure, and also correcting hue and saturation.