NHK: The World’s Technological Forefront
Notions such as 4K, VR or AR are just part of NHK (Japanese public corporation) vocabulary: since its inception, TV has always been at the forefront of technology, and a big culprit of this are the NHK Science & Technology Laboratories (STRL), created in 1930 and ‘to be blamed’ for some innovations in broadcast that are enjoyed nowadays by the whole world.
We had the opportunity to interview the director of this worldwide reference, Kohji Mitani, on an exclusive basis. He analyzed the journey of television from a technological point of view and anticipated the industry’s future.
What is the main purpose of the NHK Science & Technology Research Laboratories? Is it mandatory for you to be a technological state-of-the-art TV station?
The NHK Science & Technology Laboratories (STRL) specializes in broadcasting technologies. The institute is the only one of its kind in Japan. STRL was established in 1930, five years after the first radio broadcast went to air in Japan. Its role is “conducting investigative research necessary for the advancement and development of broadcasting and reception thereof”, which is one of the prescribed operations of NHK set forth in Japan’s Broadcasting Act. Radio and television broadcasts have always been a form of culture based on state-of-the-art technologies. As the research institute for Japan’s public broadcaster NHK, the STRL is not only focused on the latest technologies, but on the progress and development of broadcasting in the future. It plays a leading role in the research and development of cutting-edge broadcasting technologies and services. As a research institute belongs to a broadcaster, the STRL is familiar with the needs of audiences and the situation in broadcasting. We believe that it can pursue research and development in a consistent manner from the basics to the practicalities.
What have been the main technological changes that NHK has faced during the last decades?
The results of NHK’s research and development of broadcasting technologies have helped advance broadcasting technologies in Japan and elsewhere around the globe. An example is the technologies for direct satellite broadcasts to the home. NHK began research in this field in 1966, and internationally pioneered the launch of such broadcasts in 1984. NHK also commenced research on high-definition television in 1964. Its HDTV studio format became the international standard in 2000, and is spreading worldwide. These technologies developed by NHK have been acknowledged as milestones by the US-based Institute of Electrical and Electronics Engineers (IEEE). Digitization has been the major technological change that NHK has faced over the past fifteen to twenty years. The year 2000 saw the launch of HDTV-based digital satellite in Japan, followed by digital terrestrial in 2003. Digitization has accelerated the evolution of broadcasting and media technologies. The quickening of technological advances prompted NHK to begin research in ultra high-defintion TV (8K) in 1995. 8K goes above and beyond what HDTV can offer. Ultra high-definition TV (UHDTV) is a major improvement on present TV, not only in terms of the resolution, but in the reproduction of colours and luminosity, which are called WCG (Wide Color Gumut) and HDR (High Dynamic Range). Japan was able to commence satellite-based 4K/8K UHDTV broadcasts in December last year. The 8K broadcasts are a world-first.
NHK’s Science & Technology Research Laboratories have multiple lines of investigation. What is the most interesting for you and why?
NHK’s Science & Technology Research Laboratories are engaged in a wide range of research and development, from device and material technology to service technology. The STRL is engaged in a whole range of fields: technologies for equipment and devices, including cameras, microphones, displays, speakers, high-capacity recorders; video and 3D audio formats for HDTV and UHDTV; transmission technologies for terrestrial, satellite, and cable; programme production technologies harnessing artificial intelligence; Universal service technologies for conveying information universally to all viewers, including people with visual or hearing impairments. (e.g. screen captions, computerized sign language, etc.); service technologies linking broadcasts and the internet; technologies for the future, such as AR/VR and 3D TV. Research and development of the above is necessary and vital for the advancement of broadcasting. I have previously been involved in research on cameras, as well as 8K UHDTV. 8K UHDTV, which we call the ultimate in the broadcasting of two-dimensional images, has already been put into practical use. I am interested in the creation of new broadcasting and media technologies beyond 8K.
We read that you’re not only researching on new technologies, but you are also developing new equipment. Does NHK carry out this research on its own or in collaboration with other technology companies?
NHK sets its own goals for research and development. It also works with outside research institutes and corporations in fields that are close to becoming technically practical.
NHK is a unique corporation. It decides to keep researching and developing its own equipment. Almost every TV in the world decides to trust on external equipment developers. What is behind that decision?
Research and development of broadcasting technologies is in accordance with NHK’s goal of providing audiences with broadcasts of the highest quality. It’s why NHK has researched and developed broadcasting equipment according to its own specifications. State-of-the-art devices and systems are also necessary for providing audiences with programmes and experiences of the kind they have never had before. Manufacturers and corporations in many instances do not develop and invest in cutting-edge technologies that are not likely to be commercially viable. NHK has played a leading role in the research and development of broadcasting technologies. The technologies are showcased at the STRL’s annual Open House, and at the NAB Show, IBC, and so on, where we can get lots of people to comprehend what we are doing, and help trigger new movements and contribute to the development of broadcasting and culture. NHK, of course, also tries to transform the broadcasting technologies which hav been developed into domestic and international standards. Standardizing the technologies that NHK has researched and developed, I believe, enables them to be widely used in broadcasting and visual production. An example is the HDTV standard researched and developed by NHK. It is an international standard, used not only in Japan, but in other nations around the globe. The standardization and widespread use of NHK technologies also helps manufacturers develop equipment. NHK uses and carries out broadcasts with high-performance cameras and other broadcasting equipment produced by manufacturers.
Not only are you doing tests on 8K production, but also in the 8K contribution / IP transmission. Is 8K the future of broadcasting?
Japan commenced HDTV-based digital broadcasts in 2000. And about twenty years later in 2018, it commenced 4K/8K ultra high-definition telecasts. The evolution of broadcasting and media technologies has quickened since the digitization of broadcasts. There have also been considerable advances in transmission technologies, such as 5G and high-speed broadband networks. 8K equipment was very expensive initially. But now products have appeared on the market similar in price to equipment for HDTV and 4K. 8K is the ultimate in the broadcasting of two-dimensional images. I expect that it will become a standard not only in Japan, but elsewhere in other parts of the world. HDTV technologies are used not only in broadcasting, but in a wide range other visual sectors. I believe that 8K technologies will be applied not only in broadcasting, but in a wide range of other sectors, such as medicine, education, and the arts.
In addition, you’re researching technologies such as spatial 3D video system, AR/VR technology or AI. I know these are very complex fields; could you advance some of the possible applications of these technologies in broadcasting?
NHK is evolving into public media. Broadcasts will remain key, but it also wants to harness the internet in suitable ways, using it to provide content and information in a timely manner. NHK is pursuing research in this era of broadcasting and communication collaboration. It enables to incorporate new means of visual expression, such as 3D TV, augmented reality (AR), and virtual reality (VR), with a view to providing audiences with unprecedented experiences and useful information in a more user-friendly manner. NHK’s Science and Technology Research Laboratories are researching technologies for displaying natural-looking 3D images that do not require any special eyewear. We are developing our own technologies for displaying 3D images wherever the viewer’s eyes are moving, be it left or right, up or down. NHK is looking to develop in the future a system for providing individual viewers with 3D images from mobile devices. I believe, for example, that people will get more enjoyment out of a broadcast programme when they can watch 3D images associated with it from a mobile device.
Regarding augmented reality, research is also being done. We highlight a viewing style where TV performers seem to be coming out of the TV screen, and family and friends who are far away seem to be watching TV with you. Through glasses, tablets or smartphones equipped with AR technology, three-dimensional (3D) images of TV performers or family members and friends in different places are combined and displayed in full size. The STRL is also pursuing research to combine its 8K technology with virtual reality to produce high-resolution virtual-reality images. In the future, I believe we will be able to enjoy interactive, high-resolution virtual-reality images from head-mounted displays and other personal viewing devices. And I also believe that crowds of people will be able to watch and share highly engaging, high-resolution virtual reality images at public screenings featuring giant, wrap-around screens.
When it comes to artificial intelligence, NHK’s Science and Technology Research Laboratories have been researching deep-neural networks for about the past forty years. When it comes to harnessing artificial intelligence in broadcasts, research is being done on technologies to help make programme production more efficient and technologies to make broadcasts accessible to all. Among the technologies for making programme production more efficient: Automatic colorization technology for monochrome video; Japanese-English machine translation system for news articles; social media analysis technology. Among the technologies for making broadcasts accessible to all: automatic captioning for live broadcasting; automatic Sign-Language CG system; audio description generation technology.
You have also investigated 5.1 channel sound and 22.2 multi-channel technologies. Are your viewers ready for all these technologies?
NHK airs stereo audio, as well as 5.1 channel sound for its high-definition and 4K broadcasts. Stereo, 5.1 channel sound, and 22.2 multi-channel sound are provided for its 8K broadcast. Audiences can enjoy a broadcast programme in the audio mode compatible with the equipment they have at home. The STRL has been pursuing research and development on 22.2 multi-channel surround sound suitable for the home, to enable people there to enjoy the ultimate in sound quality. For example, the sound system using a binaural reproduction technique could make 22.2 multi-channel surround sound readily available in the home. (see https://www.nhk.or.jp/strl/open2018/tenji/t2_e.html). The HDMI Forum established a standard for a single HDMI cable transmitting 8K images and 22.2 multi-channel sound in May this year. I expect that a variety of sound devices to enable people to enjoy 22.2 multi-channel sound in the home will appear on the market. NHK is also letting people view a number of 8K programmes at various public screenings. These venues let visitors enjoy 8K via a giant screen and a sound system that can handle 22.2 multi-channel sound.
NHK also has developed its own play-out system. Could you tell me a little more about it?
The 8K video format, transmission format, and so on for the 4K and 8K satellite broadcasts that commenced in Japan last year use the technologies standardized from those researched and developed by NHK. NHK researched and developed a transmission method for handling 4K and 8K signals more efficiently, one that can be built into a system with an internet-protocol interface with a high affinity for not only broadcasts, but communications as well. The method has been internationally standardized as MPEG Media Transport (MMT). I believe it is a transmission technology suited to an increasingly diverse range of transmission routes and terminals.
Regarding 4K and 8K videos, they are not only concerned with resolution, but also wide colour gamut technology for more faithfully reproducing the colour of objects. High-dynamic range technology can reproduce images with a wider range of luminosity. The Hybrid Log-Gamma (HLG) format jointly developed and proposed by NHK and the BBC has been in adopted by the ITU-R as an international standard. NHK is currently evaluating the joint production for 4K and 8K broadcasts*, and so on, for use in future broadcasting facilities.
*Having an efficient system and staffing arrangements in place for producing multi-format programmes (2K, 4K, 8K).
In your opinion, where is TV Broadcast heading? What are going to be the trends of TV broadcast during the next years?
In May this year, Japan’s Diet approved amendments to the Broadcasting Act to allow NHK to provide online simulcasts of its TV broadcasts on a constant basis. I believe it will be vital to enable people to access information and content of a necessary nature at any time and any place, in accordance with their lifestyle, via TV sets, radio, smartphones, tablets, and other mobile devices. The STRL is therefore carrying out research to improve upon Hybridcast, which link together broadcasts and broadband communications. Hybridcast Connect is a new feature, a fundamental technology for linking TV sets and smartphones. For example, it easily lets people use their smartphones to select a programme they want to watch, and have it shown on a big-screen TV set. Smartphones can be used to link TV sets with other internet-of-things (IoT) devices. Broadcasts will remain key to NHK. But it is researching and developing technologies for properly harnessing the internet to make broadcasts more convenient for all audiences.