Artificial Intelligence comes standard

The integration of AI-controlled functionalities plus some technical features, make this PTZ a different camera. All this combined will be a significant leap forward in the workflows of this type of equipment.

Text: Luis Pavía

In the short time span between the IBC in Amsterdam and the BITAM Show in Madrid, we published interesting content that described some of the most important features in the PTZ camera segment. My colleague Carlos Medina also mentioned a good number of models from various manufacturers and focused correctly on the reasons why these cameras have become another tool in broadcast workflows.

Continuing with our outreach work, today we are bringing you a detailed review of Sony’s latest and most advanced model: the BRC-AM7. It is not a complement or a successor of any other camera. It is a significant step forward by making the most of all the opportunities provided by well-integrated AI to create an excellent and innovative solution.

It is not a medium-term prototype, it is a tool that already exists and works; it can be tested and there will be production available for delivery to customers beginning early next year.

A couple of years ago Sony made an important leap forward with the FR7 camera and its interchangeable optics in the PTZ environment, thus hitting an important milestone in the segment.

Sony’s surprising leap on this occasion could go almost unnoticed if we look only at the casing but, as we said, internally the camera offers plenty of interesting technical novelties, notably the integration of an AI-based detailed analysis and intense efforts on improvement of framing capabilities.

But let’s not be mistaken. Yes, it is true that for some time now many manufacturers have had equipment capable of tracking a subject autonomously. Even some of them with an autoframing capability like Sony’s own A40. But what we have in our hands today is a few steps ahead. The big difference it brings -a real innovation- is the ability to maintain perfect frames with air ranges that we can configure.

To do this, the camera uses facial analysis with recognition of the individual, head, movements of the entire skeleton, even analyzing details related to the clothing of the relevant subject. This functionality goes far beyond mere tracking.

Not only does it have an autonomous ability to track a subject; it is even capable of recognizing a specific person within a group of people and maintaining the predefined kind of frame (close-up, medium shot, wide shot, etc.) while keeping the correct proportions of air above and on the sides.

Additionally, it has enough capacity and responsiveness to follow, for instance, a tennis player in play from a position similar to that of the chair umpire. This example has been chosen by Sony as shown in one of the presentation videos available on its website. By the way, the result is impressive.

Importantly, all processing is done in-camera, with no need for external drives of any kind, and all features come standard, with no additional licenses required.

Of course, there are more new features, such as a variable ND filter and its enormous benefits, as well as two slots for recording on internal cards. And some others that had to be developed to achieve this impressive result should be mentioned such as the new servomotors. Although, just in view of the ability to frame correctly, we already dare to label it as a real innovation, different from anything we had available until now.

On the other hand, the generic features found in the segment are still there, such as fixed optics, SDI and HDMI outputs in addition to direct streaming, or sensor type. But having a signal in several outputs simultaneously is only available in the high-end models of each range. So let’s go now with the description of physical and technical features.

The sensor is a 1-inch, 14-Mpixel type with Exmor RS CMOS technology, capable of resolutions up to UHD (3840×2160) at 60p which, combined with a BIONZ XR processing engine, offers high image quality with very low noise levels. In fact, the sensor has a higher native resolution and supports up to 5K oversampling with up to 5120×2880 resolution.

The assembly is housed behind the well-known optical variable ND filter, which is identical to the one we already know from other Sony cameras, with a continuous variation range between 1/4 (2 f-stops) and 1/128 (7 f-stops).

The newly developed optics are a 20x optical zoom, with a focal length of 24 to 480 mm (equivalent to 35 mm lenses), phase and contrast detection focusing ability, which it capable of focusing accurately even in low-light conditions. The closest focus is achieved less than 1 cm from the front lens in the angle position, a distance that gradually grows to less than 1 m when we the maximum zoom is reached. These are really short close focus distances.

Brightness is quite good, starting at 2.8, although, as expected in a zoom lens of such a long range and in a compact size, it drops to 4.5 when we take the optics to the highest telephoto position.

The Clear Image function, which allows to digitally extend the maximum zoom range without perceptible degradation in the image and with no need for cropping at the wide end, reaches 30x in UHD resolution (equivalent to 24-720) and up to 40x in HD resolution (equivalent to 24-960).

To facilitate the integration of this camera into workflows involving other cameras or different needs, it is capable of working with different color responses and color curves ranging from an ITU709 to S-Cinetone or S-Log3, as well as different HLGs. This aspect makes it especially interesting in a wide range of options, from filming to live, including practically all types of sports venues, stages and halls.

By combining this capability with the remote operation possibilities, it becomes a great complement in any situations and locations where positioning traditional cameras is very complicated or even outright unfeasible.

The video signal from the camera exits in UHD or HD resolution at different frame rates via 12G-SDI and HDMI outputs. Through the 3G-SDI output, needless to say, we will only have HD signals. But through HDMI we can also have signals of 720 x 480 up to 60p and 720 x 576 up to 50p, thus facilitating the functional integration of the camera in somewhat older installations while the rest of the equipment is gradually replaced by newer models.

As for audio, we have two XLR connectors that are configurable as line/micro/micro +48 V and a 3.5 mm mini-jack, enabling our video signal to carry up to 4 independent channels in its relevant embedded audio. And, to synchronize with other equipment units within our installation, we have connectors for Genlock and TC (time code) signal input.

As for data ports, there is a socket for optical connection with an SFP+ transceiver, although the transceiver will be optional as per the customer’s criteria. Regardless of the above, the Gigabit Ethernet port allows us to have the signal available for streaming by using RTSP, RTMP, VISCA over IP, SRT and NDI/HX protocols. This same port is the one that also enables remote control and management functions.

We insist that the AV signal outputs can work in parallel, even with different resolutions that can be configured through a menu, thus facilitating operability in nearly all kinds of situations and needs.

Through the RM-IP500 remote controller we have all the usual positioning and color functions manually. And through software access to its web interface, we can have full control of the latter parameter from any device, such as tablets or computers. But we will see this in due course, later.

The power supply also takes care of versatility, since the camera can be powered by the classic 4-pin XLR-DC between 11 and 17 volts, or through PoE (Power over Ethernet), which allows us to simplify the installation to a single cable for all functions under certain circumstances.

To finish with signals, one novelty we already pointed out: the ability to record content in two internal slots, suitable indistinctly for CFexpress A-type memory or the most common SDXC card. This recording can be configured in different variants of XAVC, HEVC and AVC formats, with up to 4 audio channels in 24-bit LPCM format at 48 kHz.

Moving on to motion -a key feature in PTZ (Pan-Tilt-Zoom) equipment- the new brushless motors offer surprising performance. On the one hand, because of the very broad interval in available speeds, ranging from 180º in one second to a barely perceptible movement of 0.004º/second. No typo here: four thousandths of a degree per second.

To get an idea of the magnitude of that value, the minimum speed is even slightly lower than the movement of the stars in the sky. For curious minds, the movement of celestial bodies is 15º/hour or 180º in 12 hours…

These speeds are available within the entire range of action for both axes: from 175º to each side in horizontal movement, covering a total of 350º, and from -30º to 210º in vertical movement, for a total of 240º. This means that we will have the entire visible field safely covered regardless of the position in which it is installed, either tripod or ceiling.

But if the range of speeds is impressive, with surprising extremes, no less so are the smoothness and silence of the movement. Noise level is rated as NC25. The smoothness makes movement tremendously fluid and pleasant. And not intrusive at all. We could put it next to an orchestra without disturbing.

The silence was such that -to push the test to the limit- we even put the camera on a wooden table -our ears sticking to the table without seeing the camera- in the silence of the night, and asked someone to move the camera and zoom at any speed. Three different people, one of them with a particularly acute hearing, were unable to tell when the device was moving. Only the faint whizzing of the fan could be heard.

These impressive features do not translate into a bulky or heavy bundle. On the contrary, the size is slightly smaller than 17 x 23 x 19 cm, for a weight of only 3.7 kg. It comes with a pair of threads for direct tripod mounting, in addition to the usual ceiling mount.

The design, consisting of simple lines and matte black or neutral white finishing, means that the unit can be discreetly mounted anywhere without drawing special attention. Its two parallel tally indicators are large at the rounded ends of the arms of the mobile body, thus providing clear visibility at any angle of operation. The tally function also supports the TSL protocol and the four colors provide extended information on the operating mode in a very efficient way. For example, blue indicates that AI analysis functions are enabled and in operation.

Precisely moving on to operation, keep in mind that we have highlighted the novel autoframing capability, but let’s not lose sight of the fact that we have another significant number of automatisms such as an excellent autofocus with facial recognition, which can work independently from autoframing, or in combination, further enhancing the creative possibilities of the equipment.

We also have automatisms for exposure control, which include the variable ND filter as a manageable parameter, thus making it easier for us to maintain a consistent visual narrative, even when there are important changes in lighting levels in our scene.

In terms of gain, in our tests we assessed that up to about 5 dBs are acceptable in broadcast productions without generating perceptible noise.

To start working, once powered, we will only need to know the device’s IP address to gain access to operation from the classic remote controller -in the fashion of the RM-IP500- and we will be able to achieve exactly the same thing we would with any other PTZ camera: manual movements, position memories, movement paths, iris adjustments, zoom, focus, colorimetry, etc., just as it is possible, through the same network connection, to control all image parameters from an RCP controller.

It is highly compatible thanks to its multiple control protocols, including VISCA over IP, CGI, S700PTP, RS422, PTP-IP, and TSL tally. In this way, integrating the unit into the creation of virtual environment is easy since it is capable of exchanging positioning information in real time with the image generation equipment.

However, in order to squeeze out its full potential we would not really need a traditional controller, since it offers us all the functionalities provided by software through access to its web interface. To do this, we can simply use any device with a browser capable of pointing to an IP address within the same network segment, access being possible through virtually all operating systems: Windows, Mac, iOS, and Android.

By the way, speaking of protocols and before reviewing the menu, we should not overlook the Free-D protocol that it also features. This functionality allows the exchange of data and direct interaction with virtual content creation systems such as those from Brainstorm, Unreal Engine or Vizrt, among others. Thanks to this, its integration into productions using either VR (virtual reality) or AR (augmented reality) is simple and very efficient.

This technology is what allows us to credibly and coherently integrate real scenarios, situations and characters with others generated through a computer, be it in film studios, TV sets, concerts halls, sports venues, etc…

Now on with the menu. Once validated with username and password, on this interface we do not only have all the configuration parameters and controls available, but also the image monitor that offers us an exact replica of the viewfinder, together with touch functions to select the focus point or the subject for tracking,  etc., which means that we could be operating our camera even from a remote location. Very remote indeed.

Only on the first connection is it possible that we will be asked for validation or confirmation of a camera parameter, such as the administrator password or date/time, and we will have the opportunity to choose our preferred language from a wide range of available options, which of course include English and all major languages.

The web interface initially divides the screen into five areas with clearly differentiated functions: a narrow selection bar at the top whose central area contains the 4 tabs for the various main functions: live operation, autoframing settings, playback of recorded content and configuration settings. The side and bottom areas of this main panel will change or disappear depending on the active module.

In this same top bar we find the camera’s remote on/off button on the left and, on the right, the alarms and the lock button for the entire panel to prevent unintended actions.

When the first module -live operation- is activated most of the space is taken up by the camera viewfinder, with the presets panel on the left and the indications and access to the main parameters such as gain, shutter, iris, ND, scene and white balance on the bottom. On the right we find the main menu divided into three sections to access every single configuration item.

At the top of this right-hand panel we find the REC and AFR (autoframing) buttons. Right below, capabilities related to camera settings, AF, audio, Wi-Fi, etc. And further below, those related to PTZ motion functions. Each of them has a small top menu bar with a collection of icons to open the various subpanels that are nested, which we are not going to discuss in detail so as not to make this too long.

In the second module -PTZ settings for autoframing- we find the options to fine-tune all the details. These include detection settings and framing settings for the three main positions: close-up, mid-shot, and wide shot. Also included are settings related to response and operation speed, and one especially interesting: starting position.

When we are operating the camera, we can naturally determine which subject we want to track. But even this can be automated by establishing a starting position so that the person who enters and remains in that position for a few seconds will be registered as the main subject and monitored from that moment onwards.

Moving on to the third main module, playback of recorded content, here we not only find the usual playback functions, with media selection and access to clips, but we also have direct access to some viewfinder features. And, what is more important, on the lower right-hand side we find the buttons to access and manage the camera’s internal menu as if we had the unit in our hand.

This huge benefit entails a small drawback: In order to know what we are doing, we must have the  image menu overlay enabled, this being the reason why having this feature on is not recommended during a recording or a live performance.

But all this should not worry us in the least because, in any case, all these camera menu options are accessible from the fourth module of the interface. It gives us access to all the functions of the camera’s internal menu and we can change any parameter without interfering with the signals that may eventually be broadcast.

The internal menu is very similar to the one found in most Sony devices: very extensive and intimidating for some, and a delight -given the enormous control it allows- for others. Remember that we always have the “user” menu option, whereby we simply group the usual and necessary options for the job at hand. Thus, without having to navigate between options, we will have together and easily accessible all the options -and only those- of our choice.

With some settings we must be especially careful, since anyone having the address, username and password could gain access to the camera. To prevent this, it is possible to create users with restricted access, as in any other network device. This allows us to provide access to a limited array of functions or features, and therefore minimize potential risks and errors by less experienced operators.

Also in the network configuration it is possible to set up a series of target FTP servers for our content. This makes changing destination a quick and easy task, without the need to deal with usernames and passwords from one production and the other.

One of the settings that we will only find in this type of camera relates to PTZ functions. These allow us to set not only movement speeds, but also limits on angles so that we never go astray focusing on walls if the camera is, for example, near a corner.

Going into the part of pure operation and user experience, despite the enormous number of elements that we have under our control, all operations are quite simple, even for those who may, for the first time, operate equipment of these characteristics.

It can be a bit surprising to be sitting on an armchair and see how with a simple click on the screen of our tablet the camera will be set to following our protagonist accurately and tirelessly. And, while it is true that the armchair can be hundreds or thousands of miles away, in a live show or in an important shooting we will still feel more at ease on the set.

To control the operation, especially in situations where the subject exits the take or is lost, we have several possibilities: a waiting timer to make it easier for our main subject to re-enter or return to a predefined starting position. In this position, we can select a new main subject or the camera can be configured to auto selection.

The autoframing feature worked very well, tracking the relevant person without problems. When searching for limits, we have been dimming the lighting until the result was compromised. And yes, we succeeded in making the tracking fail, but with such a poor level of lighting that the images would have been unusable anyway.

In short, the progress of AI-aided autoframing is a new milestone that we are convinced will advance audiovisual production, opening up new and interesting areas of action. The combination of visual functionalities such as image quality, variable ND or various color spaces, the ability to integrate into broadcast environments thanks to a familiar operation and integration with VR and AR also the Free-D protocol, as well as the mechanical functionalities provided by the new servomotors coupled with the camera’s responsiveness, turns the camera into a versatile, reliable and very powerful tool for creation of new content.  Content that we are all eager to watch and enjoy.

But before finishing, it is important to make a clarification regarding the possibilities of AI analysis for subject tracking: the results it offers thanks to this feature are outstanding, but this only works with humans. At least for the time being. We are convinced that, just like photo or video cameras that originally only recognized people and are now capable of recognizing even eyes in a multitude of animal species, it is very likely that, in the not too distant future and with the appropriate updates, all these capabilities will be enhanced.

In the meantime, let’s get ready to make the most of everything we already have at our reach today without losing sight of the fact that all this was science fiction not very long ago.