Interview: TM Broadcast International magazine and Tom Peel, Director of LiveLine at SailGP

In the world of sports broadcasting, few challenges match the complexity of capturing and presenting the high-speed action of sail racing. SailGP, the global sailing league, has impulsed the viewer experience with its LiveLine technology. At the helm of this innovation is Tom Peel, Director of LiveLine at SailGP, whose team has developed a cutting-edge remote production system that brings the excitement of F50 catamaran racing to screens worldwide. With just a week to go until the Grand Final in San Francisco, where the ultimate winner of Season 4 will be crowned, the creator of this innovative solution talk to us about how it was conceived, how it was developed, and how it is now adapting to cover different sports.

In this complete exclusive interview, Peel explains all of the details that go into making LiveLine – a system which handles over 1.15 billion data requests per hour for every competing boat at America’s Cup events. It is an example of how the technology overlays graphics with augmented reality over live race footage from Bermuda and gives fans on-screen information about boat speeds, course boundaries and leaderboard positions in real time. Peel also provides insight into the benefits of remote production, how Oracle Cloud in assisting with their graphics packages and SailGP’s plans to scale this technology across sports as a whole – providing a look at what lies ahead for sports broadcasting.

Can you explain what LiveLine is and how it enhances the viewer experience for SailGP races?

LiveLine is “the patented broadcast graphics package that tells the story of SailGP racing.” The system is intended to humanize the very technical sport of sail racing for viewers. This solution, in essence, laces augmented reality graphics on top of live race footage. We overlay virtual boundaries on the racetrack, ladder lines to indicate who is leading which team and show boat speeds… how far back in the race. All of this enables non-sailing viewers to get a quick easy grasp as well on where the race is at.

The system handles more than 1 billion data requests per hour from each F50 catamaran. These data points are converted to visually appealing graphics that enrich the broadcast. LiveLine has the key advantage of being produced remotely, unlike our London studio. This method, centralized to all our international broadcasts will also minimize impact on the environment.

This tool has effectively removed the most challenging elements of sail racing and delivered compelling visual cues that have helped broaden SailGP’s viewership while making this sport easy to understand even from an armchair. It is not just a matter of displaying data -it’s about using that data to tell a compelling story that keeps viewers engaged throughout the race.

What inspired the development of LiveLine, and how long did it take to bring the concept to fruition?

How did the idea of LiveLine come about? Sailing is by its nature a very complex and untelevised sport, even for sailmen it’s complicated to figure it out. The genesis of LiveLine can be traced back to 2011, when thoughts turned towards the America’s Cup in San Francisco. Stan Honey had the idea of pitching Larry Ellison on a new visual tool to improve their sailing broadcast. Their aim was to help audiences follow the sport better as well make it more saleable, with live graphics overlaying visual context and data regarding races.

To accomplish this, our project team was tasked with creating a unique system integrating live racecourse video and high-accuracy GPS data combined with real-time performance statistics. That was accompanied by new technology – the scale of data generated by F50 catamarans requires specialist handling, using Oracle Cloud. The network generates more than 100 million data requests per hour over a one-megabyte link from each boat, converting this input into engaging graphical elements which add to the viewer experience.

It was many years from the first concept until it finally came to life, with ongoing learning and iterations continued through all parts of that journey. Its first use was in the America’s Cup, and now it has been developed further for SailGP as well other sports. The LiveLine set began two years before it was used on-air as development for the system had been in progress, including making the initial prototype smaller and faster, introducing features that would help take information during a sport depending upon where the camera is located.

How does LiveLine integrate real-time data from the F50 catamarans into the broadcast graphics?

LiveLine fuses real-time telemetry from F50 catamarans operating at speeds of more than 30 knots with the latest graphic and visualization technology, using cutting-edge GPS tracking data as well as thousands of data points collected per second via sensors around and onboard each boat (via cloud computing). Here’s how it works:

1. Sensors: 125 sensors placed strategically on each F50 catamaran that gather information about its speed, heading, environmental conditions and sea current. This data is sent via real-time being processed by the system of each boat for 1.15 billion requests per hour.
2. Data Transmission: The obtained data is transmitted from the boats to the shore, and then sent through various private fiber networks uplinking into Oracle Data Center in London. It can process data in real-time with low latency and high bandwidth.
3. Data Processing- The data is processed in Oracle Data Center with Oracle Cloud technology. This consists of processing among needless records and producing further indices for the published to broadcast. Real-Time performance information is further derived using this processed data.
4. Graphics Overlay: Data is linked to the broadcast graphics via LiveLine. During race broadcasts, this includes overlaying augmented reality (AR) graphics onto live footage that shows virtual boundaries, ladder lines and even real-time boat speeds. In large part due to the system’s use of inertial RTK GPS, it can accurately track boats within two centimeters.
5. Remote Production: The production is centralized and managed remotely from SailGP’s broadcast control room in London. This centralized model provides both consistency across broadcasts and eco-sustainability by reducing the number of on-site production teams.

LiveLine uses these technologies to bring the viewer an unprecedented view of sailing, ensuring they have a clear and instant visual context that makes this complex sport much more accessible and engaging for audiences around the world.

What were the biggest technical challenges in developing a system that could process 1.15 billion data requests per hour?

The creation of a system to execute 1.15 billion data requests hourly was not an easy task, as it posed several important technical problems. The major roadblocks that we encountered were:

• Data Volume & Filtering: The amount of data produced by the F50 catamarans is staggering. With 125 sensors in each boat, the data is coming at them around-the-clock. The biggest problem we faced was attempting not exclusively to handle this data but also trying to filter out the noise and focus on only those metrics which were relevant for a broadcast. We had to create algorithms that worked together to effectively parse and filter the data in real-time, only pulling pertinent information forward for graphics.
• Low Latency Transmission: Understanding the importance of low-latency data transmission. The data from the boats had to be sent 7000 kilometers away to the Oracle Data Center in London and rendered on broadcast graphics within a threshold that we could live with. This necessitated not just a resilient, high-speed network but that it be underlain by multiple private fiber networks to keep bandwidth up and latency low.
• Real Time Data Processing: The real-time processing of this data posed another important task. To combine all the information, including deriving two new metrics and transforming raw data into visual elements that make sense for our users, we used Oracle Cloud technology to handle this processing of data. It needed an enterprise-grade cloud computing solutions to process real-time data at scale.
• Accuracy and Precision: High positional accuracy was essential in order to obtain good tracking of the boats, this is critical for producing high-quality broadcast graphics. We were able to track the boats with 2cms accuracy because we utilized high precision RTK GPS systems. It was extremely difficult to incorporate this level of precision into the graphics system, which demanded sophisticated calibration and synchronization techniques.
• Remote Production: Managing the whole production process remotely was an added level of challenge. The nature of the shoot meant we had to centralize all our operations in our London studio so that any data and video feeds from through different global locations could be fed directly into virtual environments where needed during the live broadcast. This remote arrangement not only necessitated dependable delivery of data, but also strong systems to remotely monitor and control the installation.
• Scalability and Reliability: Ensuring the system could scale without issue, particularly when running multiple events at once in different locations was an ongoing headache. We needed to create emergency and triple backup systems so that possible errors could not disrupt the service at a time of live broadcasting.

Overcoming these obstacles allowed us to create a system that processes the huge data volumes and brings it together graphically in an onscreen viewer friendly manner by replacing traditional broadcast graphics with real-time, accurate animated equivalents.

How does the remote production workflow for LiveLine operate, and what advantages does it offer over traditional on-site broadcasting?

LiveLine’s remote production utilizes an entirely centralized workflow where the entire event is run remotely from SailGP’s broadcast studio in London rather than at the race locations. Let me digit through the detailed explanation of how does this workflow work and it is better solution than doing conventional broadcasting on-premise:

Remote Production Workflow

Data Collection & Transmission: Every F50 catamaran is packed with 125 sensors that help gather valuable live data on a wide range of points, such as boat speed and direction, weather conditions and sea currents. The data is sent from the boats to shore and then through diverse private fiber networks to the Oracle Data Center in London. This provides a low latency, high bandwidth source of truth as real-time processing often depends on this.

Data Processing: The data is then processed using Oracle Cloud technology at the Oracle Data Center. This includes sifting out all that is not needed in addition to creating extra metrics appropriate for broadcast. The feed, in turn, is used to construct real-time visualizations of performance data.

Broadcast and Graphics Overlay: LiveLine embeds the processed data into the broadcast graphics. This also involves displaying augmented reality (AR) graphics over live footage of the race, including virtual perimeters, ladder lines and boat speeds.

The entire production – from graphics generation to video feed management, is all orchestrated remotely via judges in the studio at London.

Advantages Over Traditional On-Site Broadcasting

• Stability and Consistency: This helps to keep the system stabilized, centralized and consistent during all events. It saves resources required to set up and knockdown the setup before every event, hence reducing chances of equipment failure resulting in a reliable broadcast output.
• Sustainability: With its ability to dramatically decrease the carbon footprint, remote production is a small win for sustainability. Rather than fly numerous people and equipment to each race venue, a smaller crew on the ground takes care of necessary functions while much of production is done from an offsite location.
• Cost Efficiency: The resulting financial savings are enormous, as it eliminates the usual expenses associated with moving and setting up equipment at each location. It eliminates the logistical complexities and potential delays of transporting equipment also.
• Scalability: Remote setup is very easy with a remote setup allows for scalability. With this, the production team can control multiple feeds and outputs without increasing their on-site presence. It is very useful for controlling a bunch of different international transmissions at once.
• Enhanced Viewer Experience: By utilising a centralized system, the volume of high-precision data that can be incorporated into live broadcasts in real-time is extensive and in turn supplies viewers with comprehensive visual context. Its an added benefit that enhances the experience for viewers and helps to explain such a complex sport in sailing which is often making it difficult for people unfamiliar with this type of Regatta racing entertainment.

Can you describe the role of Oracle Cloud in powering LiveLine and enabling real-time data processing?

It is from Oracle Cloud that LiveLine draws real-time data processing capabilities to achieve our broadcast graphics. As a bridging platform, it aggregates all the data from our F50 catamarans and other sources across the racecourse.

Fifty data sensors mounted on each F50 catamaran capture boat speed, direction and wind/race/sea states for a wealth of information. The boats transmit this data to shore, which is then sent over diverse private fiber networks back to the Oracle Data Center in London. As an example, we can easily process about 1.15B data requests per hour from every boat —this is a lot of information but with the Oracle Cloud infrastructure it does not overload our systems in any way and that was simply impossible to achieve even when using Google Cloud platform for computing alone.

Through Oracle Cloud, we exercise this time to filter the data and process and analyse it in real-time. To bring all the data together as quickly as possible, we turn to Oracle’s managed software solutions and Fast Connect product. This is what not only helps us generate insights but also convert the raw data into visually appealing graphics that you get to see in our broadcasts.

Scalability and performance in Oracle Cloud are important for our remote production workflow, etc. It enables us to intake tons of data from many different parts of the world and centralize those processes, here in London. This method not only keeps all our shows consistent but also makes a massive difference environmentally by cutting down the number of on-the-ground production teams we would otherwise need.

In addition, Oracle Cloud offers the required air-cooling capabilities to power our augmented reality graphics (the digital sideline markers, ladder lines and real-time stats). With the cloud infrastructure we can keep latency to a minimum, which is vital for synchronizing our graphics with live race footage.

Oracle Cloud has been at the very heart of LiveLine, making it possible for us to bring sailing alive with an experience that’s rich in data and easy-to-consume for global audiences.

How does LiveLine achieve its high level of accuracy in tracking boats and overlaying graphics?

The LiveLine technology is possible through a proactively-designed solution that takes advantage of leading edge technologies and very precise data integration. Centered around our RTK GPS system installed on every F50 catamaran (sub-two-centimetre positioning). That level of precision is vital for the high-speed, close-quarters racing that characterizes SailGP circuits.

To these we add data from up to 125 sensors on each boat – measuring parameters such as speed, course or wind and current conditions.
We pass this data in real time to our Oracle Cloud processing systems which process more than 1.15 billion queries per hour and boat, whilst we filter the processed response also in a highly scaled way using cloud services integrating knowledge created from our IoT connect layer on top of OCR/Surface Radar/Drones images/video metadata (type AI Boolean Decision Validation).

Data is then processed and overlaid onto livestreamed video footage from a helicopter-mounted camera. Steady footage is a must for precision graphic overlays, and we achieve this using gyro-stabilized camera gimbals. Our technology uses this GPS data along with video footage, and our software integrates that information to create wayfinding using virtual graphics on top of the racecourse.

Having control of our boats and all the data we produce is one core advantage. This has given us access to what is essentially every piece of information, allowing us to perfect our systems for tracking and graphics. With Siebel turned off, we have developed solutions for both GPS positioning and data processing custom-tailored to the nuance of sail racing.

The aerial-based solution enables boats to be tracked within two centimetres and graphics (like the yellow line on TV) overlay, maintaining their position irrespective of camera movements or changing conditions. This accuracy is what truly differentiates LiveLine and enables us to provide all those who watch live sailing from around the world with an exciting visual experience that illustrates where every sailor sits in each race.

Are Sail GP planning to standardize LiveLine for broader adoption across different sports and broadcasting networks?

Yes, of course. While SailGP is the change agent in this chat today, it was developed with a cross-sport approach and as such would be available to other broadcasters if they so desired (however some/much of LiveLine remains proprietary). The system, developed initially for the America’s Cup and now further adapted to SailGP is credited with providing increased viewer engagement: pool feeds are synchronous match racing — overlaid graphic analytics add clarity as vectors above each competitor provide insight in a broadcast friendly format despite being derived from live telemetry only about three seconds old. The promise we see with this solution could easily make it a game-changing part of the broadcast enhancement landscape in myriad other sports.

The solution offers advantages for other sports, such as:

Improved Viewer Experience: LiveLine overlays a clear, live visual context that makes complex sports more understandable and exciting for audiences.

Commercial The technology has virtual advertising capabilities by enabling dynamic and powerful branding real estate, with no need for physical infrastructure.

Sustainability: The entire ecosystem reduces the need for on-site production teams and infrastructure, which is a key pillar in sustainable broadcasting.

To sum up, we can say that with this system, one tool allows us to serve a great many sports. We want to amplify sports broadcasts around the world by telling more of a story and offering untapped commercial potential, through our experience with this technology.

What steps are being taken to adapt LiveLine for use in other sports beyond sailing?

To expand the capabilities of LiveLine into other sports beyond sailing we are taking several steps:

Design of the redesign: We are considering more modularity in our Sports Logic components, to make it easier for us and even others who want to integrate with our data or into other existing sports systems.

General input normalization: We are normalizing our possibilities of inputs in order to cleanly and without overweight enter the different data systems that we will face across sports Customizable graphics packages We are working on template-based graphic toolkits that you can use for various sports, each with the same base functionality that is at the heart of LiveLine. Though well-executed, we’re continuing to optimize our remote production in order for it to be effectively utilized at the different great events and locations.

AI and machine learning: We are also in early research around how we can incorporate more advanced AI and machine-learning functionality to drive better tracking of users data analytics across its myriad sports.

Scalability improvements: We are looking to make the system more scalable so that we can serve various sports with different data needs, ranging from individual competitions up mass participation races.

Collaborative partnerships – Partnering with other leagues has built a strong relationship on focussing their needs and challenges which have been accounted for in creating LiveLine.

Cross-sport testing: We are also doing proof-of-concept tests with different sports to ensure we can effectively address any sport-by- sport specific challenges in bringing our solution into additional broadcasts.

Mobile integration: We are now working to optimize the new graphics for mobile devices through apps that we’re developing like our forthcoming LiveLineVR phone app which is set up so viewers can “drink in” all of these cutting-edge graphical enhancements across a broad range of sports.

Sustainability is central as we continue to not only spotlight the sport, but also how it can be done responsibly in a Post-2020 world-with our adaptations carefully designed such that SailGP does bring 90-percent of what you see on TV from home (reducing or removing any need for large-scale infrastructure being shipped venue-by-venue).

We hope to provide many more sports with the storytelling scope and commercial benefits that a system such as ADM LiveLine can offer, taking fan engagement levels in sport up another notch across all aspects of delivering quality content, while opening doors to new partnerships.

How does the system handle varying weather conditions and their impact on data collection and transmission?

The weather doesn’t really affect broadcast or transmissions; it’s actually a problem for the shooting team and cameras. We use a range of gyro-stabilized gimbals, such as Cineflex and Shotover, to ensure steady and clear footage regardless of weather conditions. These systems are robust and capable of handling the movements caused by wind and other environmental factors.

Fog and visibility: Fog can be a challenge for camera visibility, but it does not affect our ability to track the boats. Our system can draw graphics through the fog, ensuring that viewers can still see the racecourse and boat positions accurately.

Rain and wind: We have conducted events in various weather conditions, including rain and high winds. Our helicopter-mounted cameras are gyro-stabilized, allowing them to operate effectively in windy conditions. Additionally, the data transmission and processing systems are designed to function reliably in adverse weather.

What kind of feedback have you received from viewers and broadcasting partners regarding LiveLine, and how has this shaped its development?

The response from viewers and broadcasters to LiveLine has been outstanding. Our technology has increased the opportunity to connect and engage a larger audience for sailing which is reflected in viewership growth. We’ve had 14m unique broadcast viewers tuning in on average per event and our audience has grown by more than 300% year-on-year [9].’

In turn, our broadcasting partners even love how it provides a transparent and current visual context of exactly where races are on track.
Of course, the enlightened portion of her fan base would certainly be grateful that LiveLine is shining a light on some of the black arts associated with sail racing.

While this LiveLine has raised some concerns about its use, it allows us to tell stories with a totally different perspective that wasn’t possible before. With this real-time data and graphics overlaid onto the live feed, you’re able to craft a stronger narrative that also has more interesting visual elements for fans watching at home. This has been especially useful in showing viewers the tactics and strategies of teams during races.

LiveLine virtual advertising options were praised by our commercial partners. Dynamic and context-aware brand placement within the broadcast, without physical infrastructure, was also very well received.

Finales – This feedback has significantly contributed to the direction LiveLine took in its development. We have always improved our graphics and how we communicate data to make it more user friendly. We have also continued to grow our capabilities with a wider range of advertising solutions that can be tailored and implemented by clients.

An added bonus is the great response from those who observed the technology at work live or on television; it has inspired us to think about offering [applications for LiveLineFX] beyond sailing. Also, we are currently in discussion with some of the other sports properties on how they can use our technology to increase their fan base especially sports happening out-side stadium where providing context to viewers is quite a task.

The feedback we have received overall has supported our development and will help push the boundaries of what can be achieved with LiveLine, not only for SailGP but hopefully throughout sports broadcasting.

Looking ahead, what do you see as the next big innovation in sports broadcasting technology, and how is SailGP positioning itself to lead in this area?

As we look toward the future, I truly think that one area of sports broadcasting technology innovation to pay attention to will be increased AI and machine learning integration (to help process live data at a quicker pace) in order to enhance more real-time storytelling for fans. With systems such as our LiveLineFX, we are beginning to see the effects of AI and some changes have already been made (for example with athlete tracking) or under way.

This is why SailGP seeks to be the leader in this space and from our side, we will continue work on improving our AI. One of those is advanced predictive analytics that can actually foresee the results of races and then chart important instances for viewers, in real-time. We will then be able to convey even richer stories on our broadcasts.

We’re also investing in expanding AR experiences to new areas. But then again, we’re also creating a LiveLineVR phone app that will go live from our New York event and enable it to be viewed with an entirely new race experience? in other ways. How we see AR enhancing the experience of watching broadcast esports at home, and onsite is just at its beginning.

We are also looking at options of how to adapt our technology better for different sports and make it more scalable. So, by modularizing our Sports Logic components, we are enabling additional applications for LiveLine across a broader base of events and disciplines.

Sustainability is also of utmost importance for us. With each of our innovations we are continuously trying to reduce the environmental impact so let’s do a bit this time too. Remote production is a great example of this; reducing our reliance on-site infrastructure whilst striving to deliver the highest broadcast quality.

At the end of it all, we are looking to rock up [to these venues] and open people’s eyes to how far you can take sports broadcasting. We are not just keeping up with the new innovations in this space but leading how these consumer experiences come to life through a blend of amazing tech at our disposal and unique insights on what makes sports stories so compelling.