OTT, AI and the Big Data Connection

The media delivery business has become a game of seconds. The lines have blurred between broadcast and other IP-related services for delivering media, while content creation has grown from original TV series and movies to how-to videos and social media posts. Access to content for consumers seems limitless, with digital audio and video now the preferred media for nearly all of our daily activities. With so much content being consumed for a wider variety of purposes, viewing time and audience attention spans have grown shorter, making every second count.

The main contributor to the rapid expansion of content creation and consumption has been the emergence of Over the Top (OTT) delivery, made possible by broadband connectivity to a wide range of “connected” devices. This model gives consumers access, convenience and value that wasn’t available via traditional linear services. However, Cable, Satellite, IPTV and Over the Air (OTA) delivery will not completely disappear; each will find their place in this new media delivery ecosystem. For these providers, competing in this fragmented landscape will require a mix of traditional linear services alongside new OTT services, combined with a strong data-driven approach.

Content owners maintain very little control upon turning their product over to CDNs and OTT service providers for delivery. To complicate matters, they lack insight into the viewer’s quality-of-experience (QoE), as more and more third-party services become part of the end-to-end solution. This vacuum of information begs for new methods that ensure a quality experience and proper measurement of viewer engagement.

The aggregation of quality-of-service (QoS), QoE, and viewer behavior data produces extremely large but trusted data sets. By harnessing sophisticated Machine Learning (ML) and Artificial Intelligence (AI) technologies to process this data, media enterprises can glean the valuable insights needed to improve the viewer experience. Significantly, these techniques can be used to predict – in turn allowing operators to prevent – customer-impacting problems before they occur, which is invaluable in minimizing subscriber churn.

The OTT Challenge

OTT has tremendous growth potential for the media and entertainment sector, with growth is projected to exceed 158B worldwide by 2025. OTT delivery can provide consumers with one-to-one, personalized experiences while offering providers the ability to collect immediate feedback. To maximize this opportunity, content creators will need to determine the right content, right duration, right time and right platform to reach their audience in real time.

Regardless of the end goal, though, the first question in any decision tree should be “Is the quality great?” Studies consistently place poor quality in the top four reasons why viewers abandon OTT video. And with short-form content consumption on the rise, even relatively brief problems become very noticeable – for example, imagine a five-second delay in a four-second pre-roll ad.

To complicate matters, OTT is extremely difficult to control end-to-end. OTA broadcasters controlled the entire chain through to their transmitters, while Cable, Satellite, and IPTV distribution offered a single handoff both technically and commercially.

The picture is quite different for OTT. Playout is moving to the cloud via third-party providers, as are streaming service functions including transcoding, packaging and DRM. Meanwhile, multi-CDN and multi-ISP solutions are becoming the norm for reliable delivery and reaching consumers on-the-go. This approach enables incredible scale and speed-to-market, but comes with a cost: loss of control. There could potentially be several hand-offs between separate third-party service providers, thus making a holistic, end-to-end data aggregation, monitoring and analytics system a “must-have” for a successful OTT channel.

The best way to optimize OTT-delivered content is to start with high-quality delivery to a target audience, and respond to feedback in real-time. To achieve this, many are looking toward new technologies – most notably, AI.

Enabling Technologies

AI has been talked about for decades, but adoption and useful results have been a rollercoaster ride. It didn’t really become a practical reality until the cloud, big data, and IoT enabled the capture, storage and processing of vast quantities of data.

Large datasets can hold a lot of potential value, but it is challenging to find patterns, trends, and anomalies within them. Methods and approaches from computer science, mathematics and statistics have been joined together to extract and interpret knowledge. Approaches vary from Data Warehousing and Online Analytical Processing (OLAP) to Data Mining and Machine Learning (ML).

Data Mining is defined as the process of discovering patterns in data, either automatically or semi-automatically. This process is supported by tools and practical techniques also known as Machine Learning, which are used to identify the underlying structure of the data. Data Mining techniques can be used to predict future outcomes based on historical data – for example, identifying customers unhappy with their OTT service and predicting the likelihood of them cancelling their subscription. Machine learning can support this analysis by, for example, using clustering methods to categorize customers based on their consumption habits.

There are numerous AI methods and approaches that can be used in Data Mining applications, depending on the characteristics of the available data and the questions to be answered. It is critical to pick the right set of tools and techniques. With the help of a Data Scientist, the project goals can be decomposed into subsequent tasks that can be solved by certain Machine Learning techniques. Selection of the proper model or approach requires investigation of the data, which must first be cleaned, transformed and properly ingested into the system. The path to an optimal Data Mining solution may involve iteratively exploring, building and tuning many models.

Various off-the-shelf software tools offer graphical and conceptual support for all phases of the knowledge discovery process. This eases the daily work of Data Mining experts and allows a growing number of non-experts to start knowledge discovery projects, but since every use case is unique, you will need to understand how to properly use these components. There are always factors such as exceptions to rules and errors in data that require further analysis of data and fine tuning of the models.

Big Data and AI in Action

An example of the use of AI and ML to turn Big Data into actionable business insights is a project that Qligent deployed with a large-scale provider. Their primary objective was understanding, preventing and reversing subscriber churn, but to do so they needed a better understanding of their end-customers’ experiences and consumption habits.

Working with Qligent, the provider deployed an intelligent analytics system that supplemented data collection and mining with controlled “Last Mile” probes and end-user IoT probes. A Big Data architecture was designed to process the new and legacy data in real-time, and a workflow sequence was created to process the data.

Key Performance Indicators (KPIs) and Key Quality Indictors (KQIs) were developed to create both predictive and prescriptive analytics. The complex analytical computations behind the KQIs were modeled to indicate service availability.

To simplify the understanding and use of the results, the KPIs and KQIs were broken down into three topological domains – the headend, the network and the subscriber – and designed such that any output metric lower than 95% would trigger corrective action.  By leveraging these insights, the provider realized quantifiable improvements in quality and viewer engagement while reducing support calls and churn.

The Qligent analytics system currently generates approximately 20,000 predictive tickets each week across all KQIs in all macro-regions. The number of tickets is expected to continuously drop as the provider’s first-line and second-line support teams use this information to optimize the performance and reliability of their network.

The headend and network KQIs were initially already above their minimum target of 95%, but increased another 1.4% for the headend and 1.7% for the network over the first six months with the help of analytics-driven corrective actions, and continue to grow. Interestingly, this seemingly modest improvement in quality was followed by an increase in concurrent subscriber usage. The provider was subsequently able correlate that the service quality improvements attracted more concurrent viewers and longer average viewing times.

Between 150 and 300 subscriber-related predictive tickets are generated by the system daily per macro-region, each representing an individual or small group of subscribers predicted to be affected by a critical fault in the next three to five days. The second-line support team investigates each predictive ticket, with a goal of preventing the fault from happening.

As a result, the first-line support team saw a 6.6% decrease in the number of incoming customer problem reports. Even more impressive has been an astounding 93.8% decrease in repeat calls from customers about the problems detected and sent for investigation by the analytics system. Similarly, the second-line support team saw an 86.2% decrease in customers calling multiple times about the same problems. This confirms the benefits of quickly determining the root cause of any issues.

The analytics results also enabled the provider to create a prioritized “churn prevention” list for customer service agents to proactively contact. Initially, the weekly-generated list had a large number of subscribers to call, with roughly 35% of them predicted to have a very high probability of leaving the service. After six months, the list was reduced by over 80%. Furthermore, by arming service representatives with analytics about subscribers’ preferences and past technical problems, the agents were able to demonstrate the provider’s commitment to customer service when speaking with the subscribers. Having this personalized knowledge before the call proved far more successful than generic questionnaires or robo-calls.

Conclusions

OTT enables an array of compelling new business models, including personalization at a global scale. It changes everything from the size and type of content, to how content is measured and monetized. This trend also introduced new players into the media and entertainment landscape, many of whom were early pioneers in the use of the cloud, Big Data and AI. Now, the new and traditional players alike are looking toward these technologies to gain a competitive advantage.

Bringing these technologies together can provide media organizations with valuable insights they can use to improve their subscribers’ experience. Most importantly, the use of AI and ML enables providers to predict problems before they actually occur, and thus correct them before they impact their viewers. As seen in our case study example, early project results demonstrated a direct correlation between quality improvement and end-user engagement. More viewers tuned in, watched longer, and were less likely to cancel their service after knowing their provider is staying on top of QoS and QoE issues. The only thing worse than not addressing quality problems quickly enough is not knowing about them at all.