OTT Sports Broadcasting Platform Optimization

Client

A media company running an OTT sports broadcasting platform

Industry

Media & Entertainment, Sports

Services

Data management, DevOps, Maintenance and support

Tech

Azure Front Door, Nginx, Varnish

Challenge

In 2020, a company that operates an over-the-top (OTT) platform for broadcasting sports news and major events contracted ITRex to perform regular maintenance services. Built with Symphony, the platform used an outdated (and no longer maintained!) version of the powerful PHP framework and contained a lot of bugs and unpatched security vulnerabilities. While migrating the OTT platform from AWS to Microsoft Azure, the original developers did not quite adhere to recommended practices for cloud migration and failed to provide proper documentation. As a result, ITRex was required to conduct a thorough review of the code and re-configure the platform's cloud infrastructure. The ITRex team had been low-key maintaining the platform for two years while developing a new version of the OTT solution in parallel. Things got hectic one week before the FIFA World Cup 2022. The customer considered releasing the second version of the platform before the tournament to be able to broadcast football matches to a larger audience while delivering a superior viewing experience. However, since version 2.0 was still in development, we had to make do with the original platform. When the championship began and the streaming service's daily active user base reached five million people, however, ITRex registered several performance issues that needed to be addressed ASAP.

Our task was to:

●

Improve the platform’s availability by Implementing a stronger protection against distributed denial of service (DDoS) attacks

●

Scale the platform In response to the growing workload while fixing bugs and performing ongoing maintenance and support services

●

Resolving some compliance issues, such as Google's new advertising policy and integration requirements imposed by the Dailymotion video-sharing technology platform

Solution

The OTT sports broadcasting platform's legacy code could not cope with the sudden surge in traffic and DDoS attacks. And its database became the platform's bottleneck, consuming all of the available Microsoft Azure resources. Even though the platform’s original developers had implemented advanced database replication techniques from the onset, it was practically impossible to scale the solution — either vertically by adding more computing resources or horizontally by adding more servers. To optimize the OTT platform's performance, we did the following:

●

Erased excessive information from the database and enabled data caching. The platform's database contained a lot of irrelevant data. For instance, in one of the tables, there were a whopping 20 million inaccurate or unnecessary entries, while another included seven million such entries. By removing this information, the ITRex significantly reduced the pressure on the system. And more improvements came with data caching. We started with re-configuring Varnish, a web accelerator for content-heavy dynamic websites, so that the system could use its computing resources effectively when receiving similar requests from multiple users. We also set up a load balancer to distribute requests correctly between server instances. When doing so, the ITRex team calculated the optimum ratio of proxy servers to API instances that handle back-end requests. This helped reduce the number of instances simultaneously retrieving information from the database, further minimizing the workload on the system.

●

Implemented sufficient DDoS attack protection. The older version of the OTT platform lacked a modern content delivery network (CDN) with advanced DDoS attack protection mechanisms. We solved the problem by writing custom configuration scripts for the Nginx server, creating black and white lists for servers and user IP addresses, and introducing time-bound limits on API requests. At the moment of writing this case study, the ITRex team has already swapped Nginx and proxy cache for Microsoft solutions, including Azure Front Door, a modern cloud-based content delivery network. Even though the implementation process is still in full swing, we’ve already achieved sufficient protection against DDoS attacks alongside new OTT platform performance milestones

We're currently working with the Microsoft team to implement new services — for instance, Azure Front Door, a modern cloud-based content delivery network — to achieve new performance milestones. Subsequently, we've changed the configuration of application gateways and integrations with third-party services, such as sports news aggregators.

Impact

The ITRex team applied critical fixes on short notice, bringing down platform errors to a minimum. After the World Cup 2022, no further work is scheduled for this version of the OTT sports broadcasting platform since we plan to roll out version 2.0 in the coming months.