How does the Stream Orchestrator work?

Stream Orchestrator is Mlytics proprietary AI-powered load balancer that digests real-time CDN quality data and video event data to intelligently orchestrate Multiple CDNs and P2P video delivery service to route the video streams to the best performing CDN and/or existing peers for a given location and at a given time.

In this article, we will describe how the Stream Orchestrator works. Below is the schematic diagram illustrating how it works.

Preliminary step

Let’s begin the process with the stream source configuration. On the stream source, you can specify the number of CDNs and/or enable P2P (peer-to-peer) for routing. It also defines the routing priorities between CDNs (through CDN business scores), and which domains are allowed to deliver the video (through domain allowlist).

Using the above illustration as an example:

  • CDN A score: 80
  • CDN B score: 80
  • P2P delivery: enabled
  • Allowlist: www.youtube.com

Stream Orchestration

When an end-user visit your website and played one of your videos on their browser, this will initiate the launch of the Stream Orchestrator. The CDN score will be the deciding factor on how each video segment will be downloaded and delivered.

Note: Stream delivers video streams using HTTP Live Streaming (HTS) protocol, hence video streams are delivered in segments (.ts files) which are downloaded individually through a playlist (.m3u8 file).

The higher the score of a CDN, the more likely that it will be assigned to handle urgent video segments. Once a CDN finished the delivery of a video segment, this CDN will be assigned to deliver the succeeding video segments. 

Using the above example, CDN A and CDN B have an equal likelihood to be used to deliver the first few video segments.

If the delivery of a video segment using a CDN not performing well or fails, then the next available CDN will deliver this video segment. This process is called the Mid-stream CDN switching.

For the case when the P2P delivery is enabled, if an end-user (peer) already downloaded a video segment, this end-user can deliver the video to another end-user through Web RTC. This will reduce the bandwidth consumption from the CDNs.

The routing ratio between CDNs and P2P is proportional to the location of the video segment relative to the player’s playback position. The best CDN will most likely deliver the video segment closer to the playback position, while the less performing CDN and P2P will most likely deliver the video segments farther to the playback position, with the P2P ratio increasing as the video segment gets farther from the playback position. Check the illustration below.

While obtaining the video segments, the Stream Orchestrator will also gather CDN quality data. Moreover, while the video segment is playing, the Stream Orchestrator (through MUX) will also gather video event data.

CDN quality data includes CDN bandwidth and CDN availability as metrics. While video event data includes several metrics specified by MUX (please check this link to know each metric measured on video event data).

The collected CDN quality data and video event data (through MUX) will be sent to the collector.

These collected data will be used to update the CDN score. Recalculation will be based on how well a CDN performs and how well the video is delivered such that well-performing CDNs will improve their score, while poor-performing CDNs will degrade their score. 

For example:

  • CDN A score: 90
  • CDN B score: 60

In case the user also modified the CDN business scores on the platform, these changes will also be used to update the CDN score.

After the new CDN score has been calculated, this score will be used by the Stream Orchestrator when routing new video segments to the original end-user. Consequently, the new score will affect the new routing priority.

For example, during the initial streaming stage, CDN A performs better than CDN B, this means CDN A has higher CDN quality data, and hence will improve its current score. During the mid-streaming stage, CDN A will now get a higher routing priority than CDN B.

The updating of scores is done at a specific frequency.

Video Delivery

As described above, video content can be delivered through Multi CDN and P2P. Multi CDN obtains the video content through the stream source (origin server) and delivers the content to the end-user using the HTTP protocol. While P2P obtains the video content from prior end-users who are watching the same video and delivers the video content to other end-users using the WebRTC protocol.

Since P2P delivery uses Web RTC, this will improve the privacy protection of the stream source. Moreover, since P2P delivery reduces the use of CDN infrastructures, this will reduce the cost of content delivery as well.