Understanding User Experience Metrics

This article provides a definition and overview of Stream Analytics that appear under the ‘User Experience’. 

You can find the user experience metrics on your videos if you have enabled Streams within the platform. Once enabled, navigate to ‘Analytics’ > ‘Stream’.

What are the metrics?

Smoothness score

The smoothness score measures the amount of rebuffering a viewer sees when watching a video. A higher smoothness score means the user sees less (or no) rebuffering, while a lower score means a user sees more rebuffering.

Startup time score

Startup time score estimates how happy or unhappy viewers are with the startup time. Longer startup times mean lower scores, while shorter startup times mean higher scores. Once startup time reaches a certain point (around 8 seconds), the rate of score decay decreases since additional seconds of startup after this point become less impactful.

Video quality score

The video quality score measures the visual quality a user sees by comparing the resolution of a video stream to the resolution of the player in which it is played. If a video stream is significantly upscaled, quality generally suffers, and viewers have an unacceptable experience.

Note that video quality is notoriously difficult to quantify, especially in a reference-free way (without comparing a video to a pristine master). Bitrate doesn't work, since the same bitrate may look excellent on one video and terrible on another.

Several factors contribute to actual video quality: bitrate, codec, content type, and the quality of the original source. However, if the content is encoded well and at the right bitrates, upscaling tracks reasonably well to video quality.

When are metrics collected?

User experience metrics are collected for each video starting with the first view. A video does not appear in the Stream Analytics until it has been viewed using a configured SDK profile (more about info here).