Our paper titled “Mixed-Resolution HESP for More Efficient Fast Channel Switching and Packet-Loss Repair” was accepted at the Picture Coding Symposium (PCS) 2022. This work was made in collaboration with THEO Technologies.
Hannes Mareen presented this research on December 9, 2022, in San Jose, California, USA. The poster can be viewed below (full resolution can be found here).
The poster presented at PCS 2022. Full resolution can be found here.
In low-delay live streaming, it is desired to have fast channel switching and packet-loss repair capabilities. Existing methods can provide both capabilities, but with a negative impact on the stream of steady-state users.
To minimize this impact, techniques such as the High Efficiency Streaming Protocol (HESP) utilize keyframe injection. Such techniques combine compression-efficient normal streams with corresponding companion streams that are used in case of random access or packet loss. Unfortunately, there are three downsides to such strategies. Because a companion stream is needed for every normal stream, first, distribution cost over the Content Distribution Network and second, encoding complexity are considerable costs of the current strategies. Third, injecting a companion keyframe into a normal stream causes a bitrate spike.
Therefore, this paper evaluates the impact of utilizing mixed-resolution HESP in the H.266/VVC standard. By providing a single companion stream for all normal streams of a bitrate ladder, the three mentioned downsides can be mitigated.
Only after a (a) random-access or packet-loss event, (b) a companion stream keyframe is requested (possibly with different resolution) and (c) the keyframe is decoded instead of the predicted frame at an equal timestamp.
We found that injecting a lower-resolution keyframe can effectively reduce the bitrate spike, at the cost of only a modest quality loss. The negative impact on the quality reduces over time with dynamic video content, because the subsequent P-frames replace the low-resolution content with new high-resolution information. An example of this is shown in the image and corresponding video below, which shows the effect of injecting a keyframe of half resolution and QP 32 in a normal stream of QP 22, of the dynamic RaceHorses sequence.
Example of injecting a low-resolution keyframe in dynamic video content. From left to right: normal stream, keyframe-injected stream, visualisation of difference between the two. Full video without additional YouTube compression can be found here.
In static video content, the injected information remains in low resolution as it is not replaced in subsequent P-frames. An example of this is shown in the image and corresponding video below, which shows the effect of injecting a keyframe of half resolution and QP 32 in a normal stream of QP 22, of the ChinaSpeed sequence which contains a static heads-up dispay (HUD).
Example of injecting a low-resolution keyframe in static video content. From left to right: normal stream, keyframe-injected stream, visualisation of difference between the two. Full video without additional YouTube compression can be found here.
We compared the quality loss to traditional packet-loss repair using frame copy. These traditional packet-loss repair solutions can create very perceptible artifacts in dynamic content. An example of this is shown in the image and corresponding video below, which shows the effect of applying frame copy in a normal stream of QP 22, of the dynamic RaceHorses sequence. Thus, mixed-resolution keyframe injection outperforms traditional packet-loss repair.
Example of traditional packet-loss repair using frame copy in dynamic video content. From left to right: normal stream, packet-loss-repaired stream, visualisation of difference between the two. Full video without additional YouTube compression can be found here.
In conclusion, our solution can reduce the bitrate spike and computational/distribution overhead of HESP when enabling fast channel switching or packet-loss repair.
Paper: Mixed-Resolution HESP for More Efficient Fast Channel Switching and Packet-Loss Repair
