Quality of Experience in Video Streaming: Status Quo, Pitfalls, and Guidelines

Abstract

Quality of experience (QoE) becomes both the holy grail and a free-for-all in adaptive bitrate (ABR) video streaming. On the one hand, the design, operation, and evaluation of ABR algorithms increasingly rely on QoE. On the other hand, QoE frequently receives only cursory attention in this supporting role, with many of its important aspects treated with insufficient care. As a complex subjective notion, QoE is directly measurable through subjective tests, which incur evident overhead. While an objective QoE model represents a scalable automated means for QoE assessment, QoE models proliferate without consensus on their goodness due to numerous influence factors, construction methods, and usages. The model proliferation creates a false impression that proposing a new QoE model without a proper validation is acceptable. Because the multifaceted QoE problem involves separable and often separated tasks of test conducting, model building, and model using, this separation of concerns causes additional complications. By leveraging two large real datasets of individual QoE perception, this paper reviews the status quo in QoE, identifies various pitfalls, and offers guidelines for test conducting, model building, and model using, so as to foster high standards in future work on QoE in ABR video streaming.