Characterising users experience and critical path in mobile applications

Abstract

Users Quality of Experience (QoE) analysis is paramount for telcos to drive business, and for users to verify their SLAs with the operators. Over the years both academia and industry put a lot of efforts to define methodologies and create tools to monitor QoE. In particular, web traffic QoE analysis is a well understood and ``standardized'' research area, and a varied set of tools and metrics are available to understand how content is delivered. For instance, it is easy to capture the interaction of a browser with a website (e.g., as HAR objects), to inspect content delivery critical path via the download waterfall (e.g., via Chrome, or tools like WProf), or quantify webpage retrieval performance via page load time (PLT), or Google speed index.

These tools and techniques suit well fixed access networks but cannot be directly applied to analyze mobile user QoE. Indeed, most of mobile traffic is generated by large number of applications and do not resemble to traditional web browsing traffic. Also, mobile architectures are more complex than fixed infrastructures and their main performance bottleneck is latency. In this work, we consider QoE for mobile applications by focusing on network level metrics and taking into account application level information. The main goal is to identify network activities (as DNS lookup, TCP and TLS handshake, content download, etc.) which are bottlenecks for application performance and propose countermeasures. To achieve this objective we combine active and passive on-device measurements.

On the one hand, we developed an app that leverages the Android VPN APIs (i.e., it does not require rooted devices) to monitor network traffic, while collecting also info about user activity with the device. This enable us to analyze mobile apps traffic in different scenarios (e.g., application startup, application in background/foreground without user activity, user interaction) and develop techniques to reconstruct the networking waterfall for each of them, similarly to what is currently possible for generic web browsing traffic. However, we envision a generic mobile device system component capable to inspect all traffic, and possibly tapping into user engagement (e.g., interactivity with the screen). By analysing the waterfall we aim to identify activities constituting the critical path for each scenario and application.

On the other hand, the monitoring app allows to schedule active experiments. This allows us to define a comparison baseline to contrast against other analysis, as well as further diagnose problems when the waterfall analysis suggests there may be a bottleneck.

In the poster, we present the methodology and preliminary results on a small set of applications in a testbed with real devices.