Mobile Broadband Performance Benchmarking

Mobile Broadband Performance Benchmarking

In the group of Mobile Broadband Measurement, we focus on benchmarking mobile networks. Benchmarking is the practice of comparing the performance of a company, institution, tool, or methodology to competitors, similar businesses or the general state of the art in the field, most preferably to what is considered to be the best or most successful in practice, in order to reach the standards achieved by this reference and increase the quality of own operation. Mobile network benchmarking can be undertaken by mobile network operators for deployment, operation and maintenance purposes, as well as regulatory bodies for monitoring the networks of different operators for quality assurance and network neutrality. A Quality of Service (QoS) assessment methodology for mobile communication networks, focused on end user perception of quality must be independent of the access technologies implemented by the mobile networks.

In our research group, we have developed methods to support swift, reliable and distributed measurements of performance in mobile networks for static, nomadic and high speed user scenarios for MNO such as A1 Telekom AG, OEBB-TS, T-Mobile Austria, ZTE among others.

(Resourceful) Active probing

Downlink throughput measurements in cellular mobile networks are of interest not only for researchers but also for mobile network operators and end-users. Conventional smartphone applications measure the throughput by downloading as much data as possible for a predefined duration, consuming tens of megabytes per measurement and blocking Internet connections for several seconds. Fast low-volume probing saves users’ data. It also allows for more frequent measurements, leading to higher spatial resolution along a measurement path, e.g. in case of vehicular measurements. Self-driving cars require reliable connections; swift measurements would enable a constant monitoring without severe service interruptions.

We analyze a possible reduction of data volume per test and develop software solutions for these case that allow comparing the estimated throughput to a conventional, heavy load methodology in practice. Measurements with a user equipment (UE) in live cellular mobile networks have a potential of characterizing real users’ experience but suffer from limited repeatability (small-scale fading, quickly changing cell load, varying interference from neighboring cells). To circumvent this restriction and allow validation of new methodologies, we performed measurements in a controlled LTE cell. We currently are able to achieve throughput estimates comparable to conventional apps in much shorter time (less than 50 ms) requiring much lower data volume (less than 2 MB). This enables us to retrieve high spatial and temporal resolutions in test-drives.

Setup of the measuring system in the train

The measurement setup is part of the CMPT Framework for Reference Measurements (crowdsourcing mobile performance tool). It is an Android-based Application that allows to perform automatized performance measurement tasks. It executes pre-defined experiments and reports results to a centralized database system. This enables continuous monitoring and drive testing.

Benchmarking LTE networks using Crowdsourced Measurements

In recent years, the trend in mobile network benchmarking has shifted towards crowdsourced measurements – measurements performed by anonymous end-users. While being a resourceful alternative to drive tests, crowdsourcing lacks repeatability, testing strategy and setup control, generating tests in random locations at random time points, often reflecting users’ tariffs rather than networks’ performance. At the institute we have generated the infrastructure to conduct, trace and process crowdsourced measurements. In addition to this we have imported all available open-data from Austria, Slovakia, and taken part in the MONROE H2020 project.

So far the state of the art in benchmarking on top of open-data relies on throughput medians among operators based on all available crowdsourced tests. Such analysis cannot consider circumstances under which measurements were performed and is thus biased. To tackle these problems we are and have developed a set of methods method which characterizes networks’ throughput performance as a function of received signal strength. This is achieved by filtering for throughput-degraded measurements and applying a nonlinear model our approach reaches very good quality of fit and outperforms current linear solutions. Methods are validated with real world measurement campaigns as well as lab results in anechoic rooms.

Field experiments for swift performance benchmarking

There are several Key Performance Indicators (KPI) impacting the end user perception of network communication links, one important being the available end-to-end capacity.

At the institute we are designing a swift measurement methodology for indoor benchmarking with the aspects of temporal and spatial effects in mind. Furthermore, to increase measurement speed and make this measurement possible, we have applied a modified version of the packet pattern method introduced in the resourceful probing section.

The new measurement methodology that allows for swift and repeatable measurements of the mean network performance by combining three techniques: Firstly, the instantaneous IP throughput is derived within one second by using a novel packet pattern method and an Android cellphone with an external antenna. Secondly, the long-term trend in the IP performance, that is the slowly changing average network load, is removed. Thirdly, the scenario mean of the remaining small-scale fading scenario with overlaid fast network fluctuations is obtained by statistical inference, namely, spatial systematic-sampling with an XY-positioning table. Currently we approach to extend the method towards drone based measurements with machine learning control.

Benchmarking Hybrid DSL / LTE network Internet Access

Combining different internet access technologies has been a topic of research over many decades.

While mobile broadband access can reach even the most remote locations with high data-rates, wired access allows for high availability and low latency communication. Combining these two trades is a new field of activity in the group. We are working on algorithms to merge DSL with LTE as broad band booster technology.

Open Sky Contributor

We are an active contributor for the open-sky network. As such we have been awarded with the best feeding uptime in 2018!