--- layout: post title: "5G Networked Music Performances - Will It Work?" date: 2022-04-11 10:30:00 +0200 categories: networked-music author: Aleksander Tidemann, Stefano Fasciani image: /assets/image/2022_04_01_aleksati_5g_telenor_logo.png keywords: mct, uio, networked music performances, telematic music, 5g, telenor research, lola, jacktrip, audio latency, video latency excerpt: "In collaboration with Telenor Research, we explored the prospects of doing networked music performances over 5G. Here are the preliminary results." --- 5G is the new paradigm of telecommunication and wireless networking. But what is so great about 5G? For the average user, the biggest difference from 4G to 5G will be a massive increase in speed (bandwidth). In reality, 5G is an entirely new networking infrastructure. This is one reason why there are so many high expectations of 5G. We imagine remote-controlled surgery from afar, distributed sporting events, and even stable networked music performances (NMP). At [MCT](https://www.uio.no/english/studies/programmes/mct-master/), we have been experimenting with NMPs for several years. One of the biggest challenges with playing music over the network is the exceptionally high demand for low-latency communication [2] [3] [4]. In fact, research tells us that the ideal roundtrip latency (back and forth between two locations) for synchronous music performances is around 25-30ms, with the maximum we can tolerate at around 40-50ms [5]. For some perspective: **Demo -** `30ms audio delay from left to right ear:`
In late 2021, we got in touch with Telenor to explore the feasibility of conducting NMPs over 5G. Telenor is currently involved in multiple EU-funded projects that, in part, explore the application of 5G to music technology, including [Fudge5G](https://fudge-5g.eu/en) and [5GMediaHub](https://www.5gmediahub.eu/). During a two-week period in late March 2022, we did a series of experiments on a commercial and private 5G network in collaboration with Telenor Research. In this post, we present these experiments in detail, explaining the technical setup, methods, and preliminary results. # Setup We conducted two experiments in two separate locations on two different 5G networks. ### Commercial 5G Experiment The first experiment was carried out at the Musicology Department at the University of Oslo, on March 30th 2022, on a first-generation commercial 5G network. Most commercial 5G networks today rely on a so-called Non-Stand-Alone (NSA) core network, the same as 4G. However, Telenor pre-configured our routers to access specific Access-Point-Names (APNs). This configuration enabled Peet To Peer (P2P) connectivity between our machines with faster packet routing. ### Private 5G Experiment The second experiment was carried out in Elverum, close to Terningmoen Army Base, on April 4th 2022, on a 5G Network-on-Wheels (5GNoW) solution. The 5GNoW is a private 5G network, or Non-Public Network (NPN), that relies on a Stand-Alone (SA) core network. As we understood, these kinds of systems are mostly used for experimental testing of various 5G applications in the field. ### Hardware and Software For both experiments, we used a pair of [Huawei H138-380 CPE Pro 3 5G Routers](https://nettbutikk.emcom.no/h138-380-cpe-pro-3-5g-ruter-4g-5g-ruter-wifi-6-2-x-lan-111560-p0000028124) to connect to the network. To send audio and video back and forth, we used our own portable and custom-built NMP systems. These racks are essentially bundles of high-end software, audio/video peripherals and networking tools that can provide the lowest possible latency on audio/video transmissions over the network, given that all other the pieces of the puzzle are correct. Full documentation and more detailed info about these systems is available [here](https://github.com/MCT-master/NMP-Portable-Kits/wiki). For NMPs, our go-to AV transmission software is [LoLa](https://lola.conts.it) (Low Latency AV STreaming System). This high-end application was developed at the Trieste Conservatory (Italy) in collaboration with GARR, the Italian Research and Academic Network. To provide ultra-low latency, Lola requires high-end GPU-equipped PCs, soundcard with very stable ASIO drivers (that support buffer sizes of 32 and 64 samples), and specialized [Ximea video cameras](https://www.ximea.com/en/products/cameras-filtered-by-sensor-types/mq013mg-e2). In addition to Lola, we used the JACK2 and [JackTrip](https://www.jacktrip.org) bundle as our secondary software. JackTrip is another popular audio transmission application developed by CCRMA at Stanford University (USA). JackTrip is audio-only and accomodates a wider range of soundcards and buffer sizes. This is "bad" for latency optimization but essential to tolerate more unstable connections and network jitter. # Experiments To explore to what extent 5G can accomodate NMPs, we measured the **stability**, **quality**, and **latency** of roundtrip audio and video signals using Lola and JackTrip. In any real NMP scenario, we only care about technical configurations that render stable AV transfer over time with a minimal dropouts and other unwanted artifacts. Therefore, we only measured signal latency when the best possible tradeoff between stability and quality was found. We did three tests in each experiment: ### 1) Measuring the Network Coverage and Bandwidth By using the [iPerf](https://iperf.fr/) networking utilities, the Huawei routers' own location-optimizing software, Telenor's online [coverage map](https://www.telenor.no/dekning/#dekningskart), and Ookla's [online speedtester](https://www.speedtest.net/), we were able to make network bandwidth and coverage estimates throughout the experiments, ensuring that our load did not exceed the capacity of the network. ### 2) Finding the Sweet-spots To find the best tradeoff between stability and quality, we sent a constant stream of audio and video over the network and looped the signals back to their source, as depicted in Figure 1. With this, we were able to monitor the AV quality of our connections in real-time. To fine-tune the audio, we adjusted software and hardware buffer sizes to locate the lowest possible configuration that ensured a stable audio transmission over a significant period (maybe 10minutes total). For the video, we used a similar a approach, only adjusting the framerate, compression (M-JPEG) amount, and video resolution to find the sweet-spot. ### 3) Measuring the Latency With the software and hardware parameters fine-tuned, we measured the audio and video latency with a similar loopback system: We measured the audio latency in two steps: - **Digital roundtrip time (digital RTT)** With digital RTT, we refer to the measurement of audio latency from software to software (or PC to PC), and back again. With this method, we bypassed the latency induced by our external soundcards and mixers. For the measurements, we used jackTrip in P2P mode. By utilizing the `-x1` argument client-side, we were able to record and monitor the digital RTT in real-time. - **Analog roundtrip time (analog RTT)** With analog RTT, we refer to the measurement of audio latency through the entire chain depicted in Figure 2. To make these measurements, we used another laptop with a designated audio interface. From this secondary laptop/soundcard we sent audio impulses from output 2 to the NMP kits and received the signal back again on input 2. For reference, we closed output 1 to input 1 on the soundcard and sent identical audio impulses to output 1. Then, in software, we measured the analog RTT by looking at the temporal offset between inputs 1 and 2. For video, we took advantage of the fact that our two NMP kits were in the same room. The measure the latency, we sent a Ximea video feed of me doing some claps đź‘Ź from one NMP kit to the other. While displaying the video feeds in full-screen on both computer monitors, we filmed the monitors with a secondary camera. Then, we used the footage from the secondary camera to determine the video latency by counting the offset in frames between the two monitors. # Results ### Commercial 5G Experiment Inside the Musicology building at UiO, the 5G reception was poor. After inspecting Telenor's coverage map of our location, we decided to place the routers outside and pre-configured them to be in Bridge mode, hoping it would generate better coverage, create a more stable connection between our routers, and boost overall performance. According to the routers' location-optimizing software, we achieved a stable 75% 5G coverage at [this location](https://goo.gl/maps/VJXb1KhEBAaPhz4x6). From here, we measured a stable 60Mbps bandwidth. The transmission sweet-spot for audio was achieved using jackTrip with a buffer size of 512. Unfortunately, experimenting with lower buffer sizes only resulted in massive jittery audio and dropouts. We found the optimal stereo audio settings to be the following: