{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Binaural Room Impulse Responses (BRIRs)\n", "\n", "[return to main page](index.ipynb)\n", "\n", "In this unit we will measure - with the help of our dummy head Ulf - binaural room impulse\n", "responses (BRIRs) of our seminar room. \n", "\n", "![Our Dummy Head](images/ulf.jpg)\n", "\n", "We will be using two different methods:\n", "\n", "* First, we excite the room - like in the [previous unit](rir.ipynb) - by clapping two wooden boards together.\n", " But this time, instead of using a single microphone, we will record the room response with the dummy head.\n", " We'll use the free audio recording/editing software [Audacity](http://web.audacityteam.org/) again.\n", "\n", "* Afterwards, we use the slightly more modern *sweep method*.\n", " We excite the room with a sine sweep, which we reproduce by means of a loudspeaker.\n", " The actual impulse response will be calculated from the excitation signal and the signal recorded by the dummy head.\n", "\n", "Further information will be provided during the exercises.\n", "\n", "If you cannot be with us for the measurements, you can still try the following exercises with these files (from older measurements):\n", "\n", "* using the wooden boards: [data/brir_clap.wav](data/brir_clap.wav)\n", "\n", "* using the sweep method: [data/brir_sweep.mat](data/brir_sweep.mat)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Loading the BRIRs\n", "\n", "We already know from the previous units how to load WAV files, so the first one should be easy.\n", "Note, however, that now we are dealing with a two-channel file (one channel for each ear).\n", "The resulting NumPy array will be two-dimensional and it will contain the channels along the columns.\n", "\n", "*Exercise:* Load the WAV file with the BRIRs.\n", "Use the `shape` property of the resulting array to check if the dimensions/sizes are as you expect them.\n", "How long (in seconds) are the impulse responses?" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The impulse responses obtained with the sweep method were created with Matlab®.\n", "Along with some additional information they are stored in MAT files.\n", "Luckily, SciPy [can load these kinds of files](https://docs.scipy.org/doc/scipy/reference/io.html) with the [scipy.io.loadmat()](https://docs.scipy.org/doc/scipy/reference/generated/scipy.io.loadmat.html#scipy.io.loadmat) function (as long as a certain MAT-file-version is used)." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Make sure to use the options `struct_as_record=False` and `squeeze_me=True` when loading the MAT file.\n", "\n", "*Exercise:* Load the MAT file with the other BRIRs.\n", "How long (in seconds) are the impulse responses?" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The object returned by `scipy.io.loadmat()` is a bit strange ...\n", "\n", "It is like a `dict` object which has variable names as keys.\n", "In our case, there is only one variable named `data`, which you can access with\n", "\n", " data = mat_contents['data']\n", "\n", "The `data` variable is a Matlab \"structure\" whose attributes you can access with the well-known dot-notation (but only if you used the argument `struct_as_record=False` as suggested above!).\n", "Use tab-completion (or `dir(data)`) to find out which attributes are available.\n", "\n", "For us, the most interesting attribute is `data.ir`, which holds the actual BRIR data as a two-dimensional NumPy array." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Listening to the BRIRs\n", "\n", "As we saw (or rather *heard*) in the [previous unit](rir.ipynb), listening to the impulse responses directly doesn't tell us very much, but let's do it anyway!\n", "\n", "*Exercise:* Listen to the impulse responses.\n", "Do you hear a difference?\n", "\n", "You should use `tools.normalize()` (from [tools.py](tools.py)) on both IRs before playback to adjust their volume." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "To get a clearer picture of the data, let's convolve the IRs with some signals!\n", "\n", "Note that in contrast to the previous unit, we now have to deal with two-channel impulse responses.\n", "\n", "We might want to use [scipy.signal.fftconvolve()](http://docs.scipy.org/doc/scipy/reference/generated/scipy.signal.fftconvolve.html)\n", "which can handle convolution along a desired axes.\n", "\n", "*Exercise:* Load a mono signal (e.g. from [data/xmas.wav](data/xmas.wav)) and convolve it with both BRIRs.\n", "Do you hear a difference?\n", "\n", "Use `tools.normalize()` on the results to be able to compare them with appropriate levels." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Headphone Compensation\n", "\n", "There should be a clearly audible difference between the two measured BRIRs, right?\n", "\n", "But we can still make it better.\n", "One thing that's still missing is *headphone compensation*.\n", "Load the impulse response stored in the file\n", "[data/THOMSON_HED415N_KEMAR_hcomp.wav](data/THOMSON_HED415N_KEMAR_hcomp.wav) and convolve it with the measured impulse responses to apply the headphone compensation filter.\n", "\n", "*Exercise:* Listen to the BRIRs with and without headphone compensation (after convolving some input signal with them).\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Plotting the BRIRs\n", "\n", "*Exercise:* Plot all impulse responses which were used up to now." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "*Exercise:* Estimate the time-of-flight from $t = 0$ until the direct sound hits the ears.\n", "Which distance in meters does this correspond to?\n", "Is this consistent with the actual measurement setup?" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Exercise:* Roughly estimate the signal-to-noise ratio of the measured impulse responses.\n", "Does it differ for the different measurement methods?" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Exercise:* Plot the frequency response of the different measurements.\n", "Plot the frequency logarithmically on the x-axis and the magnitude in dB on the y-axis." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Multiple Head Orientations\n", "\n", "At the beginning of this unit, we measured impulse responses for different head orientations.\n", "\n", "If you couldn't be with us for the measurements, you can use these example files:\n", "\n", "[data/brir_sweep-80.mat](data/brir_sweep-80.mat) \n", "[data/brir_sweep-40.mat](data/brir_sweep-40.mat) \n", "[data/brir_sweep.mat](data/brir_sweep.mat) (0 degree, same file as we used above) \n", "[data/brir_sweep+40.mat](data/brir_sweep+40.mat) \n", "[data/brir_sweep+80.mat](data/brir_sweep+80.mat)\n", "\n", "*Exercise:* Load all files, extract the BRIRs, convolve them with a mono signal and listen to the results." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Exercise:* Select either the left or the right ear and plot its impulse responses for each\n", "measured orientation.\n", "To do that, create a 2-dimensional array containing one impulse response per column." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Exercise:* Do the same thing with the magnitude spectra.\n", "\n", "Note: Try to add a legend to see which line corresponds to which measurement." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Let's Watch a Video!\n", "\n", "This video is from this page: [how to create animations with matplotlib](http://nbviewer.jupyter.org/github/mgeier/python-audio/blob/master/plotting/matplotlib-animation.ipynb).\n", "\n", "\n", "\n", "*Exercise:* Try to understand what's shown there.\n", "What's the meaning of the angle on the right side?\n", "\n", "Note that the impulse responses in the video were measured in an anechoic\n", "chamber, therefore they look quite different compared to our measured BRIRs." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Plotting Many Head Orientations\n", "\n", "First, have a quick look at this [public dataset](https://zenodo.org/record/55418#.YmK9actBxhE), of most interest might be the [pdf documentation](https://zenodo.org/record/55418/files/wierstorf2011_QU_KEMAR_anechoic.pdf).\n", "\n", "Then, download [QU_KEMAR_anechoic_2m.mat](https://zenodo.org/record/4459911/files/QU_KEMAR_anechoic_2m.mat).\n", "\n", "*Exercise:* Load the file and extract the variable called `irs`." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Exercise:* Plot all data from `irs.left` (or, if you prefer, `irs.right`) using [matplotlib.pyplot.pcolormesh()](https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.pcolormesh.html) or [matplotlib.pyplot.imshow()](https://matplotlib.org/stable/api/_as_gen/matplotlib.pyplot.imshow.html).\n", "\n", "You might see more if you first convert the data to dB.\n", "\n", "What can you recognize on the plot?\n", "Use the zoom feature to enlarge certain areas.\n", "\n", "Note that also in this case the data was measured in an anechoic chamber." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "*Exercise:* Try to find out what the axes show and label them accordingly." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Solutions\n", "\n", "If you had problems solving some of the exercises, don't despair!\n", "Have a look at the [example solutions](brir-solutions.ipynb).\n", "\n", "

\n", " \n", " \"CC0\"\n", " \n", "
\n", " To the extent possible under law,\n", " the person who associated CC0\n", " with this work has waived all copyright and related or neighboring\n", " rights to this work.\n", "

" ] } ], "metadata": { "kernelspec": { "display_name": "mycomac", "language": "python", "name": "mycomac" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.12" } }, "nbformat": 4, "nbformat_minor": 1 }