{ "cells": [ { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "# Classification on EmuCore\n", "#### Device: EmuCore\n", "\n", "## Introduction\n", "\n", "In this project, we have leveraged QCi’s EmuCore reservoir technology to classify emotional speech files. Below, we provide an overview of the dataset, the approach we adopted, and the results we obtained.\n", "\n", "### Reservoir Computing\n", "\n", "Reservoir computing is an approach that involves passing data through a randomly initialized neural network (the reservoir) and then using the processed data to train a simple linear model downstream. Unlike traditional neural networks, reservoir-based models are easier to set up and less computationally expensive to train. This method requires less domain expertise for different applications, making it highly versatile.\n", "\n", "![Reservoir Schematic](figures\\01.svg)\n", "\n", "### Recurrent Neural Networks (RNN)\n", "\n", "In our setup, we use a Recurrent Neural Network (RNN) architecture as our reservoir. RNNs are a class of neural networks that are particularly suited for sequential data, such as speech, because they have the ability to maintain information about previous inputs.\n", "\n", "For more information on RNNs, you can visit [Recurrent Neural Networks on Wikipedia](https://en.wikipedia.org/wiki/Recurrent_neural_network)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![RNN Model](figures/02.svg)\n", "\n", "Figure 2: A conceptual illustration of an RNN - On the left the model is shown as an input layer x, a hidden layer h, and an output layer y; the input to h is x as well the output of h from previous step. The illustration is unrolled on the right. For example, the input to h(t), hidden later at time t, is the inout (t) at time t as well as the output of hidden layer at time t-1, that is h(t-1).\n", "\n", "Below is a schematic representation of the RNN reservoir setup:\n", "\n", "![RNN Reservoir Schematic](figures/03.svg)\n", "\n", "Figure 3: A conventional reservoir computer consisting of input, reservoir, and output layers. The reservoir layer contains a large number of randomly interconnected recurrent, nonlinear nodes.\n", "This architecture is hard wired to accommodate RNN models." ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "### QCi's EmuCore\n", "\n", "QCi's EmuCore technology employs a time-delayed scheme for reservoir computing. This technique allows the reservoir to capture and process temporal dependencies in the input data effectively." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "![Time Delayed Reservoir Schematic](figures/04.svg)\n", "\n", "Figure 4: A time delay based reservoir computer. Similar to a conventional reservoir computer, the architecture includes input, reservoir, and output layers. However, the reservoir layer contain only one physical node that acts as a large set of temporally spaced virtual nodes.\n", "\n", "\n", "\n", "For more detailed information on this technology, you can refer to [this research paper](https://arxiv.org/pdf/2102.09049.pdf)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Dataset\n", "\n", "In this study, we used the RAVDESS Emotional Speech Dataset. RAVDESS\n", "dataset consists of 1440 audio files, generated by 24 speakers, which correspond\n", "to 8 different emotion. The aim is build a model that can detect which emotion\n", "an audio file corresponds to.\n", "\n", "\n", "https://www.kaggle.com/datasets/uwrfkaggler/ravdess-emotional-speech-audio\n", "\n", "\n", "Copy the files to a directory called \"ravdess_files\" under the working directory." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Method\n", "\n", "Our approach consists of the following steps," ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "from scipy.io import wavfile\n", "import matplotlib.pyplot as plt\n", "import numpy as np\n", "import warnings\n", "import os\n", "import glob\n", "from scipy.io import wavfile\n", "\n", "# Suppress warnings to keep the output clean\n", "warnings.filterwarnings(\"ignore\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "