{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Juliaで信号処理の練習\n", "## オーディオデータを取り扱う\n", "\n", "オーディオファイルのIO,再生,編集,録音とか" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "次のパッケージを使用する \n", "[SampledSignals.jl](https://github.com/JuliaAudio/SampledSignals.jl) ファイルから読み込んだ/録音したオーディオデータを保持する構造体が定義されている \n", "[LibSndFile.jl](https://github.com/JuliaAudio/SampledSignals.jl) オーディオファイルIO.wav,mp3,FLAC等いろいろ読める. \n", "[PortAudio.jl](https://github.com/JuliaAudio/PortAudio.jl) マイク入力を録音したり,処理したり. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 1. Julia上で生成した波形データの音を聞く" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "using SampledSignals" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", "

SampleBuf display requires javascript

\n", "

To enable for the whole notebook select \"Trust Notebook\" from the\n", " \"File\" menu. You can also trust this cell by re-running it. You may\n", " also need to re-run `using SampledSignals` if the module is not yet\n", " loaded in the Julia kernel, or `SampledSignals.embed_javascript()`\n", " if the Julia module is loaded but the javascript isn't initialized.

\n", "
\n", "\n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "96001-frame, 1-channel SampleBuf{Float64, 1}\n", "2.000020833333333s sampled at 48000.0Hz\n", "████████████████████████████████████████████████████████████████████████████████" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#サンプリングレート48 kHzで周波数440 Hzのサイン波 2秒分のデータを用意\n", "fs = 48000.0\n", "n = [i for i in 0:96000]\n", "wavedata = sin.(2π*440.0*n/fs)\n", "\n", "#生成したデータからSampleBufインスタンスを生成\n", "buf = SampleBuf(wavedata, fs)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "SampleBuf構造体がオーディオデータを保持する. \n", "SampleBuf.samplerate,SampleBuf.dataとして,サンプルレート,オーディオデータを保持. \n", "オーディオデータの振幅は,[-1.0, 1.0]の範囲(正確には1.0は含まれないが詳細は省略).|1.0|より大きいとクリップ. \n", "**SampleBufインスタンスをCellの出力にすると,Jupyter上ですぐに音を聞くことができる**" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", "

SampleBuf display requires javascript

\n", "

To enable for the whole notebook select \"Trust Notebook\" from the\n", " \"File\" menu. You can also trust this cell by re-running it. You may\n", " also need to re-run `using SampledSignals` if the module is not yet\n", " loaded in the Julia kernel, or `SampledSignals.embed_javascript()`\n", " if the Julia module is loaded but the javascript isn't initialized.

\n", "
\n", "\n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "96001-frame, 2-channel SampleBuf{Float64, 2}\n", "2.000020833333333s sampled at 48000.0Hz\n", "████████████████████████████████████████████████████████████████████████████████\n", "▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆▆" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#ステレオも可能\n", "#サンプルデータ*チャンネルつまりWaveLength*2の行列からSampleBufを作ればOK\n", "leftwave = wavedata\n", "rightwave = 0.2*wavedata\n", "stereodata = [leftwave rightwave]\n", "stereobuf = SampleBuf(stereodata, fs)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 2. オーディオファイルを読み込んで再生する" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": true }, "outputs": [], "source": [ "using LibSndFile" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "typeof(drumloop) = SampledSignals.SampleBuf{FixedPointNumbers.Fixed{Int16,15},2}\n" ] }, { "data": { "text/html": [ "
\n", "

SampleBuf display requires javascript

\n", "

To enable for the whole notebook select \"Trust Notebook\" from the\n", " \"File\" menu. You can also trust this cell by re-running it. You may\n", " also need to re-run `using SampledSignals` if the module is not yet\n", " loaded in the Julia kernel, or `SampledSignals.embed_javascript()`\n", " if the Julia module is loaded but the javascript isn't initialized.

\n", "
\n", "\n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "480000-frame, 2-channel SampleBuf{FixedPointNumbers.Fixed{Int16,15}, 2}\n", "10.0s sampled at 48000.0Hz\n", "▆▃▃▂▆▄▄▂▆▃▆▃▆▄▄▂▆▃▃▂▆▄▄▂▆▃▆▃▆▄▆▃▆▃▃▂▆▄▄▂▆▃▆▃▆▄▄▂▆▃▃▂▆▄▄▂▆▃▆▃▆▄▆▃▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁\n", "▆▃▃▁▆▄▄▂▆▃▆▃▆▄▄▂▆▃▃▂▆▄▃▂▆▃▆▃▆▄▆▃▆▃▃▁▆▄▄▂▆▃▆▃▆▄▄▂▆▃▃▂▆▄▃▂▆▃▆▃▆▄▆▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "drumloop = load(\"./Resource/AudioSample.wav\")\n", "@show typeof(drumloop)\n", "drumloop" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "読み込んだオーディオファイルのデータはSampleBufインスタンスとして返される \n", "従って,Jupyter上ですぐに再生可能" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 3. オーディオデータを編集する" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "先のドラムループの左チャンネルだけをノーマライズしてみる" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/plain": [ "480000×2 Array{Float64,2}:\n", " 0.000244141 0.000335693\n", " 0.000732422 0.000854492\n", " 0.00131226 0.00152588 \n", " 0.00338745 0.00366211 \n", " 0.00619507 0.00646973 \n", " 0.00479126 0.00509644 \n", " -0.00363159 -0.0032959 \n", " -0.0126038 -0.0122375 \n", " -0.0128784 -0.0123901 \n", " -0.00317383 -0.00265503 \n", " 0.00912476 0.00970459 \n", " 0.0169373 0.0175781 \n", " 0.0152283 0.0159912 \n", " ⋮ \n", " 0.0 0.0 \n", " 3.05176e-5 3.05176e-5 \n", " -3.05176e-5 -3.05176e-5 \n", " 3.05176e-5 3.05176e-5 \n", " 0.0 0.0 \n", " 0.0 0.0 \n", " 0.0 0.0 \n", " 0.0 0.0 \n", " 3.05176e-5 3.05176e-5 \n", " 0.0 0.0 \n", " 0.0 0.0 \n", " 0.0 0.0 " ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "stereodrum = convert(Array{Float64}, drumloop.data)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "オーディオフィルから読んだデータは,[FixedPointNumbers.Fixed型](https://github.com/JuliaMath/FixedPointNumbers.jl) \n", "小数点以下の桁数が固定された小数(らしい) \n", "そのまま,Float32/64と混ぜて計算できるけど,一応,先に型変換しておく" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", "

SampleBuf display requires javascript

\n", "

To enable for the whole notebook select \"Trust Notebook\" from the\n", " \"File\" menu. You can also trust this cell by re-running it. You may\n", " also need to re-run `using SampledSignals` if the module is not yet\n", " loaded in the Julia kernel, or `SampledSignals.embed_javascript()`\n", " if the Julia module is loaded but the javascript isn't initialized.

\n", "
\n", "\n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "480000-frame, 2-channel SampleBuf{Float64, 2}\n", "10.0s sampled at 48000.0Hz\n", "▇▅▅▃▇▅▅▃▇▄▇▄▇▆▅▃▇▄▅▃▇▅▅▃▇▅▇▄█▅▇▄▇▅▅▃▇▅▅▃▇▄▇▄▇▆▅▃▇▄▅▃▇▅▅▃▇▅▇▄█▅▇▄▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁\n", "▆▃▃▁▆▄▄▂▆▃▆▃▆▄▄▂▆▃▃▂▆▄▃▂▆▃▆▃▆▄▆▃▆▃▃▁▆▄▄▂▆▃▆▃▆▄▄▂▆▃▃▂▆▄▃▂▆▃▆▃▆▄▆▃▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "leftdata = stereodrum[:, 1]\n", "rightdata = stereodrum[:, 2]\n", "maxval, minval = extrema(leftdata)\n", "normalizeco = maximum([abs(maxval), abs(minval)])\n", "leftdata = leftdata/normalizeco\n", "normalizeddata = [leftdata rightdata]\n", "normalizeddrum = SampleBuf(normalizeddata, drumloop.samplerate)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "(そういえば,組み込みのnormalize関数があったか...?)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 4. マイクから録音する\n", "**これ以降のコードは一部,nbviewer上だと一部動作しないのでローカルで試してみてください**" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false }, "outputs": [ { "ename": "LoadError", "evalue": "InitError: \u001b[91mMethodError: no method matching redirect_stderr(::IJulia.IJuliaStdio{Base.PipeEndpoint})\u001b[0m\nClosest candidates are:\n redirect_stderr() at stream.jl:1034\n redirect_stderr(\u001b[91m::Union{Base.LibuvStream, IOStream}\u001b[39m) at stream.jl:1028\n redirect_stderr(\u001b[91m::Function\u001b[39m, \u001b[91m::Any\u001b[39m) at stream.jl:1082\u001b[39m\nduring initialization of module PortAudio", "output_type": "error", "traceback": [ "InitError: \u001b[91mMethodError: no method matching redirect_stderr(::IJulia.IJuliaStdio{Base.PipeEndpoint})\u001b[0m\nClosest candidates are:\n redirect_stderr() at stream.jl:1034\n redirect_stderr(\u001b[91m::Union{Base.LibuvStream, IOStream}\u001b[39m) at stream.jl:1028\n redirect_stderr(\u001b[91m::Function\u001b[39m, \u001b[91m::Any\u001b[39m) at stream.jl:1082\u001b[39m\nduring initialization of module PortAudio", "", "Stacktrace:", " [1] \u001b[1mmacro expansion\u001b[22m\u001b[22m at \u001b[1mC:\\Julia\\PKG\\v0.6\\PortAudio\\src\\suppressor.jl:15\u001b[22m\u001b[22m [inlined]", " [2] \u001b[1m__init__\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m\u001b[1m)\u001b[22m\u001b[22m at \u001b[1mC:\\Julia\\PKG\\v0.6\\PortAudio\\src\\PortAudio.jl:23\u001b[22m\u001b[22m", " [3] \u001b[1m_include_from_serialized\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::String\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\loading.jl:157\u001b[22m\u001b[22m", " [4] \u001b[1m_require_from_serialized\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Int64, ::Symbol, ::String, ::Bool\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\loading.jl:200\u001b[22m\u001b[22m", " [5] \u001b[1m_require_search_from_serialized\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Int64, ::Symbol, ::String, ::Bool\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\loading.jl:236\u001b[22m\u001b[22m", " [6] \u001b[1m_require\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Symbol\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\loading.jl:441\u001b[22m\u001b[22m", " [7] \u001b[1mrequire\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::Symbol\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\loading.jl:405\u001b[22m\u001b[22m", " [8] \u001b[1minclude_string\u001b[22m\u001b[22m\u001b[1m(\u001b[22m\u001b[22m::String, ::String\u001b[1m)\u001b[22m\u001b[22m at \u001b[1m.\\loading.jl:522\u001b[22m\u001b[22m" ] } ], "source": [ "using PortAudio" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "jupyter上だとusing PortAudio時にエラー吐くけど,今のところ録音時に問題は確認できてないのでこのまま続けます. \n", "外付けAudio IFに使われているASIOドライバ等は認識できないので,Audio IFを使っている人は既定の録音・再生デバイスをデフォルトのサウンドカードにしておいてください." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "stream = PortAudio.PortAudioStream(1, 0)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "1in/0outのストリームを作成.(とりあえず録音だけなので).\n", "マイク入力を拾えるようになる" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "
\n", "

SampleBuf display requires javascript

\n", "

To enable for the whole notebook select \"Trust Notebook\" from the\n", " \"File\" menu. You can also trust this cell by re-running it. You may\n", " also need to re-run `using SampledSignals` if the module is not yet\n", " loaded in the Julia kernel, or `SampledSignals.embed_javascript()`\n", " if the Julia module is loaded but the javascript isn't initialized.

\n", "
\n", "\n", "\n", "\n", "\n", "\n", "\n" ], "text/plain": [ "220500-frame, 1-channel SampleBuf{Float32, 2}\n", "5.0s sampled at 44100.0Hz\n", "▁▅▅▅▅▅▅▅▅▅▆▅▅▅▅▅▅▅▇▄▃▃▃▄▄▅▅▄▇▅▃▃▃▄▅▄▅▅▅▇▆▃▃▃▄▇▄▄▄▄▇▅▃▄▃▄▄▄▄▄▄▅▇▅▃▃▄▅▄▄▄▅▅▅▇▄▃▃▃▄" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "#このCellを実行すると5秒間マイクの音を録音開始\n", "#録音データはSampleBufインスタンスとして返ってくるので,再生プレイヤーが立ち上がる\n", "recbuf = read(stream, 5s)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "ノートPCのマイクで録音したせいかノイズがひどい..." ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "### 5. マイクからひろった音をリアルタイムで処理して,スピーカーに出力" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "stream.blocksize = 4096\n" ] }, { "data": { "text/html": [ "4096" ], "text/plain": [ "4096" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "stream = PortAudio.PortAudioStream(1, 1, synced=true)\n", "@show stream.blocksize" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "1in/1outのストリームを作成.synced=trueで入出力バッファの同期がとれる." ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false }, "outputs": [], "source": [ "#マイク入力にリアルタイムでトレモロをかけてみる\n", "L = stream.blocksize\n", "fs = stream.samplerate\n", "n = 0\n", "for i in 1:100\n", " #blocksize分の入力サンプルを処理して出力するという操作を繰り返す(=blocksize * 100 / fs秒分行う)\n", " inbuff = read(stream, L)\n", " audiodata = convert(Array{Float32}, inbuff.data[:, 1])\n", " \n", " for j in 1:L\n", " audiodata[j] = audiodata[j]*sin(2π*5.0*n/fs)\n", " n += 1\n", " end\n", " \n", " outbuff = SampledSignals.SampleBuf(audiodata, fs)\n", " write(stream, outbuff, L)\n", "end" ] } ], "metadata": { "kernelspec": { "display_name": "Julia 0.6.2", "language": "julia", "name": "julia-0.6" }, "language_info": { "file_extension": ".jl", "mimetype": "application/julia", "name": "julia", "version": "0.6.2" } }, "nbformat": 4, "nbformat_minor": 2 }