{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "## kmodelの作成手順\n", "\n", "kmodelの作成手順を説明したサイトは数少なく、今回は以下のサイトを参考にしました。\n", "- http://blog.shttp://blog.sipeed.com/p/518.html#http://blog.sipeed.com/p/518.html#ipeed.com/p/518.html#\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Docker環境の準備\n", "DockerHubのtakepwave/maixpyenvイメージを使ってmaixpyenvを起動します。\n", "```bash\n", "$ docker run -v `pwd`/workspace:/home/maix/workspace -i --name maixpyenv -t takepwave/maixpyenv\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Docker環境の再開\n", "\n", "上記の処理で、maixpyenvは実行しますが、一度Docker環境から抜けた場合には、以下のようにして再開します。\n", "\n", "```bash\n", "$ docker start maixpyenv\n", "$ docker exec -it maixpyenv /bin/bash\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Maix-TF-workspaceでKmodelの作成\n", "準備が整ったので、Maix-TF-workspaceを使ってKmodelを作成してみましょう。\n", "\n", "GithubからMaix-TF-workspaceをクローンします。Dockerのシェル環境には、(maix) \\$をつけて区別します。\n", "```bash\n", "(maix) $ cd ~/workspace\n", "(maix) $ git clone https://github.com/sipeed/Maix-TF-workspace.git\n", "(maix) $ cd Maix-TF-workspace/mnist\n", "(maix) $ python mnist.py\n", "```\n", "\n", "次にMaix-TF-workspaceのmnist例題を実行します。maixpyenvには、あらかじめTensorflowをインストールしていますので、すぐに例題を実行できます。mnist.pyの例題が完了するまで、私のMacbook Airで5分程度でした。\n", "\n", "```bash\n", "(maix) $ cd Maix-TF-workspace/mnist\n", "(maix) $ python mnist.py\n", "途中省略\n", "step 4800, train_accuracy 0.86\n", "step 4900, train_accuracy 0.94\n", "WARNING:tensorflow:From /usr/local/lib/python3.7/dist-packages/tensorflow/python/training/saver.py:1276: checkpoint_exists (from tensorflow.python.training.checkpoint_management) is deprecated and will be removed in a future version.\n", "ワーニングが続きます\n", "```\n", "\n", "mnist.pyの例題が完了するとmnist.pbファイルが作成されます。\n", "\n", "```bash\n", "(maix) $ ls\n", "MNIST_data logs mnist.pb mnist.py model\n", "```\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### mnist.pbからkmodelへの変換\n", "次にMaix_Toolboxを使って、mnist.pbをkmodelに変換します。\n", "\n", "最初にmnist.pbを~/Maix_Toolbox/workspaceにコピーします。\n", "```bash\n", "(maix) $ cp mnist.pb ~/Maix_Toolbox/workspace/\n", "```\n", "\n", "~/Maix_Toolbox/に移動して、pb2tflite.shを使って.pbファイルから.tflite形式に変換します。\n", "pb2tflite.shを実行すると以下の項目を入力します。\n", "- .pbファイル名: mnist.pb\n", "- 入力層名称: inputs\n", "- 出力層の名称: output\n", "- 入力画像の幅: 28\n", "- 入力画像の高さ: 28\n", "- 入力画像のチャネル数: 1\n", "\n", "```\n", "(maix) $ cd ~/Maix_Toolbox\n", "(maix) $ ./pb2tflite.sh workspace/mnist.pb\n", "This script help you generate cmd to convert *.pb to *.tflite\n", "Please put your pb into workspace dir\n", "1. pb file name: (don't include workspace)\n", "mnist.pb\n", "2. input_arrays name:\n", "inputs\n", "3. output_arrays name:\n", "output\n", "4. input width:\n", "28\n", "5. input height:\n", "28\n", "6. input channel:\n", "1\n", "-----------------------------------\n", "The command you need is:\n", "-----------------------------------\n", "toco --graph_def_file=workspace/mnist.pb --input_format=TENSORFLOW_GRAPHDEF --output_format=TFLITE --output_file=workspace/mnist.tflite --inference_type=FLOAT --input_type=FLOAT --input_arrays=inputs --output_arrays=output --input_shapes=1,28,28,1\n", "以下省略\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### tflite2kmodel.shは、確認用画像が必要\n", "次にtflite2kmodel.shを使ってtflite形式からkmodelに変換します。\n", "\n", "tflite2kmodel.shは、変換を確認するためにテスト用の画像をimagesに入れる必要があります。\n", "このページのgithubのdata/32-dataに10個の画像ファイルがありますので、そちらをimages\n", "ディレクトリにコピーします。\n", "\n", "以下のコマンドを実行して、kmodelを作成します。途中ワーニングがでますが、workspaceにmnist.kmodelが作成されます。\n", "```bash\n", "(maix) $ ./tflite2kmodel.sh workspace/mnist.tflite\n", "uasge: ./tflite2kmodel.sh xxx.tflite\n", "2019-09-07 07:40:18.267350: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA\n", "0: InputLayer -> 1x1x28x28\n", "1: K210Conv2d 1x1x28x28 -> 1x16x14x14\n", "2: K210Conv2d 1x16x14x14 -> 1x32x7x7\n", "3: Dequantize 1x32x7x7 -> 1x32x7x7\n", "4: TensorflowFlatten 1x32x7x7 -> 1x1568\n", "5: FullyConnected 1x1568 -> 1x32\n", "6: Quantize 1x32 -> 1x32\n", "7: K210AddPadding 1x32 -> 1x32x4x4\n", "8: K210Conv2d 1x32x4x4 -> 1x10x4x4\n", "9: K210RemovePadding 1x10x4x4 -> 1x10\n", "10: Dequantize 1x10 -> 1x10\n", "11: Softmax 1x10 -> 1x10\n", "12: OutputLayer 1x10\n", "KPU memory usage: 2097152 B\n", "Main memory usage: 14112 B\n", "(maix) $ ls workspace/\n", "mnist.kmodel mnist.pb mnist.tflite\n", "```\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 動作確認\n", "出来上がったmnist.kmodelを使って、数字の認識を試してみます。\n", "\n", "テストには、M5StickVのmaixpyを使います。\n", "SDカードにmnist.kmodelをmn.kmodelというファイル名でコピーし、M5StickVにSDカードを挿入します。\n", "\n", "以下のスクリプトを実行します。\n", "\n", "```python\n", "import sensor,lcd,image\n", "import KPU as kpu\n", "\n", "lcd.init()\n", "lcd.rotation(2)\n", "sensor.reset()\n", "sensor.set_pixformat(sensor.RGB565)\n", "sensor.set_framesize(sensor.QVGA)\n", "sensor.set_windowing((224, 224)) #set to 224x224 input\n", "sensor.set_hmirror(0) #flip camera\n", "task = kpu.load(\"/sd/mn.kmodel\") \n", "sensor.run(1)\n", "\n", "def detect():\n", " img = sensor.snapshot()\n", " img0=img.resize(112,112)\n", " lcd.display(img0,oft=(0,0)) #display half picture\n", " img1=img.to_grayscale(1) #convert to gray\n", " img2=img1.resize(28,28) #resize to mnist input 28x28\n", " a=img2.invert() #invert picture as mnist need\n", " a=img2.strech_char(1) #preprocessing pictures, eliminate dark corner\n", " lcd.display(img2,oft=(112,32)) #display small 28x28 picture\n", " a=img2.pix_to_ai(); #generate data for ai\n", " fmap=kpu.forward(task,img2) #run neural network model\n", " plist=fmap[:] #get result (10 digit's probability)\n", " pmax=max(plist) #get max probability\n", " max_index=plist.index(pmax) #get the digit\n", " lcd.draw_string(112,0,\"%d: %.3f\"%(max_index,pmax),lcd.WHITE,lcd.BLACK) #show result\n", "\n", "while True:\n", " detect()\n", "```\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 動作確認\n", "M5StickVを起動すると、以下のようにカメラ画像とそれを28x28の白黒反転させた画像、および認識結果を表示する\n", "画面がでます。\n", "\n", "私の手書きした数字を認識させてみました。あまり認識率はよくありませんが、数字は取れているみたいです。\n", "\n", "\n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 付録\n", "### Docker環境イメージのビルド方法\n", "GithubからDocker環境を構築する手順を説明します。\n", "\n", "- イメージ名: test/maixpyenvとします\n", "\n", "```bash\n", "$ git clone https://github.com/take-pwave/maixpy_docker.git\n", "$ cd maixpy_docker\n", "$ docker build -t test/maixpyenv .\n", "```\n", "\n", "無事イメージが作成されたら、以下のコマンドを実行します。\n", "```bash\n", "$ mkdir workspace\n", "$ docker run -v `pwd`/workspace:/home/maix/workspace -i --name maixpyenv -t test/maixpyenv\n", "```\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.3" } }, "nbformat": 4, "nbformat_minor": 2 }