{ "cells": [ { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "3S97R28TJJNU" }, "source": [ "

Data and Scripts
for Hydrological Streamline Detection Using a U-net Model with Attention Module

\n", "

\n", "$Zewei$ $Xu^{1,2}$; $Shaowen$ $Wang^{1,2}$; $Lawrence V.$ $Stanislawski^{3}$; $Zhe$ $Jiang^{4}$; $Nattapon$ $Jaroenchai^{1,2}$; $Arpan Man$ $Sainju^{4}$; $Ethan$ $Shavers^{3}$; $E. Lynn$ $Usery^{3}$; $Li$ $Chen^{2,5}$; $Zhiyu$ $Li^{1,2}$; $Bin$ $Su^{1,2}$\n", "

\n", "

\n", "$^{1}$$Department$ $of$ $Geography$ $and$ $Geographic$ $Information$ $Science$, $University$ $of$ $Illinois$ $at$ $Urbana-Champaign$, $Urbana$, $IL$, $USA$
\n", "$^{2}$$CyberGIS$ $Center$ $for$ $Advanced$ $Digital$ $and$ $Spatial$ $Studies$, $University$ $of$ $Illinois$ $at$ $Urbana-Champaign$, $Urbana$, $IL$, $USA$
\n", "$^{3}$$U.S.$ $Geological$ $Survey$, $Center$ $of$ $Excellence$ $for$ $Geospatial$ $Information$ $Science$, $Rolla$, $MO$, $USA$
\n", "$^{4}$$Department$ $of$ $Computer$ $Science$, $University$ $of$ $Alabama$, $Tuscaloosa$, $AL$, $USA$
\n", "$^{5}$$School$ $of$ $Geosciences$ $and$ $Info-Physics$, $Central$ $South$ $University$, $Changsha$, $Hunan$, $China$
\n", "$Corresponding$ $Author:$ $nj7@illinois.edu$\n", "

\n", "\n", "---\n", " \n", "**Notebook Structure:** \n", "[1. Introduction](1_Introduction.ipynb) \n", "2. Codes \n", " [2.1. Data Preprocessing](2.1_Code_Data_Preprocessing.ipynb) \n", " [2.2. Model Training](2.2_Code_Model_Training.ipynb) \n", " [2.3. Interpret the Result](2.3_Code_Interpret_the_Result%20.ipynb) " ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "ZBXLFy5IJJNY" }, "source": [ "---\n", "\n", "### Model Training\n", "\n", "The following code is to define the sctructure of the CNN, U-Net, that we will use. U-Net is a famous CNN structure for feature detecting using the combination of local pattern and global pattern. So, The U-Net is the best CNN for streamline extraction. \n", "\n", "The following code id the definition of the CNN (U-Net).\n", "\n", "We use the follow softwares:\n", "\n", "- Python 2.7\n", "- sklearn 0.18.1\n", "- scikit-image 0.16.2\n", "- gdal 3.0.2\n", "- numpy 1.17.3 \n", "- Anaconda 2.0\n", "- keras 2.0 with backend of tensorflow 1.0\n", "\n", "\n", "---\n", "\n", "\n", "## **U-net Model**\n", "\n", "\n", "
\n", "\n", "
\n", "\n", "*The attention U-net model*\n" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "AYJktLHezoxC" }, "source": [ "\n", "\n", "---\n", "\n", "\n", "## **Attention modules** \n", "\n", "\n", "We use attention gate 1 at the three shallower layers to preserve the most information and use attention gate 2, which is the revised version of attention gate 1 specifically to improve the performance of the computation of sparse metrix.\n", "\n", "
\n", "\"the\n", " \n", "

Attention gates of the U-net model (a: attention gate 1;b: attention gate 2)

\n", "\n", "
" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Requirement already satisfied: tensorflow in /opt/conda/lib/python3.7/site-packages (1.13.1)\n", "Requirement already satisfied: tensorboard<1.14.0,>=1.13.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow) (1.13.1)\n", "Requirement already satisfied: keras-applications>=1.0.6 in /opt/conda/lib/python3.7/site-packages (from tensorflow) (1.0.8)\n", "Requirement already satisfied: termcolor>=1.1.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow) (1.1.0)\n", "Requirement already satisfied: gast>=0.2.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow) (0.3.3)\n", "Requirement already satisfied: keras-preprocessing>=1.0.5 in /opt/conda/lib/python3.7/site-packages (from tensorflow) (1.1.2)\n", "Requirement already satisfied: numpy>=1.13.3 in /home/jovyan/.local/lib/python3.7/site-packages (from tensorflow) (1.20.1)\n", "Requirement already satisfied: astor>=0.6.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow) (0.8.1)\n", "Requirement already satisfied: six>=1.10.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow) (1.14.0)\n", "Requirement already satisfied: wheel>=0.26 in /opt/conda/lib/python3.7/site-packages (from tensorflow) (0.34.2)\n", "Requirement already satisfied: grpcio>=1.8.6 in /opt/conda/lib/python3.7/site-packages (from tensorflow) (1.30.0)\n", "Requirement already satisfied: protobuf>=3.6.1 in /opt/conda/lib/python3.7/site-packages (from tensorflow) (3.11.4)\n", "Requirement already satisfied: tensorflow-estimator<1.14.0rc0,>=1.13.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow) (1.13.0)\n", "Requirement already satisfied: absl-py>=0.1.6 in /opt/conda/lib/python3.7/site-packages (from tensorflow) (0.9.0)\n", "Requirement already satisfied: werkzeug>=0.11.15 in /opt/conda/lib/python3.7/site-packages (from tensorboard<1.14.0,>=1.13.0->tensorflow) (1.0.1)\n", "Requirement already satisfied: markdown>=2.6.8 in /opt/conda/lib/python3.7/site-packages (from tensorboard<1.14.0,>=1.13.0->tensorflow) (3.2.1)\n", "Requirement already satisfied: h5py in /opt/conda/lib/python3.7/site-packages (from keras-applications>=1.0.6->tensorflow) (2.10.0)\n", "Requirement already satisfied: setuptools in /opt/conda/lib/python3.7/site-packages (from protobuf>=3.6.1->tensorflow) (45.2.0.post20200209)\n", "Requirement already satisfied: mock>=2.0.0 in /opt/conda/lib/python3.7/site-packages (from tensorflow-estimator<1.14.0rc0,>=1.13.0->tensorflow) (3.0.5)\n", "Requirement already satisfied: keras in /opt/conda/lib/python3.7/site-packages (2.2.4)\n", "Requirement already satisfied: six>=1.9.0 in /opt/conda/lib/python3.7/site-packages (from keras) (1.14.0)\n", "Requirement already satisfied: numpy>=1.9.1 in /home/jovyan/.local/lib/python3.7/site-packages (from keras) (1.20.1)\n", "Requirement already satisfied: h5py in /opt/conda/lib/python3.7/site-packages (from keras) (2.10.0)\n", "Requirement already satisfied: pyyaml in /opt/conda/lib/python3.7/site-packages (from keras) (5.3)\n", "Requirement already satisfied: keras-preprocessing>=1.0.5 in /opt/conda/lib/python3.7/site-packages (from keras) (1.1.2)\n", "Requirement already satisfied: keras-applications>=1.0.6 in /opt/conda/lib/python3.7/site-packages (from keras) (1.0.8)\n", "Requirement already satisfied: scipy>=0.14 in /opt/conda/lib/python3.7/site-packages (from keras) (1.4.1)\n" ] } ], "source": [ "!pip install --user tensorflow\n", "!pip install --user keras" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "p8flybRxJJNb" }, "source": [ "### Import all dependencies " ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 81 }, "colab_type": "code", "executionInfo": { "elapsed": 2408, "status": "ok", "timestamp": 1583991740482, "user": { "displayName": "nattapon jaroenchai", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj9i_TMlYzv27Vkas3v5qWkW7ZdTBQme3RVKqYp7c4=s64", "userId": "17092454241854925654" }, "user_tz": 300 }, "id": "MRd35Jd3JJNd", "outputId": "ad9c7f5b-df61-44d0-ffbe-4c781320ce93" }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Using TensorFlow backend.\n", "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:526: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", " _np_qint8 = np.dtype([(\"qint8\", np.int8, 1)])\n", "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:527: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", " _np_quint8 = np.dtype([(\"quint8\", np.uint8, 1)])\n", "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:528: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", " _np_qint16 = np.dtype([(\"qint16\", np.int16, 1)])\n", "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:529: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", " _np_quint16 = np.dtype([(\"quint16\", np.uint16, 1)])\n", "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:530: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", " _np_qint32 = np.dtype([(\"qint32\", np.int32, 1)])\n", "/opt/conda/lib/python3.7/site-packages/tensorflow/python/framework/dtypes.py:535: FutureWarning: Passing (type, 1) or '1type' as a synonym of type is deprecated; in a future version of numpy, it will be understood as (type, (1,)) / '(1,)type'.\n", " np_resource = np.dtype([(\"resource\", np.ubyte, 1)])\n" ] } ], "source": [ "# Load all the dependencies\n", "import os\n", "import sys\n", "import random\n", "import warnings\n", "import numpy as np\n", "from numpy import genfromtxt\n", "from keras.optimizers import Adam, SGD\n", "from keras.callbacks import ModelCheckpoint, EarlyStopping, ReduceLROnPlateau\n", "from tensorflow import set_random_seed\n", "from itertools import chain\n", "from keras.layers import Layer,UpSampling2D,merge\n", "from keras.layers.normalization import BatchNormalization\n", "from keras.layers.core import SpatialDropout2D, Activation\n", "from keras.models import Model\n", "from keras.layers import Input\n", "from keras.layers.core import Dropout, Lambda\n", "from keras.layers import Reshape, Permute, Input, add, multiply\n", "from keras.layers.convolutional import Conv2D, Conv2DTranspose\n", "from keras.layers.pooling import MaxPooling2D\n", "from keras.layers.merge import concatenate\n", "from keras.callbacks import EarlyStopping, ModelCheckpoint\n", "from keras import backend as K\n", "import tensorflow as tf\n", "from keras.layers import dot\n", "\n", "# To specify the GPU ID uncomment this block and \n", "# with K.tf.device('/gpu:0'): # specify the ID of GPU here (0: the first GPU)\n", "# config = tf.ConfigProto(intra_op_parallelism_threads=4,\\\n", "# inter_op_parallelism_threads=4, allow_soft_placement=True,\\\n", "# device_count = {'CPU' : 1, 'GPU' : 1})\n", "# session = tf.Session(config=config)\n", "# K.set_session(session)\n", "\n", "set_random_seed(1337)\n", "np.random.seed(1337) # for reproducibility\n", "warnings.filterwarnings('ignore', category=UserWarning, module='skimage')" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "l5s8PK99JJNk" }, "source": [ "# Define all component for U-Net model" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "AYJktLHezoxC" }, "source": [ "\n", "\n", "---\n", "\n", "### Define Coefficient Calculations and Residual Block\n", "\n" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "colab": {}, "colab_type": "code", "id": "2tqIUMBIJJNl" }, "outputs": [], "source": [ "# Use dice coefficient function as the loss function \n", "def dice_coef(y_true, y_pred):\n", " y_true_f = K.flatten(y_true)\n", " y_pred_f = K.flatten(y_pred)\n", " intersection = K.sum(y_true_f * y_pred_f)\n", " return (2.0 * intersection + 1.0) / (K.sum(y_true_f) + K.sum(y_pred_f) + 1.0)\n", "\n", "# Jacard coefficient\n", "def jacard_coef(y_true, y_pred):\n", " y_true_f = K.flatten(y_true)\n", " y_pred_f = K.flatten(y_pred)\n", " intersection = K.sum(y_true_f * y_pred_f)\n", " return (intersection + 1.0) / (K.sum(y_true_f) + K.sum(y_pred_f) - intersection + 1.0)\n", "\n", "# calculate loss value\n", "def jacard_coef_loss(y_true, y_pred):\n", " return -jacard_coef(y_true, y_pred)\n", "\n", "# calculate loss value\n", "def dice_coef_loss(y_true, y_pred):\n", " return -dice_coef(y_true, y_pred)\n", "\n", "def Residual_CNN_block(x, size, dropout=0.0, batch_norm=True):\n", " if K.image_dim_ordering() == 'th':\n", " axis = 1\n", " else:\n", " axis = 3\n", " conv = Conv2D(size, (3, 3), padding='same')(x)\n", " if batch_norm is True:\n", " conv = BatchNormalization(axis=axis)(conv)\n", " conv = Activation('relu')(conv)\n", " conv = Conv2D(size, (3, 3), padding='same')(conv)\n", " if batch_norm is True:\n", " conv = BatchNormalization(axis=axis)(conv)\n", " conv = Activation('relu')(conv)\n", " conv = Conv2D(size, (3, 3), padding='same')(conv)\n", " if batch_norm is True:\n", " conv = BatchNormalization(axis=axis)(conv)\n", " conv = Activation('relu')(conv)\n", " return conv" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "AYJktLHezoxC" }, "source": [ "\n", "\n", "---\n", "\n", "\n", "### **Attention modules** \n", "\n", "\n", "We use attention gate 1 at the three shallower layers to preserve the most information and use attention gate 2, which is the revised version of attention gate 1 specifically to improve the performance of the computation of sparse metrix.\n", "\n", "\n", "
\n", "\"the\n", " \n", "
\n", "\n", "*Attention gates of the U-net model (a: attention gate 1;b: attention gate 2)*" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "class multiplication(Layer):\n", " def __init__(self,inter_channel = None,**kwargs):\n", " super(multiplication, self).__init__(**kwargs)\n", " self.inter_channel = inter_channel\n", " def build(self,input_shape=None):\n", " self.k = self.add_weight(name='k',shape=(1,),initializer='zeros',dtype='float32',trainable=True)\n", " def get_config(self):\n", " base_config = super(multiplication, self).get_config()\n", " config = {'inter_channel':self.inter_channel}\n", " return dict(list(base_config.items()) + list(config.items())) \n", " def call(self,inputs):\n", " g,x,x_query,phi_g,x_value = inputs[0],inputs[1],inputs[2],inputs[3],inputs[4]\n", " h,w,c = int(x.shape[1]),int(x.shape[2]),int(x.shape[3])\n", " x_query = K.reshape(x_query, shape=(-1,h*w, self.inter_channel//4))\n", " phi_g = K.reshape(phi_g,shape=(-1,h*w,self.inter_channel//4))\n", " x_value = K.reshape(x_value,shape=(-1,h*w,c))\n", " scale = dot([K.permute_dimensions(phi_g,(0,2,1)), x_query], axes=(1, 2))\n", " soft_scale = Activation('softmax')(scale)\n", " scaled_value = dot([K.permute_dimensions(soft_scale,(0,2,1)),K.permute_dimensions(x_value,(0,2,1))],axes=(1, 2))\n", " scaled_value = K.reshape(scaled_value, shape=(-1,h,w,c)) \n", " customize_multi = self.k * scaled_value\n", " layero = add([customize_multi,x])\n", " my_concat = Lambda(lambda x: K.concatenate([x[0], x[1]], axis=3))\n", " concate = my_concat([layero,g])\n", " return concate \n", " def compute_output_shape(self,input_shape):\n", " ll = list(input_shape)[1]\n", " return (None,ll[1],ll[1],ll[3]*3)\n", " def get_custom_objects():\n", " return {'multiplication': multiplication}\n", "\n", "def attention_up_and_concatenate(inputs):\n", " g,x = inputs[0],inputs[1]\n", " inter_channel = g.get_shape().as_list()[3]\n", " g = Conv2DTranspose(inter_channel, (2,2), strides=[2, 2],padding='same')(g)\n", " x_query = Conv2D(inter_channel//4, [1, 1], strides=[1, 1], data_format='channels_last')(x)\n", " phi_g = Conv2D(inter_channel//4, [1, 1], strides=[1, 1], data_format='channels_last')(g)\n", " x_value = Conv2D(inter_channel//2, [1, 1], strides=[1, 1], data_format='channels_last')(x)\n", " inputs = [g,x,x_query,phi_g,x_value]\n", " concate = multiplication(inter_channel)(inputs)\n", " return concate" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "class multiplication2(Layer):\n", " def __init__(self,inter_channel = None,**kwargs):\n", " super(multiplication2, self).__init__(**kwargs)\n", " self.inter_channel = inter_channel\n", " def build(self,input_shape=None):\n", " self.k = self.add_weight(name='k',shape=(1,),initializer='zeros',dtype='float32',trainable=True)\n", " def get_config(self):\n", " base_config = super(multiplication2, self).get_config()\n", " config = {'inter_channel':self.inter_channel}\n", " return dict(list(base_config.items()) + list(config.items())) \n", " def call(self,inputs):\n", " g,x,rate = inputs[0],inputs[1],inputs[2]\n", " scaled_value = multiply([x, rate])\n", " att_x = self.k * scaled_value\n", " att_x = add([att_x,x])\n", " my_concat = Lambda(lambda x: K.concatenate([x[0], x[1]], axis=3))\n", " concate = my_concat([att_x, g])\n", " return concate \n", " def compute_output_shape(self,input_shape):\n", " ll = list(input_shape)[1]\n", " return (None,ll[1],ll[1],ll[3]*2)\n", " def get_custom_objects():\n", " return {'multiplication2': multiplication2}\n", "\n", "def attention_up_and_concatenate2(inputs):\n", " g, x = inputs[0],inputs[1]\n", " inter_channel = g.get_shape().as_list()[3]\n", " g = Conv2DTranspose(inter_channel//2, (3,3), strides=[2, 2],padding='same')(g)\n", " g = Conv2D(inter_channel//2, [1, 1], strides=[1, 1], data_format='channels_last')(g)\n", " theta_x = Conv2D(inter_channel//4, [1, 1], strides=[1, 1], data_format='channels_last')(x)\n", " phi_g = Conv2D(inter_channel//4, [1, 1], strides=[1, 1], data_format='channels_last')(g)\n", " f = Activation('relu')(add([theta_x, phi_g]))\n", " psi_f = Conv2D(1, [1, 1], strides=[1, 1], data_format='channels_last')(f)\n", " rate = Activation('sigmoid')(psi_f)\n", " concate = multiplication2()([g,x,rate])\n", " return concate" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "_ZJh0sy6JJNs" }, "source": [ "### Construct the U-Net model" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "colab": {}, "colab_type": "code", "id": "ulfTIuNnJJNu" }, "outputs": [], "source": [ "def UNET_224(weights=None):\n", " inputs = Input((IMG_WIDTH, IMG_WIDTH, INPUT_CHANNELS))\n", " filters = 32\n", " last_dropout = 0.2\n", "# convolutiona and pooling level 1\n", " conv_224 = Residual_CNN_block(inputs,filters)\n", " pool_112 = MaxPooling2D(pool_size=(2, 2))(conv_224)\n", "# convolutiona and pooling level 2\n", " conv_112 = Residual_CNN_block(pool_112,2*filters)\n", " pool_56 = MaxPooling2D(pool_size=(2, 2))(conv_112)\n", "# convolutiona and pooling level 3\n", " conv_56 = Residual_CNN_block(pool_56,4*filters)\n", " pool_28 = MaxPooling2D(pool_size=(2, 2))(conv_56)\n", "# convolutiona and pooling level 4\n", " conv_28 = Residual_CNN_block(pool_28,8*filters)\n", " pool_14 = MaxPooling2D(pool_size=(2, 2))(conv_28)\n", "# convolutiona and pooling level 5\n", " conv_14 = Residual_CNN_block(pool_14,16*filters)\n", " pool_7 = MaxPooling2D(pool_size=(2, 2))(conv_14)\n", "# Conlovlution and feature concatenation\n", " conv_7 = Residual_CNN_block(pool_7,32*filters)\n", "# Upsampling with convolution \n", " up_14 = attention_up_and_concatenate([conv_7, conv_14]) \n", " up_conv_14 = Residual_CNN_block(up_14,16*filters)\n", "# Upsampling with convolution 2\n", " up_28 = attention_up_and_concatenate([up_conv_14, conv_28])\n", " up_conv_28 = Residual_CNN_block(up_28,8*filters)\n", "# Upsampling with convolution 3\n", " up_56 = attention_up_and_concatenate2([up_conv_28, conv_56])\n", " up_conv_56 = Residual_CNN_block(up_56,4*filters)\n", "# Upsampling with convolution 4\n", " up_112 = attention_up_and_concatenate2([up_conv_56, conv_112])\n", " up_conv_112 = Residual_CNN_block(up_112,2*filters)\n", "# Upsampling with convolution 5\n", " up_224 = attention_up_and_concatenate2([up_conv_112, conv_224])\n", " #up_224 = attention_up_and_concatenate2(up_conv_112, conv_224)\n", " up_conv_224 = Residual_CNN_block(up_224,filters,dropout = last_dropout)\n", "# 1 dimensional convolution and generate probabilities from Sigmoid function\n", " conv_final = Conv2D(OUTPUT_MASK_CHANNELS, (1, 1))(up_conv_224) \n", " conv_final = Activation('sigmoid')(conv_final)\n", "# Generate model\n", " model = Model(inputs, conv_final, name=\"UNET_224\")\n", " return model" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "2X0G0osHJJNy" }, "source": [ "\n", "### Reading the input \n", "\n", "The followig code will initiate the trainging and validating dataset. Then, is initialize the U-Net CNN with all substantial parameters." ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 87 }, "colab_type": "code", "executionInfo": { "elapsed": 110624, "status": "ok", "timestamp": 1583962497434, "user": { "displayName": "nattapon jaroenchai", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj9i_TMlYzv27Vkas3v5qWkW7ZdTBQme3RVKqYp7c4=s64", "userId": "17092454241854925654" }, "user_tz": 300 }, "id": "YU-z6VfWJJN3", "outputId": "6213857c-a336-47b2-fad8-587e70cd74b4" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(245, 224, 224, 8)\n", "(245, 224, 224, 1)\n", "(20, 224, 224, 8)\n", "(20, 224, 224, 1)\n" ] } ], "source": [ "#Path to input files\n", "root=\"/home/jovyan/shared_data/data/unet_streamline_detection/\"\n", "\n", "# n/n2: upper_half_as_train_validation/lower_half_as_train_validation, \n", "# v/v2:left_half_as_train_validation/right_half_as_train_validation\n", "m='v2'\n", "import os\n", "import numpy as np\n", "import sys\n", "aug = '_aug'+m #m is set in the first code block as 'v2' as default\n", "# read in training and validation data\n", "X_train = np.load(root+'data/generated/train_data'+aug+'.npy')#[:2000]\n", "Y_train = np.load(root+'data/generated/train_label'+aug+'.npy')#[:,:,:,np.newaxis]#[:,:,:,np.newaxis]#[:2000]\n", "X_Validation = np.load(root+'data/generated/vali_data'+aug.split('_aug')[-1]+'.npy')#[:700]\n", "Y_Validation = np.load(root+'data/generated/vali_label'+aug.split('_aug')[-1]+'.npy')\n", "s = np.arange(X_train.shape[0])\n", "np.random.shuffle(s)\n", "X_train = X_train[s]\n", "Y_train = Y_train[s]\n", "\n", "print(X_train.shape)\n", "print(Y_train.shape)\n", "print(X_Validation.shape)\n", "print(Y_Validation.shape)" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 579 }, "colab_type": "code", "executionInfo": { "elapsed": 11900, "status": "ok", "timestamp": 1583962517029, "user": { "displayName": "nattapon jaroenchai", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj9i_TMlYzv27Vkas3v5qWkW7ZdTBQme3RVKqYp7c4=s64", "userId": "17092454241854925654" }, "user_tz": 300 }, "id": "KmXFR5TMm-lW", "outputId": "a7b923ae-82f8-49b1-de1b-798216a56f20" }, "outputs": [], "source": [ "patch_size = 224\n", "\n", "IMG_WIDTH = patch_size\n", "IMG_HEIGHT = patch_size\n", "# Number of feature channels \n", "INPUT_CHANNELS = 8\n", "# Number of output masks (1 in case you predict only one type of objects)\n", "OUTPUT_MASK_CHANNELS = 1\n", "maxepoch = 10\n", "# hyperparameters\n", "learning_rate =0.0000359\n", "patience = 20\n", "\n", "# create the CNN\n", "model = UNET_224()\n", "model.compile(optimizer=Adam(lr=learning_rate),loss = dice_coef_loss,metrics=[dice_coef,'accuracy'])\n", "callbacks = [\n", " ReduceLROnPlateau(monitor='val_loss', factor=0.7, patience=patience, min_lr=1e-9, verbose=1, mode='min'),\n", " EarlyStopping(monitor='val_loss', patience=patience, verbose=0),\n", " ModelCheckpoint('unetLr03Full.h5', monitor='val_loss', save_best_only=True, verbose=0),\n", " ]\n" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "e7lgzkmMJJOB" }, "source": [ "### Train the model\n", "The following line of code is the code for training the CNN with the trainging and validating datasets. To run this code we recommend running it in an environment that have GPU. It will take several hours to finish if you run it in this notebook. \n", "\n", "The result of the trainging and validating processes are saved \n", "\n", "
\n", "\n", "> results_03 = model.fit(X_train, Y_train, validation_data=(X_Validation,Y_Validation), batch_size=16, epochs=maxepoch, callbacks=callbacks)\n", "\n", "
\n", "\n", "*Note: Due to the limitation of the resources we do not encourage you to run this training part. In order to run this model, we suggest running the model on the machine with GPU or CPU with large memory. Running the model in this environment will take several hours.*" ] }, { "cell_type": "markdown", "metadata": { "colab_type": "text", "id": "7yEekt-8JJOD" }, "source": [ "### Save the Trained model\n", "\n", "The cell below is the code for saving the result of training and validating. The code will save the results into 4 files in **\"result\"** folder. The files are as follows:\n", "1. **model_\\*_attention2.yaml:** the model structure and not its weights or its training configuration\n", "2. **model_\\*_attention2.h5:** the trained model structure and weights that we can use to predict new set of data. \n", "3. **history_\\*_attention2.pickle:** the log of the trainging and validating processes which contains the statistic of training and validating phases.\n", "4. **preds_test_total_\\*_attention2.npy:** the result of the trained model with the testing dataset." ] }, { "cell_type": "markdown", "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 385 }, "colab_type": "code", "executionInfo": { "elapsed": 947, "status": "error", "timestamp": 1583962815456, "user": { "displayName": "nattapon jaroenchai", "photoUrl": "https://lh3.googleusercontent.com/a-/AOh14Gj9i_TMlYzv27Vkas3v5qWkW7ZdTBQme3RVKqYp7c4=s64", "userId": "17092454241854925654" }, "user_tz": 300 }, "id": "eCMIM2dWJJOI", "outputId": "99d7f9d9-fa1c-4c03-c2d0-a725ba1c93fa" }, "source": [ ">from keras.models import load_model \n", ">import pickle \n", ">from keras.models import model_from_yaml \n", "> \n", ">*\\# save the trained model* \n", ">model_yaml = model.to_yaml() \n", ">with open(root+\"result/model\"+aug+\"_attention2.yaml\", \"w\") as yaml_file: \n", "> yaml_file.write(model_yaml) \n", "> \n", ">*\\# save the weights* \n", ">model.save(root+'result/model'+aug+'_attention2.h5') \n", ">*\\# save the intermdediate results and training statistics* \n", ">with open(root+'result/history'+aug+'_attention2.pickle', 'wb') as file_pi: \n", "> pickle.dump(results_03.history, file_pi, protocol=2) \n", "> \n", ">*\\# load the prediction input* \n", ">X_test = np.load(root+\"data/prediction_data.npy\") \n", ">X_test.shape \n", "> \n", ">*\\# Save the predicted labels.* \n", ">X_test = np.load(\"prediction_data.npy\") \n", ">preds_test = model.predict(X_test) \n", ">preds_test_t = (preds_test > 0.5).astype(np.uint8) \n", ">np.save(root+'result/preds_test_total'+aug+'_attention2.npy',preds_test_t) \n", "\n", "*Note: This section is disabled becasue it will always show error becasue we do not finish the training step*" ] } ], "metadata": { "accelerator": "GPU", "colab": { "collapsed_sections": [], "name": "2.2_Code_Model_Training.ipynb", "provenance": [] }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.6" } }, "nbformat": 4, "nbformat_minor": 4 }