{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Simple Example of Incremental Learning Agent w/Python + Keras\n",
"# (EXPERIMENT)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This notebook walks through an experiment with building a machine learning model that will learn incrementally.\n",
"\n",
"Why attempt to learn incrementally?\n",
"\n",
"There are two primary reasons for attempting to build a model that learns incrementally:\n",
"\n",
"1. Limited Data/Cold start problem\n",
"(Cold Start Problem~ Concerns the issue that the system cannot draw any inferences for users or items about which it has not yet gathered sufficient information). As we know, building a machine learning model requires a lot of data. This can hamper the inevitable problem of getting started (\"Guess we can add ML once we have more data...\"). But what if the model could start providing limited (admittedly low accuracy) predictions today and build up over time? \n",
"\n",
"2. Data Privacy\n",
"The transfer of large files (training) to the cloud creates data privacy and security issues. Using incremental learning, we can build a model without the need to store sensitive files in the cloud. With this method, any private data can be encrypted in transit and only the model itself is stored on disk. "
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import numpy\n",
"from keras.models import Sequential\n",
"from keras.layers import Dense\n",
"import numpy as np\n",
"import pandas as pd\n",
"\n",
"np.random.seed(4)"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"'''\n",
"This dataset is originally from the National Institute of Diabetes and Digestive and Kidney Diseases. \n",
"The objective is to predict based on diagnostic measurements whether a patient has diabetes.\n",
"https://data.world/data-society/pima-indians-diabetes-database\n",
"'''\n",
"df = pd.read_csv(\"../datasets/diabetes.csv\")"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Features & Target\n",
"X = df.iloc[:, :8].values\n",
"y = df.iloc[:, 8].values"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"from sklearn.preprocessing import MinMaxScaler\n",
"from sklearn.model_selection import train_test_split\n",
"from keras import optimizers\n",
"\n",
"# Need to rescale the features\n",
"scaler = MinMaxScaler(feature_range=(0, 1))\n",
"rescaledX = scaler.fit_transform(X)\n",
"\n",
"# Split for training and evaluation\n",
"X_train, X_test, y_train, y_test = train_test_split(rescaledX, y, test_size=0.33)"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"import hashlib\n",
"from keras.models import load_model"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"class IncrementalAgent:\n",
" \n",
" def __init__(self, name):\n",
" # Identify agent\n",
" self.name = name\n",
" # Need to keep track of predictions for evaluation of the model\n",
" self.prediction_records = {}\n",
" # Create model\n",
" self.model = Sequential()\n",
" self.model.add(Dense(12, input_dim=8, activation='relu'))\n",
" self.model.add(Dense(8, activation='relu'))\n",
" self.model.add(Dense(1, activation='sigmoid'))\n",
" # Compile model\n",
" sgd = optimizers.SGD(lr=0.01)\n",
" self.model.compile(loss='binary_crossentropy', optimizer=sgd, metrics=['accuracy'])\n",
" print(\"Model Initialized.\")\n",
"\n",
" \n",
" def train(self, feature_array, target, epoch_count, batch_count):\n",
" target_to_array = np.array([[target]])\n",
" self.model.fit(feature_array, target_to_array, epochs=epoch_count, batch_size=batch_count)\n",
" self.model.save('{}.h5'.format(self.name))\n",
" print(\"Model Trained.\")\n",
" \n",
" \n",
" def evaluate(self, feature_array, target):\n",
" # Check if the features have previously been seen\n",
" prediction_ref = hashlib.sha256(feature_array).hexdigest()\n",
" if prediction_ref in self.prediction_records:\n",
" print(\"Previous Predicted Class: {}\".format(self.prediction_records[prediction_ref]))\n",
" print(\"Actual Class: {}\".format(target))\n",
" if self.prediction_records[prediction_ref] == target:\n",
" print(\"Prediction Correct!\")\n",
" else:\n",
" print(\"Prediction Incorrect. Relearn...\")\n",
" self.train(feature_array, target, 5, 1)\n",
" print(\"Retry Predict & Update Previous Prediction\")\n",
" self.predict(feature_array)\n",
" \n",
" else:\n",
" print(\"First time seeing this sample.\")\n",
" \n",
"\n",
" def predict(self, feature_array):\n",
" # Create ID for prediction\n",
" prediction_ref = hashlib.sha256(feature_array).hexdigest()\n",
" # Actual prediction\n",
" probability_prediction = self.model.predict(feature_array)\n",
" probability_prediction = probability_prediction[0][0]\n",
" print(\"Raw Probability: {}\".format(probability_prediction))\n",
" \n",
" # Get Class Prediction\n",
" if probability_prediction > 0.5:\n",
" prediction = 1\n",
" else:\n",
" prediction = 0\n",
" \n",
" # Add Prediction to Records\n",
" self.prediction_records[prediction_ref] = prediction\n",
" \n",
" print(\"Predicted Class: {}\".format(prediction))"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Model Initialized.\n"
]
}
],
"source": [
"agent1 = IncrementalAgent('Agent1')"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"sample = np.array([X_train[2]])\n",
"target = np.array([y_train[2]])[0]"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Epoch 1/5\n",
"1/1 [==============================] - 0s 170ms/step - loss: 1.0602 - acc: 0.0000e+00\n",
"Epoch 2/5\n",
"1/1 [==============================] - 0s 1ms/step - loss: 1.0288 - acc: 0.0000e+00\n",
"Epoch 3/5\n",
"1/1 [==============================] - 0s 2ms/step - loss: 0.9885 - acc: 0.0000e+00\n",
"Epoch 4/5\n",
"1/1 [==============================] - 0s 2ms/step - loss: 0.9506 - acc: 0.0000e+00\n",
"Epoch 5/5\n",
"1/1 [==============================] - 0s 2ms/step - loss: 0.9148 - acc: 0.0000e+00\n",
"Model Trained.\n"
]
}
],
"source": [
"agent1.train(sample, target, 5, 1)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"new_sample = np.array([X_train[3]])"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Raw Probability: 0.4871627986431122\n",
"Predicted Class: 0\n"
]
}
],
"source": [
"agent1.predict(new_sample)"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"new_target = np.array([y_train[3]])[0]"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Previous Predicted Class: 0\n",
"Actual Class: 0\n",
"Prediction Correct!\n"
]
}
],
"source": [
"agent1.evaluate(new_sample, new_target)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"next_sample = np.array([X_train[10]])"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Raw Probability: 0.5264965891838074\n",
"Predicted Class: 1\n"
]
}
],
"source": [
"agent1.predict(next_sample)"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"next_target = np.array([y_train[10]])[0]"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Previous Predicted Class: 1\n",
"Actual Class: 1\n",
"Prediction Correct!\n"
]
}
],
"source": [
"agent1.evaluate(next_sample, next_target)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### Caveats:\n",
"- This example does not result in a greatly accurate model.\n",
"- There are better algorithm choices other than NNs for this dataset."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.5.2"
}
},
"nbformat": 4,
"nbformat_minor": 2
}