{"nbformat":4,"nbformat_minor":0,"metadata":{"colab":{"name":"2022-01-20-adversarial.ipynb","provenance":[{"file_id":"https://github.com/recohut/nbs/blob/main/raw/T102448%20%7C%20Adversarial%20Learning%20for%20Recommendation.ipynb","timestamp":1644653104262}],"collapsed_sections":[],"mount_file_id":"1Y4-jjSksUSJwsRiEhXrobQnraBT-U1XA","authorship_tag":"ABX9TyMxJ0o/Vv98HHolXe8JcWFk"},"kernelspec":{"name":"python3","display_name":"Python 3"},"language_info":{"name":"python"}},"cells":[{"cell_type":"markdown","metadata":{"id":"OX59ktWF29bu"},"source":["# Adversarial Training (Regularization) on a Recommender System"]},{"cell_type":"markdown","metadata":{"id":"bce1rkKF2-Ii"},"source":["Adversarial training (also adversarial regularization) is a defense strategy against adversarial perturbations. The main intuition is to increase the robustness of a recommender system on minimal adversarial perturbation of model parameters by adding further training iterations that takes into account the application of such perturbations."]},{"cell_type":"markdown","metadata":{"id":"AWUR7Onl3Dfb"},"source":["In this notebook, we become familiar with the usefulness of the adversarial regularization by:\n","- Training classical model-based recommender (BPR-MF) on a small part of Movielens-1M\n","- Attacking the learned model with FGSM-like Adversarial Perturbations\n","- Adversarially Training the model-based recommender (BPR-MF) with Adversarial Personalized Ranking (APR)\n","- Attacking the robustified model"]},{"cell_type":"markdown","metadata":{"id":"zNLxxtYj35Ry"},"source":["### Imports"]},{"cell_type":"code","metadata":{"id":"vx6mrrnv-_g_"},"source":["!pip install -q timethis"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"mzYu3R8B37SC"},"source":["import numpy as np\n","import tensorflow as tf\n","from abc import ABC\n","from timethis import timethis"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"Xf9-n3ii3yaj"},"source":["### Global settings"]},{"cell_type":"code","metadata":{"id":"QmgqS8C730q6"},"source":["np.random.seed(42)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"UaY2poEh3mMX"},"source":["### Utils"]},{"cell_type":"code","metadata":{"id":"YwPuM5DJ3n_9"},"source":["import time\n","from functools import wraps\n","\n","# A simple decorator\n","def timethis(func):\n","    @wraps(func)\n","    def wrapper(*args, **kwargs):\n","        start = time.time()\n","        r = func(*args, **kwargs)\n","        end = time.time()\n","        print(end-start)\n","        return r\n","    return wrapper"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"5ZUmfAuI3JSF"},"source":["### Evaluator"]},{"cell_type":"code","metadata":{"id":"FjUioN5J3Xw3"},"source":["import numpy as np\n","from multiprocessing import Pool\n","from multiprocessing import cpu_count\n","import sys\n","import math\n","from time import time\n","\n","\n","_feed_dict = None\n","_dataset = None\n","_model = None\n","_sess = None\n","_K = None\n","\n","\n","def _init_eval_model(data):\n","    global _dataset\n","    _dataset = data\n","\n","    pool = Pool(cpu_count() - 1)\n","    feed_dicts = pool.map(_evaluate_input, range(_dataset.num_users))\n","    pool.close()\n","    pool.join()\n","\n","    return feed_dicts\n","\n","\n","def _evaluate_input(user):\n","    # generate items_list\n","    try:\n","        test_item = _dataset.test[user][1]\n","        item_input = set(range(_dataset.num_items)) - set(_dataset.train_list[user])\n","        if test_item in item_input:\n","            item_input.remove(test_item)\n","        item_input = list(item_input)\n","        item_input.append(test_item)\n","        user_input = np.full(len(item_input), user, dtype='int32')[:, None]\n","        item_input = np.array(item_input)[:, None]\n","        return user_input, item_input\n","    except:\n","        print('******' + user)\n","        return 0, 0\n","\n","\n","def _eval_by_user(user):\n","    # get predictions of data in testing set\n","    user_input, item_input = _feed_dicts[user]\n","    predictions, _, _ = _model.get_inference(user_input, item_input)\n","\n","    neg_predict, pos_predict = predictions[:-1], predictions[-1]\n","    position = (neg_predict.numpy() >= pos_predict.numpy()).sum()\n","\n","    # calculate from HR@1 to HR@100, and from NDCG@1 to NDCG@100, AUC\n","    hr, ndcg, auc = [], [], []\n","    for k in range(1, _K + 1):\n","        hr.append(position < k)\n","        ndcg.append(math.log(2) / math.log(position + 2) if position < k else 0)\n","        auc.append(\n","            1 - (position / len(neg_predict)))  # formula: [#(Xui>Xuj) / #(Items)] = [1 - #(Xui<=Xuj) / #(Items)]\n","\n","    return hr, ndcg, auc\n","\n","\n","class Evaluator:\n","    def __init__(self, model, data, k):\n","        \"\"\"\n","        Class to manage all the evaluation methods and operation\n","        :param data: dataset object\n","        :param k: top-k evaluation\n","        \"\"\"\n","        self.data = data\n","        self.k = k\n","        self.eval_feed_dicts = _init_eval_model(data)\n","        self.model = model\n","\n","    def eval(self, epoch=0, results={}, epoch_text='', start_time=0):\n","        \"\"\"\n","        Runtime Evaluation of Accuracy Performance (top-k)\n","        :return:\n","        \"\"\"\n","        global _model\n","        global _K\n","        global _dataset\n","        global _feed_dicts\n","        _dataset = self.data\n","        _model = self.model\n","        _K = self.k\n","        _feed_dicts = self.eval_feed_dicts\n","\n","        res = []\n","        for user in range(self.model.data.num_users):\n","            res.append(_eval_by_user(user))\n","\n","        hr, ndcg, auc = (np.array(res).mean(axis=0)).tolist()\n","        print(\"%s %.3f Performance@%d \\tHR: %.4f\\tnDCG: %.4f\\tAUC: %.4f\" % (\n","            epoch_text, time() - start_time, _K, hr[_K - 1], ndcg[_K - 1], auc[_K - 1]))\n","\n","        if len(epoch_text) != '':\n","            results[epoch] = {'hr': hr, 'ndcg': ndcg, 'auc': auc[0]}\n","\n","    def store_recommendation(self, attack_name=\"\"):\n","        \"\"\"\n","        Store recommendation list (top-k) in order to be used for the ranksys framework (sisinflab)\n","        attack_name: The name for the attack stored file\n","        :return:\n","        \"\"\"\n","        results = self.model.get_full_inference().numpy()\n","        with open('{0}{1}_best{2}_top{3}_rec.tsv'.format(self.model.path_output_rec_result,\n","                                                         attack_name + self.model.path_output_rec_result.split('/')[\n","                                                             -2],\n","                                                         self.model.best,\n","                                                         self.k),\n","                  'w') as out:\n","            for u in range(results.shape[0]):\n","                results[u][self.data.train_list[u]] = -np.inf\n","                top_k_id = results[u].argsort()[-self.k:][::-1]\n","                top_k_score = results[u][top_k_id]\n","                for i, value in enumerate(top_k_id):\n","                    out.write(str(u) + '\\t' + str(value) + '\\t' + str(top_k_score[i]) + '\\n')\n","\n","    def evaluate(self):\n","        \"\"\"\n","        Runtime Evaluation of Accuracy Performance (top-k)\n","        \"\"\"\n","        global _model\n","        global _K\n","        global _dataset\n","        global _feed_dicts\n","        _dataset = self.data\n","        _model = self.model\n","        _K = self.k\n","        _feed_dicts = self.eval_feed_dicts\n","\n","        res = []\n","        for user in range(self.model.data.num_users):\n","            res.append(_eval_by_user(user))\n","\n","        hr, ndcg, auc = (np.array(res).mean(axis=0)).tolist()\n","        print(\"Performance@%d\\n\\tHR: %.4f\\tnDCG: %.4f\\tAUC: %.4f\" % (\n","            _K, hr[_K - 1], ndcg[_K - 1], auc[_K - 1]))\n","\n","        return hr[_K - 1], ndcg[_K - 1], auc[_K - 1]"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"l3x1_EGH3-FN"},"source":["## Data"]},{"cell_type":"markdown","metadata":{"id":"NgQf4Uvs4Gjk"},"source":["### Dataloader"]},{"cell_type":"code","metadata":{"id":"A5101miN4J8I"},"source":["import scipy.sparse as sp\n","import numpy as np\n","from multiprocessing import Pool\n","from multiprocessing import cpu_count\n","import pandas as pd\n","from scipy.sparse import dok_matrix\n","from time import time\n","\n","\n","_user_input = None\n","_item_input_pos = None\n","_batch_size = None\n","_index = None\n","_model = None\n","_train = None\n","_test = None\n","_num_items = None\n","\n","\n","def _get_train_batch(i):\n","    \"\"\"\n","    Generation of a batch in multiprocessing\n","    :param i: index to control the batch generayion\n","    :return:\n","    \"\"\"\n","    user_batch, item_pos_batch, item_neg_batch = [], [], []\n","    begin = i * _batch_size\n","    for idx in range(begin, begin + _batch_size):\n","        user_batch.append(_user_input[_index[idx]])\n","        item_pos_batch.append(_item_input_pos[_index[idx]])\n","        j = np.random.randint(_num_items)\n","        while j in _train[_user_input[_index[idx]]]:\n","            j = np.random.randint(_num_items)\n","        item_neg_batch.append(j)\n","    return np.array(user_batch)[:, None], np.array(item_pos_batch)[:, None], np.array(item_neg_batch)[:, None]\n","\n","\n","class DataLoader(object):\n","    \"\"\"\n","    Load train and test dataset\n","    \"\"\"\n","\n","    def __init__(self, path_train_data, path_test_data):\n","        \"\"\"\n","        Constructor of DataLoader\n","        :param path_train_data: relative path for train file\n","        :param path_test_data: relative path for test file\n","        \"\"\"\n","        self.num_users, self.num_items = self.get_length(path_train_data, path_test_data)\n","        self.load_train_file(path_train_data)\n","        self.load_train_file_as_list(path_train_data)\n","        self.load_test_file(path_test_data)\n","        self._user_input, self._item_input_pos = self.sampling()\n","        print('{0} - Loaded'.format(path_train_data))\n","        print('{0} - Loaded'.format(path_test_data))\n","\n","    def get_length(self, train_name, test_name):\n","        train = pd.read_csv(train_name, sep='\\t', header=None)\n","        test = pd.read_csv(test_name, sep='\\t', header=None)\n","        try:\n","            train.columns = ['user', 'item', 'r', 't']\n","            test.columns = ['user', 'item', 'r', 't']\n","            data = train.copy()\n","            data = data.append(test, ignore_index=True)\n","        except:\n","            train.columns = ['user', 'item', 'r']\n","            test.columns = ['user', 'item', 'r']\n","            data = train.copy()\n","            data = data.append(test, ignore_index=True)\n","\n","        return data['user'].nunique(), data['item'].nunique()\n","\n","    def load_train_file(self, filename):\n","        \"\"\"\n","        Read /data/dataset_name/train file and Return the matrix.\n","        \"\"\"\n","        # Get number of users and items\n","        # self.num_users, self.num_items = 0, 0\n","        with open(filename, \"r\") as f:\n","            line = f.readline()\n","            while line is not None and line != \"\":\n","                arr = line.split(\"\\t\")\n","                u, i = int(arr[0]), int(arr[1])\n","                # self.num_users = max(self.num_users, u)\n","                # self.num_items = max(self.num_items, i)\n","                line = f.readline()\n","\n","        # Construct URM\n","        self.train = sp.dok_matrix((self.num_users + 1, self.num_items + 1), dtype=np.float32)\n","        with open(filename, \"r\") as f:\n","            line = f.readline()\n","            while line is not None and line != \"\":\n","                arr = line.split(\"\\t\")\n","                user, item, rating = int(arr[0]), int(arr[1]), float(arr[2])\n","                if rating > 0:\n","                    self.train[user, item] = 1.0\n","                line = f.readline()\n","\n","        # self.num_users = self.train.shape[0]\n","        # self.num_items = self.train.shape[1]\n","\n","    def load_train_file_as_list(self, filename):\n","        # Get number of users and items\n","        u_ = 0\n","        self.train_list, items = [], []\n","        with open(filename, \"r\") as f:\n","            line = f.readline()\n","            index = 0\n","            while line is not None and line != \"\":\n","                arr = line.split(\"\\t\")\n","                u, i = int(arr[0]), int(arr[1])\n","                if u_ < u:\n","                    index = 0\n","                    self.train_list.append(items)\n","                    items = []\n","                    u_ += 1\n","                index += 1\n","                items.append(i)\n","                line = f.readline()\n","        self.train_list.append(items)\n","\n","    def load_test_file(self, filename):\n","        self.test = []\n","        with open(filename, \"r\") as f:\n","            line = f.readline()\n","            while line != None and line != \"\":\n","                arr = line.split(\"\\t\")\n","                user, item = int(arr[0]), int(arr[1])\n","                self.test.append([user, item])\n","                line = f.readline()\n","\n","    def sampling(self):\n","        _user_input, _item_input_pos = [], []\n","        for (u, i) in self.train.keys():\n","            # positive instance\n","            _user_input.append(u)\n","            _item_input_pos.append(i)\n","        return _user_input, _item_input_pos\n","\n","    def shuffle(self, batch_size=512):\n","        \"\"\"\n","        Shuffle dataset to create batch with batch size\n","        Variable are global to be faster!\n","        :param batch_size: default 512\n","        :return: set of all generated random batches\n","        \"\"\"\n","        global _user_input\n","        global _item_input_pos\n","        global _batch_size\n","        global _index\n","        global _model\n","        global _train\n","        global _num_items\n","\n","        _user_input, _item_input_pos = self._user_input, self._item_input_pos\n","        _batch_size = batch_size\n","        _index = list(range(len(_user_input)))\n","        _train = self.train_list\n","        _num_items = self.num_items\n","\n","        np.random.shuffle(_index)\n","        _num_batches = len(_user_input) // _batch_size\n","        pool = Pool(cpu_count())\n","        res = pool.map(_get_train_batch, range(_num_batches))\n","        pool.close()\n","        pool.join()\n","\n","        user_input = [r[0] for r in res]\n","        item_input_pos = [r[1] for r in res]\n","        item_input_neg = [r[2] for r in res]\n","        return user_input, item_input_pos, item_input_neg"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"1D9J4pA44iDp"},"source":["### Download"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"yacgk5lF4EaX","executionInfo":{"status":"ok","timestamp":1633177854810,"user_tz":-330,"elapsed":972,"user":{"displayName":"Sparsh Agarwal","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"13037694610922482904"}},"outputId":"12246c2d-48e7-4676-c26b-e51984b5f64c"},"source":["!wget -q --show-progress https://github.com/sisinflab/HandsOn-ECIR2021/raw/master/data/movielens-500/trainingset.tsv\n","!wget -q --show-progress https://github.com/sisinflab/HandsOn-ECIR2021/raw/master/data/movielens-500/testset.tsv"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["trainingset.tsv.2   100%[===================>]   1.40M  --.-KB/s    in 0.1s    \n","testset.tsv.2       100%[===================>]   9.83K  --.-KB/s    in 0s      \n"]}]},{"cell_type":"markdown","metadata":{"id":"2SKLoD2v3-y4"},"source":["First, we will load a short version of Movielens 1M dataset, which has been pre-processed and stored as a TSV file with the following structure: user_id, item_id, rating, timestamp. We have already divided the dataset in training and test sets using the leave-one-out evaluation protocol. We have used a small version with 500 users to reduce the computation time. To execute with the full dataset, you can change 'movielens-500' with 'movielens'."]},{"cell_type":"markdown","metadata":{"id":"o5wsntg265S_"},"source":["### Load"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"BZ0aZ2b-4gNj","executionInfo":{"status":"ok","timestamp":1633177857251,"user_tz":-330,"elapsed":1613,"user":{"displayName":"Sparsh Agarwal","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"13037694610922482904"}},"outputId":"8cacb0b5-cc56-4a86-e006-2d69f325ee32"},"source":["data = DataLoader(path_train_data='trainingset.tsv', path_test_data='testset.tsv')\n","\n","print('\\nStatistics:\\nNumber of Users: {0}\\nNumber of Items: {1}\\nTraining User-Item Ratings: {2}'.format(data.num_users, data.num_items, len(data.train)))"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["trainingset.tsv - Loaded\n","testset.tsv - Loaded\n","\n","Statistics:\n","Number of Users: 500\n","Number of Items: 3172\n","Training User-Item Ratings: 73371\n"]}]},{"cell_type":"markdown","metadata":{"id":"9CE7Zo7-62FY"},"source":["## Model"]},{"cell_type":"markdown","metadata":{"id":"1_scRTiS4qP7"},"source":["### Define the model"]},{"cell_type":"markdown","metadata":{"id":"-0hXnqt06Y7p"},"source":["We will define a new Tensorflow 2 model class to define the model (BPR-MF). For a matter of simplicity we have also implemented the adversarial attack and defense strategies,, that will be used in the later sections."]},{"cell_type":"code","metadata":{"id":"mnAmNEmQ6hIz"},"source":["class RecommenderModel(tf.keras.Model, ABC):\n","    \"\"\"\n","    This class represents a recommender model.\n","    You can load a pretrained model by specifying its ckpt path and use it for training/testing purposes.\n","    \"\"\"\n","\n","    def __init__(self, data, path_output_rec_result, path_output_rec_weight, rec):\n","        super(RecommenderModel, self).__init__()\n","        self.rec = rec\n","        self.data = data\n","        self.num_items = data.num_items\n","        self.num_users = data.num_users\n","        self.path_output_rec_result = path_output_rec_result\n","        self.path_output_rec_weight = path_output_rec_weight\n","\n","    def train(self):\n","        pass"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"u9BG_LpS6s5U"},"source":["TOPK = 100 # Top-K"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"id":"0cr8QzL16tF5"},"source":["class BPRMF(RecommenderModel):\n","    def __init__(self, data_loader, path_output_rec_result, path_output_rec_weight):\n","        super(BPRMF, self).__init__(data_loader, path_output_rec_result, path_output_rec_weight, 'bprmf')\n","        self.embedding_size = 64\n","        self.learning_rate = 0.05\n","        self.reg = 0\n","        self.epochs = 5\n","        self.batch_size = 512\n","        self.verbose = 1\n","        self.evaluator = Evaluator(self, data, TOPK)\n","\n","        self.initialize_model_parameters()\n","        self.initialize_perturbations()\n","        self.initialize_optimizer()\n","\n","    def initialize_model_parameters(self):\n","        \"\"\"\n","            Initialize Model Parameters\n","        \"\"\"\n","        self.embedding_P = tf.Variable(tf.random.truncated_normal(shape=[self.num_users, self.embedding_size], mean=0.0, stddev=0.01))  # (users, embedding_size)\n","        self.embedding_Q = tf.Variable(tf.random.truncated_normal(shape=[self.num_items, self.embedding_size], mean=0.0, stddev=0.01))  # (items, embedding_size)\n","        self.h = tf.constant(1.0, tf.float32, [self.embedding_size, 1])\n","\n","    def initialize_optimizer(self):\n","        \"\"\"\n","            Optimizer\n","        \"\"\"\n","        self.optimizer = tf.keras.optimizers.Adagrad(learning_rate=self.learning_rate)\n","\n","    def initialize_perturbations(self):\n","        \"\"\"\n","            Set delta variables useful to store delta perturbations,\n","        \"\"\"\n","        self.delta_P = tf.Variable(tf.zeros(shape=[self.num_users, self.embedding_size]), trainable=False)\n","        self.delta_Q = tf.Variable(tf.zeros(shape=[self.num_items, self.embedding_size]), trainable=False)\n","\n","    def get_inference(self, user_input, item_input_pos):\n","        \"\"\"\n","            Generate Prediction Matrix with respect to passed users and items identifiers\n","        \"\"\"\n","        self.embedding_p = tf.reduce_sum(tf.nn.embedding_lookup(self.embedding_P + self.delta_P, user_input), 1)\n","        self.embedding_q = tf.reduce_sum(tf.nn.embedding_lookup(self.embedding_Q + self.delta_Q, item_input_pos), 1)\n","\n","        return tf.matmul(self.embedding_p * self.embedding_q,self.h), self.embedding_p, self.embedding_q  # (b, embedding_size) * (embedding_size, 1)\n","\n","    def get_full_inference(self):\n","        \"\"\"\n","            Get Full Predictions useful for Full Store of Predictions\n","        \"\"\"\n","        return tf.matmul(self.embedding_P + self.delta_P, tf.transpose(self.embedding_Q + self.delta_Q))\n","\n","    @timethis\n","    def _train_step(self, batches):\n","        \"\"\"\n","            Apply a Single Training Step (across all the batches in the dataset).\n","        \"\"\"\n","        user_input, item_input_pos, item_input_neg = batches\n","\n","        for batch_idx in range(len(user_input)):\n","            with tf.GradientTape() as t:\n","                t.watch([self.embedding_P, self.embedding_Q])\n","\n","                # Model Inference\n","                self.output_pos, embed_p_pos, embed_q_pos = self.get_inference(user_input[batch_idx],\n","                                                                               item_input_pos[batch_idx])\n","                self.output_neg, embed_p_neg, embed_q_neg = self.get_inference(user_input[batch_idx],\n","                                                                               item_input_neg[batch_idx])\n","                self.result = tf.clip_by_value(self.output_pos - self.output_neg, -80.0, 1e8)\n","\n","                self.loss = tf.reduce_sum(tf.nn.softplus(-self.result))\n","\n","                # Regularization Component\n","                self.reg_loss = self.reg * tf.reduce_mean(tf.square(embed_p_pos) + tf.square(embed_q_pos) + tf.square(embed_q_neg))\n","\n","                # Loss Function\n","                self.loss_opt = self.loss + self.reg_loss\n","\n","            gradients = t.gradient(self.loss_opt, [self.embedding_P, self.embedding_Q])\n","            self.optimizer.apply_gradients(zip(gradients, [self.embedding_P, self.embedding_Q]))\n","    \n","    @timethis\n","    def train(self):\n","        for epoch in range(self.epochs):\n","            batches = self.data.shuffle(self.batch_size)\n","            self._train_step(batches)\n","            print('Epoch {0}/{1}'.format(epoch+1, self.epochs))\n","\n","    @timethis\n","    def _adversarial_train_step(self, batches, epsilon):\n","        \"\"\"\n","            Apply a Single Training Step (across all the batches in the dataset).\n","        \"\"\"\n","        user_input, item_input_pos, item_input_neg = batches\n","        adv_reg = 1\n","\n","        for batch_idx in range(len(user_input)):\n","            with tf.GradientTape() as t:\n","                t.watch([self.embedding_P, self.embedding_Q])\n","\n","                # Model Inference\n","                self.output_pos, embed_p_pos, embed_q_pos = self.get_inference(user_input[batch_idx],\n","                                                                               item_input_pos[batch_idx])\n","                self.output_neg, embed_p_neg, embed_q_neg = self.get_inference(user_input[batch_idx],\n","                                                                               item_input_neg[batch_idx])\n","                self.result = tf.clip_by_value(self.output_pos - self.output_neg, -80.0, 1e8)\n","\n","                self.loss = tf.reduce_sum(tf.nn.softplus(-self.result))\n","\n","                # Regularization Component\n","                self.reg_loss = self.reg * tf.reduce_mean(tf.square(embed_p_pos) + tf.square(embed_q_pos) + tf.square(embed_q_neg))\n","\n","                # Adversarial Regularization Component\n","                ##  Execute the Adversarial Attack on the Current Model (Perturb Model Parameters)\n","                self.execute_adversarial_attack(epsilon)\n","                ##  Inference on the Adversarial Perturbed Model\n","                self.output_pos_adver, _, _ = self.get_inference(user_input[batch_idx], item_input_pos[batch_idx])\n","                self.output_neg_adver, _, _ = self.get_inference(user_input[batch_idx], item_input_neg[batch_idx])\n","\n","                self.result_adver = tf.clip_by_value(self.output_pos_adver - self.output_neg_adver, -80.0, 1e8)\n","                self.loss_adver = tf.reduce_sum(tf.nn.softplus(-self.result_adver))\n","\n","                # Loss Function\n","                self.adversarial_regularizer = adv_reg * self.loss_adver # AMF = Adversarial Matrix Factorization\n","                self.bprmf_loss = self.loss + self.reg_loss\n","\n","                self.amf_loss = self.bprmf_loss + self.adversarial_regularizer\n","\n","            gradients = t.gradient(self.amf_loss, [self.embedding_P, self.embedding_Q])\n","            self.optimizer.apply_gradients(zip(gradients, [self.embedding_P, self.embedding_Q]))\n","\n","        self.initialize_perturbations()\n","\n","\n","    @timethis\n","    def adversarial_train(self, adversarial_epochs, epsilon):\n","        for epoch in range(adversarial_epochs):\n","            batches = self.data.shuffle(self.batch_size)\n","            self._adversarial_train_step(batches, epsilon)\n","            print('Epoch {0}/{1}'.format(self.epochs+epoch+1, self.epochs+adversarial_epochs))\n","\n","    def execute_adversarial_attack(self, epsilon):\n","        user_input, item_input_pos, item_input_neg = self.data.shuffle(len(self.data._user_input))\n","        self.initialize_perturbations()\n","\n","        with tf.GradientTape() as tape_adv:\n","            tape_adv.watch([self.embedding_P, self.embedding_Q])\n","            # Evaluate Current Model Inference\n","            output_pos, embed_p_pos, embed_q_pos = self.get_inference(user_input[0],\n","                                                                      item_input_pos[0])\n","            output_neg, embed_p_neg, embed_q_neg = self.get_inference(user_input[0],\n","                                                                      item_input_neg[0])\n","            result = tf.clip_by_value(output_pos - output_neg, -80.0, 1e8)\n","            loss = tf.reduce_sum(tf.nn.softplus(-result))\n","            loss += self.reg * tf.reduce_mean(\n","                tf.square(embed_p_pos) + tf.square(embed_q_pos) + tf.square(embed_q_neg))\n","        # Evaluate the Gradient\n","        grad_P, grad_Q = tape_adv.gradient(loss, [self.embedding_P, self.embedding_Q])\n","        grad_P, grad_Q = tf.stop_gradient(grad_P), tf.stop_gradient(grad_Q)\n","\n","        # Use the Gradient to Build the Adversarial Perturbations (https://doi.org/10.1145/3209978.3209981)\n","        self.delta_P = tf.nn.l2_normalize(grad_P, 1) * epsilon\n","        self.delta_Q = tf.nn.l2_normalize(grad_Q, 1) * epsilon"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"k8gvf2v46wCD"},"source":["## Initialize and Train The Model"]},{"cell_type":"code","metadata":{"id":"PJg3jn7h7Grs"},"source":["!mkdir -p rec_result rec_weights"],"execution_count":null,"outputs":[]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"HyKeZ_8n7CHH","executionInfo":{"status":"ok","timestamp":1633178123010,"user_tz":-330,"elapsed":15956,"user":{"displayName":"Sparsh Agarwal","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"13037694610922482904"}},"outputId":"92e55739-b1b2-4630-ed15-ac52fe2cd013"},"source":["recommender_model = BPRMF(data, 'rec_result/', 'rec_weights/')\n","recommender_model.train()"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["2.111112356185913\n","Epoch 1/5\n","2.135735273361206\n","Epoch 2/5\n","2.1026523113250732\n","Epoch 3/5\n","2.0907275676727295\n","Epoch 4/5\n","2.070622682571411\n","Epoch 5/5\n","15.236634731292725\n"]}]},{"cell_type":"markdown","metadata":{"id":"gIYX8PnS7JuW"},"source":["## Evaluate The Model"]},{"cell_type":"markdown","metadata":{"id":"XFWdk-ol8EwX"},"source":["The evaluation is computed on TOPK recommendation lists (default K = 100)."]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"4qy6ACNxAPB9","executionInfo":{"status":"ok","timestamp":1633178535451,"user_tz":-330,"elapsed":2172,"user":{"displayName":"Sparsh Agarwal","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"13037694610922482904"}},"outputId":"99d4a990-7ce4-4f55-b1f0-0c6636bcdbe0"},"source":["before_adv_hr, before_adv_ndcg, before_adv_auc = recommender_model.evaluator.evaluate()"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Performance@100\n","\tHR: 0.2760\tnDCG: 0.0656\tAUC: 0.8229\n"]}]},{"cell_type":"markdown","metadata":{"id":"EH4cOZeZBprW"},"source":["## Adversarial Attack Against The Model\n","We can attack the model with adversarial perturbation and measure the performance after the attack. Epsilon is the perturbation budget."]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"YRpVSN56By9G","executionInfo":{"status":"ok","timestamp":1633178576310,"user_tz":-330,"elapsed":1442,"user":{"displayName":"Sparsh Agarwal","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"13037694610922482904"}},"outputId":"07ca572d-ee39-4f0c-e5bf-a0550e1353a5"},"source":["epsilon = 0.5\n","print('Running the Attack with Epsilon = {0}'.format(epsilon))\n","recommender_model.execute_adversarial_attack(epsilon=epsilon)\n","print('The model has been Adversarially Perturbed.')"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Running the Attack with Epsilon = 0.5\n","The model has been Adversarially Perturbed.\n"]}]},{"cell_type":"markdown","metadata":{"id":"KqsPksbpB1yQ"},"source":["## Evaluate the Effects of the Adversarial Attack\n","We will now evaluate the performance of the attacked model."]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"P5oI0_wWB9ft","executionInfo":{"status":"ok","timestamp":1633178610580,"user_tz":-330,"elapsed":2156,"user":{"displayName":"Sparsh Agarwal","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"13037694610922482904"}},"outputId":"d8abbeb9-1aea-4c2e-c41f-41123991072d"},"source":["after_adv_hr, after_adv_ndcg, after_adv_auc = recommender_model.evaluator.evaluate()\n","\n","print('HR decreases by %.2f%%' % ((1-after_adv_hr/before_adv_hr)*100))\n","print('nDCG decreases by %.2f%%' % ((1-after_adv_ndcg/before_adv_ndcg)*100))\n","print('AUC decreases by %.2f%%' % ((1-after_adv_auc/before_adv_auc)*100))"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Performance@100\n","\tHR: 0.2200\tnDCG: 0.0514\tAUC: 0.7935\n","HR decreases by 20.29%\n","nDCG decreases by 21.58%\n","AUC decreases by 3.57%\n"]}]},{"cell_type":"markdown","metadata":{"id":"d0QyTKufB953"},"source":["## Implement The Adversarial Training/Regularization\n","We have identified the clear performance degradation of the recommender under adversarial attack. Now, we can test whether the adversarial regularization will make the model more robust."]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"4hJLvP6jCQ--","executionInfo":{"status":"ok","timestamp":1633178897616,"user_tz":-330,"elapsed":189048,"user":{"displayName":"Sparsh Agarwal","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"13037694610922482904"}},"outputId":"3d6c782b-4b98-47c0-a50c-259990856cb6"},"source":["recommender_model.adversarial_train(adversarial_epochs=1, epsilon=epsilon)"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["187.35591626167297\n","Epoch 6/6\n","188.3142421245575\n"]}]},{"cell_type":"markdown","metadata":{"id":"mst7p8IFCVji"},"source":["## Evaluated The Adversarially Defended Model before the Attack"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"N2kEcR6ZCZ2M","executionInfo":{"status":"ok","timestamp":1633178957304,"user_tz":-330,"elapsed":2236,"user":{"displayName":"Sparsh Agarwal","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"13037694610922482904"}},"outputId":"bcf622e1-55f4-4e2a-96c1-a3c51ab39084"},"source":["before_adv_hr, before_adv_ndcg, before_adv_auc = recommender_model.evaluator.evaluate()"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Performance@100\n","\tHR: 0.2920\tnDCG: 0.0686\tAUC: 0.8336\n"]}]},{"cell_type":"markdown","metadata":{"id":"nCLmbcDmCYP-"},"source":["## Adversarial Attack Against The Defended Model"]},{"cell_type":"code","metadata":{"id":"RcPRL_1aCi-Q"},"source":["recommender_model.execute_adversarial_attack(epsilon=epsilon)"],"execution_count":null,"outputs":[]},{"cell_type":"markdown","metadata":{"id":"OleMekZjCiBb"},"source":["##Evaluate the Effects of the Adversarial Attack against the Defended Model"]},{"cell_type":"code","metadata":{"colab":{"base_uri":"https://localhost:8080/"},"id":"TlxieDX-Cke3","executionInfo":{"status":"ok","timestamp":1633178967878,"user_tz":-330,"elapsed":2568,"user":{"displayName":"Sparsh Agarwal","photoUrl":"https://lh3.googleusercontent.com/a/default-user=s64","userId":"13037694610922482904"}},"outputId":"ad7b6166-566f-486b-a2f6-064da29fc946"},"source":["after_adv_hr, after_adv_ndcg, after_adv_auc = recommender_model.evaluator.evaluate()\n","\n","print('HR decreases by %.2f%%' % ((1-after_adv_hr/before_adv_hr)*100))\n","print('nDCG decreases by %.2f%%' % ((1-after_adv_ndcg/before_adv_ndcg)*100))\n","print('AUC decreases by %.2f%%' % ((1-after_adv_auc/before_adv_auc)*100))"],"execution_count":null,"outputs":[{"output_type":"stream","name":"stdout","text":["Performance@100\n","\tHR: 0.2500\tnDCG: 0.0591\tAUC: 0.8048\n","HR decreases by 14.38%\n","nDCG decreases by 13.80%\n","AUC decreases by 3.46%\n"]}]},{"cell_type":"markdown","metadata":{"id":"L-mWToBfCMq8"},"source":["## Consequences\n","At this point, we have seen that the adversarial training has been effective in reducing the effectiveness of the FGSM-based adversarial attack against the recommender model. Furthermore, we have also identified another important consequences of the adversarial regularization. If we compare the performance of the model before and after the attack we can identify that there has been a performance improvement. For this reason, several recent works have implemented this robustification technique as an additional model component to increase the accuracy power of the recommender model."]},{"cell_type":"markdown","metadata":{"id":"FVxushStCH_O"},"source":["## References\n","- Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, Lars Schmidt-Thieme: BPR: Bayesian Personalized Ranking from Implicit Feedback. UAI 2009: 452-461\n","- Xiangnan He, Zhankui He, Xiaoyu Du, Tat-Seng Chua: Adversarial Personalized Ranking for Recommendation. SIGIR 2018: 355-364\n","- https://github.com/merrafelice/HandsOn-RecSys2020"]}]}