---
title: "IMDB - LSTM"
author: "Dr. Hua Zhou @ UCLA"
date: "3/7/2019"
output: 
  html_document:
    toc: true
    toc_depth: 4
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```
```{r}
sessionInfo()
```

Source: <https://tensorflow.rstudio.com/keras/articles/examples/imdb_lstm.html>

## Prepare data

From documentation:  

> Dataset of 25,000 movies reviews from IMDB, labeled by sentiment (positive/negative). Reviews have been preprocessed, and each review is encoded as a sequence of word indexes (integers). For convenience, words are indexed by overall frequency in the dataset, so that for instance the integer "3" encodes the 3rd most frequent word in the data. This allows for quick filtering operations such as: "only consider the top 10,000 most common words, but eliminate the top 20 most common words".

Retrieve IMDB data:
```{r}
library(keras)

max_features <- 20000
batch_size <- 32

# Cut texts after this number of words (among top max_features most common words)
maxlen <- 80  

cat('Loading data...\n')
imdb <- dataset_imdb(num_words = max_features)
imdb$train$x[[1]]
imdb$train$y[[1]]
```

```{r}
x_train <- imdb$train$x
y_train <- imdb$train$y
x_test <- imdb$test$x
y_test <- imdb$test$y

cat(length(x_train), 'train sequences\n')
cat(length(x_test), 'test sequences\n')

cat('Pad sequences (samples x time)\n')
x_train <- pad_sequences(x_train, maxlen = maxlen)
x_test <- pad_sequences(x_test, maxlen = maxlen)
cat('x_train shape:', dim(x_train), '\n')
cat('x_test shape:', dim(x_test), '\n')
```

## Build model

```{r}
cat('Build model...\n')
model <- keras_model_sequential()
model %>%
  layer_embedding(input_dim = max_features, output_dim = 128) %>% 
  layer_lstm(units = 64, dropout = 0.2, recurrent_dropout = 0.2) %>% 
  layer_dense(units = 1, activation = 'sigmoid')

# Try using different optimizers and different optimizer configs
model %>% compile(
  loss = 'binary_crossentropy',
  optimizer = 'adam',
  metrics = c('accuracy')
)
summary(model)
```

## Training

```{r}
cat('Train...\n')
system.time({
model %>% fit(
  x_train, y_train,
  batch_size = batch_size,
  epochs = 1,
  validation_data = list(x_test, y_test)
)
})
```

## Testing

```{r}
scores <- model %>% evaluate(
  x_test, y_test,
  batch_size = batch_size
)
```

```{r}
cat('Test score:', scores[[1]])
cat('Test accuracy', scores[[2]])
```