# Training Factorization Machine with FTRL on Spark on Angel

> FM(Factorization Machine) is an algorithm based on matrix decomposition which can predict any real-valued vector.

> Its main advantages include: 

- can handle highly sparse data; 
- linear computational complexity

> FTRL (Follow-the-regularized-leader) is an optimization algorithm which is widely deployed by online learning. Employing FTRL is easy in Spark-on-Angel and you can train a model with billions, even ten billions, dimensions once you have enough machines.

Here, we will use FTRL Optimizer to update the parameters of FM.

If you are not familiar with how to programming on Spark-on-Angel, please first refer to [Programming Guide for Spark-on-Angel](https://github.com/Angel-ML/angel/blob/master/docs/programmers_guide/spark_on_angel_programing_guide_en.md);

## Factorization Model

![model](http://latex.codecogs.com/png.latex?\dpi{150}\hat{y}(x)=b+\sum_{i=1}^n{w_ix_i}+\sum_{i=1}^{n-1}\sum_{j=i+1}^n<v_i,v_j>x_ix_j)

where ![](http://latex.codecogs.com/png.latex?\dpi{100}\inline%20<v_i,v_j>) is the dot of two k-dimension vector:

![dot](http://latex.codecogs.com/png.latex?\dpi{150}\inline%20<v_i,v_j>=\sum_{i=1}^kv_{i,f}\cdot%20v_{j,f})

model parameters:
![parameter](http://latex.codecogs.com/png.latex?\dpi{100}\inlinew_0\in%20w\in%20R^n,V\in%20R^{n\times%20k})
, where n is the number of feature, ![](http://latex.codecogs.com/png.latex?\dpi{100}\inline%20v_i) represents feature i composed by k factors, k is a hyperparameter that determines the factorization.


## Using the FTRL-FM

```scala

import com.tencent.angel.ml.matrix.RowType
import com.tencent.angel.spark.ml.online_learning.FtrlFM

// allocate a ftrl optimizer with (lambda1, lambda2, alpha, beta)
val optim = new FtrlFM(lambda1, lambda2, alpha, beta)
// initializing the model
optim.init(dim, factor)
```

There are four hyper-parameters for the FTRL optimizer, which are lambda1, lambda2, alpha and beta. We allocate a FTRL optimizer with these four hyper-parameters. The next step is to initialized a FtrlFM model. There are two matrixs for FtrlFM, including `first` and `second`, the `first` contains the z, n and w in which z and n are used to init or update parameter w in FM, the `second` contains the z, n and v in which z and n are used to init or update parameter v in FM. In the aboving code, we allocate `first` a sparse distributed matrix with 3 rows and dim columns, and allocate `second` a sparse distributed matrix with 3 * factor rows and dim columns.

### set the dimension
In the scenaro of online learning, the index of features can be range from (int.min, int.max), which is usually generated by a hash function. In Spark-on-Angel, you can set the dim=-1 when your feature index range from (int.min, int.max) and rowType is sparse. If the feature index range from [0, n), you can set the dim=n.


## Training with Spark

### loading data
Using the interface of RDD to load data and parse them to vectors.
```scala
val data = sc.textFile(input).repartition(partNum)
      .map(s => (DataLoader.parseIntFloat(s, dim), DataLoader.parseLabel(s, false)))
      .map {
        f =>
          f._1.setY(f._2)
          f._1
      }
```
### training model
```scala
val size = data.count()
for (epoch <- 1 to numEpoch) {
    val totalLoss = data.mapPartitions {
        case iterator =>
        // for each partition
          val loss = iterator
            .sliding(batchSize, batchSize)
            .zipWithIndex
            .map(f => optim.optimize(f._2, f_1.toArray)).sum
          Iterator.single(loss)
    }.sum()
    println(s"epoch=$epoch loss=${totalLoss / size}")
}
```


### saving model
```scala
output = "hdfs://xxx"
optim.weight
optim.save(output + "/back")
optim.saveWeight(output)
```

The example code can be find at https://github.com/Angel-ML/angel/blob/master/spark-on-angel/examples/src/main/scala/com/tencent/angel/spark/examples/cluster/FtrlFMExample.scala