Simple linear regression

Simple linear regression is performed between two variables, of the form:

y = Ax + e

Where y is the metric you want to predict (dependent variable), x the metric you want to use to predict y (regressors/predictors), A being the regression coefficient of variable x and e being the bit not predicted, sometimes called noise.

This you may also recognise as the equation for a straight line on a x-y plot, where A is the slope and e is the intercept, and that is pretty much what linear regression is, where the technique of least-squares is used to fit a line to some points.

An example applied to digital marketing would be predicting transactions from sessions. The regression coefficient in this case would represent a derived conversion rate.

library(knitr) # nice tables
sc_ga_data <- read.csv("./data/search_console_plus_ga.csv", stringsAsFactors = FALSE)

kable(head(sc_ga_data))

model <- lm(transactions ~ impressions, data = sc_ga_data)

library(broom) ## nice tidy model results

## overall model statistics
glance(model)

##    r.squared adj.r.squared    sigma statistic    p.value df    logLik
## 1 0.04580986    0.03496679 18.41266  4.224805 0.04280121  2 -388.8667
##        AIC      BIC deviance df.residual
## 1 783.7333 791.2328  29834.3          88

## coefficients
tidy(model)

##          term     estimate    std.error statistic      p.value
## 1 (Intercept) 6.333054e+01 5.7041878936 11.102463 2.027362e-18
## 2 impressions 3.905562e-04 0.0001900116  2.055433 4.280121e-02

## put the linear regression line (not accouting for errors)
plot(sc_ga_data$impressions, sc_ga_data$transactions)
abline(reg = coef(model))

Multi-linear regression

Multi-linear regression is the same as simple linear regression, but you have more predictors - the below shows three (x, y, ø) with corresponding regression coefficients (A, B and C)

y = Ax + Bz + Cø + e

Plotting a multi-linear regression is harder for more than 3 variables, as you need a multi-dimensional space, but its exactly the same principle - in this case the coefficients A, B and C show how much y would change if their respective regressor changed by 1 (and everything else stayed the same)

In R, you create multi-linear regression via the lm() function again, but you just add more variables to the model formula:

model2 <- lm(transactions ~ impressions + sessions + position, data = sc_ga_data)
## overall model statistics
glance(model2)

##   r.squared adj.r.squared    sigma statistic     p.value df    logLik
## 1 0.1642764     0.1351233 17.43101  5.634947 0.001421253  4 -382.9012
##        AIC      BIC deviance df.residual
## 1 775.8025 788.3015 26130.25          86

## coefficients
tidy(model2)

##          term      estimate    std.error  statistic    p.value
## 1 (Intercept) 69.8256229916 2.997209e+01  2.3296885 0.02216593
## 2 impressions -0.0001550714 2.905029e-04 -0.5338034 0.59485465
## 3    sessions  0.0394295484 1.331140e-02  2.9620894 0.00395002
## 4    position -1.3288129470 1.161496e+00 -1.1440533 0.25577588

The r-squared value has improved a little so we can explain 16% of transactions from our model.

How do you think the coefficients of multiple linear regression applied to all Google Analytics sessions compare to the actual conversion rate registered in the E-commerce reports? What about the multi-touch reports?

Logistic regression

The above regressions are all for continuous data - metrics that can take any value. They don’t work when the thing you want to predict is more categorical e.g. TRUE or FALSE. An example of this could be predicting if a customer is going to transact in a web session or not.

In R, you create logistic regression by using the glm() function and passing in a family argument. In this case we choose binomial (another name for logistic), but the argument takes many other names too for different types of regression - see ?family for details.

model <- glm(sale ~ saw.promotion + existing.customer + hour.of.day, 
             data = webdata, 
             family = binomial)