---
title: "Exercise template for 'Regression fundamentals'"
author: "Your name here"
format:
  html: default
  pdf: default
---

```{r}
#| message: false
#| include: false
#| warning: false
library(dplyr, warn.conflicts = FALSE)
library(ggplot2)
library(farr)
library(modelsummary)
library(tidyr)         # pivot_wider()
library(kableExtra)    # kbl()
library(fixest)        # feols()
```

```{r}
#| include: false
test_scores <- test_scores |> mutate(grade = as.integer(grade))

camp_scores <-
  test_scores |>
  inner_join(camp_attendance, by = "id") |>
  rename(treat = camp) |>
  mutate(post = grade >= 7)
```

```{r}
#| include: false
test_summ <-
  camp_scores |>
  filter(grade %in% 6:7L) |>
  group_by(post, treat) |>
  summarize(score = mean(score), .groups = "drop") |>
  pivot_wider(names_from = post, values_from = score) |>
  rename(post = `TRUE`, pre = `FALSE`) |>
  mutate(change = post - pre)
```

```{r}
#| include: false
mean_by_grade <-
  test_scores |>
  group_by(grade) |>
  summarize(score = mean(score),
            .groups = "drop") 
```

```{r}
#| include: false
fm_grade <- lm(score ~ factor(grade), data = test_scores)

fm_id <- lm(score ~ treat * post + factor(grade) + factor(id),
            data = camp_scores, x = TRUE)
```

```{r}
#| include: false
demean <- function(x) x - mean(x)

camp_scores_demean <-
  camp_scores |>
  group_by(id) |>
  mutate(score = demean(score)) |>
  group_by(grade) |>
  mutate(score = demean(score)) 
  
fm_demean <- lm(score ~ treat * post, 
                data = camp_scores_demean, x = TRUE)
```

## Running regressions in R

### Exercises

1. In using `pivot_wider()` in Chapter 2, we supplied a value to the `id_cols` argument, but we omitted that in creating `test_summ`.
If we wanted to be explicit, what value would we need to provide for that argument in the code creating `test_summ`?

2. What is the relation between the means in the following table and the regression coefficients in `fm_grade`?

```{r}
#| echo: false
mean_by_grade |> kbl(booktabs = TRUE, digits = 3)
```

3. Why is there no estimated coefficient for `factor(grade)5` in `fm_grade`?

4. Now let's return to our earlier regression specification, except this time we include fixed effects for `grade` (see code and output below).
We now have *two* fixed effects omitted: `factor(grade)5` and `factor(grade)8`.
Why are we now losing two fixed effects, while above we lost just one?
(*Hint*: Which variables can be expressed as linear combinations of the `grade` indicators?)

```{r}
#| label: fm_dd_fe
fm_dd_fe <- lm(score ~ treat * post + factor(grade),
               data = camp_scores)
```

```{r}
#| echo: false
tibble(term = names(coef(fm_dd_fe)), estimate = coef(fm_dd_fe)) |>
  kbl(booktabs = TRUE, digits = 4)
```

5.  In words, what are we doing to create `camp_scores_demean`?
Intuitively, why might this affect the need to use fixed effects?

6. Can you relate the coefficients from the regression stored in `fm_demean` to the numbers in the following table?
Which of these estimated coefficients is meaningful? 
All of them? Some of them? None of them?

```{r}
#| echo: false
camp_scores_demean |>
  group_by(grade, treat) |> 
  summarize(score = mean(score), .groups = "drop") |>
  kbl(booktabs = TRUE, digits = 4)
```

7.  The `feols()` function from the `fixest` package offers a succinct syntax for adding fixed effects and uses computationally efficient algorithms (much like our demeaning approach above) in estimating these. \indexc{feols()}
What is the same in the results below and the two specifications we estimated above? 
What is different?
Why might these differences exist?
What is the `I()` function doing here?
What happens if we omit it (i.e., just include `post * treat`)?

```{r}
fefm <- feols(score ~ I(post * treat) | grade + id, data = camp_scores)
coefficients(fefm)
```

## Frisch-Waugh-Lovell theorem {#fwl}

```{r}
#| include: false
fm <- lm(ta ~ big_n + cfo + size + lev + mtb + 
           factor(fyear) * (inv_at + I(d_sale - d_ar) + ppe),
         data = comp, na.action = na.exclude)
```

```{r}
#| include: false
fm_aux_ta <- lm(ta ~ big_n + size + lev + mtb +
                  factor(fyear) * (inv_at + I(d_sale - d_ar) + ppe),
                data = comp, na.action = na.exclude)

fm_aux_cfo <- lm(cfo ~ big_n + size + lev + mtb +
                   factor(fyear) * (inv_at + I(d_sale - d_ar) + ppe),
                 data = comp, na.action = na.exclude)

aux_data <- tibble(ta = resid(fm_aux_ta),
                   cfo = resid(fm_aux_cfo))
fm_aux <- lm(ta ~ cfo, data = aux_data)
```

### Exercises

1. Verify the Frisch-Waugh-Lovell theorem using `big_n` and `lev` in place of `cfo` and produce plots like [the book's FWL plot](https://iangow.github.io/far_book/reg-basics.html#fig-aux-all) for each variable.
Does the plot with `big_n` as the independent variable seem less helpful?

2. Above we said that the standard errors of the main regression and the auxiliary regression using the Frisch-Waugh-Lovell theorem should be "very similar".
Confirm that the standard errors are similar across the variants of `fm` and `fm_aux` that you calculated for the previous question.
(*Hint*: For the `big_n` variant, compare `summary(fm_aux)$coefficients["big_n", ]` with `summary(fm)$coefficients["big_nTRUE", ]`.)
Can you guess what might explain any differences?
(*Hint*: Compare `fm$df.residual` and `fm_aux$df.residual` and perhaps use `sqrt()`.)

3. In words, what effect does converting `fyear` into a factor and interacting it with `inv_at`, `I(d_sale - d_ar)` and `ppe` have?
(*Hint:* It may be helpful to visually inspect the more complete regression output produced without `coef_omit = "(fyear|ppe|inv_at|d_sale)"`.)