---
title: "String Manipulation With stringr"
subtitle: Biostat 203B
author: "Dr. Hua Zhou @ UCLA"
date: today
format:
  html:
    theme: cosmo
    embed-resources: true
    number-sections: true
    toc: true
    toc-depth: 4
    toc-location: left
    code-fold: false
knitr:
  opts_chunk: 
    fig.align: 'center'
    fig.width: 6
    fig.height: 4
    message: FALSE
    cache: false
---

Display machine information for reproducibility.
```{r}
sessionInfo()
```

# String | r4ds chapter 14

We encounter text data often: sentiment analysis, electronic health records (doctor notes), twitter data, websites, ...

## stringr

- stringr package, by Hadley Wickham et al, provides utilities for handling strings.

- Included in tidyverse.
```{r}
library("tidyverse")
```
```{r}
# load htmlwidgets
library(htmlwidgets)
```

- Main functions in stringr:    
    - `str_detect(string, pattern)`: Detect the presence or absence of a pattern in a string.  
    - `str_locate(string, pattern)`: Locate the first position of a pattern and return a matrix with start and end.  
    - `str_extract(string, pattern)`: Extract text corresponding to the first match.  
    - `str_match(string, pattern)`: Extract matched groups from a string.  
    - `str_split(string, pattern)`: Split string into pieces and returns a list of character vectors.  
    - `str_replace(string, pattern, replacement)`: Replace the first matched pattern and returns a character vector.
    
- Variants with an `_all` suffix will match more than 1 occurrence of the pattern in a given string.  
- Most functions are vectorized.

## Basics

- Strings are enclosed by double quotes or single quotes:
```{r}
string1 <- "This is a string"
string2 <- 'If I want to include a "quote" inside a string, I use single quotes'
```

- Literal single or double quote:
```{r}
double_quote <- "\"" # or '"'
single_quote <- '\'' # or "'"
```

----

- Printed representation:
```{r}
x <- c("\"", "\\")
x
```
vs `cat` (how R views your string after all special characters have been parsed):    
```{r}
cat(x)
```
vs `writeLines()` (how R views your string after all special characters have been parsed):
```{r}
writeLines(x)
```

----    

- Other special characters: `"\n"` (new line), `"\t"` (tab), ... Check
```{r}
#| eval: false
?"'"
```
for a complete list.
```{r}
cat("a\"b")
cat("a\tb")
cat("a\nb")
```

- Unicode
```{r}
x <- "\u00b5"
x
```

- Character vector (vector of strings):
```{r}
c("one", "two", "three")
```

## String length

- Length of a single string:
```{r}
str_length("R for data science")
```

- Lengths of a character vector:
```{r}
str_length(c("a", "R for data science", NA))
```

## Combining strings

- Combine two or more strings
```{r}
str_c("x", "y")
str_c("x", "y", "z")
```

- Separator:
```{r}
str_c("x", "y", sep = ", ")
```

----

- `str_c()` is vectorised:
```{r}
str_c("prefix-", c("a", "b", "c"), "-suffix")
```

- Objects of length 0 are silently dropped:
```{r}
name <- "Hadley"
time_of_day <- "morning"
birthday <- FALSE

str_c(
  "Good ", 
  time_of_day, 
  " ", 
  name,
  if (birthday) " and HAPPY BIRTHDAY",
  "."
)
```

----

- Combine a vector of strings:
```{r}
# this is not collapsing
str_c(c("x", "y", "z"))
# collapse
str_c(c("x", "y", "z"), collapse = ", ")
# equivalently
str_flatten(c("x", "y", "z"), collapse = ", ")
```

## Subsetting strings

- By position:
```{r}
str_sub("Apple", 1, 3)
# vectorized
x <- c("Apple", "Banana", "Pear")
str_sub(x, 1, 3)
```

- Negative numbers count backwards from end:
```{r}
str_sub(x, -3, -1)
```

----

- Out of range:
```{r}
str_sub("a", 1, 5)
str_sub("a", 2, 5)
```

- Assignment to a substring:
```{r}
str_sub(x, 1, 1) <- str_to_lower(str_sub(x, 1, 1))
x
```

## Matching patterns with regular expressions

- Match exact strings:
```{r}
x <- c("apple", "banana", "pear")
str_view(x, "an")
```

- `.` matches any character apart from a newline:
```{r}
str_view(x, ".a.")
```

- **Regex escapes live on top of regular string escapes, so there needs to be two levels of escapes.**  
    To match a literal `.`:
```{r}
#| error: true
# doesn't work because "a\.c" is treated as a regular expression
str_view(c("abc", "a.c", "bef"), "a\.c")
```
```{r}
# regular expression needs double escape
str_view(c("abc", "a.c", "bef"), "a\\.c")
```
To match a literal `\`:
```{r}
str_view("a\\b", "\\\\")
```

<p align="center">
<img src="./escape.png" height="125">
</p>

## Anchors

- `^` matches the start of the string:
```{r}
x <- c("apple", "banana", "pear")
str_view(x, "^a")
```

- `$` matches the end of the string:
```{r}
str_view(x, "a$")
```

- To force a regular expression to only match a complete string:
```{r}
x <- c("apple pie", "apple", "apple cake")
str_view(x, "^apple$")
```

## Alternation  

- `[abc]`: matches a, b, or c. Same as `(a|b|c)`.
```{r}
str_view(c("grey", "gray"), "gr(e|a)y")
```
```{r}
str_view(c("grey", "gray"), "gr[ea]y")
```

- `[^abc]`: matches anything except a, b, or c.

- `[a-z]`: matches any lower case letter.

## Repetition

- 
`?`: 0 or 1  
`+`: 1 or more  
`*`: 0 or more

- Examples:
```{r}
x <- "1888 is the longest year in Roman numerals: MDCCCLXXXVIII"
# match either C or CC, being greedy here
str_view(x, "CC?")
# greedy matches
str_view(x, "CC+")
# greedy matches
str_view(x, 'C[LX]+')
```

- Specify number of matches:  
`{n}`: exactly n   
`{n,}`: n or more  
`{,m}`: at most m  
`{n,m}`: between n and m

- Examples:
```{r}
str_view(x, "C{2}")
# greedy matches
str_view(x, "C{2,}")
# greedy matches
str_view(x, "C{2,3}")
```

- Greedy (default) vs lazy (put `?` after repetition):
```{r}
# lazy matches
str_view(x, 'C{2,3}?')
# lazy matches
str_view(x, 'C[LX]+?')
```

- What went wrong here?
```{r}
text = "<div class='main'> <div> <a href='here.pdf'>Here!</a> </div> </div>"
str_extract(text, "<div>.*</div>")
```
If we add ? after a quantifier, the matching will be lazy (find the shortest possible match, not the longest).
```{r}
str_extract(text, "<div>.*?</div>")
```

## Grouping and backreference

<!-- - Group together parts of a regular expression for modification or capture. -->
<!--     - `(a|b)`: match literal a or b, group either   -->
<!--     - `a(bc)`: match literal a or abc, group bc or ""   -->
<!--     - `(?:abc)`: non-capturing group   -->
<!--     - `(abc)def(hig)`: match abcdefhig, group abc and hig -->

<!-- - Example -->
<!--     ```{r} -->
<!--     text = c("Bob Smith", "Alice Smith", "Apple") -->
<!--     str_extract(text, "^[:alpha:]+") -->
<!--     str_match(text, "^([:alpha:]+) [:alpha:]+") -->
<!--     str_match(text, "^([:alpha:]+) ([:alpha:]+)") -->
<!--     ``` -->

- `fruit` is a character vector pre-defined in `stringr` package:
```{r}
fruit
```

- Parentheses define groups, which can be back-referenced as `\1`, `\2`, ...
```{r}
# with match = TRUE, only show matched strings
str_view(fruit, "(..)\\1")
```

## Detect matches

- `str_detect` return Booleans.
```{r}
x <- c("apple", "banana", "pear")
str_detect(x, "e")
```

- Vector `words` contains 980 commonly used words:
```{r}
length(words)
head(words)
```

- Examples:
```{r}
# How many common words start with t?
sum(str_detect(words, "^t"))
# What proportion of common words end with a vowel?
mean(str_detect(words, "[aeiou]$"))
```

- Find words that end with `x`:
```{r}
words[str_detect(words, "x$")]
```
same as
```{r}
str_subset(words, "x$")
```

- Filter a data frame:
```{r}
df <- tibble(
  word = words, 
  i = seq_along(word)
)
df %>% 
  filter(str_detect(word, "x$"))
```

- `str_count()` tells how many matches are found:
```{r}
x <- c("apple", "banana", "pear")
str_count(x, "a")
```
```{r}
# On average, how many vowels per word?
mean(str_count(words, "[aeiou]"))
```

- Matches never overlap:
```{r}
str_count("abababa", "aba")
str_view("abababa", "aba")
```

- Mutate a data frame:
```{r}
df %>% 
  mutate(
    vowels = str_count(word, "[aeiou]"),
    consonants = str_count(word, "[^aeiou]")
  )
```

## Extract matches

- `sentences` is a collection of 720 phrases:
```{r}
length(sentences)
head(sentences)
```

- Suppose we want to find all sentences that contain a colour.

- Create a collection of colours:
```{r}
(colours <- c("red", "orange", "yellow", "green", "blue", "purple"))
```

```{r}
(colour_match <- str_c(colours, collapse = "|"))
```

- Select the sentences that contain a colour, and then extract the colour to figure out which one it is:    
```{r}
(has_colour <- str_subset(sentences, colour_match))
```

```{r}
(matches <- str_extract(has_colour, colour_match))
```

- `str_extract()` only extracts the first match.
```{r}
more <- sentences[str_count(sentences, colour_match) > 1]
str_view(more, colour_match)
```

- `str_extract_all()` extracts all matches:
```{r}
str_extract_all(more, colour_match)
```

- Setting `simplify = TRUE` in `str_extract_all()` will return a matrix with short matches expanded to the same length as the longest:
```{r}
str_extract_all(more, colour_match, simplify = TRUE)
```
```{r}
x <- c("a", "a b", "a b c")
str_extract_all(x, "[a-z]", simplify = TRUE)
```

## Grouped matches

- `str_extract()` gives us the complete match:
```{r}
# why "([^ ]+)" match a word?
noun <- "(a|the) ([^ ]+)"

has_noun <- sentences %>%
  str_subset(noun) %>%
  head(10)
has_noun %>%
  str_extract(noun)
```

- `str_match()` gives each individual component:
```{r}
has_noun %>%
  str_match(noun)
```

- `tidyr::extract()` works with tibble:
```{r}
tibble(sentence = sentences) %>%
  tidyr::extract(
    sentence, c("article", "noun"), "(a|the) ([^ ]+)",
    remove = FALSE
  )
```

## Replacing matches

- Replace the first match:
```{r}
x <- c("apple", "pear", "banana")
str_replace(x, "[aeiou]", "-")
```

- Replace all matches:
```{r}
str_replace_all(x, "[aeiou]", "-")
```

- Multiple replacement:
```{r}
x <- c("1 house", "2 cars", "3 people")
str_replace_all(x, c("1" = "one", "2" = "two", "3" = "three"))
```

- Back-reference:
```{r}
# flip the order of the second and third words
sentences %>% 
  str_replace("([^ ]+) ([^ ]+) ([^ ]+)", "\\1 \\3 \\2") %>% 
  head(5)
```

## Splitting

- Split a string into pieces:
```{r}
sentences %>%
  head(5) %>% 
  str_split(" ")
```
    
- Use `simplify = TRUE` to return a matrix:    
```{r}
sentences %>%
  head(5) %>% 
  str_split(" ", simplify = TRUE)
```

# More regular expressions

## Character classes

<p align="center">
<img src="./regex_characterclass.png" height="250">
</p>

- A number of built-in convenience classes of characters:
    - `.`: Any character except new line `\n`.
    - `\s`: White space.  
    - `\S`: Not white space.  
    - `\d`: Digit (0-9). 
    - `\D`: Not digit.
    - `\w`: Word (A-Z, a-z, 0-9, or _).
    - `\W`: Not word.
    
- Example: How to match a telephone number with the form (###) ###-####?
```{r}
#| error: true
# This produces error. Why?
text = c("apple", "(219) 733-8965", "(329) 293-8753")
str_detect(text, "(\d\d\d) \d\d\d-\d\d\d\d")
```
```{r}
# This doesn't give the answer we want. Why?
text = c("apple", "(219) 733-8965", "(329) 293-8753")
str_detect(text, "(\\d\\d\\d) \\d\\d\\d-\\d\\d\\d\\d")
# This should yield the desired answer
str_detect(text, "\\(\\d\\d\\d\\) \\d\\d\\d-\\d\\d\\d\\d")
```

## Lists and ranges

- Lists and ranges:
    - `[abc]`: List (a or b or c)  
    - `[^abc]`: Excluded list (not a or b or c)  
    - `[a-q]`: Range lower case letter from a to q  
    - `[A-Q]`: Range upper case letter from A to Q  
    - `[0-7]`: Digit from 0 to 7  

```{r}
text = c("apple", "(219) 733-8965", "(329) 293-8753")
str_replace_all(text, "[aeiou]", "") # strip all vowels
str_replace_all(text, "[13579]", "*")
str_replace_all(text, "[1-5a-ep]", "*")
```

# Teaser: how to validate email address using RegEx

<https://r4ds.had.co.nz/strings.html#tools>

# Cheat sheet  

[Posit cheat sheet](https://rstudio.github.io/cheatsheets/html/strings.html) is extremely helpful.
