This page was last updated on September 27, 2019.
In this tutorial we will provide one example of how simulated data can be used to test hypotheses.
This tutorial also illustrates how to approach hypothesis testing, and how to prepare your answer to questions that involve hypothesis tests.
tigerstats
packagelibrary(tigerstats, warn.conflicts = FALSE, quietly = TRUE)
When defending territories along streams, males of the damselfly species Calopteryx maculata (pictured here) often battle intruders in the air, flying around and around in circles with their foe for minutes at a time. No one knows whether these battles are flown in a consistent direction, i.e. predominantly clockwise or counter-clockwise, as would be expected if the damselflies exhibited a form of “handedness”, like many animals do.
A researcher was curious about this because he had worked with these damselflies for years and witnessed many territorial bouts (see exmaple research here).
The researcher conducted a study (fictional) in which he video-recorded 20 male damselflies defending territories (all randomly sampled from a population), and determined the predominant direction of flight during circular flight battles. One battle per damselfly was recorded, and each battle was known to involve a unique combattant.
He found that in 17 out of 20 bouts the damselflies flew in the counter-clockwise direction.
Should this result be considered evidence of handedness in this population?
For the damselfly example, we have a random trial involving 20 independent battles. We’ll arbitrarily consider a counter-clockwise battle as a success, and we’ll define the test statistic as the number of “successes”" in the random trial.
If there is no preference in flight direction, then we would expect half (10 out of 20) the battles to be flown in the counter-clockwise direction, which equals a proportion of 0.5. Thus:
H0: The proportion of bouts flown in a counter-clockwise direction is 0.5.
HA: The proportion of bouts flown in a counter-clockwise direction is not 0.5.
We’ll set \(\alpha\) = 0.05.
As indicated by the HA statement, we’ll use a 2-tailed alternative hypothesis because we have no reason to exclude the possibility that clockwise battles are in fact more predominant.
For our damselfly example, we can easily simulate the random trial (20 territorial flight battles, each yielding one of 2 outcomes) by recognizing that it is analagous to flipping a coin 20 times, and tallying the number of times we get “heads” (success).
The tigerstats
package includes a function called rflip
that simulates flipping a coin. Look at the help file associated with this function:
?rflip
We can generate an appropriate null distribution by repeating this random trial many thousands of times (at least 10000) using conditions that reflect a true null hypothesis (i.e. in which the probability of getting a “heads” is indeed 0.5), and each time recording the value of the test statistic (i.e. the number of times we get “heads”).
We’ll use the code we learned in a previous tutorial, but here we explicitly define a number.of.trials
variable (stored as an integer object) so that we need only change this one line of code if in future we wish to use a different number of trials:
set.seed(144)
number.of.trials <- 10000
fly.battles <- do(number.of.trials) * rflip(20, prob = 0.5)
head(fly.battles)
## n heads tails prop
## 1 20 8 12 0.40
## 2 20 10 10 0.50
## 3 20 11 9 0.55
## 4 20 10 10 0.50
## 5 20 10 10 0.50
## 6 20 8 12 0.40
Now let’s visualize the null distribution using the barplot
function we learned about in the preceding tutorial “Simulate random trials”:
First we need to tabulate the results, and then calculate the relative frequencies:
fly.battles.table <- xtabs(~ heads, data = fly.battles) # tabulate results
fly.battles.table.rel <- fly.battles.table / number.of.trials # calculate relative frequencies
fly.battles.table.rel
## heads
## 1 2 3 4 5 6 7 8 9 10
## 0.0001 0.0002 0.0012 0.0052 0.0169 0.0350 0.0746 0.1218 0.1562 0.1739
## 11 12 13 14 15 16 17 18
## 0.1648 0.1197 0.0737 0.0361 0.0146 0.0044 0.0015 0.0001
And visualize them with a bar chart:
barplot(fly.battles.table.rel,
las = 1,
ylab = "Proportion of 10000 random trials",
ylim = c(0, 0.2),
xlim = c(0, 20))
Figure 2: Null distribution for 10000 random trials of 20 circular flight battles, tallying the number of ‘counter-clockwise’ battles in each
NOTE: This bar chart is not showing all of the potential outcomes (0 through 20) because in our simulation we did not actually observe any of the extreme outcomes 0, 19, and 20.
The null distribution above shows us that if the null hypothesis was true, the most probable outcome would be 10 counter-clockwise battles out of 20. But other outcomes are of course possible, with decreasing probability towards zero and twenty.
The P-value is defined as the probability of observing an outcome as extreme or more extreme as the observed value of the test statistic, if the null hypothesis were true.
In our damselfly study, we observed 17 out of 20 battles to be in the counter-clockwise direction.
We can calculate a P-value as follows:
The code for completing these tasks is below.
NOTE: In future tutorials you’ll learn how to calculate P-values using built-in R functions and theoretical probability distributions.
In the Starter R Tutorials you learned how to use logical comparison operators such as >
and <=
. We’ll use these here.
We can take advantage of the fact that the logical value TRUE
can be treated by R as the value 1.
Let’s find out how many values within the variable heads
in our fly.battles
data frame are equal to or greater than 17:
## [1] 16
So a total of 16 test statistic values out of the 10000 in the null distribution were as extreme as our observed test statistic value of 17.
Let’s calculate the P-value now:
## [1] 0.0032
So the P-value associated with our observed test statistic is 0.0032.
It is important to write a concluding statement that talks about the actual findings of the study. For example:
In their territorial bouts, the damselflies flew in a counter-clockwise direction significantly more than expected under a true null hypothesis (17 counter-clockwise flights out of n = 20 trials; P-value = 0.0032). We therefore conclude that there is evidence of “handedness” in this population of C. maculata.
It is also crucial that you report the sample size (here, number of random trials), the value of the observed test statistic (here 17), and the associated P-value.
When we learn new types of statistical test, we’ll adjust our concluding statements accordingly.
Getting started:
library
Data frame structure:
head
str
Tabulation:
xtabs
Simulation:
set.seed
sample
do
rflip
(from the mosaic
package in the tigerstats
package)Math:
sum
Graphs:
barplot
(different from the barchartGC
function used in previous tutorials)