{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Statistical Inference" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "### The Method of Comparison\n", "The basic method to mitigate confounding is to compare(at least) two groups, one that receives _treatment_ and a _control group_ that does not (or that gets a different treatment).\n", "To minimize bias, the treatment group and the control group should be as similar as possible but for the fact that\n", "one gets treatment and the other does not.\n", "\n", "If subjects _self-select_ for treatment, that generally results in bias. So does allowing the experimenter flexibility to select the groups.\n", "The best way to minimize bias, and to be able to quantify the uncertainty in the resulting inferences, is to assign subjects to treatment or control _randomly_.\n", "\n", "For human subjects, the mere fact of receiving treatment—even a treatment with no real effect—can\n", "produce changes in response. This is called _the placebo effect_.\n", "For that reason, it is important that human subjects be _blind_ to whether they are treated or not, for instance,\n", "by giving subjects in the control group a _placebo_.\n", "That makes the treatment and control groups more similar.\n", "Both groups receive something: the difference is in _what_ they\n", "receive, rather than _whether_ they receive anything.\n", "\n", "Also, subjective elements can deliberately or inadvertently enter the assessment of subjects' responses to treatment,\n", "making it important for the people assessing the responses to be _blind_ to which subjects received treatment.\n", "When neither the subjects nor the assessors know who was treated, the experiment is _double blind_.\n", "\n", "See [SticiGui: Does Treatment Have an Effect?](http://www.stat.berkeley.edu/~stark/SticiGui/Text/experiments.htm) for more discussion." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Example: Smoking and Cancer\n", "See Freedman, 2009, Chapter 1.\n", "\n", "Smokers get more heart attacks, lung cancer, and other diseases than non-smokers.\n", "Is it because they smoke?\n", "\n", "Most smokers are male, and gender matters for many of those diseases.\n", "So does age, exposure to other environmental agents such as air pollution, etc.\n", "How can we tell whether smoking is responsible for the increased morbidity and mortality?\n", "\n", "### Example: HIP trial of the early 1960s\n", "See Freedman, 2009.\n", "700,000 members of a NY health plan, including 62,000 women between age 40 and 64, who were\n", "randomly assigned to be screened for breast cancer or not.\n", "This is a controlled, randomized experiment.\n", "\n", "\n", "### Example: Snow's study of the origins of Cholera\n", "See [SticiGui](http://www.stat.berkeley.edu/~stark/SticiGui/Text/experiments.htm#cholera)\n", "This is a natural experiment, but a spectacularly good one. Indeed, it helped establish the germ theory\n", "of disease.\n", "\n", "### Example: Yule's study of the Causes of Pauperism\n", "Freedman, 2009.\n", "This is a regression model applied to data from an observational study in an attempt to make causal inferences." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The 2-sample problem \n", "\n", "Suppose we have a group of $N$ individuals who are randomized into two groups, a _treatment_ group of size $N_t$ and a _control_ group of size $N_c = N - N_t$.\n", "Label the individuals from $1$ to $N$.\n", "Let ${\\mathcal T}$ denote the labels of individuals assigned to treatment and ${\\mathcal C}$ denote \n", "the labels of those assigned to control.\n", "\n", "For each of the $N$ individuals, we measure a quantitative (real-valued) response.\n", "Each individual $i$ has two _potential responses_: the response $c_i $individual would have if assigned to \n", "the control group, and the response $t_i$ the individual would have if assigned to the treatment group.\n", "\n", "We assume that individual $i$'s response depends _only_ on that individual's assigment, and not on anyone else's assignment.\n", "This is the assumption of _non-interference_. \n", "In some cases, this assumption is reasonable; in others, it is not.\n", "\n", "For instance, imagine testing a vaccine for a communicable disease.\n", "If you and I have contact, whether you get the disease might depend on whether I am vaccinated—and _vice versa_—since if the vaccine protects me from illness, I won't infect you.\n", "Similarly, suppose we are testing the effectiveness of an advertisement for a product.\n", "If you and I are connected and you buy the product, I might be more likely to buy it, even if I don't\n", "see the advertisement.\n", "\n", "Conversely, suppose that \"treatment\" is exposure to a carcinogen, and the response whether the\n", "subject contracts cancer. \n", "On the assumption that cancer is not communicable, my exposure and your disease\n", "status have no connection.\n", "\n", "The _strong null hypothesis_ is that individual by individual, treatment makes no difference whatsoever: $c_i = t_i$ for all $i$.\n", "\n", "If so, any differences between statistics computed for the treatment and control groups are entirely due to the luck of the draw: which individuals happened to be assigned to treatment and which to control.\n", "\n", "We can find the _null distribution_ of any statistic computed from the responses of the two groups: if the strong null hypothesis is true, we know what individual $i$'s response would have been whether assigned to treatment or to control—namely, the same.\n", "\n", "For instance, suppose we suspect that treatment tends to increase response: in general, $t_i \\ge c_i$.\n", "Then we might expect $\\bar{c} = \\frac{1}{N_c} \\sum_{i \\in {\\mathcal C}} c_i$ to be less than\n", "$\\bar{t} = \\frac{1}{N_t} \\sum_{i \\in {\\mathcal T}} t_i$.\n", "How large a difference between $\\bar{c}$ and $\\bar{t}$ would be evidence that treatment increases the response,\n", "beyond what might happen by chance through the luck of the draw?\n", "\n", "This amounts to asking whether the observed difference in means between the two groups is a high percentile\n", "of the distribution of that difference in means, calculated on the assumption that the null hypothesis is true.\n", "\n", "Because of how subjects are assigned to treatment or to control, all allocations of $N_t$ subjects to\n", "treatment are equally likely.\n", "\n", "One way to partition the $N$ subjects randomly into a group of size $N_c$ and a group of size $N_t$ is\n", "to permute the $N$ subjects at random, then take the first $N_c$ in the permuted list to be the control\n", "group, and the remaining $N_t$ to be the treatment group.\n", "\n", "[Note: TO DO discussion of how to construct a random permutation. Issues with assigning random numbers to\n", "all items of the list. Compare with Knuth's algorithm, also for computational efficiency.]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Aside: Random number generation\n", "\n", "[To do.]\n", "Most computers cannot generate true random numbers (there are rare exceptions that have _hardware random\n", "number generators_).\n", "Instead, they generate _psdudo-random numbers_ using algorithms called _pseudo-random number generators_ (PRNGs).\n", "\n", "Current high-end statistics packages and programming languages (e.g., R, Python) use the Mersenne Twister\n", "PRNG.\n", "The Mersenne Twister has a very long period ($2^{19937}-1$) and passed the [DIEHARD tests](https://en.wikipedia.org/wiki/Diehard_tests) for equidistribution, etc.\n", "However, it is not adequate for cryptography (the _state space_ is so small that its future values can be predicted\n", "from a relatively small number of observations).\n", "_Linear congruential_ generators are generally not adequate for statistics.\n", "In particular, beware the algorithms in _Numerical Recipes_ and the Excel PRNG.\n", "\n", "Even the Mersenne Twister runs into trouble generating random permutations of long vectors using the\n", "naiive approach (assign a random number to each element of the vector, then sort by those numbers).\n", "The period of the Mersenne Twister is about $4 \\times 10^{6002}$. That's less than the number of\n", "permutations of 2081 objects.\n", "\n", "[To do: explain Knuth's algorithm]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Significance level and power\n", "\n", "[To do.]\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Permutation tests\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Gender Bias in Teaching Evaluations\n", "MacNell, Driscoll, and Hunt (2014. [What's in a Name: Exposing Gender Bias in Student Ratings of Teaching](http://link.springer.com/article/10.1007%2Fs10755-014-9313-4), _Innovative Higher Education_) conducted a controlled, randomized experiment on\n", "the effect of students' perception of instructors' gender on teaching evaluations\n", "in an online course.\n", "Students in the class did not know the instructors' true genders.\n", "\n", "MacNell et al. randomized 43 students in an online course into four groups: two taught by a female\n", "instructor and two by a male instructor.\n", "One of the groups taught by each instructor was led to believe the instructor was male;\n", "the other was led to believe the instructor was female.\n", "Comparable instructor biographies were given to all students.\n", "Instructors treated the groups identically, including returning assignments at the same time.\n", "\n", "When students thought the instructor was female, students rated the instructor lower, on average,\n", "in every regard.\n", "\n", "
Characteristic | F - M |
---|---|
Caring | -0.52 |
Consistent | -0.47 |
Enthusiastic | -0.57 |
Fair | -0.76 |
Feedback | -0.47 |
Helpful | -0.46 |
Knowledgeable | -0.35 |
Praise | -0.67 |
Professional | -0.61 |
Prompt | -0.80 |
Respectful | -0.61 |
Responsive | -0.22 |
group | professional | respect | caring | enthusiastic | communicate | helpful | feedback | prompt | consistent | fair | responsive | praised | knowledgeable | clear | overall | gender | age | tagender | taidgender | |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1 | 3 | 5 | 5 | 4 | 4 | 4 | 3 | 4 | 4 | 4 | 4 | 4 | 4 | 3 | 5 | 4 | 2 | 1990 | 0 | 1 |
2 | 3 | 4 | 4 | 4 | 4 | 5 | 5 | 5 | 5 | 3 | 4 | 5 | 5 | 5 | 5 | 4 | 1 | 1992 | 0 | 1 |
3 | 3 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 2 | 1991 | 0 | 1 |
4 | 3 | 5 | 5 | 5 | 5 | 5 | 3 | 5 | 5 | 5 | 5 | 3 | 5 | 5 | 5 | 5 | 2 | 1991 | 0 | 1 |
5 | 3 | 5 | 5 | 5 | 5 | 5 | 5 | 5 | 3 | 4 | 5 | 5 | 5 | 5 | 5 | 5 | 2 | 1992 | 0 | 1 |
sprintf {base} | R Documentation |
A wrapper for the C function sprintf
, that returns a character\n",
"vector containing a formatted combination of text and variable values.\n",
"
\n", "sprintf(fmt, ...)\n", "gettextf(fmt, ..., domain = NULL)\n", "\n", "\n", "\n", "
fmt | \n",
"\n",
" a character vector of format strings, each of up to 8192 bytes. \n", " |
... | \n",
"\n",
" values to be passed into |
domain | \n",
"\n",
" see |
sprintf
is a wrapper for the system sprintf
C-library\n",
"function. Attempts are made to check that the mode of the values\n",
"passed match the format supplied, and R's special values (NA
,\n",
"Inf
, -Inf
and NaN
) are handled correctly.\n",
"
gettextf
is a convenience function which provides C-style\n",
"string formatting with possible translation of the format string.\n",
"
The arguments (including fmt
) are recycled if possible a whole\n",
"number of times to the length of the longest, and then the formatting\n",
"is done in parallel. Zero-length arguments are allowed and will give\n",
"a zero-length result. All arguments are evaluated even if unused, and\n",
"hence some types (e.g., \"symbol\"
or \"language\"
, see\n",
"typeof
) are not allowed.\n",
"
The following is abstracted from Kernighan and Ritchie (see\n", "References): however the actual implementation will follow the C99\n", "standard and fine details (especially the behaviour under user error)\n", "may depend on the platform.\n", "
\n", "The string fmt
contains normal characters,\n",
"which are passed through to the output string, and also conversion\n",
"specifications which operate on the arguments provided through\n",
"...
. The allowed conversion specifications start with a\n",
"%
and end with one of the letters in the set\n",
"aAdifeEgGosxX%
. These letters denote the following types:\n",
"
d
, i
, o
, x
, X
Integer\n",
"value, o
being octal, \n",
"x
and X
being hexadecimal (using the same case for\n",
"a-f
as the code). Numeric variables with exactly integer\n",
"values will be coerced to integer. Formats d
and i
\n",
"can also be used for logical variables, which will be converted to\n",
"0
, 1
or NA
.\n",
"
f
Double precision value, in “fixed\n",
"point” decimal notation of the form "[-]mmm.ddd". The number of\n",
"decimal places ("d") is specified by the precision: the default is 6;\n",
"a precision of 0 suppresses the decimal point. Non-finite values\n",
"are converted to NA
, NaN
or (perhaps a sign followed\n",
"by) Inf
.\n",
"
e
, E
Double precision value, in\n",
"“exponential” decimal notation of the\n",
"form [-]m.ddde[+-]xx
or [-]m.dddE[+-]xx
.\n",
"
g
, G
Double precision value, in %e
or\n",
"%E
format if the exponent is less than -4 or greater than or\n",
"equal to the precision, and %f
format otherwise.\n",
"(The precision (default 6) specifies the number of\n",
"significant digits here, whereas in %f, %e
, it is\n",
"the number of digits after the decimal point.)\n",
"
a
, A
Double precision value, in binary notation\n",
"of the form [-]0xh.hhhp[+-]d
. This is a binary fraction\n",
"expressed in hex multiplied by a (decimal) power of 2. The number\n",
"of hex digits after the decimal point is specified by the precision:\n",
"the default is enough digits to represent exactly the internal\n",
"binary representation. Non-finite values are converted to NA
,\n",
"NaN
or (perhaps a sign followed by) Inf
. Format\n",
"%a
uses lower-case for x
, p
and the hex\n",
"values: format %A
uses upper-case.\n",
"
This should be supported on all platforms as it is a feature of C99.\n",
"The format is not uniquely defined: although it would be possible\n",
"to make the leading h
always zero or one, this is not\n",
"always done. Most systems will suppress trailing zeros, but a few\n",
"do not. On a well-written platform, for normal numbers there will\n",
"be a leading one before the decimal point plus (by default) 13\n",
"hexadecimal digits, hence 53 bits. The treatment of denormalized\n",
"(aka ‘subnormal’) numbers is very platform-dependent.\n",
"
s
Character string. Character NA
s are\n",
"converted to \"NA\"
.\n",
"
%
Literal %
(none of the extra formatting\n",
"characters given below are permitted in this case).\n",
"
Conversion by as.character
is used for non-character\n",
"arguments with s
and by as.double
for\n",
"non-double arguments with f, e, E, g, G
. NB: the length is\n",
"determined before conversion, so do not rely on the internal\n",
"coercion if this would change the length. The coercion is done only\n",
"once, so if length(fmt) > 1
then all elements must expect the\n",
"same types of arguments.\n",
"
In addition, between the initial %
and the terminating\n",
"conversion character there may be, in any order:\n",
"
m.n
Two numbers separated by a period, denoting the\n",
"field width (m
) and the precision (n
).
-
Left adjustment of converted argument in its field.
\n", "+
Always print number with sign: by default only\n", "negative numbers are printed with a sign.
\n", "Prefix a space if the first character is not a sign.
\n", "0
For numbers, pad to the field width with leading\n", "zeros. For characters, this zero-pads on some platforms and is\n", "ignored on others.
\n", "#
specifies “alternate output” for numbers, its\n",
"action depending on the type:\n",
"For x
or X
, 0x
or 0X
will be prefixed\n",
"to a non-zero result. For e
, e
, f
, g
\n",
"and G
, the output will always have a decimal point; for\n",
"g
and G
, trailing zeros will not be removed.\n",
"
Further, immediately after %
may come 1$
to 99$
\n",
"to refer to numbered argument: this allows arguments to be\n",
"referenced out of order and is mainly intended for translators of\n",
"error messages. If this is done it is best if all formats are\n",
"numbered: if not the unnumbered ones process the arguments in order.\n",
"See the examples. This notation allows arguments to be used more than\n",
"once, in which case they must be used as the same type (integer,\n",
"double or character).\n",
"
A field width or precision (but not both) may be indicated by an\n",
"asterisk *
: in this case an argument specifies the desired\n",
"number. A negative field width is taken as a '-' flag followed by a\n",
"positive field width. A negative precision is treated as if the\n",
"precision were omitted. The argument should be integer, but a double\n",
"argument will be coerced to integer.\n",
"
There is a limit of 8192 bytes on elements of fmt
, and on\n",
"strings included from a single %
letter conversion\n",
"specification.\n",
"
Field widths and precisions of %s
conversions are interpreted\n",
"as bytes, not characters, as described in the C standard.\n",
"
The C doubles used for R numerical vectors have signed zeros, which\n",
"sprintf
may output as -0
, -0.000
....\n",
"
A character vector of length that of the longest input. If any\n",
"element of fmt
or any character argument is declared as UTF-8,\n",
"the element of the result will be in UTF-8 and have the encoding\n",
"declared as UTF-8. Otherwise it will be in the current locale's\n",
"encoding.\n",
"
The format string is passed down the OS's sprintf
function, and\n",
"incorrect formats can cause the latter to crash the R process . R\n",
"does perform sanity checks on the format, but not all possible user\n",
"errors on all platforms have been tested, and some might be terminal.\n",
"
The behaviour on inputs not documented here is ‘undefined’,\n", "which means it is allowed to differ by platform.\n", "
\n", "\n", "\n", "Original code by Jonathan Rougier.\n", "
\n", "\n", "\n", "Kernighan, B. W. and Ritchie, D. M. (1988)\n", "The C Programming Language. Second edition, Prentice Hall.\n", "Describes the format options in table B-1 in the Appendix.\n", "
\n", "The C Standards, especially ISO/IEC 9899:1999 for ‘C99’. Links\n", "can be found at http://developer.r-project.org/Portability.html.\n", "
\n", "man sprintf
on a Unix-alike system.\n",
"
formatC
for a way of formatting vectors of numbers in a\n",
"similar fashion.\n",
"
paste
for another way of creating a vector combining\n",
"text and values.\n",
"
gettext
for the mechanisms for the automated translation\n",
"of text.\n",
"
\n", "## be careful with the format: most things in R are floats\n", "## only integer-valued reals get coerced to integer.\n", "\n", "sprintf(\"%s is %f feet tall\\n\", \"Sven\", 7.1) # OK\n", "try(sprintf(\"%s is %i feet tall\\n\", \"Sven\", 7.1)) # not OK\n", " sprintf(\"%s is %i feet tall\\n\", \"Sven\", 7 ) # OK\n", "\n", "## use a literal % :\n", "\n", "sprintf(\"%.0f%% said yes (out of a sample of size %.0f)\", 66.666, 3)\n", "\n", "## various formats of pi :\n", "\n", "sprintf(\"%f\", pi)\n", "sprintf(\"%.3f\", pi)\n", "sprintf(\"%1.0f\", pi)\n", "sprintf(\"%5.1f\", pi)\n", "sprintf(\"%05.1f\", pi)\n", "sprintf(\"%+f\", pi)\n", "sprintf(\"% f\", pi)\n", "sprintf(\"%-10f\", pi) # left justified\n", "sprintf(\"%e\", pi)\n", "sprintf(\"%E\", pi)\n", "sprintf(\"%g\", pi)\n", "sprintf(\"%g\", 1e6 * pi) # -> exponential\n", "sprintf(\"%.9g\", 1e6 * pi) # -> \"fixed\"\n", "sprintf(\"%G\", 1e-6 * pi)\n", "\n", "## no truncation:\n", "sprintf(\"%1.f\", 101)\n", "\n", "## re-use one argument three times, show difference between %x and %X\n", "xx <- sprintf(\"%1$d %1$x %1$X\", 0:15)\n", "xx <- matrix(xx, dimnames = list(rep(\"\", 16), \"%d%x%X\"))\n", "noquote(format(xx, justify = \"right\"))\n", "\n", "## More sophisticated:\n", "\n", "sprintf(\"min 10-char string '%10s'\",\n", " c(\"a\", \"ABC\", \"and an even longer one\"))\n", "\n", "## Platform-dependent bad example from qdapTools 1.0.0:\n", "## may pad with spaces or zeroes.\n", "sprintf(\"%09s\", month.name)\n", "\n", "n <- 1:18\n", "sprintf(paste0(\"e with %2d digits = %.\", n, \"g\"), n, exp(1))\n", "\n", "## Using arguments out of order\n", "sprintf(\"second %2$1.0f, first %1$5.2f, third %3$1.0f\", pi, 2, 3)\n", "\n", "## Using asterisk for width or precision\n", "sprintf(\"precision %.*f, width '%*.3f'\", 3, pi, 8, pi)\n", "\n", "## Asterisk and argument re-use, 'e' example reiterated:\n", "sprintf(\"e with %1$2d digits = %2$.*1$g\", n, exp(1))\n", "\n", "## re-cycle arguments\n", "sprintf(\"%s %d\", \"test\", 1:3)\n", "\n", "## binary output showing rounding/representation errors\n", "x <- seq(0, 1.0, 0.1); y <- c(0,.1,.2,.3,.4,.5,.6,.7,.8,.9,1)\n", "cbind(x, sprintf(\"%a\", x), sprintf(\"%a\", y))\n", "\n", "\n", "
group | grade | tagender | taidgender | |
---|---|---|---|---|
1 | 3 | 77.4 | 0 | 1 |
2 | 3 | 89.02 | 0 | 1 |
3 | 3 | 53.5 | 0 | 1 |
4 | 3 | 88.32 | 0 | 1 |
5 | 3 | 90.02 | 0 | 1 |
sd {stats} | R Documentation |
This function computes the standard deviation of the values in\n",
"x
.\n",
"If na.rm
is TRUE
then missing values are removed before\n",
"computation proceeds.\n",
"
\n", "sd(x, na.rm = FALSE)\n", "\n", "\n", "\n", "
x | \n",
"\n",
" a numeric vector or an R object which is coercible to one\n",
"by |
na.rm | \n",
"\n",
" logical. Should missing values be removed? \n", " |
Like var
this uses denominator n - 1.\n",
"
The standard deviation of a zero-length vector (after removal of\n",
"NA
s if na.rm = TRUE
) is not defined and gives an error.\n",
"The standard deviation of a length-one vector is NA
.\n",
"
var
for its square, and mad
, the most\n",
"robust alternative.\n",
"
\n", "sd(1:2) ^ 2\n", "\n", "\n", "