{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Probability, Random Vectors, and the Multivariate Normal Distribution" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Probability\n", "\n", "This course assumes that you have already had an introduction to probability, random variables, expectation, and so on, so the treatment is very cursory.\n", "The treatment is quite informal, omitting important technical concepts such as sigma-algebras,\n", "measurability, and the like.\n", "\n", "Modern probability theory starts with Kolmogorov's Axioms; here is an informal startement of the axioms.\n", "For more (but still a very informal treatment), see these chapters of SticiGui: \n", "[Probability: Philosophy and Mathematical Background](http://www.stat.berkeley.edu/~stark/SticiGui/Text/probabilityPhilosophy.htm),\n", "[Set theory](http://www.stat.berkeley.edu/~stark/SticiGui/Text/sets.htm),\n", "and\n", "[Probability: Axioms and Fundaments](http://www.stat.berkeley.edu/~stark/SticiGui/Text/probabilityAxioms.htm).\n", "\n", "Let $S$ denote the _outcome space_, the set of all possible outcomes of a random experiment, \n", "and let $\\{A_i\\}_{i=1}^\\infty$ be subsets of $S$.\n", "(Note that here $A$ denotes a subset, not a matrix.)\n", "Then any probability function ${\\mathbb P}$ must satisfy these axioms:\n", "\n", "1. For every $A \\subset S$, ${\\mathbb P}(A) \\ge 0$ (probabilities are nonnegative)\n", "2. ${\\mathbb P}(S) = 1$ (the chance that _something_ happens is 100%)\n", "3. If $A_i \\cap A_j = \\emptyset$ for $i \\ne j$, then ${\\mathbb P} \\cup_{i=1}^\\infty A_i = \\sum_{i=1}^\\infty {\\mathbb P}(A_i)$\n", "(If a countable collection of events is _pairwise disjoint_, then the chance that any of the\n", "events occurs is the sum of the chances that they occur individually.)\n", "\n", "These axioms have many useful consequences, among them that ${\\mathbb P}(\\emptyset) = 0$, ${\\mathbb P}(A^c) = 1 - {\\mathbb P}(A)$,\n", "and ${\\mathbb P}(A \\cup B) = {\\mathbb P}(A) + {\\mathbb P}(B) - {\\mathbb P}(AB)$.\n", "\n", "