{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Confidence Bounds for the Mean of a Finite Population\n",
    "\n",
    "\n",
    "## Some canonical problems\n",
    "### Address constructing nonparametric confidence bounds in a variety of contexts\n",
    "\n",
    "1. Financial audits, including healthcare, tax, and due diligence audits\n",
    "   * Is a set of of asset accounts (e.g., accounts receivable) materially _overstated_? By how much?\n",
    "   * Is a set of of liability accounts (e.g., accounts payable) materially _understated_? By how much?\n",
    "   * Of a collection of bills that were written off, what fraction of the value was written off in compliance with applicable policy?\n",
    "   * Is a set of bills for healthcare _overstated_? By how much?  \n",
    "      Health Care Fraud Prevention and Enforcement Action Team (HEAT)  \n",
    "      Since 2007, >1,400 defendants collectively falsely billed Medicare >\\$4.8 billion.\n",
    "      In 2011, HEAT federal health care fraud takedown involving $530 million.\n",
    "      \n",
    "      http://www.hhs.gov/news/press/2014pres/05/20140513b.html\n",
    "      http://www.hhs.gov/news/press/2013pres/05/20130514a.html\n",
    "   \n",
    "2. Election auditing\n",
    "   * Was the margin for the reported winner _overstated_? By how much?\n",
    "   * Did the reported winners really win?\n",
    "   * Were the correct numbers of seats allocated to each party?\n",
    "   \n",
    "3. Damages in litigation\n",
    "   * Were employees paid all the overtime they were due? If not, how much is owed?\n",
    "   * Were employees given all the meal and rest breaks they were due? If not, what is owed?\n",
    "   * Were mortgage origination fees \"padded\"? By how much?\n",
    "   * Was a patent infringed? If so, how much is due in royalties or license fees?\n",
    "   * In a collection of mortgages, what fraction of the original balances were in loans that did not comply with the applicable underwriting guidelines?\n",
    "   \n",
    "4. Causal inference in experiments, marketing, etc.\n",
    "   * Does a tool for targeted advertising work? How much does it increase sales?\n",
    "   * Does a purported remedy work? How large is its effect?\n",
    "   \n",
    "All these can be reduced to inferences about the mean of a nonnegative or bounded finite population.  \n",
    "That is, there are:\n",
    "\n",
    "1. known numbers $N > 0$,  $ \\{\\ell_j\\}_{j=1}^N$, and $\\{u_j\\}_{j=1}^N$, with $\\ell_j \\le u_j,\\; \\forall j$\n",
    "2. an unknown population of numbers $\\{x_j\\}_{j=1}^N$ that satisfy $\\ell_j \\le x_j \\le u_j,\\; \\forall j$\n",
    "\n",
    "We want to draw inferences about \n",
    "$$\\mu \\equiv \\frac{1}{N} \\sum_{j=1}^N x_j.$$\n",
    "\n",
    "We want more than a point estimate:\n",
    "+ rigorous hypothesis tests\n",
    "+ exact or conservative confidence intervals\n",
    "+ generally, 1-sided confidence interval: lower confidence bound or upper confidence bound, depending on context.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Mean, not median\n",
    "\n",
    "Nonparametric confidence bounds for population percentiles require only a random sample.\n",
    "(They can be constructed by invert binomial or hypergeometric tests; see, e.g., Lehmann.)\n",
    "   \n",
    "Nonparametric confidence bounds for the *mean* require prior information, because a single item \n",
    "   can have an arbitrarily large influence on the population mean.\n",
    "   Unless the distribution is symmetric, the mean need not equal the median.\n",
    "   In many applications, the mean (or population total) is relevant but the median is not.\n",
    "\n",
    "For instance, in both financial and election auditing one might like to determine from a random sample \n",
    "   whether a reported total differs materially from the total that a full manual audit would \n",
    "   show.\n",
    "   \n",
    "In auditing, the population of errors typically consists mostly of zeros or \n",
    "   small values, but occasional large values are possible, which makes the normal approximation \n",
    "   inadequate.\n",
    "   The mean error in the population determines the total error; the median\n",
    "   error is not directly relevant."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### General note:\n",
    "\n",
    "Most of the methods we consider will assume that we are sampling _with_ replacement.  There are three main reasons:\n",
    "\n",
    "1. In the applications contemplated, the sample size is a small fraction of the population size, so there is not much difference between sampling with and without replacment.\n",
    "2. The math is simpler for sampling with replacement.\n",
    "3. In great generality, uncertainties for sampling with replacement are larger than for sampling without replacement (see Hoeffding); hence, these results give a conservative impression of what is possible.\n",
    "\n",
    "Many of the approaches do have extensions to sampling without replacement, but the bookkeeping is more complicated.\n",
    "\n",
    "Many of the methods we will consider draw samples with replacement, but not with equal probabilities. In particular, selecting items with probability proportional to a bound on \"size\" turns out to be very helpful."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Next topic: Why not use the normal approximation?](normApprox.ipynb)\n",
    "\n",
    "[Index of topics](index.ipynb)\n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}