\n", "\n", "## Content of the lecture\n", "\n", " * Systematics\n", " * Shape analysis\n", " * Test statistic for discovery and exclusions" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "Restart from the on/off problem.\n", "\n", "We observe the number of events in the signal region $n_{SR}$ and in the control region $n_{CR}$. $\\alpha$ is the ratio of the expected background in the CR with respect to the SR. We can write the likelihood as:\n", "\n", "$$L(s, b|n_{SR}, n_{CR}) = \\text{Pois}(n_{SR}|s + b) \\text{Pois}(n_{CR}|\\alpha b)$$\n", "\n", "The test statistics is based on the profiled likelihood ratio:\n", "\n", "$$-2\\log\\lambda = -2\\log\\frac{\\sup_{b \\in [0, \\infty], s\\in\\{0\\}}{L(s, b)}}{\\sup_{b\\in [0, \\infty], s\\in [0, \\infty]}{L(s, b)}} = -2\\log\\frac{L(0, \\hat{\\hat{b}}(s=0))}{L(\\hat{s}, \\hat{b})}$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "## Systematics\n", "\n", "How we can incorporate systematics inside our model? Suppose we know the parameter $\\alpha$ with a relative uncertainty $\\sigma_\\alpha$. We can imagine that there is another measurement (auxiliary measurement), that measured $\\alpha$ (the parameter of the model of the auxiliary measurement) and observed $a$. We can write the likelihood of the auxiliary measurement as $L(a|\\alpha) = N(a|\\alpha, \\delta_\\alpha)$, if assume that $a$ is normally distributed aroud the true value $\\alpha$ and $\\delta\\alpha$ is the absolute error on $\\alpha$ ($\\delta\\alpha=\\sigma_\\alpha a$)\n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "We can join the auxiliary measurement with our on/off model:\n", "\n", "$$L(s, b, a|N_{SR}, N_{CR}, \\alpha) = L(s, b|N_{SR}, N_{CR}) L(a|\\alpha)$$\n", "\n", "or more explicitely:\n", "\n", "$$\\text{Pois}(N_{SR}|s + b) \\text{Pois}(N_{CR}|\\alpha b) \\times N(a|\\alpha, \\delta_\\alpha)$$\n", "\n", "Usually this is written as:\n", "\n", "$$\\text{Pois}(N_{SR}|s + b) \\text{Pois}(N_{CR}|a (1 + \\sigma_\\alpha\\theta_\\alpha) b) \\times N(0|\\theta_\\alpha, 1)$$\n", "\n", "The first term is called the \"physical\" pdf, while the second is called the \"constraints\".\n" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "scrolled": true, "slideshow": { "slide_type": "subslide" } }, "outputs": [], "source": [ "#ws = ws_onoff_withsys = ws_onoff.Clone(\"ws_onoff_withsys\")\n", "f = ROOT.TFile.Open(\"onoff.root\")\n", "ws_onoff = f.Get(\"ws_onoff\")\n", "\n", "# create the term kalpha = (1 + sigma * theta) with a relative error of 20%\n", "ws_onoff.factory('expr:kalpha(\"1 + @0 * @1\", {sigma_alpha[0.2], theta_alpha[0, -5, 5]})')\n", "ws_onoff.factory('prod:alpha_x_kappa(alpha, kalpha)')\n", "# create new pdf model replacing alpha -> alpha_x_kalpha\n", "ws_onoff.factory('EDIT:model_with_sys(model, alpha=alpha_x_kappa)')\n", " \n", "# create new workspace\n", "ws_onoff_sys = ROOT.RooWorkspace('ws_onoff_sys')\n", "getattr(ws_onoff_sys, 'import')(ws_onoff.pdf('model_with_sys'))\n", "# create the constraint\n", "ws_onoff_sys.factory(\"Gaussian:constraint_alpha(global_alpha[0, -5, 5], theta_alpha, 1)\")\n", "ws_onoff_sys.var(\"global_alpha\").setConstant(True)\n", "# final pdf\n", "model = ws_onoff_sys.factory(\"PROD:model_constrained(model_with_sys, constraint_alpha)\")" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "RooWorkspace(ws_onoff_sys) ws_onoff_sys contents\n", "\n", "variables\n", "---------\n", "(alpha,b,global_alpha,n_cr,n_sr,s,sigma_alpha,theta_alpha)\n", "\n", "p.d.f.s\n", "-------\n", "RooPoisson::N_CR_model_with_sys[ x=n_cr mean=alpha_x_b_model_with_sys ] = 0\n", "RooPoisson::N_SR[ x=n_sr mean=s_plus_b ] = 0\n", "RooGaussian::constraint_alpha[ x=global_alpha mean=theta_alpha sigma=1 ] = 1\n", "RooProdPdf::model_constrained[ model_with_sys * constraint_alpha ] = 0\n", "RooProdPdf::model_with_sys[ N_SR * N_CR_model_with_sys ] = 0\n", "\n", "functions\n", "--------\n", "RooProduct::alpha_x_b_model_with_sys[ alpha_x_kappa * b ] = 500\n", "RooProduct::alpha_x_kappa[ alpha * kalpha ] = 10\n", "RooFormulaVar::kalpha[ actualVars=(sigma_alpha,theta_alpha) formula=\"1+@0*@1\" ] = 1\n", "RooAddition::s_plus_b[ s + b ] = 50\n", "\n" ] } ], "source": [ "ws_onoff_sys.Print()" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "data": { "image/svg+xml": [ "" ], "text/plain": [ "
![](\"img/figaux_004a.png\")
"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"![](\"img/figaux_007a.png\")
"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"![](\"img/figaux_007b.png\")
"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "subslide"
}
},
"source": [
"![](\"img/atlas_mH_ws.jpg\")
"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Topic not coverted, but important if you want to do hypothesis test in HEP\n",
"\n",
" * Look elesewhere effect\n",
" * CLs\n",
" * Asimov dataset"
]
},
{
"cell_type": "markdown",
"metadata": {
"slideshow": {
"slide_type": "slide"
}
},
"source": [
"## Last (big) exercize\n",
"Redo one of the example where we have use the frequentist calculator, without using it, reimplementing the toy generation and the computation of the test statistic by hand"
]
}
],
"metadata": {
"celltoolbar": "Slideshow",
"hide_input": false,
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.16"
},
"latex_envs": {
"bibliofile": "biblio.bib",
"cite_by": "apalike",
"current_citInitial": 1,
"eqLabelWithNumbers": true,
"eqNumInitial": 0
}
},
"nbformat": 4,
"nbformat_minor": 1
}