{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Reading material\n", "\n", "Different authors present content in different ways.\n", "Try a few and pick one that you find most readable/accessible.\n", "\n", "\n", "\n", "## Books about Bayesian inference\n", "\n", "### General\n", "\n", "* \"Bayesian data analysis\" (Gelman): http://www.stat.columbia.edu/~gelman/book/\n", "* \"Introduction to Probability\" (Dimitri Bertsekas): covers basics: random variable, combinatorics, derived distributions, what is a PDF, how to work with them, expectation values\n", "* \"Probability and Statistics\" (Morris H. DeGroot): covers classical approaches to hypothesis testing and frequentist analysis, so you can really understand, e.g., the classical tests, what p-values really are, why are they used and how and when they are useful. \n", "\n", "### for physical sciences\n", "\n", "* \"Bayesian Logical Data Analysis for the Physical Sciences\" (Gregory)\n", "* \"Data Analysis - A Bayesian Tutorial\" (Sivia): building an intuition for how Bayesian statistics works\n", "* \"Scientific Inference\" (Simon Vaughan): basic probability theory, statistical thinking, model and data representation in computers. likelihood function, graphical summaries, basic Monte Carlo. \n", "* \"Bayesian reasoning in data analysis - A critical introduction\" (D'Agostini) https://www.roma1.infn.it/~dagos/WSPC/ differences in uncertainties, confidence vs. credible intervals, frequentist/maximum likelihood, Bayesian inference and how they relate\n", "* \"An Introduction to Statistical Learning with Applications in R\" (James, Witten, Hastie & Tibshirani) http://www-bcf.usc.edu/~gareth/ISL/ available online\n", "* \"Machine Learning: An Algorithmic Perspective\" (Stephen Marsland) http://seat.massey.ac.nz/personal/s.r.marsland/MLbook.html\n", "* \"Statistics, Data Mining, and Machine Learning in Astronomy\" (Zeljko Ivezic, Andy Connolly, Jake VanderPlas, Alex Gray) for scikit-learn and astroml\n", "* https://itp.tugraz.at/LV/wvl/Statistik/A_WS_pdf.pdf \"Wahrscheinlichkeitstheorie, Statistik und Datenanalyse\", von der Linden & Prüll\n", "\n", "## Lectures\n", "\n", "A similar course is available in full online at https://kipac.github.io/StatisticalMethods/\n", "\n", "\n", "* Annual astrostatistics summer school at Penn State: http://astrostatistics.psu.edu/ \n", "* Stefano Andreon's page http://www.brera.mi.astro.it/~stefano.andreon/ (Click on teaching)\n", "* Brendon Brewer's teaching page https://www.stat.auckland.ac.nz/~brewer/teaching.html (Click on STATS 331)\n", "* James M. Cordes's teaching page http://hosting.astro.cornell.edu/~cordes/A6523/\n", "* James Long's SAMSI course https://www.stat.tamu.edu/~jlong/astrostat/\n", "* Jarle Brinchmann's Databases & Data mining course https://github.com/jbrinchmann/DDM2017\n", "* Astrostatistics & Machine Learning course https://github.com/dirac-institute/uw-astr598-w18\n", "\n", "\n", "## Research Papers on State-of-the-Art Methods\n", "\n", "In the course, we will go through some of these (you do not have to read them beforehand).\n", "\n", "\n", "### Markov Chain Monte Carlo\n", "\n", "(see also the end of the 3-MCMC notebook for references)\n", "\n", "* https://arxiv.org/abs/1701.02434: A Conceptual Introduction to Hamiltonian Monte Carlo\n", "* https://arxiv.org/abs/1909.12313: A Conceptual Introduction to Markov Chain Monte Carlo Methods\n", "\n", "Talks:\n", "\n", "* By Michael Betancourt: https://www.youtube.com/results?search_query=michael+betancourt\n", "* Writing https://betanalpha.github.io/writing/\n", "\n", "Diagnostics:\n", "\n", "* https://arxiv.org/abs/1903.08008: Modern Convergence diagnostics\n", "* https://arxiv.org/abs/2003.07900: Convergence diagnostic with machine learning\n", "\n", "Recommended state-of-the-art tools:\n", "\n", "* Stan https://mc-stan.org\n", "* There are also younger implementations in Python (PyMC3, https://docs.pymc.io/) and Julia (BAT, https://bat.github.io/BAT.jl/stable/).\n", "* Ensemble MCMC methods like [emcee](https://emcee.readthedocs.io/en/stable/) and [zeus](https://zeus-mcmc.readthedocs.io/) are popular for gradient-free inference. These are wrapped in autoemcee https://johannesbuchner.github.io/autoemcee/, which runs until convergence is reached.\n", "\n", "### Importance Sampling\n", "\n", "(there are many resources online)\n", "\n", "* Combination with Variational Bayes and initialization: https://arxiv.org/abs/1304.7808\n", "\n", "Recommended tools:\n", "\n", "* Programming Library: https://pypmc.github.io/ supports Variational Bayes and Expectation Maximization algorithms for proposal optimization, and MPI for sampling.\n", "* snowline: https://johannesbuchner.github.io/snowline/ wraps the above, together with minuit to find a initial guess.\n", "\n", "### Nested Sampling\n", "\n", "* https://arxiv.org/abs/2101.09675: Nested Sampling Methods\n", "* https://ui.adsabs.harvard.edu/search/?q=author%3A%22%5ESkilling%22+title%3ANested : papers by Skilling\n", "* https://arxiv.org/abs/1805.03924: Connection to Sequential/Population Monte Carlo\n", "\n", "LRPS techniques:\n", "\n", "* https://arxiv.org/abs/astro-ph/0508461: The Ellipsoidal Nested sampling technique\n", "* https://arxiv.org/abs/1904.02180: Illustrations of various techniques\n", "\n", "Diagnostics\n", "\n", "* https://arxiv.org/abs/1804.06406 - Several visualisations and tests, for example looking at consistency across runs.\n", "* https://arxiv.org/abs/2006.03371 - looks where the newly sampled point fits in the existing points likelihood distribution. Is its rank / order uniformly distributed?\n", "\n", "Recommended state-of-the-art, open-source tools:\n", "\n", "* [UltraNest](https://johannesbuchner.github.io/UltraNest/)\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.10" } }, "nbformat": 4, "nbformat_minor": 4 }