# Resources on Automatic Differentiation (AD)

Compiled by Peter Sharpe

-----

Lectures, Videos, GitHub repos, blog posts:

* [YouTube: "What is Automatic Differentiation"](https://www.youtube.com/watch?v=wG_nF1awSSY) - a "3Blue1Brown"-style video introducing AD
* [Medium: "Automatic Differentiation Step by Step"](https://marksaroufim.medium.com/automatic-differentiation-step-by-step-24240f97a6e6)
* [GitHub: "Differentiation for Hackers"](https://github.com/MikeInnes/diff-zoo) - includes runnable examples in Julia
* [Lecture: "Automatic differentiation"](http://videolectures.net/deeplearning2017_johnson_automatic_differentiation/) - by Matthew J. Johnson, a huge contributor in the current AD landscape (developer of autograd, JAX).
* [Lecture: "Intuition behind reverse mode algorithmic differentiation"](https://youtu.be/twTIGuVhKbQ) - by Joris Gillis, one of the developers of CasADi (the AD library underneath AeroSandbox).
* [Blog: "Reverse-mode automatic differentiation: a tutorial"](https://rufflewind.com/2016-12-30/reverse-mode-automatic-differentiation)
* [Lecture: "Large-Scale Multidisciplinary Design Optimization of Aerospace Systems"](https://www.pathlms.com/siam/courses/479/sections/678/thumbnail_video_presentations/5169) - by Joaquim Martins. Section on AD starts at ~17:00.

Academic literature:

* Griewank and Walther, "Evaluating Derivatives: Principles and Techniques of Algorithmic Differentiation", 2008. Great comprehensive overview.
* ["Automatic Differentiation in Machine Learning: a Survey" on ArXiV by Baydin et. al.](https://arxiv.org/abs/1502.05767) is another great read.
* ["Automatic differentiation of algorithms"](https://www.sciencedirect.com/science/article/pii/S0377042700004222?via%3Dihub) by Bartholomew-Biggs, et. al.