{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Useful Packages\n",
    "\n",
    "The number of registered packages in the [General](https://github.com/JuliaRegistries/General/blob/master/Registry.toml) registry is quite vast (about 2800 on **11/12/2018**) and growing every day. So if you have a reasonably standard problem you want to solve, there is a good chance that someone has already made an effort.\n",
    "\n",
    "To give you some ideas about the community and what is available, here is a non-exhaustive list, which is naturally influenced by my own background. I'm happy to adapt it, if you feel something is wrong / missing.\n",
    "\n",
    "Without further ado:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Development and coding workflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [Revise](https://github.com/timholy/Revise.jl) is one of the absolute must-have packages. It watches your filesystem for changes to a Julia package or script and reloads it automatically in the current REPL / Jupyter kernel. If you are the developer of a Julia package or usually find yourself coding in one terminal and using the code in another, this package is indispensable.\n",
    "- [JuliaFormatter](https://github.com/domluna/JuliaFormatter.jl): Basic automatic source-code formatter for Julia.\n",
    "- [Infiltrator](https://github.com/JuliaDebug/Infiltrator.jl): Drop into an interactive REPL at an arbitrary place in your code. Great for debugging and testing.\n",
    "- [Debugger](https://github.com/JuliaDebug/Debugger.jl): A gdb-like Julia debugger, which I use every day.\n",
    "- [Rebugger](https://github.com/timholy/Rebugger.jl): Another debugger. Preferred over Debugger by some people.\n",
    "- [Juno](https://junolab.org/): Not actually a package, but the development environment endorsed Julia computing.\n",
    "- Surely there is Julia support for [your favourite editor](https://github.com/JuliaEditorSupport), too."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Reports and source-code documentation\n",
    "- [Documenter](https://github.com/JuliaDocs/Documenter.jl) is the goto source code documentation tool for Julia.\n",
    "- [Weave](http://weavejl.mpastell.com/stable/) allows to mix markdown and Julia, where the Julia code is automatically executed and resulting plots, data, tables are included into the final document."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## More linear algebra\n",
    "\n",
    "- [IterativeSolvers](https://juliamath.github.io/IterativeSolvers.jl/dev/) is a rich library for iterative methods with the most important iterative solver algorithms (Conjugate Gradient, GMRES, BiCGStab, Chebychev iteration, ...)\n",
    "- [StaticArrays](https://github.com/JuliaArrays/StaticArrays.jl): Arrays of compile-time known size. Has a huge speed improvement for example when used for 3D vectors or 3D rotations.\n",
    "- [BlockArrays](https://github.com/JuliaArrays/BlockArrays.jl): Special array for block-wise storing or treating arrays \n",
    "- [LazyArrays](https://github.com/JuliaArrays/LazyArrays.jl): Makes array operations lazy.\n",
    "- [TensorOperations](https://github.com/Jutho/TensorOperations.jl): Performs higher-order tensor operations in convenient syntax. Kernels are generated with respect to the operation performad and are automatically parallelised. GPU support in the making. An example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import Pkg; Pkg.add(\"TensorOperations\")\n",
    "using TensorOperations\n",
    "N = 3\n",
    "I = randn(N, N, N, N); C = randn(N, N);\n",
    "@tensor It[i,j,k,l] := I[α,β,γ,δ] * C[α, i] * C[β, j] * C[γ, k] * C[δ, l]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Automatic Differentiation (AD)\n",
    "- [Zygote](https://github.com/FluxML/Zygote.jl): Adjoint-mode AD, we already used it in [Dancing_Particles](05_Dancing_Particles.ipynb)\n",
    "- [ForwardDiff](https://github.com/JuliaDiff/ForwardDiff.jl): Forward-mode AD\n",
    "- [FiniteDifferences](https://github.com/JuliaDiff/FiniteDifferences.jl): Classic finite-difference approximations\n",
    "- [ChainRules](https://github.com/JuliaDiff/ChainRules.jl): Generic interface for automatice differentiation in Julia, allows to couple different kinds of AD"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualisation and interaction\n",
    "- Numerous plot packages were already mentioned in [Dancing_Particles](05_Dancing_Particles.ipynb).\n",
    "- [Interact](https://github.com/JuliaGizmos/Interact.jl) is great for adding interactive sliders or buttons to a notebook. For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import Pkg; Pkg.add(\"Interact\")\n",
    "using Interact\n",
    "using Plots\n",
    "Vmorse(r; re=0.7, α=1.3, D=10) = D*(1 - exp(-α * (r - re)))^2 - D\n",
    "\n",
    "r = collect(0:0.1:10)\n",
    "mp = @manipulate for α in slider(0:0.1:4; label=\"Steepness α\"), D in slider(0:0.4:30, label=\"depth D\"),\n",
    "                     re in slider(0:0.1:4, label=\"Equilibrium re\")\n",
    "    p = plot(r, Vmorse.(r, re=re, α=α, D=D), label=\"Vmorse re=$re α=$α D=$D\", ylim=(-30, 30))\n",
    "end"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Symbolic manipulation\n",
    "- [SymPy](https://github.com/JuliaPy/SymPy.jl): Julia wrapper of the popular python package"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Standard file formats\n",
    "\n",
    "Plenty of standard file formats in scientific computing and elsewhere can be used in Julia:\n",
    "- [JSON](https://github.com/JuliaIO/JSON.jl) and [YAML](https://github.com/BioJulia/YAML.jl) (TOML is in the standard library)\n",
    "- [HDF5](https://github.com/JuliaIO/HDF5.jl)\n",
    "- [MAT](https://github.com/JuliaIO/MAT.jl): Support for Matlab's mat files\n",
    "- [NCDatasets](https://github.com/Alexander-Barth/NCDatasets.jl): Support for netcdf\n",
    "- [JLD](https://github.com/JuliaIO/JLD.jl) and [JLD2](https://github.com/JuliaIO/JLD2.jl): Julia's own file format to reproducably dump and load any Julia object."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "-----"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chemistry and Materials Science\n",
    "\n",
    "Not too many projects in chemistry and materials science have adopted Julia so far. Here are a few:\n",
    "- [Chemfiles](https://github.com/chemfiles/Chemfiles.jl): Wrapper around the C library of the same name, which understands quite a few file formats in quantum chemistry.\n",
    "- [ASE](https://github.com/JuliaMolSim/ASE.jl): Wrapper around the python library of the same name for atomistic simulations in materials science\n",
    "- [JuLIP](https://github.com/JuliaMolSim/JuLIP.jl): Library for generating and fitting interatomic potentials\n",
    "- [DFTK](https://github.com/JuliaMolSim/DFTK.jl): Toolkit for density-functional theory and related models in extended systems (shameless advertisement)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PDEs and ODEs\n",
    "- Most of the tooling for solving differential equations in Julia is collected in the [JuliaDiffEq](https://juliadiffeq.org/) organisation. The most important package is [DifferentialEquations](https://github.com/JuliaDiffEq/DifferentialEquations.jl), where basically all the strings are collected. The package mostly concentrates on ordinary differential equations in various kinnds (random, stochastic, algebraic, mixed discrete / continuous ...)\n",
    "- For PDEs one big ecosystem is [JuliaFEM](http://www.juliafem.org/), mostly centered around the [JuliaFEM package](https://github.com/JuliaFEM/JuliaFEM.jl). It brings a suite for finite-element modelling and all sorts of supporting tools (mesh generation, node renumbering). They claim to be able to handle between 10M to 100M degrees of freedom."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Optimisation-related packages\n",
    "\n",
    "The optimisation community has adopted Julia rather early and the respective Julia libraries are at a very good shape.\n",
    "\n",
    "- The driving force behind the Julia optimisation community is the [JuliaOpt](https://www.juliaopt.org/) organisation with its working horse [JuMP](https://github.com/JuliaOpt/JuMP.jl). This package defines a metalanguage for optimisation problems, which can be combined with about 20 open-source or commercial optimisation solvers. A pretty exhaustive set of problem classes are supported: Linear programming, (mixed) integer programming, semidefinite programming, nonlinear programming, ...\n",
    "\n",
    "- For certain highly specific use cases a number of specialised packages have emerged:\n",
    "    - [Optim](https://github.com/JuliaNLSolvers/Optim.jl) for continuous optimisation problems including simple manifolds\n",
    "    - [LsqFit](https://github.com/JuliaNLSolvers/LsqFit.jl) for least-squares problems\n",
    "    - [Roots](https://github.com/JuliaMath/Roots.jl) for finding roots of functions\n",
    "    - [NLsolve](https://github.com/JuliaNLSolvers/NLsolve.jl) for general non-linear systems of equations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data science and statistics\n",
    "Apart from a few examples, the most important statistics and data science packages are associated with the [JuliaStats](https://github.com/JuliaStats) and [JuliaData](https://github.com/JuliaData) organisations. In particular there are:\n",
    "- `Statistics` in the standard library.\n",
    "- [DataFrames](https://github.com/JuliaData/DataFrames.jl): Package for working with tabular data (\"the pandas of Julia\")\n",
    "- [PrettyTables](https://github.com/ronisbr/PrettyTables.jl): Nicely formatted tables\n",
    "- [StatsKit](https://github.com/JuliaStats/StatsKit.jl): Metapackage loading a full suite of Statistics-related tools\n",
    "- [Distributions](https://github.com/JuliaStats/Distributions.jl): Defines standard distributions and their properties"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Machine Learning\n",
    "\n",
    "- In Julia two important frameworks exist, namely [Flux](https://github.com/FluxML/Flux.jl) and [Knet](https://github.com/denizyuret/Knet.jl). Their philosophy seems to differ a little, such that both seem to have their respective pros and cons.\n",
    "- Bindings for popular packages in foreign languages exist as well ([TensorFlow](https://github.com/malmaud/TensorFlow.jl),  [Apache MXNet](https://github.com/apache/incubator-mxnet)). These bindings seem to be less actively developed than the above packages, however."
   ]
  }
 ],
 "metadata": {
  "@webio": {
   "lastCommId": "0bba521938544a5096c2349d28e65280",
   "lastKernelId": "e13bd870-ac07-446d-a160-212f94b7a565"
  },
  "kernelspec": {
   "display_name": "Julia 1.3.0",
   "language": "julia",
   "name": "julia-1.3"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.3.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}