{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ " **Chapter 1: [Introduction](CH1_00-Introduction.ipynb)** \n", "\n", "\n", "
\n", "\n", "# Prerequisites\n", "\n", "[Download](https://raw.githubusercontent.com/gduscher/MSE672-Introduction-to-TEM//main/Introduction/CH1_02-Prerequisites.ipynb)\n", "\n", "[![OpenInColab](https://colab.research.google.com/assets/colab-badge.svg)](\n", " https://colab.research.google.com/github/gduscher/MSE672-Introduction-to-TEM/blob/main/Introduction/CH1_02-Prerequisites.ipynb)\n", " \n", "part of\n", "\n", " **[MSE672: Introduction to Transmission Electron Microscopy](../_MSE672_Intro_TEM.ipynb)**\n", "\n", "**Spring 2026**
\n", "by Gerd Duscher\n", "\n", "Microscopy Facilities
\n", "Institute of Advanced Materials & Manufacturing
\n", "Materials Science & Engineering
\n", "The University of Tennessee, Knoxville\n", "\n", "\n", "Background and methods to analysis and quantification of data acquired with transmission electron microscopes." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Language\n", "The notebooks are all in python 3. \n", "\n", "At this point the common version is python 3.9" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "cd\n", "## Packages\n", "The idea behind any python program is to make use of the highly efficient libraries that already exist.\n", "\n", "I use [anaconda3](https://www.anaconda.com/distribution/) (not miniconda) which is available for all major operating systems.\n", "\n", "We use a few modules that come with every python installation, like:\n", "\n", "* math\n", "* sys\n", "* os\n", "\n", "We use mostly the common packages for scientific computing in python (all included in anaconda3)\n", "The most important ones are:\n", "* [Numpy](https://www.numpy.org/) - the numerical library\n", "* [Scipy](https://www.scipy.org/scipylib/index.html) the scientific library\n", "* [Matplotlib](https://www.matplotlib.org/) the interactive plotting library \n", "\n", "\n", "These packages are expected to be installed on your computer to run the notebooks of this book.\n", "These packages are already installed in Google colab.\n", "\n", "For specialist applications we do not reinvent the wheel and use those on an as needed basis.\n", "\n", "Example is the library to register a stack of images:\n", "* [SimpleITK](https://www.simpleitk.org)\n", "integration of AI in image analysis and for high performing computer algorithms\n", "* [pyNSID](https://pycroscopy.github.io/pyNSID/about.html)\n", "the atomistic simulation program is used for crystallographic data\n", "* [ase](https://wiki.fysik.dtu.dk/ase/)\n", "together with a symmetry package\n", "* [spglib](https://atztogo.github.io/spglib/)\n", "\n", "\n", "For dialogs, we use the capabilities of Jupyter Widgets provided by:\n", "* [ipywidgets](https://ipywidgets.readthedocs.io/en/stable/)\n", "\n", "All routines that are introduced in the notebooks are also available (for analysis) in the provided package \n", "* [pyTEMlib](https://github.com/pycroscopy/pyTEMlib)\n", "\n", "\n", "If you install **[pyTEMlib](#TEM-Library)** with the code cell below all packages you need for this book will be installed." ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "## Basic Installation of Python Environment\n", "\n", "I recommend installing the free **anaconda3** from this [link](https://www.anaconda.com/products/individual).\n", "\n", "If you have an old version of anaconda, please reinstall the new version.\n", "The lecture is based on fairly new packages of scipy and numpy, dask, and h5py, which will create problems in old anaconda versions.\n", "\n", "Once you have installed anaconda\n", "type \n", ">pip install pyTEMlib\n", "\n", "Alternatively you can just run the code cell below. Be aware that the installation process may take a while." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "done\n" ] } ], "source": [ "import sys\n", "import importlib.metadata\n", "def test_package(package_name):\n", " \"\"\"Test if package exists and returns version or -1\"\"\"\n", " try:\n", " version = importlib.metadata.version(package_name)\n", " except importlib.metadata.PackageNotFoundError:\n", " version = '-1'\n", " return version\n", "\n", "# pyTEMlib setup ------------------\n", "if test_package('pyTEMlib') < '0.2026.1.0':\n", " print('installing pyTEMlib')\n", " !{sys.executable} -m pip install --upgrade pyTEMlib\n", "\n", "# ------------------------------\n", "print('done')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Data Format\n", "All data in this course are stored in the file format of\n", "* [pyNSID](https://pycroscopy.github.io/pyNSID/about.html)\n", "\n", "which is based on\n", "* [HDF5](https://www.h5py.org/)\n", "\n", "\n", "In the memory the data are stored in dictionaries in the \n", "* [sidpy](https://pycroscopy.github.io/sidpy) data format\n", " \n", "which provides a lot of functionalities for plotting and is the basis for any other package of the [pycroscopy](https://github.com/pycroscopy) ecosystem.\n", "\n", "## Notebook preamble\n", "As a minimum Any notebook in this course has to have the following libraries loaded :" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [], "source": [ "# import matplotlib and numpy \n", "# Define iteractivity with this **magic** command\n", "# use \"inline\" instead of \"widget\" for non-interactive plots\n", "%matplotlib widget\n", "\n", "import matplotlib.pylab as plt\n", "import numpy as np" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Numpy\n", "\n", "The calculations depend on **Numpy** and an installation of that package that is compiled to include BLAS and LAPACK libraries will be much faster than the standalone version.\n", "For example the numpy installed on ubuntu with *> sudo apt-get install python3-numpy* or at windows you can install the numpy package from Gohlke's webpage which compiled against the MKL library of Intel. If you used anaconda3, everything is already optimized.\n", "\n", "The command below lets you see what you have" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Build Dependencies:\n", " blas:\n", " detection method: pkgconfig\n", " found: true\n", " include directory: C:/Users/gduscher/AppData/Local/anaconda3/Library/include\n", " lib directory: C:/Users/gduscher/AppData/Local/anaconda3/Library/lib\n", " name: mkl-sdl\n", " openblas configuration: unknown\n", " pc file directory: C:\\miniconda3\\conda-bld\\numpy_and_numpy_base_1763980694785\\_h_env\\Library\\lib\\pkgconfig\n", " version: '2025'\n", " lapack:\n", " detection method: pkgconfig\n", " found: true\n", " include directory: C:/Users/gduscher/AppData/Local/anaconda3/Library/include\n", " lib directory: C:/Users/gduscher/AppData/Local/anaconda3/Library/lib\n", " name: mkl-sdl\n", " openblas configuration: unknown\n", " pc file directory: C:\\miniconda3\\conda-bld\\numpy_and_numpy_base_1763980694785\\_h_env\\Library\\lib\\pkgconfig\n", " version: '2025'\n", "Compilers:\n", " c:\n", " commands: cl.exe\n", " linker: link\n", " name: msvc\n", " version: 19.29.30159\n", " c++:\n", " commands: cl.exe\n", " linker: link\n", " name: msvc\n", " version: 19.29.30159\n", " cython:\n", " commands: cython\n", " linker: cython\n", " name: cython\n", " version: 3.1.4\n", "Machine Information:\n", " build:\n", " cpu: x86_64\n", " endian: little\n", " family: x86_64\n", " system: windows\n", " host:\n", " cpu: x86_64\n", " endian: little\n", " family: x86_64\n", " system: windows\n", "Python Information:\n", " path: C:\\miniconda3\\conda-bld\\numpy_and_numpy_base_1763980694785\\_h_env\\python.exe\n", " version: '3.13'\n", "SIMD Extensions:\n", " baseline:\n", " - SSE\n", " - SSE2\n", " - SSE3\n", " found:\n", " - SSSE3\n", " - SSE41\n", " - POPCNT\n", " - SSE42\n", " - AVX\n", " - F16C\n", "\n", "numpy version: 2.3.5\n", "scipy version: 1.16.3\n" ] } ], "source": [ "## What is numpy compiled against\n", "np.__config__.show()\n", "print('numpy version: ',np.version.version)\n", "import scipy as sp\n", "print('scipy version: ',sp.__version__)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## TEM Library\n", "\n", "\n", "You will have to run the code cell below **at least once** to install the library with the programs needed for the analysis of data.\n", "\n", "The code cell below will install pyTEMlib directly from pypi" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "pycharm": { "is_executing": true }, "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "done\n" ] } ], "source": [ "import sys\n", "import importlib.metadata\n", "def test_package(package_name):\n", " \"\"\"Test if package exists and returns version or -1\"\"\"\n", " try:\n", " version = importlib.metadata.version(package_name)\n", " except importlib.metadata.PackageNotFoundError:\n", " version = -1\n", " return version\n", "\n", "# pyTEMlib setup ------------------\n", "if test_package('pyTEMlib') < '0.2026.1.0':\n", " print('installing pyTEMlib')\n", " !{sys.executable} -m pip install --upgrade pyTEMlib\n", "print('done')" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "Now we load pyTEMlib to make it available for this notebook." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "pyTEM version: 0.2026.1.1\n" ] } ], "source": [ "import pyTEMlib\n", "print(f'pyTEM version: {pyTEMlib.__version__}')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Test\n", "Let's test if the installation was successful and plot a unit cell. You can rotate the plot around, zoom and select. Try it!" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Atoms(symbols='C4', pbc=False, cell=[[2.46772414, 0.0, 0.0], [-1.2338620699999996, 2.1371117947721068, 0.0], [0.0, 0.0, 6.711]])\n" ] }, { "data": { "text/html": [ "\n", " \n", " ASE atomic visualization\n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "\n", "\n", "\n", " \n", " \n", "\n", "\n" ], "text/plain": [ "" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# ---- Input ------------\n", "crystal_name = 'graphite'\n", "# -----------------------\n", "\n", "# make a structure dictionary with crystal_tools\n", "atoms = pyTEMlib.crystal_tools.structure_by_name(crystal_name)\n", "print(atoms)\n", "import ase.visualize\n", "ase.visualize.view(atoms*(4,4,1), viewer='x3d')\n" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Help on function structure_by_name in module pyTEMlib.crystal_tools:\n", "\n", "structure_by_name(crystal_name: str) -> ase.atoms.Atoms | None\n", " Provides crystal structure in ase.Atoms format.\n", " Additional information is stored in the info attribute as a dictionary\n", "\n", " Parameter\n", " ---------\n", " crystal_name: str\n", " Please note that the chemical expressions are not case-sensitive.\n", "\n", " Returns\n", " -------\n", " atoms: ase.Atoms\n", " structure\n", "\n", " Example\n", " -------\n", " >> # for a list of pre-defined crystal structures\n", " >> import pyTEMlib.crystal_tools\n", " >> print(pyTEMlib.crystal_tools.crystal_data_base.keys())\n", " >>\n", " >> atoms = pyTEMlib.crystal_tools.structure_by_name('Silicon')\n", " >> print(atoms)\n", " >> print(atoms.info)\n", "\n" ] } ], "source": [ "help(pyTEMlib.crystal_tools.structure_by_name)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Get used to changing parameters and type **silicon** instead of **Graphite** in the code cell above." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Summary\n", "\n", "We now have tested all tools to load data and save our analysis.\n", "\n", "We are ready to go" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Navigation\n", "- **Back [Python as it is used here](CH1_01-Introduction_Python.ipynb)** \n", "- **Next: [Matplotlib and Numpy for Micrographs](CH1_03-Data_Representation.ipynb)** \n", "- **Chapter 1: [Introduction](CH1_00-Introduction.ipynb)** \n", "- **List of Content: [Front](../_MSE672_Intro_TEM.ipynb)** \n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Appendix\n", "\n", "I am using some extensions to jupyter notebooks which can be installed with the following cells\n", "\n", "I like mostly the table of content extension that shows where in the notebook I am and lets me jump to different parts easily.\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Known nbextensions:\n", " config dir: C:\\Users\\gdusc\\anaconda3\\etc\\jupyter\\nbconfig\n", " notebook section\n", " jupyter-jsmol/extension enabled \n", " jupyterlab-plotly/extension enabled \n", " jupyter-js-widgets/extension enabled \n" ] }, { "name": "stderr", "output_type": "stream", "text": [ " - Validating: ok\n", " - Validating: ok\n", " - Validating: ok\n" ] } ], "source": [ "# Install a pip package in the current Jupyter kernel\n", "import sys\n", "!{sys.executable} -m jupyter nbextension list" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Use one of the following (uncomment pip and comment out the conda command line, if you are not in an anaconda environment)" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Collecting package metadata (current_repodata.json): ...working... done\n", "Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.\n", "Collecting package metadata (repodata.json): ...working... done\n", "Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "PackagesNotFoundError: The following packages are not available from current channels:\n", "\n", " - jupyter_contrib_nbextensions\n", "\n", "Current channels:\n", "\n", " - https://repo.anaconda.com/pkgs/main/win-64\n", " - https://repo.anaconda.com/pkgs/main/noarch\n", " - https://repo.anaconda.com/pkgs/r/win-64\n", " - https://repo.anaconda.com/pkgs/r/noarch\n", " - https://repo.anaconda.com/pkgs/msys2/win-64\n", " - https://repo.anaconda.com/pkgs/msys2/noarch\n", "\n", "To search for alternate channels that may provide the conda package you're\n", "looking for, navigate to\n", "\n", " https://anaconda.org\n", "\n", "and use the search bar at the top of the page.\n", "\n", "\n" ] } ], "source": [ "import sys\n", "\n", "#!{sys.executable} -m pip install jupyter_contrib_nbextensions\n", "!conda install --yes --prefix {sys.prefix} jupyter_contrib_nbextensions" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] }, { "cell_type": "code", "execution_count": 3, "metadata": { "pycharm": { "is_executing": true }, "scrolled": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Enabling notebook extension toc2...\n", " - Validating: problems found:\n", " - require? X toc2\n" ] } ], "source": [ "\n", "!{sys.executable} -m jupyter nbextension enable toc2" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "data": { "application/javascript": [ "$('
').css({position: 'fixed', top: '120px', left: 0}).appendTo(document.body);\n", "$.getScript('https://kmahelona.github.io/ipython_notebook_goodies/ipython_notebook_toc.js');\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "%%javascript\n", "$('
').css({position: 'fixed', top: '120px', left: 0}).appendTo(document.body);\n", "$.getScript('https://kmahelona.github.io/ipython_notebook_goodies/ipython_notebook_toc.js');" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.5" }, "toc": { "base_numbering": "3", "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": { "height": "calc(100% - 180px)", "left": "10px", "top": "150px", "width": "242.4px" }, "toc_section_display": true, "toc_window_display": true }, "varInspector": { "cols": { "lenName": 16, "lenType": 16, "lenVar": 40 }, "kernels_config": { "python": { "delete_cmd_postfix": "", "delete_cmd_prefix": "del ", "library": "var_list.py", "varRefreshCmd": "print(var_dic_list())" }, "r": { "delete_cmd_postfix": ") ", "delete_cmd_prefix": "rm(", "library": "var_list.r", "varRefreshCmd": "cat(var_dic_list()) " } }, "types_to_exclude": [ "module", "function", "builtin_function_or_method", "instance", "_Feature" ], "window_display": true }, "vscode": { "interpreter": { "hash": "838e0debddb5b6f29d3d8c39ba50ae8c51920a564d3bac000e89375a158a81de" } } }, "nbformat": 4, "nbformat_minor": 4 }