{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ " **Chapter 1: [Introduction](CH1_00-Introduction.ipynb)** \n", "\n", "\n", "
\n", "\n", "# Prerequisites\n", "\n", "[Download](https://raw.githubusercontent.com/gduscher/MSE672-Introduction-to-TEM//main/Introduction/CH1_02-Prerequisites.ipynb)\n", "\n", "[![OpenInColab](https://colab.research.google.com/assets/colab-badge.svg)](\n", " https://colab.research.google.com/github/gduscher/MSE672-Introduction-to-TEM/blob/main/Introduction/CH1_02-Prerequisites.ipynb)\n", " \n", "part of\n", "\n", " **[MSE672: Introduction to Transmission Electron Microscopy](../_MSE672_Intro_TEM.ipynb)**\n", "\n", "**Spring 2024**\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Gerd Duscher Khalid Hattar
Microscopy Facilities Tennessee Ion Beam Materials Laboratory
Materials Science & Engineering Nuclear Engineering
Institute of Advanced Materials & Manufacturing
The University of Tennessee, Knoxville
\n", "\n", "Background and methods to analysis and quantification of data acquired with transmission electron microscopes." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Language\n", "The notebooks are all in python 3. \n", "\n", "At this point the common version is python 3.9" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "cd\n", "## Packages\n", "The idea behind any python program is to make use of the highly efficient libraries that already exist.\n", "\n", "I use [anaconda3](https://www.anaconda.com/distribution/) (not miniconda) which is available for all major operating systems.\n", "\n", "We use a few modules that come with every python installation, like:\n", "\n", "* math\n", "* sys\n", "* os\n", "\n", "We use mostly the common packages for scientific computing in python (all included in anaconda3)\n", "The most important ones are:\n", "* [Numpy](https://www.numpy.org/) - the numerical library\n", "* [Scipy](https://www.scipy.org/scipylib/index.html) the scientific library\n", "* [Matplotlib](https://www.matplotlib.org/) the interactive plotting library \n", "\n", "\n", "These packages are expected to be installed on your computer to run the notebooks of this book.\n", "These packages are already installed in Google colab.\n", "\n", "For specialist applications we do not reinvent the wheel and use those on an as needed basis.\n", "\n", "Example is the library to register a stack of images:\n", "* [SimpleITK](https://www.simpleitk.org)\n", "integration of AI in image analysis and for high performing computer algorithms\n", "* [pyNSID](https://pycroscopy.github.io/pyNSID/about.html)\n", "the atomistic simulation program is used for crystallographic data\n", "* [ase](https://wiki.fysik.dtu.dk/ase/)\n", "together with a symmetry package\n", "* [spglib](https://atztogo.github.io/spglib/)\n", "\n", "\n", "For dialogs, we use the capabilities of Jupyter Widgets provided by:\n", "* [ipywidgets](https://ipywidgets.readthedocs.io/en/stable/)\n", "\n", "All routines that are introduced in the notebooks are also available (for analysis) in the provided package \n", "* [pyTEMlib](https://github.com/pycroscopy/pyTEMlib)\n", "\n", "\n", "If you install **[pyTEMlib](#TEM-Library)** with the code cell below all packages you need for this book will be installed." ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "## Basic Installation of Python Environment\n", "\n", "I recommend installing the free **anaconda3** from this [link](https://www.anaconda.com/products/individual).\n", "\n", "If you have an old version of anaconda, please reinstall the new version.\n", "The lecture is based on fairly new packages of scipy and numpy, dask, and h5py, which will create problems in old anaconda versions.\n", "\n", "Once you have installed anaconda\n", "type \n", ">pip install pyTEMlib\n", "\n", "Alternatively you can just run the code cell below. Be aware that the installation process may take a while." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "done\n" ] } ], "source": [ "import sys\n", "import importlib.metadata\n", "def test_package(package_name):\n", " \"\"\"Test if package exists and returns version or -1\"\"\"\n", " try:\n", " version = importlib.metadata.version(package_name)\n", " except importlib.metadata.PackageNotFoundError:\n", " version = '-1'\n", " return version\n", "\n", "# pyTEMlib setup ------------------\n", "if test_package('pyTEMlib') < '0.2023.9.1':\n", " print('installing pyTEMlib')\n", " !{sys.executable} -m pip install --upgrade git+https://github.com/pycroscopy/pyTEMlib.git@main -q --upgrade\n", "\n", "if 'google.colab' in sys.modules:\n", " !{sys.executable} -m pip install numpy==1.24.4\n", "# ------------------------------\n", "print('done')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Data Format\n", "All data in this course are stored in the data format of\n", "* [pyNSID](https://pycroscopy.github.io/pyNSID/about.html)\n", "\n", "which is based on\n", "* [HDF5](https://www.h5py.org/)\n", "\n", "\n", "\n", "## Notebook preamble\n", "As a minimum Any notebook in this course has to have the following libraries loaded :" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [], "source": [ "# import matplotlib and numpy \n", "# Define iteractivity with this **magic** command\n", "# use \"inline\" instead of \"widget\" for non-interactive plots\n", "%matplotlib widget\n", "\n", "import matplotlib.pylab as plt\n", "import numpy as np" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Numpy\n", "\n", "The calculations depend on **Numpy** and an installation of that package that is compiled to include BLAS and LAPACK libraries will be much faster than the standalone version.\n", "For example the numpy installed on ubuntu with *> sudo apt-get install python3-numpy* or at windows you can install the numpy package from Gohlke's webpage which compiled against the MKL library of Intel. If you used anaconda3, everything is already optimized.\n", "\n", "The command below lets you see what you have" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Build Dependencies:\n", " blas:\n", " detection method: pkgconfig\n", " found: true\n", " include directory: /c/opt/64/include\n", " lib directory: /c/opt/64/lib\n", " name: openblas64\n", " openblas configuration: USE_64BITINT=1 DYNAMIC_ARCH=1 DYNAMIC_OLDER= NO_CBLAS=\n", " NO_LAPACK= NO_LAPACKE= NO_AFFINITY=1 USE_OPENMP= SKYLAKEX MAX_THREADS=2\n", " pc file directory: C:/opt/64/lib/pkgconfig\n", " version: 0.3.23.dev\n", " lapack:\n", " detection method: internal\n", " found: true\n", " include directory: unknown\n", " lib directory: unknown\n", " name: dep2628220032720\n", " openblas configuration: unknown\n", " pc file directory: unknown\n", " version: 1.26.3\n", "Compilers:\n", " c:\n", " commands: cl\n", " linker: link\n", " name: msvc\n", " version: 19.29.30153\n", " c++:\n", " commands: cl\n", " linker: link\n", " name: msvc\n", " version: 19.29.30153\n", " cython:\n", " commands: cython\n", " linker: cython\n", " name: cython\n", " version: 3.0.7\n", "Machine Information:\n", " build:\n", " cpu: x86_64\n", " endian: little\n", " family: x86_64\n", " system: windows\n", " host:\n", " cpu: x86_64\n", " endian: little\n", " family: x86_64\n", " system: windows\n", "Python Information:\n", " path: C:\\Users\\runneradmin\\AppData\\Local\\Temp\\cibw-run-frm2g0yv\\cp311-win_amd64\\build\\venv\\Scripts\\python.exe\n", " version: '3.11'\n", "SIMD Extensions:\n", " baseline:\n", " - SSE\n", " - SSE2\n", " - SSE3\n", " found:\n", " - SSSE3\n", " - SSE41\n", " - POPCNT\n", " - SSE42\n", " - AVX\n", " - F16C\n", " - FMA3\n", " - AVX2\n", " - AVX512F\n", " - AVX512CD\n", " - AVX512_SKX\n", " - AVX512_CLX\n", " - AVX512_CNL\n", " - AVX512_ICL\n", "\n", "numpy version: 1.26.3\n", "scipy version: 1.11.4\n" ] } ], "source": [ "## What is numpy compiled against\n", "np.__config__.show()\n", "print('numpy version: ',np.version.version)\n", "import scipy as sp\n", "print('scipy version: ',sp.__version__)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## TEM Library\n", "\n", "\n", "You will have to run the code cell below **at least once** to install the library with the programs needed for the analysis of data.\n", "\n", "The code cell below will install pyTEMlib directly from pypi" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "pycharm": { "is_executing": true }, "scrolled": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "installing pyTEMlib\n", "done\n" ] } ], "source": [ "import sys\n", "import importlib.metadata\n", "def test_package(package_name):\n", " \"\"\"Test if package exists and returns version or -1\"\"\"\n", " try:\n", " version = importlib.metadata.version(package_name)\n", " except importlib.metadata.PackageNotFoundError:\n", " version = -1\n", " return version\n", "\n", "# pyTEMlib setup ------------------\n", "if test_package('pyTEMlib') < '0.2023.9.1':\n", " print('installing pyTEMlib')\n", " !{sys.executable} -m pip install --upgrade git+https://github.com/pycroscopy/pyTEMlib.git@main -q --upgrade\n", "\n", "if 'google.colab' in sys.modules:\n", " !{sys.executable} -m pip install numpy==1.24.4\n", "# ------------------------------\n", "print('done')" ] }, { "cell_type": "markdown", "metadata": { "pycharm": { "name": "#%% md\n" } }, "source": [ "Now we load pyTEMlib to make it available for this notebook." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "pyTEM version: 0.2023.1.0\n" ] } ], "source": [ "import pyTEMlib\n", "print(f'pyTEM version: {pyTEMlib.__version__}')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Test\n", "Let's test if the installation was successful and plot a unit cell. You can rotate the plot around, zoom and select. Try it!" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "You don't have igor2 installed. If you wish to open igor files, you will need to install it (pip install igor2) before attempting.\n", "You don't have gwyfile installed. If you wish to open .gwy files, you will need to install it (pip install gwyfile) before attempting.\n", "Symmetry functions of spglib enabled\n", "Using kinematic_scattering library version {_version_ } by G.Duscher\n" ] }, { "data": { "text/plain": [ "Atoms(symbols='C4', pbc=False, cell=[[2.46772414, 0.0, 0.0], [-1.2338620699999996, 2.1371117947721068, 0.0], [0.0, 0.0, 6.711]])" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pyTEMlib.kinematic_scattering as ks # Import kinematic scattering library from pyTEMlib\n", "\n", "# make a structure dictionary\n", "atoms = ks.structure_by_name('Graphite')\n", "\n", "atoms" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from ase.visualize import view\n", "view(atoms*(4,4,1))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Get used to changing parameters and type **silicon** instead of **Graphite** in the code cell above." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Summary\n", "\n", "We now have tested all tools to load data and save our analysis.\n", "\n", "We are ready to go" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Navigation\n", "- **Back [Python as it is used here](CH1_01-Introduction_Python.ipynb)** \n", "- **Next: [Matplotlib and Numpy for Micrographs](CH1_03-Data_Representation.ipynb)** \n", "- **Chapter 1: [Introduction](CH1_00-Introduction.ipynb)** \n", "- **List of Content: [Front](../_MSE672_Intro_TEM.ipynb)** \n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Appendix\n", "\n", "I am using some extensions to jupyter notebooks which can be installed with the following cells\n", "\n", "I like mostly the table of content extension that shows where in the notebook I am and lets me jump to different parts easily.\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Known nbextensions:\n", " config dir: C:\\Users\\gdusc\\anaconda3\\etc\\jupyter\\nbconfig\n", " notebook section\n", " jupyter-jsmol/extension enabled \n", " jupyterlab-plotly/extension enabled \n", " jupyter-js-widgets/extension enabled \n" ] }, { "name": "stderr", "output_type": "stream", "text": [ " - Validating: ok\n", " - Validating: ok\n", " - Validating: ok\n" ] } ], "source": [ "# Install a pip package in the current Jupyter kernel\n", "import sys\n", "!{sys.executable} -m jupyter nbextension list" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Use one of the following (uncomment pip and comment out the conda command line, if you are not in an anaconda environment)" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Collecting package metadata (current_repodata.json): ...working... done\n", "Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.\n", "Collecting package metadata (repodata.json): ...working... done\n", "Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "\n", "PackagesNotFoundError: The following packages are not available from current channels:\n", "\n", " - jupyter_contrib_nbextensions\n", "\n", "Current channels:\n", "\n", " - https://repo.anaconda.com/pkgs/main/win-64\n", " - https://repo.anaconda.com/pkgs/main/noarch\n", " - https://repo.anaconda.com/pkgs/r/win-64\n", " - https://repo.anaconda.com/pkgs/r/noarch\n", " - https://repo.anaconda.com/pkgs/msys2/win-64\n", " - https://repo.anaconda.com/pkgs/msys2/noarch\n", "\n", "To search for alternate channels that may provide the conda package you're\n", "looking for, navigate to\n", "\n", " https://anaconda.org\n", "\n", "and use the search bar at the top of the page.\n", "\n", "\n" ] } ], "source": [ "import sys\n", "\n", "#!{sys.executable} -m pip install jupyter_contrib_nbextensions\n", "!conda install --yes --prefix {sys.prefix} jupyter_contrib_nbextensions" ] }, { "cell_type": "markdown", "metadata": {}, "source": [] }, { "cell_type": "code", "execution_count": 3, "metadata": { "pycharm": { "is_executing": true }, "scrolled": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Enabling notebook extension toc2...\n", " - Validating: problems found:\n", " - require? X toc2\n" ] } ], "source": [ "\n", "!{sys.executable} -m jupyter nbextension enable toc2" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "pycharm": { "is_executing": true } }, "outputs": [ { "data": { "application/javascript": [ "$('
').css({position: 'fixed', top: '120px', left: 0}).appendTo(document.body);\n", "$.getScript('https://kmahelona.github.io/ipython_notebook_goodies/ipython_notebook_toc.js');\n" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "%%javascript\n", "$('
').css({position: 'fixed', top: '120px', left: 0}).appendTo(document.body);\n", "$.getScript('https://kmahelona.github.io/ipython_notebook_goodies/ipython_notebook_toc.js');" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "pyTEMlib", "language": "python", "name": "pytemlib" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.7" }, "toc": { "base_numbering": "3", "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": { "height": "calc(100% - 180px)", "left": "10px", "top": "150px", "width": "242.4px" }, "toc_section_display": true, "toc_window_display": true }, "varInspector": { "cols": { "lenName": 16, "lenType": 16, "lenVar": 40 }, "kernels_config": { "python": { "delete_cmd_postfix": "", "delete_cmd_prefix": "del ", "library": "var_list.py", "varRefreshCmd": "print(var_dic_list())" }, "r": { "delete_cmd_postfix": ") ", "delete_cmd_prefix": "rm(", "library": "var_list.r", "varRefreshCmd": "cat(var_dic_list()) " } }, "types_to_exclude": [ "module", "function", "builtin_function_or_method", "instance", "_Feature" ], "window_display": true }, "vscode": { "interpreter": { "hash": "838e0debddb5b6f29d3d8c39ba50ae8c51920a564d3bac000e89375a158a81de" } } }, "nbformat": 4, "nbformat_minor": 4 }