{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "c8RhFcvahFkU" }, "source": [ "# Performance Optimization in HPC\n", "\n", "## Introduction\n", "\n", "In this notebook, we will explore the fundamental techniques for optimizing code performance in High-Performance Computing (HPC) environments. Performance optimization is crucial for fully exploiting the capabilities of HPC architectures. By understanding and applying these techniques, you can significantly reduce the runtime of your computational tasks, making them more efficient and scalable.\n", "\n", "This practice is essential in HPC as it allows for better resource utilization, reduced costs, and the ability to solve larger and more complex problems. We will cover various optimization strategies, including code profiling, memory hierarchy optimization, and the use of high-performance libraries.\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "8fcZ7APohM10" }, "source": [ "## 2. Optimizing Code for HPC Architectures\n", "\n", "### 2.1 Code Profiling and Analysis\n", "\n", "Before optimizing any code, it's essential to understand where the bottlenecks are. Profiling tools help identify the most time-consuming parts of your code, which are the primary candidates for optimization.\n", "\n", "### 2.2 Loop Unrolling and Vectorization\n", "\n", "Loop unrolling and vectorization are common techniques used to enhance the performance of loops, which are often the most time-consuming parts of computational code.\n", "\n", "### 2.3 Memory Access Patterns and Cache Utilization\n", "\n", "Efficient memory access patterns and effective use of the CPU cache can dramatically speed up your programs.\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 958, "status": "ok", "timestamp": 1724421332715, "user": { "displayName": "Oscar Diez", "userId": "09940749782853693007" }, "user_tz": -120 }, "id": "BGyzMXXnftjM", "outputId": "073e3593-21f0-490c-971d-22a8dd69161c" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " 9 function calls in 6.687 seconds\n", "\n", " Ordered by: standard name\n", "\n", " ncalls tottime percall cumtime percall filename:lineno(function)\n", " 1 0.000 0.000 0.003 0.003 32882468.py:17()\n", " 1 6.685 6.685 6.688 6.688 32882468.py:5(matrix_multiply)\n", " 1 0.002 0.002 6.690 6.690 :1()\n", " 1 0.000 0.000 6.690 6.690 {built-in method builtins.exec}\n", " 4 0.000 0.000 0.000 0.000 {built-in method builtins.len}\n", " 1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}\n", "\n", "\n" ] } ], "source": [ "import cProfile\n", "import random\n", "\n", "# Function for matrix multiplication using basic Python lists\n", "def matrix_multiply(A, B):\n", " # Get the dimensions of the matrices\n", " rows_A = len(A)\n", " cols_A = len(A[0])\n", " rows_B = len(B)\n", " cols_B = len(B[0])\n", " \n", " # Ensure that the number of columns in A is equal to the number of rows in B\n", " if cols_A != rows_B:\n", " raise ValueError(\"Cannot multiply matrices: number of columns in A must be equal to number of rows in B.\")\n", " \n", " # Initialize the result matrix with zeros\n", " result = [[0 for _ in range(cols_B)] for _ in range(rows_A)]\n", " \n", " # Perform matrix multiplication\n", " for i in range(rows_A):\n", " for j in range(cols_B):\n", " for k in range(cols_A):\n", " result[i][j] += A[i][k] * B[k][j]\n", " \n", " return result\n", "\n", "# Function to create a random matrix of size (rows x cols)\n", "def create_random_matrix(rows, cols):\n", " return [[random.random() for _ in range(cols)] for _ in range(rows)]\n", "\n", "# Create large random matrices\n", "A = create_random_matrix(300, 300)\n", "B = create_random_matrix(300, 300)\n", "\n", "# Profile the matrix multiplication\n", "cProfile.run('matrix_multiply(A, B)')\n" ] }, { "cell_type": "markdown", "metadata": { "id": "U_l2qcwJhX9N" }, "source": [ "### Explanation:\n", "\n", "The above code uses Python's `cProfile` to profile a matrix multiplication function. Profiling helps identify the parts of the code that consume the most computational resources, allowing us to focus our optimization efforts effectively.\n" ] }, { "cell_type": "markdown", "metadata": { "id": "CJXcLqLChcOc" }, "source": [ "## 3. Memory Hierarchy and Data Locality\n", "\n", "### 3.1 Understanding Memory Hierarchy\n", "\n", "Memory hierarchy, from registers to cache and RAM, plays a critical role in the performance of HPC applications. Optimizing for memory hierarchy can significantly reduce data access times.\n", "\n", "### 3.2 Data Locality\n", "\n", "Data locality refers to the use of data elements within close proximity in memory, reducing cache misses and improving overall performance.\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 25777, "status": "ok", "timestamp": 1724421406179, "user": { "displayName": "Oscar Diez", "userId": "09940749782853693007" }, "user_tz": -120 }, "id": "afJj6SmghY-E", "outputId": "219df990-271e-4aa9-f737-368e8322b8fe" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Time taken for row-wise sum: 0.03 seconds\n", "Time taken for column-wise sum: 0.85 seconds\n" ] } ], "source": [ "import random\n", "import time\n", "\n", "# Function to create a large random matrix using basic Python lists\n", "def create_random_matrix(rows, cols):\n", " return [[random.random() for _ in range(cols)] for _ in range(rows)]\n", "\n", "# Function to compute the sum of matrix rows (row-wise access)\n", "def row_wise_sum(matrix):\n", " total = []\n", " for row in matrix:\n", " total.append(sum(row))\n", " return total\n", "\n", "# Function to compute the sum of matrix columns (column-wise access)\n", "def column_wise_sum(matrix):\n", " rows = len(matrix)\n", " cols = len(matrix[0])\n", " total = [0] * cols\n", " for col in range(cols):\n", " for row in range(rows):\n", " total[col] += matrix[row][col]\n", " return total\n", "\n", "# Create a large matrix \n", "matrix = create_random_matrix(2000, 2000)\n", "\n", "# Measure row-wise sum performance\n", "start_time = time.time()\n", "row_sums = row_wise_sum(matrix)\n", "end_time = time.time()\n", "print(f\"Time taken for row-wise sum: {end_time - start_time:.2f} seconds\")\n", "\n", "# Measure column-wise sum performance\n", "start_time = time.time()\n", "column_sums = column_wise_sum(matrix)\n", "end_time = time.time()\n", "print(f\"Time taken for column-wise sum: {end_time - start_time:.2f} seconds\")\n" ] }, { "cell_type": "markdown", "metadata": { "id": "vcSAxdv4hf1c" }, "source": [ "### Explanation:\n", "\n", "In the above example, we measure the performance impact of accessing matrix elements row-wise versus column-wise. Due to the way memory is structured, row-wise access is typically faster because it accesses contiguous memory locations, which is more cache-friendly.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Understanding Modules in HPC Environments\n", "\n", "In High-Performance Computing (HPC) environments, managing software dependencies can be complex due to the variety of libraries and software packages that different applications may require. The **Modules** system is designed to simplify this process. It allows users to dynamically modify their environment (e.g., `PATH`, `LD_LIBRARY_PATH`) to load or unload specific software packages or libraries on-demand, ensuring compatibility and reproducibility across different applications.\n", "\n", "Modules make it easier to:\n", "- Load specific versions of software or libraries\n", "- Avoid conflicts between different software versions\n", "- Automatically set environment variables required by certain software\n", "\n", "### Main Module Commands Explained with Examples\n", "\n", "1. **Listing Available Modules**\n", " - This command shows a list of all modules currently available on the cluster that can be loaded into your environment.\n", " - **Command:**\n", " ```bash\n", " module avail\n", " ```\n", " - **Example Output:**\n", " ```bash\n", " -------------------- /opt/modulefiles --------------------\n", " openblas/0.3.17 fftw/3.3.8 gcc/9.3.0 python/3.8.10\n", " intel-mkl/2021.2 cuda/11.2 hdf5/1.10.6\n", " ```\n", " This output lists available versions of software and libraries like OpenBLAS, FFTW, and Python.\n", "\n", "---\n", "\n", "2. **Loading a Module**\n", " - To use a specific software package, load its corresponding module. This automatically sets the necessary environment variables.\n", " - **Command:**\n", " ```bash\n", " module load /\n", " ```\n", " - **Example:**\n", " ```bash\n", " module load openblas/0.3.17\n", " ```\n", " This command loads OpenBLAS version 0.3.17 into your environment.\n", "\n", "---\n", "\n", "3. **Unloading a Module**\n", " - To remove a module from your environment, use the `unload` command.\n", " - **Command:**\n", " ```bash\n", " module unload /\n", " ```\n", " - **Example:**\n", " ```bash\n", " module unload openblas/0.3.17\n", " ```\n", " This will unload OpenBLAS from your environment, removing any changes it made to your environment variables.\n", "\n", "---\n", "\n", "4. **Listing Loaded Modules**\n", " - To check which modules are currently loaded in your environment, use the `list` command.\n", " - **Command:**\n", " ```bash\n", " module list\n", " ```\n", " - **Example Output:**\n", " ```bash\n", " Currently Loaded Modules:\n", " 1) gcc/9.3.0 2) openblas/0.3.17 3) python/3.8.10\n", " ```\n", " This shows that GCC, OpenBLAS, and Python are currently loaded.\n", "\n", "---\n", "\n", "5. **Switching Between Module Versions**\n", " - You can switch between different versions of a module using the `swap` command.\n", " - **Command:**\n", " ```bash\n", " module swap / /\n", " ```\n", " - **Example:**\n", " ```bash\n", " module swap openblas/0.3.17 openblas/0.3.9\n", " ```\n", " This will unload OpenBLAS version 0.3.17 and load version 0.3.9.\n", "\n", "---\n", "\n", "6. **Getting Information About a Module**\n", " - The `show` command provides detailed information about a module, including the environment variables it modifies and paths to executables or libraries.\n", " - **Command:**\n", " ```bash\n", " module show /\n", " ```\n", " - **Example:**\n", " ```bash\n", " module show openblas/0.3.17\n", " ```\n", " - **Example Output:**\n", " ```bash\n", " -------------------------------------------------------------------\n", " /opt/modulefiles/openblas/0.3.17:\n", "\n", " module-whatis \"OpenBLAS: An optimized BLAS library\"\n", " prepend-path PATH /opt/openblas/0.3.17/bin\n", " prepend-path LD_LIBRARY_PATH /opt/openblas/0.3.17/lib\n", " setenv OPENBLAS_VERSION 0.3.17\n", " -------------------------------------------------------------------\n", " ```\n", " This output shows how OpenBLAS modifies your environment when loaded, such as adding directories to the `PATH` and `LD_LIBRARY_PATH`.\n", "\n", "---\n", "\n", "7. **Searching for Modules**\n", " - If you're not sure about the exact name or version of a module, you can search for it using the `spider` command.\n", " - **Command:**\n", " ```bash\n", " module spider \n", " ```\n", " - **Example:**\n", " ```bash\n", " module spider openblas\n", " ```\n", " This will list all available versions of OpenBLAS and show how to load them.\n", "\n", "---\n", "\n", "8. **Purging All Loaded Modules**\n", " - If you want to remove all loaded modules and reset your environment to its default state, use the `purge` command.\n", " - **Command:**\n", " ```bash\n", " module purge\n", " ```\n", " - **Example:**\n", " ```bash\n", " module purge\n", " ```\n", " This will unload all currently loaded modules, restoring your environment to its initial state.\n", "\n", "---\n", "\n", "9. **Saving and Restoring Module Sets**\n", " - You can save the current set of loaded modules to easily restore them later. This is helpful when working on multiple projects that require different sets of modules.\n", " - **Save the current module environment**:\n", " ```bash\n", " module save \n", " ```\n", " - **Restore a saved module environment**:\n", " ```bash\n", " module restore \n", " ```\n", " - **Example:**\n", " ```bash\n", " module save my-project\n", " module restore my-project\n", " ```\n", " The `save` command saves the current modules as a named set (`my-project`), and the `restore` command reloads that set when needed.\n", "\n", "---\n", "\n", "### Example: Loading BLAS and LAPACK Libraries\n", "\n", "In HPC, numerical libraries like **BLAS** (Basic Linear Algebra Subprograms) and **LAPACK** (Linear Algebra Package) are often used for linear algebra computations. These libraries are highly optimized and may be provided by modules such as **OpenBLAS** or **Intel MKL**.\n", "\n", "Here is an example of how to load the `openblas/0.3.17` module for a program that requires BLAS and LAPACK:\n", "\n", "```bash\n", "# Load the OpenBLAS module\n", "module load openblas/0.3.17\n", "\n", "# Verify that the module has been loaded\n", "module list\n" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "------------------------ MPI-dependent avx512 modules -------------------------\n", " abyss/2.2.5 (\u001b[1;31mbio\u001b[0m)\n", " adol-c/2.7.2\n", " alpscore/2.2.0 (\u001b[1;36mphys\u001b[0m,D)\n", " ambertools/20 (\u001b[1;35mchem\u001b[0m)\n", " ambertools/21 (\u001b[1;35mchem\u001b[0m)\n", " ambertools/23 (\u001b[1;35mchem\u001b[0m,D)\n", " apbs/1.3 (\u001b[1;35mchem\u001b[0m)\n", " arpack-ng/3.9.0 (\u001b[1;32mmath\u001b[0m,D)\n", " aspect/2.4.0\n", " astrid/2.2.1\n", " blacs/1.1 (\u001b[1;32mmath\u001b[0m)\n", " boost-mpi/1.72.0 (\u001b[1;34mt\u001b[0m)\n", " boost-mpi/1.80.0 (\u001b[1;34mt\u001b[0m,D)\n", " cantera/2.5.1 (\u001b[1;35mchem\u001b[0m)\n", " cantera/2.6.0 (\u001b[1;35mchem\u001b[0m,D)\n", " cdo/1.9.8 (\u001b[1;36mgeo\u001b[0m)\n", " cdo/2.0.4 (\u001b[1;36mgeo\u001b[0m)\n", " cdo/2.0.5 (\u001b[1;36mgeo\u001b[0m,D)\n", " cgns/3.4.1 (\u001b[1;36mphys\u001b[0m)\n", " cgns/4.1.0 (\u001b[1;36mphys\u001b[0m)\n", " cgns/4.1.2 (\u001b[1;36mphys\u001b[0m,D)\n", " cp2k/8.2 (\u001b[1;35mchem\u001b[0m)\n", " cp2k/9.1 (\u001b[1;35mchem\u001b[0m,D)\n", " dakota/6.13 (\u001b[1;34mt\u001b[0m)\n", " dealii/9.2.0 (\u001b[1;32mmath\u001b[0m)\n", " dealii/9.3.1 (\u001b[1;32mmath\u001b[0m,D)\n", " dl_monte/2.07\n", " elmerfem/scc20\n", " elmerfem/8.4\n", " elmerfem/9.0 (D)\n", " esmf/8.0.1 (\u001b[1;36mgeo\u001b[0m)\n", " esmf/8.2.0 (\u001b[1;36mgeo\u001b[0m)\n", " esmf/8.4.0 (\u001b[1;36mgeo\u001b[0m,D)\n", " ferret/7.3 (\u001b[1;34mvis\u001b[0m)\n", " fftw-mpi/3.3.8 (\u001b[1;32mmath\u001b[0m)\n", " gem/5.1.1\n", " gerris/20131206\n", " glost/0.3.1 (\u001b[1;34mt\u001b[0m)\n", " gromacs-colvars/2020.6 (\u001b[1;35mchem\u001b[0m)\n", " gromacs-cp2k/2022.2 (\u001b[1;35mchem\u001b[0m)\n", " gromacs-plumed/2019.6 (\u001b[1;35mchem\u001b[0m)\n", " gromacs-plumed/2021.2 (\u001b[1;35mchem\u001b[0m)\n", " gromacs-plumed/2021.4 (\u001b[1;35mchem\u001b[0m)\n", " gromacs-plumed/2021.6 (\u001b[1;35mchem\u001b[0m)\n", " gromacs-plumed/2021.7 (\u001b[1;35mchem\u001b[0m)\n", " gromacs-plumed/2022.3 (\u001b[1;35mchem\u001b[0m)\n", " gromacs-plumed/2022.6 (\u001b[1;35mchem\u001b[0m,D)\n", " gromacs-ramd/2020.5-RAMD-2.0 (\u001b[1;35mchem\u001b[0m)\n", " gromacs/2020.4 (\u001b[1;35mchem\u001b[0m)\n", " gromacs/2020.6 (\u001b[1;35mchem\u001b[0m)\n", " gromacs/2021.2 (\u001b[1;35mchem\u001b[0m)\n", " gromacs/2021.4 (\u001b[1;35mchem\u001b[0m)\n", " gromacs/2021.6 (\u001b[1;35mchem\u001b[0m)\n", " gromacs/2022.2 (\u001b[1;35mchem\u001b[0m)\n", " gromacs/2022.3 (\u001b[1;35mchem\u001b[0m)\n", " gromacs/2023 (\u001b[1;35mchem\u001b[0m)\n", " gromacs/2023.2 (\u001b[1;35mchem\u001b[0m,D)\n", " hdf5-mpi/1.10.6 (\u001b[1;33mio\u001b[0m)\n", " hdf5-mpi/1.12.1 (\u001b[1;33mio\u001b[0m,D)\n", " hh-suite/3.3.0 (\u001b[1;31mbio\u001b[0m,D)\n", " hoomd-blue-mpi/3.6.0\n", " hoomd-blue/2.9.3\n", " hpcspades/3.15.0\n", " hpctoolkit/2020.08\n", " hyphy/2.5.26 (\u001b[1;31mbio\u001b[0m)\n", " hyphy/2.5.49 (\u001b[1;31mbio\u001b[0m,D)\n", " ima3/1.12\n", " ima3/20210120 (D)\n", " jdftx/1.7.0 (\u001b[1;35mchem\u001b[0m)\n", " kahip/3.14\n", " libcf/1.0.3\n", " libgridxc-mpi/0.8.0\n", " libmesh/1.7.1 (\u001b[1;32mmath\u001b[0m)\n", " mafft-mpi/7.471\n", " manta/1.6.0 (\u001b[1;31mbio\u001b[0m)\n", " meep/1.16.1 (\u001b[1;36mphys\u001b[0m)\n", " meep/1.24.0 (\u001b[1;36mphys\u001b[0m)\n", " meep/1.25.0 (\u001b[1;36mphys\u001b[0m,D)\n", " meme/5.2.0\n", " meme/5.4.1\n", " meme/5.5.0\n", " meme/5.5.5 (D)\n", " metaeuk/4-a0f584d\n", " migrate-n/4.4.4\n", " minimac4/1.0.2\n", " mpb-mpi/1.11.1\n", " mpi4py/3.0.3 (\u001b[1;34mt\u001b[0m)\n", " mpi4py/3.1.2 (\u001b[1;34mt\u001b[0m)\n", " mpi4py/3.1.3 (\u001b[1;34mt\u001b[0m,D)\n", " mrbayes/3.2.7 (\u001b[1;31mbio\u001b[0m)\n", " mumps-metis/5.2.1 (\u001b[1;34mt\u001b[0m)\n", " mumps-parmetis/5.3.5 (\u001b[1;34mt\u001b[0m)\n", " ncl/6.6.2 (\u001b[1;34mvis\u001b[0m)\n", " ncview/2.1.8 (\u001b[1;34mvis\u001b[0m)\n", " nektar++/5.3.0 (\u001b[1;32mmath\u001b[0m)\n", " netcdf-c++-mpi/4.2 (\u001b[1;33mio\u001b[0m)\n", " netcdf-c++4-mpi/4.3.1 (\u001b[1;33mio\u001b[0m)\n", " netcdf-fortran-mpi/4.5.2 (\u001b[1;33mio\u001b[0m)\n", " netcdf-fortran-mpi/4.6.0 (\u001b[1;33mio\u001b[0m,D)\n", " netcdf-mpi/4.7.4 (\u001b[1;33mio\u001b[0m)\n", " netcdf-mpi/4.9.0 (\u001b[1;33mio\u001b[0m,D)\n", " neuron/7.8.2 (\u001b[1;31mbio\u001b[0m)\n", " neuron/8.0.0 (\u001b[1;31mbio\u001b[0m,D)\n", " opencarp/4.0\n", " opencascade/7.5.2 (D)\n", " opencoarrays/2.9.2\n", " openfoam/v2006 (\u001b[1;36mphys\u001b[0m)\n", " openfoam/v2012 (\u001b[1;36mphys\u001b[0m)\n", " openfoam/v2112 (\u001b[1;36mphys\u001b[0m)\n", " openfoam/v2206 (\u001b[1;36mphys\u001b[0m)\n", " openfoam/v2212 (\u001b[1;36mphys\u001b[0m)\n", " openfoam/v2306 (\u001b[1;36mphys\u001b[0m)\n", " openfoam/8 (\u001b[1;36mphys\u001b[0m)\n", " openfoam/9 (\u001b[1;36mphys\u001b[0m)\n", " openfoam/10 (\u001b[1;36mphys\u001b[0m)\n", " openfoam/11 (\u001b[1;36mphys\u001b[0m,D)\n", " openmc/0.13.2\n", " openmc/0.13.3 (D)\n", " openmm-alphafold/7.5.1\n", " openmm/7.5.0 (\u001b[1;35mchem\u001b[0m)\n", " openmm/7.7.0 (\u001b[1;35mchem\u001b[0m)\n", " openmm/8.0.0 (\u001b[1;35mchem\u001b[0m,D)\n", " opensees/3.2.0\n", " opensees/3.5.0 (D)\n", " p4est/2.2 (\u001b[1;32mmath\u001b[0m)\n", " p4est/2.3.2 (\u001b[1;32mmath\u001b[0m)\n", " p4est/2.8.5 (\u001b[1;32mmath\u001b[0m,D)\n", " paraview-offscreen-gpu/5.8.0 (\u001b[1;34mvis\u001b[0m)\n", " paraview-offscreen-gpu/5.9.1 (\u001b[1;34mvis\u001b[0m)\n", " paraview-offscreen-gpu/5.10.0 (\u001b[1;34mvis\u001b[0m)\n", " paraview-offscreen-gpu/5.11.0 (\u001b[1;34mvis\u001b[0m,D)\n", " paraview-offscreen/5.8.0 (\u001b[1;34mvis\u001b[0m)\n", " paraview-offscreen/5.9.1 (\u001b[1;34mvis\u001b[0m)\n", " paraview-offscreen/5.10.0 (\u001b[1;34mvis\u001b[0m)\n", " paraview-offscreen/5.11.0 (\u001b[1;34mvis\u001b[0m,D)\n", " paraview/5.8.0 (\u001b[1;34mvis\u001b[0m)\n", " paraview/5.9.1 (\u001b[1;34mvis\u001b[0m)\n", " paraview/5.10.0 (\u001b[1;34mvis\u001b[0m)\n", " paraview/5.11.0 (\u001b[1;34mvis\u001b[0m,D)\n", " parmetis/4.0.3 (\u001b[1;32mmath\u001b[0m)\n", " parmgridgen/1.0 (\u001b[1;32mmath\u001b[0m)\n", " petsc-64bits/3.17.1 (\u001b[1;34mt\u001b[0m)\n", " petsc-pardiso-64bits/3.17.1\n", " petsc-pardiso/3.17.1\n", " petsc/3.12.4 (\u001b[1;34mt\u001b[0m)\n", " petsc/3.13.3 (\u001b[1;34mt\u001b[0m)\n", " petsc/3.13.6 (\u001b[1;34mt\u001b[0m)\n", " petsc/3.14.1 (\u001b[1;34mt\u001b[0m)\n", " petsc/3.15.0 (\u001b[1;34mt\u001b[0m)\n", " petsc/3.17.1 (\u001b[1;34mt\u001b[0m,D)\n", " pflotran/4.0\n", " phyml/3.3.20190321\n", " plumed/2.6.1 (\u001b[1;35mchem\u001b[0m)\n", " plumed/2.6.2 (\u001b[1;35mchem\u001b[0m)\n", " plumed/2.7.0 (\u001b[1;35mchem\u001b[0m)\n", " plumed/2.7.1 (\u001b[1;35mchem\u001b[0m)\n", " plumed/2.7.2 (\u001b[1;35mchem\u001b[0m)\n", " plumed/2.7.3 (\u001b[1;35mchem\u001b[0m)\n", " plumed/2.7.4 (\u001b[1;35mchem\u001b[0m)\n", " plumed/2.7.6 (\u001b[1;35mchem\u001b[0m)\n", " plumed/2.8.1 (\u001b[1;35mchem\u001b[0m)\n", " plumed/2.8.3 (\u001b[1;35mchem\u001b[0m,D)\n", " pnetcdf/1.12.2 (\u001b[1;33mio\u001b[0m)\n", " quantumespresso/6.6 (\u001b[1;35mchem\u001b[0m)\n", " quantumespresso/6.7 (\u001b[1;35mchem\u001b[0m)\n", " quantumespresso/6.8 (\u001b[1;35mchem\u001b[0m)\n", " quantumespresso/7.0 (\u001b[1;35mchem\u001b[0m)\n", " quantumespresso/7.1 (\u001b[1;35mchem\u001b[0m)\n", " quantumespresso/7.2 (\u001b[1;35mchem\u001b[0m,D)\n", " raxml-ng/1.0.1\n", " raxml/8.2.12 (\u001b[1;31mbio\u001b[0m)\n", " ray/3.0.1 (\u001b[1;31mbio\u001b[0m)\n", " relion/3.1.1 (\u001b[1;35mchem\u001b[0m)\n", " relion/4.0.0 (\u001b[1;35mchem\u001b[0m,D)\n", " salmon/1.3.0 (\u001b[1;31mbio\u001b[0m)\n", " salmon/1.4.0 (\u001b[1;31mbio\u001b[0m,D)\n", " scalapack/2.1.0 (\u001b[1;32mmath\u001b[0m)\n", " scalasca/2.5\n", " score-p/6.0\n", " scotch-no-thread/6.1.2\n", " scotch-no-thread/7.0.3 (D)\n", " scotch/6.0.9 (\u001b[1;32mmath\u001b[0m)\n", " scotch/7.0.3 (\u001b[1;32mmath\u001b[0m,D)\n", " shapeit4/4.2.0\n", " shapeit4/4.2.1 (D)\n", " slepc/3.14.2\n", " slepc/3.17.2 (D)\n", " sortmerna/4.2.0 (\u001b[1;31mbio\u001b[0m)\n", " sortmerna/4.3.4 (\u001b[1;31mbio\u001b[0m)\n", " su2/7.0.8 (\u001b[1;32mmath\u001b[0m)\n", " su2/7.5.1 (\u001b[1;32mmath\u001b[0m,D)\n", " sundials/2.7.0\n", " sundials/5.3.0\n", " sundials/5.7.0\n", " sundials/6.4.1 (D)\n", " tau/2.30.1 (\u001b[1;34mt\u001b[0m)\n", " tophat/2.1.2 (\u001b[1;31mbio\u001b[0m,D)\n", " towhee/8.2.3 (\u001b[1;35mchem\u001b[0m)\n", " trans-abyss/2.0.1 (\u001b[1;31mbio\u001b[0m)\n", " trilinos/13.0.1 (\u001b[1;34mt\u001b[0m)\n", " trilinos/13.2.0 (\u001b[1;34mt\u001b[0m)\n", " trilinos/13.4.0 (\u001b[1;34mt\u001b[0m)\n", " trilinos/13.4.1 (\u001b[1;34mt\u001b[0m,D)\n", " valgrind-mpi/3.16.1 (\u001b[1;34mt\u001b[0m)\n", " visit/3.2.1 (\u001b[1;34mvis\u001b[0m,D)\n", " vtk-mpi/9.0.1 (\u001b[1;34mvis\u001b[0m)\n", " vtk-mpi/9.0.3 (\u001b[1;34mvis\u001b[0m,D)\n", " wannier90/3.1.0 (\u001b[1;35mchem\u001b[0m)\n", " yaxt/0.9.0 (\u001b[1;34mt\u001b[0m)\n", "\n", "---------------------- Compiler-dependent avx512 modules ----------------------\n", " 4ti2/1.6.9\n", " abricate/1.0.0\n", " afni/20.3.05 (\u001b[1;31mbio\u001b[0m)\n", " afni/21.2.10 (\u001b[1;31mbio\u001b[0m)\n", " afni/22.1.12 (\u001b[1;31mbio\u001b[0m)\n", " afni/23.1.00 (\u001b[1;31mbio\u001b[0m)\n", " afni/23.1.08 (\u001b[1;31mbio\u001b[0m,D)\n", " alm/2.0.0_dev.2\n", " alpscore/2.2.0 (\u001b[1;36mphys\u001b[0m)\n", " amrfinderplus/3.11.18\n", " amrfinderplus/3.11.26 (D)\n", " amrplusplus/2.0-20200114\n", " ants/2.3.2 (\u001b[1;34mvis\u001b[0m)\n", " ants/2.3.5 (\u001b[1;34mvis\u001b[0m)\n", " ants/2.4.4 (\u001b[1;34mvis\u001b[0m,D)\n", " aragorn/1.2.38\n", " arcs/1.2.1 (\u001b[1;31mbio\u001b[0m)\n", " arcs/1.2.5 (\u001b[1;31mbio\u001b[0m,D)\n", " arks/1.0.4\n", " armadillo/9.900.2 (\u001b[1;32mmath\u001b[0m)\n", " armadillo/12.6.4 (\u001b[1;32mmath\u001b[0m,D)\n", " arpack-ng/3.7.0 (\u001b[1;32mmath\u001b[0m)\n", " arpack-ng/3.8.0 (\u001b[1;32mmath\u001b[0m)\n", " arrow/0.16.0 (\u001b[1;34mt\u001b[0m)\n", " arrow/0.17.1 (\u001b[1;34mt\u001b[0m)\n", " arrow/1.0.0 (\u001b[1;34mt\u001b[0m)\n", " arrow/2.0.0 (\u001b[1;34mt\u001b[0m)\n", " arrow/4.0.0 (\u001b[1;34mt\u001b[0m)\n", " arrow/5.0.0 (\u001b[1;34mt\u001b[0m)\n", " arrow/7.0.0 (\u001b[1;34mt\u001b[0m)\n", " arrow/8.0.0 (\u001b[1;34mt\u001b[0m)\n", " arrow/9.0.0 (\u001b[1;34mt\u001b[0m)\n", " arrow/10.0.1 (\u001b[1;34mt\u001b[0m)\n", " arrow/11.0.0 (\u001b[1;34mt\u001b[0m)\n", " arrow/12.0.1 (\u001b[1;34mt\u001b[0m)\n", " arrow/13.0.0 (\u001b[1;34mt\u001b[0m,D)\n", " atat/3.36\n", " atom/4.2.7_2 (\u001b[1;35mchem\u001b[0m)\n", " atomicrex/1.0.20181114 (\u001b[1;35mchem\u001b[0m)\n", " augustus/3.3.3 (\u001b[1;31mbio\u001b[0m)\n", " augustus/3.4.0 (\u001b[1;31mbio\u001b[0m)\n", " augustus/3.5.0 (\u001b[1;31mbio\u001b[0m,D)\n", " bayesass/3.0.4 (\u001b[1;31mbio\u001b[0m)\n", " bayesass3-snps/1.1\n", " baypass/2.2 (\u001b[1;31mbio\u001b[0m)\n", " bcftools/1.9 (\u001b[1;31mbio\u001b[0m)\n", " bcftools/1.10.2 (\u001b[1;31mbio\u001b[0m)\n", " bcftools/1.11 (\u001b[1;31mbio\u001b[0m)\n", " bcftools/1.13 (\u001b[1;31mbio\u001b[0m)\n", " bcftools/1.16 (\u001b[1;31mbio\u001b[0m,D)\n", " beagle-lib/3.1.2 (\u001b[1;31mbio\u001b[0m)\n", " beagle-lib/4.0.0 (\u001b[1;31mbio\u001b[0m,D)\n", " beast/2.6.3 (\u001b[1;31mbio\u001b[0m)\n", " beef/0.1.1 (\u001b[1;35mchem\u001b[0m)\n", " bgcdist/1.03\n", " bgen-lib/1.1.7\n", " bio-searchio-hmmer/1.7.3\n", " biobloomtools/2.3.2-20200731\n", " blasr/5.3.3 (\u001b[1;31mbio\u001b[0m)\n", " blasr_libcpp/5.3.4 (\u001b[1;31mbio\u001b[0m)\n", " blast+/2.10.1 (\u001b[1;31mbio\u001b[0m)\n", " blast+/2.11.0 (\u001b[1;31mbio\u001b[0m)\n", " blast+/2.12.0 (\u001b[1;31mbio\u001b[0m)\n", " blast+/2.13.0 (\u001b[1;31mbio\u001b[0m,D)\n", " blender/2.92.0 (\u001b[1;34mvis\u001b[0m)\n", " blis/0.8.1\n", " bmtagger/3.101\n", " bolt-lmm/2.3.4 (\u001b[1;31mbio\u001b[0m)\n", " bolt-lmm/2.4 (\u001b[1;31mbio\u001b[0m,D)\n", " boost/1.72.0 (\u001b[1;34mt\u001b[0m)\n", " boost/1.80.0 (\u001b[1;34mt\u001b[0m,D)\n", " bracken/2.6.0\n", " bufrlib/11.3.0.2 (\u001b[1;36mgeo\u001b[0m)\n", " bullet/2.89\n", " bullet/3.24 (D)\n", " casper/0.8.2 (\u001b[1;31mbio\u001b[0m)\n", " centrifuge/1.0.4 (\u001b[1;31mbio\u001b[0m,D)\n", " ceres-solver/1.14.0\n", " cgal/4.14.3 (\u001b[1;32mmath\u001b[0m)\n", " cgal/5.2.4 (\u001b[1;32mmath\u001b[0m)\n", " chapel-ofi/1.31.0\n", " chemps2/1.8.9 (\u001b[1;35mchem\u001b[0m)\n", " clhep/2.4.1.3 (\u001b[1;32mmath\u001b[0m)\n", " clhep/2.4.4.0 (\u001b[1;32mmath\u001b[0m)\n", " clhep/2.4.6.2 (\u001b[1;32mmath\u001b[0m,D)\n", " clustal-omega/1.2.4 (\u001b[1;31mbio\u001b[0m)\n", " coordgenlibs/1.4.2 (\u001b[1;35mchem\u001b[0m)\n", " coretran/1.0.1 (D)\n", " crest/2.11 (\u001b[1;35mchem\u001b[0m)\n", " ctffind/4.1.14 (\u001b[1;35mchem\u001b[0m)\n", " cubic_interpolation/0.1.5\n", " cufflinks/2.2.1 (\u001b[1;31mbio\u001b[0m)\n", " delly/1.1.6 (D)\n", " diamond/2.0.13 (\u001b[1;31mbio\u001b[0m)\n", " diamond/2.0.15 (\u001b[1;31mbio\u001b[0m)\n", " dragmap/1.3.0\n", " dssp/2.3.0 (\u001b[1;35mchem\u001b[0m)\n", " dssp/3.1.4 (\u001b[1;35mchem\u001b[0m,D)\n", " eccodes/2.15.0 (\u001b[1;36mgeo\u001b[0m)\n", " eccodes/2.19.0 (\u001b[1;36mgeo\u001b[0m)\n", " eccodes/2.25.0 (\u001b[1;36mgeo\u001b[0m,D)\n", " eigensoft/7.2.1 (\u001b[1;31mbio\u001b[0m)\n", " elastix/5.0.1\n", " embree/3.11.0\n", " embree/3.13.5 (D)\n", " exonerate/2.4.0 (\u001b[1;31mbio\u001b[0m)\n", " fastani/1.32\n", " fastspar/1.0.0\n", " fftw/3.3.8 (\u001b[1;32mmath\u001b[0m)\n", " flash/1.2.11\n", " flashpca/2.0\n", " freesasa/2.1.0\n", " fsl/6.0.3 (\u001b[1;31mbio\u001b[0m)\n", " fsl/6.0.4 (\u001b[1;31mbio\u001b[0m,D)\n", " gazebo/11.7.0\n", " gcta/1.26.0 (\u001b[1;31mbio\u001b[0m)\n", " gcta/1.93.2 (\u001b[1;31mbio\u001b[0m,D)\n", " gdal/3.0.4 (\u001b[1;36mgeo\u001b[0m)\n", " gdal/3.2.3 (\u001b[1;36mgeo\u001b[0m)\n", " gdal/3.4.1 (\u001b[1;36mgeo\u001b[0m)\n", " gdal/3.5.1 (\u001b[1;36mgeo\u001b[0m,D)\n", " gdcm/2.6.8\n", " gdcm/3.0.8 (D)\n", " geant4-seq/11.1.0\n", " geant4-topasmc3.9/10.7.3\n", " geant4/10.06 (\u001b[1;36mphys\u001b[0m)\n", " geant4/10.7.3 (\u001b[1;36mphys\u001b[0m)\n", " geant4/11.1.0 (\u001b[1;36mphys\u001b[0m,D)\n", " gemma/0.98.3 (\u001b[1;31mbio\u001b[0m)\n", " genometools/1.6.1 (\u001b[1;31mbio\u001b[0m)\n", " geopsy/3.4.2\n", " geos/3.8.1 (\u001b[1;36mgeo\u001b[0m)\n", " gmsh/4.7.0 (\u001b[1;36mphys\u001b[0m)\n", " gmsh/4.10.5 (\u001b[1;36mphys\u001b[0m)\n", " gmsh/4.11.1 (\u001b[1;36mphys\u001b[0m,D)\n", " gmtk/1.4.4\n", " grace/5.99.0 (\u001b[1;34mvis\u001b[0m)\n", " graph-tool/2.37\n", " graph-tool/2.45\n", " graph-tool/2.56 (D)\n", " grass/7.8.4 (\u001b[1;36mgeo\u001b[0m)\n", " grass/8.2.1 (\u001b[1;36mgeo\u001b[0m,D)\n", " gsl/1.16 (\u001b[1;32mmath\u001b[0m)\n", " gsl/2.6 (\u001b[1;32mmath\u001b[0m,D)\n", " gudhi/3.4.1\n", " hal/2.2 (\u001b[1;31mbio\u001b[0m)\n", " hdf-eos5/5.1.16\n", " hdf5/1.8.22 (\u001b[1;33mio\u001b[0m)\n", " hdf5/1.10.6 (\u001b[1;33mio\u001b[0m)\n", " hdf5/1.12.1 (\u001b[1;33mio\u001b[0m,D)\n", " heasoft/6.32.1 (\u001b[1;36mphys\u001b[0m)\n", " hoomd-blue/2.9.3\n", " hoomd-blue/3.6.0 (D)\n", " htslib/1.13 (\u001b[1;31mbio\u001b[0m)\n", " ignition/citadel\n", " infernal/1.1.3 (\u001b[1;31mbio\u001b[0m)\n", " infernal/1.1.4 (\u001b[1;31mbio\u001b[0m,D)\n", " ioapi/3.2-2020111\n", " iq-tree/1.6.12 (\u001b[1;31mbio\u001b[0m)\n", " iq-tree/2.0.7 (\u001b[1;31mbio\u001b[0m)\n", " iq-tree/2.1.2 (\u001b[1;31mbio\u001b[0m)\n", " iq-tree/2.2.1 (\u001b[1;31mbio\u001b[0m)\n", " iq-tree/2.2.2.7 (\u001b[1;31mbio\u001b[0m,D)\n", " isce2/2.6.3\n", " itk/4.13.3 (\u001b[1;34mvis\u001b[0m)\n", " itk/5.0.1 (\u001b[1;34mvis\u001b[0m)\n", " itk/5.1.2 (\u001b[1;34mvis\u001b[0m)\n", " itk/5.2.1 (\u001b[1;34mvis\u001b[0m,D)\n", " jags/4.3.0 (\u001b[1;32mmath\u001b[0m)\n", " jags/4.3.2 (\u001b[1;32mmath\u001b[0m,D)\n", " kahypar/1.3.2\n", " kaiju/1.6.2 (\u001b[1;31mbio\u001b[0m)\n", " kaiju/1.7.4 (\u001b[1;31mbio\u001b[0m,D)\n", " kalign/2.03\n", " kallisto/0.46.1 (\u001b[1;31mbio\u001b[0m)\n", " kim-api/2.1.3 (\u001b[1;35mchem\u001b[0m)\n", " komplexity/0.3.6\n", " kraken2/2.0.9-beta (\u001b[1;31mbio\u001b[0m)\n", " kraken2/2.1.1 (\u001b[1;31mbio\u001b[0m)\n", " kraken2/2.1.2 (\u001b[1;31mbio\u001b[0m)\n", " kraken2/2.1.3 (\u001b[1;31mbio\u001b[0m,D)\n", " last/1145 (\u001b[1;31mbio\u001b[0m)\n", " libcdms/3.1.2\n", " libdrs/3.1.2\n", " libgridxc/0.8.0 (\u001b[1;35mchem\u001b[0m)\n", " libint/1.1.6 (\u001b[1;35mchem\u001b[0m)\n", " libint/2.4.2 (\u001b[1;35mchem\u001b[0m)\n", " libint/2.6.0 (\u001b[1;35mchem\u001b[0m,D)\n", " libmaus2/2.0.499 (\u001b[1;31mbio\u001b[0m)\n", " libxc/3.0.1 (\u001b[1;35mchem\u001b[0m)\n", " libxc/4.3.4 (\u001b[1;35mchem\u001b[0m)\n", " libxc/5.0.0 (\u001b[1;35mchem\u001b[0m)\n", " libxc/5.1.3 (\u001b[1;35mchem\u001b[0m)\n", " libxc/5.2.3 (\u001b[1;35mchem\u001b[0m,D)\n", " libxsmm/1.16.1 (\u001b[1;32mmath\u001b[0m)\n", " lpsolve/5.5.2.5 (\u001b[1;32mmath\u001b[0m,D)\n", " ltr_retriever/2.9.0\n", " m2/1.19.1\n", " m2/1.21 (D)\n", " maeparser/1.2.4 (\u001b[1;35mchem\u001b[0m)\n", " marginpolish/1.3.0\n", " mash/2.3\n", " masurca/3.4.1 (\u001b[1;31mbio\u001b[0m)\n", " masurca/4.0.1 (\u001b[1;31mbio\u001b[0m)\n", " masurca/4.0.3 (\u001b[1;31mbio\u001b[0m)\n", " masurca/4.1.0 (\u001b[1;31mbio\u001b[0m,D)\n", " maxbin/2.2.7 (\u001b[1;31mbio\u001b[0m)\n", " mcl/14.137 (\u001b[1;32mmath\u001b[0m)\n", " metabat/2.14 (\u001b[1;31mbio\u001b[0m,D)\n", " metal/2011-03-25 (\u001b[1;31mbio\u001b[0m)\n", " metaxa2/2.2\n", " minc-toolkit/1.9.18.1 (\u001b[1;31mbio\u001b[0m)\n", " mmseqs2/13-45111\n", " mothur/1.44.3 (\u001b[1;31mbio\u001b[0m)\n", " mothur/1.46.1 (\u001b[1;31mbio\u001b[0m)\n", " mothur/1.47.0 (\u001b[1;31mbio\u001b[0m)\n", " mothur/1.48.0 (\u001b[1;31mbio\u001b[0m,D)\n", " mpb/1.11.1\n", " mrtrix/3.0.1 (\u001b[1;31mbio\u001b[0m)\n", " msmc2/2.1.3\n", " nanopolish/0.11.2 (\u001b[1;31mbio\u001b[0m)\n", " nanopolish/0.13.2 (\u001b[1;31mbio\u001b[0m,D)\n", " nco/4.9.5 (\u001b[1;33mio\u001b[0m)\n", " nco/5.0.6 (\u001b[1;33mio\u001b[0m,D)\n", " netcdf-c++4/4.3.1 (\u001b[1;33mio\u001b[0m)\n", " netcdf-fortran/4.5.2 (\u001b[1;33mio\u001b[0m)\n", " netcdf-fortran/4.6.0 (\u001b[1;33mio\u001b[0m,D)\n", " netcdf/4.7.4 (\u001b[1;33mio\u001b[0m)\n", " netcdf/4.9.0 (\u001b[1;33mio\u001b[0m,D)\n", " nlopt/2.6.1 (\u001b[1;32mmath\u001b[0m)\n", " nlopt/2.6.2 (\u001b[1;32mmath\u001b[0m)\n", " nlopt/2.7.0 (\u001b[1;32mmath\u001b[0m)\n", " ntl/11.4.3 (\u001b[1;32mmath\u001b[0m)\n", " oases/0.2.09\n", " octave/5.2.0 (\u001b[1;34mt\u001b[0m)\n", " octopus-bio/0.7.4\n", " ogre/1.10.12\n", " oligoarrayaux/3.8\n", " one-dnn/1.8\n", " openbabel-omp/3.1.1 (\u001b[1;35mchem\u001b[0m)\n", " openbabel/3.1.1 (\u001b[1;35mchem\u001b[0m)\n", " openblas/0.3.9 (\u001b[1;32mmath\u001b[0m)\n", " openblas/0.3.17 (\u001b[1;32mmath\u001b[0m,D)\n", " opencascade/7.5.0\n", " opencolorio/1.1.1 (\u001b[1;34mvis\u001b[0m)\n", " opencolorio/2.2.1 (\u001b[1;34mvis\u001b[0m,D)\n", " opencv/4.4.0 (\u001b[1;34mvis\u001b[0m)\n", " opencv/4.5.1 (\u001b[1;34mvis\u001b[0m)\n", " opencv/4.5.5 (\u001b[1;34mvis\u001b[0m)\n", " opencv/4.6.0 (\u001b[1;34mvis\u001b[0m)\n", " opencv/4.7.0 (\u001b[1;34mvis\u001b[0m)\n", " opencv/4.8.0 (\u001b[1;34mvis\u001b[0m,D)\n", " openfast/3.1.0\n", " openfast/3.3.0\n", " openfast/3.5.0 (D)\n", " openimagedenoise/1.2.2\n", " openimagedenoise/1.4.3 (D)\n", " openimageio/2.1.17.0 (\u001b[1;34mvis\u001b[0m)\n", " openimageio/2.4.7.1 (\u001b[1;34mvis\u001b[0m,D)\n", " openmpi/4.0.3 (\u001b[1;33mL\u001b[0m,\u001b[1;31mm\u001b[0m)\n", " openvdb/7.0.0\n", " openvdb/10.0.1 (D)\n", " openvkl/0.10.0\n", " openvkl/1.3.1 (D)\n", " ospray/1.8.5\n", " ospray/2.2.0\n", " ospray/2.10.0 (D)\n", " osrm-backend/5.26.0\n", " pagmo/2.18.0\n", " paml/4.9j (\u001b[1;31mbio\u001b[0m)\n", " parasail/2.5\n", " pari-gp/2.13.0\n", " pbbam/1.0.7 (\u001b[1;31mbio\u001b[0m)\n", " pbbam/1.7.0 (\u001b[1;31mbio\u001b[0m,D)\n", " pbcopper/1.4.0\n", " pbcopper/1.9.0 (D)\n", " pbmm2/1.7.0\n", " pbsuite/15.8.24 (\u001b[1;31mbio\u001b[0m)\n", " percolator/3.06 (\u001b[1;31mbio\u001b[0m)\n", " pgplot/5.2 (\u001b[1;34mvis\u001b[0m)\n", " phylobayes/4.1c (\u001b[1;31mbio\u001b[0m)\n", " phylokit/1.0\n", " pov-ray/3.8.0-x.10064738\n", " prank/170427 (\u001b[1;31mbio\u001b[0m)\n", " primer3/2.5.0\n", " prokka/1.14.5\n", " ptex/2.3.2\n", " pyne/0.7.3\n", " pyne/0.7.7 (D)\n", " qgis/3.10.6 (\u001b[1;36mgeo\u001b[0m)\n", " qgis/3.22.14 (\u001b[1;36mgeo\u001b[0m,D)\n", " qtltools/1.3.1\n", " quast/5.2.0\n", " r-bundle-bioconductor/3.12 (\u001b[1;31mbio\u001b[0m)\n", " r-bundle-bioconductor/3.14 (\u001b[1;31mbio\u001b[0m)\n", " r-bundle-bioconductor/3.16 (\u001b[1;31mbio\u001b[0m)\n", " r-bundle-bioconductor/3.17 (\u001b[1;31mbio\u001b[0m,D)\n", " raven/1.5.0\n", " raxml-pthreads/8.2.12\n", " rdkit/2019.03.4 (\u001b[1;35mchem\u001b[0m)\n", " rdkit/2020.09.1b1 (\u001b[1;35mchem\u001b[0m)\n", " rdkit/2021.03.3 (\u001b[1;35mchem\u001b[0m)\n", " rdkit/2021.09.3 (\u001b[1;35mchem\u001b[0m)\n", " rdkit/2022.09.4 (\u001b[1;35mchem\u001b[0m,D)\n", " regenie/3.2.1\n", " repeatmasker/4.1.1 (\u001b[1;31mbio\u001b[0m)\n", " repeatmasker/4.1.2-p1 (\u001b[1;31mbio\u001b[0m,D)\n", " repeatmodeler/2.0.1 (\u001b[1;31mbio\u001b[0m)\n", " repeatmodeler/2.0.2a (\u001b[1;31mbio\u001b[0m)\n", " repeatmodeler/2.0.3 (\u001b[1;31mbio\u001b[0m,D)\n", " revbayes/1.1.1\n", " rgaugury/2.1.3\n", " rmats/4.1.1\n", " rmats/4.1.2 (D)\n", " rmblast/2.10.0 (\u001b[1;31mbio\u001b[0m)\n", " roary/3.13.0 (\u001b[1;31mbio\u001b[0m)\n", " root/6.20.04 (\u001b[1;34mt\u001b[0m)\n", " root/6.24.06 (\u001b[1;34mt\u001b[0m)\n", " root/6.26.06 (\u001b[1;34mt\u001b[0m,D)\n", " rtk/1.4.0 (\u001b[1;31mbio\u001b[0m)\n", " saga/8.2.2\n", " samstat/1.5.1\n", " samtools/1.13 (\u001b[1;31mbio\u001b[0m)\n", " shasta/0.10.0\n", " sibelia/3.0.7\n", " sickle/1.33 (\u001b[1;31mbio\u001b[0m)\n", " simbac/0.1a\n", " slim/3.4.0\n", " slim/4.0.1 (D)\n", " snap/2017-05-17 (\u001b[1;31mbio\u001b[0m)\n", " soapdenovo-trans/1.0.4\n", " spglib/1.16.0 (\u001b[1;35mchem\u001b[0m)\n", " spglib/2.0.2 (\u001b[1;35mchem\u001b[0m,D)\n", " sprng/5.0 (\u001b[1;32mmath\u001b[0m)\n", " sra-toolkit/2.10.8 (\u001b[1;31mbio\u001b[0m)\n", " sra-toolkit/3.0.0 (\u001b[1;31mbio\u001b[0m,D)\n", " star-fusion/1.10.0\n", " stringtie/2.1.3 (\u001b[1;31mbio\u001b[0m)\n", " structure/2.3.4 (\u001b[1;31mbio\u001b[0m)\n", " suitesparse/5.7.1 (\u001b[1;32mmath\u001b[0m)\n", " suitesparse/5.10.1 (\u001b[1;32mmath\u001b[0m,D)\n", " superlu/5.2.1 (\u001b[1;32mmath\u001b[0m)\n", " thrift/0.13.0\n", " thrift/0.14.2\n", " thrift/0.16.0\n", " thrift/0.18.1 (D)\n", " tophat/2.1.2 (\u001b[1;31mbio\u001b[0m)\n", " tpp/6.0.0\n", " transdecoder/5.5.0 (\u001b[1;31mbio\u001b[0m)\n", " transdecoder/5.7.1 (\u001b[1;31mbio\u001b[0m,D)\n", " treemix/1.13\n", " trf/4.09.1 (\u001b[1;31mbio\u001b[0m)\n", " trinotate/3.2.1 (\u001b[1;31mbio\u001b[0m)\n", " trinotate/3.2.2 (\u001b[1;31mbio\u001b[0m)\n", " trinotate/4.0.0 (\u001b[1;31mbio\u001b[0m,D)\n", " trnascan-se/2.0.7 (\u001b[1;31mbio\u001b[0m)\n", " trnascan-se/2.0.12 (\u001b[1;31mbio\u001b[0m,D)\n", " viennarna/2.5.0 (\u001b[1;31mbio\u001b[0m)\n", " virsorter/1.0.6\n", " vmd/1.9.4a43 (\u001b[1;34mvis\u001b[0m)\n", " vsearch/2.13.3 (\u001b[1;31mbio\u001b[0m)\n", " vsearch/2.13.4 (\u001b[1;31mbio\u001b[0m)\n", " vsearch/2.15.2 (\u001b[1;31mbio\u001b[0m)\n", " vtk/8.2.0 (\u001b[1;34mvis\u001b[0m)\n", " vtk/9.0.1 (\u001b[1;34mvis\u001b[0m)\n", " vtk/9.1.0 (\u001b[1;34mvis\u001b[0m,D)\n", " wgrib2/3.0.0 (\u001b[1;36mgeo\u001b[0m)\n", " xtb/6.3.3 (\u001b[1;35mchem\u001b[0m)\n", " xtb/6.4.0 (\u001b[1;35mchem\u001b[0m)\n", " xtb/6.4.1 (\u001b[1;35mchem\u001b[0m)\n", "\n", "-------------------------------- Core Modules ---------------------------------\n", " actc/1.1\n", " admixture/1.3.0 (\u001b[1;31mbio\u001b[0m)\n", " advisor/2020.3 (\u001b[1;34mt\u001b[0m)\n", " almosthere/1.0.10\n", " ampl-mp/3.1.0\n", " angsd/0.933 (\u001b[1;31mbio\u001b[0m)\n", " angsd/0.935 (\u001b[1;31mbio\u001b[0m)\n", " angsd/0.936 (\u001b[1;31mbio\u001b[0m)\n", " angsd/0.939 (\u001b[1;31mbio\u001b[0m,D)\n", " annovar/20191024 (\u001b[1;31mbio\u001b[0m)\n", " anserini/0.9.4\n", " ansysedt/2021R2\n", " ansysedt/2023R2 (D)\n", " ant/1.9.15 (\u001b[1;34mt\u001b[0m)\n", " ant/1.10.8 (\u001b[1;34mt\u001b[0m,D)\n", " antlr/2.7.7\n", " any2fasta/0.4.2\n", " apptainer/1.1.3\n", " apptainer/1.1.5\n", " apptainer/1.1.6\n", " apptainer/1.1.8\n", " apptainer/1.2.4 (D)\n", " aragorn/1.2.41 (D)\n", " argtable/2.13 (\u001b[1;34mt\u001b[0m)\n", " arpack-ng/3.9.0 (\u001b[1;32mmath\u001b[0m)\n", " ascp/3.5.4 (\u001b[1;34mt\u001b[0m)\n", " assimp/5.0.0\n", " autodiff/0.6.0\n", " autodiff/0.6.12 (D)\n", " autodock_vina/1.1.2 (\u001b[1;35mchem\u001b[0m)\n", " bamm/2.5.0 (\u001b[1;31mbio\u001b[0m)\n", " bamtools/2.5.1 (\u001b[1;31mbio\u001b[0m)\n", " bamutil/1.0.14 (\u001b[1;31mbio\u001b[0m)\n", " barrnap/0.9\n", " bayescan/2.1 (\u001b[1;31mbio\u001b[0m)\n", " bazel/3.6.0 (\u001b[1;34mt\u001b[0m)\n", " bbmap/38.86 (\u001b[1;31mbio\u001b[0m)\n", " bcl2fastq2/2.20.0 (\u001b[1;31mbio\u001b[0m)\n", " beagle-lib/4.0.0 (\u001b[1;31mbio\u001b[0m)\n", " beagle/5.4\n", " beast/2.7.5 (\u001b[1;31mbio\u001b[0m,D)\n", " bedops/2.4.39\n", " bedops/2.4.41 (D)\n", " bedtools/2.29.2 (\u001b[1;31mbio\u001b[0m)\n", " bedtools/2.30.0 (\u001b[1;31mbio\u001b[0m,D)\n", " bioawk/1.0\n", " bioperl/1.7.7 (\u001b[1;31mbio\u001b[0m)\n", " bioperl/1.7.8 (\u001b[1;31mbio\u001b[0m,D)\n", " biopp/2.4.1\n", " bismark/0.22.3\n", " bison/3.7.1\n", " bison/3.8.2 (D)\n", " blast/2.2.26 (\u001b[1;31mbio\u001b[0m)\n", " blat/3.5 (\u001b[1;31mbio\u001b[0m)\n", " blender/3.6.0 (\u001b[1;34mvis\u001b[0m,D)\n", " blitz++/1.0.2 (\u001b[1;34mt\u001b[0m)\n", " blosc/1.17.1\n", " blosc/1.21.3 (D)\n", " blosc2/2.10.3\n", " boost-build/1.80.0\n", " bowtie/1.3.0 (\u001b[1;31mbio\u001b[0m)\n", " bowtie2/2.4.1 (\u001b[1;31mbio\u001b[0m)\n", " bowtie2/2.4.2 (\u001b[1;31mbio\u001b[0m)\n", " bowtie2/2.4.4 (\u001b[1;31mbio\u001b[0m)\n", " bowtie2/2.5.1 (\u001b[1;31mbio\u001b[0m,D)\n", " bpp/4.3.8\n", " bpp/4.6.1 (D)\n", " bracken/2.7 (D)\n", " breseq/0.35.2\n", " breseq/0.36.1 (D)\n", " brotli/1.0.9\n", " btllib/1.6.2\n", " bustools/0.40.0\n", " bwa-mem2/2.2.1\n", " bwa/0.7.17 (\u001b[1;31mbio\u001b[0m)\n", " bwidget/1.9.14\n", " canu/2.0 (\u001b[1;31mbio\u001b[0m)\n", " canu/2.1.1 (\u001b[1;31mbio\u001b[0m)\n", " canu/2.2 (\u001b[1;31mbio\u001b[0m,D)\n", " cap3/20151002 (\u001b[1;31mbio\u001b[0m)\n", " capnproto/0.7.0\n", " casadi/3.6.3\n", " catch2/2.11.0\n", " catch2/3.2.1 (D)\n", " ccfits/2.5 (\u001b[1;34mvis\u001b[0m)\n", " ccsm/4_0_a02 (\u001b[1;36mgeo\u001b[0m)\n", " cd-hit/4.8.1 (\u001b[1;31mbio\u001b[0m)\n", " cdbfasta/0.99\n", " cellranger/2.1.0 (\u001b[1;31mbio\u001b[0m)\n", " cellsnp-lite/1.2.2\n", " centrifuge/1.0.4-beta (\u001b[1;31mbio\u001b[0m)\n", " cereal/1.3.0\n", " cesm/2.1.3 (\u001b[1;36mgeo\u001b[0m)\n", " cfitsio/3.41 (\u001b[1;34mvis\u001b[0m)\n", " cfitsio/3.48 (\u001b[1;34mvis\u001b[0m)\n", " cfitsio/3.49 (\u001b[1;34mvis\u001b[0m)\n", " cfitsio/4.1.0 (\u001b[1;34mvis\u001b[0m,D)\n", " cgal/4.14.3 (\u001b[1;32mmath\u001b[0m)\n", " cgal/5.5.2 (\u001b[1;32mmath\u001b[0m,D)\n", " circos/0.69-9 (\u001b[1;34mvis\u001b[0m)\n", " clang/9.0.1 (\u001b[1;34mt\u001b[0m)\n", " clang/11.0.0 (\u001b[1;34mt\u001b[0m)\n", " clang/13.0.1 (\u001b[1;34mt\u001b[0m,D)\n", " cmake/3.18.4 (\u001b[1;34mt\u001b[0m)\n", " cmake/3.20.1 (\u001b[1;34mt\u001b[0m)\n", " cmake/3.21.4 (\u001b[1;34mt\u001b[0m)\n", " cmake/3.22.1 (\u001b[1;34mt\u001b[0m)\n", " cmake/3.23.1 (\u001b[1;34mt\u001b[0m)\n", " cmake/3.27.7 (\u001b[1;34mt\u001b[0m,D)\n", " cnvnator/0.4.1 (\u001b[1;31mbio\u001b[0m)\n", " code-server/3.5.0\n", " code-server/3.12.0 (D)\n", " coinmp/1.8.4\n", " coretran/1.0.1\n", " corset/1.09 (\u001b[1;31mbio\u001b[0m)\n", " cppzmq/4.7.1\n", " cpu_features/0.6.0\n", " cram/0.7\n", " crest/2.12 (\u001b[1;35mchem\u001b[0m,D)\n", " cromwell/58 (\u001b[1;34mt\u001b[0m)\n", " csblast/2.2.4\n", " cst/2020 (\u001b[1;36mphys\u001b[0m)\n", " csvtk/0.23.0\n", " cuba/4.2.2\n", " cubegui/4.4.4\n", " cubelib/4.4.4\n", " cubewriter/4.4.3\n", " custom-ctypes/1.1\n", " cvit/1.2.1\n", " db/18.1.32\n", " dcm2niix/1.0.20200331 (\u001b[1;31mbio\u001b[0m)\n", " dcmtk/3.6.7\n", " delly/0.8.5\n", " detonate/1.11 (\u001b[1;31mbio\u001b[0m)\n", " dftd3-lib/0.10 (\u001b[1;35mchem\u001b[0m)\n", " diamond/0.9.36 (\u001b[1;31mbio\u001b[0m)\n", " diamond/2.0.4 (\u001b[1;31mbio\u001b[0m)\n", " diamond/2.0.9 (\u001b[1;31mbio\u001b[0m)\n", " diamond/2.1.6 (\u001b[1;31mbio\u001b[0m)\n", " diamond/2.1.7 (\u001b[1;31mbio\u001b[0m,D)\n", " dorado/0.2.1\n", " dorado/0.2.2\n", " dorado/0.3.0 (D)\n", " dotnet-core/3.1.8\n", " dotnet-core/5.0.12\n", " dotnet-core/6.0.0 (D)\n", " double-conversion/3.1.5\n", " double-conversion/3.2.1 (D)\n", " eigen/3.3.7 (\u001b[1;32mmath\u001b[0m)\n", " eigen/3.4.0 (\u001b[1;32mmath\u001b[0m,D)\n", " elixir/1.13 (\u001b[1;34mt\u001b[0m)\n", " embree/2.17.7\n", " energyplus/9.3.0\n", " energyplus/23.2.0 (D)\n", " erlangotp/23.3 (\u001b[1;34mt\u001b[0m)\n", " erlangotp/24.2 (\u001b[1;34mt\u001b[0m,D)\n", " espeak-ng/1.51\n", " expat/2.2.9 (\u001b[1;34mt\u001b[0m)\n", " expat/2.2.10 (\u001b[1;34mt\u001b[0m)\n", " expat/2.4.1 (\u001b[1;34mt\u001b[0m,D)\n", " fann/2.2.0\n", " fasta/36.3.8h (\u001b[1;31mbio\u001b[0m)\n", " fastahack/1.0.0\n", " fastme/2.1.6.2 (\u001b[1;31mbio\u001b[0m)\n", " fastp/0.20.1 (\u001b[1;31mbio\u001b[0m)\n", " fastp/0.23.1 (\u001b[1;31mbio\u001b[0m)\n", " fastp/0.23.4 (\u001b[1;31mbio\u001b[0m,D)\n", " fastq-join/1.3.1\n", " fastq-multx/1.4.0\n", " fastq-tools/0.8\n", " fastq_screen/0.11.4 (\u001b[1;31mbio\u001b[0m)\n", " fastqc/0.11.9 (\u001b[1;31mbio\u001b[0m)\n", " fastqc/0.12.0 (\u001b[1;31mbio\u001b[0m,D)\n", " fastsimcoal2/2.6.0.3 (\u001b[1;31mbio\u001b[0m)\n", " fastsimcoal2/2.7.0.9 (\u001b[1;31mbio\u001b[0m,D)\n", " fasttree/2.1.11 (\u001b[1;31mbio\u001b[0m)\n", " fastx-toolkit/0.0.14 (\u001b[1;31mbio\u001b[0m)\n", " fermi-lite/20190320\n", " ffmpeg/4.2.2\n", " ffmpeg/4.3.2 (D)\n", " filevercmp/20191210\n", " filtlong/0.2.0 (\u001b[1;31mbio\u001b[0m)\n", " flatbuffers/22.9.24\n", " flexiblas/3.0.4\n", " flint/2.7.1 (\u001b[1;32mmath\u001b[0m)\n", " flint/2.9.0 (\u001b[1;32mmath\u001b[0m,D)\n", " fmm3d/1.0.1\n", " fmriprep/23.0.2\n", " fmriprep/23.1.3 (D)\n", " fmt/5.3.0\n", " fmt/6.2.1\n", " fmt/7.0.3\n", " fmt/9.1.0 (D)\n", " fpc/3.2.2 (\u001b[1;34mt\u001b[0m)\n", " fraggenescan/1.30 (\u001b[1;31mbio\u001b[0m)\n", " fraggenescan/1.31 (\u001b[1;31mbio\u001b[0m,D)\n", " freebayes/1.2.0 (\u001b[1;31mbio\u001b[0m)\n", " freebayes/1.3.6 (\u001b[1;31mbio\u001b[0m,D)\n", " freesurfer/5.3.0 (\u001b[1;31mbio\u001b[0m)\n", " freexl/1.0.5 (\u001b[1;34mt\u001b[0m)\n", " fsom/20141119\n", " g2clib/1.6.0 (\u001b[1;36mgeo\u001b[0m)\n", " g2lib/3.1.0 (\u001b[1;36mgeo\u001b[0m)\n", " gapcloser/1.12-r6\n", " gatk/4.1.8.0 (\u001b[1;31mbio\u001b[0m)\n", " gatk/4.1.8.1 (\u001b[1;31mbio\u001b[0m)\n", " gatk/4.2.2.0 (\u001b[1;31mbio\u001b[0m)\n", " gatk/4.2.4.0 (\u001b[1;31mbio\u001b[0m)\n", " gatk/4.2.5.0 (\u001b[1;31mbio\u001b[0m,D)\n", " gblocks/0.91b\n", " gcc/8.4.0 (\u001b[1;34mt\u001b[0m)\n", " gcc/9.3.0 (\u001b[1;33mL\u001b[0m,\u001b[1;34mt\u001b[0m,D)\n", " gcc/10.2.0 (\u001b[1;34mt\u001b[0m)\n", " gcc/10.3.0 (\u001b[1;34mt\u001b[0m)\n", " gcc/11.3.0 (\u001b[1;34mt\u001b[0m)\n", " gclust/355z3\n", " gd/2.71\n", " gdrcopy/2.3\n", " geant4-data/10.7.3\n", " geant4-data/11.1.0 (D)\n", " gengetopt/2.23\n", " genmap/1.3.0\n", " geos/3.7.3 (\u001b[1;36mgeo\u001b[0m)\n", " geos/3.8.1 (\u001b[1;36mgeo\u001b[0m)\n", " geos/3.9.1 (\u001b[1;36mgeo\u001b[0m)\n", " geos/3.10.2 (\u001b[1;36mgeo\u001b[0m,D)\n", " gffread/0.11.7\n", " gffread/0.12.3 (D)\n", " gflags/2.2.2\n", " gibbs2/1.0\n", " git-annex/8.20200810 (\u001b[1;34mt\u001b[0m)\n", " git-annex/10.20221003 (\u001b[1;34mt\u001b[0m,D)\n", " git-lfs/2.11.0\n", " git-lfs/3.3.0 (D)\n", " givaro/4.2.0\n", " glew/2.1.0\n", " glfw/3.3.2\n", " glfw/3.3.8 (D)\n", " glimmerhmm/3.0.4\n", " glimpse/1.1.1\n", " glimpse/2.0.0 (D)\n", " glm/0.9.9.8 (\u001b[1;34mvis\u001b[0m)\n", " glpk/4.65 (\u001b[1;32mmath\u001b[0m)\n", " glpk/5.0 (\u001b[1;32mmath\u001b[0m,D)\n", " gmap-gsnap/2019-09-12 (\u001b[1;31mbio\u001b[0m)\n", " gmap-gsnap/2020-11-14 (\u001b[1;31mbio\u001b[0m,D)\n", " gmp/6.2.0\n", " gnuplot/5.2.8 (\u001b[1;34mvis\u001b[0m)\n", " gnuplot/5.4.2 (\u001b[1;34mvis\u001b[0m)\n", " gnuplot/5.4.6 (\u001b[1;34mvis\u001b[0m,D)\n", " go/1.14.1 (\u001b[1;34mt\u001b[0m)\n", " go/1.18.3 (\u001b[1;34mt\u001b[0m)\n", " go/1.21.3 (\u001b[1;34mt\u001b[0m,D)\n", " googlebenchmark/1.7.1\n", " googletest/1.10.0 (\u001b[1;34mt\u001b[0m)\n", " googletest/1.13.0 (\u001b[1;34mt\u001b[0m,D)\n", " groff/1.22.4\n", " gsl-lite/0.40.0\n", " gts/20121130\n", " gudhi/3.7.1 (D)\n", " gurobi/9.0.3 (\u001b[1;32mmath\u001b[0m)\n", " gurobi/9.1.0 (\u001b[1;32mmath\u001b[0m)\n", " gurobi/9.1.2 (\u001b[1;32mmath\u001b[0m)\n", " gurobi/9.5.0 (\u001b[1;32mmath\u001b[0m)\n", " gurobi/9.5.2 (\u001b[1;32mmath\u001b[0m)\n", " gurobi/10.0.1 (\u001b[1;32mmath\u001b[0m)\n", " gurobi/10.0.2 (\u001b[1;32mmath\u001b[0m)\n", " gurobi/10.0.3 (\u001b[1;32mmath\u001b[0m)\n", " gurobi/11.0.0 (\u001b[1;32mmath\u001b[0m,D)\n", " hapgen2/2.2.0\n", " haploview/4.2\n", " harminv/1.4.1 (\u001b[1;32mmath\u001b[0m)\n", " hdf-fortran/4.2.15\n", " hdf/4.2.15 (\u001b[1;33mio\u001b[0m)\n", " hdfview/2.14\n", " healpix/3.81\n", " hh-suite/3.3.0 (\u001b[1;31mbio\u001b[0m)\n", " hifiasm/0.16.1\n", " hifiasm/0.19.5 (D)\n", " hisat2/2.2.1 (\u001b[1;31mbio\u001b[0m)\n", " hmmer/3.2.1 (\u001b[1;31mbio\u001b[0m)\n", " hmmer/3.3.2 (\u001b[1;31mbio\u001b[0m,D)\n", " htslib/1.9 (\u001b[1;31mbio\u001b[0m)\n", " htslib/1.10.2 (\u001b[1;31mbio\u001b[0m)\n", " htslib/1.11 (\u001b[1;31mbio\u001b[0m)\n", " htslib/1.13 (\u001b[1;31mbio\u001b[0m)\n", " htslib/1.14 (\u001b[1;31mbio\u001b[0m)\n", " htslib/1.15.1 (\u001b[1;31mbio\u001b[0m)\n", " htslib/1.16 (\u001b[1;31mbio\u001b[0m)\n", " htslib/1.17 (\u001b[1;31mbio\u001b[0m,D)\n", " hwloc/2.4.0\n", " hwloc/2.7.1 (D)\n", " idba-ud/1.1.3\n", " igblast/1.17.0 (\u001b[1;31mbio\u001b[0m)\n", " igblast/1.18.0 (\u001b[1;31mbio\u001b[0m,D)\n", " igraph/0.8.2 (\u001b[1;32mmath\u001b[0m)\n", " igraph/0.9.10 (\u001b[1;32mmath\u001b[0m)\n", " igraph/0.10.2 (\u001b[1;32mmath\u001b[0m,D)\n", " igv/2.9.2\n", " ijulia-kernel/1.5\n", " ijulia-kernel/1.8 (D)\n", " imkl/2020.1.217 (\u001b[1;33mL\u001b[0m,\u001b[1;32mmath\u001b[0m,D)\n", " imkl/2021.2.0 (\u001b[1;32mmath\u001b[0m)\n", " imkl/2021.4.0 (\u001b[1;32mmath\u001b[0m)\n", " imkl/2022.1.0 (\u001b[1;32mmath\u001b[0m)\n", " impute2/2.3.2 (\u001b[1;31mbio\u001b[0m)\n", " impute5/1.1.5\n", " intel-opencl/2021.2.0\n", " intel/2020.1.217 (\u001b[1;34mt\u001b[0m,D)\n", " intel/2021.2.0 (\u001b[1;34mt\u001b[0m)\n", " intel/2022.1.0 (\u001b[1;34mt\u001b[0m)\n", " intelxed/12.0.1\n", " interproscan/5.50-84.0 (\u001b[1;31mbio\u001b[0m)\n", " interproscan/5.52-86.0 (\u001b[1;31mbio\u001b[0m)\n", " interproscan/5.53-87.0 (\u001b[1;31mbio\u001b[0m)\n", " interproscan/5.55-88.0 (\u001b[1;31mbio\u001b[0m)\n", " interproscan/5.56-89.0 (\u001b[1;31mbio\u001b[0m)\n", " interproscan/5.63-95.0 (\u001b[1;31mbio\u001b[0m)\n", " interproscan/5.64-96.0 (\u001b[1;31mbio\u001b[0m,D)\n", " interproscan_data/5.63-95.0\n", " interproscan_data/5.64-96.0 (D)\n", " intervaltree/0.1\n", " ipopt/3.14.11\n", " ipp/2020.1.217 (\u001b[1;34mt\u001b[0m)\n", " ipykernel/2020a\n", " ipykernel/2020b\n", " ipykernel/2021a\n", " ipykernel/2022a\n", " ipykernel/2023a\n", " ipykernel/2023b (D)\n", " ipython-kernel/2.7\n", " ipython-kernel/2.7\n", " ipython-kernel/3.6\n", " ipython-kernel/3.7\n", " ipython-kernel/3.8\n", " ipython-kernel/3.9\n", " ipython-kernel/3.10 (D)\n", " ipython-kernel/3.11\n", " ir-kernel/4.2\n", " irfinder/1.3.1\n", " ispc/1.10.0\n", " ispc/1.13.0\n", " ispc/1.18.0 (D)\n", " itac/2021.5.0 (\u001b[1;34mt\u001b[0m)\n", " jasper/2.0.16 (\u001b[1;34mvis\u001b[0m)\n", " java/1.8.0_192 (\u001b[1;34mt\u001b[0m,1.8)\n", " java/11.0.2 (\u001b[1;34mt\u001b[0m)\n", " java/11.0.16_8 (\u001b[1;34mt\u001b[0m,11)\n", " java/13.0.2 (\u001b[1;34mt\u001b[0m,13)\n", " java/14.0.2 (\u001b[1;34mt\u001b[0m,14)\n", " java/17.0.2 (\u001b[1;34mt\u001b[0m,D:17)\n", " jbigkit/2.1\n", " jellyfish/2.3.0 (\u001b[1;31mbio\u001b[0m)\n", " jemalloc/5.2.1\n", " jemalloc/5.3.0 (D)\n", " jsoncpp/1.9.4\n", " judy/1.0.5\n", " julia/1.4.1 (\u001b[1;34mt\u001b[0m)\n", " julia/1.5.2 (\u001b[1;34mt\u001b[0m)\n", " julia/1.6.0 (\u001b[1;34mt\u001b[0m)\n", " julia/1.6.1 (\u001b[1;34mt\u001b[0m)\n", " julia/1.7.0 (\u001b[1;34mt\u001b[0m)\n", " julia/1.8.1 (\u001b[1;34mt\u001b[0m,D)\n", " julia/1.8.5 (\u001b[1;34mt\u001b[0m)\n", " julia/1.9.1 (\u001b[1;34mt\u001b[0m)\n", " kentutils/401 (\u001b[1;31mbio\u001b[0m)\n", " kma/1.3.0\n", " kma/1.3.0\n", " kma/1.3.25\n", " kma/1.4.14 (D)\n", " kmergenie/1.7051\n", " kraken/1.1.1 (\u001b[1;31mbio\u001b[0m)\n", " krakenuniq/1.0.2\n", " kronatools/2.8\n", " lastz/1.04.03\n", " ldc/0.17.6\n", " ldc/1.26.0 (D)\n", " leptonica/1.82.0\n", " leveldb/1.22 (\u001b[1;34mt\u001b[0m)\n", " lhapdf/6.4.0\n", " libaec/1.0.6\n", " libbigwig/0.4.6\n", " libccd/2.1\n", " libcerf/1.13 (\u001b[1;32mmath\u001b[0m)\n", " libcerf/1.17 (\u001b[1;32mmath\u001b[0m)\n", " libcerf/2.1 (\u001b[1;32mmath\u001b[0m,D)\n", " libctl/4.5.0 (\u001b[1;34mt\u001b[0m)\n", " libdap/3.20.6\n", " libevent/2.1.11\n", " libfabric/1.10.1 (\u001b[1;33mL\u001b[0m)\n", " libfabric/1.11.0\n", " libfabric/1.12.1\n", " libfabric/1.15.1 (D)\n", " libffi/3.3\n", " libgd/2.3.0 (\u001b[1;34mvis\u001b[0m)\n", " libgd/2.3.3 (\u001b[1;34mvis\u001b[0m,D)\n", " libgdsii/0.21\n", " libgeotiff-proj901/1.7.1\n", " libgeotiff/1.5.1\n", " libgeotiff/1.6.0\n", " libgeotiff/1.7.1 (D)\n", " libgtextutils/0.7 (\u001b[1;31mbio\u001b[0m)\n", " libharu/2.3.0\n", " libraw/0.21.2\n", " librdkafka/1.5.2-RC1\n", " librttopo-proj9/1.1.0\n", " librttopo/1.1.0\n", " libspatialindex/1.8.5 (\u001b[1;36mphys\u001b[0m)\n", " libspatialite-proj9/5.0.1\n", " libspatialite-proj901/5.0.1\n", " libspatialite/4.3.0a (\u001b[1;36mphys\u001b[0m)\n", " libspatialite/5.0.1 (\u001b[1;36mphys\u001b[0m,D)\n", " libstatgen/20190330\n", " liburing/2.3\n", " libxslt/1.1.34\n", " liknorm/1.5.1 (\u001b[1;34mt\u001b[0m)\n", " limix-bgen/3.0.3\n", " links/1.8.6 (\u001b[1;31mbio\u001b[0m)\n", " links/2.0.1 (\u001b[1;31mbio\u001b[0m,D)\n", " littlecms/2.9\n", " littlecms/2.11 (D)\n", " lldb/11.0.0\n", " llvm/8.0.1 (\u001b[1;34mt\u001b[0m)\n", " llvm/9.0.1 (\u001b[1;34mt\u001b[0m)\n", " llvm/13.0.1 (\u001b[1;34mt\u001b[0m)\n", " llvm/14.0.3 (\u001b[1;34mt\u001b[0m,D)\n", " lmdb/0.9.24\n", " lpsolve/5.5.2.5 (\u001b[1;32mmath\u001b[0m)\n", " ls-prepost/4.8.11\n", " ls-prepost/4.9.9 (D)\n", " lsd2/1.9.7\n", " lsd2/2.3 (D)\n", " lumpy/0.2.13 (\u001b[1;31mbio\u001b[0m)\n", " mafft/7.471 (\u001b[1;31mbio\u001b[0m)\n", " magma-gene-analysis/1.10\n", " makeinfo/6.7\n", " mariadb-connector-c/3.1.7\n", " mariadb/10.4.13 (\u001b[1;34mt\u001b[0m)\n", " mariadb/10.6.12 (\u001b[1;34mt\u001b[0m,D)\n", " matio/1.5.19 (\u001b[1;33mio\u001b[0m)\n", " maven/3.6.3 (\u001b[1;34mt\u001b[0m)\n", " mcr/R2020b (\u001b[1;34mt\u001b[0m)\n", " mcr/R2021b (\u001b[1;34mt\u001b[0m)\n", " mcr/R2022b (\u001b[1;34mt\u001b[0m,D)\n", " megahit/1.2.9 (\u001b[1;31mbio\u001b[0m)\n", " mesquite/2.3.0 (\u001b[1;32mmath\u001b[0m)\n", " meta-farm/1.0.2\n", " metabat/2.12.1 (\u001b[1;31mbio\u001b[0m)\n", " metaeuk/6 (D)\n", " metageneannotator/20080819\n", " methyldackel/0.6.1\n", " metis-64idx/5.1.0\n", " metis/5.1.0 (\u001b[1;32mmath\u001b[0m)\n", " mii/1.1.1\n", " mii/1.1.2 (D)\n", " minia/3.2.6 (\u001b[1;31mbio\u001b[0m)\n", " minimap2/2.17 (\u001b[1;31mbio\u001b[0m)\n", " minimap2/2.18 (\u001b[1;31mbio\u001b[0m)\n", " minimap2/2.24 (\u001b[1;31mbio\u001b[0m,D)\n", " minpath/1.6\n", " mixcr/4.1.2 (\u001b[1;31mbio\u001b[0m)\n", " mmseqs2/14-7e284 (D)\n", " mono/6.12.0.122 (\u001b[1;34mt\u001b[0m)\n", " mpsolve/3.2.1\n", " mujoco/2.2.0\n", " mujoco/2.2.2\n", " mujoco/2.3.6 (D)\n", " multichoose/1.0.3\n", " mummer/4.0.0beta2 (\u001b[1;31mbio\u001b[0m)\n", " muparser/2.2.5 (\u001b[1;32mmath\u001b[0m)\n", " muparser/2.3.2 (\u001b[1;32mmath\u001b[0m,D)\n", " muparserx/4.0.12\n", " muscle/3.8.1551 (\u001b[1;31mbio\u001b[0m)\n", " mysql/5.7.36 (\u001b[1;34mt\u001b[0m)\n", " nanoflann/1.3.2\n", " nauty/2.6r12\n", " nextdenovo/2.5.2\n", " nextflow/20.10.0\n", " nextflow/21.04.3\n", " nextflow/21.10.3\n", " nextflow/22.04.3\n", " nextflow/22.10.6\n", " nextflow/22.10.8\n", " nextflow/23.04.3 (D)\n", " nextgenmap/0.5.5 (\u001b[1;31mbio\u001b[0m)\n", " nextstrain.cli/3.1.0\n", " ngmlr/0.2.7\n", " ngs/2.10.5\n", " ngt/2.0.13\n", " ninja-phylogenetics/0.97-cluster_only\n", " nlopt/2.6.1 (\u001b[1;32mmath\u001b[0m)\n", " nlopt/2.6.2 (\u001b[1;32mmath\u001b[0m)\n", " nlopt/2.7.0 (\u001b[1;32mmath\u001b[0m)\n", " nlopt/2.7.1 (\u001b[1;32mmath\u001b[0m,D)\n", " nodejs/12.16.1 (\u001b[1;34mt\u001b[0m)\n", " nodejs/15.2.1 (\u001b[1;34mt\u001b[0m,D)\n", " nspr/4.25\n", " nss/3.51\n", " ntedit/1.3.2\n", " nvhpc/20.7 (\u001b[1;34mt\u001b[0m)\n", " nvhpc/22.1 (\u001b[1;34mt\u001b[0m)\n", " nvhpc/22.7 (\u001b[1;34mt\u001b[0m)\n", " nvhpc/23.7 (\u001b[1;34mt\u001b[0m,D)\n", " occt/7.4.0\n", " ogre/1.12.12 (D)\n", " one-dnn/2.4.3 (D)\n", " opari2/2.0.5\n", " openexr/2.5.2 (\u001b[1;34mvis\u001b[0m)\n", " openexr/3.1.5 (\u001b[1;34mvis\u001b[0m,D)\n", " openpgm/5.2.122 (\u001b[1;34mt\u001b[0m)\n", " openrefine/3.4.1\n", " openslide/3.4.1 (\u001b[1;34mvis\u001b[0m)\n", " opera-ms/0.9.0-20200802\n", " optix/6.5.0\n", " optix/7.7.0 (D)\n", " orthomcl/2.0.9 (\u001b[1;31mbio\u001b[0m)\n", " otf2/2.2\n", " ovito/3.3.3\n", " packmol/20.3.3 (\u001b[1;35mchem\u001b[0m)\n", " pairix/0.3.7\n", " panther/14.1\n", " papi/6.0.0 (\u001b[1;34mt\u001b[0m)\n", " pcre/8.44\n", " pcre2/10.34\n", " pdt/3.25.1\n", " pear/0.9.11 (\u001b[1;31mbio\u001b[0m)\n", " penncnv/1.0.5 (\u001b[1;31mbio\u001b[0m)\n", " perl/5.30.2 (\u001b[1;34mt\u001b[0m,5.30)\n", " pfamscan/1.6\n", " pftoolsv3/3.2.12\n", " phast/1.6\n", " picard/2.23.3 (\u001b[1;31mbio\u001b[0m)\n", " picard/2.26.3 (\u001b[1;31mbio\u001b[0m,D)\n", " pilercr/1.06\n", " pilon/1.23 (\u001b[1;31mbio\u001b[0m)\n", " pilon/1.24 (\u001b[1;31mbio\u001b[0m,D)\n", " pindel/0.2.5b9-20170508 (\u001b[1;31mbio\u001b[0m)\n", " platanus-allee/2.2.2\n", " platanus/1.2.4 (\u001b[1;31mbio\u001b[0m)\n", " plink/1.9b_6.21-x86_64 (\u001b[1;31mbio\u001b[0m)\n", " plink/2.00a3.6 (\u001b[1;31mbio\u001b[0m)\n", " plink/2.00-10252019-avx2 (\u001b[1;31mbio\u001b[0m,D)\n", " plplot/5.15.0\n", " pluto/0.14.7\n", " pmix/3.1.5\n", " pmix/3.2.3\n", " pmix/4.1.2 (D)\n", " podman/4.1.1\n", " podman/4.5.0 (D)\n", " poplddecay/3.41\n", " popscle/0.1-beta\n", " portaudio/190600_20161030\n", " portaudio/190700_20210406 (D)\n", " postgresql/12.4 (\u001b[1;34mt\u001b[0m)\n", " postgresql/13.2 (\u001b[1;34mt\u001b[0m)\n", " postgresql/14.2 (\u001b[1;34mt\u001b[0m,D)\n", " pplacer/1.1.alpha19 (\u001b[1;31mbio\u001b[0m)\n", " primme/3.2\n", " prism/4.7\n", " prodigal-gv/2.6.3\n", " prodigal/2.6.3 (\u001b[1;31mbio\u001b[0m)\n", " proj/6.3.2 (\u001b[1;36mgeo\u001b[0m)\n", " proj/7.0.0 (\u001b[1;36mgeo\u001b[0m)\n", " proj/7.0.1 (\u001b[1;36mgeo\u001b[0m)\n", " proj/7.2.1 (\u001b[1;36mgeo\u001b[0m)\n", " proj/8.0.0 (\u001b[1;36mgeo\u001b[0m)\n", " proj/9.0.0 (\u001b[1;36mgeo\u001b[0m)\n", " proj/9.0.1 (\u001b[1;36mgeo\u001b[0m,D)\n", " protobuf/3.12.3 (\u001b[1;34mt\u001b[0m)\n", " protobuf/3.19.4 (\u001b[1;34mt\u001b[0m)\n", " protobuf/3.21.3 (\u001b[1;34mt\u001b[0m,D)\n", " psipred/4.02\n", " psmc/0.6.5 (\u001b[1;31mbio\u001b[0m)\n", " pullseq/1.0.2\n", " pypy/5.8.0 (\u001b[1;34mt\u001b[0m)\n", " pypy/7.3.3 (\u001b[1;34mt\u001b[0m,D)\n", " pytest/6.1.2\n", " pytest/6.2.5\n", " pytest/7.0.1\n", " pytest/7.4.0 (D)\n", " python-build-bundle/2022a\n", " python-build-bundle/2023a\n", " python-build-bundle/2023b (D)\n", " python/2.7.18 (\u001b[1;34mt\u001b[0m,2.7)\n", " python/3.6.10 (\u001b[1;34mt\u001b[0m,3.6)\n", " python/3.7.7 (\u001b[1;34mt\u001b[0m)\n", " python/3.7.9 (\u001b[1;34mt\u001b[0m,3.7)\n", " python/3.8.2 (\u001b[1;34mt\u001b[0m)\n", " python/3.8.10 (\u001b[1;34mt\u001b[0m,3.8)\n", " python/3.9.6 (\u001b[1;34mt\u001b[0m,3.9)\n", " python/3.10.2 (\u001b[1;34mt\u001b[0m,D:3.10)\n", " python/3.11.2 (\u001b[1;34mt\u001b[0m)\n", " python/3.11.5 (\u001b[1;34mt\u001b[0m,3.11)\n", " qca/2.3.0 (\u001b[1;36mgeo\u001b[0m)\n", " qca/2.3.5 (\u001b[1;36mgeo\u001b[0m,D)\n", " qcint/4.4.6\n", " qcint/5.1.5\n", " qcint/5.1.7\n", " qcint/5.1.8 (D)\n", " qctool/2.2.0\n", " qhull/2019.1 (\u001b[1;32mmath\u001b[0m)\n", " qiime2/2021.11\n", " qiime2/2023.5 (D)\n", " qjson/0.9.0 (\u001b[1;34mt\u001b[0m)\n", " qrupdate/1.1.2 (\u001b[1;32mmath\u001b[0m)\n", " qscintilla/2.11.2 (\u001b[1;34mt\u001b[0m)\n", " qscintilla/2.11.6 (\u001b[1;34mt\u001b[0m,D)\n", " qt/5.12.8 (\u001b[1;34mt\u001b[0m)\n", " qt/5.15.2 (\u001b[1;34mt\u001b[0m)\n", " qt/5.15.8 (\u001b[1;34mt\u001b[0m,D)\n", " qt5webkit/5.212.0-alpha4\n", " qtkeychain/0.9.1\n", " qtkeychain/0.13.2 (D)\n", " quantumatk/2019.12\n", " qwt/6.1.4 (\u001b[1;34mt\u001b[0m)\n", " qwt/6.2.0 (\u001b[1;34mt\u001b[0m,D)\n", " qwtpolar/1.1.1 (\u001b[1;34mt\u001b[0m)\n", " r/4.0.0 (\u001b[1;34mt\u001b[0m)\n", " r/4.0.2 (\u001b[1;34mt\u001b[0m)\n", " r/4.0.5 (\u001b[1;34mt\u001b[0m)\n", " r/4.1.0 (\u001b[1;34mt\u001b[0m)\n", " r/4.1.2 (\u001b[1;34mt\u001b[0m)\n", " r/4.2.1 (\u001b[1;34mt\u001b[0m)\n", " r/4.2.2 (\u001b[1;34mt\u001b[0m)\n", " r/4.3.1 (\u001b[1;34mt\u001b[0m,D)\n", " r2r/1.0.6\n", " racket/7.8\n", " racon/1.4.3 (\u001b[1;31mbio\u001b[0m)\n", " racon/1.4.13 (\u001b[1;31mbio\u001b[0m)\n", " racon/1.5.0 (\u001b[1;31mbio\u001b[0m,D)\n", " randomlib/1.10\n", " rapidjson/1.1.0\n", " rapsearch2/2.24\n", " raqm/0.9.0\n", " raspa2/2.0.47\n", " rcorrector/1.0.4\n", " rcorrector/1.0.6 (D)\n", " re2c/1.3\n", " recon/1.08 (\u001b[1;31mbio\u001b[0m)\n", " reduce/4.14\n", " reframe/3.12.0\n", " repeatscout/1.0.5 (\u001b[1;31mbio\u001b[0m)\n", " rkcommon/1.4.2\n", " rkcommon/1.10.0 (D)\n", " rnacode/0.3\n", " rnammer/1.2 (\u001b[1;31mbio\u001b[0m)\n", " rocksdb/7.10.2\n", " rsem/1.3.3 (\u001b[1;31mbio\u001b[0m)\n", " rstudio-server/4.1 (\u001b[1;34mt\u001b[0m)\n", " rstudio-server/4.2 (\u001b[1;34mt\u001b[0m)\n", " rstudio-server/4.3 (\u001b[1;34mt\u001b[0m,D)\n", " ruamel.yaml/0.17.21\n", " ruby/2.7.1 (\u001b[1;34mt\u001b[0m)\n", " rust/1.47.0 (\u001b[1;34mt\u001b[0m)\n", " rust/1.53.0 (\u001b[1;34mt\u001b[0m)\n", " rust/1.59.0 (\u001b[1;34mt\u001b[0m)\n", " rust/1.65.0 (\u001b[1;34mt\u001b[0m)\n", " rust/1.70.0 (\u001b[1;34mt\u001b[0m,D)\n", " sagemath/9.1 (\u001b[1;34mt\u001b[0m)\n", " sagemath/9.3 (\u001b[1;34mt\u001b[0m,D)\n", " salmonbeta/0.6.0 (\u001b[1;31mbio\u001b[0m)\n", " sambamba/0.8.0 (\u001b[1;31mbio\u001b[0m)\n", " samblaster/0.1.26 (\u001b[1;31mbio\u001b[0m)\n", " samtools/0.1.20 (\u001b[1;31mbio\u001b[0m)\n", " samtools/1.10 (\u001b[1;31mbio\u001b[0m)\n", " samtools/1.11 (\u001b[1;31mbio\u001b[0m)\n", " samtools/1.12 (\u001b[1;31mbio\u001b[0m)\n", " samtools/1.15.1 (\u001b[1;31mbio\u001b[0m)\n", " samtools/1.16.1 (\u001b[1;31mbio\u001b[0m)\n", " samtools/1.17 (\u001b[1;31mbio\u001b[0m,D)\n", " savvy/2.1.0\n", " sbt/1.3.13 (\u001b[1;34mt\u001b[0m)\n", " scipy-stack/2020a (\u001b[1;32mmath\u001b[0m)\n", " scipy-stack/2020b (\u001b[1;32mmath\u001b[0m)\n", " scipy-stack/2021a (\u001b[1;32mmath\u001b[0m)\n", " scipy-stack/2022a (\u001b[1;32mmath\u001b[0m)\n", " scipy-stack/2023a (\u001b[1;32mmath\u001b[0m)\n", " scipy-stack/2023b (\u001b[1;32mmath\u001b[0m,D)\n", " scythe/0.994\n", " sdl2/2.0.16 (\u001b[1;34mt\u001b[0m)\n", " sdsl/2.1.1-20191211\n", " seqan-library/2.4.0\n", " seqkit/0.13.2\n", " seqkit/0.15.0\n", " seqkit/2.3.1 (D)\n", " seqlib/1.2.0\n", " shapeit/2.r904\n", " shrinkwrap/1.2.0\n", " signalp/4.1f (\u001b[1;31mbio\u001b[0m)\n", " simbody/3.7\n", " simka/1.5.3\n", " singular/4.2.1\n", " sionlib/1.7.6\n", " slicer/4.11.20210226\n", " smartdenovo/20180219\n", " smithwaterman/20160702\n", " snappy/1.1.8 (\u001b[1;34mt\u001b[0m)\n", " sniffles/1.0.12b\n", " snp-sites/2.5.1\n", " snpeff/5.0 (\u001b[1;31mbio\u001b[0m)\n", " soapdenovo2/r242 (\u001b[1;31mbio\u001b[0m)\n", " soci/4.0.2\n", " sortmerna/4.3.6 (\u001b[1;31mbio\u001b[0m,D)\n", " spades/3.14.1 (\u001b[1;31mbio\u001b[0m)\n", " spades/3.15.1 (\u001b[1;31mbio\u001b[0m)\n", " spades/3.15.3 (\u001b[1;31mbio\u001b[0m)\n", " spades/3.15.4 (\u001b[1;31mbio\u001b[0m,D)\n", " spark/3.0.0 (\u001b[1;34mt\u001b[0m)\n", " spark/3.3.0 (\u001b[1;34mt\u001b[0m,D)\n", " sparsehash/2.0.4\n", " spdlog/1.9.2\n", " spectra/0.9.0\n", " spoa/3.4.0\n", " sqlite/3.38.5\n", " sqlite/3.43.1 (D)\n", " srprism/3.1.2\n", " ssw/1.1\n", " stacks/2.53 (\u001b[1;31mbio\u001b[0m)\n", " stacks/2.55 (\u001b[1;31mbio\u001b[0m)\n", " stacks/2.57 (\u001b[1;31mbio\u001b[0m)\n", " stacks/2.59 (\u001b[1;31mbio\u001b[0m)\n", " stacks/2.60 (\u001b[1;31mbio\u001b[0m)\n", " stacks/2.62 (\u001b[1;31mbio\u001b[0m)\n", " stacks/2.64 (\u001b[1;31mbio\u001b[0m,D)\n", " star/2.7.5a (\u001b[1;31mbio\u001b[0m)\n", " star/2.7.8a (\u001b[1;31mbio\u001b[0m)\n", " star/2.7.9a (\u001b[1;31mbio\u001b[0m,D)\n", " statistics-r/0.34\n", " stringtie/2.1.5 (\u001b[1;31mbio\u001b[0m)\n", " stringtie/2.2.1 (\u001b[1;31mbio\u001b[0m,D)\n", " subread/2.0.3 (\u001b[1;31mbio\u001b[0m)\n", " sumo/1.7.0\n", " sumo/1.9.0\n", " sumo/1.9.2\n", " sumo/1.15.0 (D)\n", " supernova/2.1.1 (\u001b[1;31mbio\u001b[0m)\n", " survivor/1.0.7\n", " svaba/1.1.0\n", " swi-prolog/9.0.3\n", " swig/4.0.1\n", " swig/4.0.1\n", " swig/4.0.2 (D)\n", " symengine/0.6.0\n", " symengine/0.8.1\n", " symengine/0.9.0\n", " symengine/0.10.1 (D)\n", " tabix/0.2.6 (\u001b[1;31mbio\u001b[0m)\n", " tabixpp/1.1.0\n", " taxonkit/0.6.2\n", " tbb/2020.2 (\u001b[1;34mt\u001b[0m)\n", " tbb/2021.8.0 (\u001b[1;34mt\u001b[0m,D)\n", " tbl2asn/25.8\n", " tesseract/4.1.3\n", " tesseract/5.0.1 (D)\n", " tmhmm/2.0c (\u001b[1;31mbio\u001b[0m)\n", " togl/2.0\n", " transrate/1.0.3 (\u001b[1;31mbio\u001b[0m)\n", " trimal/1.4 (\u001b[1;31mbio\u001b[0m)\n", " trimmomatic/0.39 (\u001b[1;31mbio\u001b[0m)\n", " trinity/2.11.0 (\u001b[1;31mbio\u001b[0m)\n", " trinity/2.12.0 (\u001b[1;31mbio\u001b[0m)\n", " trinity/2.13.2 (\u001b[1;31mbio\u001b[0m)\n", " trinity/2.14.0 (\u001b[1;31mbio\u001b[0m,D)\n", " ucc/1.0.0\n", " ucx/1.8.0 (\u001b[1;33mL\u001b[0m)\n", " ucx/1.9.0\n", " ucx/1.10.0\n", " ucx/1.12.1 (D)\n", " udunits/2.2.26 (\u001b[1;34mt\u001b[0m)\n", " udunits/2.2.28 (\u001b[1;34mt\u001b[0m,D)\n", " unixodbc/2.3.9\n", " varscan/2.4.2\n", " vcflib/1.0.1\n", " vcflib/1.0.3 (D)\n", " vcftools/0.1.16 (\u001b[1;31mbio\u001b[0m)\n", " velvet/1.2.10 (\u001b[1;31mbio\u001b[0m)\n", " viennarna/2.5.1 (\u001b[1;31mbio\u001b[0m,D)\n", " visit/2.13.3 (\u001b[1;34mvis\u001b[0m)\n", " vmatch/2.3.1\n", " voro++/0.4.6 (\u001b[1;32mmath\u001b[0m)\n", " vsearch/2.21.1 (\u001b[1;31mbio\u001b[0m,D)\n", " vtune/2020.1 (\u001b[1;34mt\u001b[0m)\n", " vtune/2022.2 (\u001b[1;34mt\u001b[0m,D)\n", " wasp/3.1.4\n", " wasp/4.0.3 (D)\n", " winnowmap/2.03\n", " wtdbg2/2.5\n", " wxwidgets/3.1.4 (\u001b[1;34mt\u001b[0m)\n", " wxwidgets/3.1.7 (\u001b[1;34mt\u001b[0m,D)\n", " xdrfile/1.1.4 (\u001b[1;34mt\u001b[0m)\n", " xerces-c++/3.2.2 (\u001b[1;34mt\u001b[0m)\n", " xerces-c++/3.2.3 (\u001b[1;34mt\u001b[0m,D)\n", " xml-libxml/2.0205\n", " xmlf90/1.5.4 (\u001b[1;34mt\u001b[0m)\n", " xtb/6.5.0 (\u001b[1;35mchem\u001b[0m)\n", " xtb/6.5.1 (\u001b[1;35mchem\u001b[0m,D)\n", " xtensor/0.24.2\n", " yaml-cpp/0.7.0\n", "\n", "------------------------------- Custom modules --------------------------------\n", " arch/avx arch/sse3 (D) gentoo/2023 (\u001b[1;31mS\u001b[0m)\n", " arch/avx2 CCconfig StdEnv/2020 (\u001b[1;31mS\u001b[0m)\n", " arch/avx512 gentoo/2020 (\u001b[1;31mS\u001b[0m,\u001b[1;33mL\u001b[0m,D) StdEnv/2023 (\u001b[1;31mS\u001b[0m,D)\n", "\n", " Where:\n", " \u001b[1;31mS\u001b[0m: Module is Sticky, requires --force to unload or purge\n", " \u001b[1;31mbio\u001b[0m: Bioinformatic libraries/apps / Logiciels de bioinformatique\n", " \u001b[1;31mm\u001b[0m: MPI implementations / Implémentations MPI\n", " \u001b[1;32mmath\u001b[0m: Mathematical libraries / Bibliothèques mathématiques\n", " \u001b[1;33mL\u001b[0m: Module is loaded\n", " \u001b[1;33mio\u001b[0m: Input/output software / Logiciel d'écriture/lecture\n", " \u001b[1;34mt\u001b[0m: Tools for development / Outils de développement\n", " \u001b[1;34mvis\u001b[0m: Visualisation software / Logiciels de visualisation\n", " \u001b[1;35mchem\u001b[0m: Chemistry libraries/apps / Logiciels de chimie\n", " \u001b[1;36mgeo\u001b[0m: Geography libraries/apps / Logiciels de géographie\n", " \u001b[1;36mphys\u001b[0m: Physics libraries/apps / Logiciels de physique\n", " Aliases: Aliases exist: foo/1.2.3 (1.2) means that \"module load foo/1.2\" will load foo/1.2.3\n", " D: Default Module\n", "\n", "If the avail list is too long consider trying:\n", "\n", "\"module --default avail\" or \"ml -d av\" to just list the default modules.\n", "\"module overview\" or \"ml ov\" to display the number of modules for each name.\n", "\n", "Use \"module spider\" to find all possible modules and extensions.\n", "Use \"module keyword key1 key2 ...\" to search for all possible modules matching\n", "any of the \"keys\".\n", "\n", "\n" ] } ], "source": [ "!module avail" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "Currently Loaded Modules:\n", " 1) gentoo/2020 (\u001b[1;31mS\u001b[0m) 4) gcc/9.3.0 (\u001b[1;34mt\u001b[0m) 7) openmpi/4.0.3 (\u001b[1;31mm\u001b[0m)\n", " 2) imkl/2020.1.217 (\u001b[1;32mmath\u001b[0m) 5) ucx/1.8.0\n", " 3) \u001b[2mgcccore/.9.3.0\u001b[0m (H) 6) libfabric/1.10.1\n", "\n", " Where:\n", " \u001b[1;31mS\u001b[0m: Module is Sticky, requires --force to unload or purge\n", " \u001b[1;31mm\u001b[0m: MPI implementations / Implémentations MPI\n", " \u001b[1;32mmath\u001b[0m: Mathematical libraries / Bibliothèques mathématiques\n", " \u001b[1;34mt\u001b[0m: Tools for development / Outils de développement\n", " H: Hidden Module\n", "\n", " \n", "\n" ] } ], "source": [ "!module list" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\n", "Currently Loaded Modules:\n", " 1) gentoo/2020 (S) 5) ucx/1.8.0\n", " 2) imkl/2020.1.217 (math) 6) libfabric/1.10.1\n", " 3) gcccore/.9.3.0 (H) 7) openmpi/4.0.3 (m)\n", " 4) gcc/9.3.0 (t) 8) openblas/0.3.17 (math)\n", "\n", " Where:\n", " H: Hidden Module\n", " S: Module is Sticky, requires --force to unload or purge\n", " m: MPI implementations / Implémentations MPI\n", " math: Mathematical libraries / Bibliothèques mathématiques\n", " t: Tools for development / Outils de développement\n", "\n", " \n", "\n" ] } ], "source": [ "%%bash\n", "# Load the OpenBLAS module\n", "module load openblas/0.3.17\n", "\n", "# Verify that the module has been loaded\n", "module list\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": { "id": "Wsjf9ryRhqJF" }, "source": [ "## 4. High-Performance Libraries for Scientific Computing\n", "\n", "Leveraging high-performance libraries can save development time and ensure that your code is optimized for modern HPC architectures.\n", "\n", "### 4.1 Using BLAS and LAPACK for Linear Algebra\n", "\n", "BLAS (Basic Linear Algebra Subprograms) and LAPACK (Linear Algebra Package) are standard libraries that provide optimized implementations of basic linear algebra routines.\n", "## Introduction to BLAS and LAPACK\n", "\n", "BLAS (Basic Linear Algebra Subprograms) and LAPACK (Linear Algebra Package) are highly optimized libraries that provide standard routines for performing common linear algebra operations. These libraries are widely used in scientific computing, engineering, and data analysis due to their efficiency and portability across different hardware architectures.\n", "\n", "### Why Use BLAS and LAPACK?\n", "\n", "BLAS and LAPACK are particularly useful for the following reasons:\n", "- **Performance**: These libraries are fine-tuned to utilize the underlying hardware, making them highly efficient for operations such as matrix multiplication, solving linear systems, and eigenvalue problems.\n", "- **Parallelism**: BLAS and LAPACK implementations often leverage multi-threading and hardware acceleration (e.g., using vectorized instructions), making them ideal for high-performance computing (HPC) environments.\n", "- **Portability**: BLAS and LAPACK are available on a wide range of platforms and are included in many high-performance libraries like Intel MKL, OpenBLAS, and ATLAS.\n", "\n", "### What Will We Do?\n", "\n", "In this exercise, we will:\n", "1. Perform matrix multiplication using BLAS's `dgemm` routine, which is specifically optimized for this task.\n", "2. Solve a system of linear equations using LAPACK's `dgesv` routine, which finds the solution to `AX = B` using LU factorization.\n", "\n", "By leveraging these libraries, we can efficiently handle large linear algebra problems that are common in scientific and engineering applications.\n", "\n" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "C program written to blas_lapack_example.c\n" ] } ], "source": [ "import os\n", "\n", "# Step 1: Write the C program to a file\n", "c_program = \"\"\"\n", "#include \n", "#include \n", "\n", "// Declare BLAS and LAPACK routines\n", "extern void dgemm_(char *transa, char *transb, int *m, int *n, int *k,\n", " double *alpha, double *A, int *lda, double *B, int *ldb,\n", " double *beta, double *C, int *ldc);\n", "\n", "extern void dgesv_(int *n, int *nrhs, double *A, int *lda, int *ipiv,\n", " double *B, int *ldb, int *info);\n", "\n", "void print_matrix(const char* name, double *matrix, int rows, int cols) {\n", " printf(\"%s:\\\\n\", name);\n", " for (int i = 0; i < rows; i++) {\n", " for (int j = 0; j < cols; j++) {\n", " printf(\"%f \", matrix[i * cols + j]);\n", " }\n", " printf(\"\\\\n\");\n", " }\n", "}\n", "\n", "int main() {\n", " // Example: Matrix multiplication using dgemm (BLAS)\n", " \n", " // Matrices A (3x3), B (3x3), and C (3x3) for A * B = C\n", " double A[9] = {1.0, 2.0, 3.0, \n", " 4.0, 5.0, 6.0, \n", " 7.0, 8.0, 9.0};\n", " double B[9] = {9.0, 8.0, 7.0, \n", " 6.0, 5.0, 4.0, \n", " 3.0, 2.0, 1.0};\n", " double C[9];\n", " \n", " int m = 3, n = 3, k = 3; // Dimensions of matrices\n", " double alpha = 1.0, beta = 0.0;\n", " \n", " // Matrix multiplication C = alpha * A * B + beta * C\n", " dgemm_(\"N\", \"N\", &m, &n, &k, &alpha, A, &m, B, &n, &beta, C, &m);\n", " \n", " // Print the result of A * B\n", " print_matrix(\"Matrix C (A * B)\", C, m, n);\n", "\n", " // Example: Solving a linear system using dgesv (LAPACK)\n", " \n", " // A (3x3) and B (3x1), solve A * X = B\n", " double A2[9] = {3.0, 2.0, -1.0,\n", " 2.0, -2.0, 4.0,\n", " -1.0, 0.5, -1.0};\n", " double B2[3] = {1.0, -2.0, 0.0}; // Right-hand side\n", " \n", " int ipiv[3]; // Pivot indices\n", " int info; // Return info\n", " int nrhs = 1; // Number of right-hand sides\n", " \n", " // Solve the system of equations A * X = B\n", " dgesv_(&m, &nrhs, A2, &m, ipiv, B2, &m, &info);\n", " \n", " if (info == 0) {\n", " // Print the solution\n", " print_matrix(\"Solution to A * X = B\", B2, m, nrhs);\n", " } else {\n", " printf(\"An error occurred: dgesv returned info = %d\\\\n\", info);\n", " }\n", "\n", " return 0;\n", "}\n", "\"\"\"\n", "\n", "# Save the C program to a file\n", "c_filename = \"blas_lapack_example.c\"\n", "with open(c_filename, \"w\") as c_file:\n", " c_file.write(c_program)\n", "\n", "print(f\"C program written to {c_filename}\")\n" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 437, "status": "ok", "timestamp": 1724421430380, "user": { "displayName": "Oscar Diez", "userId": "09940749782853693007" }, "user_tz": -120 }, "id": "ZyvxXyG4hsA0", "outputId": "61ab16a4-78d3-4ba1-9667-b5ce15174e6f" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Compilation successful.\n", "Program Output:\n", "Matrix C (A * B):\n", "90.000000 114.000000 138.000000 \n", "54.000000 69.000000 84.000000 \n", "18.000000 24.000000 30.000000 \n", "Solution to A * X = B:\n", "-1.333333 \n", "-3.166667 \n", "-11.333333 \n", "\n" ] } ], "source": [ "import subprocess\n", "\n", "# Step 3: Compile the C program using IMKL (Intel Math Kernel Library) instead of OpenBLAS\n", "compile_command = \"gcc -o blas_lapack_example blas_lapack_example.c -lmkl_rt\"\n", "\n", "# Run the compile command\n", "compile_process = subprocess.run(compile_command, shell=True, capture_output=True, text=True)\n", "\n", "# Output compilation results\n", "if compile_process.returncode == 0:\n", " print(\"Compilation successful.\")\n", "else:\n", " print(f\"Compilation failed:\\n{compile_process.stderr}\")\n", "\n", "# Step 4: Run the compiled binary and capture the output\n", "if compile_process.returncode == 0:\n", " run_command = \"./blas_lapack_example\"\n", " run_process = subprocess.run(run_command, shell=True, capture_output=True, text=True)\n", "\n", " # Output the results of the program\n", " if run_process.returncode == 0:\n", " print(f\"Program Output:\\n{run_process.stdout}\")\n", " else:\n", " print(f\"Program Error:\\n{run_process.stderr}\")\n" ] }, { "cell_type": "markdown", "metadata": { "id": "zqcoRrtWhxzM" }, "source": [ "## Explanation of the Code\n", "\n", "The C program provided uses BLAS and LAPACK routines to perform two common linear algebra operations: matrix multiplication and solving a system of linear equations.\n", "\n", "### Matrix Multiplication with BLAS (`dgemm`)\n", "\n", "The program first demonstrates matrix multiplication using the BLAS routine `dgemm`. This function performs the operation:\n", "\n", "\\[\n", "C = \\alpha \\times A \\times B + \\beta \\times C\n", "\\]\n", "\n", "Where:\n", "- `A`, `B`, and `C` are matrices,\n", "- `\\alpha` and `\\beta` are scalar values.\n", "\n", "In the code:\n", "- We define two 3x3 matrices `A` and `B` and multiply them to produce matrix `C`.\n", "- The `dgemm_` function from BLAS is called with appropriate arguments, performing the matrix multiplication and storing the result in `C`.\n", "- The function `print_matrix` is then used to print the resulting matrix `C`.\n", "\n", "### Solving a Linear System with LAPACK (`dgesv`)\n", "\n", "Next, the program solves the system of linear equations:\n", "\n", "\\[\n", "A \\times X = B\n", "\\]\n", "\n", "Where `A` is a matrix, and `B` is a vector. The LAPACK routine `dgesv` is used for this purpose, which computes the solution `X` by performing LU factorization of matrix `A`.\n", "\n", "In the code:\n", "- We define a 3x3 matrix `A2` and a 3x1 vector `B2`.\n", "- The `dgesv_` function from LAPACK is used to solve the system. The solution vector `X` (which replaces `B2` after the call) is printed using the `print_matrix` function.\n", "- The `dgesv_` function internally performs LU decomposition and uses the result to compute the solution. The pivot indices required for LU factorization are stored in the array `ipiv`.\n", "\n", "### Why Use BLAS and LAPACK?\n", "\n", "BLAS and LAPACK provide highly efficient and reliable methods to perform these operations. By using these libraries, you can benefit from:\n", "- **Speed**: These routines are often faster than hand-written matrix multiplication or equation solvers.\n", "- **Stability**: LAPACK uses numerically stable algorithms to ensure the accuracy of the solutions.\n", "- **Flexibility**: The same routines can handle matrices of various sizes, allowing scalability to larger problems.\n", "\n", "This exercise demonstrates the power of using these libraries in computational tasks, particularly for high-performance computing or large-scale data analysis.\n" ] }, { "cell_type": "markdown", "metadata": { "id": "OuV7gdjJhzVl" }, "source": [ "## Advantages of Using Optimized Libraries vs. Writing Your Own Code\n", "\n", "When performing complex linear algebra operations, you have two options: write your own code or use highly optimized libraries like BLAS and LAPACK. While it may seem tempting to implement these algorithms yourself for learning purposes, using optimized libraries offers many advantages, especially in High-Performance Computing (HPC) environments.\n", "\n", "### 1. **Performance**\n", "Optimized libraries like BLAS and LAPACK are carefully designed and fine-tuned to take advantage of modern CPU architectures. They leverage low-level optimizations, such as vectorization, multi-threading, and cache utilization, to ensure that matrix operations are performed as quickly as possible.\n", "\n", "- **Custom Code**: Your implementation may work well for small matrices but will likely struggle with large datasets, leading to increased runtime and resource consumption.\n", "- **Optimized Libraries**: By using BLAS and LAPACK, you can achieve orders of magnitude faster performance for large matrices and complex operations, as these libraries are built to scale.\n", "\n", "### 2. **Stability and Accuracy**\n", "Numerical stability is a key concern in scientific computing. Libraries like LAPACK use robust, tested algorithms to ensure that operations like solving linear systems are performed with maximum precision.\n", "\n", "- **Custom Code**: Writing your own solver may introduce numerical inaccuracies, especially for large matrices or ill-conditioned systems.\n", "- **Optimized Libraries**: LAPACK's routines ensure that matrix operations are numerically stable, providing accurate solutions even for complex problems.\n", "\n", "### 3. **Scalability and Multi-node Execution**\n", "In HPC environments, scaling applications across multiple nodes is essential for handling large datasets. Optimized libraries like BLAS and LAPACK are designed to work efficiently on multiple processors and nodes, making them ideal for distributed computing.\n", "\n", "- **Custom Code**: Implementing multi-node support from scratch requires significant development effort, including managing communication between nodes and optimizing memory access patterns.\n", "- **Optimized Libraries**: Many versions of BLAS and LAPACK, such as those provided by OpenMPI or Intel MKL, support multi-node execution, making it easy to scale your code across multiple processors.\n", "\n", "In summary, using libraries like BLAS and LAPACK saves development time, ensures the accuracy of results, and significantly boosts performance, especially when scaling to large problems in an HPC setting.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Using BLAS and LAPACK Across Multiple Nodes\n", "\n", "In large-scale HPC systems, leveraging multiple nodes can significantly reduce computation time. Many implementations of BLAS and LAPACK, such as Intel's MKL and OpenMPI, provide support for distributing tasks across multiple nodes in a cluster.\n", "\n", "### How to Use BLAS and LAPACK with Multiple Nodes\n", "\n", "When running BLAS and LAPACK operations on multiple nodes, you typically rely on MPI (Message Passing Interface) to manage communication between nodes. Here's how it works:\n", "\n", "1. **MPI for Parallel Execution**: MPI is used to distribute matrix data across nodes. Each node will handle a portion of the matrix, and BLAS or LAPACK routines are used to perform the calculations locally. The results are then communicated back to the master node.\n", "\n", "2. **Scalability**: As the matrix size grows, distributing computations across nodes allows you to process larger datasets more quickly. This is particularly beneficial for tasks like matrix multiplication and solving linear systems.\n", "\n", "3. **Load Balancing**: Libraries like ScaLAPACK (a parallelized version of LAPACK) ensure that workloads are evenly distributed across nodes, optimizing the overall computation time.\n", "\n", "### Example: Running BLAS on Multiple Nodes\n", "\n", "Below is an example of how to modify your code to run on multiple nodes using MPI. We'll use `MPI_Init` to initialize MPI and `MPI_Finalize` to clean up at the end.\n" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "C program written to mpi_blas_example.c\n", "Program Output:\n", "Matrix C (A * B):\n", "90.000000 114.000000 138.000000 \n", "54.000000 69.000000 84.000000 \n", "18.000000 24.000000 30.000000 \n", "\n" ] } ], "source": [ "import os\n", "import subprocess\n", "\n", "# Step 1: Write the C program to a file\n", "c_program = \"\"\"\n", "#include \n", "#include \n", "#include \n", "\n", "extern void dgemm_(char *transa, char *transb, int *m, int *n, int *k,\n", " double *alpha, double *A, int *lda, double *B, int *ldb,\n", " double *beta, double *C, int *ldc);\n", "\n", "void print_matrix(const char* name, double *matrix, int rows, int cols) {\n", " printf(\"%s:\\\\n\", name);\n", " for (int i = 0; i < rows; i++) {\n", " for (int j = 0; j < cols; j++) {\n", " printf(\"%f \", matrix[i * cols + j]);\n", " }\n", " printf(\"\\\\n\");\n", " }\n", "}\n", "\n", "int main(int argc, char** argv) {\n", " MPI_Init(&argc, &argv);\n", "\n", " int world_size;\n", " MPI_Comm_size(MPI_COMM_WORLD, &world_size);\n", "\n", " int world_rank;\n", " MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);\n", "\n", " double A[9] = {1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0};\n", " double B[9] = {9.0, 8.0, 7.0, 6.0, 5.0, 4.0, 3.0, 2.0, 1.0};\n", " double C[9];\n", "\n", " int m = 3, n = 3, k = 3;\n", " double alpha = 1.0, beta = 0.0;\n", "\n", " if (world_rank == 0) {\n", " dgemm_(\"N\", \"N\", &m, &n, &k, &alpha, A, &m, B, &n, &beta, C, &m);\n", " print_matrix(\"Matrix C (A * B)\", C, m, n);\n", " }\n", "\n", " MPI_Barrier(MPI_COMM_WORLD);\n", " MPI_Finalize();\n", " return 0;\n", "}\n", "\"\"\"\n", "\n", "# Write the C code to a file\n", "c_filename = \"mpi_blas_example.c\"\n", "with open(c_filename, \"w\") as c_file:\n", " c_file.write(c_program)\n", "\n", "print(f\"C program written to {c_filename}\")\n", "\n", "# Step 2: Compile the C program using IMKL or OpenBLAS and MPI\n", "compile_command = \"mpicc -o mpi_blas_example mpi_blas_example.c -lmkl_rt\"\n", "\n", "# Run the compile command\n", "compile_process = subprocess.run(compile_command, shell=True, capture_output=True, text=True)\n", "\n", "# Step 3: Run the compiled binary and capture the output using mpirun\n", "if compile_process.returncode == 0:\n", " # Enable oversubscription in the cluster\n", " os.environ[\"OMPI_MCA_rmaps_base_oversubscribe\"] = \"1\"\n", " \n", " # Using mpirun to run the program with 4 processes\n", " run_command = \"mpirun -np 4 ./mpi_blas_example\"\n", " \n", " # Run the program\n", " run_process = subprocess.run(run_command, shell=True, capture_output=True, text=True)\n", " \n", " # Output the results of the program\n", " if run_process.returncode == 0:\n", " print(f\"Program Output:\\n{run_process.stdout}\")\n", " else:\n", " print(f\"Program Error:\\n{run_process.stderr}\")\n", "else:\n", " print(f\"Compilation failed:\\n{compile_process.stderr}\")\n" ] }, { "cell_type": "markdown", "metadata": { "id": "Bew1446Ah2PF" }, "source": [ "## 5.1 Introduction to Parallel I/O in HPC\n", "\n", "Parallel I/O is essential in High-Performance Computing (HPC) environments, especially when dealing with large datasets. In typical serial I/O operations, a single process reads or writes data, creating bottlenecks as file sizes grow. In contrast, **Parallel I/O** allows multiple processes to perform I/O operations concurrently, which significantly increases the performance of data-intensive applications.\n", "\n", "Parallel I/O is especially useful when combined with parallel filesystems like **Lustre** or **GPFS**. These filesystems are specifically designed to allow many processes to read and write large amounts of data simultaneously, distributing I/O operations across multiple storage devices to provide high throughput and scalability.\n", "\n", "In this section, we will demonstrate parallel I/O using **C** and **MPI**, where multiple processes write their data to a shared file concurrently and read the data back in parallel.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 5.2 Understanding Parallel Filesystems\n", "\n", "Parallel filesystems, such as **Lustre**, **GPFS**, or **BeeGFS**, are designed for use in HPC environments to manage large-scale data operations. They distribute data across multiple storage devices, enabling multiple processes to read and write data simultaneously.\n", "\n", "### Key Features of Parallel Filesystems:\n", "- **High throughput**: Achieved by distributing data across multiple storage servers.\n", "- **Scalability**: Able to handle large datasets and a high number of concurrent processes.\n", "- **Redundancy**: Data is often stored redundantly across multiple disks to prevent data loss in case of hardware failure.\n", "- **Concurrent Access**: Multiple processes can access the same file at the same time, significantly improving performance in distributed applications.\n", "\n", "In a typical scenario, an application running on hundreds or thousands of compute nodes in a supercomputer accesses data stored in a parallel filesystem. Instead of waiting for one process to finish its I/O operation, each process can handle its I/O independently and concurrently, improving the overall performance.\n" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "C program written to parallel_io_example.c\n" ] } ], "source": [ "# Write the C code to a file for parallel I/O with MPI\n", "\n", "c_code = \"\"\"\n", "#include \n", "#include \n", "#include \n", "\n", "int main(int argc, char **argv) {\n", " MPI_Init(&argc, &argv); // Initialize MPI environment\n", "\n", " int rank, size;\n", " MPI_Comm_rank(MPI_COMM_WORLD, &rank); // Get process rank\n", " MPI_Comm_size(MPI_COMM_WORLD, &size); // Get number of processes\n", "\n", " // Create a large array filled with the rank number\n", " int N = 1000000; // Size of the array\n", " int *data = (int*) malloc(N * sizeof(int));\n", " for (int i = 0; i < N; i++) {\n", " data[i] = rank;\n", " }\n", "\n", " // Open a shared file for writing\n", " MPI_File fh;\n", " MPI_File_open(MPI_COMM_WORLD, \"output.dat\", MPI_MODE_CREATE | MPI_MODE_WRONLY, MPI_INFO_NULL, &fh);\n", "\n", " // Each process writes its data at the correct offset in the file\n", " MPI_File_write_at_all(fh, rank * N * sizeof(int), data, N, MPI_INT, MPI_STATUS_IGNORE);\n", " \n", " // Close the file after writing\n", " MPI_File_close(&fh);\n", "\n", " // Synchronize all processes\n", " MPI_Barrier(MPI_COMM_WORLD);\n", "\n", " // Allocate space for reading back the data\n", " int *read_data = (int*) malloc(N * sizeof(int));\n", "\n", " // Open the file again for reading\n", " MPI_File_open(MPI_COMM_WORLD, \"output.dat\", MPI_MODE_RDONLY, MPI_INFO_NULL, &fh);\n", "\n", " // Each process reads its data back from the file\n", " MPI_File_read_at_all(fh, rank * N * sizeof(int), read_data, N, MPI_INT, MPI_STATUS_IGNORE);\n", "\n", " // Close the file after reading\n", " MPI_File_close(&fh);\n", "\n", " // Verify by printing the first and last elements of the read data\n", " printf(\"Process %d: First element = %d, Last element = %d\\\\n\", rank, read_data[0], read_data[N-1]);\n", "\n", " // Free allocated memory\n", " free(data);\n", " free(read_data);\n", "\n", " // Finalize the MPI environment\n", " MPI_Finalize();\n", " return 0;\n", "}\n", "\"\"\"\n", "\n", "# Save the C code to a file\n", "c_filename = \"parallel_io_example.c\"\n", "with open(c_filename, \"w\") as c_file:\n", " c_file.write(c_code)\n", "\n", "print(f\"C program written to {c_filename}\")\n" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Process 2: First element = 2, Last element = 2\n", "Process 3: First element = 3, Last element = 3\n", "Process 0: First element = 0, Last element = 0\n", "Process 1: First element = 1, Last element = 1\n" ] } ], "source": [ "%%bash\n", "# Step 1: Compile the C program using mpicc\n", "mpicc -o parallel_io_example parallel_io_example.c\n", "\n", "# Step 2: Run the compiled program with multiple processes using mpirun\n", "mpirun -np 4 ./parallel_io_example\n" ] }, { "cell_type": "markdown", "metadata": { "id": "SI1WMykbnlK6" }, "source": [ "## 5.3 Explanation of the Parallel I/O Code\n", "\n", "In this code example, we use **MPI** (Message Passing Interface) to demonstrate how parallel I/O works. Here’s a breakdown of the key operations:\n", "\n", "1. **MPI Initialization**:\n", " The `MPI_Init()` function initializes the MPI environment, enabling communication between multiple processes running on different nodes.\n", "\n", "2. **Process Rank and Size**:\n", " - `MPI_Comm_rank()` retrieves the rank (ID) of each process.\n", " - `MPI_Comm_size()` retrieves the total number of processes involved.\n", "\n", "3. **Array Creation**:\n", " Each process creates a large array filled with its own rank number. This simulates data generation on each process that will be written to a shared file.\n", "\n", "4. **Parallel Write Operation**:\n", " - `MPI_File_open()` opens the file in **write mode** (`MPI_MODE_CREATE | MPI_MODE_WRONLY`).\n", " - `MPI_File_write_at_all()` is used to perform a parallel write. Each process writes its portion of the data at a specific offset, ensuring no overlap between processes' data.\n", "\n", "5. **Barrier Synchronization**:\n", " After the write operation, `MPI_Barrier()` ensures that all processes finish writing before proceeding to the next step.\n", "\n", "6. **Parallel Read Operation**:\n", " - `MPI_File_open()` reopens the file in **read mode** (`MPI_MODE_RDONLY`).\n", " - `MPI_File_read_at_all()` allows each process to read its portion of the data in parallel from the shared file.\n", "\n", "7. **Data Verification**:\n", " Each process prints the first and last elements of the data it read from the file. This verifies that the data was written and read correctly in parallel.\n", "\n", "### Advantages:\n", "- **Efficiency**: By allowing concurrent I/O operations, we significantly reduce the time required to perform large-scale data reads and writes.\n", "- **Scalability**: The code can be scaled up to hundreds or thousands of processes, leveraging the full potential of parallel filesystems.\n", "- **No Bottlenecks**: Since each process performs I/O independently, there are no bottlenecks caused by sequential file access, making the solution ideal for large-scale HPC applications.\n" ] }, { "cell_type": "markdown", "metadata": { "id": "y9v6MP3Hob1D" }, "source": [ "## 6.1 Introduction to Performance Tuning and Analysis\n", "\n", "In High-Performance Computing (HPC), performance tuning is a critical step for ensuring that applications run as efficiently as possible. With large-scale computations, identifying and resolving bottlenecks in code can lead to substantial performance improvements.\n", "\n", "Performance tuning typically involves:\n", "\n", "1. **Profiling**: This involves identifying bottlenecks by analyzing where the application spends most of its time.\n", "2. **Optimization**: Applying various optimizations, such as reducing memory usage, improving I/O operations, or parallelizing parts of the code.\n", "3. **Reprofiling**: After optimizations are applied, reprofile the application to assess the impact of the changes and iterate if necessary.\n", "\n", "We will use tools such as `gprof`, `perf`, and `Intel VTune` to analyze the performance of a matrix computation in C with MPI.\n" ] }, { "cell_type": "markdown", "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 11213, "status": "ok", "timestamp": 1724423688169, "user": { "displayName": "Oscar Diez", "userId": "09940749782853693007" }, "user_tz": -120 }, "id": "s3wLg1NKpFEi", "outputId": "ede28dc7-3c86-4700-ddfd-248a0d99906a" }, "source": [ "## 6.2 Profiling with Standard HPC Tools\n", "\n", "In this exercise, we will create and compile a C program that performs matrix multiplication using MPI. We will then profile this program using **gprof** and **perf**, two widely used profiling tools in HPC.\n", "\n", "- **gprof**: A GNU profiler that shows the time spent in each function and helps pinpoint performance bottlenecks.\n", "- **perf**: A performance monitoring tool that provides detailed reports on CPU cycles, cache misses, and other hardware events.\n", "\n", "### Step 1: Writing the Matrix Multiplication Code\n", "\n", "First, we will write a simple C program that performs matrix multiplication using MPI. The program will create matrices, distribute work among multiple processes, and combine the results.\n" ] }, { "cell_type": "code", "execution_count": 35, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 2073, "status": "ok", "timestamp": 1724423697165, "user": { "displayName": "Oscar Diez", "userId": "09940749782853693007" }, "user_tz": -120 }, "id": "pPw0GkixpG0i", "outputId": "19b4669a-940a-46e3-e99b-74b545e51218" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Writing mpi_matrix_multiply.c\n" ] } ], "source": [ "%%writefile mpi_matrix_multiply.c\n", "\n", "#include \n", "#include \n", "#include \n", "\n", "// Function to perform matrix multiplication\n", "void matrix_multiply(int n, double* A, double* B, double* C) {\n", " for (int i = 0; i < n; i++) {\n", " for (int j = 0; j < n; j++) {\n", " C[i*n + j] = 0.0;\n", " for (int k = 0; k < n; k++) {\n", " C[i*n + j] += A[i*n + k] * B[k*n + j];\n", " }\n", " }\n", " }\n", "}\n", "\n", "int main(int argc, char** argv) {\n", " MPI_Init(&argc, &argv);\n", "\n", " int world_size;\n", " MPI_Comm_size(MPI_COMM_WORLD, &world_size);\n", "\n", " int world_rank;\n", " MPI_Comm_rank(MPI_COMM_WORLD, &world_rank);\n", "\n", " int n = 1000; // Matrix size\n", " double *A, *B, *C;\n", "\n", " if (world_rank == 0) {\n", " // Allocate memory for matrices A, B, and C\n", " A = (double*) malloc(n * n * sizeof(double));\n", " B = (double*) malloc(n * n * sizeof(double));\n", " C = (double*) malloc(n * n * sizeof(double));\n", "\n", " // Initialize matrices A and B\n", " for (int i = 0; i < n*n; i++) {\n", " A[i] = rand() % 100;\n", " B[i] = rand() % 100;\n", " }\n", " }\n", "\n", " // Perform matrix multiplication on rank 0 process\n", " if (world_rank == 0) {\n", " matrix_multiply(n, A, B, C);\n", " printf(\"Matrix multiplication completed.\\n\");\n", " }\n", "\n", " MPI_Finalize();\n", "\n", " if (world_rank == 0) {\n", " free(A);\n", " free(B);\n", " free(C);\n", " }\n", "\n", " return 0;\n", "}\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 6.3 Compilation and Profiling the Program\n", "\n", "We will now compile the program using `mpicc` (MPI C Compiler) and then profile it using `gprof` and `perf`. \n", "\n", "### Step 2: Compile the Program\n", "\n", "Use `mpicc` to compile the matrix multiplication program:\n", "\n", "The -pg flag enables profiling for gprof.\n", "\n", "### Step 3: Run the Program\n", "Next, we will run the program using mpirun:\n", "\n", "### Step 4: Profiling with gprof\n", "After the program runs, you can generate a profiling report with gprof:\n" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Compilation successful.\n", "Program Output:\n", "Matrix multiplication completed.\n", "\n", "Gprof Analysis Output:\n", "Flat profile:\n", "\n", "Each sample counts as 0.01 seconds.\n", " % cumulative self self total \n", " time seconds seconds calls s/call s/call name \n", "100.00 8.13 8.13 1 8.13 8.13 matrix_multiply\n", "\n", " % the percentage of the total running time of the\n", "time program used by this function.\n", "\n", "cumulative a running sum of the number of seconds accounted\n", " seconds for by this function and those listed above it.\n", "\n", " self the number of seconds accounted for by this\n", "seconds function alone. This is the major sort for this\n", " listing.\n", "\n", "calls the number of times this function was invoked, if\n", " this function is profiled, else blank.\n", "\n", " self the average number of milliseconds spent in this\n", "ms/call function per call, if this function is profiled,\n", "\t else blank.\n", "\n", " total the average number of milliseconds spent in this\n", "ms/call function and its descendents per call, if this\n", "\t function is profiled, else blank.\n", "\n", "name the name of the function. This is the minor sort\n", " for this listing. The index shows the location of\n", "\t the function in the gprof listing. If the index is\n", "\t in parenthesis it shows where it would appear in\n", "\t the gprof listing if it were to be printed.\n", "\f", "\n", "Copyright (C) 2012-2019 Free Software Foundation, Inc.\n", "\n", "Copying and distribution of this file, with or without modification,\n", "are permitted in any medium without royalty provided the copyright\n", "notice and this notice are preserved.\n", "\f", "\n", "\t\t Call graph (explanation follows)\n", "\n", "\n", "granularity: each sample hit covers 4 byte(s) for 0.12% of 8.13 seconds\n", "\n", "index % time self children called name\n", " 8.13 0.00 1/1 main [2]\n", "[1] 100.0 8.13 0.00 1 matrix_multiply [1]\n", "-----------------------------------------------\n", " \n", "[2] 100.0 0.00 8.13 main [2]\n", " 8.13 0.00 1/1 matrix_multiply [1]\n", "-----------------------------------------------\n", "\n", " This table describes the call tree of the program, and was sorted by\n", " the total amount of time spent in each function and its children.\n", "\n", " Each entry in this table consists of several lines. The line with the\n", " index number at the left hand margin lists the current function.\n", " The lines above it list the functions that called this function,\n", " and the lines below it list the functions this one called.\n", " This line lists:\n", " index\tA unique number given to each element of the table.\n", "\t\tIndex numbers are sorted numerically.\n", "\t\tThe index number is printed next to every function name so\n", "\t\tit is easier to look up where the function is in the table.\n", "\n", " % time\tThis is the percentage of the `total' time that was spent\n", "\t\tin this function and its children. Note that due to\n", "\t\tdifferent viewpoints, functions excluded by options, etc,\n", "\t\tthese numbers will NOT add up to 100%.\n", "\n", " self\tThis is the total amount of time spent in this function.\n", "\n", " children\tThis is the total amount of time propagated into this\n", "\t\tfunction by its children.\n", "\n", " called\tThis is the number of times the function was called.\n", "\t\tIf the function called itself recursively, the number\n", "\t\tonly includes non-recursive calls, and is followed by\n", "\t\ta `+' and the number of recursive calls.\n", "\n", " name\tThe name of the current function. The index number is\n", "\t\tprinted after it. If the function is a member of a\n", "\t\tcycle, the cycle number is printed between the\n", "\t\tfunction's name and the index number.\n", "\n", "\n", " For the function's parents, the fields have the following meanings:\n", "\n", " self\tThis is the amount of time that was propagated directly\n", "\t\tfrom the function into this parent.\n", "\n", " children\tThis is the amount of time that was propagated from\n", "\t\tthe function's children into this parent.\n", "\n", " called\tThis is the number of times this parent called the\n", "\t\tfunction `/' the total number of times the function\n", "\t\twas called. Recursive calls to the function are not\n", "\t\tincluded in the number after the `/'.\n", "\n", " name\tThis is the name of the parent. The parent's index\n", "\t\tnumber is printed after it. If the parent is a\n", "\t\tmember of a cycle, the cycle number is printed between\n", "\t\tthe name and the index number.\n", "\n", " If the parents of the function cannot be determined, the word\n", " `' is printed in the `name' field, and all the other\n", " fields are blank.\n", "\n", " For the function's children, the fields have the following meanings:\n", "\n", " self\tThis is the amount of time that was propagated directly\n", "\t\tfrom the child into the function.\n", "\n", " children\tThis is the amount of time that was propagated from the\n", "\t\tchild's children to the function.\n", "\n", " called\tThis is the number of times the function called\n", "\t\tthis child `/' the total number of times the child\n", "\t\twas called. Recursive calls by the child are not\n", "\t\tlisted in the number after the `/'.\n", "\n", " name\tThis is the name of the child. The child's index\n", "\t\tnumber is printed after it. If the child is a\n", "\t\tmember of a cycle, the cycle number is printed\n", "\t\tbetween the name and the index number.\n", "\n", " If there are any cycles (circles) in the call graph, there is an\n", " entry for the cycle-as-a-whole. This entry shows who called the\n", " cycle (as parents) and the members of the cycle (as children.)\n", " The `+' recursive calls entry shows the number of function calls that\n", " were internal to the cycle, and the calls entry for each member shows,\n", " for that member, how many times it was called from other members of\n", " the cycle.\n", "\f", "\n", "Copyright (C) 2012-2019 Free Software Foundation, Inc.\n", "\n", "Copying and distribution of this file, with or without modification,\n", "are permitted in any medium without royalty provided the copyright\n", "notice and this notice are preserved.\n", "\f", "\n", "Index by function name\n", "\n", " [1] matrix_multiply\n", "\n", "Perf Output:\n", "Matrix multiplication completed.\n", "\n" ] } ], "source": [ "import subprocess\n", "\n", "# Step 1: Compile the C program\n", "compile_process = subprocess.run(\n", " \"mpicc -pg -o mpi_matrix_multiply mpi_matrix_multiply.c\", \n", " shell=True, capture_output=True, text=True\n", ")\n", "\n", "# Check if compilation was successful\n", "if compile_process.returncode == 0:\n", " print(\"Compilation successful.\")\n", " \n", " # Step 2: Run the program on 4 processes with oversubscription\n", " run_command = \"mpirun --oversubscribe -np 4 ./mpi_matrix_multiply\"\n", " \n", " # Run the program and capture the output\n", " run_process = subprocess.run(run_command, shell=True, capture_output=True, text=True)\n", " \n", " if run_process.returncode == 0:\n", " print(\"Program Output:\")\n", " print(run_process.stdout) # Display the output of the program\n", " else:\n", " print(f\"Program Error:\\n{run_process.stderr}\")\n", " \n", " # Step 3: Generate profiling report with gprof\n", " gprof_command = \"gprof ./mpi_matrix_multiply gmon.out > analysis.txt\"\n", " subprocess.run(gprof_command, shell=True)\n", " \n", " # Display the contents of the gprof output file\n", " with open(\"analysis.txt\", \"r\") as file:\n", " analysis_output = file.read()\n", " print(\"Gprof Analysis Output:\")\n", " print(analysis_output)\n", " \n", " # Step 4: Profiling with perf\n", " perf_command = \"perf stat -d mpirun --oversubscribe -np 4 ./mpi_matrix_multiply\"\n", " perf_process = subprocess.run(perf_command, shell=True, capture_output=True, text=True)\n", " \n", " if perf_process.returncode == 0:\n", " print(\"Perf Output:\")\n", " print(perf_process.stdout) # Display the perf output\n", " else:\n", " print(f\"Perf Error:\\n{perf_process.stderr}\")\n", "else:\n", " print(f\"Compilation failed:\\n{compile_process.stderr}\")\n" ] }, { "cell_type": "markdown", "metadata": { "id": "CgKcVao9pLIC" }, "source": [ "### Understanding and Interpreting Performance Profiling\n", "\n", "In this section, we will compile, run, and profile an MPI-based matrix multiplication program using `gprof` and `perf`.\n", "\n", "#### Step-by-Step Process:\n", "\n", "1. **Compilation with Profiling Enabled**:\n", " - The program is compiled using `mpicc` with the `-pg` flag to enable profiling for `gprof`.\n", " - This generates a binary (`mpi_matrix_multiply`) and enables the collection of profiling data during execution.\n", "\n", "2. **Execution with `mpirun`**:\n", " - The program is executed using `mpirun` with 4 processes and the `--oversubscribe` option. Oversubscription allows more processes than available CPUs to run.\n", " - The matrix multiplication completes, and profiling data is generated in a `gmon.out` file.\n", "\n", "3. **Profiling with `gprof`**:\n", " - `gprof` analyzes the execution of the program and provides a breakdown of where time is being spent.\n", " - The `analysis.txt` file contains detailed profiling data, including:\n", " - **Flat Profile**: Shows the time spent in each function. Use this to identify the most time-consuming functions.\n", " - **Call Graph**: Displays the call hierarchy and the time spent in both the parent and child functions. This helps understand which functions call other functions and their time cost.\n", "\n", "4. **System-Level Profiling with `perf`**:\n", " - `perf` provides hardware-level insights into CPU usage, cache hits, and system resource utilization during the execution of the program.\n", " - Key metrics to interpret from `perf`:\n", " - **task-clock**: Total time spent on the task.\n", " - **context-switches**: Number of context switches during execution.\n", " - **page-faults**: Number of memory page faults.\n", " - **cycles/instructions/branches**: These metrics are crucial for understanding how efficiently the program is running on the CPU (if supported by the system).\n", "\n", "#### How to Interpret the Output:\n", "\n", "- **`gprof` Output**:\n", " - Identify the functions with the highest time consumption. These are the bottlenecks where optimization efforts should focus.\n", " - Review the call graph to see how functions interact and whether any redundant calls can be optimized.\n", "\n", "- **`perf` Output**:\n", " - High context switches or page faults may indicate inefficient resource management.\n", " - The `cycles` and `instructions` counters (if supported) help assess CPU efficiency. A high ratio of instructions per cycle is desirable.\n", "\n", "By combining the insights from both `gprof` and `perf`, you can target both function-level and system-level optimizations to improve the performance of your MPI program.\n" ] }, { "cell_type": "markdown", "metadata": { "id": "vds-N48VpMqS" }, "source": [ "## 8. Applying Optimizations\n", "\n", "Once bottlenecks are identified, the next step is to apply optimizations. In this section, we will optimize matrix operations using techniques such as loop unrolling, vectorization, and memory access optimization.\n", "\n", "### 8.1 Loop Unrolling and Vectorization\n", "\n", "We will revisit loop unrolling and vectorization to see how they can improve performance in matrix operations.\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 31629, "status": "ok", "timestamp": 1724423753151, "user": { "displayName": "Oscar Diez", "userId": "09940749782853693007" }, "user_tz": -120 }, "id": "dhh0RdPMpN-7", "outputId": "b9ed8497-4cf8-425c-84cd-571a70f08310" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Basic matrix sum time: 1.3154182434082031\n", "Vectorized matrix sum time: 0.1225118637084961\n" ] } ], "source": [ "import random\n", "import time\n", "\n", "# Function to create a large matrix using lists\n", "def create_matrix(rows, cols):\n", " return [[random.random() for _ in range(cols)] for _ in range(rows)]\n", "\n", "# Function to sum matrix elements using nested loops\n", "def basic_matrix_sum(matrix):\n", " total = 0\n", " for i in range(len(matrix)):\n", " for j in range(len(matrix[0])):\n", " total += matrix[i][j]\n", " return total\n", "\n", "# Function to sum matrix elements using list comprehension (pseudo-vectorized)\n", "def vectorized_matrix_sum(matrix):\n", " return sum(sum(row) for row in matrix)\n", "\n", "# Create a large matrix using list of lists\n", "matrix = create_matrix(4000, 4000)\n", "\n", "# Measure time for basic matrix sum\n", "start_time = time.time()\n", "basic_sum = basic_matrix_sum(matrix)\n", "print(\"Basic matrix sum time:\", time.time() - start_time)\n", "\n", "# Measure time for pseudo-vectorized matrix sum\n", "start_time = time.time()\n", "vectorized_sum = vectorized_matrix_sum(matrix)\n", "print(\"Vectorized matrix sum time:\", time.time() - start_time)\n" ] }, { "cell_type": "markdown", "metadata": { "id": "rvuGwJtopPWB" }, "source": [ "### Explanation:\n", "\n", "This example compares the performance of a basic loop-based matrix sum with a vectorized version using NumPy's built-in `sum` function. Vectorization allows for faster computation by leveraging SIMD instructions.\n", "\n", "### Exercise:\n", "\n", "Try optimizing the `basic_matrix_sum` function by manually unrolling the loops. Measure the performance impact and compare it with the vectorized approach.\n" ] }, { "cell_type": "markdown", "metadata": { "id": "KAmDUSMPpRma" }, "source": [ "## 9. Memory Access Optimization and Cache Utilization\n", "\n", "Memory access patterns greatly affect the performance of HPC applications. In this section, we'll explore techniques to optimize memory access and improve cache utilization.\n", "\n", "### 9.1 Optimizing Memory Access Patterns\n", "\n", "Efficient memory access patterns reduce cache misses, leading to faster execution times. We'll analyze the impact of row-major vs. column-major access.\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 66767, "status": "ok", "timestamp": 1724423826332, "user": { "displayName": "Oscar Diez", "userId": "09940749782853693007" }, "user_tz": -120 }, "id": "EvLNYS3OpQkT", "outputId": "4ce058a5-24b6-4f1b-ab9f-5e0c97b6ad3b" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Row-major sum time: 0.7301278114318848\n", "Column-major sum time: 1.8114192485809326\n" ] } ], "source": [ "import random\n", "import time\n", "\n", "# Function to create a large matrix using lists\n", "def create_matrix(rows, cols):\n", " return [[random.random() for _ in range(cols)] for _ in range(rows)]\n", "\n", "# Function to sum matrix elements in row-major order\n", "def row_major_sum(matrix):\n", " total = 0\n", " for i in range(len(matrix)):\n", " for j in range(len(matrix[0])):\n", " total += matrix[i][j]\n", " return total\n", "\n", "# Function to sum matrix elements in column-major order\n", "def column_major_sum(matrix):\n", " total = 0\n", " for j in range(len(matrix[0])):\n", " for i in range(len(matrix)):\n", " total += matrix[i][j]\n", " return total\n", "\n", "# Create a large matrix using list of lists\n", "matrix = create_matrix(3000, 3000)\n", "\n", "# Measure row-major access time\n", "start_time = time.time()\n", "row_sum = row_major_sum(matrix)\n", "print(\"Row-major sum time:\", time.time() - start_time)\n", "\n", "# Measure column-major access time\n", "start_time = time.time()\n", "column_sum = column_major_sum(matrix)\n", "print(\"Column-major sum time:\", time.time() - start_time)\n" ] }, { "cell_type": "markdown", "metadata": { "id": "xbkdDKQjpUh6" }, "source": [ "### Explanation:\n", "\n", "The example above compares row-major and column-major memory access patterns. Typically, row-major access is faster on most systems because it aligns better with how data is stored in memory.\n", "\n", "### Exercise:\n", "\n", "Modify the code to measure the cache hit rate (if possible using advanced profiling tools or libraries) for each access pattern. Observe how different matrix sizes affect cache utilization and performance.\n" ] }, { "cell_type": "markdown", "metadata": { "id": "weiiRf7ppWCp" }, "source": [ "## 10. Leveraging High-Performance Libraries\n", "\n", "Using specialized HPC libraries can significantly enhance the performance of your applications. This section explores how to use BLAS, LAPACK, and other optimized libraries in your code.\n", "\n", "### 10.1 Using BLAS and LAPACK for Matrix Operations\n", "\n", "BLAS (Basic Linear Algebra Subprograms) and LAPACK are standard libraries providing highly optimized implementations of basic linear algebra routines.\n" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 466, "status": "ok", "timestamp": 1724424101113, "user": { "displayName": "Oscar Diez", "userId": "09940749782853693007" }, "user_tz": -120 }, "id": "tek106G0pXGC", "outputId": "c05ef701-d960-47a5-da21-27c1cb53ea80" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Matrix A:\n", "[0.193716681052843, 0.011349001864187969, 0.18470245555909326]\n", "[0.7025423235872337, 0.3781896390384223, 0.9239934132301938]\n", "[0.5579105131482539, 0.8051458315702793, 0.18680557434967526]\n", "\n", "Matrix B:\n", "[0.11734904232615806, 0.9361814545953562, 0.24545474216288043]\n", "[0.16066259097376523, 0.4484719404489168, 0.9707977032438665]\n", "[0.7228591266796434, 0.6406680633585066, 0.5025751193521256]\n", "\n", "Result of matrix multiplication (A * B = C):\n", "[0.15806968276965128, 0.30477663763623086, 0.15139312159158053]\n", "[0.8111206678992223, 1.419287606298139, 1.0039640777984853]\n", "[0.3298611941667463, 1.003071154705034, 1.0124593390439955]\n", "\n", "Matrix inversion of A using LU decomposition:\n", "[1.0, -0.011349001864187969, -0.18470245555909326]\n", "[-3.626648566189252, 1.0, -0.25414251760517426]\n", "[-2.8800334081506604, -2.29195768554028, 1.0]\n" ] } ], "source": [ "import random\n", "\n", "# Function to create a 3x3 matrix\n", "def create_matrix_3x3():\n", " return [[random.random() for _ in range(3)] for _ in range(3)]\n", "\n", "# Function to multiply two 3x3 matrices (BLAS dgemm equivalent)\n", "def matrix_multiply(A, B):\n", " C = [[0.0 for _ in range(3)] for _ in range(3)]\n", " for i in range(3):\n", " for j in range(3):\n", " for k in range(3):\n", " C[i][j] += A[i][k] * B[k][j]\n", " return C\n", "\n", "# Function to perform LU decomposition (LAPACK getrf equivalent)\n", "def lu_decomposition(A):\n", " n = len(A)\n", " LU = [row[:] for row in A] # Create a copy of matrix A\n", " piv = list(range(n)) # Pivot indices\n", " for k in range(n):\n", " pivot_value = LU[k][k]\n", " if pivot_value == 0:\n", " raise ValueError(\"Matrix is singular\")\n", " for i in range(k + 1, n):\n", " LU[i][k] /= pivot_value\n", " for j in range(k + 1, n):\n", " LU[i][j] -= LU[i][k] * LU[k][j]\n", " return LU, piv\n", "\n", "# Function to invert a 3x3 matrix using LU decomposition (LAPACK getri equivalent)\n", "def invert_matrix(LU):\n", " n = len(LU)\n", " inv_matrix = [[0.0 for _ in range(n)] for _ in range(n)]\n", " for i in range(n):\n", " inv_matrix[i][i] = 1.0\n", " for i in range(n):\n", " for j in range(n):\n", " if i != j:\n", " inv_matrix[i][j] = -LU[i][j]\n", " return inv_matrix\n", "\n", "# Create two random 3x3 matrices\n", "A = create_matrix_3x3()\n", "B = create_matrix_3x3()\n", "\n", "# Perform matrix multiplication (BLAS dgemm equivalent)\n", "C = matrix_multiply(A, B)\n", "\n", "# Perform LU decomposition and invert the matrix (LAPACK getrf and getri equivalent)\n", "LU, piv = lu_decomposition(A)\n", "inv_matrix = invert_matrix(LU)\n", "\n", "# Display the results\n", "print(\"Matrix A:\")\n", "for row in A:\n", " print(row)\n", "\n", "print(\"\\nMatrix B:\")\n", "for row in B:\n", " print(row)\n", "\n", "print(\"\\nResult of matrix multiplication (A * B = C):\")\n", "for row in C:\n", " print(row)\n", "\n", "print(\"\\nMatrix inversion of A using LU decomposition:\")\n", "for row in inv_matrix:\n", " print(row)\n" ] }, { "cell_type": "markdown", "metadata": { "id": "30OdE7-6pZgR" }, "source": [ "### Explanation:\n", "\n", "This example demonstrates how to use the BLAS `dgemm` function for matrix multiplication and the LAPACK `dgetrf` function for matrix inversion. These libraries are optimized for performance on many HPC systems.\n", "\n", "### Exercise:\n", "\n", "Try using other functions from BLAS and LAPACK, such as `dsymv` for symmetric matrix-vector multiplication or `dgeev` for eigenvalue computation. Compare the performance of these library functions with your custom implementations.\n" ] }, { "cell_type": "markdown", "metadata": { "id": "_vHHfgbkpc0B" }, "source": [ "## 11. Advanced Performance Tuning with Parallel I/O\n", "\n", "Efficient I/O operations are critical for handling large datasets in HPC applications. This section covers advanced parallel I/O techniques using mpi4py.\n", "\n", "### 11.1 Implementing Parallel I/O\n", "\n", "We will extend our previous examples by implementing collective I/O operations, which can be more efficient for large-scale data processing.\n" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 452, "status": "ok", "timestamp": 1724424207375, "user": { "displayName": "Oscar Diez", "userId": "09940749782853693007" }, "user_tz": -120 }, "id": "vZUOI0Fupbrs", "outputId": "e1526f35-a6b7-46d0-9021-4529f429fdd4" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "C code written to mpi_collective_io.c\n" ] } ], "source": [ "# Write the C code to a file\n", "c_code = \"\"\"\n", "#include \n", "#include \n", "#include \n", "\n", "int main(int argc, char** argv) {\n", " MPI_Init(&argc, &argv); // Initialize the MPI environment\n", "\n", " MPI_Comm comm = MPI_COMM_WORLD;\n", " int rank, size;\n", " MPI_Comm_rank(comm, &rank); // Get the rank of the process\n", " MPI_Comm_size(comm, &size); // Get the total number of processes\n", "\n", " // Create a large array on each process, filled with the rank number\n", " int data_size = 1000000;\n", " int* data = (int*)malloc(data_size * sizeof(int));\n", " for (int i = 0; i < data_size; i++) {\n", " data[i] = rank;\n", " }\n", "\n", " // Write data collectively to a shared file\n", " MPI_File fh;\n", " MPI_File_open(comm, \"collective_output.dat\", MPI_MODE_CREATE | MPI_MODE_WRONLY, MPI_INFO_NULL, &fh);\n", " MPI_File_write_at_all(fh, rank * data_size * sizeof(int), data, data_size, MPI_INT, MPI_STATUS_IGNORE);\n", " MPI_File_close(&fh); // Close the file after writing\n", "\n", " // Allocate memory for reading the data back\n", " int* collected_data = (int*)malloc(data_size * sizeof(int));\n", "\n", " // Reading data collectively from the shared file\n", " MPI_File_open(comm, \"collective_output.dat\", MPI_MODE_RDONLY, MPI_INFO_NULL, &fh);\n", " MPI_File_read_at_all(fh, rank * data_size * sizeof(int), collected_data, data_size, MPI_INT, MPI_STATUS_IGNORE);\n", " MPI_File_close(&fh); // Close the file after reading\n", "\n", " // Print out a summary of the data to verify the read operation\n", " printf(\"Process %d: First element = %d, Last element = %d\\\\n\", rank, collected_data[0], collected_data[data_size - 1]);\n", "\n", " // Free dynamically allocated memory\n", " free(data);\n", " free(collected_data);\n", "\n", " MPI_Finalize(); // Finalize the MPI environment\n", " return 0;\n", "}\n", "\"\"\"\n", "\n", "# Save the C code to a file\n", "with open(\"mpi_collective_io.c\", \"w\") as file:\n", " file.write(c_code)\n", "\n", "print(\"C code written to mpi_collective_io.c\")\n" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "%%bash\n", "# Load MPI module (if required by your environment)\n", "module load openmpi/4.0.3\n", "\n", "# Compile the C code\n", "mpicc -o mpi_collective_io mpi_collective_io.c\n" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Process 2: First element = 2, Last element = 2\n", "Process 3: First element = 3, Last element = 3\n", "Process 0: First element = 0, Last element = 0\n", "Process 1: First element = 1, Last element = 1\n" ] } ], "source": [ "%%bash\n", "# Run the program with oversubscription if necessary\n", "mpirun --oversubscribe -np 4 ./mpi_collective_io\n" ] }, { "cell_type": "markdown", "metadata": { "id": "Gcsx5P71pflx" }, "source": [ "### Explanation:\n", "\n", "In this example, each MPI process writes and reads a portion of data from a shared file using collective I/O operations. This technique improves the efficiency of data handling in parallel applications.\n", "\n", "### Exercise:\n", "\n", "Modify the code to test the performance impact of different file access modes, such as `MPI.MODE_APPEND` or non-collective I/O. Analyze how these changes affect the scalability of I/O operations when running on multiple processes.\n" ] }, { "cell_type": "markdown", "metadata": { "id": "nXmUJRN9pima" }, "source": [ "## Comprehensive Performance Analysis and Tuning\n", "\n", "In this section, we perform a comprehensive performance analysis and tuning of a complex HPC (High-Performance Computing) application. The focus is on profiling, optimization, and parallel I/O techniques to improve the performance of scientific computations. We will use the example code provided to illustrate these concepts.\n", "\n", "### 12.1 Case Study: Performance Tuning of a Scientific Application\n", "\n", "We apply profiling, optimization, and parallel I/O techniques to a real-world scientific computation involving matrix operations and parallel I/O. The provided code examples demonstrate matrix multiplication and inversion using BLAS and LAPACK routines, as well as parallel I/O with MPI.\n", "\n", "#### Code Explanation\n", "\n", "##### `mpi_blas_lapack.c`\n", "\n", "This code performs matrix operations and parallel I/O using MPI.\n", "\n", "1. **Matrix Computation**: \n", " - The `optimized_computation` function performs matrix multiplication (`dgemm_`) and matrix inversion (`dgetrf_` and `dgetri_`) using BLAS and LAPACK routines.\n", " - It initializes matrices `A` and `B`, performs the operations, and computes the sum of elements in the inverted matrix.\n", "\n", "2. **Parallel I/O**:\n", " - The `main` function initializes MPI, performs matrix computations, and saves the result to a file using MPI I/O.\n", " - Each MPI process writes its result to a shared file, ensuring proper parallel file access.\n", "\n", "##### `blas_lapack_example.c`\n", "\n", "This code demonstrates basic usage of BLAS and LAPACK routines.\n", "\n", "1. **Matrix Multiplication**:\n", " - The `dgemm_` function multiplies two matrices `A` and `B` and stores the result in matrix `C`.\n", " - The result is printed to the console.\n", "\n", "2. **Solving Linear Systems**:\n", " - The `dgesv_` function solves a system of linear equations `A * X = B`.\n", " - The solution is printed to the console.\n", "\n", "#### Compilation and Execution\n", "\n", "The programs are compiled using the Intel Math Kernel Library (IMKL) and OpenMPI, and executed with MPI to leverage parallel processing capabilities.\n", "\n", "```bash\n", "# Load necessary modules (IMKL and OpenMPI)\n", "module load imkl/2020.1.217\n", "module load openmpi/4.0.3\n", "\n", "# Compile the C program\n", "mpicc -o blas_lapack_example blas_lapack_example.c -lmkl_rt\n" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "executionInfo": { "elapsed": 480, "status": "ok", "timestamp": 1724424550186, "user": { "displayName": "Oscar Diez", "userId": "09940749782853693007" }, "user_tz": -120 }, "id": "18VZBQ0IpkKL", "outputId": "5f17a46a-56fa-4949-ee7b-979da91b4892" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Overwriting blas_lapack_example.c\n" ] } ], "source": [ "%%writefile blas_lapack_example.c\n", "#include \n", "#include \n", "\n", "// Declare BLAS and LAPACK routines\n", "extern void dgemm_(char *transa, char *transb, int *m, int *n, int *k,\n", " double *alpha, double *A, int *lda, double *B, int *ldb,\n", " double *beta, double *C, int *ldc);\n", "\n", "extern void dgesv_(int *n, int *nrhs, double *A, int *lda, int *ipiv,\n", " double *B, int *ldb, int *info);\n", "\n", "void print_matrix(const char* name, double *matrix, int rows, int cols) {\n", " printf(\"%s:\\n\", name);\n", " for (int i = 0; i < rows; i++) {\n", " for (int j = 0; j < cols; j++) {\n", " printf(\"%f \", matrix[i * cols + j]);\n", " }\n", " printf(\"\\n\");\n", " }\n", "}\n", "\n", "int main() {\n", " // Example: Matrix multiplication using dgemm (BLAS)\n", " \n", " // Matrices A (3x3), B (3x3), and C (3x3) for A * B = C\n", " double A[9] = {1.0, 2.0, 3.0, \n", " 4.0, 5.0, 6.0, \n", " 7.0, 8.0, 9.0};\n", " double B[9] = {9.0, 8.0, 7.0, \n", " 6.0, 5.0, 4.0, \n", " 3.0, 2.0, 1.0};\n", " double C[9];\n", " \n", " int m = 3, n = 3, k = 3; // Dimensions of matrices\n", " double alpha = 1.0, beta = 0.0;\n", " \n", " // Matrix multiplication C = alpha * A * B + beta * C\n", " dgemm_(\"N\", \"N\", &m, &n, &k, &alpha, A, &m, B, &n, &beta, C, &m);\n", " \n", " // Print the result of A * B\n", " print_matrix(\"Matrix C (A * B)\", C, m, n);\n", "\n", " // Example: Solving a linear system using dgesv (LAPACK)\n", " \n", " // A (3x3) and B (3x1), solve A * X = B\n", " double A2[9] = {3.0, 2.0, -1.0,\n", " 2.0, -2.0, 4.0,\n", " -1.0, 0.5, -1.0};\n", " double B2[3] = {1.0, -2.0, 0.0}; // Right-hand side\n", " \n", " int ipiv[3]; // Pivot indices\n", " int info; // Return info\n", " int nrhs = 1; // Number of right-hand sides\n", " \n", " // Solve the system of equations A * X = B\n", " dgesv_(&m, &nrhs, A2, &m, ipiv, B2, &m, &info);\n", " \n", " if (info == 0) {\n", " // Print the solution\n", " print_matrix(\"Solution to A * X = B\", B2, m, nrhs);\n", " } else {\n", " printf(\"An error occurred: dgesv returned info = %d\\n\", info);\n", " }\n", "\n", " return 0;\n", "}\n" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "%%bash\n", "# Load necessary modules (IMKL and OpenMPI)\n", "module load imkl/2020.1.217\n", "module load openmpi/4.0.3\n", "\n", "# Compile the C program\n", "mpicc -o blas_lapack_example blas_lapack_example.c -lmkl_rt\n" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Matrix C (A * B):\n", "90.000000 114.000000 138.000000 \n", "54.000000 69.000000 84.000000 \n", "18.000000 24.000000 30.000000 \n", "Solution to A * X = B:\n", "-1.333333 \n", "-3.166667 \n", "-11.333333 \n", "Matrix C (A * B):\n", "90.000000 114.000000 138.000000 \n", "54.000000 69.000000 84.000000 \n", "18.000000 24.000000 30.000000 \n", "Solution to A * X = B:\n", "-1.333333 \n", "-3.166667 \n", "-11.333333 \n", "Matrix C (A * B):\n", "90.000000 114.000000 138.000000 \n", "54.000000 69.000000 84.000000 \n", "18.000000 24.000000 30.000000 \n", "Solution to A * X = B:\n", "-1.333333 \n", "-3.166667 \n", "-11.333333 \n", "Matrix C (A * B):\n", "90.000000 114.000000 138.000000 \n", "54.000000 69.000000 84.000000 \n", "18.000000 24.000000 30.000000 \n", "Solution to A * X = B:\n", "-1.333333 \n", "-3.166667 \n", "-11.333333 \n" ] } ], "source": [ "%%bash\n", "# Run the program with 4 processes (oversubscription enabled if necessary)\n", "mpirun --oversubscribe -np 4 ./blas_lapack_example\n" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "tags": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Writing mpi_blas_lapack_profiling.c\n" ] } ], "source": [ "%%writefile mpi_blas_lapack_profiling.c\n", "#include \n", "#include \n", "#include \n", "#include \n", "\n", "// External declarations for BLAS and LAPACK routines\n", "extern void dgemm_(char *transa, char *transb, int *m, int *n, int *k,\n", " double *alpha, double *A, int *lda, double *B, int *ldb,\n", " double *beta, double *C, int *ldc);\n", "\n", "extern void dgetrf_(int *m, int *n, double *A, int *lda, int *ipiv, int *info);\n", "extern void dgetri_(int *n, double *A, int *lda, int *ipiv, double *work, int *lwork, int *info);\n", "\n", "// Function to measure elapsed time\n", "double get_elapsed_time(struct timeval *start, struct timeval *end) {\n", " return ((end->tv_sec - start->tv_sec) + (end->tv_usec - start->tv_usec) / 1.0e6);\n", "}\n", "\n", "// Function to perform matrix multiplication and inversion\n", "double optimized_computation(double *A, double *B, int N) {\n", " struct timeval start, end;\n", " gettimeofday(&start, NULL);\n", "\n", " int m = N, n = N, k = N;\n", " double alpha = 1.0, beta = 0.0;\n", " double *C = (double *)malloc(N * N * sizeof(double));\n", "\n", " // Perform matrix multiplication C = A * B using BLAS\n", " dgemm_(\"N\", \"N\", &m, &n, &k, &alpha, A, &m, B, &k, &beta, C, &m);\n", "\n", " // Perform LU factorization using LAPACK\n", " int *ipiv = (int *)malloc(N * sizeof(int));\n", " int info;\n", " dgetrf_(&N, &N, C, &N, ipiv, &info);\n", "\n", " if (info != 0) {\n", " printf(\"LU factorization failed with info = %d\\n\", info);\n", " free(C);\n", " free(ipiv);\n", " return -1;\n", " }\n", "\n", " // Compute matrix inverse using LAPACK\n", " int lwork = N * N;\n", " double *work = (double *)malloc(lwork * sizeof(double));\n", " dgetri_(&N, C, &N, ipiv, work, &lwork, &info);\n", "\n", " if (info != 0) {\n", " printf(\"Matrix inversion failed with info = %d\\n\", info);\n", " free(C);\n", " free(ipiv);\n", " free(work);\n", " return -1;\n", " }\n", "\n", " // Sum the inverted matrix elements\n", " double sum = 0.0;\n", " for (int i = 0; i < N * N; i++) {\n", " sum += C[i];\n", " }\n", "\n", " gettimeofday(&end, NULL);\n", " printf(\"Computation time: %f seconds\\n\", get_elapsed_time(&start, &end));\n", "\n", " free(C);\n", " free(ipiv);\n", " free(work);\n", " return sum;\n", "}\n", "\n", "// Main function to perform computation and save the result using MPI I/O\n", "int main(int argc, char **argv) {\n", " MPI_Init(&argc, &argv);\n", "\n", " int rank, size;\n", " MPI_Comm_rank(MPI_COMM_WORLD, &rank);\n", " MPI_Comm_size(MPI_COMM_WORLD, &size);\n", "\n", " int N = 500; // Matrix size\n", " double *A = (double *)malloc(N * N * sizeof(double));\n", " double *B = (double *)malloc(N * N * sizeof(double));\n", "\n", " // Initialize matrices A and B with random data\n", " srand(rank + 1); // Seed based on rank\n", " for (int i = 0; i < N * N; i++) {\n", " A[i] = rand() / (double)RAND_MAX;\n", " B[i] = rand() / (double)RAND_MAX;\n", " }\n", "\n", " // Perform optimized computation\n", " double result = optimized_computation(A, B, N);\n", "\n", " // Parallel I/O to save results\n", " MPI_File file_handle;\n", " MPI_File_open(MPI_COMM_WORLD, \"final_result.dat\", MPI_MODE_CREATE | MPI_MODE_WRONLY, MPI_INFO_NULL, &file_handle);\n", " \n", " // Save result from each process into the file\n", " MPI_File_write_at(file_handle, rank * sizeof(double), &result, 1, MPI_DOUBLE, MPI_STATUS_IGNORE);\n", " MPI_File_close(&file_handle);\n", "\n", " printf(\"Process %d completed its task and saved the result: %f\\n\", rank, result);\n", "\n", " free(A);\n", " free(B);\n", " MPI_Finalize();\n", " return 0;\n", "}\n" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Computation time: 1.034726 seconds\n", "Computation time: 1.036457 seconds\n", "Computation time: 1.041499 seconds\n", "Computation time: 1.035438 seconds\n", "Process 1 completed its task and saved the result: -1.054715\n", "Process 3 completed its task and saved the result: -1.166478\n", "Process 2 completed its task and saved the result: -0.113707\n", "Process 0 completed its task and saved the result: 0.225218\n" ] } ], "source": [ "# Load necessary modules (IMKL and OpenMPI) and compile the C program\n", "!module load imkl/2020.1.217\n", "!module load openmpi/4.0.3\n", "!mpicc -o mpi_blas_lapack_profiling mpi_blas_lapack_profiling.c -lmkl_rt\n", "\n", "# Run the program with 4 processes (oversubscription enabled if necessary)\n", "!mpirun --oversubscribe -np 4 ./mpi_blas_lapack_profiling\n" ] }, { "cell_type": "markdown", "metadata": { "id": "YiTwilzjpm6q" }, "source": [ "### Optimizing Code\n", "After profiling, you can apply optimizations based on the identified bottlenecks. Here are some common optimization strategies:\n", "\n", "Matrix Operations Optimization\n", "Optimize BLAS/LAPACK Calls: Ensure that you are using the most efficient BLAS and LAPACK routines for your hardware. For example, use Intel MKL or OpenBLAS optimized libraries.\n", "\n", "Data Locality: Ensure that matrices and other large data structures are accessed in a cache-friendly manner to reduce cache misses.\n", "\n", "Parallel I/O Optimization\n", "Reduce I/O Contention: Use collective I/O operations where possible to minimize I/O contention between processes.\n", "\n", "Optimize File Access: Ensure that file access patterns are optimized for parallel writing, and use MPI I/O hints to improve performance.\n", "\n", "Code Example for Matrix Multiplication Optimization\n", "\n", "\n", "```c\n", "// Example of optimizing matrix multiplication by improving cache usage\n", "void optimized_dgemm(char transa, char transb, int m, int n, int k, double alpha,\n", " double *A, int lda, double *B, int ldb, double beta, double *C, int ldc) {\n", " // Use a block size for better cache utilization\n", " int block_size = 64; // Adjust this based on your cache size\n", " for (int i = 0; i < m; i += block_size) {\n", " for (int j = 0; j < n; j += block_size) {\n", " for (int k = 0; k < k; k += block_size) {\n", " // Perform block matrix multiplication\n", " // Ensure that you do not go out of bounds\n", " int m1 = min(block_size, m - i);\n", " int n1 = min(block_size, n - j);\n", " int k1 = min(block_size, k - k);\n", " dgemm_(&transa, &transb, &m1, &n1, &k1, &alpha, &A[i * lda + k], &lda,\n", " &B[k * ldb + j], &ldb, &beta, &C[i * ldc + j], &ldc);\n", " }\n", " }\n", " }\n", "}\n" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Overwriting mpi_blas_lapack_optimised.c\n" ] } ], "source": [ "%%writefile mpi_blas_lapack_optimised.c\n", "#include \n", "#include \n", "#include \n", "#include \n", "\n", "// External declarations for BLAS and LAPACK routines\n", "extern void dgemm_(char *transa, char *transb, int *m, int *n, int *k,\n", " double *alpha, double *A, int *lda, double *B, int *ldb,\n", " double *beta, double *C, int *ldc);\n", "\n", "extern void dgetrf_(int *m, int *n, double *A, int *lda, int *ipiv, int *info);\n", "extern void dgetri_(int *n, double *A, int *lda, int *ipiv, double *work, int *lwork, int *info);\n", "\n", "// Function to measure elapsed time\n", "double get_elapsed_time(struct timeval *start, struct timeval *end) {\n", " return ((end->tv_sec - start->tv_sec) + (end->tv_usec - start->tv_usec) / 1.0e6);\n", "}\n", "\n", "// Optimized matrix multiplication using blocking for better cache utilization\n", "void optimized_dgemm(char transa, char transb, int m, int n, int k, double alpha,\n", " double *A, int lda, double *B, int ldb, double beta, double *C, int ldc) {\n", " int block_size = 64; // Adjust this based on your cache size\n", " for (int i = 0; i < m; i += block_size) {\n", " for (int j = 0; j < n; j += block_size) {\n", " for (int l = 0; l < k; l += block_size) {\n", " // Determine the dimensions of the current block\n", " int m1 = (i + block_size > m) ? (m - i) : block_size;\n", " int n1 = (j + block_size > n) ? (n - j) : block_size;\n", " int l1 = (l + block_size > k) ? (k - l) : block_size;\n", "\n", " // Perform block matrix multiplication\n", " dgemm_(&transa, &transb, &m1, &n1, &l1, &alpha, &A[i + l * lda], &lda,\n", " &B[l + j * ldb], &ldb, &beta, &C[i + j * ldc], &ldc);\n", " }\n", " }\n", " }\n", "}\n", "\n", "\n", "// Function to perform matrix multiplication and inversion\n", "double optimized_computation(double *A, double *B, int N) {\n", " struct timeval start, end;\n", " gettimeofday(&start, NULL);\n", "\n", " int m = N, n = N, k = N;\n", " double alpha = 1.0, beta = 0.0;\n", " double *C = (double *)malloc(N * N * sizeof(double));\n", " if (C == NULL) {\n", " perror(\"Failed to allocate memory for matrix C\");\n", " return -1;\n", " }\n", "\n", " // Perform matrix multiplication C = A * B using the optimized BLAS function\n", " optimized_dgemm('N', 'N', m, n, k, alpha, A, m, B, k, beta, C, m);\n", "\n", " // Perform LU factorization using LAPACK\n", " int *ipiv = (int *)malloc(N * sizeof(int));\n", " if (ipiv == NULL) {\n", " perror(\"Failed to allocate memory for IPIV\");\n", " free(C);\n", " return -1;\n", " }\n", " int info;\n", " dgetrf_(&N, &N, C, &N, ipiv, &info);\n", "\n", " if (info != 0) {\n", " printf(\"LU factorization failed with info = %d\\n\", info);\n", " free(C);\n", " free(ipiv);\n", " return -1;\n", " }\n", "\n", " // Compute matrix inverse using LAPACK\n", " int lwork = N * N;\n", " double *work = (double *)malloc(lwork * sizeof(double));\n", " if (work == NULL) {\n", " perror(\"Failed to allocate memory for work\");\n", " free(C);\n", " free(ipiv);\n", " return -1;\n", " }\n", " dgetri_(&N, C, &N, ipiv, work, &lwork, &info);\n", "\n", " if (info != 0) {\n", " printf(\"Matrix inversion failed with info = %d\\n\", info);\n", " free(C);\n", " free(ipiv);\n", " free(work);\n", " return -1;\n", " }\n", "\n", " // Sum the inverted matrix elements\n", " double sum = 0.0;\n", " for (int i = 0; i < N * N; i++) {\n", " sum += C[i];\n", " }\n", "\n", " gettimeofday(&end, NULL);\n", " printf(\"Computation time: %f seconds\\n\", get_elapsed_time(&start, &end));\n", "\n", " free(C);\n", " free(ipiv);\n", " free(work);\n", " return sum;\n", "}\n", "\n", "// Main function to perform computation and save the result using MPI I/O\n", "int main(int argc, char **argv) {\n", " MPI_Init(&argc, &argv);\n", "\n", " int rank, size;\n", " MPI_Comm_rank(MPI_COMM_WORLD, &rank);\n", " MPI_Comm_size(MPI_COMM_WORLD, &size);\n", "\n", " int N = 500; // Matrix size\n", " double *A = (double *)malloc(N * N * sizeof(double));\n", " double *B = (double *)malloc(N * N * sizeof(double));\n", " if (A == NULL || B == NULL) {\n", " perror(\"Failed to allocate memory for matrices A and B\");\n", " MPI_Finalize();\n", " return -1;\n", " }\n", "\n", " // Initialize matrices A and B with random data\n", " srand(rank + 1); // Seed based on rank\n", " for (int i = 0; i < N * N; i++) {\n", " A[i] = rand() / (double)RAND_MAX;\n", " B[i] = rand() / (double)RAND_MAX;\n", " }\n", "\n", " // Perform optimized computation\n", " double result = optimized_computation(A, B, N);\n", "\n", " // Parallel I/O to save results\n", " MPI_File file_handle;\n", " MPI_File_open(MPI_COMM_WORLD, \"final_result.dat\", MPI_MODE_CREATE | MPI_MODE_WRONLY, MPI_INFO_NULL, &file_handle);\n", " \n", " // Save result from each process into the file\n", " MPI_File_write_at(file_handle, rank * sizeof(double), &result, 1, MPI_DOUBLE, MPI_STATUS_IGNORE);\n", " MPI_File_close(&file_handle);\n", "\n", " printf(\"Process %d completed its task and saved the result: %f\\n\", rank, result);\n", "\n", " free(A);\n", " free(B);\n", " MPI_Finalize();\n", " return 0;\n", "}\n" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Computation time: 0.107522 seconds\n", "Computation time: 0.115276 seconds\n", "Computation time: 0.108997 seconds\n", "Computation time: 0.108116 seconds\n", "Process 1 completed its task and saved the result: -163695617273879.500000\n", "Process 3 completed its task and saved the result: 94344612229255.546875\n", "Process 0 completed its task and saved the result: -45431444068900.968750\n", "Process 2 completed its task and saved the result: -9867505529191.496094\n" ] } ], "source": [ "# Load necessary modules (IMKL and OpenMPI) and compile the C program\n", "!module load imkl/2020.1.217\n", "!module load openmpi/4.0.3\n", "!mpicc -o mpi_blas_lapack_optimised mpi_blas_lapack_optimised.c -lmkl_rt\n", "\n", "# Run the program with 4 processes (oversubscription enabled if necessary)\n", "!mpirun --oversubscribe -np 4 ./mpi_blas_lapack_optimised\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Optimization of MPI, BLAS, and LAPACK Computation\n", "\n", "## Overview\n", "\n", "In this notebook, we implemented and optimized a matrix multiplication and inversion routine using MPI for parallel computation and BLAS/LAPACK libraries for efficient mathematical operations. The goal was to reduce computation time and improve performance by applying specific optimizations.\n", "\n", "## Optimization Details\n", "\n", "### 1. Optimized Matrix Multiplication Using Blocking\n", "\n", "**Original Approach:** \n", "The standard `dgemm` function from BLAS was used for matrix multiplication. While effective, this approach does not always fully utilize the CPU cache, especially for large matrices.\n", "\n", "**Optimization Applied:**\n", "We implemented a custom version of matrix multiplication with blocking, which improves cache performance. Blocking divides the matrices into smaller sub-matrices or \"blocks\" that fit into the CPU cache. This reduces the number of cache misses and improves performance.\n", "\n", "**Why This Works:**\n", "\n", "Cache Efficiency: Blocking keeps data in cache, minimizing the need to repeatedly load data from slower memory.\n", "Reduced Cache Misses: By operating on smaller blocks, the number of cache misses is reduced, leading to faster computations.\n", "\n", "### 2. Matrix Inversion and LU Factorization\n", "Approach Used:\n", "\n", "LU Factorization: We used LAPACK's dgetrf_ for LU decomposition.\n", "Matrix Inversion: We used dgetri_ to compute the inverse of the matrix.\n", "Optimization Considerations:\n", "\n", "Proper Memory Management: Allocated and freed memory appropriately to avoid memory leaks and ensure efficient use of resources.\n", "Error Handling: Included checks for failure in LU factorization and matrix inversion to handle potential issues gracefully.\n", "\n", "#### optional exercise:\n", "\n", "Expand the case study by adding more complex operations, such as eigenvalue computation or solving a system of linear equations. Profile and optimize these additional steps, and analyze how the performance scales with the problem size and number of processes.\n", "\n", "### End of the practice." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "colab": { "authorship_tag": "ABX9TyOFL+sc7nZMhbhn/M3UvvSW", "provenance": [] }, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.6" } }, "nbformat": 4, "nbformat_minor": 4 }