{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "\n", "# IDScalarWaveNRPy: An Einstein Toolkit Initial Data Thorn for the Scalar Wave Equation\n", "\n", "## Author: Terrence Pierre Jacques & Zach Etienne\n", "### Formatting improvements courtesy Brandon Clark\n", "\n", "## The notebook outlines the construction of an Einstein Toolkit (ETK) thorn for setting up initial data for the scalar wave initial value problem. The ETK module employs NRPy+ to transform SymPy expressions for initial data into a C-code kernel, which is then integrated with the ETK infrastructure. The tutorial demonstrates the conversion of SymPy expressions into C-code, interfacing with the ETK, constructing C-code calling functions, defining module interactions with ETK, and the addition of C-code to the ETK compilation list.\n", "\n", "[comment]: <> (Notebook Status and Validation Notes: TODO)\n", "\n", "### NRPy+ Source Code for this module: [ScalarWave/InitialData.py](../edit/ScalarWave/InitialData.py) [\\[**tutorial**\\]](Tutorial-ScalarWave.ipynb) Contructs the SymPy expressions for spherical gaussian and plane-wave initial data\n", "\n", "## Introduction:\n", "In this part of the tutorial, we will construct an Einstein Toolkit (ETK) thorn (module) that will set up *initial data* for the scalar wave initial value problem. In a [previous tutorial notebook](Tutorial-ScalarWave.ipynb), we used NRPy+ to construct the SymPy expressions for either spherical Gaussian or plane-wave initial data. This thorn is largely based on and should function similarly to the $\\text{IDScalarWaveC}$ thorn included in the Einstein Toolkit (ETK) $\\text{CactusWave}$ arrangement.\n", "\n", "We will construct this thorn in two steps.\n", "\n", "1. Call on NRPy+ to convert the SymPy expressions for the initial data into one C-code kernel.\n", "1. Write the C code and linkages to the Einstein Toolkit infrastructure (i.e., the .ccl files) to complete this Einstein Toolkit module." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "# Table of Contents\n", "$$\\label{toc}$$ \n", "\n", "This notebook is organized as follows\n", "\n", "1. [Step 1](#initializenrpy): Call on NRPy+ to convert the SymPy expression for the scalar wave initial data into a C-code kernel\n", "1. [Step 2](#einstein): Interfacing with the Einstein Toolkit\n", " 1. [Step 2.a](#einstein_c): Constructing the Einstein Toolkit C-code calling functions that include the C code kernels\n", " 1. [Step 2.b](#einstein_ccl): CCL files - Define how this module interacts and interfaces with the larger Einstein Toolkit infrastructure\n", " 1. [Step 2.c](#einstein_list): Add the C code to the Einstein Toolkit compilation list\n", "1. [Step 3](#latex_pdf_output): Output this notebook to $\\LaTeX$-formatted PDF file" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "# Step 1: Initialize needed Python/NRPy+ modules \\[Back to [top](#toc)\\]\n", "\n", "$$\\label{initializenrpy}$$" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "execution": { "iopub.execute_input": "2021-03-07T17:07:06.416555Z", "iopub.status.busy": "2021-03-07T17:07:06.415823Z", "iopub.status.idle": "2021-03-07T17:07:06.736279Z", "shell.execute_reply": "2021-03-07T17:07:06.736809Z" } }, "outputs": [], "source": [ "# Step 1: Import needed core NRPy+ modules\n", "from outputC import lhrh # NRPy+: Core C code output module\n", "import finite_difference as fin # NRPy+: Finite difference C code generation module\n", "import NRPy_param_funcs as par # NRPy+: Parameter interface\n", "import grid as gri # NRPy+: Functions having to do with numerical grids\n", "import loop as lp # NRPy+: Generate C code loops\n", "import reference_metric as rfm # NRPy+: Reference metric support\n", "import cmdline_helper as cmd # NRPy+: Multi-platform Python command-line interface\n", "import os, sys # Standard Python modules for multiplatform OS-level functions\n", "\n", "# Step 1a: Create directories for the thorn if they don't exist.\n", "# Create directory for WaveToyNRPy thorn & subdirectories in case they don't exist.\n", "outrootdir = \"IDScalarWaveNRPy/\"\n", "cmd.mkdir(os.path.join(outrootdir))\n", "outdir = os.path.join(outrootdir,\"src\") # Main C code output directory\n", "cmd.mkdir(outdir)\n", "\n", "# Step 1b: This is an Einstein Toolkit (ETK) thorn. Here we\n", "# tell NRPy+ that gridfunction memory access will\n", "# therefore be in the \"ETK\" style.\n", "par.set_parval_from_str(\"grid::GridFuncMemAccess\",\"ETK\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "# Step 2: Interfacing with the Einstein Toolkit \\[Back to [top](#toc)\\]\n", "$$\\label{einstein}$$\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "## Step 2.a: Constructing the Einstein Toolkit C-code calling functions that include the C code kernels \\[Back to [top](#toc)\\]\n", "$$\\label{einstein_c}$$\n", "\n", "Using SymPy, we construct the exact expressions for all scalar wave initial data currently supported in NRPy, documented in [Tutorial-ScalarWave.ipynb](Tutorial-ScalarWave.ipynb). We write the generated C codes into different C files, corresponding to the type of initial data they may want to choose at run time. Note that the code below can be easily extensible to include other types of initial data." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "execution": { "iopub.execute_input": "2021-03-07T17:07:06.747213Z", "iopub.status.busy": "2021-03-07T17:07:06.746545Z", "iopub.status.idle": "2021-03-07T17:07:06.994996Z", "shell.execute_reply": "2021-03-07T17:07:06.995530Z" } }, "outputs": [], "source": [ "# Step 1c: Call the InitialData() function from within the\n", "# ScalarWave/InitialData.py module.\n", "import ScalarWave.InitialData as swid\n", "\n", "_time = par.Cparameters(\"REAL\", __name__, \"time\", 0.0)\n", "\n", "# Step 1e: Call the InitialData() function to set up initial data.\n", "# Options include:\n", "# \"PlaneWave\": monochromatic (single frequency/wavelength) plane wave\n", "# \"SphericalGaussian\": spherically symmetric Gaussian, with default stdev=3\n", "ID_options = [\"PlaneWave\", \"SphericalGaussian\"]\n", "for ID in ID_options:\n", " gri.glb_gridfcs_list = []\n", "\n", "# Within the ETK, the 3D gridfunctions x, y, and z store the\n", "# Cartesian grid coordinates. Setting the gri.xx[] arrays\n", "# to point to these gridfunctions forces NRPy+ to treat\n", "# the Cartesian coordinate gridfunctions properly --\n", "# reading them from memory as needed.\n", " x,y,z = gri.register_gridfunctions(\"AUX\",[\"x\",\"y\",\"z\"])\n", " rfm.xx[0] = x\n", " rfm.xx[1] = y\n", " rfm.xx[2] = z\n", " swid.InitialData(WaveType=ID,\n", " default_sigma=0.25,\n", " default_k0=1.0,\n", " default_k1=0.,\n", " default_k2=0.)\n", "\n", " # Step 1f: Register uu and vv gridfunctions so they can be written to by NRPy.\n", " uu,vv = gri.register_gridfunctions(\"EVOL\",[\"uu\",\"vv\"])\n", "\n", " # Step 1g: Set the uu and vv gridfunctions to the uu_ID & vv_ID variables\n", " # defined by InitialData_PlaneWave().\n", " uu = swid.uu_ID\n", " vv = swid.vv_ID\n", "\n", " # Step 1h: Create the C code output kernel.\n", " ScalarWave_ID_SymbExpressions = [\\\n", " lhrh(lhs=gri.gfaccess(\"out_gfs\",\"uu\"),rhs=uu),\\\n", " lhrh(lhs=gri.gfaccess(\"out_gfs\",\"vv\"),rhs=vv),]\n", "\n", " ScalarWave_ID_CcodeKernel = fin.FD_outputC(\"returnstring\",ScalarWave_ID_SymbExpressions)\n", "\n", " ScalarWave_ID_looped = lp.loop([\"i2\",\"i1\",\"i0\"],[\"0\",\"0\",\"0\"],[\"cctk_lsh[2]\",\"cctk_lsh[1]\",\"cctk_lsh[0]\"],\\\n", " [\"1\",\"1\",\"1\"],[\"#pragma omp parallel for\",\"\",\"\"],\"\",\\\n", " ScalarWave_ID_CcodeKernel.replace(\"time\",\"cctk_time\"))\n", "\n", " # Write the C code kernel to file.\n", " with open(os.path.join(outdir,\"ScalarWave_\"+ID+\"ID.h\"), \"w\") as file:\n", " file.write(str(ScalarWave_ID_looped))\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "## Step 2. b: CCL files - Define how this module interacts and interfaces with the larger Einstein Toolkit infrastructure \\[Back to [top](#toc)\\]\n", "$$\\label{einstein_ccl}$$\n", "\n", "Writing a module (\"thorn\") within the Einstein Toolkit requires that three \"ccl\" files be constructed, all in the root directory of the thorn.\n", "\n", "1. `interface.ccl`: defines the gridfunction groups needed, and provides keywords denoting what this thorn provides and what it should inherit from other thorns. Specifically, this file governs the interaction between this thorn and others; more information can be found in the [official Einstein Toolkit documentation](https://einsteintoolkit.org/usersguide/UsersGuide.html#x1-179000D2.2). \n", "With \"implements\", we give our thorn its unique name. By \"inheriting\" other thorns, we tell the Toolkit that we will rely on variables that exist and are declared \"public\" within those functions." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "execution": { "iopub.execute_input": "2021-03-07T17:07:07.002041Z", "iopub.status.busy": "2021-03-07T17:07:07.001387Z", "iopub.status.idle": "2021-03-07T17:07:07.004037Z", "shell.execute_reply": "2021-03-07T17:07:07.003516Z" } }, "outputs": [], "source": [ "evol_gfs_list = []\n", "for i in range(len(gri.glb_gridfcs_list)):\n", " if gri.glb_gridfcs_list[i].gftype == \"EVOL\":\n", " evol_gfs_list.append( gri.glb_gridfcs_list[i].name+\"GF\")\n", "\n", "# NRPy+'s finite-difference code generator assumes gridfunctions\n", "# are alphabetized; not sorting may result in unnecessary\n", "# cache misses.\n", "evol_gfs_list.sort()\n", "\n", "with open(os.path.join(outrootdir,\"interface.ccl\"), \"w\") as file:\n", " file.write(\"\"\"\n", "# With \"implements\", we give our thorn its unique name.\n", "implements: IDScalarWaveNRPy\n", "\n", "# By \"inheriting\" other thorns, we tell the Toolkit that we\n", "# will rely on variables/function that exist within those\n", "# functions.\n", "inherits: WaveToyNRPy grid\n", "\"\"\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "2. `param.ccl`: specifies free parameters within the thorn, enabling them to be set at runtime. It is required to provide allowed ranges and default values for each parameter. More information on this file's syntax can be found in the [official Einstein Toolkit documentation](https://einsteintoolkit.org/usersguide/UsersGuide.html#x1-184000D2.3)." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "execution": { "iopub.execute_input": "2021-03-07T17:07:07.017819Z", "iopub.status.busy": "2021-03-07T17:07:07.017139Z", "iopub.status.idle": "2021-03-07T17:07:07.020135Z", "shell.execute_reply": "2021-03-07T17:07:07.019542Z" } }, "outputs": [], "source": [ "def keep_param__return_type(paramtuple):\n", " keep_param = True # We'll not set some parameters in param.ccl;\n", " # e.g., those that should be #define'd like M_PI.\n", " typestring = \"\"\n", " # Separate thorns within the ETK take care of grid/coordinate parameters;\n", " # thus we ignore NRPy+ grid/coordinate parameters:\n", " if paramtuple.module == \"grid\" or paramtuple.module == \"reference_metric\" or paramtuple.parname == \"wavespeed\":\n", " keep_param = False\n", "\n", " partype = paramtuple.type\n", " if partype == \"bool\":\n", " typestring += \"BOOLEAN \"\n", " elif partype == \"REAL\":\n", " if paramtuple.defaultval != 1e300: # 1e300 is a magic value indicating that the C parameter should be mutable\n", " typestring += \"CCTK_REAL \"\n", " else:\n", " keep_param = False\n", " elif partype == \"int\":\n", " typestring += \"CCTK_INT \"\n", " elif partype == \"#define\":\n", " keep_param = False\n", " elif partype == \"char\":\n", " # FIXME: char/string parameter types should in principle be supported\n", " print(\"Error: parameter \"+paramtuple.module+\"::\"+paramtuple.parname+\n", " \" has unsupported type: \\\"\"+ paramtuple.type + \"\\\"\")\n", " sys.exit(1)\n", " else:\n", " print(\"Error: parameter \"+paramtuple.module+\"::\"+paramtuple.parname+\n", " \" has unsupported type: \\\"\"+ paramtuple.type + \"\\\"\")\n", " sys.exit(1)\n", " return keep_param, typestring\n", "\n", "paramccl_str=\"\"\"\n", "# This param.ccl file was automatically generated by NRPy+.\n", "# You are advised against modifying it directly; instead\n", "# modify the Python code that generates it.\n", "\n", "shares: grid\n", "\n", "USES KEYWORD type\n", "\n", "shares: WaveToyNRPy\n", "\n", "USES REAL wavespeed\n", "\n", "restricted:\n", "CCTK_KEYWORD initial_data \"Type of initial data\"\n", "{\"\"\"\n", "\n", "for ID in ID_options:\n", " paramccl_str +='''\n", " \"'''+ID+'''\" :: \"'''+ID+'\"'\n", "\n", "paramccl_str +='''\n", "} \"'''+ID+'''\"\n", "\n", "'''\n", "paramccl_str +=\"\"\"\n", "restricted:\n", "\n", "\"\"\"\n", "\n", "for i in range(len(par.glb_Cparams_list)):\n", " # keep_param is a boolean indicating whether we should accept or reject\n", " # the parameter. singleparstring will contain the string indicating\n", " # the variable type.\n", " keep_param, singleparstring = keep_param__return_type(par.glb_Cparams_list[i])\n", "\n", " if keep_param:\n", " parname = par.glb_Cparams_list[i].parname\n", " partype = par.glb_Cparams_list[i].type\n", " singleparstring += parname + \" \\\"\"+ parname +\" (see NRPy+ for parameter definition)\\\"\\n\"\n", " singleparstring += \"{\\n\"\n", " if partype != \"bool\":\n", " singleparstring += \" *:* :: \\\"All values accepted. NRPy+ does not restrict the allowed ranges of parameters yet.\\\"\\n\"\n", " singleparstring += \"} \"+str(par.glb_Cparams_list[i].defaultval)+\"\\n\\n\"\n", "\n", " paramccl_str += singleparstring\n", "with open(os.path.join(outrootdir,\"param.ccl\"), \"w\") as file:\n", " file.write(paramccl_str)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "3. `schedule.ccl`: allocates storage for gridfunctions, defines how the thorn's functions should be scheduled in a broader simulation, and specifies the regions of memory written to or read from gridfunctions. More information on this file's syntax can be found in the [official Einstein Toolkit documentation](https://einsteintoolkit.org/usersguide/UsersGuide.html#x1-187000D2.4). \n", "\n", "We specify here the standardized ETK \"scheduling bins\" in which we want each of our thorn's functions to run." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "execution": { "iopub.execute_input": "2021-03-07T17:07:07.024841Z", "iopub.status.busy": "2021-03-07T17:07:07.024200Z", "iopub.status.idle": "2021-03-07T17:07:07.026740Z", "shell.execute_reply": "2021-03-07T17:07:07.027215Z" } }, "outputs": [], "source": [ "with open(os.path.join(outrootdir,\"schedule.ccl\"), \"w\") as file:\n", " file.write(\"\"\"\n", "# This schedule.ccl file was automatically generated by NRPy+.\n", "# You are advised against modifying it directly; instead\n", "# modify the Python code that generates it.\n", "\n", "if (CCTK_EQUALS (initial_data, \"PlaneWave\"))\n", "{\n", " schedule IDScalarWaveNRPy_param_check at CCTK_PARAMCHECK\n", " {\n", " LANG: C\n", " OPTIONS: global\n", " } \"Check sanity of parameters\"\n", "}\n", "\n", "schedule IDScalarWaveNRPy_InitialData at CCTK_INITIAL as WaveToy_InitialData\n", "{\n", " STORAGE: WaveToyNRPy::scalar_fields[3]\n", " LANG: C\n", "} \"Initial data for 3D wave equation\"\n", "\"\"\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "## Step 2.c: Add the C code to the Einstein Toolkit compilation list \\[Back to [top](#toc)\\]\n", "$$\\label{einstein_list}$$\n", "\n", "We will also need `make.code.defn`, which indicates the list of files that need to be compiled. This thorn only has the one C file to compile." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "execution": { "iopub.execute_input": "2021-03-07T17:07:07.031873Z", "iopub.status.busy": "2021-03-07T17:07:07.031278Z", "iopub.status.idle": "2021-03-07T17:07:07.033553Z", "shell.execute_reply": "2021-03-07T17:07:07.034016Z" } }, "outputs": [], "source": [ "make_code_defn_list = []\n", "def append_to_make_code_defn_list(filename):\n", " if filename not in make_code_defn_list:\n", " make_code_defn_list.append(filename)\n", " return os.path.join(outdir,filename)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "execution": { "iopub.execute_input": "2021-03-07T17:07:07.038484Z", "iopub.status.busy": "2021-03-07T17:07:07.037872Z", "iopub.status.idle": "2021-03-07T17:07:07.040186Z", "shell.execute_reply": "2021-03-07T17:07:07.040656Z" } }, "outputs": [], "source": [ "with open(append_to_make_code_defn_list(\"InitialData.c\"),\"w\") as file:\n", " file.write(\"\"\"\n", "\n", "#include \n", "#include \n", "#include \n", "\n", "#include \"cctk.h\"\n", "#include \"cctk_Parameters.h\"\n", "#include \"cctk_Arguments.h\"\n", "\n", "void IDScalarWaveNRPy_param_check(CCTK_ARGUMENTS) {\n", " DECLARE_CCTK_ARGUMENTS;\n", " DECLARE_CCTK_PARAMETERS;\n", "\n", " if (kk0 == 0 && kk1 == 0 && kk2 == 0) {\n", " CCTK_WARN(0,\"kk0==kk1==kk2==0: Zero wave vector cannot be normalized. Set one of the kk's to be != 0.\");\n", " }\n", "}\n", "\n", "void IDScalarWaveNRPy_InitialData(CCTK_ARGUMENTS)\n", "{\n", " DECLARE_CCTK_ARGUMENTS\n", " DECLARE_CCTK_PARAMETERS\n", "\n", " const CCTK_REAL *xGF = x;\n", " const CCTK_REAL *yGF = y;\n", " const CCTK_REAL *zGF = z;\n", "\n", " if (CCTK_EQUALS (initial_data, \"PlaneWave\")) {\n", " #include \"ScalarWave_PlaneWaveID.h\"\n", " } else if (CCTK_EQUALS (initial_data, \"SphericalGaussian\")) {\n", " #include \"ScalarWave_SphericalGaussianID.h\"\n", " }\n", "}\n", "\"\"\")" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "execution": { "iopub.execute_input": "2021-03-07T17:07:07.047207Z", "iopub.status.busy": "2021-03-07T17:07:07.046510Z", "iopub.status.idle": "2021-03-07T17:07:07.048669Z", "shell.execute_reply": "2021-03-07T17:07:07.049180Z" } }, "outputs": [], "source": [ "with open(os.path.join(outdir,\"make.code.defn\"), \"w\") as file:\n", " file.write(\"\"\"\n", "# Main make.code.defn file for thorn WaveToyNRPy\n", "\n", "# Source files in this directory\n", "SRCS =\"\"\")\n", " filestring = \"\"\n", " for i in range(len(make_code_defn_list)):\n", " filestring += \" \"+make_code_defn_list[i]\n", " if i != len(make_code_defn_list)-1:\n", " filestring += \" \\\\\\n\"\n", " else:\n", " filestring += \"\\n\"\n", " file.write(filestring)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "# Step 3: Output this notebook to $\\LaTeX$-formatted PDF file \\[Back to [top](#toc)\\]\n", "$$\\label{latex_pdf_output}$$\n", "\n", "The following code cell converts this Jupyter notebook into a proper, clickable $\\LaTeX$-formatted PDF file. After the cell is successfully run, the generated PDF may be found in the root NRPy+ tutorial directory, with filename\n", "[Tutorial-ETK_thorn-IDScalarWaveNRPy.pdf](Tutorial-ETK_thorn-IDScalarWaveNRPy.pdf). (Note that clicking on this link may not work; you may need to open the PDF file through another means.)" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "execution": { "iopub.execute_input": "2021-03-07T17:07:07.056634Z", "iopub.status.busy": "2021-03-07T17:07:07.055732Z", "iopub.status.idle": "2021-03-07T17:07:08.539512Z", "shell.execute_reply": "2021-03-07T17:07:08.540238Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[WARNING] Duplicate link reference '[comment]' at line 17 column 1\n", "Created Tutorial-ETK_thorn-IDScalarWaveNRPy.tex, and compiled LaTeX file to\n", " PDF file Tutorial-ETK_thorn-IDScalarWaveNRPy.pdf\n" ] } ], "source": [ "import cmdline_helper as cmd # NRPy+: Multi-platform Python command-line interface\n", "cmd.output_Jupyter_notebook_to_LaTeXed_PDF(\"Tutorial-ETK_thorn-IDScalarWaveNRPy\")" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.1" } }, "nbformat": 4, "nbformat_minor": 4 }