{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "4fb34a18",
   "metadata": {},
   "source": [
    "# Darcy in 3D: native workspaces and CPU/GPU scaling\n",
    "\n",
    "This notebook uses the public local/global equation API for anisotropic Darcy in the unit cube. Its default CPU control uses 4 × 4 × 4 macrocells, four Q1 subdivisions per coordinate inside each macrocell and unsplit tensor-Q1 normal traces. CPU/GPU campaigns and all available historical panels remain accessible separately.\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "K(\\boldsymbol x)&=m(\\boldsymbol x)D,\n",
    "& m(\\boldsymbol x)&=\\exp\\left(\\prod_{j=1}^{3}\\sin(20\\pi x_j)\\right),\\\\\n",
    "p_{\\mathrm{exact}}(\\boldsymbol x)&=\\prod_{j=1}^{3}\\sin(\\pi x_j),\n",
    "& f&=-\\nabla\\cdot(K\\nabla p_{\\mathrm{exact}}).\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "The tensor $D$ is symmetric positive definite; zero exterior pressure fixes the physical solution without a mean gauge. Local native workspaces reuse geometry, spaces, compiled forms and geometry-only trace/mean blocks. Every cell's material operator and source, and every solver factorization, remain fresh.\n",
    "\n",
    "$$\n",
    "\\begin{aligned}\n",
    "a_K(p,v)&=\\int_K K\\nabla p\\cdot\\nabla v, &L_K(v)&=\\int_Kfv,\\\\\n",
    "b_K(\\lambda,v)&=\\int_{\\partial K}\\lambda v,\n",
    "&c_K(p,\\mu)&=-\\int_{\\partial K}p\\mu.\n",
    "\\end{aligned}\n",
    "$$\n",
    "\n",
    "The custom interface supplies the canonical-to-outward-normal transport. The global multiplier system also retains one local pressure constant. Reported Darcy flux is the broken physical field $-K\\nabla p_h$; no fine-cell H(div) property is assumed.\n",
    "\n",
    "The PyMHM distribution contains only the library. The first cell explicitly\n",
    "downloads a checksum-verified companion archive and prepares the declared data\n",
    "using its local Python helpers. You can inspect these support files in `ROOT`.\n",
    "Complete studies and historical replay retain their existing opt-in flags.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7b8bea1b",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path\n",
    "import os\n",
    "import sys\n",
    "from pymhm.io.workspace import workspace_from_archive\n",
    "\n",
    "# Download verified support files; this operation does not execute them.\n",
    "COMPANION_URL = \"https://ipes-lncc.github.io/pymhm/downloads/89f5d1044875a054f6e835d7c48de1a0fd94683d71d0cbcbec493008963dd924/darcy_3d_parallel_scalability-companion.zip\"\n",
    "COMPANION_SHA256 = \"89f5d1044875a054f6e835d7c48de1a0fd94683d71d0cbcbec493008963dd924\"\n",
    "WORKSPACE = Path(\n",
    "    os.environ.get(\"PYMHM_WORKSPACE\", Path.cwd() / \".pymhm-companions\" / COMPANION_SHA256)\n",
    ")\n",
    "ROOT = workspace_from_archive(COMPANION_URL, sha256=COMPANION_SHA256, directory=WORKSPACE)\n",
    "os.environ[\"PYMHM_WORKSPACE\"] = str(ROOT)\n",
    "if str(ROOT) not in sys.path:\n",
    "    sys.path.insert(0, str(ROOT))\n",
    "\n",
    "# Explicitly prepare declared data with the downloaded Python helpers.\n",
    "from scripts.notebook_reproduction import notebook_workspace\n",
    "\n",
    "ROOT = notebook_workspace(\"introduction/darcy_3d_parallel_scalability.ipynb\", directory=ROOT)\n",
    "root = ROOT\n",
    "print(\"Workspace:\", ROOT)\n",
    "\n",
    "\n",
    "import inspect\n",
    "import numpy as np\n",
    "from IPython.display import Code, display\n",
    "from pymhm import MeshHierarchy, bind_interface, bind_problem\n",
    "from pymhm.core.equations import Equation\n",
    "from pymhm.core.multiscale import assemble\n",
    "from pymhm.execution.cpu import ExecutionConfig\n",
    "from examples.introduction import scaling_3d as study\n",
    "\n",
    "DATA = study.DarcyData(period=0.1)\n",
    "FINE_N, MACRO_COUNT = 16, 4\n",
    "TRACE_DEGREE, ASSEMBLY_QUADRATURE_DEGREE = 1, 8\n",
    "print({\"anisotropy\": DATA.anisotropy, \"source_control\": study.verify_source(DATA)})\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14eaa382",
   "metadata": {},
   "source": [
    "These are the actual executed physical-data and local-equation declarations. Native form compilation, workspace ownership and the importable spawn workers live in [scaling_3d.py](https://github.com/ipes-lncc/pymhm/tree/main/examples/introduction/scaling_3d.py). `create_cell_workspace` contains the full UFL volume and face forms; `LocalProvider` owns resource lifetime and orientation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fd9127eb",
   "metadata": {},
   "outputs": [],
   "source": [
    "display(Code(inspect.getsource(study.symbolic_data), language=\"python\"))\n",
    "display(Code(inspect.getsource(study.assemble_cell_equations), language=\"python\"))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28b02c83",
   "metadata": {},
   "source": [
    "Bind the hierarchy and the custom normal-trace interface, then assemble and solve through the generic API. A spawned worker imports the provider directly; no cell compilation or temporary Python module is needed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "376ff8de",
   "metadata": {},
   "outputs": [],
   "source": [
    "macro = study.macro_mesh(MACRO_COUNT)\n",
    "provider = study.LocalProvider(\n",
    "    macro, FINE_N // MACRO_COUNT, data=DATA,\n",
    "    trace_degree=TRACE_DEGREE, quadrature_degree=ASSEMBLY_QUADRATURE_DEGREE,\n",
    ")\n",
    "problem = bind_problem(\n",
    "    MeshHierarchy(macro, provider.local_mesh),\n",
    "    study.TensorFaceInterface(macro, TRACE_DEGREE), provider,\n",
    "    global_equation=Equation(0, 0), retained=1,\n",
    ")\n",
    "system = assemble(problem, execution=ExecutionConfig(\n",
    "    \"process\", 2, native_threads=1, batch_size=2, pipeline=True,\n",
    "))\n",
    "solution = system.solve()\n",
    "print(study.mhm_physical_errors(system, solution, data=DATA, order=7))\n",
    "print({\"trace_dofs\": problem.trace_size, \"relative_residual\": solution.residual})\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4972527a",
   "metadata": {},
   "source": [
    "The default study repeats the original small serial/process and LU/AMG controls, independently assembles conforming Q1 references, checks physical errors and saves the executed basis for replay. Spatial slices compare pressure and vector flux with the exact solution and show the actual macro mesh. Historical timing and accelerator panels retain their own source revisions and hardware."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "799f438b",
   "metadata": {},
   "outputs": [],
   "source": [
    "RUN_SMALL_REPRODUCTION = True\n",
    "RUN_LARGE_CAMPAIGN = False\n",
    "RUN_GPU_COMPONENT = False\n",
    "RUN_FULL_GPU_WORKFLOW = False\n",
    "reproduction = study.run_study(\n",
    "    RUN_SMALL_REPRODUCTION=RUN_SMALL_REPRODUCTION,\n",
    "    RUN_LARGE_CAMPAIGN=RUN_LARGE_CAMPAIGN,\n",
    "    RUN_GPU_COMPONENT=RUN_GPU_COMPONENT,\n",
    "    RUN_FULL_GPU_WORKFLOW=RUN_FULL_GPU_WORKFLOW,\n",
    "    data=DATA, trace_degree=TRACE_DEGREE,\n",
    "    quadrature_degree=ASSEMBLY_QUADRATURE_DEGREE, small_fine_n=FINE_N,\n",
    ")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f6f38bba",
   "metadata": {},
   "source": [
    "Set `RUN_LARGE_CAMPAIGN=True` for the complete 64/96/128-axis CPU strong/weak campaign (32 workers at the largest configuration). `RUN_GPU_COMPONENT=True` measures original local condensation on two GPUs; `RUN_FULL_GPU_WORKFLOW=True` includes CPU assembly, dedicated GPU solves, ordered global assembly and reconstruction with both cuDSS and AMGX. Optional device runs require the corresponding locked accelerator environment and two usable devices.\n",
    "\n",
    "The full helpers preserve all resource checks, synchronization, coefficient/basis archives and physical comparisons. The default run never treats archived timings as measurements of the current execution."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (Pixi introduction)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.13"
  },
  "pymhm_editorial_revision": {
   "date": "2026-10-06",
   "preserved": "Code cells, execution counts, execution timestamps and numerical outputs",
   "preserved_code_cells_sha256": "56b38a70982211ab1a1022b89fc96fd614444cea2bff0691b26d666bfb18cb1a",
   "scope": "Markdown citations and bibliography only"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}