{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0",
   "metadata": {},
   "source": [
    "# Test problem 2: Flow past a cylinder (DFG 2D-3 benchmark)\n",
    "\n",
    "Author: Jørgen S. Dokken\n",
    "\n",
    "In this section, we will turn our attention to a slightly more challenging problem: flow past a cylinder. The geometry and parameters are taken from the [DFG 2D-3 benchmark](https://www.featflow.de/en/benchmarks/cfdbenchmarking/flow/dfg_benchmark3_re100.html) in FeatFlow.\n",
    "\n",
    "To be able to solve this problem efficiently and ensure numerical stability, we will substitute our first order backward difference scheme with a Crank-Nicholson discretization in time, and a semi-implicit Adams-Bashforth approximation of the non-linear term.\n",
    "\n",
    "```{admonition} Computationally demanding demo\n",
    "This demo is computationally demanding, with a run-time up to 15 minutes, as it is using parameters from the DFG 2D-3 benchmark, which consists of 12800 time steps. It is adviced to download this demo and  not run it in a browser. This runtime of the demo can be increased by using 2 or 3 mpi processes.\n",
    "```\n",
    "\n",
    "The computational geometry we would like to use is\n",
    "![Fluid channel with a circular obstacle](turek.png)\n",
    "\n",
    "The kinematic velocity is given by $\\nu=0.001=\\frac{\\mu}{\\rho}$ and the inflow velocity profile is specified as\n",
    "\n",
    "$$\n",
    "    u(x,y,t) = \\left( \\frac{4Uy(0.41-y)}{0.41^2}, 0 \\right)\n",
    "$$\n",
    "\n",
    "$$\n",
    "    U=U(t) = 1.5\\sin(\\pi t/8)\n",
    "$$\n",
    "\n",
    "which has a maximum magnitude of $1.5$ at $y=0.41/2$. We do not use any scaling for this problem since all exact parameters are known.\n",
    "\n",
    "## Mesh generation\n",
    "\n",
    "As in the [Deflection of a membrane](./../chapter1/membrane_code.ipynb) we use GMSH to generate the mesh. We fist create the rectangle and obstacle.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1",
   "metadata": {},
   "outputs": [],
   "source": [
    "import gmsh\n",
    "import os\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import tqdm.autonotebook\n",
    "\n",
    "from mpi4py import MPI\n",
    "from petsc4py import PETSc\n",
    "\n",
    "from basix.ufl import element\n",
    "\n",
    "from dolfinx.fem import (\n",
    "    Constant,\n",
    "    Function,\n",
    "    functionspace,\n",
    "    assemble_scalar,\n",
    "    dirichletbc,\n",
    "    form,\n",
    "    locate_dofs_topological,\n",
    "    set_bc,\n",
    ")\n",
    "from dolfinx.fem.petsc import (\n",
    "    apply_lifting,\n",
    "    assemble_matrix,\n",
    "    assemble_vector,\n",
    "    create_vector,\n",
    "    create_matrix,\n",
    "    set_bc,\n",
    ")\n",
    "from dolfinx.geometry import bb_tree, compute_collisions_points, compute_colliding_cells\n",
    "from dolfinx.io import VTXWriter, gmshio\n",
    "from ufl import (\n",
    "    FacetNormal,\n",
    "    Measure,\n",
    "    TestFunction,\n",
    "    TrialFunction,\n",
    "    as_vector,\n",
    "    div,\n",
    "    dot,\n",
    "    dx,\n",
    "    inner,\n",
    "    lhs,\n",
    "    grad,\n",
    "    nabla_grad,\n",
    "    rhs,\n",
    ")\n",
    "\n",
    "gmsh.initialize()\n",
    "\n",
    "L = 2.2\n",
    "H = 0.41\n",
    "c_x = c_y = 0.2\n",
    "r = 0.05\n",
    "gdim = 2\n",
    "mesh_comm = MPI.COMM_WORLD\n",
    "model_rank = 0\n",
    "if mesh_comm.rank == model_rank:\n",
    "    rectangle = gmsh.model.occ.addRectangle(0, 0, 0, L, H, tag=1)\n",
    "    obstacle = gmsh.model.occ.addDisk(c_x, c_y, 0, r, r)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2",
   "metadata": {},
   "source": [
    "The next step is to subtract the obstacle from the channel, such that we do not mesh the interior of the circle.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3",
   "metadata": {},
   "outputs": [],
   "source": [
    "if mesh_comm.rank == model_rank:\n",
    "    fluid = gmsh.model.occ.cut([(gdim, rectangle)], [(gdim, obstacle)])\n",
    "    gmsh.model.occ.synchronize()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4",
   "metadata": {},
   "source": [
    "To get GMSH to mesh the fluid, we add a physical volume marker\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5",
   "metadata": {},
   "outputs": [],
   "source": [
    "fluid_marker = 1\n",
    "if mesh_comm.rank == model_rank:\n",
    "    volumes = gmsh.model.getEntities(dim=gdim)\n",
    "    assert len(volumes) == 1\n",
    "    gmsh.model.addPhysicalGroup(volumes[0][0], [volumes[0][1]], fluid_marker)\n",
    "    gmsh.model.setPhysicalName(volumes[0][0], fluid_marker, \"Fluid\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6",
   "metadata": {},
   "source": [
    "To tag the different surfaces of the mesh, we tag the inflow (left hand side) with marker 2, the outflow (right hand side) with marker 3 and the fluid walls with 4 and obstacle with 5. We will do this by computing the center of mass for each geometrical entity.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7",
   "metadata": {},
   "outputs": [],
   "source": [
    "inlet_marker, outlet_marker, wall_marker, obstacle_marker = 2, 3, 4, 5\n",
    "inflow, outflow, walls, obstacle = [], [], [], []\n",
    "if mesh_comm.rank == model_rank:\n",
    "    boundaries = gmsh.model.getBoundary(volumes, oriented=False)\n",
    "    for boundary in boundaries:\n",
    "        center_of_mass = gmsh.model.occ.getCenterOfMass(boundary[0], boundary[1])\n",
    "        if np.allclose(center_of_mass, [0, H / 2, 0]):\n",
    "            inflow.append(boundary[1])\n",
    "        elif np.allclose(center_of_mass, [L, H / 2, 0]):\n",
    "            outflow.append(boundary[1])\n",
    "        elif np.allclose(center_of_mass, [L / 2, H, 0]) or np.allclose(\n",
    "            center_of_mass, [L / 2, 0, 0]\n",
    "        ):\n",
    "            walls.append(boundary[1])\n",
    "        else:\n",
    "            obstacle.append(boundary[1])\n",
    "    gmsh.model.addPhysicalGroup(1, walls, wall_marker)\n",
    "    gmsh.model.setPhysicalName(1, wall_marker, \"Walls\")\n",
    "    gmsh.model.addPhysicalGroup(1, inflow, inlet_marker)\n",
    "    gmsh.model.setPhysicalName(1, inlet_marker, \"Inlet\")\n",
    "    gmsh.model.addPhysicalGroup(1, outflow, outlet_marker)\n",
    "    gmsh.model.setPhysicalName(1, outlet_marker, \"Outlet\")\n",
    "    gmsh.model.addPhysicalGroup(1, obstacle, obstacle_marker)\n",
    "    gmsh.model.setPhysicalName(1, obstacle_marker, \"Obstacle\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8",
   "metadata": {},
   "source": [
    "In our previous meshes, we have used uniform mesh sizes. In this example, we will have variable mesh sizes to resolve the flow solution in the area of interest; close to the circular obstacle. To do this, we use GMSH Fields.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create distance field from obstacle.\n",
    "# Add threshold of mesh sizes based on the distance field\n",
    "# LcMax -                  /--------\n",
    "#                      /\n",
    "# LcMin -o---------/\n",
    "#        |         |       |\n",
    "#       Point    DistMin DistMax\n",
    "res_min = r / 3\n",
    "if mesh_comm.rank == model_rank:\n",
    "    distance_field = gmsh.model.mesh.field.add(\"Distance\")\n",
    "    gmsh.model.mesh.field.setNumbers(distance_field, \"EdgesList\", obstacle)\n",
    "    threshold_field = gmsh.model.mesh.field.add(\"Threshold\")\n",
    "    gmsh.model.mesh.field.setNumber(threshold_field, \"IField\", distance_field)\n",
    "    gmsh.model.mesh.field.setNumber(threshold_field, \"LcMin\", res_min)\n",
    "    gmsh.model.mesh.field.setNumber(threshold_field, \"LcMax\", 0.25 * H)\n",
    "    gmsh.model.mesh.field.setNumber(threshold_field, \"DistMin\", r)\n",
    "    gmsh.model.mesh.field.setNumber(threshold_field, \"DistMax\", 2 * H)\n",
    "    min_field = gmsh.model.mesh.field.add(\"Min\")\n",
    "    gmsh.model.mesh.field.setNumbers(min_field, \"FieldsList\", [threshold_field])\n",
    "    gmsh.model.mesh.field.setAsBackgroundMesh(min_field)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "10",
   "metadata": {},
   "source": [
    "## Generating the mesh\n",
    "\n",
    "We are now ready to generate the mesh. However, we have to decide if our mesh should consist of triangles or quadrilaterals. In this demo, to match the DFG 2D-3 benchmark, we use second order quadrilateral elements.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "11",
   "metadata": {},
   "outputs": [],
   "source": [
    "if mesh_comm.rank == model_rank:\n",
    "    gmsh.option.setNumber(\"Mesh.Algorithm\", 8)\n",
    "    gmsh.option.setNumber(\"Mesh.RecombinationAlgorithm\", 2)\n",
    "    gmsh.option.setNumber(\"Mesh.RecombineAll\", 1)\n",
    "    gmsh.option.setNumber(\"Mesh.SubdivisionAlgorithm\", 1)\n",
    "    gmsh.model.mesh.generate(gdim)\n",
    "    gmsh.model.mesh.setOrder(2)\n",
    "    gmsh.model.mesh.optimize(\"Netgen\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12",
   "metadata": {},
   "source": [
    "## Loading mesh and boundary markers\n",
    "\n",
    "As we have generated the mesh, we now need to load the mesh and corresponding facet markers into DOLFINx.\n",
    "To load the mesh, we follow the same structure as in [Deflection of a membrane](./../chapter1/membrane_code.ipynb), with the difference being that we will load in facet markers as well.\n",
    "To learn more about the specifics of the function below, see [A GMSH tutorial for DOLFINx](https://jsdokken.com/src/tutorial_gmsh.html).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "13",
   "metadata": {},
   "outputs": [],
   "source": [
    "mesh_data = gmshio.model_to_mesh(gmsh.model, mesh_comm, model_rank, gdim=gdim)\n",
    "mesh = mesh_data.mesh\n",
    "assert mesh_data.facet_tags is not None\n",
    "ft = mesh_data.facet_tags\n",
    "ft.name = \"Facet markers\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14",
   "metadata": {},
   "source": [
    "## Physical and discretization parameters\n",
    "\n",
    "Following the DGF-2 benchmark, we define our problem specific parameters\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "15",
   "metadata": {},
   "outputs": [],
   "source": [
    "t = 0\n",
    "T = 8  # Final time\n",
    "dt = 1 / 1600  # Time step size\n",
    "num_steps = int(T / dt)\n",
    "k = Constant(mesh, PETSc.ScalarType(dt))\n",
    "mu = Constant(mesh, PETSc.ScalarType(0.001))  # Dynamic viscosity\n",
    "rho = Constant(mesh, PETSc.ScalarType(1))  # Density"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16",
   "metadata": {},
   "source": [
    "```{admonition} Reduced end-time of problem\n",
    "In the current demo, we have reduced the run time to one second to make it easier to illustrate the concepts of the benchmark. By increasing the end-time `T` to 8, the runtime in a notebook is approximately 25 minutes. If you convert the notebook to a python file and use `mpirun`, you can reduce the runtime of the problem.\n",
    "```\n",
    "\n",
    "## Boundary conditions\n",
    "\n",
    "As we have created the mesh and relevant mesh tags, we can now specify the function spaces `V` and `Q` along with the boundary conditions. As the `ft` contains markers for facets, we use this class to find the facets for the inlet and walls.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "17",
   "metadata": {},
   "outputs": [],
   "source": [
    "v_cg2 = element(\"Lagrange\", mesh.topology.cell_name(), 2, shape=(mesh.geometry.dim,))\n",
    "s_cg1 = element(\"Lagrange\", mesh.topology.cell_name(), 1)\n",
    "V = functionspace(mesh, v_cg2)\n",
    "Q = functionspace(mesh, s_cg1)\n",
    "\n",
    "fdim = mesh.topology.dim - 1\n",
    "\n",
    "# Define boundary conditions\n",
    "\n",
    "\n",
    "class InletVelocity:\n",
    "    def __init__(self, t):\n",
    "        self.t = t\n",
    "\n",
    "    def __call__(self, x):\n",
    "        values = np.zeros((gdim, x.shape[1]), dtype=PETSc.ScalarType)\n",
    "        values[0] = (\n",
    "            4 * 1.5 * np.sin(self.t * np.pi / 8) * x[1] * (0.41 - x[1]) / (0.41**2)\n",
    "        )\n",
    "        return values\n",
    "\n",
    "\n",
    "# Inlet\n",
    "u_inlet = Function(V)\n",
    "inlet_velocity = InletVelocity(t)\n",
    "u_inlet.interpolate(inlet_velocity)\n",
    "bcu_inflow = dirichletbc(\n",
    "    u_inlet, locate_dofs_topological(V, fdim, ft.find(inlet_marker))\n",
    ")\n",
    "# Walls\n",
    "u_nonslip = np.array((0,) * mesh.geometry.dim, dtype=PETSc.ScalarType)\n",
    "bcu_walls = dirichletbc(\n",
    "    u_nonslip, locate_dofs_topological(V, fdim, ft.find(wall_marker)), V\n",
    ")\n",
    "# Obstacle\n",
    "bcu_obstacle = dirichletbc(\n",
    "    u_nonslip, locate_dofs_topological(V, fdim, ft.find(obstacle_marker)), V\n",
    ")\n",
    "bcu = [bcu_inflow, bcu_obstacle, bcu_walls]\n",
    "# Outlet\n",
    "bcp_outlet = dirichletbc(\n",
    "    PETSc.ScalarType(0), locate_dofs_topological(Q, fdim, ft.find(outlet_marker)), Q\n",
    ")\n",
    "bcp = [bcp_outlet]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "18",
   "metadata": {},
   "source": [
    "## Variational form\n",
    "\n",
    "As opposed to [Pouseille flow](./ns_code1.ipynb), we will use a Crank-Nicolson discretization, and an semi-implicit Adams-Bashforth approximation.\n",
    "The first step can be written as\n",
    "\n",
    "$$\n",
    "\\rho\\left(\\frac{u^*- u^n}{\\delta t} + \\left(\\frac{3}{2}u^{n} - \\frac{1}{2} u^{n-1}\\right)\\cdot \\frac{1}{2}\\nabla (u^*+u^n) \\right) - \\frac{1}{2}\\mu \\Delta( u^*+ u^n )+ \\nabla p^{n-1/2} = f^{n+\\frac{1}{2}} \\qquad \\text{ in } \\Omega\n",
    "$$\n",
    "\n",
    "$$\n",
    "u^{*}=g(\\cdot, t^{n+1}) \\qquad \\text{ on } \\partial \\Omega_{D}\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\frac{1}{2}\\nu \\nabla (u^*+u^n) \\cdot n = p^{n-\\frac{1}{2}} \\qquad \\text{ on } \\partial \\Omega_{N}\n",
    "$$\n",
    "\n",
    "where we have used the two previous time steps in the temporal derivative for the velocity, and compute the pressure staggered in time, at the time between the previous and current solution. The second step becomes\n",
    "\n",
    "$$\n",
    "\\nabla^2 \\phi = \\frac{\\rho}{\\delta t} \\nabla \\cdot u^* \\qquad\\text{in } \\Omega,\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\nabla \\phi \\cdot n = 0 \\qquad \\text{on } \\partial \\Omega_D,\n",
    "$$\n",
    "\n",
    "$$\n",
    "\\phi = 0 \\qquad\\text{on } \\partial\\Omega_N\n",
    "$$\n",
    "\n",
    "where $p^{n+\\frac{1}{2}}=p^{n-\\frac{1}{2}} + \\phi$.\n",
    "Finally, the third step is\n",
    "\n",
    "$$\n",
    "\\rho (u^{n+1}-u^{*}) = -\\delta t \\nabla\\phi.\n",
    "$$\n",
    "\n",
    "We start by defining all the variables used in the variational formulations.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19",
   "metadata": {},
   "outputs": [],
   "source": [
    "u = TrialFunction(V)\n",
    "v = TestFunction(V)\n",
    "u_ = Function(V)\n",
    "u_.name = \"u\"\n",
    "u_s = Function(V)\n",
    "u_n = Function(V)\n",
    "u_n1 = Function(V)\n",
    "p = TrialFunction(Q)\n",
    "q = TestFunction(Q)\n",
    "p_ = Function(Q)\n",
    "p_.name = \"p\"\n",
    "phi = Function(Q)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20",
   "metadata": {},
   "source": [
    "Next, we define the variational formulation for the first step, where we have integrated the diffusion term, as well as the pressure term by parts.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "21",
   "metadata": {},
   "outputs": [],
   "source": [
    "f = Constant(mesh, PETSc.ScalarType((0, 0)))\n",
    "F1 = rho / k * dot(u - u_n, v) * dx\n",
    "F1 += inner(dot(1.5 * u_n - 0.5 * u_n1, 0.5 * nabla_grad(u + u_n)), v) * dx\n",
    "F1 += 0.5 * mu * inner(grad(u + u_n), grad(v)) * dx - dot(p_, div(v)) * dx\n",
    "F1 += dot(f, v) * dx\n",
    "a1 = form(lhs(F1))\n",
    "L1 = form(rhs(F1))\n",
    "A1 = create_matrix(a1)\n",
    "b1 = create_vector(L1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22",
   "metadata": {},
   "source": [
    "Next we define the second step\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "23",
   "metadata": {},
   "outputs": [],
   "source": [
    "a2 = form(dot(grad(p), grad(q)) * dx)\n",
    "L2 = form(-rho / k * dot(div(u_s), q) * dx)\n",
    "A2 = assemble_matrix(a2, bcs=bcp)\n",
    "A2.assemble()\n",
    "b2 = create_vector(L2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24",
   "metadata": {},
   "source": [
    "We finally create the last step\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25",
   "metadata": {},
   "outputs": [],
   "source": [
    "a3 = form(rho * dot(u, v) * dx)\n",
    "L3 = form(rho * dot(u_s, v) * dx - k * dot(nabla_grad(phi), v) * dx)\n",
    "A3 = assemble_matrix(a3)\n",
    "A3.assemble()\n",
    "b3 = create_vector(L3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "26",
   "metadata": {},
   "source": [
    "As in the previous tutorials, we use PETSc as a linear algebra backend.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "27",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Solver for step 1\n",
    "solver1 = PETSc.KSP().create(mesh.comm)\n",
    "solver1.setOperators(A1)\n",
    "solver1.setType(PETSc.KSP.Type.BCGS)\n",
    "pc1 = solver1.getPC()\n",
    "pc1.setType(PETSc.PC.Type.JACOBI)\n",
    "\n",
    "# Solver for step 2\n",
    "solver2 = PETSc.KSP().create(mesh.comm)\n",
    "solver2.setOperators(A2)\n",
    "solver2.setType(PETSc.KSP.Type.MINRES)\n",
    "pc2 = solver2.getPC()\n",
    "pc2.setType(PETSc.PC.Type.HYPRE)\n",
    "pc2.setHYPREType(\"boomeramg\")\n",
    "\n",
    "# Solver for step 3\n",
    "solver3 = PETSc.KSP().create(mesh.comm)\n",
    "solver3.setOperators(A3)\n",
    "solver3.setType(PETSc.KSP.Type.CG)\n",
    "pc3 = solver3.getPC()\n",
    "pc3.setType(PETSc.PC.Type.SOR)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "28",
   "metadata": {},
   "source": [
    "## Verification of the implementation compute known physical quantities\n",
    "\n",
    "As a further verification of our implementation, we compute the drag and lift coefficients over the obstacle, defined as\n",
    "\n",
    "$$\n",
    "    C_{\\text{D}}(u,p,t,\\partial\\Omega_S) = \\frac{2}{\\rho L U_{mean}^2}\\int_{\\partial\\Omega_S}\\rho \\nu n \\cdot \\nabla u_{t_S}(t)n_y -p(t)n_x~\\mathrm{d} s,\n",
    "$$\n",
    "\n",
    "$$\n",
    "    C_{\\text{L}}(u,p,t,\\partial\\Omega_S) = -\\frac{2}{\\rho L U_{mean}^2}\\int_{\\partial\\Omega_S}\\rho \\nu n \\cdot \\nabla u_{t_S}(t)n_x + p(t)n_y~\\mathrm{d} s,\n",
    "$$\n",
    "\n",
    "where $u_{t_S}$ is the tangential velocity component at the interface of the obstacle $\\partial\\Omega_S$, defined as $u_{t_S}=u\\cdot (n_y,-n_x)$, $U_{mean}=1$ the average inflow velocity, and $L$ the length of the channel. We use `UFL` to create the relevant integrals, and assemble them at each time step.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "29",
   "metadata": {},
   "outputs": [],
   "source": [
    "n = -FacetNormal(mesh)  # Normal pointing out of obstacle\n",
    "dObs = Measure(\"ds\", domain=mesh, subdomain_data=ft, subdomain_id=obstacle_marker)\n",
    "u_t = inner(as_vector((n[1], -n[0])), u_)\n",
    "drag = form(2 / 0.1 * (mu / rho * inner(grad(u_t), n) * n[1] - p_ * n[0]) * dObs)\n",
    "lift = form(-2 / 0.1 * (mu / rho * inner(grad(u_t), n) * n[0] + p_ * n[1]) * dObs)\n",
    "if mesh.comm.rank == 0:\n",
    "    C_D = np.zeros(num_steps, dtype=PETSc.ScalarType)\n",
    "    C_L = np.zeros(num_steps, dtype=PETSc.ScalarType)\n",
    "    t_u = np.zeros(num_steps, dtype=np.float64)\n",
    "    t_p = np.zeros(num_steps, dtype=np.float64)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30",
   "metadata": {},
   "source": [
    "We will also evaluate the pressure at two points, one in front of the obstacle, $(0.15, 0.2)$, and one behind the obstacle, $(0.25, 0.2)$. To do this, we have to find which cell contains each of the points, so that we can create a linear combination of the local basis functions and coefficients.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "31",
   "metadata": {},
   "outputs": [],
   "source": [
    "tree = bb_tree(mesh, mesh.geometry.dim)\n",
    "points = np.array([[0.15, 0.2, 0], [0.25, 0.2, 0]])\n",
    "cell_candidates = compute_collisions_points(tree, points)\n",
    "colliding_cells = compute_colliding_cells(mesh, cell_candidates, points)\n",
    "front_cells = colliding_cells.links(0)\n",
    "back_cells = colliding_cells.links(1)\n",
    "if mesh.comm.rank == 0:\n",
    "    p_diff = np.zeros(num_steps, dtype=PETSc.ScalarType)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32",
   "metadata": {},
   "source": [
    "## Solving the time-dependent problem\n",
    "\n",
    "```{admonition} Stability of the Navier-Stokes equation\n",
    "Note that the current splitting scheme has to fullfil the a [Courant–Friedrichs–Lewy condition](https://en.wikipedia.org/wiki/Courant%E2%80%93Friedrichs%E2%80%93Lewy_condition). This limits the spatial discretization with respect to the inlet velocity and temporal discretization.\n",
    "Other temporal discretization schemes such as the second order backward difference discretization or Crank-Nicholson discretization with Adams-Bashforth linearization are better behaved than our simple backward difference scheme.\n",
    "```\n",
    "\n",
    "As in the previous example, we create output files for the velocity and pressure and solve the time-dependent problem. As we are solving a time dependent problem with many time steps, we use the `tqdm`-package to visualize the progress. This package can be installed with `pip3`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pathlib import Path"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "34",
   "metadata": {},
   "outputs": [],
   "source": [
    "folder = Path(\"results\")\n",
    "folder.mkdir(exist_ok=True, parents=True)\n",
    "vtx_u = VTXWriter(mesh.comm, folder / \"dfg2D-3-u.bp\", [u_], engine=\"BP4\")\n",
    "vtx_p = VTXWriter(mesh.comm, folder / \"dfg2D-3-p.bp\", [p_], engine=\"BP4\")\n",
    "vtx_u.write(t)\n",
    "vtx_p.write(t)\n",
    "progress = tqdm.autonotebook.tqdm(desc=\"Solving PDE\", total=num_steps)\n",
    "for i in range(num_steps):\n",
    "    progress.update(1)\n",
    "    # Update current time step\n",
    "    t += dt\n",
    "    # Update inlet velocity\n",
    "    inlet_velocity.t = t\n",
    "    u_inlet.interpolate(inlet_velocity)\n",
    "\n",
    "    # Step 1: Tentative velocity step\n",
    "    A1.zeroEntries()\n",
    "    assemble_matrix(A1, a1, bcs=bcu)\n",
    "    A1.assemble()\n",
    "    with b1.localForm() as loc:\n",
    "        loc.set(0)\n",
    "    assemble_vector(b1, L1)\n",
    "    apply_lifting(b1, [a1], [bcu])\n",
    "    b1.ghostUpdate(addv=PETSc.InsertMode.ADD_VALUES, mode=PETSc.ScatterMode.REVERSE)\n",
    "    set_bc(b1, bcu)\n",
    "    solver1.solve(b1, u_s.x.petsc_vec)\n",
    "    u_s.x.scatter_forward()\n",
    "\n",
    "    # Step 2: Pressure corrrection step\n",
    "    with b2.localForm() as loc:\n",
    "        loc.set(0)\n",
    "    assemble_vector(b2, L2)\n",
    "    apply_lifting(b2, [a2], [bcp])\n",
    "    b2.ghostUpdate(addv=PETSc.InsertMode.ADD_VALUES, mode=PETSc.ScatterMode.REVERSE)\n",
    "    set_bc(b2, bcp)\n",
    "    solver2.solve(b2, phi.x.petsc_vec)\n",
    "    phi.x.scatter_forward()\n",
    "\n",
    "    p_.x.petsc_vec.axpy(1, phi.x.petsc_vec)\n",
    "    p_.x.scatter_forward()\n",
    "\n",
    "    # Step 3: Velocity correction step\n",
    "    with b3.localForm() as loc:\n",
    "        loc.set(0)\n",
    "    assemble_vector(b3, L3)\n",
    "    b3.ghostUpdate(addv=PETSc.InsertMode.ADD_VALUES, mode=PETSc.ScatterMode.REVERSE)\n",
    "    solver3.solve(b3, u_.x.petsc_vec)\n",
    "    u_.x.scatter_forward()\n",
    "\n",
    "    # Write solutions to file\n",
    "    vtx_u.write(t)\n",
    "    vtx_p.write(t)\n",
    "\n",
    "    # Update variable with solution form this time step\n",
    "    with (\n",
    "        u_.x.petsc_vec.localForm() as loc_,\n",
    "        u_n.x.petsc_vec.localForm() as loc_n,\n",
    "        u_n1.x.petsc_vec.localForm() as loc_n1,\n",
    "    ):\n",
    "        loc_n.copy(loc_n1)\n",
    "        loc_.copy(loc_n)\n",
    "\n",
    "    # Compute physical quantities\n",
    "    # For this to work in paralell, we gather contributions from all processors\n",
    "    # to processor zero and sum the contributions.\n",
    "    drag_coeff = mesh.comm.gather(assemble_scalar(drag), root=0)\n",
    "    lift_coeff = mesh.comm.gather(assemble_scalar(lift), root=0)\n",
    "    p_front = None\n",
    "    if len(front_cells) > 0:\n",
    "        p_front = p_.eval(points[0], front_cells[:1])\n",
    "    p_front = mesh.comm.gather(p_front, root=0)\n",
    "    p_back = None\n",
    "    if len(back_cells) > 0:\n",
    "        p_back = p_.eval(points[1], back_cells[:1])\n",
    "    p_back = mesh.comm.gather(p_back, root=0)\n",
    "    if mesh.comm.rank == 0:\n",
    "        t_u[i] = t\n",
    "        t_p[i] = t - dt / 2\n",
    "        C_D[i] = sum(drag_coeff)\n",
    "        C_L[i] = sum(lift_coeff)\n",
    "        # Choose first pressure that is found from the different processors\n",
    "        for pressure in p_front:\n",
    "            if pressure is not None:\n",
    "                p_diff[i] = pressure[0]\n",
    "                break\n",
    "        for pressure in p_back:\n",
    "            if pressure is not None:\n",
    "                p_diff[i] -= pressure[0]\n",
    "                break\n",
    "progress.close()\n",
    "vtx_u.close()\n",
    "vtx_p.close()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "35",
   "metadata": {},
   "source": [
    "## Verification using data from FEATFLOW\n",
    "\n",
    "As FEATFLOW has provided data for different discretization levels, we compare our numerical data with the data provided using `matplotlib`.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "36",
   "metadata": {},
   "outputs": [],
   "source": [
    "if mesh.comm.rank == 0:\n",
    "    if not os.path.exists(\"figures\"):\n",
    "        os.mkdir(\"figures\")\n",
    "    num_velocity_dofs = V.dofmap.index_map_bs * V.dofmap.index_map.size_global\n",
    "    num_pressure_dofs = Q.dofmap.index_map_bs * V.dofmap.index_map.size_global\n",
    "\n",
    "    turek = np.loadtxt(\"bdforces_lv4\")\n",
    "    turek_p = np.loadtxt(\"pointvalues_lv4\")\n",
    "    fig = plt.figure(figsize=(25, 8))\n",
    "    l1 = plt.plot(\n",
    "        t_u,\n",
    "        C_D,\n",
    "        label=r\"FEniCSx  ({0:d} dofs)\".format(num_velocity_dofs + num_pressure_dofs),\n",
    "        linewidth=2,\n",
    "    )\n",
    "    l2 = plt.plot(\n",
    "        turek[1:, 1],\n",
    "        turek[1:, 3],\n",
    "        marker=\"x\",\n",
    "        markevery=50,\n",
    "        linestyle=\"\",\n",
    "        markersize=4,\n",
    "        label=\"FEATFLOW (42016 dofs)\",\n",
    "    )\n",
    "    plt.title(\"Drag coefficient\")\n",
    "    plt.grid()\n",
    "    plt.legend()\n",
    "    plt.savefig(\"figures/drag_comparison.png\")\n",
    "\n",
    "    fig = plt.figure(figsize=(25, 8))\n",
    "    l1 = plt.plot(\n",
    "        t_u,\n",
    "        C_L,\n",
    "        label=r\"FEniCSx  ({0:d} dofs)\".format(num_velocity_dofs + num_pressure_dofs),\n",
    "        linewidth=2,\n",
    "    )\n",
    "    l2 = plt.plot(\n",
    "        turek[1:, 1],\n",
    "        turek[1:, 4],\n",
    "        marker=\"x\",\n",
    "        markevery=50,\n",
    "        linestyle=\"\",\n",
    "        markersize=4,\n",
    "        label=\"FEATFLOW (42016 dofs)\",\n",
    "    )\n",
    "    plt.title(\"Lift coefficient\")\n",
    "    plt.grid()\n",
    "    plt.legend()\n",
    "    plt.savefig(\"figures/lift_comparison.png\")\n",
    "\n",
    "    fig = plt.figure(figsize=(25, 8))\n",
    "    l1 = plt.plot(\n",
    "        t_p,\n",
    "        p_diff,\n",
    "        label=r\"FEniCSx ({0:d} dofs)\".format(num_velocity_dofs + num_pressure_dofs),\n",
    "        linewidth=2,\n",
    "    )\n",
    "    l2 = plt.plot(\n",
    "        turek[1:, 1],\n",
    "        turek_p[1:, 6] - turek_p[1:, -1],\n",
    "        marker=\"x\",\n",
    "        markevery=50,\n",
    "        linestyle=\"\",\n",
    "        markersize=4,\n",
    "        label=\"FEATFLOW (42016 dofs)\",\n",
    "    )\n",
    "    plt.title(\"Pressure difference\")\n",
    "    plt.grid()\n",
    "    plt.legend()\n",
    "    plt.savefig(\"figures/pressure_comparison.png\")"
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "formats": "ipynb,py:light"
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}