\n",
"\n",
"# Hadoop MiniCluster\n",
" \n",
" \n",
"\n",
"The Hadoop MiniCluster is a lightweight, single-node Hadoop cluster that is primarily used for testing and development purposes. It provides a way to simulate a distributed Hadoop environment on a single machine, allowing developers to experiment with and test their Hadoop applications without the need for a full-scale, multi-node cluster.\n",
"\n",
"Quote from [the official Hadoop documentation](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/CLIMiniCluster.html):\n",
"\n",
"\n",
"> *Using the CLI MiniCluster, users can simply start and stop a single-node Hadoop cluster with a single command, and without the need to set any environment variables or manage configuration files. The CLI MiniCluster starts both a YARN/MapReduce & HDFS clusters.*\n",
"\n",
"\n",
"In this notebook we download the Hadoop core and guide you through the steps required to launch the MiniCluster."
],
"metadata": {
"id": "OFLizmmFvTjK"
}
},
{
"cell_type": "code",
"source": [
"# @title\n",
"from IPython.core.display import HTML\n",
"HTML(\"\"\"\n",
"
\n",
"\n",
" Note:\n",
"While the MiniCluster is useful for many development and testing scenarios, it's important to note that it does not fully replicate the complexities and challenges of a true multi-node Hadoop cluster. For production-scale testing or performance evaluations, a larger, more representative cluster setup is recommended.\n",
"
\n",
"\n",
" Note:\n",
"While the MiniCluster is useful for many development and testing scenarios, it's important to note that it does not fully replicate the complexities and challenges of a true multi-node Hadoop cluster. For production-scale testing or performance evaluations, a larger, more representative cluster setup is recommended.\n",
"
\n"
]
},
"metadata": {},
"execution_count": 1
}
]
},
{
"cell_type": "markdown",
"source": [
"## About this tutorial\n",
"\n",
"This tutorial dives deep to help you really get the hang of things, explaining every step in Big Data processing, even if it takes a bit of time, so just hang in there and be patient! "
],
"metadata": {
"id": "piPXIeS3KPDd"
}
},
{
"cell_type": "markdown",
"source": [
"# Preamble"
],
"metadata": {
"id": "Plpb6qoZ8EVh"
}
},
{
"cell_type": "markdown",
"source": [
"## Download Hadoop"
],
"metadata": {
"id": "xvhtO--1RWug"
}
},
{
"cell_type": "code",
"source": [
"import urllib.request\n",
"import os\n",
"import shutil\n",
"import tarfile\n",
"import logging\n",
"import subprocess\n",
"import time\n",
"import sys\n",
"############\n",
"# COSTANTS #\n",
"############\n",
"\n",
"# URL for downloading Hadoop (archive site https://archive.apache.org/dist/)\n",
"HADOOP_URL = \"https://archive.apache.org/dist/hadoop/core/hadoop-3.4.0/hadoop-3.4.0.tar.gz\"\n",
"\n",
"# logging level (should be one of: DEBUG, INFO, WARNING, ERROR, CRITICAL)\n",
"LOGGING_LEVEL = \"INFO\" #@param [\"DEBUG\", \"INFO\", \"WARNING\", \"ERROR\", \"CRITICAL\"]\n",
"\n",
"# setup logging\n",
"for handler in logging.root.handlers[:]:\n",
" logging.root.removeHandler(handler)\n",
"\n",
"logging_level = getattr(logging, LOGGING_LEVEL.upper(), 10)\n",
"\n",
"logging.basicConfig(level=logging_level, \\\n",
" format='%(asctime)s - %(levelname)s: %(message)s', \\\n",
" datefmt='%d-%b-%y %I:%M:%S %p')\n",
"\n",
"logger = logging.getLogger('my_logger')\n",
"\n",
"JAVA_PATH = '/usr/lib/jvm/java-11-openjdk-amd64'\n",
"\n",
"# true if running on Google Colab\n",
"IN_COLAB = 'google.colab' in sys.modules\n",
"if IN_COLAB:\n",
" from google.colab import output\n",
"\n",
"# setup logging\n",
"for handler in logging.root.handlers[:]:\n",
" logging.root.removeHandler(handler)\n",
"\n",
"logging_level = getattr(logging, LOGGING_LEVEL.upper(), 10)\n",
"\n",
"logging.basicConfig(level=logging_level, \\\n",
" format='%(asctime)s - %(levelname)s: %(message)s', \\\n",
" datefmt='%d-%b-%y %I:%M:%S %p')\n",
"\n",
"logger = logging.getLogger('my_logger')\n",
"\n",
"# set variable JAVA_HOME (install Java if necessary)\n",
"def is_java_installed():\n",
" os.environ['JAVA_HOME'] = os.path.realpath(shutil.which(\"java\")).split('/bin')[0]\n",
" return os.environ['JAVA_HOME']\n",
"\n",
"def install_java():\n",
" # Uncomment and modify the desired version\n",
" # java_version= 'openjdk-11-jre-headless'\n",
" # java_version= 'default-jre'\n",
" # java_version= 'openjdk-17-jre-headless'\n",
" # java_version= 'openjdk-18-jre-headless'\n",
" java_version= 'openjdk-19-jre-headless'\n",
"\n",
" print(f\"Java not found. Installing {java_version} ... (this might take a while)\")\n",
" try:\n",
" cmd = f\"apt install -y {java_version}\"\n",
" subprocess_output = subprocess.run(cmd, shell=True, check=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT, text=True)\n",
" stdout_result = subprocess_output.stdout\n",
" # Process the results as needed\n",
" logger.info(\"Done installing Java {}\".format(java_version))\n",
" os.environ['JAVA_HOME'] = os.path.realpath(shutil.which(\"java\")).split('/bin')[0]\n",
" logger.info(\"JAVA_HOME is {}\".format(os.environ['JAVA_HOME']))\n",
" except subprocess.CalledProcessError as e:\n",
" # Handle the error if the command returns a non-zero exit code\n",
" logger.warn(\"Command failed with return code {}\".format(e.returncode))\n",
" logger.warn(\"stdout: {}\".format(e.stdout))\n",
"\n",
"# Install Java if not available\n",
"if is_java_installed():\n",
" logger.info(\"Java is already installed: {}\".format(os.environ['JAVA_HOME']))\n",
"else:\n",
" logger.info(\"Installing Java\")\n",
" install_java()\n",
"\n",
"\n",
"# download Hadoop\n",
"file_name = os.path.basename(HADOOP_URL)\n",
"if os.path.isfile(file_name):\n",
" logger.info(\"{} already exists, not downloading\".format(file_name))\n",
"else:\n",
" logger.info(\"Downloading {}\".format(file_name))\n",
" urllib.request.urlretrieve(HADOOP_URL, file_name)\n",
"\n",
"# uncompress archive\n",
"dir_name = file_name[:-7]\n",
"if os.path.exists(dir_name):\n",
" logger.info(\"{} is already uncompressed\".format(file_name))\n",
"else:\n",
" logger.info(\"Uncompressing {}\".format(file_name))\n",
" tar = tarfile.open(file_name)\n",
" tar.extractall()\n",
" tar.close()\n",
"\n",
"# environment variables\n",
"os.environ['HADOOP_HOME'] = os.path.join(os.getcwd(), dir_name)\n",
"logger.info(\"HADOOP_HOME is {}\".format(os.environ['HADOOP_HOME']))\n",
"os.environ['PATH'] = ':'.join([os.path.join(os.environ['HADOOP_HOME'], 'bin'), os.environ['PATH']])\n",
"logger.info(\"PATH is {}\".format(os.environ['PATH']))\n"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "6223c729-12a3-4010-b232-4d0fe4de25c3",
"id": "qUVKZ2j0vTjS"
},
"execution_count": 2,
"outputs": [
{
"output_type": "stream",
"name": "stderr",
"text": [
"04-Aug-24 07:09:23 PM - INFO: Java is already installed: /usr/lib/jvm/java-11-openjdk-amd64\n",
"04-Aug-24 07:09:23 PM - INFO: Downloading hadoop-3.4.0.tar.gz\n",
"04-Aug-24 07:10:00 PM - INFO: Uncompressing hadoop-3.4.0.tar.gz\n",
"04-Aug-24 07:10:26 PM - INFO: HADOOP_HOME is /content/hadoop-3.4.0\n",
"04-Aug-24 07:10:26 PM - INFO: PATH is /content/hadoop-3.4.0/bin:/opt/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/tools/node/bin:/tools/google-cloud-sdk/bin\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"### Check environment variables used by `mapred`\n",
"\n",
"The following steps are not needed but they might be useful to get familiar the `mapred` command.\n",
"\n",
"`mapred minicluster` is the command that we are going to use to start the MiniCluster once a few variables and libraries are taken care of."
],
"metadata": {
"id": "wWzin-_oRuTl"
}
},
{
"cell_type": "code",
"source": [
"!mapred -h"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "FmCr_cI05Fvm",
"outputId": "13ab3991-d6d1-40b2-e11f-b903f0703b40"
},
"execution_count": 3,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Usage: mapred [OPTIONS] SUBCOMMAND [SUBCOMMAND OPTIONS]\n",
" or mapred [OPTIONS] CLASSNAME [CLASSNAME OPTIONS]\n",
" where CLASSNAME is a user-provided Java class\n",
"\n",
" OPTIONS is none or any of:\n",
"\n",
"--config dir Hadoop config directory\n",
"--debug turn on shell script debug mode\n",
"--help usage information\n",
"\n",
" SUBCOMMAND is one of:\n",
"\n",
"\n",
" Admin Commands:\n",
"\n",
"frameworkuploader mapreduce framework upload\n",
"hsadmin job history server admin interface\n",
"\n",
" Client Commands:\n",
"\n",
"classpath prints the class path needed for running mapreduce subcommands\n",
"envvars display computed Hadoop environment variables\n",
"job manipulate MapReduce jobs\n",
"minicluster CLI MiniCluster\n",
"pipes run a Pipes job\n",
"queue get information regarding JobQueues\n",
"sampler sampler\n",
"version print the version\n",
"\n",
" Daemon Commands:\n",
"\n",
"historyserver run job history servers as a standalone daemon\n",
"\n",
"SUBCOMMAND may print help when invoked w/o parameters or with -h.\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"!mapred envvars"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "381816ac-2a99-4a5e-c3ee-58ba5322d8ec",
"id": "XSF0r1CJvTjT"
},
"execution_count": 4,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"JAVA_HOME='/usr/lib/jvm/java-11-openjdk-amd64'\n",
"HADOOP_MAPRED_HOME='/content/hadoop-3.4.0'\n",
"MAPRED_DIR='share/hadoop/mapreduce'\n",
"MAPRED_LIB_JARS_DIR='share/hadoop/mapreduce/lib'\n",
"HADOOP_CONF_DIR='/content/hadoop-3.4.0/etc/hadoop'\n",
"HADOOP_TOOLS_HOME='/content/hadoop-3.4.0'\n",
"HADOOP_TOOLS_DIR='share/hadoop/tools'\n",
"HADOOP_TOOLS_LIB_JARS_DIR='share/hadoop/tools/lib'\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## Set `HADOOP_TOOLS_LIB_JARS_DIR`\n",
"\n",
"This variable needs to point to the folder containing Hadoop libraries. As you can see in the output of `mapred envvars`, by default this is set incorrectly to `share/hadoop/tools/lib`."
],
"metadata": {
"id": "-vNiekW25pbB"
}
},
{
"cell_type": "code",
"source": [
"os.environ['HADOOP_TOOLS_LIB_JARS_DIR'] = os.path.join(os.environ['HADOOP_HOME'], 'share/hadoop/tools/lib/') #IMPORTANT"
],
"metadata": {
"id": "EKAwTLHdvTjU"
},
"execution_count": 5,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"## Install the `mockito` library\n",
"\n",
"To figure out which version of `mockito` is compatible with the current version of Hadoop, check this page for library dependency analysis: https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/dependency-analysis.html\n",
"\n",
"We need Mockito 2.28.2."
],
"metadata": {
"id": "zrUFZyHOSBb5"
}
},
{
"cell_type": "markdown",
"source": [
"After a new Hadoop installation, the mockito library is not yet there!"
],
"metadata": {
"id": "pQrBSymx6xdT"
}
},
{
"cell_type": "code",
"source": [
"!find hadoop-3.4.0 -name \"mockito*\""
],
"metadata": {
"id": "84AGO_GwvTjU"
},
"execution_count": 6,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Download it from the Maven repository."
],
"metadata": {
"id": "ztngaw_S67Hd"
}
},
{
"cell_type": "code",
"source": [
"!wget --no-clobber https://repo1.maven.org/maven2/org/mockito/mockito-core/2.28.2/mockito-core-2.28.2.jar"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "99ccdabf-d9bf-46d3-8caa-96e2d54eb0f7",
"id": "9W4RaUGbvTjV"
},
"execution_count": 7,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"--2024-08-04 19:10:27-- https://repo1.maven.org/maven2/org/mockito/mockito-core/2.28.2/mockito-core-2.28.2.jar\n",
"Resolving repo1.maven.org (repo1.maven.org)... 199.232.192.209, 199.232.196.209, 2a04:4e42:4c::209, ...\n",
"Connecting to repo1.maven.org (repo1.maven.org)|199.232.192.209|:443... connected.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 591179 (577K) [application/java-archive]\n",
"Saving to: ‘mockito-core-2.28.2.jar’\n",
"\n",
"\rmockito-core-2.28.2 0%[ ] 0 --.-KB/s \rmockito-core-2.28.2 100%[===================>] 577.32K --.-KB/s in 0.05s \n",
"\n",
"2024-08-04 19:10:27 (10.3 MB/s) - ‘mockito-core-2.28.2.jar’ saved [591179/591179]\n",
"\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Install the library in a location where it can be found."
],
"metadata": {
"id": "rlWXc92bSjeE"
}
},
{
"cell_type": "code",
"source": [
"shutil.copy('mockito-core-2.28.2.jar', os.path.join(os.environ['HADOOP_HOME'],'share/hadoop/mapreduce/'))"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 0
},
"id": "jk6CvZZY4ehO",
"outputId": "05d80737-cc4c-4c61-cc8a-1b84f3440671"
},
"execution_count": 8,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"'/content/hadoop-3.4.0/share/hadoop/mapreduce/mockito-core-2.28.2.jar'"
],
"application/vnd.google.colaboratory.intrinsic+json": {
"type": "string"
}
},
"metadata": {},
"execution_count": 8
}
]
},
{
"cell_type": "code",
"source": [
"os.listdir(os.path.join(os.environ['HADOOP_HOME'],'share/hadoop/mapreduce/'))"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "b08dae4a-5d9b-4627-c900-d5244c27f6db",
"id": "x5KDVOgFvTjU"
},
"execution_count": 9,
"outputs": [
{
"output_type": "execute_result",
"data": {
"text/plain": [
"['hadoop-mapreduce-client-core-3.4.0.jar',\n",
" 'sources',\n",
" 'hadoop-mapreduce-client-uploader-3.4.0.jar',\n",
" 'hadoop-mapreduce-client-jobclient-3.4.0.jar',\n",
" 'hadoop-mapreduce-client-shuffle-3.4.0.jar',\n",
" 'hadoop-mapreduce-client-common-3.4.0.jar',\n",
" 'mockito-core-2.28.2.jar',\n",
" 'hadoop-mapreduce-client-hs-3.4.0.jar',\n",
" 'hadoop-mapreduce-client-app-3.4.0.jar',\n",
" 'hadoop-mapreduce-client-jobclient-3.4.0-tests.jar',\n",
" 'hadoop-mapreduce-examples-3.4.0.jar',\n",
" 'hadoop-mapreduce-client-nativetask-3.4.0.jar',\n",
" 'hadoop-mapreduce-client-hs-plugins-3.4.0.jar',\n",
" 'jdiff']"
]
},
"metadata": {},
"execution_count": 9
}
]
},
{
"cell_type": "markdown",
"source": [
"## Create folders for the datanodes\n",
"\n",
"These folders are needed for the correct functioning of the MiniCluster."
],
"metadata": {
"id": "FweWN3enS3HV"
}
},
{
"cell_type": "code",
"source": [
"!mkdir -p ./target/test/data/dfs/{name-0-1,name-0-2}"
],
"metadata": {
"id": "KZg6NLydvTjU"
},
"execution_count": 10,
"outputs": []
},
{
"cell_type": "code",
"source": [
"!ls ./target/test/data/dfs/"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "XThvV4kuVYpU",
"outputId": "56e8287a-47e4-4359-c029-1f5fbdad7bd0"
},
"execution_count": 11,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"name-0-1 name-0-2\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"# Start the MiniCluster\n",
"\n",
"\n",
"To see a full list of options run `mapred minicluster -help`."
],
"metadata": {
"id": "7Odx4SZbOhL4"
}
},
{
"cell_type": "code",
"source": [
"!mapred minicluster -help"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "R7YqK3brVG2J",
"outputId": "7ef5aa31-196e-4d68-8aee-c43a45fd4932"
},
"execution_count": 12,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"usage: ...\n",
" -D Options to pass into configuration object\n",
" -datanodes How many datanodes to start (default 1)\n",
" -format Format the DFS (default false)\n",
" -help Prints option help.\n",
" -jhsport JobHistoryServer port (default 0--we choose)\n",
" -namenode URL of the namenode (default is either the DFS\n",
" cluster or a temporary dir)\n",
" -nnhttpport NameNode HTTP port (default 0--we choose)\n",
" -nnport NameNode port (default 0--we choose)\n",
" -nodemanagers How many nodemanagers to start (default 1)\n",
" -nodfs Don't start a mini DFS cluster\n",
" -nomr Don't start a mini MR cluster\n",
" -rmport ResourceManager port (default 0--we choose)\n",
" -writeConfig Save configuration to this XML file.\n",
" -writeDetails Write basic information to this JSON file.\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"If you are not running this notebook for the first time or have edited the `core-site.xml` file you should now empty it to get the default initial configuration.\n",
"\n",
"**Note:** the file `core-site.xml` needs to exist and contain the lines\n",
"\n",
"```\n",
"\n",
"\n",
"\n",
"```\n"
],
"metadata": {
"id": "CKX9vXJ9MRlI"
}
},
{
"cell_type": "code",
"source": [
"# check if the file is there\n",
"!find $HADOOP_HOME -name \"core-site.xml\""
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "VANlMygeMukl",
"outputId": "c9069a09-fb71-4262-87b4-365664f7de9f"
},
"execution_count": 13,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"/content/hadoop-3.4.0/etc/hadoop/core-site.xml\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"# view the contents of the file\n",
"!cat $(find $HADOOP_HOME -name \"core-site.xml\")"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "FyNiiZki_1s1",
"outputId": "e38a9975-38f8-49f9-c347-f7d5327f74e7"
},
"execution_count": 14,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
"\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"with open(os.environ['HADOOP_HOME']+'/etc/hadoop/core-site.xml', 'w') as file:\n",
" file.write(\"\\n\")"
],
"metadata": {
"id": "w4ihWl6hMz2s"
},
"execution_count": 15,
"outputs": []
},
{
"cell_type": "code",
"source": [
"!cat $(find $HADOOP_HOME -name \"core-site.xml\")"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "C4QcevYKdvf7",
"outputId": "901ed557-b095-488d-85b1-bbeccbfd6cba"
},
"execution_count": 16,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"\n",
""
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## `mapred minicluster -format`\n",
"\n",
"Finally, we are all set up to start the MiniCluster.\n",
"\n",
"Make sure to include the `-format` option to initialize and format the filesystem.\n",
"\n",
"Other than that, we are using the defaults for all the other options.\n"
],
"metadata": {
"id": "_2vNXuN1AE6q"
}
},
{
"cell_type": "markdown",
"source": [
"Note that this process runs forever thus blocking the notebook. In order to proceed with the rest of the notebook, just interrupt the running cell.\n",
"\n",
"We'll see later how to run the MiniCluster as a subprocess without blocking the notebook's cells execution.\n",
"\n",
"Uncomment the next cell to launch the MiniCluster!"
],
"metadata": {
"id": "y7jGgVy3WF4S"
}
},
{
"cell_type": "code",
"source": [
"#!mapred minicluster -format"
],
"metadata": {
"id": "5c46g83gvTjX"
},
"execution_count": 17,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"If the MiniMRCluster started correctly, you should see a line like this at the bottom:\n",
"\n",
" 2024-01-14 13:53:15,112 INFO mapreduce.MiniHadoopClusterManager: Started MiniMRCluster"
],
"metadata": {
"id": "IlU0eS5_QGbq"
}
},
{
"cell_type": "markdown",
"source": [
"# Stop the MiniCluster process\n",
"\n",
"To continue to work with this notebook, you need to stop the MiniCluster by terminating the execution of the previous cell."
],
"metadata": {
"id": "Oc0xM9VNRei5"
}
},
{
"cell_type": "markdown",
"source": [
""
],
"metadata": {
"id": "fJGsLuZ46pXf"
}
},
{
"cell_type": "markdown",
"source": [
"# Start the MiniCluster as a subprocess\n",
"\n",
"It is convenient to start the MiniCluster as a subprocess in order prevent it from blocking the execution of other notebook cells.\n",
"\n",
"The MiniCluster is a Java process with multiple listening ports.\n",
"\n"
],
"metadata": {
"id": "F9BfFS27vTjX"
}
},
{
"cell_type": "markdown",
"source": [
"## Use `lsof` to show listening ports\n",
"\n",
"To check ports that have listening services use `lsof`.\n",
"\n",
"The output should look like this:\n",
"\n",
"```\n",
"COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME\n",
"node 7 root 21u IPv6 19666 0t0 TCP *:8080 (LISTEN)\n",
"kernel_manager_ 20 root 3u IPv4 18322 0t0 TCP 172.28.0.12:6000 (LISTEN)\n",
"colab-fileshim. 61 root 3u IPv4 19763 0t0 TCP 127.0.0.1:3453 (LISTEN)\n",
"jupyter-noteboo 79 root 7u IPv4 19989 0t0 TCP 172.28.0.12:9000 (LISTEN)\n",
"python3 428 root 21u IPv4 25506 0t0 TCP 127.0.0.1:38217 (LISTEN)\n",
"python3 467 root 3u IPv4 26425 0t0 TCP 127.0.0.1:43729 (LISTEN)\n",
"python3 467 root 5u IPv4 26426 0t0 TCP 127.0.0.1:60229 (LISTEN)\n",
"```"
],
"metadata": {
"id": "0X1zImnU5Afb"
}
},
{
"cell_type": "code",
"source": [
"!lsof -n -i -P +c0 -sTCP:LISTEN"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "v_ieJekiBv-2",
"outputId": "25c07f4f-56ba-4431-9c8a-f4cd72a25b7c"
},
"execution_count": 18,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME\n",
"node 7 root 21u IPv6 20065 0t0 TCP *:8080 (LISTEN)\n",
"kernel_manager_ 25 root 6u IPv4 19487 0t0 TCP 172.28.0.12:6000 (LISTEN)\n",
"colab-fileshim. 70 root 3u IPv4 19330 0t0 TCP 127.0.0.1:3453 (LISTEN)\n",
"jupyter-noteboo 91 root 7u IPv4 20298 0t0 TCP 172.28.0.12:9000 (LISTEN)\n",
"python3 2301 root 21u IPv4 84721 0t0 TCP 127.0.0.1:37185 (LISTEN)\n",
"python3 2340 root 3u IPv4 86086 0t0 TCP 127.0.0.1:35717 (LISTEN)\n",
"python3 2340 root 5u IPv4 86087 0t0 TCP 127.0.0.1:45663 (LISTEN)\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Options used in `lsof`:\n",
"\n",
"* `-i` specifies that you want to display only network files, that is open network connections\n",
"* `-n` and `-P` tell `lsof` to show IP addresses (`-n`) and ports (`-P`) in numeric form. This makes `lsof` faster as it saves the time for name lookups.\n",
"* the option `+c0` is used to show a longer substring of the name of the UNIX command associated with the process (https://linux.die.net/man/8/lsof)\n",
"* `-sTCP:LISTEN` filters for TCP connections in state `LISTEN`\n",
"\n",
"See also [lsof and listening ports](https://unix.stackexchange.com/questions/26887/lsof-and-listening-ports) on Stackexchange."
],
"metadata": {
"id": "tg9V_qXGZ1fe"
}
},
{
"cell_type": "markdown",
"source": [
"## Start the subprocess\n",
"\n",
"Start the MiniCluster as a subprocess using Python's [`subprocess` library](https://docs.python.org/3/library/subprocess.html).\n",
"\n",
"The files `out.txt` and `err.txt` will contain respectively the standard output and the standard error emitted by the `mapred minicluster` command."
],
"metadata": {
"id": "a-OKONXJBwkf"
}
},
{
"cell_type": "code",
"source": [
"import subprocess\n",
"with open('out.txt', \"w\") as stdout_file, open('err.txt', \"w\") as stderr_file:\n",
" process = subprocess.Popen(\n",
" [\"mapred\", \"minicluster\", \"-format\"],\n",
" stdout=stdout_file,\n",
" stderr=stderr_file\n",
" )"
],
"metadata": {
"id": "aTBwruBkAuP_"
},
"execution_count": 19,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Wait for a couple of seconds because the services might not be available immediately."
],
"metadata": {
"id": "gY484JgGCVUd"
}
},
{
"cell_type": "code",
"source": [
"if not IN_COLAB:\n",
" time.sleep(30)\n",
"else:\n",
" time.sleep(10)"
],
"metadata": {
"id": "dk_FGmuOCoBW"
},
"execution_count": 20,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Now check for listening ports again (you can also refresh the next cell with ctrl-enter)."
],
"metadata": {
"id": "6iBMkb6fCqUU"
}
},
{
"cell_type": "code",
"source": [
"!lsof -n -i -P +c0 -sTCP:LISTEN"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "Rt9J9bj4A_5m",
"outputId": "a18dc323-201b-4680-c488-897b64a3737e"
},
"execution_count": 21,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME\n",
"node 7 root 21u IPv6 20065 0t0 TCP *:8080 (LISTEN)\n",
"kernel_manager_ 25 root 6u IPv4 19487 0t0 TCP 172.28.0.12:6000 (LISTEN)\n",
"colab-fileshim. 70 root 3u IPv4 19330 0t0 TCP 127.0.0.1:3453 (LISTEN)\n",
"jupyter-noteboo 91 root 7u IPv4 20298 0t0 TCP 172.28.0.12:9000 (LISTEN)\n",
"python3 2301 root 21u IPv4 84721 0t0 TCP 127.0.0.1:37185 (LISTEN)\n",
"python3 2340 root 3u IPv4 86086 0t0 TCP 127.0.0.1:35717 (LISTEN)\n",
"python3 2340 root 5u IPv4 86087 0t0 TCP 127.0.0.1:45663 (LISTEN)\n",
"java 2805 root 341u IPv4 98240 0t0 TCP 127.0.0.1:46505 (LISTEN)\n",
"java 2805 root 351u IPv4 98317 0t0 TCP 127.0.0.1:44975 (LISTEN)\n",
"java 2805 root 361u IPv4 98320 0t0 TCP 127.0.0.1:42669 (LISTEN)\n",
"java 2805 root 364u IPv4 100111 0t0 TCP 127.0.0.1:44643 (LISTEN)\n",
"java 2805 root 393u IPv4 98335 0t0 TCP 127.0.0.1:34731 (LISTEN)\n",
"java 2805 root 394u IPv4 98342 0t0 TCP 127.0.0.1:36923 (LISTEN)\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"You should have gotten something like this (a total of $18$ listening ports associated with the MiniCluster process):\n",
"\n",
"```\n",
"COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME\n",
"node 6 root 21u IPv6 17373 0t0 TCP *:8080 (LISTEN)\n",
"kernel_manager_ 20 root 3u IPv4 17180 0t0 TCP 172.28.0.12:6000 (LISTEN)\n",
"colab-fileshim. 58 root 3u IPv4 19499 0t0 TCP 127.0.0.1:3453 (LISTEN)\n",
"jupyter-noteboo 75 root 7u IPv4 19658 0t0 TCP 172.28.0.12:9000 (LISTEN)\n",
"python3 4081 root 21u IPv4 108431 0t0 TCP 127.0.0.1:44519 (LISTEN)\n",
"python3 4108 root 3u IPv4 109755 0t0 TCP 127.0.0.1:46699 (LISTEN)\n",
"python3 4108 root 5u IPv4 109756 0t0 TCP 127.0.0.1:51813 (LISTEN)\n",
"java 17261 root 347u IPv4 390097 0t0 TCP 127.0.0.1:38817 (LISTEN)\n",
"java 17261 root 357u IPv4 391184 0t0 TCP 127.0.0.1:41631 (LISTEN)\n",
"java 17261 root 367u IPv4 390801 0t0 TCP 127.0.0.1:34651 (LISTEN)\n",
"java 17261 root 370u IPv4 390804 0t0 TCP 127.0.0.1:35015 (LISTEN)\n",
"java 17261 root 399u IPv4 390814 0t0 TCP 127.0.0.1:46503 (LISTEN)\n",
"java 17261 root 400u IPv4 390817 0t0 TCP 127.0.0.1:44665 (LISTEN)\n",
"java 17261 root 423u IPv4 401418 0t0 TCP *:8031 (LISTEN)\n",
"java 17261 root 440u IPv4 395796 0t0 TCP *:10033 (LISTEN)\n",
"java 17261 root 450u IPv4 400174 0t0 TCP *:19888 (LISTEN)\n",
"java 17261 root 455u IPv4 396244 0t0 TCP 127.0.0.1:43877 (LISTEN)\n",
"java 17261 root 465u IPv4 400367 0t0 TCP *:8088 (LISTEN)\n",
"java 17261 root 470u IPv4 400420 0t0 TCP *:8033 (LISTEN)\n",
"java 17261 root 490u IPv4 401422 0t0 TCP *:8030 (LISTEN)\n",
"java 17261 root 500u IPv4 400426 0t0 TCP 127.0.0.1:37335 (LISTEN)\n",
"java 17261 root 510u IPv4 400470 0t0 TCP 127.0.0.1:42337 (LISTEN)\n",
"java 17261 root 520u IPv4 401450 0t0 TCP 127.0.0.1:40401 (LISTEN)\n",
"java 17261 root 530u IPv4 401454 0t0 TCP *:42359 (LISTEN)\n",
"java 17261 root 531u IPv4 401457 0t0 TCP 127.0.0.1:38543 (LISTEN)\n",
"```\n",
"\n",
"The `java` process is the one responsible for providing the MiniCluster services by listening on several ports.\n",
"\n",
"There are two known ports for the Web interfaces (see [https://hadoop.apache.org/docs/.../ClusterSetup.html#Web_Interfaces](https://hadoop.apache.org/docs/r3.0.1/hadoop-project-dist/hadoop-common/ClusterSetup.html#Web_Interfaces)):\n",
"* $8088$ the port to the ResourceManager\n",
"* $19888$ the port to the MapReduce JobHistory Server\n",
"\n"
],
"metadata": {
"id": "crP9O3O9DfUT"
}
},
{
"cell_type": "markdown",
"source": [
"### Access the Web UI"
],
"metadata": {
"id": "rHb_H6-rt2NQ"
}
},
{
"cell_type": "markdown",
"source": [
"Let us check the Web interface at port $8088$."
],
"metadata": {
"id": "vUo-K1HmKTIv"
}
},
{
"cell_type": "code",
"source": [
"!wget http://localhost:8088"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "GOKETlOVKTRg",
"outputId": "c3c0a621-d346-4841-8c14-9697c1dd1f84"
},
"execution_count": 22,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"--2024-08-04 19:10:41-- http://localhost:8088/\n",
"Resolving localhost (localhost)... 127.0.0.1, ::1\n",
"Connecting to localhost (localhost)|127.0.0.1|:8088... failed: Connection refused.\n",
"Connecting to localhost (localhost)|::1|:8088... failed: Cannot assign requested address.\n",
"Retrying.\n",
"\n",
"--2024-08-04 19:10:42-- (try: 2) http://localhost:8088/\n",
"Connecting to localhost (localhost)|127.0.0.1|:8088... failed: Connection refused.\n",
"Connecting to localhost (localhost)|::1|:8088... failed: Cannot assign requested address.\n",
"Retrying.\n",
"\n",
"--2024-08-04 19:10:44-- (try: 3) http://localhost:8088/\n",
"Connecting to localhost (localhost)|127.0.0.1|:8088... failed: Connection refused.\n",
"Connecting to localhost (localhost)|::1|:8088... failed: Cannot assign requested address.\n",
"Retrying.\n",
"\n",
"--2024-08-04 19:10:47-- (try: 4) http://localhost:8088/\n",
"Connecting to localhost (localhost)|127.0.0.1|:8088... connected.\n",
"HTTP request sent, awaiting response... 302 Found\n",
"Location: http://localhost:8088/cluster [following]\n",
"--2024-08-04 19:10:47-- http://localhost:8088/cluster\n",
"Reusing existing connection to localhost:8088.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 14065 (14K) [text/html]\n",
"Saving to: ‘index.html’\n",
"\n",
"index.html 100%[===================>] 13.74K --.-KB/s in 0.001s \n",
"\n",
"2024-08-04 19:10:48 (24.8 MB/s) - ‘index.html’ saved [14065/14065]\n",
"\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"We can serve the ResourceManager UI in the browser through Google Colab."
],
"metadata": {
"id": "jO6uA0_aLbX_"
}
},
{
"cell_type": "code",
"source": [
"if IN_COLAB:\n",
" # serve the Web UI on Colab\n",
" print(\"Click on the link below to open the Resource Manager Web UI 🚀\")\n",
" output.serve_kernel_port_as_window(8088, path='/node')"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"id": "4a1mpPpYL25c",
"outputId": "2f72432f-1ab6-41d4-a3ac-39d72dd23e0f"
},
"execution_count": 23,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Click on the link below to open the Resource Manager Web UI 🚀\n"
]
},
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"application/javascript": [
"(async (port, path, text, element) => {\n",
" if (!google.colab.kernel.accessAllowed) {\n",
" return;\n",
" }\n",
" element.appendChild(document.createTextNode(''));\n",
" const url = await google.colab.kernel.proxyPort(port);\n",
" const anchor = document.createElement('a');\n",
" anchor.href = new URL(path, url).toString();\n",
" anchor.target = '_blank';\n",
" anchor.setAttribute('data-href', url + path);\n",
" anchor.textContent = text;\n",
" element.appendChild(anchor);\n",
" })(8088, \"/node\", \"https://localhost:8088/node\", window.element)"
]
},
"metadata": {}
}
]
},
{
"cell_type": "markdown",
"source": [
"The port $19888$ is redirected to the same page as port $8088$, so it won't be very useful. I'm not sure if this is due to a missing configuration parameter or if it's a bug."
],
"metadata": {
"id": "zoL28RXvRyZV"
}
},
{
"cell_type": "code",
"source": [
"!wget http://localhost:19888"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "Ll29MYBUSAhg",
"outputId": "efcf5335-4eae-40df-b101-603983590c27"
},
"execution_count": 24,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"--2024-08-04 19:10:48-- http://localhost:19888/\n",
"Resolving localhost (localhost)... 127.0.0.1, ::1\n",
"Connecting to localhost (localhost)|127.0.0.1|:19888... connected.\n",
"HTTP request sent, awaiting response... 302 Found\n",
"Location: http://localhost:19888/cluster [following]\n",
"--2024-08-04 19:10:48-- http://localhost:19888/cluster\n",
"Reusing existing connection to localhost:19888.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 14065 (14K) [text/html]\n",
"Saving to: ‘index.html.1’\n",
"\n",
"\rindex.html.1 0%[ ] 0 --.-KB/s \rindex.html.1 100%[===================>] 13.74K --.-KB/s in 0s \n",
"\n",
"2024-08-04 19:10:48 (104 MB/s) - ‘index.html.1’ saved [14065/14065]\n",
"\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"if IN_COLAB:\n",
" # serve the Web UI on Colab\n",
" print(\"Click on the link below to open the MapReduce JobHistory Server Web UI 🚀\")\n",
" output.serve_kernel_port_as_window(19888, path='/node')"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/",
"height": 52
},
"id": "XwxSD_Ez4mA5",
"outputId": "2ceef797-d3a6-4da5-a55c-4d223815dbe9"
},
"execution_count": 25,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Click on the link below to open the MapReduce JobHistory Server Web UI 🚀\n"
]
},
{
"output_type": "display_data",
"data": {
"text/plain": [
""
],
"application/javascript": [
"(async (port, path, text, element) => {\n",
" if (!google.colab.kernel.accessAllowed) {\n",
" return;\n",
" }\n",
" element.appendChild(document.createTextNode(''));\n",
" const url = await google.colab.kernel.proxyPort(port);\n",
" const anchor = document.createElement('a');\n",
" anchor.href = new URL(path, url).toString();\n",
" anchor.target = '_blank';\n",
" anchor.setAttribute('data-href', url + path);\n",
" anchor.textContent = text;\n",
" element.appendChild(anchor);\n",
" })(19888, \"/node\", \"https://localhost:19888/node\", window.element)"
]
},
"metadata": {}
}
]
},
{
"cell_type": "markdown",
"source": [
"### Use ngrok to access the Web UI"
],
"metadata": {
"id": "qq0i7cnLuSvm"
}
},
{
"cell_type": "markdown",
"source": [
"In the free tier of Google Colab this functionality might not be available (see https://research.google.com/colaboratory/faq.html#limitations-and-restrictions). As an alternative, you can use [ngrok](https://ngrok.com/) after signing up for a free account.\n",
"\n",
"Check the NGROK box below if you want to use ngrok.\n",
"\n"
],
"metadata": {
"id": "z6c_QA-fsJib"
}
},
{
"cell_type": "code",
"source": [
"# you should set this to True\n",
"NGROK = False #@param {type:\"boolean\"}"
],
"metadata": {
"id": "jD7pB1dHGaMB"
},
"execution_count": 26,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"We are going to use the Python ngrok client `pyngrok` (see the [Colab example](https://pyngrok.readthedocs.io/en/latest/integrations.html#colab-http-example))."
],
"metadata": {
"id": "24l7XugFGsfQ"
}
},
{
"cell_type": "code",
"source": [
"if NGROK:\n",
" !pip install pyngrok\n",
" from pyngrok import ngrok, conf\n",
" import getpass\n",
"\n",
" print(\"Enter your authtoken, which can be copied from https://dashboard.ngrok.com/get-started/your-authtoken\")\n",
" conf.get_default().auth_token = getpass.getpass()"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "Ed-77ALaqlel",
"outputId": "9d6e8bda-bd81-4f79-f324-d10e9a7d9ba1"
},
"execution_count": 27,
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Collecting pyngrok\n",
" Downloading pyngrok-7.2.0-py3-none-any.whl.metadata (7.4 kB)\n",
"Requirement already satisfied: PyYAML>=5.1 in /usr/local/lib/python3.10/dist-packages (from pyngrok) (6.0.1)\n",
"Downloading pyngrok-7.2.0-py3-none-any.whl (22 kB)\n",
"Installing collected packages: pyngrok\n",
"Successfully installed pyngrok-7.2.0\n",
"Enter your authtoken, which can be copied from https://dashboard.ngrok.com/get-started/your-authtoken\n",
"··········\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"After entering the ngrok authorization token, you can open a connection."
],
"metadata": {
"id": "mI2SrsWBrOeI"
}
},
{
"cell_type": "code",
"source": [
"if NGROK:\n",
" # Open a ngrok tunnel to the HTTP server\n",
" public_url = ngrok.connect(19888).public_url"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "xvvmU6ZArDz_",
"outputId": "5e3d79f5-5e51-4861-c9e9-6a4cb921af9d"
},
"execution_count": 28,
"outputs": [
{
"output_type": "stream",
"name": "stderr",
"text": [
"04-Aug-24 07:12:03 PM - INFO: Opening tunnel named: http-19888-9ba3fee7-3b4b-40cb-ad20-2d842c826538\n"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": []
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"04-Aug-24 07:12:04 PM - INFO: Overriding default auth token\n",
"04-Aug-24 07:12:04 PM - INFO: t=2024-08-04T19:12:04+0000 lvl=info msg=\"no configuration paths supplied\"\n",
"04-Aug-24 07:12:04 PM - INFO: t=2024-08-04T19:12:04+0000 lvl=info msg=\"using configuration at default config path\" path=/root/.config/ngrok/ngrok.yml\n",
"04-Aug-24 07:12:04 PM - INFO: t=2024-08-04T19:12:04+0000 lvl=info msg=\"open config file\" path=/root/.config/ngrok/ngrok.yml err=nil\n",
"04-Aug-24 07:12:04 PM - INFO: t=2024-08-04T19:12:04+0000 lvl=info msg=\"starting web service\" obj=web addr=127.0.0.1:4040 allow_hosts=[]\n",
"04-Aug-24 07:12:04 PM - INFO: t=2024-08-04T19:12:04+0000 lvl=info msg=\"client session established\" obj=tunnels.session\n",
"04-Aug-24 07:12:04 PM - INFO: t=2024-08-04T19:12:04+0000 lvl=info msg=\"tunnel session started\" obj=tunnels.session\n",
"04-Aug-24 07:12:04 PM - INFO: t=2024-08-04T19:12:04+0000 lvl=info msg=start pg=/api/tunnels id=1cc987793bc5e340\n",
"04-Aug-24 07:12:05 PM - INFO: t=2024-08-04T19:12:04+0000 lvl=info msg=end pg=/api/tunnels id=1cc987793bc5e340 status=200 dur=447.85µs\n",
"04-Aug-24 07:12:05 PM - INFO: t=2024-08-04T19:12:05+0000 lvl=info msg=start pg=/api/tunnels id=7f6c4a0097fbee1f\n",
"04-Aug-24 07:12:05 PM - INFO: t=2024-08-04T19:12:05+0000 lvl=info msg=end pg=/api/tunnels id=7f6c4a0097fbee1f status=200 dur=160.467µs\n",
"04-Aug-24 07:12:05 PM - INFO: t=2024-08-04T19:12:05+0000 lvl=info msg=start pg=/api/tunnels id=756a04f173da4cf6\n",
"04-Aug-24 07:12:05 PM - INFO: t=2024-08-04T19:12:05+0000 lvl=info msg=\"started tunnel\" obj=tunnels name=http-19888-9ba3fee7-3b4b-40cb-ad20-2d842c826538 addr=http://localhost:19888 url=https://3a1d-34-74-204-234.ngrok-free.app\n",
"04-Aug-24 07:12:05 PM - INFO: t=2024-08-04T19:12:05+0000 lvl=info msg=end pg=/api/tunnels id=756a04f173da4cf6 status=201 dur=48.830762ms\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"if NGROK:\n",
" print(f'Click on {public_url} to open the MapReduce JobHistory Server Web UI')"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "uzpDDM-xrbHu",
"outputId": "3f228a4c-655c-4dd2-a4f9-61ce2e2f68f8"
},
"execution_count": 29,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Click on https://3a1d-34-74-204-234.ngrok-free.app to open the MapReduce JobHistory Server Web UI\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"You can safely ignore the warning since we are not disclosing any confidential information and proceed with clicking on the \"Visit site\" button."
],
"metadata": {
"id": "WV2BBzEwvv_X"
}
},
{
"cell_type": "markdown",
"source": [
""
],
"metadata": {
"id": "PTN1yVvhwYUr"
}
},
{
"cell_type": "markdown",
"source": [
"## Stop the MiniCluster subprocess\n",
"\n",
"To stop the MiniCluster subprocess use `process.kill()` (remember: `process` is the variable name for the MiniCluster subprocess)."
],
"metadata": {
"id": "7UQg9PjpNJRj"
}
},
{
"cell_type": "code",
"source": [
"process.kill()"
],
"metadata": {
"id": "9_UGJUPYNWuH"
},
"execution_count": 30,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"The Java process should now be gone."
],
"metadata": {
"id": "F5nVzcd2NcgZ"
}
},
{
"cell_type": "code",
"source": [
"!lsof -n -i -P +c0 -sTCP:LISTEN"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "PGJ0Bs0ENbDZ",
"outputId": "32bf4e82-d198-43c4-d745-648b5a79d7e3"
},
"execution_count": 31,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME\n",
"node 7 root 21u IPv6 20065 0t0 TCP *:8080 (LISTEN)\n",
"kernel_manager_ 25 root 6u IPv4 19487 0t0 TCP 172.28.0.12:6000 (LISTEN)\n",
"colab-fileshim. 70 root 3u IPv4 19330 0t0 TCP 127.0.0.1:3453 (LISTEN)\n",
"jupyter-noteboo 91 root 7u IPv4 20298 0t0 TCP 172.28.0.12:9000 (LISTEN)\n",
"python3 2301 root 21u IPv4 84721 0t0 TCP 127.0.0.1:37185 (LISTEN)\n",
"python3 2340 root 3u IPv4 86086 0t0 TCP 127.0.0.1:35717 (LISTEN)\n",
"python3 2340 root 5u IPv4 86087 0t0 TCP 127.0.0.1:45663 (LISTEN)\n",
"ngrok 3698 root 6u IPv4 116605 0t0 TCP 127.0.0.1:4040 (LISTEN)\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"In case there are still some `java` processes lingering around, kill them with"
],
"metadata": {
"id": "Nc5RhrAl8o_F"
}
},
{
"cell_type": "code",
"source": [
"!pkill -f java"
],
"metadata": {
"id": "p62GHDXh1fiU"
},
"execution_count": 32,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Verify that the Java processes are gone."
],
"metadata": {
"id": "F9H_aFvWARei"
}
},
{
"cell_type": "markdown",
"source": [
"# Start the MiniCluster with your own ports\n",
"\n",
"This time we will also set the ports for various services:\n",
"\n",
" mapred minicluster -format -jhsport 8900 -nnhttpport 8901 -nnport 8902 -rmport 8903\n",
"\n",
"Ports:\n",
"\n",
"port number | description\n",
"--- | ---\n",
"8900 | JobHistoryServer port\n",
"8901 | NameNode HTTP port\n",
"8902 | NameNode port\n",
"8903 | ResourceManager port\n",
"\n",
"\n",
"**Note:** we chose these port numbers (8900, 8901, 8902, 8903) arbitrarily, you can pick other numbers as long as they do not conflict with ports that are already is use."
],
"metadata": {
"id": "Tn6kOUmvA0FS"
}
},
{
"cell_type": "code",
"source": [
"import subprocess\n",
"with open('out.txt', \"w\") as stdout_file, open('err.txt', \"w\") as stderr_file:\n",
" process = subprocess.Popen(\n",
" [\"mapred\", \"minicluster\", \"-format\", \"-jhsport\", \"8900\", \"-nnhttpport\", \"8901\", \"-nnport\", \"8902\", \"-rmport\", \"8903\"],\n",
" stdout=stdout_file,\n",
" stderr=stderr_file\n",
" )"
],
"metadata": {
"id": "fvmYd5HhvTjX"
},
"execution_count": 33,
"outputs": []
},
{
"cell_type": "code",
"source": [
"if not IN_COLAB:\n",
" time.sleep(30)\n",
"else:\n",
" time.sleep(10)"
],
"metadata": {
"id": "UzhdeackOYAa",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "f147bd51-4fcd-42fd-8070-dd0fdf605c9d"
},
"execution_count": 34,
"outputs": [
{
"output_type": "stream",
"name": "stderr",
"text": [
"04-Aug-24 07:12:10 PM - WARNING: t=2024-08-04T19:12:10+0000 lvl=warn msg=\"failed to open private leg\" id=3f1ee8ca6795 privaddr=localhost:19888 err=\"dial tcp 127.0.0.1:19888: connect: connection refused\"\n",
"04-Aug-24 07:12:10 PM - WARNING: t=2024-08-04T19:12:10+0000 lvl=warn msg=\"failed to open private leg\" id=166fe8e39b26 privaddr=localhost:19888 err=\"dial tcp 127.0.0.1:19888: connect: connection refused\"\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"List the Java ports. These should be all ports associated with the MIniCluster `java` process.\n",
"\n",
"Note: `grep \"^COMMAND\\|java\"` means \"filter out the lines that begin with the string `COMMAND` or that contain the string `java`\". This is to preserve the header line."
],
"metadata": {
"id": "kmW_0xbWPb8k"
}
},
{
"cell_type": "code",
"source": [
"!lsof -n -i -P +c0 -sTCP:LISTEN | grep \"^COMMAND\\|java\""
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "4f7d404c-2269-4dd3-acdc-35ba4775ab25",
"id": "5w_SHSeSvTjX"
},
"execution_count": 35,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME\n",
"java 3727 root 341u IPv4 119325 0t0 TCP 127.0.0.1:8901 (LISTEN)\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"You should now see\n",
"\n",
"```\n",
"COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME\n",
"java 30246 root 347u IPv4 671659 0t0 TCP 127.0.0.1:8901 (LISTEN)\n",
"java 30246 root 357u IPv4 672029 0t0 TCP 127.0.0.1:8902 (LISTEN)\n",
"java 30246 root 367u IPv4 674008 0t0 TCP 127.0.0.1:44721 (LISTEN)\n",
"java 30246 root 370u IPv4 672082 0t0 TCP 127.0.0.1:36789 (LISTEN)\n",
"java 30246 root 399u IPv4 674082 0t0 TCP 127.0.0.1:37975 (LISTEN)\n",
"java 30246 root 400u IPv4 674085 0t0 TCP 127.0.0.1:37675 (LISTEN)\n",
"java 30246 root 423u IPv4 681319 0t0 TCP *:8031 (LISTEN)\n",
"java 30246 root 440u IPv4 680955 0t0 TCP *:10033 (LISTEN)\n",
"java 30246 root 450u IPv4 681014 0t0 TCP *:19888 (LISTEN)\n",
"java 30246 root 455u IPv4 681987 0t0 TCP 127.0.0.1:8900 (LISTEN)\n",
"java 30246 root 465u IPv4 682035 0t0 TCP *:8088 (LISTEN)\n",
"java 30246 root 470u IPv4 681313 0t0 TCP *:8033 (LISTEN)\n",
"java 30246 root 490u IPv4 681325 0t0 TCP *:8030 (LISTEN)\n",
"java 30246 root 500u IPv4 682046 0t0 TCP 127.0.0.1:8903 (LISTEN)\n",
"java 30246 root 510u IPv4 681370 0t0 TCP 127.0.0.1:34521 (LISTEN)\n",
"java 30246 root 520u IPv4 682057 0t0 TCP 127.0.0.1:36981 (LISTEN)\n",
"java 30246 root 530u IPv4 681373 0t0 TCP *:46657 (LISTEN)\n",
"java 30246 root 531u IPv4 682061 0t0 TCP 127.0.0.1:39897 (LISTEN)\n",
"```\n",
"\n",
"Our ports $8900$, $8901$, $8902$, and $8903$ are included in the list."
],
"metadata": {
"id": "oIt1BO5EPRIE"
}
},
{
"cell_type": "markdown",
"source": [
"The log messages are in the file `err.txt`. The last 10 lines should look like this:\n",
"\n",
"```\n",
"2023-12-27 21:36:08,820 INFO server.MiniYARNCluster: All Node Managers connected in MiniYARNCluster\n",
"2023-12-27 21:36:08,820 INFO v2.MiniMRYarnCluster: MiniMRYARN ResourceManager address: localhost:8903\n",
"2023-12-27 21:36:08,821 INFO v2.MiniMRYarnCluster: MiniMRYARN ResourceManager web address: 0.0.0.0:8088\n",
"2023-12-27 21:36:08,821 INFO v2.MiniMRYarnCluster: MiniMRYARN HistoryServer address: localhost:8900\n",
"2023-12-27 21:36:08,822 INFO v2.MiniMRYarnCluster: MiniMRYARN HistoryServer web address: 26769af38ddc:19888\n",
"2023-12-27 21:36:08,823 INFO mapreduce.MiniHadoopClusterManager: Started MiniMRCluster\n",
"```"
],
"metadata": {
"id": "W19UpXR5NXbu"
}
},
{
"cell_type": "code",
"source": [
"!tail err.txt"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "B6K9s-A9NdBa",
"outputId": "3f270543-87f5-4468-b61d-afbf06715e63"
},
"execution_count": 36,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"2024-08-04 19:12:16,167 INFO util.GSet: capacity = 2^17 = 131072 entries\n",
"2024-08-04 19:12:16,183 INFO common.Storage: Lock on /content/target/test/data/dfs/name-0-1/in_use.lock acquired by nodename 3727@446a375b7fe4\n",
"2024-08-04 19:12:16,191 INFO common.Storage: Lock on /content/target/test/data/dfs/name-0-2/in_use.lock acquired by nodename 3727@446a375b7fe4\n",
"2024-08-04 19:12:16,195 INFO namenode.FileJournalManager: Recovering unfinalized segments in /content/target/test/data/dfs/name-0-1/current\n",
"2024-08-04 19:12:16,195 INFO namenode.FileJournalManager: Recovering unfinalized segments in /content/target/test/data/dfs/name-0-2/current\n",
"2024-08-04 19:12:16,195 INFO namenode.FSImage: No edit log streams selected.\n",
"2024-08-04 19:12:16,195 INFO namenode.FSImage: Planning to load image: FSImageFile(file=/content/target/test/data/dfs/name-0-1/current/fsimage_0000000000000000000, cpktTxId=0000000000000000000)\n",
"2024-08-04 19:12:16,348 INFO namenode.FSImageFormatPBINode: Loading 1 INodes.\n",
"2024-08-04 19:12:16,360 INFO namenode.FSImageFormatPBINode: Successfully loaded 1 inodes\n",
"2024-08-04 19:12:16,383 INFO namenode.FSImageFormatPBINode: Completed update blocks map and name cache, total waiting duration 0ms.\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Check the NameNode's HTTP port."
],
"metadata": {
"id": "OdzAzlPmcB0h"
}
},
{
"cell_type": "code",
"source": [
"!wget http://localhost:8901"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "025b4bc8-6dc4-47a4-cb88-f5814900e236",
"id": "M8S8O_4WvTjY"
},
"execution_count": 37,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"--2024-08-04 19:12:16-- http://localhost:8901/\n",
"Resolving localhost (localhost)... 127.0.0.1, ::1\n",
"Connecting to localhost (localhost)|127.0.0.1|:8901... connected.\n",
"HTTP request sent, awaiting response... 302 Found\n",
"Location: http://localhost:8901/index.html [following]\n",
"--2024-08-04 19:12:16-- http://localhost:8901/index.html\n",
"Reusing existing connection to localhost:8901.\n",
"HTTP request sent, awaiting response... 200 OK\n",
"Length: 1079 (1.1K) [text/html]\n",
"Saving to: ‘index.html.2’\n",
"\n",
"index.html.2 100%[===================>] 1.05K --.-KB/s in 0s \n",
"\n",
"2024-08-04 19:12:17 (117 MB/s) - ‘index.html.2’ saved [1079/1079]\n",
"\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## Start the NameNode Web UI\n",
"\n",
"Serve the NameNode UI in the browser through Google Colab (path should be set to `/index.html` as in the output of `wget` otherwise the URL won't work)."
],
"metadata": {
"id": "aKA3MLV5cIEX"
}
},
{
"cell_type": "code",
"source": [
"if IN_COLAB and not NGROK:\n",
" # serve the Web UI on Colab\n",
" print(\"Click on the link below to open the NameNode Web UI 🚀\")\n",
" output.serve_kernel_port_as_window(8901, path='/index.html')\n",
"else:\n",
" if NGROK:\n",
" # disconnect previous tunnels (note: you can have max 3 tunnels open!)\n",
" # see: https://pyngrok.readthedocs.io/en/latest/index.html#get-active-tunnels\n",
" tunnels = ngrok.get_tunnels()\n",
" for t in tunnels:\n",
" ngrok.disconnect(t.public_url)\n",
" # Open a ngrok tunnel to the HTTP server on port 8901\n",
" public_url = ngrok.connect(8901).public_url\n",
" print(f'Click on {public_url} to open the NameNode Web UI 🚀')"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "8f1a1bff-a2eb-45d8-92eb-fcc08da112be",
"id": "k-LdeVwBvTjY"
},
"execution_count": 38,
"outputs": [
{
"output_type": "stream",
"name": "stderr",
"text": [
"04-Aug-24 07:12:17 PM - INFO: t=2024-08-04T19:12:17+0000 lvl=info msg=start pg=/api/tunnels id=618f6ef4fc692b19\n",
"04-Aug-24 07:12:17 PM - INFO: t=2024-08-04T19:12:17+0000 lvl=info msg=end pg=/api/tunnels id=618f6ef4fc692b19 status=200 dur=265.638µs\n",
"04-Aug-24 07:12:17 PM - INFO: Disconnecting tunnel: https://3a1d-34-74-204-234.ngrok-free.app\n",
"04-Aug-24 07:12:17 PM - INFO: t=2024-08-04T19:12:17+0000 lvl=info msg=start pg=/api/tunnels/http-19888-9ba3fee7-3b4b-40cb-ad20-2d842c826538 id=26cf928d2ad02baa\n",
"04-Aug-24 07:12:17 PM - INFO: t=2024-08-04T19:12:17+0000 lvl=info msg=end pg=/api/tunnels/http-19888-9ba3fee7-3b4b-40cb-ad20-2d842c826538 id=26cf928d2ad02baa status=204 dur=36.380055ms\n",
"04-Aug-24 07:12:17 PM - INFO: t=2024-08-04T19:12:17+0000 lvl=info msg=\"failed to accept connection: Listener closed\" obj=tunnels.session clientid=cf31c5d7baff4c19c11cf349a9ee2705\n",
"04-Aug-24 07:12:17 PM - WARNING: t=2024-08-04T19:12:17+0000 lvl=warn msg=\"Stopping forwarder\" name=http-19888-9ba3fee7-3b4b-40cb-ad20-2d842c826538 acceptErr=\"failed to accept connection: Listener closed\"\n",
"04-Aug-24 07:12:17 PM - INFO: t=2024-08-04T19:12:17+0000 lvl=info msg=\"Error handling the forwarder accept error\" error=\"no tunnel found with requested name\"\n",
"04-Aug-24 07:12:17 PM - INFO: Opening tunnel named: http-8901-dbbf6775-d1b1-4b5c-ad30-fc781622caef\n",
"04-Aug-24 07:12:17 PM - INFO: t=2024-08-04T19:12:17+0000 lvl=info msg=start pg=/api/tunnels id=3d43722d46a8ba3a\n",
"04-Aug-24 07:12:17 PM - INFO: t=2024-08-04T19:12:17+0000 lvl=info msg=\"started tunnel\" obj=tunnels name=http-8901-dbbf6775-d1b1-4b5c-ad30-fc781622caef addr=http://localhost:8901 url=https://f668-34-74-204-234.ngrok-free.app\n"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Click on https://f668-34-74-204-234.ngrok-free.app to open the NameNode Web UI 🚀\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"04-Aug-24 07:12:17 PM - INFO: t=2024-08-04T19:12:17+0000 lvl=info msg=end pg=/api/tunnels id=3d43722d46a8ba3a status=201 dur=46.18765ms\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"By clicking on the above link you should see the NameNode's Web UI in your browser:\n",
"\n"
],
"metadata": {
"id": "zya4FAgnTGHz"
}
},
{
"cell_type": "markdown",
"source": [
""
],
"metadata": {
"id": "GXZifg7YhvBa"
}
},
{
"cell_type": "markdown",
"source": [
"# Test the HDFS filesystem"
],
"metadata": {
"id": "PWLoLFJabAev"
}
},
{
"cell_type": "markdown",
"source": [
"## Create a directory\n",
"\n",
"**Note:** in order to use the MiniCluster's Hadoop filesystem you need to specify the full path prepending `hdfs://localhost:8902/` otherwise `hdfs` will write to the local filesystem."
],
"metadata": {
"id": "9jA_tBoV647p"
}
},
{
"cell_type": "code",
"source": [
"%%bash\n",
"# create a folder my_dir\n",
"hdfs dfs -mkdir hdfs://localhost:8902/my_dir"
],
"metadata": {
"id": "wCYTzNvlfzRO"
},
"execution_count": 39,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"List the contents of `my_dir` (should be empty)."
],
"metadata": {
"id": "NiIBxgBi8KQh"
}
},
{
"cell_type": "code",
"source": [
"!hdfs dfs -ls hdfs://localhost:8902/my_dir"
],
"metadata": {
"id": "yATUZZ5hbEHh"
},
"execution_count": 40,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Upload the local folder `sample_data` to `my_dir` on HDFS"
],
"metadata": {
"id": "5vKshDCX8Sc2"
}
},
{
"cell_type": "code",
"source": [
"!ls -lh sample_data"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "PiVF1RFzeLXd",
"outputId": "77ff6b8e-af93-41a4-9b57-846d5b4a693c"
},
"execution_count": 41,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"total 55M\n",
"-rwxr-xr-x 1 root root 1.7K Jan 1 2000 anscombe.json\n",
"-rw-r--r-- 1 root root 295K Aug 1 13:24 california_housing_test.csv\n",
"-rw-r--r-- 1 root root 1.7M Aug 1 13:24 california_housing_train.csv\n",
"-rw-r--r-- 1 root root 18M Aug 1 13:24 mnist_test.csv\n",
"-rw-r--r-- 1 root root 35M Aug 1 13:24 mnist_train_small.csv\n",
"-rwxr-xr-x 1 root root 930 Jan 1 2000 README.md\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Check the total size of the local folder `sample_data` using the command `du` (\"du\" stands for \"disk usage\" and the `-h` option stands for \"human\" as it will format file sizes in a “human-readable” fashion, e.g 55M instead of 55508) ."
],
"metadata": {
"id": "ayePcxZ7TuXm"
}
},
{
"cell_type": "code",
"source": [
"!du -h sample_data"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "Kp119M2bTrzy",
"outputId": "b16cff16-1d95-468a-f93f-1ac802305fe5"
},
"execution_count": 42,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"55M\tsample_data\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Upload `sample_data` to HDFS."
],
"metadata": {
"id": "4zer_jX7T2ZD"
}
},
{
"cell_type": "code",
"source": [
"!hdfs dfs -put sample_data hdfs://localhost:8902/my_dir/"
],
"metadata": {
"id": "WfV62y1yeu6O"
},
"execution_count": 43,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Check"
],
"metadata": {
"id": "gP1gYl7t8Sh9"
}
},
{
"cell_type": "code",
"source": [
"!hdfs dfs -ls hdfs://localhost:8902/my_dir"
],
"metadata": {
"id": "mL8pfXzVUBS_",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "83f6dd1d-c0dd-48b2-9eb7-86a1e96e7d4e"
},
"execution_count": 44,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Found 1 items\n",
"drwxr-xr-x - root supergroup 0 2024-08-04 19:12 hdfs://localhost:8902/my_dir/sample_data\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Check the size of `my_dir` on HDFS using the [HDFS equivalent of `du`](https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/FileSystemShell.html#du)."
],
"metadata": {
"id": "7h7gL8tOlS17"
}
},
{
"cell_type": "code",
"source": [
"!hdfs dfs -du -h hdfs://localhost:8902/my_dir"
],
"metadata": {
"id": "1zCBrlOEUKpU",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "813e3030-27bc-4deb-b058-8e288fabbce6"
},
"execution_count": 45,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"54.2 M 162.6 M hdfs://localhost:8902/my_dir/sample_data\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Check the contents of the HDFS folder `my_dir`"
],
"metadata": {
"id": "IQDzG-TOEB9a"
}
},
{
"cell_type": "code",
"source": [
"!hdfs dfs -ls -R -h hdfs://localhost:8902/my_dir"
],
"metadata": {
"id": "U1XOv7NZhY9F",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "91925578-7e05-4296-a538-2f2842d1bb7a"
},
"execution_count": 46,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"drwxr-xr-x - root supergroup 0 2024-08-04 19:12 hdfs://localhost:8902/my_dir/sample_data\n",
"-rw-r--r-- 3 root supergroup 930 2024-08-04 19:12 hdfs://localhost:8902/my_dir/sample_data/README.md\n",
"-rw-r--r-- 3 root supergroup 1.7 K 2024-08-04 19:12 hdfs://localhost:8902/my_dir/sample_data/anscombe.json\n",
"-rw-r--r-- 3 root supergroup 294.1 K 2024-08-04 19:12 hdfs://localhost:8902/my_dir/sample_data/california_housing_test.csv\n",
"-rw-r--r-- 3 root supergroup 1.6 M 2024-08-04 19:12 hdfs://localhost:8902/my_dir/sample_data/california_housing_train.csv\n",
"-rw-r--r-- 3 root supergroup 17.4 M 2024-08-04 19:12 hdfs://localhost:8902/my_dir/sample_data/mnist_test.csv\n",
"-rw-r--r-- 3 root supergroup 34.8 M 2024-08-04 19:12 hdfs://localhost:8902/my_dir/sample_data/mnist_train_small.csv\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"You should now see in the Web interface that the \"DFS used\" has increased (you might need to refresh the NameNode UI Web page):"
],
"metadata": {
"id": "tuI4QmYnCyvp"
}
},
{
"cell_type": "markdown",
"source": [
"![dfs_used.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAACQ4AAAEECAYAAABj6TuGAAAMa2lDQ1BJQ0MgUHJvZmlsZQAASImVVwdYU8kWnluSkJDQAqFICb0JIr1ICaFFEJAONkISSCgxJgQVOyIquHYRxYquiii6FkAWFbGXRbH3xYLKyrpYUBSVNyEBXfeV753vmzv/PXPmP+XO3HsHAM1erkSSi2oBkCfOl8aFBzNTUtOYpOcAAVRABkygzeXJJKzY2CgAZbD/u7y/Ca2hXHNScP1z/L+KDl8g4wGAjIc4gy/j5UHcDAC+gSeR5gNAVOgtp+ZLFHguxLpSGCDEqxU4S4l3KXCGEjcN2CTEsSG+AoAalcuVZgGgcR/qmQW8LMij8RliFzFfJAZAczjEATwhlw+xIvbheXmTFbgCYjtoL4EYxgO8M77jzPobf8YQP5ebNYSVeQ2IWohIJsnlTv8/S/O/JS9XPujDBjaqUBoRp8gf1vB2zuRIBaZC3CXOiI5R1BriXhFfWXcAUIpQHpGotEeNeTI2rB9gQOzC54ZEQmwMcZg4NzpKpc/IFIVxIIarBZ0myuckQGwA8SKBLDReZbNFOjlO5Quty5SyWSr9Oa50wK/C10N5TiJLxf9GKOCo+DGNQmFCMsQUiK0KREnREGtA7CzLiY9U2YwqFLKjB22k8jhF/FYQxwnE4cFKfqwgUxoWp7IvzZMN5ottEYo40Sp8IF+YEKGsD3aKxx2IH+aCXRGIWYmDPAJZStRgLnxBSKgyd+yFQJwYr+LpleQHxynn4hRJbqzKHrcQ5IYr9BYQu8sK4lVz8aR8uDiV/HimJD82QRknXpjNHR2rjAdfDqIAG4TA3SeHLQNMBtlA1NpV3wXvlCNhgAukIAsIgJNKMzgjeWBEDK/xoBD8CZEAyIbmBQ+MCkAB1H8Z0iqvTiBzYLRgYEYOeAZxHogEufBePjBLPOQtCTyFGtE/vHNh48F4c2FTjP97/aD2m4YFNVEqjXzQI1Nz0JIYSgwhRhDDiPa4ER6A++FR8BoEmyvujfsM5vHNnvCM0EZ4TLhBaCfcmSQqkv4Q5RjQDvnDVLXI+L4WuA3k9MCDcX/IDplxBm4EnHB36IeFB0LPHlDLVsWtqArzB+6/ZfDd01DZkV3IKFmfHES2+3GmhoOGxxCLotbf10cZa8ZQvdlDIz/6Z39XfT7sI3+0xBZhB7Gz2AnsPNaE1QMmdhxrwC5hRxV4aHU9HVhdg97iBuLJgTyif/jjqnwqKilzqXHpdPmsHMsXTMtXbDz2ZMl0qShLmM9kwa+DgMkR85yHM11dXN0AUHxrlK+vt4yBbwjCuPBNV/QOAH9+f39/0zddFNzrhxbA7f/sm872GHxN6ANwrownlxYodbjiQoBvCU240wyBKbAEdjAfV+AJ/EAQCAWjQQxIAKlgIqyyEK5zKZgKZoJ5oASUgeVgDVgPNoNtYBfYCw6AetAEToAz4CK4Am6Ae3D1dICXoBu8B30IgpAQGkJHDBEzxBpxRFwRbyQACUWikDgkFUlHshAxIkdmIvORMmQlsh7ZilQjvyBHkBPIeaQNuYM8QjqRN8gnFEOpqC5qgtqgI1BvlIVGognoBDQLnYIWosXoUrQCrUL3oHXoCfQiegNtR1+iPRjA1DEGZo45Yd4YG4vB0rBMTIrNxkqxcqwKq8Ua4XO+hrVjXdhHnIjTcSbuBFdwBJ6I8/Ap+Gx8Cb4e34XX4afwa/gjvBv/SqARjAmOBF8Ch5BCyCJMJZQQygk7CIcJp+Fe6iC8JxKJDKIt0QvuxVRiNnEGcQlxI3EfsZnYRnxC7CGRSIYkR5I/KYbEJeWTSkjrSHtIx0lXSR2kXjV1NTM1V7UwtTQ1sVqRWrnabrVjalfVnqv1kbXI1mRfcgyZT55OXkbeTm4kXyZ3kPso2hRbij8lgZJNmUepoNRSTlPuU96qq6tbqPuoj1UXqc9Vr1Dfr35O/ZH6R6oO1YHKpo6nyqlLqTupzdQ71Lc0Gs2GFkRLo+XTltKqaSdpD2m9GnQNZw2OBl9jjkalRp3GVY1XmmRNa02W5kTNQs1yzYOalzW7tMhaNlpsLa7WbK1KrSNat7R6tOnaI7VjtPO0l2jv1j6v/UKHpGOjE6rD1ynW2aZzUucJHaNb0tl0Hn0+fTv9NL1Dl6hrq8vRzdYt092r26rbraej566XpDdNr1LvqF47A2PYMDiMXMYyxgHGTcYnfRN9lr5Af7F+rf5V/Q8GwwyCDAQGpQb7DG4YfDJkGoYa5hiuMKw3fGCEGzkYjTWaarTJ6LRR1zDdYX7DeMNKhx0YdtcYNXYwjjOeYbzN+JJxj4mpSbiJxGSdyUmTLlOGaZBptulq02OmnWZ0swAzkdlqs+NmfzD1mCxmLrOCeYrZbW5sHmEuN99q3mreZ2FrkWhRZLHP4oElxdLbMtNytWWLZbeVmdUYq5lWNVZ3rcnW3tZC67XWZ60/2NjaJNsstKm3eWFrYMuxLbStsb1vR7MLtJtiV2V33Z5o722fY7/R/ooD6uDhIHSodLjsiDp6OoocNzq2DScM9xkuHl41/JYT1YnlVOBU4/TImeEc5VzkXO/8aoTViLQRK0acHfHVxcMl12W7y72ROiNHjywa2TjyjauDK8+10vW6G80tzG2OW4Pba3dHd4H7JvfbHnSPMR4LPVo8vnh6eUo9az07vay80r02eN3y1vWO9V7ifc6H4BPsM8enyeejr6dvvu8B37/8nPxy/Hb7vRhlO0owavuoJ/4W/lz/rf7tAcyA9IAtAe2B5oHcwKrAx0GWQfygHUHPWfasbNYe1qtgl2Bp8OHgD2xf9ix2cwgWEh5SGtIaqhOaGLo+9GGYRVhWWE1Yd7hH+Izw5ghCRGTEiohbHBMOj1PN6R7tNXrW6FOR1Mj4yPWRj6McoqRRjWPQMaPHrBpzP9o6WhxdHwNiODGrYh7E2sZOif11LHFs7NjKsc/iRsbNjDsbT4+fFL87/n1CcMKyhHuJdonyxJYkzaTxSdVJH5JDklcmt6eMSJmVcjHVKFWU2pBGSktK25HWMy503JpxHeM9xpeMvznBdsK0CecnGk3MnXh0kuYk7qSD6YT05PTd6Z+5Mdwqbk8GJ2NDRjePzVvLe8kP4q/mdwr8BSsFzzP9M1dmvsjyz1qV1SkMFJYLu0Rs0XrR6+yI7M3ZH3Jicnbm9Ocm5+7LU8tLzzsi1hHniE9NNp08bXKbxFFSImmf4jtlzZRuaaR0hwyRTZA15OvCn/pLcjv5AvmjgoCCyoLeqUlTD07Tniaedmm6w/TF058XhhX+PAOfwZvRMtN85ryZj2axZm2djczOmN0yx3JO8ZyOueFzd82jzMuZ91uRS9HKonfzk+c3FpsUzy1+siB8QU2JRom05NZCv4WbF+GLRItaF7stXrf4aym/9EKZS1l52eclvCUXfhr5U8VP/Uszl7Yu81y2aTlxuXj5zRWBK3at1F5ZuPLJqjGr6lYzV5eufrdm0prz5e7lm9dS1srXtldEVTSss1q3fN3n9cL1NyqDK/dtMN6weMOHjfyNVzcFbardbLK5bPOnLaItt7eGb62rsqkq30bcVrDt2fak7Wd/9v65eofRjrIdX3aKd7bvitt1qtqrunq38e5lNWiNvKZzz/g9V/aG7G2odarduo+xr2w/2C/f/8cv6b/cPBB5oOWg98HaQ9aHNhymHy6tQ+qm13XXC+vbG1Ib2o6MPtLS6Nd4+FfnX3c2mTdVHtU7uuwY5Vjxsf7jhcd7miXNXSeyTjxpmdRy72TKyeunxp5qPR15+tyZsDMnz7LOHj/nf67pvO/5Ixe8L9Rf9LxYd8nj0uHfPH473OrZWnfZ63LDFZ8rjW2j2o5dDbx64lrItTPXOdcv3oi+0XYz8ebtW+Nvtd/m335xJ/fO67sFd/vuzb1PuF/6QOtB+UPjh1W/2/++r92z/eijkEeXHsc/vveE9+TlU9nTzx3Fz2jPyp+bPa9+4fqiqTOs88of4/7oeCl52ddV8qf2nxte2b069FfQX5e6U7o7Xktf979Z8tbw7c537u9aemJ7Hr7Pe9/3obTXsHfXR++PZz8lf3reN/Uz6XPFF/svjV8jv97vz+vvl3Cl3IFfAQw2NDMTgDc7AaClAkCH5zbKOOVZcEAQ5fl1AIH/hJXnxQHxBKAWdorfeHYzAPths5kLuYMAUPzCJwQB1M1tqKlElunmquSiwpMQobe//60JAKRGAL5I+/v7Nvb3f9kOg70DQPMU5RlUIUR4ZtgSoEA3DFK2gh9EeT79Lscfe6CIwB382P8LRN2PY8nL7lUAAACKZVhJZk1NACoAAAAIAAQBGgAFAAAAAQAAAD4BGwAFAAAAAQAAAEYBKAADAAAAAQACAACHaQAEAAAAAQAAAE4AAAAAAAAAkAAAAAEAAACQAAAAAQADkoYABwAAABIAAAB4oAIABAAAAAEAAAkOoAMABAAAAAEAAAEEAAAAAEFTQ0lJAAAAU2NyZWVuc2hvdL9VossAAAAJcEhZcwAAFiUAABYlAUlSJPAAAAHXaVRYdFhNTDpjb20uYWRvYmUueG1wAAAAAAA8eDp4bXBtZXRhIHhtbG5zOng9ImFkb2JlOm5zOm1ldGEvIiB4OnhtcHRrPSJYTVAgQ29yZSA2LjAuMCI+CiAgIDxyZGY6UkRGIHhtbG5zOnJkZj0iaHR0cDovL3d3dy53My5vcmcvMTk5OS8wMi8yMi1yZGYtc3ludGF4LW5zIyI+CiAgICAgIDxyZGY6RGVzY3JpcHRpb24gcmRmOmFib3V0PSIiCiAgICAgICAgICAgIHhtbG5zOmV4aWY9Imh0dHA6Ly9ucy5hZG9iZS5jb20vZXhpZi8xLjAvIj4KICAgICAgICAgPGV4aWY6UGl4ZWxZRGltZW5zaW9uPjI2MDwvZXhpZjpQaXhlbFlEaW1lbnNpb24+CiAgICAgICAgIDxleGlmOlBpeGVsWERpbWVuc2lvbj4yMzE4PC9leGlmOlBpeGVsWERpbWVuc2lvbj4KICAgICAgICAgPGV4aWY6VXNlckNvbW1lbnQ+U2NyZWVuc2hvdDwvZXhpZjpVc2VyQ29tbWVudD4KICAgICAgPC9yZGY6RGVzY3JpcHRpb24+CiAgIDwvcmRmOlJERj4KPC94OnhtcG1ldGE+CidKQukAAAAcaURPVAAAAAIAAAAAAAAAggAAACgAAACCAAAAggAAUN9677UQAABAAElEQVR4AezdCbxV8/7/8W8DSZEmIa66KEMUwi+Uoswumefpyg0hIVKhmzFDSDIUERmvebqEa/jF5afENaQBl4Qk6QoJ/97f67P+373O2tM5e5+zz9mv7+Nx715r7bXWXuu51unxWF/v9fnW+21FczQEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBAoK4F6BIfK6npzsggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIeAGCQ9wICCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAgiUoQDBoTK86JwyAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIEh7gHEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBAoQwGCQ2V40TllBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQIDnEPIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCBQhgIEh8rwonPKCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggQHOIeQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEECgDAUIDpXhReeUEUAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBAgOMQ9gAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIBAGQoQHCrDi84pI4AAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCBAcIh7AAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQACBMhQgOFSGF51TRgABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEECA4BD3AAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACZShAcKgMLzqnjAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIBAQYNDc+fORRQBBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQSKJPDHP/6xYHsmOFQwSnaEAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggEBxBUo+OLTuuusWV4C9I4AAAggggAACCCCAAAIIIIAAAggggAACaQQ+++wz/w39lGmAWIwAAggggAACCCCAAAIIIFArBex5l+BQrbx8HDQCCCCAAAIIIIAAAggggAACCCCAAAIIVIeAdaQSHKoObX4DAQQQQAABBBBAAAEEEECgugTseZfgUHWJ8zsIIIAAAggggAACCCCAAAIIIIAAAgggUOsErCOV4FCtu3QcMAIIIIAAAggggAACCCCAQAYBe94lOJQBia8QQAABBBBAAAEEEEAAAQQQQAABBBBAoLwFrCOV4FB53wecPQIIIIAAAggggAACCCBQ1wTseZfgUF27spwPAggggAACCCCAAAIIIIAAAggggAACCBRMwDpSCQ4VjJQdIYAAAggggAACCCCAAAIIlICAPe8SHCqBi8EhIIAAAggggAACCCCAAAIIIIAAAggggEBpClhHKsGh0rw+HBUCCCCAAAIIIIAAAggggEDlBOx5l+BQ5fzYCgEEEEAAAQQQQAABBBBAAAEEEEAAAQTKQMA6UgkOlcHF5hQRQAABBBBAAAEEEEAAgTISsOddgkNldNE5VQQQQAABBBBAAAEEEEAAAQQQQAABBBDIT8A6UgkO5efG2ggggAACCCCAAAIIIIAAAqUtYM+7BIdK+zpxdAgggAACCCCAAAIIIIAAAggggAACCCBQgwLWkUpwqAYvAj+NAAIIIIAAAggggAACCCBQcAF73iU4VHBadogAAggggAACCCCAAAIIIIAAAggggAACdUXAOlIJDtWVK8p5IIAAAggggAACCCCAAAIISMCedwkOcT8ggAACCCCAAAIIIIAAAggggAACCCCAAAJpBKwjleBQGiAWI4AAAggggAACCCCAAAII1EoBe94lOFQrLx8HjQACCCCAAAIIIIAAAggggAACCCCAAALVIWAdqQSHqkOb30AAAQQQQAABBBBAAAEEEKguAXveJThUXeL8DgIIIIAAAggggAACCCCAAAIIIIAAAgjUOgHrSCU4VOsuHQeMAAIIIIAAAggggAACCCCQQcCedwkOZUDiKwQQQAABBBBAAAEEEEAAAQQQQAABBBAobwHrSCU4VN73AWePAAIIIIAAAggggAACCNQ1AXveJThU164s54MAAggggAACCCCAAAIIIIAAAggggAACBROwjlSCQwUjZUcIIIAAAggggAACCCCAAAIlIGDPuwSHSuBicAgIIIAAAggggAACCCCAAAIIIIAAAgggUJoC1pFKcKg0rw9HhQACCCCAAAIIIIAAAgggUDkBe94lOFQ5P7ZCAAEEEEAAAQQQQAABBBBAAAEEEEAAgTIQsI5UgkNlcLE5RQQQQAABBBBAAAEEEECgjATseZfgUBlddE4VAQQQQAABBBBAAAEEEEAAAQQQQAABBPITsI5UgkP5ubE2AggggAACCCCAAAIIIIBAaQvY8y7BodK+ThwdAggggAACCCCAAAIIIIAAAggggAACCNSggHWkEhyqwYvATyOAAAIIIIAAAggggAACCBRcwJ53CQ4VnJYdIoAAAggggAACCCCAAAIIIIAAAggggEBdEbCOVIJDdeWKch4IIIAAAggggAACCCCAAAISsOddgkPcDwgggAACCCCAAAIIIIAAAggggAACCCCAQBoB60glOJQGiMUIIIAAAggggAACCCCAAAK1UsCedwkO1crLx0EjgAACCCCAAAIIIIAAAggggAACCCCAQHUIWEcqwaHq0OY3EEAAAQQQQAABBBBAAAEEqkvAnncJDlWXOL+DAAIIIIAAAggggAACCCCAAAIIIIAAArVOwDpSCQ7VukvHASOAAAIIIIAAAggggAACCGQQsOddgkMZkPgKAQQQQAABBBBAAAEEEEAAAQQQQAABBMpbwDpSCQ6V933A2SOAAAIIIIAAAggggAACdU3AnncJDtW1K8v5IIAAAggggAACCCCAAAIIIIAAAggggEDBBKwjleBQwUjZEQIIIIAAAggggAACCCCAQAkI2PNu2QaHvv76a/fWW2+5OXPmuFmzZrn58+e75s2buxYtWrgtttjCdevWzbVt27YELlVuh/DTTz+5F154wc2bN8998803TvPdu3d3O+20k5s9e7Z74oknoh2pk+OAAw6I5plIFpDb22+/HX3ZqlUr16NHj2g+34m6ds/le/5VWf+XX35xY8eOdb/99lu0m0MPPdS1adMmmmcCAQQQQAABBBBAAAEEEEAAAQQQKIaAdaQSHCqGbs3tU/3C7733nu8bfv/9992CBQt8//Caa67pNt10U9e7d++s/cOvvPKKW7x4ccaTWH311V27du3cOuus4xo0aJBxXb5EAAEEEEAAAQQQQAABBKpTwJ53yy44pODB448/7i699NKs3rvvvrsbMGCADxNlXbkGV/j888/dySef7L766quUozjqqKPcSSed5P7xj3+48847L/quc+fObty4cdE8E8kCd999txszZkz0pQJlN954YzSf60RdvOdyPfdCracgXK9evVJ2N378eN+JYws/+ugjt3TpUpt1f/jDH9xqq60WzTOBAAIIIIAAAggggAACCCCAAAIIVEbAOlILHRx655133IwZM3x4RZ9qG220kWvfvr3ba6+93IYbbpjz4aof65lnnnEbbLCBGzZsWMbtJk+e7L799tuM6+jLDh06+PBM1hVzWOHVV191Z555pl/zggsucLvttlvard544w337rvvun/9619OgZ411ljD9/N07NjR9e3b1zVr1izttrl88fPPP7tbbrnF3XnnnVlX33XXXd2QIUNco0aNEtc94ogjnPqkcm1av3///gSIcgVjPQQQQAABBBBAAAEEECiqgD3vllVw6D//+Y8bPny4++c//5kzrqoQKaCw9tpr57xNda944YUX+o6B+O8SHIqL5DdfiOBQXb3n8pOs+tq5BIf69evnO5Xs10aMGOH69Oljs3wigAACCCCAAAIIIIAAAggggAAClRKwjtRCBocefPBBd+WVV2Y8Hr0QqKBJ/fr1M67366+/uv3339+/VKhwzW233ZZx/e233z7j9/blLrvs4kaOHGmzlf5cuHChO+SQQ6IXvvSC49577524v5tvvtlNnDgx8TtbmC14ZOslff773/922n7mzJkVvtZLaPo+3vQS5qhRoxJfUMs3OKR9b7XVVu7yyy93TZo0if8U8wgggAACCCCAAAIIIIBAtQrY825ZBYdGjx7t7r///ryhVUZWb6CsssoqeW9bHRskPewr8KTgkIZzouJQ5a5CIYJDdfWeq5xo5bciOFR5O7ZEAAEEEEAAAQQQQAABBBBAAIGqCVhHaqGCQ/fee6+79tpro4NaddVV3eabb+5WWmklN23atChgoxWGDh3qqw9FKydM3H777e6mm27y32QLDn3zzTdpQzvxXRciOKRQ09lnn+1UcchauuCQzkHnYs1cPvzwQ7do0SJb7D8VjtK55tOWLVvmDj/8cKfq7daOPPJI77veeutFAa1PP/3UPfzww059g9Y22WQTN2HCBJuNPsPg0FVXXeXWWmut6DtN/PDDDz7Q9dRTT7mXX345+u4vf/mLO+aYY6J5JhBAAAEEEEAAAQQQQACBmhCw592yCQ598MEH7vjjj69grXKz+p/GmdbbLyqFqwpD8XbKKaf4N3ziy2t6XmNo77HHHimHEe9Q0Doas9uaxtXOp9SxbVdun1UNDtXVe64m7gMN96ZKYfq0pqHjwjezqDhkMnwigAACCCCAAAIIIIAAAggggEAhBawjtRDBIfU/7rPPPtHhHX300e7EE0+MQiuqXq1KRBp2TE3hmYceeiil2o2GGfvkk0/8EFnPPfece/PNN6P9ZQsOaQgw9aGoqVqz+lfSNf12VYeBj4ek9FtJwSGFeQ488MDoUAYPHuz2228/P6/wkfrZFMzR0GVqlQk1TZo0yY0bN85vr3O75JJL3Lbbbuvnk/5PQZ9zzjkn+ioprBQGh/72t7+lrVqvPi29YPjAAw/4/Wk4urvuuivaNxMIIIAAAggggAACCCCAQE0I2PNuWQSHfvnlF3fCCSdUKEGrh/Jjjz22gr/GF9dbH2HTw+Tf//73xPGn58+f74M5c+fO9Q/sCuZoPHE9AGpc8qRKRXpY1PjXevBVa9CggV9f8wpIaF8//vija9WqlV8ef4jXet99951/Q0bnFjaN0d22bVv/u/pt7efrr7+OVtGY3K1bt47mwwk9pOstno8//thvr/NQ+Vwd7+zZs6NVGzZsmBI+0nY6HmstWrRwa665ps36T40fHgaYtFCdGfXq1fPff/nllylvD+kc1Dmhfb/++us+2NWrVy8Xv2nVWTJr1ix/fHr7SBWitF8d+8orr+z3ne7/dG+oBLHOed68ed5Nb3hpH1UJDhXznlu6dKnvHNJx2x+y7hN1XnXp0iXxHpXRF198ETG0bNnS3wNLlizx95vsdV/oftX49Y0bN47WTZrQ22n6fb2BJTetr2Ow7ZO2CZd9//33/h7T/aDtNVa9hgPUdUs3LKBKSIfBIf196ZjVobZ8+XJ3xhlnpPyNn3XWWW7nnXf294DMwr8BHYvOM6nUtzqiwqY3xXR8NAQQQAABBBBAAAEEEEAAAQQQKE8B638pRHBI4RIFYNQUfvnrX/8a9Y2ZrvryVEX8q6++8ou0frdu3exrpz6PqVOnRvPhRLbg0LPPPuuH6tI2CrGoD6xYTf1tSX2vScEhVXu/4YYb/KGoCtDJJ59c4bDCcJGO20I4FVZMWCBLCyLpa7n37t07Yc3URWPHjo0CPromp512WsoKuQaHtFH8JcN01zDlB5hBAAEEEEAAAQQQQAABBIooYM+78QxGVX6y3or/qP//y4FUZU8rtlVwRq0QD+RJQSAFcW688Ub/G0n/N2zYMPf888+nfKW3QBRWsKZwiErUZhp3Ww+xF110kdt4441tM/+psIMqHYVND7uDBg1KO5a2HqpVNldt+vTpTlWQMrW99trLlzOOD1Wm8brvueeelE01FNSll14avc0UfqlSvP3793enn356tFhBqilTpkTzOjb9jjW9IaRzCZuCR3qLKmwKY9mbSxrb+5FHHom+VuUkBT70No61yy67zPXo0cPPKjylN66ss8XWCT/V0aBKU0nhLQVJdJ3ffvvtcBM/3bdvX99xos4Ba9nuGVtPn8W65/Smk94Gk0tS03U57rjjKlTHCjultN1BBx3kOnXq5MdTj+9L+9D49WGHlP2W7nm9naXx5tM13V+6V5K21za65jqHdE1/FwruhQGiTEOV6V5Tx1G6ttlmm/mOuOuuuy5lFQXs9F3YFIQ65JBDwkXu4osvdgqsqSkkGJbM1jJ5t2nTRpM0BBBAAAEEEEAAAQQQQAABBBCogwLWkVqIfkr1bT366KNe6YILLnC77bZbotjVV18dBWMUolEfl7WqBIdUNUd9ImrqZ9LLjMVoGqJLfSZ68ax58+Z+iDDrZ0sKDoUu8aBUeHzbb799NPu///u/FUJX0ZexCfUv3n///X6p+jplkPRCWWwz/+KbhjdT03k88cQTKavkExxSf7ddx6R9peyYGQQQQAABBBBAAAEEEECgGgTsebcsgkN6oNN//A9bGEAJl9v0xysq7jz99NM26z/33nvvKMikSi0qmTtjxoyUddLNxIcPSwoOKXChh+l0TcOLKahk450PGDAg3ap+ea7BIVV/OfPMMxMDNOl+oDqCQ+oIiL95Y9dNARb5h+OjpztWBVhGjRqV0hGiak/yi4+Pnm4fWp5PcKgY99ytt96aOIxe0jFrjPSwalY8OKTOiWznrvBPGKJRUEvLwvLXSb9ty1RWO+zMUa5QJaDjHSy2fvipvwUNGdi0aVO/uKrBIYX3FAYLmwJl8WpdCqJdccUV4WpOJb+tAlNYzttW0nWJBwPtOz4RQAABBBBAAAEEEEAAAQQQQKD2C1hHaiGCQ6qYrGrjaqqyk65zdsyYMb4attZTH4/6eqwtXrzYKZhjbcGCBVE/ULaKQ6q0oz7PYg+VFQaB9DKXql1b/2xScEjrmIsqDzVr1sxOL/rUPqx/J9+KQ3op0l4eVN/PDjvsEO23KhP5BIfCqkrdu3f3L/RV5bfZFgEEEEAAAQQQQAABBBCoqoA976Z7Nq3M/ku24pAqC91xxx0p5/Tiiy/6AE7Kwjxmwof3XDcLx7lOCg7lsh97E0kPunrgzdT23XdfPw53topDsslUfSnpN6ojOJT0uxYc0ptZms61nXrqqe6www6LVg87aaKFWSbyCQ4V+p7TMGP7779/hSPUdYhXDLKVdN1tqLZ4cMjWyfapII1V01EFLlVoird0x7D11ls7/Z1Yi48Lb8vTfVrwTd9nCg6pTHSmwJ1dt5NOOikl6JfUQaYw2iuvvBId0u677+7OP//8aJ7gUETBBAIIIIAAAggggAACCCCAAAJlI2AdqYUIDr311ltOQ8Cr7bTTTikvuhmoXphT1W69+KaWLegSDsOVLTj05z//2b3//vv+Za+uXbv6vhLNq69SLy1qe1Wqrsq5vvDCC74Kuo5dwRpVTQ9fsksKDmndTE3Dt6mPxvptdIzq38u1aVgy60Mr5BBt2YJDCnh98skn7rXXXkup4K3KT7KmIYAAAggggAACCCCAAAI1KWDPu2URHBoyZIhTUMhaVUvBhuNp2z71louGX9IDnx609QCqSiRhC0MI6YJDKlerykZNmjTxw3ap6krYjjrqKKcAhLWFCxe6ffbZx2b95+OPP+5atGgRLcsUHNID85/+9Kfowdk26tevn9tqq618VRqFYOLBjOoODqnjYq211vLjomu4Nr1dZA/7OmaFQ4YPH+5atmzpOxDkFh6zro/KEderVy9xGDHdExqjvHXr1k5ljjVkWrh/+41cA1aFvud0TVWtJ2w21JY6TjT8l4Z6C1tYCSddcEidGwr46H7UUHwzZ84Md+HC8I69kWYr6JqodLTMvvvuO6dy04899ph97T+t5PXy5ct9KebwmmgFOek+07BxGh4tHHJM99gzzzzjy0ZnCg6pkpH+p7fvFOyxppBdnz59/DXXdU8Kmz344IP+vtI2y5Ytcz179rTN/We8NDbBoRQeZhBAAAEEEEAAAQQQQAABBBAoCwHrSK1KmCYfqHHjxvmh4m2bJ5980q2xxho2W+Ezn+BQGKCpsKNggV5YVD+k+lTyaaoKpP4m9aup70h9dHqxrTLBoYcffth9++23bvbs2W769OlR9WxVqpaR+vNyadrHnnvuGa360ksvuYYNG0bzNqE+NvVhZWqqSh0O7xYGh9SX1ahRo2hz9WfF+xe1jvrAVCGdhgACCCCAAAIIIIAAAgjUtIA975ZFcEhv6OgB09pmm20WjeVty/L5VLlclZYNW1iZxZaHY5HbMgUrFG5JCg7tsssu/sHR1tVn/Nj1JtKll14arVLV4NCUKVNSKqpox2effXZU9lfzOlYNWRUObVVdwSGFWlRZSEEqa/EglJarapI6I6xpCDMNvxY2C9rILx5wue+++1LepFJ4SA5hs8o14bJ00/HrVtV7buLEiSn3nMJeCjpZ09tLun/CZtWZtCwpOKQOIB2nNV1nBdfU2WQtDNnpLa533nnHvvJvjvXq1Suaf/31193AgQOjeU1Yx5Y6d/R2WdhU3We//faLFr3xxht+KLRowYoJnXeHDh0yVhyy9RV2C4NDCvIpOGQt3kmk5eeee64Pzml62rRpfvg6TavpHn/qqadSKpMp+KS/67CdddZZKfdO+B3TCCCAAAIIIIAAAggggAACCCBQ+wWsI7XYwSG9mHXNNdf4ocRMbejQof7FLptP+sw1OJTUN6KX7bp06eKWLFnih6cPgy7qWzvggAOSfjJxmSolDRgwIKr4rJfUVPFZrTLBIfUbhf1U9qNhVXdblukzfBFszTXXdAokJTUdu/qHMjUNtxb2h4XBoUzbhd+pj00vRSaFl8L1mEYAAQQQQAABBBBAAAEEii1gz7tlERyKP8BtsskmbsKECZU2VlBg6tSp0fZhJaFo4YoJhQw0jFLYFDrSw3hScEgVXfTWT9j0MKoHa2t6G0VVUKxVNTikYMbNN99su/NhCYWJ4i0cg1vfVUdwSL+hSjurrLJKyuEkHUtoopWtoyLc8Pjjj3cnnHCCr9g0Y8aM6CsFjuJD2enL+HBm+QSHCn3PRQf7+4TuH/0Rz58/382aNctpGLF4NZ9MwSHZWjWfcN+jR4/2lZnCZc8995zT21TxpipB8+bN88egkta6T8POJa1vwaGwc8j2Ex8uUNdMgS5VD7Km+12VpjJVHLJ1swWHtF58KLIdd9zRjRo1yu8i/jafhobT3zoNAQQQQAABBBBAAAEEEEAAAQTKW8A6UosVHPr11199P82VV16Z0rdy2GGH+SBOtqo/uQaH1H+jocrU1Dekl6PU32Vt8eLFTn1D6jOyds899zhV+Mmlhf2M8dBR2DeU61Bl8Rfz7Bh07Ko8rhcsc2nqB7QK7pmCQ/Fh7pP2nSk4tOuuu7pmzZqlbKa+srlz5/rh4cIvOnfu7Kt3169fP1zMNAIIIIAAAggggAACCCBQrQL2vFsWwSE9jKpKjTU9XCaFY+z7bJ/xt11OPfVUpwf5eNNDv4IJYbMKJ0nBIauuEq6v4Z/0do61QgeH4sEkDRt1/fXX289FnxqHe9CgQdF83DBufOCBB6asrw1V9SmscKNlGmJrtdVW06QfaktDhFkLQx22TJ9JFYPC79NN2zFp+K2wepJCPvFqONqHhiULA0X5BIfiHnGvdMeYafnPP//sK+Co8ybb20/aT6bgULoKSCrVrHs0bApq2T8UCqop3PP0009XCCqF29i0BYcUTtP9bU1vs2k4v1xboYJDSRW2FLpSOC0e9rKQX67HyHoIIIAAAggggAACCCCAAAIIIFA3BawjtRjBIVXDUYBHoR5rqgB9zjnnuB49etiijJ+5BodUcUjVpBVmUd9Q0vmo/0nhIqverorX8Rcjkw5G+9Uw8mrqv7ziiiv88PO2bmWCQ7atqiHJSX2kb775pi3281bRKFqYMKGX31S929rLL7+cMtyYLX/22Wf9C3I2b5/qTzKPiy66yO288872VUp/UqZKSHpRThXrFQ6zphfW9OIaDQEEEEAAAQQQQAABBBCoKQF73rU8QCGOo96KB6D/XyqkinvUmxhqSQ+w+e46HgDR9umqqNi+Naa1qrmErW3btn7Yoh122CFc7IdsUhglqcVDKieffLIfDiopOJT0cFns4JDe/tGwXtb0ZsyFF15os9FnWNJXC+NBmHhQxkI60Q5WTOQbHNpnn33ckCFDwl346XgloAorpFmgylDaX/xtpPgDv22uENPll19us/4NLN1LubRC3nMaB14hNB2nAju5tkzBoXyqZNkQb+qk0fBmH330Ua6HEFUcUjWt8NjTBdTS7bhQwaGkId3UYbPRRhu5fffdN/p5ddApIMVbXxEJEwgggAACCCCAAAIIIIAAAgiUrYB1pBain9IQFdC59dZb3e23326L/KeGkT/mmGNckyZNUpZnmsk1OJRpH+F3YcgnXR9SuL6mFS6yatjqw2rdunXKKgrr6OU0Nb2A2bNnTz+tzulcz1VmeoHz7bff9tvKSn2t2Zq6rMP+XAV42rRpk22z6Pthw4b5at9aEO9HDF9ES+rbjXby+4TCT+rvVdtuu+18haffv+IDAQQQQAABBBBAAAEEEKh2AXveLYvgkIa7uuSSS1KQrfJPysJgZtKkSU5DF4Xt7rvvduuvv37Kg7C+Vxnbo446KlzVT2vope7du6csP//8850euEslOKRgTFjlJ92wXarYpHCQtWzBoaRhnl5//XU3cOBA24X/zFRxKCl8pI3iFYdUYjh8+E/5gWCmY8eOTmGk+LoKE2l5vMWHRMun4lCh7zkNrRcfXk/XQG84derUyYea9JbS559/Hp1GpuDQ9ttvn/KGk200c+ZMd9xxx9ms/9T90apVK98REw7xpi9VOUhBLA3/p2kNBRc2qzgUHwJNJa5V6jrXVqjgkH5vxIgRvtKV/bbuMx3/yJEjbZEP9+XS8RRtwAQCCCCAAAIIIIAAAggggAACCNRZAetILVRwSC8sqo/MAjCCU//KgAEDnF5czLcVOjgUDu2V68tfvXv3ThlmLddzuOaaa9zmm2/u/u///s9voj4o9dOka0899VTUh6N+1/Clv3TbaLmqKFlVp6FDh7p0L4Em7SMeiqpMxSHb7wcffOCOP/54P6sX1xTSoiGAAAIIIIAAAggggAACNSVgz7tlERz68MMP3bHHHptirbDJ/fff7ysIpXyxYkYhBT08qmyvtTDooKDJiy++aF/54chGjRoVzdtE+CBoy6x6S6kEh8K3XOwYVRlm9dVXt1n/qRDKo48+Gi2LB4fiYYykYcbuu+8+p86AsFUmOBQ/5qTfCn8jPq1gjAIy1lSqOD48l74bPHiwe+WVV2y1vCoOFfqeiw+Pp2M+++yzo7LKSSG1TMEhdUwo3FSvXr3o/DSh4cNUHjtsMliwYIHr27dvuNiPJb/HHntEy+JVqfSFBYf0t6bwUNjiZaFV0SgM72hdlbjeYIMN/N9kr169ws3d+PHj3aabbhot69evny9bbQt0T/bp08dmo8+pU6c6haysyUKdYKpCZk1v/G288cY2yycCCCCAAAIIIIAAAggggAACCJSxgHWkFiI4pKrSqmCjl/TU1MemfikFbyrbcg0OKZyjqj0rrbSS69q1a9qfU5+NhkpTU3+Q+qCytaoEh/Qy2sEHH+x/ItuLe2+88YY7/fTT/brpXoxLOtbwRUT1806ePDmnStOqohQO1VbVikNhvxTBoaQrxTIEEEAAAQQQQAABBBCoTgF73i2L4JBghw8fnhIM0DKFBRRU0EOaNQ1lpBCQAi1hUyUVextEoQKFFsJ22223OVW0saZOAL29EgaM9J0FZUolOKRQiAIyYdNDtwxsmKakSkHx4FC8ooxMVammYcOGftd6k0qVmcLAjr4wD03Hqx+lqzgUdl5oO7V7773Xrbfeev+dWfH/8tUQVMuXL4+WKXyjTpH4sFlaIV6iOCn0la3jIvqh3ycKdc8pwBbvPNJQaDoea3pjSm9OhS1TcEjraZi6Aw44INpEZirxrM4maxaYe+2119ygQYNssf+MB39UkWvMmDEp61hwKOwUsRXkEwaPwjLYto5dl8pUHIrv3/apDjL9bhgMtO/0qc4qBaiS2nfffed0LmFTBavVVlstXMQ0AggggAACCCCAAAIIIIAAAgjUIQHrSC1EcGjatGm+spB41L+mPsWwT6sybLkGh8IX09T3oT6QpBaGbNQfpD66bE1D26vPJV179dVX3U033eS/1lBs9oKYzl1Bph49ekSbPv/8826VVVaJ5sMJ9QFee+21fpH6auPVr8N1w2kdn4YVs6b+wXh/m31nn+rfVUXqsDJUVYNDV1xxhe+H1G9069bNXXXVVfZzfCKAAAIIIIAAAggggAAC1S5gz7tlExzSEE5JD7l6QNeDavv27d0XX3zhQwHhcE+6MlpHwyqpVK7aokWLKpSz1TpnnHGGL6X7/fff+yCLHnLDpvG7NQ63WqkEh1SpRg/NNga5Ha8qMnXp0sWHK8KqO/a9znfKlCk2632uu+66aF4TKrGsyjgKW6i6zZtvvpnyvWYqExyS/0EHHZQS/FDARR0FKm2saj/qRFBHTNgmTpzoOnTo4MI3k+x7na/KQWv89Y8//tgHYOLBknyDQ4W655YtWxaN+27Hq6HxFMTS9VOlHw3FF7+GWmbjxWuM9aSOCP1NqJNCgTkNhaZOlLCpbLbe+Jo+fbo75ZRTwq/8/mxbdf7obbl403Vv0aKF37+Gg4ub6o06VfbRNdGbXrq21sJh8yoTHFLVsP79+/u/38aNG9tu/adCZQ8++GDKMpsJQ4K2zD6TqipRnch0+EQAAQQQQAABBBBAAAEEEECgbgpYR2ohgkPhC4nqn9ALfFVtuQaHwtCK+oTUlxmvRv3ee++lhHFURTocPk19OL/99ps/ZPUdrrzyyjkdvvpJre/ovPPOc3vvvXfKduFQYieeeGKF6vFa+csvv/T9mNa/pPPRC125Nr2Id8cdd0SrZwpFqTL2DTfc4F+MjDZYMVHZ4JBCSLfffrtTNXpruYSXbF0+EUAAAQQQQAABBBBAAIFiCNjzbtkEh4SoBzO9xZNvu/766311onC7O++80z88hssyTStoo/BGs2bN/GqlEhzSwSRV8Ml0LvouHhyaO3eur1aTbbv495UJDmkfCi2df/758d2lndcbRHoYt6ZqOwq75NPyDQ5p34W65xTuiod6VNVJgRrrLImfSzhUV7rgUHybcF77V7imUaNGiWE5ravAVjywFO7DKgZpWb7HoLfHttlmG7+7XIJDSRWetPFmm22W0imjZTNmzPDBK03Hmyonrb/++vHFfp7gUCILCxFAAAEEEEAAAQQQQAABBBCo0wLWkVqI4JBeXLOX3fTSUzy4kwSpoditjyTp+1yDQ2G1I+1n11139UOR6YXKb7/91mkoM4WZrKkytSruWNMLbN27d7dZd99997lcTbIFh+IVzzVEmvrztH+9pKljV5DH+sHUTzd27FjXoEGD6HiyTejFOb3YGVbb1ktxW265pdtggw38C4Xz5893c+bMcer7td8K95spOKSXKONVqVUNXS8Xzp49O2V/qoSvlzCt4nv4G0wjgAACCCCAAAIIIIAAAtUlYM+7ZRUcEq4qoKgSS65tyJAhTpVS4k1VYFTB5bHHHot/VWFeIRuV+A0f8EspOKS3hBTquPrqqyscuy1Q5Robe13L4sEhLVPFGv0vXVMFGT0kh62ywSHtIyybHO4zPq0OAIVomjZtGn2lPwBV04lXl4pWWDGx9dZbp1RJqkxwSPsrxD33wgsv+KHvwuMLp3U9LOBjy/fdd99oPPp8QzvaxzXXXOO23XZb252/P9IN4aWVkkJEYXhJfzP6e8olsBX/u8slOJQuTJYUHNJbXvq7Disc6RzCKkeajzeCQ3ER5hFAAAEEEEAAAQQQQAABBBCo+wLWkZprSCadiIbyUrgk33baaae5Qw89NO1muQaHtIO77rrLB27S7uz3LxRsueyyy1L604oZHNLP6iXBsMJ5umNUP5gqB6Ubai3ddlquvsmLL77YzZw5M9Nq0XdyUMVsVeRWyxQcijbKMqHwkqpka4g2GgIIIIAAAggggAACCCBQkwL2vFt2wSGhf/rppz48pKoj6Zoe4hUsadOmTbpV/PJnn33WXX755SlvjIQbKHihMrw2zJl9pzdcdtllF5v1nw8//LDTkFlhGzdunJs0aVK0aMcdd3SjRo2K5vU20J577hnNa+LJJ590a6yxRrQsXlEoXTji/fffdyo/rJLEVkWmY8eOTsNiKXQjD2uqRvPEE0/YbPSp4JCq1MQDGSo3fMghh/g3maKVV0zIr0mTJn5RfPgorX/66aeHq1eY1ttICjzZ8cZXUGUhvaGU9PaOhlBTFaKpU6fGN3PaTmOsh+fcuXNnp+tRmVaIe07hH/1+/G0ndWAMHjzYd6yMHz8+5fDsza94cEj3kYIz6gCKXytdc7no/MP2448/OlXfShriS2/Iyeqss87y1Xxsu/h9osCOhv3TfpKa1h85cmSFCl9JHWvxIcJ0fHobTPsPjZKCQ/rt+N+WlmkoQXXepGsffPCBO/7441O+tiHwUhYygwACCCCAAAIIIIAAAggggAACdUbAOlKrGhxSaOXoo4/O2yXTkFra2cKFC6MXHzfZZJOML/ZpfYVzNGxXuhfqVGXo8MMPr9CfFg8O6QWzXMM7eiFRQ5SpDR061KkvKd70cqOqV4dVj+LraIg1DTO/+uqrx7/KeV7noX7YpH4224n6aNU3qcCQqhrpuun66UXGMPxly227pE/1d+m6qKqRXizt2rVr0mosQwABBBBAAAEEEEAAAQSqXcCed8syOGTaChoocKIhoATSsmVLXzVFlVPiQR/bJulTYQiVsf3kk0/8/1SSViV+NdxRWOUmadtSXaYyugrbWOBGD9NhYEnhkkzDvmnMcXkowKROFb0JVKwmf3V0fPzxx/5TD/bt2rXzv9uwYcOsP6vOFb1ltHjxYn/d9BBfrDd+qnrPKeykY9WbZK1bt/bDcFnwKtOJJgWHdD11PArDfP311/5e3WijjfzfQaZ9yVolm3Uscta9kItzuE9z0D2ie0V/K/ptBfVyKdEd7is+rX3rfPSpjh39Dcoq3lR+W2/shS0pvBd+zzQCCCCAAAIIIIAAAggggAACCJSfgHWkVjU4VEpyCs/MmjXL94mqb2zttdf2/aJt27YtWr9YruevSu3qs9WLeAsWLHAK3she/1P/baGaDNSnqz5F/U/9W+oTloWCPtYvWqjfYz8IIIAAAggggAACCCCAQKkJ2PNuWQeHSu2i1MTxqLrQ5MmTU35aFVdUpcWahonSm0aqSGRN45mr0hKtdgikCw7VjqMvzlGOHTvWl+e2vW+33XZu9OjRNssnAggggAACCCCAAAIIIIAAAggg4AWsI7UuBYe4tAgggAACCCCAAAIIIIAAAgjY8y7BoTK/F7744gu3//77pyioxLCCQnqDR5VtHnnkETdt2rSUddKVFE5ZiZmSESA49N9LoZLSGqJw7ty5vvR1eIEuueQS17Nnz3AR0wgggAACCCCAAAIIIIAAAggggICvyiMGgkPcDAgggAACCCCAAAIIIIAAAnVJgOBQXbqaVTyXAQMGVAgGZdqlgkWTJk1yjRs3zrQa35WQAMGh/16Mu+66y6nSULxpeLsHHngg7yHX4vthHgEEEEAAAQQQQAABBBBAAAEE6p6AdaQSHKp715YzQgABBBBAAAEEEEAAAQTKWcCed6k4VM53we/nvmTJEjdw4MCUocjSsWiIsuHDh7umTZumW4XlJShAcOi/FyVdcGjEiBGuT58+JXjlOCQEEEAAAQQQQAABBBBAAAEEEKhpAetIJThU01eC30cAAQQQQAABBBBAAAEEECikgD3vEhwqpGot3tfy5cvd1KlT3aOPPuoDRIsWLfJns+qqq7oNN9zQtW/f3nXq1Mntsccern79+rX4TMvz0KdMmeLGjx8fnfy2227rBg0aFM2Xy8Tdd9/txowZ40/X7u0jjjjCKRBHQwABBBBAAAEEEEAAAQQQQAABBJIErCOV4FCSDssQQAABBBBAAAEEEEAAAQRqq4A97xIcqq1XsMjH/csvv7ilS5e61VZbrci/xO4RqF4B3dtqDRo0qN4f5tcQQAABBBBAAAEEEEAAAQQQQKBWClhHKsGhWnn5OGgEEEAAAQQQQAABBBBAAIE0Ava8S3AoDRCLEUAAAQQQQAABBBBAAAEEEEAAAQQQQAAB60glOMS9gAACCCCAAAIIIIAAAgggUJcE7HmX4FBduqqcCwIIIIAAAggggAACCCCAAAIIIIAAAggUVMA6UgkOFZSVnSGAAAIIIIAAAggggAACCNSwgD3vEhyq4QvBzyOAAAIIIIAAAggggAACCCCAAAIIIIBA6QpYRyrBodK9RhwZAggggAACCCCAAAIIIIBA/gL2vEtwKH87tkAAAQQQQAABBBBAAAEEEEAAAQQQQACBMhGwjlSCQ2VywTlNBBBAAAEEEEAAAQQQQKBMBOx5l+BQmVxwThMBBBBAAAEEEEAAAQQQQAABBBBAAAEE8hewjlSCQ/nbsQUCCCCAAAIIIIAAAggggEDpCtjzLsGh0r1GHBkCCCCAAAIIIIAAAggggAACCCCAAAII1LCAdaQSHKrhC8HPI4AAAggggAACCCCAAAIIFFTAnncJDhWUlZ0hgAACCCCAAAIIIIAAAggggAACCCCAQF0SsI5UgkN16apyLggggAACCCCAAAIIIIAAAva8S3CIewEBBBBAAAEEEEAAAQQQQAABBBBAAAEEEEgjYB2pBIfSALEYAQQQQAABBBBAAAEEEECgVgrY8y7BoVp5+ThoBBBAAAEEEEAAAQQQQAABBBBAAAEEEKgOAetIJThUHdr8BgIIIIAAAggggAACCCCAQHUJ2PMuwaHqEud3EEAAAQQQQAABBBBAAAEEEEAAAQQQQKDWCVhHKsGhWnfpOGAEEEAAAQQQQAABBBBAAIEMAva8S3AoAxJfIYAAAggggAACCCCAAAIIIIAAAggggEB5C1hHKsGh8r4POHsEEEAAAQQQQAABBBBAoK4J2PMuwaG6dmU5HwQQQAABBBBAAAEEEEAAAQQQQAABBBAomIB1pBIcKhgpO0IAAQQQQAABBBBAAAEEECgBAXveJThUAheDQ0AAAQQQQAABBBBAAAEEEEAAAQQQQACB0hSwjlSCQ6V5fTgqBBBAAAEEEEAAAQQQQACBygnY8y7Bocr5sRUCCCCAAAIIIIAAAggggAACCCCAAAIIlIGAdaQSHCqDi80pIoAAAggggAACCCCAAAJlJGDPuwSHyuiic6oIIIAAAggggAACCCCAAAIIIIAAAgggkJ+AdaQSHMrPjbURQAABBBBAAAEEEEAAAQRKW8CedwkOlfZ14ugQQAABBBBAAAEEEEAAAQQQQAABBBBAoAYFrCOV4FANXgR+GgEEEEAAAQQQQAABBBBAoOAC9rxLcKjgtOwQAQQQQAABBBBAAAEEEEAAAQQQQAABBOqKgHWkEhyqK1eU80AAAQQQQAABBBBAAAEEEJCAPe8SHOJ+QAABBBBAAAEEEEAAAQQQQAABBBBAAAEE0ghYRyrBoTRALEYAAQQQQAABBBBAAAEEEKiVAva8S3CoVl4+DhoBBBBAAAEEEEAAAQQQQAABBBBAAAEEqkPAOlIJDlWHNr+BAAIIIIAAAggggAACCCBQXQL2vFvywaHqAuF3EEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBAoJwGCQ+V0tTlXBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQR+Fyj54FAhD5CrjgACCCCAAAIIIIAAAggggAACCCCAAAII5CMwd+5cvzr9lPmosS4CCCCAAAIIIIAAAggggECpCxTjebfebytaoU68GAdYqGNjPwgggAACCCCAAAIIIIAAAggggAACCCBQHgL0U5bHdeYsEUAAAQQQQAABBBBAAIFyEyjG8y7BoXK7izhfBBBAAAEEEEAAAQQQQAABBBBAAAEE6rhAMTpS6zgZp4cAAggggAACCCCAAAIIIFALBIrxvEtwqBZceA4RAQQQQAABBBBAAAEEEEAAAQQQQAABBHIXKEZHau6/zpoIIIAAAggggAACCCCAAAIIFEegGM+7BIeKc63YKwIIIIAAAggggAACCCCAAAIIIIAAAgjUkEAxOlJr6FT4WQQQQAABBBBAAAEEEEAAAQQigWI87xIciniZQAABBBBAAAEEEEAAAQQQQAABBBBAAIG6IFCMjtS64MI5IIAAAggggAACCCCAAAII1G6BYjzvEhyq3fcER48AAggggAACCCCAAAIIIIAAAggggAACMYFidKTGfoJZBBBAAAEEEEAAAQQQQAABBKpdoBjPuwSHqv0y8oMIIIAAAggggAACCCCAAAIIIIAAAgggUEyBYnSkFvN42TcCCCCAAAIIIIAAAggggAACuQgU43mX4FAu8qyDAAIIIIAAAggggAACCCCAAAIIIIAAArVGoBgdqbXm5DlQBBBAAAEEEEAAAQQQQACBOitQjOddgkN19nbhxBBAAAEEEEAAAQQQQAABBBBAAAEEEChPgWJ0pJanJGeNAAIIIIAAAggggAACCCBQSgLFeN4lOFRKV5hjQQABBBBAAAEEEEAAAQQQQAABBBBAAIEqCxSjI7XKB8UOEEAAAQQQQAABBBBAAAEEEKiiQDGedwkOVfGisDkCCCCAAAIIIIAAAggggAACCCCAAAIIlJZAMTpSS+sMORoEEEAAAQQQQAABBBBAAIFyFCjG8y7BoXK8kzhnBBBAAAEEEEAAAQQQQAABBBBAAAEE6rBAMTpS6zAXp4YAAggggAACCCCAAAIIIFBLBIrxvEtwqJZcfA4TAQQQQAABBBBAAAEEEEAAAQQQQAABBHITKEZHam6/zFoIIIAAAggggAACCCCAAAIIFE+gGM+7BIeKd73YMwIIIIAAAggggAACCCCAAAIIIIAAAgjUgEAxOlJr4DT4SQQQQAABBBBAAAEEEEAAAQRSBIrxvEtwKIWYGQQQQAABBBBAAAEEEEAAAQQQQAABBBCo7QLF6Eit7SYcPwIIIIAAAggggAACCCCAQO0XKMbzLsGh2n9fcAYIIIAAAggggAACCCCAAAIIIIAAAgggEAgUoyM12D2TCCCAAAIIIIAAAggggAACCNSIQDGedwkO1cil5EcRQAABBBBAAAEEEEAAAQQQQAABBBBAoFgCxehILdaxsl8EEEAAAQQQQAABBBBAAAEEchUoxvMuwaFc9VkPAQQQQAABBBBAAAEEEEAAAQQQQAABBGqFQDE6UmvFiXOQCCCAAAIIIIAAAggggAACdVqgGM+7BIfq9C3DySGAAAIIIIAAAggggAACCCCAAAIIIFB+AsXoSC0/Rc4YAQQQQAABBBBAAAEEEECg1ASK8bxLcKjUrjLHgwACCCCAAAIIIIAAAggggAACCCCAAAJVEihGR2qVDoiNEUAAAQQQQAABBBBAAAEEECiAQDGedwkOFeDCsAsEEEAAAQQQQAABBBBAAAEEEEAAAQQQKB2BYnSkls7ZcSQIIIAAAggggAACCCCAAALlKlCM512CQ+V6N3HeCCCAAAIIIIAAAggggAACCCCAAAII1FGBYnSk1lEqTgsBBBBAAAEEEEAAAQQQQKAWCRTjeZfgUC26AThUBBBAAAEEEEAAAQQQQAABBBBAAAEEEMguUIyO1Oy/yhoIIIAAAggggAACCCCAAAIIFFegGM+7BIeKe83YOwIIIIAAAggggAACCCCAAAIIIIAAAghUs0AxOlKr+RT4OQQQQAABBBBAAAEEEEAAAQQqCBTjebdWBYcWLFjgpk+f7mbPnu0+/PBD9/nnn7vmzZu7li1bui5durjtt9/erbvuuhXgSnXBTz/95J577jn32WefuYULFzrN77TTTq5Xr15u1qxZ7rHHHosOXed18MEHR/NMJAvI7a233oq+bN26tevZs2c0n23ioYcecj/88EO21VzDhg3dGmus4e+3jh07ugYNGmTdhhWqT+C3337z/1boftC/FTNnzvQ/vuaaa/prpn8rttpqK7fyyitX30HVol+67777/L9Ldsj6d2nrrbe2WT4RQAABBBBAAAEEEEAAAQQQKHmBYnSkFuuk33vvPTdnzpyMu2/UqJFr166dW3/99Z2maQgggAACCCCAAAIIIIAAAuUpUIzn3VoRHFII4NFHH3UjR47MeuX33HNPN3DgQNeiRYus69bkCvPmzXP9+vVzX331VcphHHvssW7AgAHu+eefd4MHD46+23LLLd0tt9wSzTORLHDXXXe50aNHR1927tzZTZgwIZrPNtGjRw+3dOnSbKulfK97bdSoUT68lvJFGc3oH6fvv/8+OmN1Yq2++urRfHVOfPHFF+7iiy92r776asaf1XU799xz3c4775xxvXL88s9//rObMWNGdOr6N/XII4+M5vU3EnZorrLKKm6jjTaKvmcCAQQQQAABBBBAAAEEEEAAgZoWKEZHqs5JLzLec889/iUlveTYvn17/3LSFlts4TbbbDNXv379vE/96quvdpMnT855O70QNWLECP9CZc4bsSICCCCAAAIIIIAAAggggECdECjG827JB4eWLFnizjvvvKwhgPAKKxAwceJEt84664SLS2p62LBh7umnn65wTASHKpDktaAmgkN2gEOHDnV9+/a12bL6PO6449w777wTnbOCO7vttls0X10TU6ZM8WGgfH7vhBNOcP37989nkzq/brbg0Ouvv+5OPvnkyEGVnJ588slongkEEEAAAQQQQAABBBBAAAEEalqgGB2pjzzySMYXGw8//HA3aNCgvE893+CQfkDP4tdff7374x//mPfvsQECCCCAAAIIIIAAAggggEDtFSjG827JB4euuOIKd++99+Z91dq2beu3UyWMUmxdu3atcFgKPCk4pE4GKg5V4MlpQU0Gh3SAqoxVyoG1nBArsVIpBIf0xt+f/vSnShy9c+eff36lt63UD5b4RgSHSvwCcXgIIIAAAggggAACCCCAAAJZBQrdkfryyy+7M844I+V3Naz3p59+mlJRXBXG//KXv6Ssl20mDA7pRZ2ePXumbLJs2TL37bff+hcr1fdlbZtttnHjxo2zWT4RQAABBBBAAAEEEEAAAQTKQKDQz7siK+ngkMb3Pvrooytc2t13393pf3qj5uuvv3b//Oc/3U033VRhvdNOOy1x+worVvOCxYsXu1122SXlVy+44AK3zz77RMu0zuzZs6N5DfvEUEARR9qJQgeH+vTpU6FyznfffeevjSpGffPNNynHcswxx7hTTz01ZVk5zJRCcOjMM890L774Ygr3iIc7ogAAEJ9JREFUqquu6k4//XTXqVMnpxDhxx9/7If8++CDDyqs99RTT7kmTZqkLC/XmX/9619O97m1du3apQTiqDhkMnwigAACCCCAAAIIIIAAAgiUqkAhO1J/+eUXt//++7t58+b50+3du7cfKqxRo0Z+ftq0ae7EE0+MKK677jqn4cRybWFw6K9//avbc8890276zDPP+OrstsKzzz7LkGWGwScCCCCAAAIIIIAAAgggUAYChXzeNa6SDQ7pgVwhjPh/4NdbN8cff7wdf/T59ttvV1iu0MALL7zgGjRoEK1nE6pOomDOnDlz3EcffeQUzNlwww19GKlDhw4+ZGDr2udvv/3mdBF+/fVXv0j7VXhJ86+99prf348//uhatWrlNthgA9e5c2fb1H9qPf3HeHUy6NzCdtttt7n11lvP/64CDtqPxkm3po4IlSBOajoXOek8Gjdu7M9DbzzpeD/88MNok5VWWiklfKTt9LaSNR13/Df0RlMYYKpXr57beOONnT7VvvzyS7dw4ULbhVt33XW9pfYtEwW7FJKSR9gWLVrkZs2a5f+n8I0qRGm/ugYrr7xyuGqFad0bn3zyiT9nWWrbLl26+GBDoYNDAwYM8FWgKhzEigU6hgMOOCDlq+22286NHTs2ZZlmdO3tfvv3v//t/XS++l/c3Db+z3/+4+bPn2+zrlmzZn5declWxtpW11oGYdN27777rvvss8/c2muv7YMz8XXC9TX9008/+ftF97j+LnRvKDCia6f/6drGm45x+fLlTk7h3+o555zjFLrStdTfYdgqYxFunzT90ksvVSgFrvO94YYbKtj8/PPPTsenbcI2ePBgd/DBB4eL/LQMdc30qfu9efPm3l1/323atKmwfvxvRn+T7du3d7pvX331VffxivBS/fr1va2uvyqNZWry1TZ6g1HHIfPWrVv731blsmxhJ/07oGPXvw+6B7W/tdZay/+9bLnllon/Ptrv2HHpHtJ5a1v9voJDGkLSms7hvvvu87NNmzb1f9f6XWstW7ZMtNK/D1999ZWt5uL/RkVfMIEAAggggAACCCCAAAIIIIBAngKF7EgNwzrqC7zjjjsq9B1OnjzZKQCkpuCPAkC5tnyCQ+pfUP+TtUmTJrlNNtnEZvlEAAEEEEAAAQQQQAABBBCo4wKFfN41qpINDiUFgfQf6idMmGDHXuHz3HPPdVOmTElZrv+YHY71rYfrm2++OeN+FDi47LLLKjx06z+Yx0sFa2gqhSb0H9rjTf9Rfvjw4e4Pf/iD/yr+9lF8fc2r6pCqD8WHKtM+HnzwwZRNFPQYOXKkU+WbeNt00039cSloZU0BjjAsoaCEfseaQhNaFjaFew477LBwkd9GQSu1Sy65JOW4dOxLly51GmLO2pVXXhm5KTTywAMPuFGjRtnXFT4VqlJZ56Rh5hSm0nWeMWNGhe0U4tG101td1rLdM7aeffbo0cMfv81nCg5pnfiQTknXSaENuSQds/ah+1P3yeabb67ZqD322GP+7TVboLfZevXq5YYOHWqLok/tY/z48T6oo8o7qsIVb+mCMVpP11lhmqT72Paj6l8nnXSSD3fYsn333Td6286WhZ86J4XirFXWwrZP93nKKadUOGeFhrbddtvETZLuawWdLr300mh9hfw0rzf30jWFtoYNG+ZDf7ZOfN+6J7Qf+dubibauPlUZ7cgjj/RhonC5pvWPvq639pmuaXg27Tvp70XBnBEjRvjAUtL2Cp6pfLquY9ji9/XAgQP9MYYdpeH64bT+/vQ3qn8HrHXr1s2NGTPGZqNPDRH35JNPRvPxvx/9+x2eu0KdBx10ULQ+EwgggAACCCCAAAIIIIAAAgikEyhkR6qerdVPo5auf0X9CDvvvHN0OK+88kris3q0QjCRT3BIL+rstNNO0XP3I488UuGlqWDXTCKAAAIIIIAAAggggAACCNQxgUI+7xpNyQaH4qEJHXAYQLETCD9VUSP8j9D6Tv9RXZV81PQAr1DF9OnT/Xy2/9N/1Nb21pKCQ/oP3ZnCFhpe7Pbbb/eBjjfffDPrGOe5Boe+//57HzhIF0axYw4/qyM4tOOOOzp1jITNrptCWxoLfurUqeHXidMq5zx69OiUaij6A+jfv3+F4cESd/D7wmIHhxSm0ZB61uz62Xx8SCdbnvSp4fYURLEW/xvQ9QvDGLaefcreqtrYsvinAh0HHnhgymIN0aXgUi5NQSB1Zqn6jFo+waGqWGQ7tl133TXlvlCQ6t57740qYyVtr1BKWNVLVXVUdlxNVXAU5okPRZe0H1XbUTjKKjrFg0PaRutk2pc6/K666qqU3auqlIJruTTdNwrmhNW6VHFKQa9M94ztW9WD7Ny1rCrBoeuvv95Xevvb3/5mu/efGkYurI6kEKGCmOHxxSvKKVQV/nuhYJ+9vZmyc2YQQAABBBBAAAEEEEAAAQQQiAkUsiM17HfQi33qD0xq6kuwisy33nqr22KLLZJWq7Asn+DQO++84zRkvJr6itJVW6/wIyxAAAEEEEAAAQQQQAABBBCoEwKFfN41kJINDmm4p7BSiQ5Yw/xoKJvKtmuuucbdeeedeW2uikLrrLOO3yYpOJTLzmxscoV89B/kMzX9x3v9R/xsFYdkkzQkVqZ9V0dwKOn3LTj08MMPu4suuihplcRlVuXEvjz11FPTVk6xdeKfxQwOvfHGGz6YEf6m3kDba6+9/CJVhOrbt2/KUEzhuvFpXR9VdLHKMfHgUHz9ysyrwoz2a8P3KTizxx575LUrVXxRdSI13a+ZgnPmX1WLTAeoEJ2CN2FTwOyEE04IF+U1ffnll7v7778/523CwEtScCiXHalKlypKqSlUo4BXJtv4Pm+55RanKme2vUJt1lkZXzdpXkGf9ddf33+VLjikim4Kn2VqqvSk+0uVjMKmYFR4nd5//3131FFHhav46mVh5yvBoRQeZhBAAAEEEEAAAQQQQAABBPIQKFRHqoZyVxVotXjfWvxwVMFclbbVFAbSCzC5tGzBIQ0driHM9Syt37CXcNJVP8rlN1kHAQQQQAABBBBAAAEEEECgdgoU6nk3PPuSDQ6dffbZ/o0ZO1hV7FCoorJNQwTFh+NRhRAFWTQO+JIlS5wqkOg/voctHJM8XXBIQ2upMlHTpk39f/hW5ZiwHXvssSmVQxYuXOh22223cBV/bjpHa5mCQ+oc2H333aNOAttGYQlVHlm0aJFT1Y946CDeuaHOhUIPVWbHok9VW1IVF4UQVPVp7733TjlmhUoUtGnVqpUfQu3GG29MOWZdH4WN6tWr55KGrpOXwkVt2rTx2z/00EMp+9cxWHBF07m0+FBlqrATVgHSPuSrP0a94RU2+eqtM52Pmt4sU4gibKpipWunIbtUSvquu+4Kv/ZDv+ktNrWk4NDGG2/sK03pXlPQQ/uIN+3/8MMP99WHNJRcONST1lUgpn379n4zDaEVrwyjwM0hhxziNBydqilpKK74EFvqBGvXrp1TeWz9T9c49FBYTseha6f/VdUifo7h/MyZM90RRxwRLvJDw6n6U2Vb+Cah9qF9KSylUJeqEakSkP2DrO/DSjjpgkMqV66/+0aNGvnhBeNDDCrUpaBiw4YN3SeffOI09F7YFNhRUEveOmcFg8I2aNAgf921LKmKlI5RQR0Lj2lou7CFnY3pgkNaX6EmVY8KqyFpn48//rjfXf369f29p0BaWGVJIbpwmL2JEyf6f6fsGHRvx4OdBIdMh08EEEAAAQQQQAABBBBAAIF8Bey5XVWJq9IU2NEzrVp8iO34fseNG+cmTJjgF6s/Zb/99ouvkjgfBoe0QthHqPnw+VrzapmGPv/vGvw/AggggAACCCCAAAIIIIBAXRQo1PNuaFOywaHDDjssJfCgAEe8AlF4ItmmNYyPhgwL2xNPPOFDJ+GyUaNG+QBRuOzvf/+7a9mypUsKDvXp08cpfBG2+LGrioiqiViranBIASpVJQrbkCFDUoIGCkIpeBB2LFRXcGibbbbxw8qFwxLFg1A69smTJ7sOHTpEp6EhidTpETaFCzp16uRGjhxZISSjoJANQ6dtXn75ZT8UWrh9VYND4b4yTSs4obBM165do9Xi1Xji94FWjA91piHarrvuOr+PpOBQWAEraagnha0U6rHKXEmBK/0d6e9JQ/cpzBK2gw8+2ClAErYvv/wyqqJkyw899FB31lln2awvkR0Ghy6++OKUcFxVLaIfSph46aWXnEIzYVNw7n/+53/CRTlPWzDPNlBY6I477nBrrbWWLXJhR6AWqhNSwUO1pOBQt27d3LXXXusUqlHTtdPfsCr4hM2q/qialYZVtKaOyUmTJqUMvaay5KG5wmLmoGo/GhrRmu6Lu+++278Zacvi4aBwuLT4d/HqX/Fh53T/x4eJ1H0sN2vq9FRYygz69euXMmykzlf/doZNYcLwHPX3fOKJJ4arMI0AAggggAACCCCAAAIIIIBAokChOlL1UpW9vJOtn0kvxKjiuVpYnTjxAIOF8eBQ8FXaSfXh6Pm/WbNmadfhCwQQQAABBBBAAAEEEEAAgbonUKjn3VCmZINDevi1E9YBb7rppin/ETo8iVym9R++X3nllWjVsJJQtHDFRFKlj5tvvtlttdVWicEhVXSxCjG2H1XRUejDWhgG0bKqBof05pKCC9bigSBbrqCUAlPW4usVo+KQfiMcbst+O+lYFKQI2y+//OJUNSlsCheoEyQeMlA1IwUh4i0+nFm2Dp349vGKQ/Hv081rDHv9tg0BpnPZbrvtUlZX5akddtghZZmCQOG9oi8VhJFjPDikZS+++GJKeCQe8FBVpwsvvDD6jaRwkKogbbvttu7dd991qpYVNlVMCoeKsu9UKSkMhqgKU1hZKx5iCYNDhbCw40j6lEkYstE6VQkOxX9DIR9VXJo/f76viKUwVmih9bMFh1RZqmPHjim7TgrTpTvuZcuW+ZLkOgb9u6hhGxUuClsYHIpXTDrppJMqDJOojk+VOLfWvHnzKEgWv68qExxKGopMQSL9W550X8pUASQaAggggAACCCCAAAIIIIAAAoUQsH7FqlYc0os5Nhz3jjvuGAWDko5RL8kpBKRW2eCQ+lw23HDDlN1rqDJVFp8zZ06FlwT1ElLr1q1T1mcGAQQQQAABBBBAAAEEEECg7goU6nk3FCrZ4FA81BIPvYQnkcu0gkIaYsha/D+E23KFBBSqCJuVFk6qOJQUCIhX2ih0cCgeTIqHOOzY4xV84oZx46RqM0nVUxR40DBWagpOKWxiLRyyyZbpM6liUPh9umk7pngQQm96xasTaR9jx45NqUxVXcEh/XYY3Pj888/98HVanm+zqkLx4FBSOez4NdQwY2H46scff3Tq1AqbBYcU1FAgKGyqJGMVYcLlqnYTBr3i91Km4FAhLMJjiU8n3aPDhw+vMDRhfLts89qvroGq5ISVu5K2yxYcUjWsxo0bp2yqfcZDh/HKYQo76jrlMkyj3X8//PCD6969e8pvJQUcU1aIzRQiOKQh1VSSPRzmTvem7tF//OMfKRWr0v0bFjssZhFAAAEEEEAAAQQQQAABBBDIWaBQHalhv0O251e9ZHXLLbf4Y1SlYVVgzqWFFYdU0Vr9mOna9OnT/XDq1lfRu3dvd9lll6VbneUIIIAAAggggAACCCCAAAJ1TKBQz7shy/8DAAD//+gFJdgAAEAASURBVO3dB7QURdrG8UIEE2IARHQNLGYXRcG0ChhWjIA5IZhQXIIBFQMqKKjAroKCAuoawCyomDO46iLGFUyoa2LNmFBMoHw+9fn21vTtmTvDTF/m3vnXOTA93dXp1z2cU8Vbb9Vb9FtxJSrvvPOOP9If//jHoo94+eWXu2uvvTbjOE8++aRbbrnlMtaFX3788Uf30UcfhavcH/7wB9egQQO31VZbZaw/99xzXZcuXTLW2ZdOnTq5L7/80r66fv36uSOOOMJ99913bscdd4zWa+Huu+92a6yxRsa6yy67zE2YMCFa9+c//9lpnZUvvvjC7bbbbvbVfz788MNu1VVXjdY9/vjjbsCAAdH3tdde291xxx3++wknnOD+9a9/Rdt23313N3To0Oi7LcyaNcsdddRR9tUtv/zy7p///Gf0XcfXeawcdNBBGefU+rfeessdeuihVsV/ap/GjRv75QsvvDC6Lq3o2rWrO+ecc/y28C8ZTp8+PVyV1/Kee+7pj7fddttl1B82bJj7y1/+krFOX2Ska7Ky+eabu3/84x/2tdrPDh06uO+//z6qd+SRR7rDDjss+q6fy1dffeVefPFFN2LEiGi9LTzxxBNuhRVWcDNnznRHH320rS7oc/LkyW6dddZx99xzjzvvvPOifcN3wFbGn+Gxxx7revXqZZudfhM77LBD9F0LY8eO9b+HiRMnuksvvTTapvdP72FS0Xt+/vnnZ2x69tln3VJLLeXX6T3T+2blggsuiN7xUljYcZM+9bz03MLSs2dPd/zxx4erqix//PHH7ocffojW67k1b97cf3/jjTfc4YcfHm2rbkH/5t12222+Wvw3k8s1/r6deOKJrnv37v44d911V+LvOtu16D3t37+/e//9993++++fUe3KK690W265Zca6XF+OOeYY9/LLL0dVTjrppAwPPfvevXtH21dbbTV3//33R99t4eqrr3bjxo2zr26jjTZyN9xwg9Pvd9KkSdF6/ZuhfzsoCCCAAAIIIIAAAggggAACCJRKoFT9lHPnznXqe1NZf/313c0335z1EocPH+5uv/12v139RjvvvHPWuuGGSy65xN10001+lfpf1B+Wq8T7/GbMmOHq16+faxe2IYAAAggggAACCCCAAAII1BGBUrV3Q4565Ro4lBSocPbZZ7t99tknvP6M5euuu86NGTMmY53+c3rdddd1++23n/vggw+ibX379nUKComXX375xW2zzTYZq63BXi6BQ/FgnWydFvHgo+oChw488EB3+umnZ9z7M88842QVllyBQ0nBR9p3yJAhbsqUKdFhFGjQvn376Hu2hY033tgHFMQDv7IFGlx//fVu9OjR0eGKDRzK9p7oBAqIUGBEWBQwtskmm/gAtnhgmjp9cgW+2XH69OnjA7PSDhx69NFH3RlnnGGn9Z/ZOpoUxKdgPivxYJhcgUMK5ivWws6b7TMe7Kf3S++bggaTioKN1OkXBolZAN7nn3/uA2/CbTpG69atnYIA9U6q0zAM1ssVOKR9wyArfbfSrl07W/Sf+m3rXuKBOVZJgUbaZ9NNN3X33nuvu/POO22TD3BT4JCCHnWMsKjjcpdddglX5VwuVeBQUhCTAowUlBUGZz722GNupZVWynlNbEQAAQQQQAABBBBAAAEEEECgEIFSdaQuXLjQbbvttv7U6lubNm1aNJAqfj1nnnmme+SRR/zqQgbxFBo4FO+/VP/AWmutFb8cviOAAAIIIIAAAggggAACCNRBgVK1d0Oasg0cmj17tuvWrVt4rU7BAMrC0bBhw4z1+vLTTz+5XXfdNSMQIMzQctppp7mpU6dG++k/4NUoj5fXXnvN9ejRI2O1giYUNFAugUMKTgkzGOliw2Aeu3hlfQkDC+KBQ8q6FGYJSTLRKKqLL77YDuk/w3PFg5iyBQ7FrznpXBkniX1RoIGywFjZd9993cCBA+1r9KnAiTCrUpqBQ0899ZRTJpawKHBNnUnxDhzVUYYfBZ3kW9IKHFLAk4JPkn5j2a4x7qoMUGGAVq7AoVJYVGcW/32rfq5AQ2UHimeMspGADz30UMa7pd+NslYpQM9KPFtTdYFD+ndL2c/CkuRv/9bEf1fK1HPFFVdEmb50nHjmMcs4pKxY8UC7pAA4ZefSO2ylVatWTkFrKqUKHNKx4r9dXaeNotT2Qv8t0D4UBBBAAAEEEEAAAQQQQAABBKoTKGVHqgaDffbZZ/6U6lNIyrau/o+ddtop6pu0rNTVXae2Fxo4NG/evIxsRkn9DvmclzoIIIAAAggggAACCCCAAAK1T6CU7V27+7INHNIFhqN07ILbtm3rLrroooxpvTTdkNaFQTCqrymbNHWTylVXXeXGjx/vl+0vTZmj/5C38uuvv/qMO2GAkbZZoEy5BA4pMEaBHGHRdFTqZLCpo5IyBcUDh/7+97+7W265JTqMssjIcOmll/brNM2VpnwKA3a0wTy0HA9wyBY4pM6SU045RbtERYELCu6yIl9NYaSRXFaUKWrrrbd28SAnbVfGldVXX92quqSgrzQDh5LOp2egQAgVWdiPVt8POOCAKhl+5B0GOmm6LJvqLa3AIf0O9DtKmuJLHVzKTmPvka47adouZesKM1HFA4c0xdpee+2l3X0p1sKOk+0zPj2Y1bNsYfZdn8rmc+qpp0YdebZNz0G/kfg0icpAFmZbUn0FjIVBN9UFDmlaPb3bVvRvzVlnneWU9SksDz74oGvatKkPXtT7ZUUBPTK2on/zNN1hmBXJAodUJx6so9+ZgnWWXXZZfwidXxmWwqw/mt5M/+aqFBo4FM9A5Q/y+19JwYfhdsuyFK6z5eeffz7qmNU6/d4LmXLNjsMnAggggAACCCCAAAIIIIBA5QlYn0xSkE+hGmFfgdrMf/3rX6scIswerIzF8UF/VXYIVhQaOKSp5tWvYCVbBmnbzicCCCCAAAIIIIAAAggggEDdEShle9dUyjpw6MMPP/TTVNnF2qf+c1/T7qjh//HHH7unn37aqW5YVGfy5MmuWbNmfnXS9D2qo0wlmlpq/vz5/j/W4/+Rr6xHJ598sj9GuQQOaQSTphULp17TBSojk/5TXcEEYTCKv/jf/tL9husVSKCOibAocETZfL7++munwJXnnnsu3OyXFydwSP6aZi4MdFAww/HHH+8U3KPsKzfeeKN74YUXMs6ndRtuuGHi1E26XwVw6FM/jlGjRmUcXwdKM3AoKWOMsjN17NjR34MCdBSwFha9Twr40LOQo7LIhEX+f/vb3/yqtAOHdJKxY8f6bDrhNSjIRUEoCmJS0JACicLnproPPPBA9NvS93jgUOfOnX1gke5T07MVa6FzVFcUPKjffLwoSKpNmzY+U5nuJx4YqPrHHXec/6NlTT+nrExhUZrxVVZZxX3xxRdOAW/xIMQwu1m2ICa9F3r2jRo1cpqKMf5vjbZbdi8FPYa/BWU803XVr1/fvfvuu/656f0JyyGHHOIDorQu229bdZSdTSMR4/uPHDkymj6w0MAhnVO/1ebNm/spx+rVq6dVvmhEpkZmZisWsJW0PZ5ViexESUqsQwABBBBAAAEEEEAAAQQQSBIoZUdqvK2vgXjrrbdedFoN8FEb9qWXXvLrFNSjwXD5lkICh9SOVpZl66sJBwLlez7qIYAAAggggAACCCCAAAII1F6BUrZ3TaGsA4d0kfoPfP2HeaHFsqqE+11//fUZ0yuF25KWFfSg4I2VVlrJby6XwCFdTFIGn6R7CNfFA4f+85//uIMPPjisktfy4gQO6cDx0VDVnaxTp04+o5HViwcR2Ppcn2kGDr3//vtOnTNh0fRwCg5R+fnnn33mF/vhhvWyLSs7i02JlVbg0JVXXhllbfn22299ZiDrbMp2XeH6MMjG1idlB9M2Bbxo+q1iLew8uT7nzp3rO+UKuRcdTxmFNApQQTkq06ZNiwJw/Irf/1JwUDxYz7YreM0ynsU7E61OdZ/hsw87DG0//X6VMSjMEmTb9KkOSRttqGxhXbp0yVo33E/L+p3o31kL+KkucCjXvx0KhlOWsLBoJGZSEKKyHg0dOjSsmrEc/80TOJTBwxcEEEAAAQQQQAABBBBAAIEcAtYfU4qMQzqNsgErs4/Kmmuu6bp37+6ngv/oo4/8oKyXX3452qbpzJZZZhn/PZ+/wn4A9aW0bNmyym6ffvqpU59D2C+gDMAayGMDJ6vsxAoEEEAAAQQQQAABBBBAAIE6J1Dq9q6Ayj5wSBep7ByacijfoqmeunbtWqW6gheUPWXKlClVtsVX6D/pNZVX+B/g5RQ4tGjRInf77be7ESNGxC89+r7zzjtnZBWJBw6pooJI9CdbURCLOiXCsriBQzrGkCFD8vJXSmcF4ay44orRqefMmeOz2MSzS0UVflvYaqutMgIU0gwcSnofFIQyZsyYKABDWYk0XV4+wSx6lnpmVmoicEjnevHFF/0UamHHk11D/FOZYxScYlNe2fZsQWEWOKR6xVjYear7VHabQYMGZbwDufbR+6EMT+pos6J/J44++ugqU/TZdn0mBRHpebVo0cL/Xg499NCwus+KpWvLVnr37u3Padv/+9//+gxd9j3pM34N6gjV9IsNGzb01TWVmjJyVVf0jPRvXZMmTaKq1QUOqaL+jU36Ler933bbbaNjaSHbv+FhlqOMHX7/QuBQkgrrEEAAAQQQQAABBBBAAAEE8hEodUeqspVrYEw4tXjSdSj79BZbbJG0Keu6MHAoa6XYBvVBqZ8t7NOIVeErAggggAACCCCAAAIIIIBAHRQodXtXRLUicEgXqkwfagxbyl+tixdN83TKKae41VdfPb4p4/tDDz3kg1KyBXOo4T148OAqo3VUXxkvwqIsI8o2Ehb9x/l1110XrYpnyfjqq6/crrvuGm3XgqYtWnnllaN18YxCCuBRRpJ4UWeF0iO/8sorUTaUjTbayB155JF+SqS+fftGu6gjQQEe8aLAoUmTJmWMWFIdZZZRAIRcw6Jr0zRWKsOGDfP72nZNcdW/f3/7mvj5zDPP+ICnbNlbBgwY4A444AC31FJLVdl/3rx57txzz3UKiogX7adgivCe1VETny4svl/4XVmOwgAaBS706NEjrJKxnBQ8oVFl4Wg2XbOyuWj6qKSi4BW922ussUbGZr1bulcrOqaOHZZ4ph9NcaVAJSualmr77be3r/4zzDhkG3TPCs5LcrU6CsjRFGRJRRlulNFLo9zC31UYOKT9Ftci6ZzZ1v3666/+t6KsY+G1hPUVRHf66ae7PfbYI/E902hBPZN4lhz9hvSOH3TQQVX+LZCNjJIyDun3pTTimiotLLoO/YYUKBcvzz//vL+GeHCO3nH9xjQFnJ53WOLPSMF2ekfi57V9FAg2cODAKqMg41Ol6Xy677D8+9//9lOmhVOqaXtS4JCeexgUp3q6d/27Z4FOWhcvOm84vWI4lV+8Lt8RQAABBBBAAAEEEEAAAQQQCAXS6Ej95ptvnPqf4m1hnVft9fPOO89nXw6vI5/lUaNG+cFAueqqHb3hhhu6DTbYwG266aZOWXyT+s5yHYNtCCCAAAIIIIAAAggggAACtV8gjfZurQkcssenAAVNEaWpcpSVo2nTpm6dddbxjfNC0vIquEDBAe+9957/07hxY58GeN11183IcmPnrQ2fCxcu9B0G1mlwxx13ZEz1pYAiZSTJVj755BNvscoqq7i11lrL/8d+trrFrpe/AiLeffdd/9m8eXPvr/MuvfTS1R7+iy++cK+//rpTh43SNyuwqkGDBtXutyQrKHhC767uecGCBf66FQwUBowtyevTuZVFSb8J/WMjW/225KugJpvKK9c16vf5+eefux9++ME/x0aNGlUJrNP+NWGhrFy6Ft3L22+/7adL0+9bHXn6kytgxe5RWZL0zH755Ren34/2t+m8rE78MylwSEFAuh69s/p3R++qnr1Sm9vvNX4cfVf2I+2j34pSnG+22WZVAhqT9gvXmYOeq949nVudjOutt16VzFHhfvkua4o4vSv6TesaFUgZz0ilY8WzGB144IE+eCvf81APAQQQQAABBBBAAAEEEEAAgUIE0uhItfOr/2TmzJk+u7L6ddR/0qZNm5xtfNuXTwQQQAABBBBAAAEEEEAAAQSKEUijvVvrAoeKAawr+7766qtu4sSJGbfTrVu3jBFNyjSjzDNh+uSOHTu6iy++OGM/viCAQGkFsgUOlfYstetoCuBSdqewXHPNNT4QKlzHMgIIIIAAAggggAACCCCAAAKlEkijI7VU18ZxEEAAAQQQQAABBBBAAAEEEFhcgTTauwQOLe7TWIL7ffzxx1WmjFLmkn79+rkmTZq4zz77zCnbUDxtcnwaoyV4C5wagTorQODQ/x7tXXfd5bMR3XfffT7zk21RtqVbb7212uxNVp9PBBBAAAEEEEAAAQQQQAABBAoVSKMjtdBroD4CCCCAAAIIIIAAAggggAACpRZIo71L4FCpn1INHa9Xr15VAoNynVqBRTfffHOq04/lOj/bEKgUAQKH/vek27Vr978vwdK5557runTpEqxhEQEEEEAAAQQQQAABBBBAAIHSCqTRkVraK+RoCCCAAAIIIIAAAggggAACCBQukEZ7l8Chwp9DWewxb94817dv34ypyLJdmKYoGzx4sFtxxRWzVWE9AgiUSIDAof9BJgUOrb322j7bUIMGDf5XkSUEEEAAAQQQQAABBBBAAAEESiyQRkdqiS+RwyGAAAIIIIAAAggggAACCCBQsEAa7V0Chwp+DOWzw8KFC91TTz3lNB3Qa6+95r788kt/ccsvv7xbf/31XatWrVzr1q3dXnvt5ZZaaqnyuXCuBIE6LDBnzhx34oknRne4zDLL+Gxf0YoKWggDhxQwtM0227jjjjvOrbLKKhWkwK0igAACCCCAAAIIIIAAAggsCYE0OlKXxH1wTgQQQAABBBBAAAEEEEAAAQRCgTTauwQOhcK1fPmXX35x8+fPd40bN67ld8LlI4BAXRBYtGiR079L9evXd/Xq1asLt8Q9IIAAAggggAACCCCAAAII1BKBNDpSa8mtc5kIIIAAAggggAACCCCAAAJ1WCCN9i6BQ3X4heHWEEAAAQQQQAABBBBAAAEEEEAAAQQQqESBNDpSK9GRe0YAAQQQQAABBBBAAAEEECgvgTTauwQOldcz5moQQAABBBBAAAEEEEAAAQQQQAABBBBAoEiBNDpSi7wkdkcAAQQQQAABBBBAAAEEEECgaIE02rsEDhX9WDgAAggggAACCCCAAAIIIIAAAggggAACCJSTQBodqeV0f1wLAggggAACCCCAAAIIIIBAZQqk0d4lcKgy3yXuGgEEEEAAAQQQQAABBBBAAAEEEEAAgTorkEZHap3F4sYQQAABBBBAAAEEEEAAAQRqjUAa7V0Ch2rN4+dCEUAAAQQQQAABBBBAAAEEEEAAAQQQQCAfgTQ6UvM5L3UQQAABBBBAAAEEEEAAAQQQSFMgjfYugUNpPjGOjQACCCCAAAIIIIAAAggggAACCCCAAAI1LpBGR2qN3wQnRAABBBBAAAEEEEAAAQQQQCAmkEZ7l8ChGDJfEUAAAQQQQAABBBBAAAEEEEAAAQQQQKB2C6TRkVq7Rbh6BBBAAAEEEEAAAQQQQACBuiCQRnuXwKG68GZwDwgggAACCCCAAAIIIIAAAggggAACCCAQCaTRkRodnAUEEEAAAQQQQAABBBBAAAEElpBAGu1dAoeW0MPktAgggAACCCCAAAIIIIAAAggggAACCCCQjkAaHanpXClHRQABBBBAAAEEEEAAAQQQQCB/gTTauwQO5e9PTQQQQAABBBBAAAEEEEAAAQQQQAABBBCoBQJpdKTWgtvmEhFAAAEEEEAAAQQQQAABBOq4QBrtXQKH6vhLw+0hgAACCCCAAAIIIIAAAggggAACCCBQaQJpdKRWmiH3iwACCCCAAAIIIIAAAgggUH4CabR3CRwqv+fMFSGAAAIIIIAAAggggAACCCCAAAIIIIBAEQJpdKQWcTnsigACCCCAAAIIIIAAAggggEBJBNJo7xI4VJJHw0EQQAABBBBAAAEEEEAAAQQQQAABBBBAoFwE0uhILZd74zoQQAABBBBAAAEEEEAAAQQqVyCN9i6BQ5X7PnHnCCCAAAIIIIAAAggggAACCCCAAAII1EmBNDpS6yQUN4UAAggggAACCCCAAAIIIFCrBNJo7xI4VKteAS4WAQQQQAABBBBAAAEEEEAAAQQQQAABBKoTSKMjtbpzsh0BBBBAAAEEEEAAAQQQQACBtAXSaO8SOJT2U+P4CCCAAAIIIIAAAggggAACCCCAAAIIIFCjAml0pNboDXAyBBBAAAEEEEAAAQQQQAABBBIE0mjvEjiUAM0qBBBAAAEEEEAAAQQQQAABBBBAAAEEEKi9Aml0pNZeDa4cAQQQQAABBBBAAAEEEECgrgik0d4lcKiuvB3cBwIIIIAAAggggAACCCCAAAIIIIAAAgh4gTQ6UqFFAAEEEEAAAQQQQAABBBBAYEkLpNHeJXBoST9Vzo8AAggggAACCCCAAAIIIIAAAggggAACJRVIoyO1pBfIwRBAAAEEEEAAAQQQQAABBBBYDIE02rupBA4txr2xCwIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCFQj8Mc//rGaGvlvJnAofytqIoAAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCwRAXKPnDoD3/4wxIF4uQIIIAAAggggAACCCCAAAIIIIAAAgggULkC//3vf/3N009Zue8Ad44AAggggAACCCCAAAII1EUBa+8SOFQXny73hAACCCCAAAIIIIAAAggggAACCCCAAAIlEbCOVAKHSsLJQRBAAAEEEEAAAQQQQAABBMpEwNq7BA6VyQPhMhBAAAEEEEAAAQQQQAABBBBAAAEEEECg/ASsI5XAofJ7NlwRAggggAACCCCAAAIIIIDA4gtYe5fAocU3ZE8EEEAAAQQQQAABBBBAAAEEEEAAAQQQqOMC1pFK4FAdf9DcHgIIIIAAAggggAACCCBQYQLW3iVwqMIePLeLAAIIIIAAAggggAACCCCAAAIIIIAAAvkLWEcqgUP5m1ETAQQQQAABBBBAAAEEEECg/AWsvUvgUPk/K64QAQQQQAABBBBAAAEEEEAAAQQQQAABBJaQgHWkEji0hB4Ap0UAAQQQQAABBBBAAAEEEEhFwNq7BA6lwstBEUAAAQQQQAABBBBAAAEEEEAAAQQQQKAuCFhHKoFDdeFpcg8IIIAAAggggAACCCCAAAImYO1dAodMhE8EEEAAAQQQQAABBBBAAAEEEEAAAQQQQCAmYB2pBA7FYPiKAAIIIIAAAggggAACCCBQqwWsvUvgUK1+jFw8AggggAACCCCAAAIIIIAAAggggAACCKQpYB2pBA6lqcyxEUAAAQQQQAABBBBAAAEEalrA2rsEDtW0POdDAAEEEEAAAQQQQAABBBBAAAEEEEAAgVojYB2pBA7VmkfGhSKAAAIIIIAAAggggAACCOQhYO1dAofywKIKAggggAACCCCAAAIIIIAAAggggAACCFSmgHWkEjhUmc+fu0YAAQQQQAABBBBAAAEE6qqAtXcJHKqrT5j7QgABBBBAAAEEEEAAAQQQQAABBBBAAIGiBawjlcChoik5AAIIIIAAAggggAACCCCAQBkJWHuXwKEyeihcCgIIIIAAAggggAACCCCAAAIIIIAAAgiUl4B1pBI4VF7PhatBAAEEEEAAAQQQQAABBBAoTsDauwQOFefI3ggggAACCCCAAAIIIIAAAggggAACCCBQhwWsI5XAoTr8kLk1BBBAAAEEEEAAAQQQQKACBay9S+BQBT58bhkBBBBAAAEEEEAAAQQQQAABBBBAAAEE8hOwjlQCh/LzohYCCCCAAAIIIIAAAggggEDtELD2LoFDteN5cZUIIIAAAggggAACCCCAAAIIIIAAAgggsAQErCOVwKElgM8pEUAAAQQQQAABBBBAAAEEUhOw9i6BQ6kRc2AEEEAAAQQQQAABBBBAAAEEEEAAAQQQqO0C1pFK4FBtf5JcPwIIIIAAAggggAACCCCAQChg7V0Ch0IVlhFAAAEEEEAAAQQQQAABBBBAAAEEEEAAgUDAOlIJHApQWEQAAQQQQAABBBBAAAEEEKj1AtbeJXCo1j9KbgABBBBAAAEEEEAAAQQQQAABBBBAAAEE0hKwjlQCh9IS5rgIIIAAAggggAACCCCAAAJLQsDauwQOLQl9zokAAggggAACCCCAAAIIIIAAAggggAACtULAOlIJHKoVj4uLRAABBBBAAAEEEEAAAQQQyFPA2rsEDuUJRjUEEEAAAQQQQAABBBBAAAEEEEAAAQQQqDwB60glcKjynj13jAACCCCAAAIIIIAAAgjUZQFr7xI4VJefMveGAAIIIIAAAggggAACCCCAAAIIIIAAAkUJWEcqgUNFMbIzAggggAACCCCAAAIIIIBAmQlYe5fAoTJ7MFwOAggggAACCCCAAAIIIIAAAggggAACCJSPgHWkEjhUPs+EK0EAAQQQQAABBBBAAAEEEChewNq7BA4Vb8kREEAAAQQQQAABBBBAAAEEEEAAAQQQQKCOClhHKoFDdfQBc1sIIIAAAggggAACCCCAQIUKWHuXwKEyfQF+/PFHt+yyy5bp1XFZCCCAAAIIIIAAAggggAACCCCAAAIIVIaAdaQSOFQZz5u7RAABBBBAAAEEEEAAAQQqRcDauxUROPTrr7+6yZMnu19++SXn823YsKFr0qSJa9WqlVtzzTVdvXr1stZ/8skn3eeff551e7YNq6yyittpp50yNi9atMg98cQT7vHHH3dz5szxf77//nu3/PLLu5YtW7oNNtjA7b///m5xH9bcuXPdo48+mnHOjh07uhYtWmSssy96OZ566in76j87derkVl111Yx15fzl7bffdjNnzowusWnTpq5Dhw7RdxYQQAABBBBAAAEEEEAAAQQQQAABBBDIR8A6UgkcykcrnToLFixwDz/8cMbB1de34oorZqzL9mX69Onuyy+/jDZvuummbt11142+qy/0m2++ib4nLTRu3Njvs8Yaa7j69esnVSl43fz5893QoUOd7m/QoEF530/BJ6qQHW677Tb37LPPus6dOzv1f1MQQAABBBBAAAEEEEAgt4C1dxc3FiXp6PV+C4BZlLRhcda98847frdSNMjV8Cq0obDhhhu64447zm233XaJl//Xv/7Vvfzyy4nbcq1cb7313IQJE6IqCnD5+9//nhHkEm2MLSh456STTnIrr7xybEvury+++KLr27dvRiWd889//nPGOvsybdo0d9ZZZ9lX/3nVVVc5NahrS7n55pvd6NGjo8vdbLPN3Lhx46LvLCCAAAIIIIAAAggggAACCCCAAAIIIJCPgHWklqKfUuf74Ycf3LXXXpvPqX2fZlKfnAZI3nfffe65555z6kf99NNP3VprreUDW3bccceC+0LtYkpxbXYs+zz33HP9QMm9997bD4609YV8fvXVV26vvfbK2OWMM85wXbp0yViX9EWZ3XfeeeeMTSeccII75JBDonXdunVz7777bvS9ugXVP/7444sOIFL/pfoxt9lmGzdy5Mgqp/3oo4/cpEmT3Jtvvuk0OHSdddZxbdq0cerr3Hjjjd1SSy1VZZ98V+gd0kBWDWj94IMPXIMGDfxxdWydQwNgsxWZ/utf/3JvvPGG79fWYNjVV1/dv4Pt2rVzu+22mz9etv1/+ukn995777mFCxf6Abz59HdfdNFF3kGfOldSeemll1yfPn38gNw77rjDKdiLggACCCCAAAIIIIAAAtkFrL1L4FB2I7/l6KOPdj179qxSqxSBQ2rwHn744U6f+ZbNN9/cXXbZZTkbXvFjETjkfKOXwKH4m8F3BBBAAAEEEEAAAQQQQAABBBBAAIHqBKwjtVSBQwoCOfLII6s7rd8+YMAAt88++2TU1fWcc845bvbs2Rnrwy9t27Z1F1xwQcGBE8VeW3gNWta1HnTQQX61+kF79+4dr5LX96TAoS233NKNGTOm2v0VGHPmmWdm1Cs2cEgH0/mHDx/uVlhhhYxj5/vlP//5j+vevbuvPnbsWKd+37Dcc889TkEy2YoCn3Qfi1PmzZvnTjzxxKzvkDLhX3PNNW7ttdeucngFDSloS5l9spXVVlvNjRgxwmfSD+soWOmmm25yut+w7LDDDu7000/3swGE623Z+rc1EFYDYnMVDaBV/a5du/pj5qrLNgQQQAABBBBAAAEEKl3A2rsEDuXxJpx66qluv/32y6hZisAhHVcjM5KKUt5qRElSOeCAA1z//v2TNiWus4ZVuJGMQ6EGywgggAACCCCAAAIIIIAAAggggAACCCQLWEdqqQKHkrJ9J5/ZuXjgkDKrK0t6GDSkICFlbFHmoTBrTvv27d2wYcNcvXr1sh2+yvpiri1+MGWTUTCIpglTKXXgkI559913u6ZNm2oxa1HGo0cffTRje67AoYsvvrhKRhtlYvrss8/cAw884J588snoWL169XJHHHFE9D3fBSXuVxCVMtonBUA9/fTT7rTTTss4nOrpXdR1WMk26NW2J33quahv+9VXX402qy9aWXzUj2xFGYeuvvpq16JFC1vllClIfdovvPBCtE5BQi1btnQzZsyI1mlBx7zuuutco0aNovUKhFJAVFLR+W655ZYq07X9+uuv7phjjvHv/MSJE12rVq2Sdo/WWdYhrRg/frxr3bp1tI0FBBBAAAEEEEAAAQQQyBSw9m7FBg4p+EZpU1XUUNMc12pYK5AnHrCjERb333+/a9iwYaQYDxzS6AsbIRJVSlhYaaWVnEZQfPvttz5la1hFKXOV5naDDTbwaW41x7XSDcenDdM+aqDmO5c2gUNkHArfM5YRQAABBBBAAAEEEEAAAQQQQAABBPIXsI7UUgUO3Xjjje7yyy/3F6BAiWWWWSbrxagvcdlll422K1BGwUAqCthQxp3wuhQgo0AZK1dccYWfdsq+V/dZzLXp2OpXlZemsFJ/qqbAspJG4FDSgE87nz4V8LPLLruEq/xyrsChyZMnZwTLhDurH1lTimn6MBUFzMis0KJgqlNOOcXvdskll7htt902OoSy8hx88MFRH7X6jJVhyt6TMDBGO8X3jw6UZeHBBx90559/frQ1zHakbELnnXeen75MFTQd2j/+8Y+o7kMPPeS3a4X6zC+99FJnU+n9/PPP7vnnn/fv3/fff+/3CQPftM0yJOmZKDOQgop0PQrWUtHgXT3TsNg73blz5yqZo8J64fJRRx3lA4023HDDvKcFDPdnGQEEEEAAAQQQQACBShGw9m7FBg6pAbTrrrtWed4K1hk0aFCVTEBqcO++++5R/XjgkIKBlH4135IUzKO0uZpLOl6S0ulqtIYCjPIpSecqVcahTz75xM2aNSsa7aJ5o5s3b+7WX3/9vEZzfPjhh05pefVHI6bU2NZ9aU726uboViNanQ9KoazjrLnmmv6cGs2iucE1R7gVzc3NVGWmwScCCCCAAAIIIIAAAggggAACCCCAQL4C1pEaBujku29SPU1vNWXKFKcMK/fdd19SlazrNP2Y7aPMQ0lTnimQxAJb+vTp4wcqZj1gbEMx16ZDaSqpbKVUgUMawGkBSdX1+U2dOtUNHDjQX1K43+IGDulACopSph8r2TLK2/b4p4KPevbs6V5//XUf/HXHHXdk9INaoIz2U1+pAnfC4DGtV8DZZZddpkXfZx0Gi/mVOf7SgFobOKt+8N122y2jtga8aho0TQ+nonPZlGUnn3xylFkoW/9yGJi07777RpmTFMR2ww03+Hu6/vrr3dJLLx2dV/eoP/HfhIKRNNWdsizddddd3ivaKceCfiP6ragouE7ZmigIIIAAAggggAACCCBQVcDauwQOVbVx3333nevUqVPGlvXWW89NmDAhWlds4FA4OkMH1QgNrUvKIqSRHuoIUKCMFTX6czXErZ4+0wgc0mgdNfBCk/CcWlbDVil127RpE9/kNOpEI6vuvPPOKtu0Qh4adaNgraR0ynPnznVnn322mzlzZpX91SBV8JCN3FKFpE4EjR566623ov0VsBSfki7ayAICCCCAAAIIIIAAAggggAACCCCAQEUKWEdqqQKHlGlF/XXbbLONz15TCKqmbFLAiUq2gYVhViJlSFc/Zr6lmGvTOXL1V5YqcEj9fMpUYwE76l/UQMakov7Dxx9/3G9SsI6m3lIpJnBIU8LpXlTigS5+ZTV/aRCmpjhT0bOJZ7EPg8P69+/vFOgTL/Fs9rrHeHBRfB9917usQByVXP3RFuSjeuF0bPvss080VdrDDz+cMQ2Z6qpokKcFtIXvuDIJ6ZkpKMkyD/3/Hs738R5//PH+a3jc2267zY0aNcopg9Cxxx5r1av9DDNNdezY0WmKNAoCCCCAAAIIIIAAAghUFbD2LoFDVW38msGDBzs1UsISNsCKDRxSA18N/bDsuOOOTg0kG8ERbitmOY3AoaS5wbNd4zXXXOM22mijaLNGiCgQKpyPO9oYWwgbprZJU8qpE8NGvdj6XJ9JgUMKTLI51rVvoVmjcp2PbQgggAACCCCAAAIIIIAAAggggAACdUPAOlJLFTi01157+X6tpACK6sQ0ZdVjjz3mq2m6qb/85S9Vdgn7Nc844wzXpUuXKnWyrSjm2nTMzz//PGPwozKpazorlVIFDmlqLAXTKKO8ykknnRQFw/gVv/+lgYvms/nmm7u99947ykJTTOCQsuYosEalffv2TlmaCikKXlJ/qYqy7Gg6sLDYM9C6W2+91WdmD7fbsk3Hpe/jx4/PK/v7Aw884IYMGeIPoQGb2TIVKcDHpgwLg3/Un61Brsr8ni37fpgxKcw4pAxFyq609dZb+2Aguw993nPPPT64R8FMjzzyiB9IquAo7a+ibEMKFiukhP33yjxlU70VcgzqIoAAAggggAACCCBQ1wWsvUvgUJYnrWw0NreyVdEIB+sgCBse2q75kuMjJWw/+9TIF2XCUVEDS/NTJxUFDqlBpkw9f/rTn1yzZs2SquW9rtSBQ9OmTXNnnXVWxvnVqFMDXJ0Db7/9dsa2tm3bZkwbFnZeZFTM8kWN6U022STaGqbEjVZWs0DgUDVAbEYAAQQQQAABBBBAAAEEEEAAAQQQSBSwjlTrF0yslOfKMJjFMs3Mnj3bvfLKKz6wQYPvFBijrDBJgRLhNFDqj1NGHfUjLrfccj5gR/2Xo0ePjq5G2YeaNm0afc+1UOy1JR07DCIpVeCQ+mE1TZdNsaXAGwXgxEt4bgXBaGqsYcOG+WqFBg4pg83777/vnnnmGXfllVdGp7r22mt9v3C0Io+FMOBHgVUNGjSI9vr666/dnnvu6b/r+eoeshULxNF2BS8piKm6IjdNPaYyYMAApwxCSUUDPm1bPBN/Un1bpynQZGtToV144YVOg2VVwgz8/fr185mHlGleU79pkKjev3Bg59ixY93EiRN9Vvr999/fH6OQv2z6M+2j+27Xrl0hu1MXAQQQQAABBBBAAIGKELD2LoFDWR63sgup4R0WTX21xRZb+FXxwKGwXrbleONYjfibb745W/VovYKNlFK1Q4cOfsqtpKm7osoJC6UOHAobpTpdfGRNOMe2XY5GQqkD4+WXX66SHlnTgynT0k8//eQbw1dddZVvKNq+2m4jXMJUvrZdKYHVIFWA1dNPP+3niFdDMywEDoUaLCOAAAIIIIAAAggggAACCCCAAAII5CtgHamlCBzSgLsePXpUe2r1d2ng3vbbb59Rd9GiRT5jjAKIwqJgGgV7hBm6lZFn1113DavlXC722pIOHgbvxPtGk+pnW6f7UiYeFd2rAnZOP/109+STT/p1kyZNigZs+hW//XXmmWc6BeaoKIBKWXTyCRxSwE6YnUZ9lvG+RtVR5p7tttvOHz/fv8L70CBMBceERe+aTSWmwaUW5BPWsWUFMGm6OpV8M0sNHTrU3X///X4fTYm20047+eX4X+FUX7kCmOSiLELz58/3U5SF2d01aFZG1pf966+/ut69e/tpyXS+1VZbzU+v9sEHH0SntwxLn376qc82pH5x9Z+HwVVR5WoWnnvuOXfiiSf6WsW8e9Wchs0IIIAAAggggAACCNRqAWvvEjiU5TGqkaOprMKigBmbp7sUgUO//PKL7wCwBm54rmzLalCq0VvIdGalDhxSkM/MmTOjS9S1qNGp0ScqalhqFIc6MqwcffTRvjE4cuRId/vtt9tqP5e2UuSGjT8bTWKV1Di97777fINd81ErdW1YwkxQWq/godNOOy2s4gOuxo0bl7FOAUqvvvpqtK5169ZVpo+LNrKAAAIIIIAAAggggAACCCCAAAIIIFCRAtaRWorAIfUDKuAlLMqYo341BUs8++yz4SYfIKNAmbCoT7Fnz55OmYqyFU3RNWjQIFe/fv1sVaqsL8W1xQ+aZuBQeGxlrDnssMOi03/33XeuU6dO/vuWW27pxowZ44OH8gkcig5SzYKyomsqLWUyyrcou476SVU6d+7s+3nDfV9//fWofzJpIGRYVwE1ll2qV69e7ogjjgg3Jy5rcKYCqFSqy8Jj/eCqq3cj6V1SZiFNGxcvLVu29EFNYZ+v6nzxxRc+O9JTTz2VsYuCiPS+2qBdBTWpPzhXcFPGARK+hEFYCvCKzy6QsAurEEAAAQQQQAABBBCoOAFr7xI4lOXRa6SEAoXCormnlS5YpRSBQzqORlr885//9MEw4YgMbctWNNJCo0mS0hUn7VPqwCGNVlJq2XhRA2+rrbbyU5apQW7TsoX14tOMaU5rpecNy3vvvVdlbnDNha5jx92zpcqNn6e6hnZ4fpYRQAABBBBAAAEEEEAAAQQQQAABBBAwAetILUXgUBjsoX6tSy65JGMqsXfffdcHUCj7j4oG7Gm6JgvAUKDGOeec4xRgYkWD7po3b+60b1jUH6bja3s+pdhrSzpHGNxTTNaXMFOPZRxSthsFSKnYOruGRx55xDvqu2XjUdahfAKHFHC00kor2aH8p871zjvvZLhrgwZ5Kkv9UkstlVE/2xcFhp100kl+c7du3VyfPn0yqob9uArcifdPh5WVnUd9pirHHXecO/LII/1yrr/CvlVN5aWgtWwln8Chzz//3B166KFVMjLpmHq/FQzUqlWrjFNosOlrr73mMxTJVf9BIUd7Ty3zla7t6quv9hmLtM/DDz/sZsyY4adB0/R7bdq0cV27do1+Gxkn+e3LN9984/bYYw+/OlsfcnwfviOAAAIIIIAAAgggUGkC1t4lcCjLk1ejWiluwxLOCR42slRHQTOWLjfcJ1xWA0iBMtmKMvW89NJL7t///rf/DLPhxPcJp++Kb4t/Dxucti3MnmTr7HPatGk+E5J916ey82h+dZWpU6e6gQMH+uVcf6m+goLCRqbmxlba5EKLGpmat1zGYcrlpAa2jq3sQhMmTIhOQ+BQRMECAggggAACCCCAAAIIIIAAAggggEABAtaRWorAIU3LpAAf9QO2b9/erbDCClWuJJ7FJRzMeOyxx0YZtBVUpOnM1O+logGKCpj529/+FgVyKKijX79+Vc6RtKLYa0s6ZpqBQzqfgqgee+wxf2pN62VZ2sNpzO6991636qqr5p1xaPLkya5FixZJt+MzrN95550ZAT3K4qO+2nyKBmNqUKZKUpYgC5rRdsuUpOWkoqAavRsqAwYMcOp3ra7ofVHfr8oVV1zhg2/8l9hfP/74o9NUYyoK6NFzzFX07qnP9plnnvFZsvQOq2jKvbvuuitrcE/SMW1AqLLSqz9d16IM/AoaihcFBCmTULNmzeKbnDJz6TemoutQBiMKAggggAACCCCAAAIIZApYe5fAoUyX6Fs8MEgbwpSs8e077LCDGzFiRLR/KRY04kJBRGpcxdO3qhGca47r8PyzZs3yDdFwnaYWs8ZfuF7LYQPWtmne8DAt8vXXX+/Gjx9vm3N+du/e3WcKUgNSTotT1KhV4FDHjh0zds92H1OmTMnIWkTgUAYbXxBAAAEEEEAAAQQQQAABBBBAAAEE8hSwjtRSBA7leUp3/vnnuwcffNBXV7/Y3nvv7QOGFDhk5fbbb3drrrmmfY0+FbzRv3//6PsTTzxRUOBGtGOWhaRry1LVB5yce+65fnOpMw7poOEAyOOPP9716NHDffvtt74fUds1iHPUqFFaLEngkD/Qb3/deOONPtOQvm+zzTZu5MiRtinnp4KSbMosZR466KCDMuprKi9NYaZSXZYcDQxV1nyVCy+80O24445+Oddf4WBZBZhtv/32idXnzp3runTp4rcV0g+tHT755BOnZ62+bZVcA1h9heCv5557zp144ok+4Gf48OF+i6ZUs37wtm3b+qz0b731VhQwJn/dV7169YIj/f9iOKDVpmirUokVCCCAAAIIIIAAAghUsIC1dwkcSngJ4qN6VEXZc5R1x0qxgUMapaEpuaysu+66UWpdWxd+ho06W6/RNMstt5x9zfqp0UKHHHJIxvbTTjvNz8GdsfL3L2qIqUEWFo2kUcrjsChzkBqnaqDrHLmKjRCJZxxSQE88XW3Scfbcc0+3ySabVGnMarSJNabD/W644QY/asbWEThkEnwigAACCCCAAAIIIIAAAggggAACCBQiYB2pNRk4FA7as2molDlHASIqu+yyixsyZEjW2zjggAP8lE6qoCnI1llnnax1C92QdG3ZjpF2xqEwM07Lli19QE84KFJZ0y1LfL5TleXKOGT3+cYbb7ijjz7afy0km80999zjLrroIr+fgsCUrT0sCxcudB06dPCrlOlH03NlmwYtzLaUK3tQePzrrrvOXXnllX6Vgsv0niQVTcum4B8Vy3z08ccfO5tCT/25a6yxRtKuft0FF1wQZfhRBqEDDzwwa13boEGnmm5N57jpppuc+svDILCDDz7YnXDCCVGAkPqrFfykkjTtWjiIVbMFaHAuBQEEEEAAAQQQQAABBDIFrL1L4FCmi1uwYIHT6JRwnnBVCRuZ+l5s4JBGoWhUkBU1BB944IGso3/Chq3to4Cdhg0b2tesn/PmzXO77757xnbN/63RQUklTOVr2x9//HG37LLL+pTH3333na32n40bN/YjSV5++WX3/PPPR43CsJIa0j179nSnnHKKmz59erRJI0jU6Mu3qDE9e/bsqLpGvmie8nhRet4wSxOBQ3EhviOAAAIIIIAAAggggAACCCCAAAII5CNgHanFBg5pejJlF1dp0qSJ22CDDbKePsyyoj489eUpAGj06NF+n+qmIFM2m2effdbXTQqqiJ+42GuLH8++px04pPNo6i8FC6lYJiDLLnP//fe7lVde2W8L+1cVgBIOtOzWrZufQk4V8wkc0vE1RZlKIYFD6s9VBikVnV/XES/hwEvdjwKi4kXTcCk7u2X10RR1SdPexfcLg6pyBZ9psKgyBalYv66m+lJAkMoxxxzj//gvCX+F06ipP3j//fdPqJW5yq5t3333dRr0qqL+ZvXDqyiIqlGjRn5Zfy1atMhn4tIUaeoftgxJVkF92J06dfJfN954Yx9cZNv4RAABBBBAAAEEEEAAgf8XsPYugUO/vxEKGNKUXhqpYPNihy+LGrkK7rFSbOCQgoTio4Kyjb5Qw10jQNRQslJIY0eNKI3qsLml7RhJKWzVyBw0aJBV8Z9hOlrNwa6GdFjCeda1Xg1WnU+NNisa2aPgq7DTQ9uUclejXBSUZEWBQRMnTrSv/lOdHU2bNs1I02wV4tmQwhE/VicpcOjFF190yppkRRmVtthiC/vKJwIIIIAAAggggAACCCCAAAIIIIAAAs46UosNHPr000+jDODKgKI+raTplRQUooF21penLDEKMgqDVcIpuJIeURh8kk9QSbHXlnQNWlcTgUMaPKhBhCrKoDNp0iS/vN1220XTgmlFKQOHlOlGz08lfh6/Mstfb775ps+qo80a6GnTuIXVx40b5yZMmOBXKQOPMk7Fi03ppfWFnD/M4KO+7ttuu82tuuqq8cP7gbUzZ87065WFX9n49V0DblWqe//CQZ1jxozxWYv8jln++umnn9x+++3n+5OVlUmBdSrhfU6dOtUts8wyGUewzFoK4tL+YdGUabZuhx12cCNGjAg3s4wAAggggAACCCCAAAK/CVh7t2IDhzQSRA10K2EWG1tnn0lpY4sNHJozZ05iph01YjS3dIsWLZxGRXz44YduypQpUUeBXVOhc4JrZJH+xItGlmy00UZ+hIYCbpRZKF50r927d/erFWDVsWPHjCqaK1pThlmDTtOWaSSKjXhRZUupHDb27CDaX/N5K2XyjBkznAKRwoAeNWI14qR+/foZjUXbX8+xb9++rlmzZn76N428Cs+tekmBQ/HsRzQgTZRPBBBAAAEEEEAAAQQQQAABBBBAAAETsI7UYgOHdDwLdNDy0KFD3c4776zFjHLLLbf4wXdaqQF9yjqjfrEwuEfbLr30UrfVVltpMaNY5hat1HRSFkij7wpGsvtRX1qYzaaYa9Oxk0pNBA79/PPPbs8996zSH6ipvPbYY4/oskoROKTprzRVm4JprFhGKPue6zPMFJRtYKim6urRo0d0GA2w1NRgVuKDTJU9vmvXrrbZ9ym/9tpr/rsCbTbffPNomxbOPvvsqA84KSt9GIilPnQF8mi6tDDoSMe5/PLLEwdhhgFuqhdmfdL3pKKpyRRgpIz1NgWc6n3xxReuc+fOfhcNLFUfshVltVf/rsrYsWOr3KcGjarPWCVbdie/kb8QQAABBBBAAAEEEKhgAWsfVmzgUL7PXqMnLr74Yt84D/cpNnBIx1Kj/ZJLLgkPm9eyOgyuvfZat9xyy+VVX5UUiHPYYYdVaUBXdwA1DjXCxYKCVD+cPzvcX3U1OiQetKM6YQM3HJET7p9tOd74jQf8ZNsvXE/gUKjBMgIIIIAAAggggAACCCCAAAIIIIBAvgLWkVqKwCENmNM0TlY0YNEGEX788cdO00HdfvvtttlPF6VBd1Y0zZWmu7LSu3dvv78ydX/55Zd+8J0yFFmJZxzXNmX/VlGwjYJIrBR7bXac8LMmAod0Pt3nvffeG57aW6y44orRunwDhzRoMtxPB1i4cKEPulJQT9j3ueWWW/ogLwXW5FvCbDzxLPd2jHCqOQV/qV+3bdu2Tu+InqFlA9I2BZaFmXg0HZ7eCxX11+qdCku4Xes1xZem9GrcuLEftGnT4Wmb3re9995bi77o3dH5rfTr188HD+n9U5CP7kfXY6W6Kc1U75tvvommMtMzCrP+a3v4ziu4Tf28GrhqvyNltNeA2QYNGqh6VNQfrYAiFU27Fv6OokosIIAAAggggAACCCBQ4QLW3iVwKMeLoEaTGmnhNFpWvRSBQzqWGi2aMzrfosbeFVdc4bPz5LuP1QtHWdi6XJ9qpI0cOdK1bt06o5pGlygDUZgVKKNC7ItlG7LV8+fP93NTq6FdXdF5lAI3TNusl1fPxdI1Jx1DDekXXngh2kTgUETBAgIIIIAAAggggAACCCCAAAIIIIBAAQLWkVqKwCFlnFHmbmV1qa4kZUlRv5qCMRQ4UV1RMMjgwYMzqinoI1vgULHXlnGi37/UVODQM8884/r37x9dQlJm8XwDh6KDVLNw6KGH+n7LeMBKNbu5W2+91WeLUj0F6agfM170nE844QT3+uuvxzdlfE/KtBMGBiUFDukAmr5u0KBBGceKf5HhsGHDfLYh26YM+Zo+LVe/rNXV9Gbqx67ORwY333yziw8eteN89dVXvi856Z1X//X48eMzMjLZfmGA1mOPPVZE4y/WAAAPF0lEQVTQIFw7Bp8IIIAAAggggAACCNR1AWvvVkTgkEaEdOjQodpnqoaGRmm0adPGp7FVuthsRaMpwsCU9u3bu+HDh2ernnO9pgjTiJ5cHQa6tqOOOsqPvkgKZMp5gmCjUhor7asaS7mKOhb69Onjp/9KqqcRJGqUxUfyhHWVGUlBQzvttFNG4I/qqCNCU7Cp8RiO0rH91agdMmRI1vmv582b55QGWKlv40UZidZaay0fXGTblJLXRpjYurDxqHUaTXTRRRfZZj4RQAABBBBAAAEEEEAAAQQQQAABBBCIpvYqReCQOH/88UefVUhTXiX1i6kfUP1e2TKkaFCfpnbS/klF+yvoRJli4plwwiws2q5sLmEp9trCY2l56tSpbuDAgX61pt/SAMHFKcpKY9OOJU3xtWDBAr/dPBUUs9tuu2WcSv2YykykoiAjZa+xomurbpCj+it1bk0bpini2rVrZ7sX9PnOO++4ww8/3O+jabk0PVdS0T3LToNB40X9rsoKr+CceJk1a5br1auXX50tcEgb5TFq1KjEd1BBa8patPTSS8cP7zRVmvqX77zzzirbtELvn57zPvvsk7h/uJMCkPQcdD833HBD1vp6L/Xuqi949uzZvr78ZbfyyiuHh/TLusZddtnFL2+33XZ+RoEqlViBAAIIIIAAAggggAACUXu3IgKHasvzVjphpZtVcI/+NGzY0LVo0SL6U0zAUNxAjehXX33VZw3SqA11IqghqbSyf/rTn/IegaFRJm+++aZ77733fMpeXfNqq63m/+jlindOxK9DU5vNmTPH76/rUAeM5lZv3rx5lWCj+L76rgAmNRbVkNZ+arhXN4ol6TisQwABBBBAAAEEEEAAAQQQQAABBBBAIEnARmCWKnDIzqGsMgpW0fEVGKGBcPqjfrHq+tR0jK+//tq99dZbfn/1K2o/BWCoT65Ro0Z2msX6LPbaFuukFbSTDUpVf6wyIdWvXz/r3av/9ZVXXvF9sAqS0TNWdvV83pGsB/19w6JFi3y/rIKNFJCmvmgNwGzSpEl1u7q5c+f6fl1lAlJfc7NmzXzfrq5PwUP5FA2oVbYoBWIlBUHlc4ykOpqe7YILLvCbFBy19dZbJ1VjHQIIIIAAAggggAACFS9g7V0Chyr+VQAAAQQQQAABBBBAAAEEEEAAAQQQQAABBLIJWEdqqQOHsp2P9XVfYPr06U5Z01WUxV7Z7CmlE9B0fprmLSk7VenOwpEQQAABBBBAAAEEEKj9AtbeJXCo9j9L7gABBBBAAAEEEEAAAQQQQAABBBBAAAEEUhKwjlQCh1ICrtDDnnzyyW7GjBk+e9C4ceMqVKH0t62p3fr27esPPHr0aNe2bdvSn4QjIoAAAggggAACCCBQRwSsvUvgUB15oNwGAggggAACCCCAAAIIIIAAAggggAACCJRewDpSCRwqvW0lH3HOnDnu4IMP9gSXXnqpn66rkj1Kde82DVynTp3c4MGDS3VYjoMAAggggAACCCCAQJ0UsPYugUN18vFyUwgggAACCCCAAAIIIIAAAggggAACCCBQCgHrSCVwqBSaHCMUmDBhglO2oc0228x/httYLlxg1qxZrlevXn7Hu+++2zVt2rTwg7AHAggggAACCCCAAAIVJGDtXQKHKuihc6sIIIAAAggggAACCCCAAAIIIIAAAgggUJiAdaQSOFSYG7WrF1iwYIEbPny4mz9/vjv11FNdkyZNqt+JGlkFFCw0ffp017FjR7f77rtnrccGBBBAAAEEEEAAAQQQ+H8Ba+8SOMQbgQACCCCAAAIIIIAAAggggAACCCCAAAIIZBGwjlQCh7IAsRoBBBBAAAEEEEAAAQQQQKBWClh7l8ChWvn4uGgEEEAAAQQQQAABBBBAAAEEEEAAAQQQqAkB60glcKgmtDkHAggggAACCCCAAAIIIIBATQlYe5fAoZoS5zwIIIAAAggggAACCCCAAAIIIIAAAgggUOsErCOVwKFa9+i4YAQQQAABBBBAAAEEEEAAgRwC1t4lcCgHEpsQQAABBBBAAAEEEEAAAQQQQAABBBBAoLIFrCOVwKHKfg+4ewQQQAABBBBAAAEEEECgrglYe5fAobr2ZLkfBBBAAAEEEEAAAQQQQAABBBBAAAEEECiZgHWkEjhUMlIOhAACCCCAAAIIIIAAAgggUAYC1t4lcKgMHgaXgAACCCCAAAIIIIAAAggggAACCCCAAALlKWAdqQQOlefz4aoQQAABBBBAAAEEEEAAAQQWT8DauwQOLZ4feyGAAAIIIIAAAggggAACCCCAAAIIIIBABQhYRyqBQxXwsLlFBBBAAAEEEEAAAQQQQKCCBKy9S+BQBT10bhUBBBBAAAEEEEAAAQQQQAABBBBAAAEEChOwjlQChwpzozYCCCCAAAIIIIAAAggggEB5C1h7l8Ch8n5OXB0CCCCAAAIIIIAAAggggAACCCCAAAIILEEB60glcGgJPgROjQACCCCAAAIIIIAAAgggUHIBa+8SOFRyWg6IAAIIIIAAAggggAACCCCAAAIIIIAAAnVFwDpSCRyqK0+U+0AAAQQQQAABBBBAAAEEEJCAtXcJHOJ9QAABBBBAAAEEEEAAAQQQQAABBBBAAAEEsghYRyqBQ1mAWI0AAggggAACCCCAAAIIIFArBay9S+BQrXx8XDQCCCCAAAIIIIAAAggggAACCCCAAAII1ISAdaQSOFQT2pwDAQQQQAABBBBAAAEEEECgpgSsvUvgUE2Jcx4EEEAAAQQQQAABBBBAAAEEEEAAAQQQqHUC1pFK4FCte3RcMAIIIIAAAggggAACCCCAQA4Ba+8SOJQDiU0IIIAAAggggAACCCCAAAIIIIAAAgggUNkC1pFK4FBlvwfcPQIIIIAAAggggAACCCBQ1wSsvUvgUF17stwPAggggAACCCCAAAIIIIAAAggggAACCJRMwDpSCRwqGSkHQgABBBBAAAEEEEAAAQQQKAMBa+8SOFQGD4NLQAABBBBAAAEEEEAAAQQQQAABBBBAAIHyFLCOVAKHyvP5cFUIIIAAAggggAACCCCAAAKLJ2DtXQKHFs+PvRBAAAEEEEAAAQQQQAABBBBAAAEEEECgAgSsI5XAoQp42NwiAggggAACCCCAAAIIIFBBAtbeJXCogh46t4oAAggggAACCCCAAAIIIIAAAggggAAChQlYRyqBQ4W5URsBBBBAAAEEEEAAAQQQQKC8Bay9S+BQeT8nrg4BBBBAAAEEEEAAAQQQQAABBBBAAAEElqCAdaQSOLQEHwKnRgABBBBAAAEEEEAAAQQQKLmAtXcJHCo5LQdEAAEEEEAAAQQQQAABBBBAAAEEEEAAgboiYB2pBA7VlSfKfSCAAAIIIIAAAggggAACCEjA2rtlHzjE40IAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAIHSCxA4VHpTjogAAggggAACCCCAAAIIIIAAAggggAACCCCAAAIIIIAAAggggAACCCCAQNkLlH3gUCkvsOyfBheIAAIIIIAAAggggAACCCCAAAIIIIAAAmUl8M477/jroZ+yrB4LF4MAAggggAACCCCAAAIIIFCkQBrt3XqLfitFXle0exoXGB2cBQQQQAABBBBAAAEEEEAAAQQQQAABBBBAIA8B+inzQKIKAggggAACCCCAAAIIIIBArRNIo71L4FCtew24YAQQQAABBBBAAAEEEEAAAQQQQAABBBDIJZBGR2qu87ENAQQQQAABBBBAAAEEEEAAgZoQSKO9S+BQTTw5zoEAAggggAACCCCAAAIIIIAAAggggAACNSaQRkdqjV08J0IAAQQQQAABBBBAAAEEEEAgi0Aa7V0Ch7JgsxoBBBBAAAEEEEAAAQQQQAABBBBAAAEEaqdAGh2ptVOCq0YAAQQQQAABBBBAAAEEEKhLAmm0dwkcqktvCPeCAAIIIIAAAggggAACCCCAAAIIIIAAAi6NjlRYEUAAAQQQQAABBBBAAAEEEFjSAmm0dwkcWtJPlfMjgAACCCCAAAIIIIAAAggggAACCCCAQEkF0uhILekFcjAEEEAAAQQQQAABBBBAAAEEFkMgjfYugUOL8SDYBQEEEEAAAQQQQAABBBBAAAEEEEAAAQTKVyCNjtTyvVuuDAEEEEAAAQQQQAABBBBAoFIE0mjvEjhUKW8P94kAAggggAACCCCAAAIIIIAAAggggECFCKTRkVohdNwmAggggAACCCCAAAIIIIBAGQuk0d4lcKiMHziXhgACCCCAAAIIIIAAAggggAACCCCAAAKFC6TRkVr4VbAHAggggAACCCCAAAIIIIAAAqUVSKO9S+BQaZ8RR0MAAQQQQAABBBBAAAEEEEAAAQQQQACBJSyQRkfqEr4lTo8AAggggAACCCCAAAIIIICAS6O9S+AQLxYCCCCAAAIIIIAAAggggAACCCCAAAII1CmBNDpS6xQQN4MAAggggAACCCCAAAIIIFArBdJo7xI4VCtfBS4aAQQQQAABBBBAAAEEEEAAAQQQQAABBLIJpNGRmu1crEcAAQQQQAABBBBAAAEEEECgpgTSaO8SOFRTT4/zIIAAAggggAACCCCAAAIIIIAAAggggECNCKTRkVojF85JEEAAAQQQQAABBBBAAAEEEMghkEZ7l8ChHOBsQgABBBBAAAEEEEAAAQQQQAABBBBAAIHaJ5BGR2rtU+CKEUAAAQQQQAABBBBAAAEE6ppAGu1dAofq2lvC/SCAAAIIIIAAAggggAACCCCAAAIIIFDhAml0pFY4KbePAAIIIIAAAggggAACCCBQBgJptHcJHCqDB8slIIAAAggggAACCCCAAAIIIIAAAggggEDpBNLoSC3d1XEkBBBAAAEEEEAAAQQQQAABBBZPII32LoFDi/cs2AsBBBBAAAEEEEAAAQQQQAABBBBAAAEEylQgjY7UMr1VLgsBBBBAAAEEEEAAAQQQQKCCBNJo7xI4VEEvELeKAAIIIIAAAggggAACCCCAAAIIIIBAJQik0ZFaCW7cIwIIIIAAAggggAACCCCAQHkLpNHeJXCovJ85V4cAAggggAACCCCAAAIIIIAAAggggAACBQqk0ZFa4CVQHQEEEEAAAQQQQAABBBBAAIGSC6TR3v0/jfGT8TDQeIYAAAAASUVORK5CYII=\n",
")"
],
"metadata": {
"id": "6LX998zzDDaa"
}
},
{
"cell_type": "markdown",
"source": [
"Remove the folder `my_dir`"
],
"metadata": {
"id": "CJd-_xeU8wTH"
}
},
{
"cell_type": "code",
"source": [
"!hdfs dfs -rm -r hdfs://localhost:8902/my_dir"
],
"metadata": {
"id": "fug-vO6d8y00",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "cdf39571-e507-4d86-c84e-743a194609e8"
},
"execution_count": 47,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Deleted hdfs://localhost:8902/my_dir\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Now the DFS used should be back to ~$4$MB."
],
"metadata": {
"id": "uJnm8IIdVbXw"
}
},
{
"cell_type": "markdown",
"source": [
"\n",
"\n"
],
"metadata": {
"id": "CU3gbnQSTL47"
}
},
{
"cell_type": "markdown",
"source": [
"## The default filesystem\n",
"\n",
"By default, `hdfs` will use the local filesystem so you need to prepend `hdfs://...` if you want to use the HDFS filesystem.\n",
"\n",
"If you do not want to use the prefix `hdfs://localhost:8902/` in the filenames, you could set the property `fs.defaultFS` in `core-site.xml` or else use the option `-fs` like this:\n",
"\n",
" hdfs dfs -fs hdfs://localhost:8902/"
],
"metadata": {
"id": "DQQHj14wVnum"
}
},
{
"cell_type": "code",
"source": [
"!hdfs dfs -fs hdfs://localhost:8902/ -ls /"
],
"metadata": {
"id": "iTvSHQntiReX",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "132c3dd4-56ac-415a-988b-157fdefc611b"
},
"execution_count": 48,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Found 2 items\n",
"drwxrwxrwx - root supergroup 0 2024-08-04 19:12 /content\n",
"drwxr-xr-x - root supergroup 0 2024-08-04 19:12 /user\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"This is the same as"
],
"metadata": {
"id": "PEteAOIhWOXo"
}
},
{
"cell_type": "code",
"source": [
"!hdfs dfs -ls hdfs://localhost:8902/"
],
"metadata": {
"id": "oh2ddzKYWQEE",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "3a19d10c-d679-4c54-ebb7-521f8a475765"
},
"execution_count": 49,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Found 2 items\n",
"drwxrwxrwx - root supergroup 0 2024-08-04 19:12 hdfs://localhost:8902/content\n",
"drwxr-xr-x - root supergroup 0 2024-08-04 19:12 hdfs://localhost:8902/user\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"And also the same as"
],
"metadata": {
"id": "iv1KeqOYWkRx"
}
},
{
"cell_type": "code",
"source": [
"!hdfs dfs -D fs.defaultFS=hdfs://localhost:8902/ -ls /"
],
"metadata": {
"id": "Q67buUSoWmwS",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "65dd651d-22d2-4001-aa28-3c3ae31efb08"
},
"execution_count": 50,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Found 2 items\n",
"drwxrwxrwx - root supergroup 0 2024-08-04 19:12 /content\n",
"drwxr-xr-x - root supergroup 0 2024-08-04 19:12 /user\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"With the option `-D` we can set any variable on the fly (in this case we set `fs.defaultFS` to be the HDFS filesystem).\n",
"\n",
"**Note:** the `-D` option should come before any other option."
],
"metadata": {
"id": "hofC7_S1mmvm"
}
},
{
"cell_type": "markdown",
"source": [
"If you configure the property `fs.defaultFS` in `core-site.xml`, you can also use `hdfs dfs -ls /`."
],
"metadata": {
"id": "ImKd8-1QW5X5"
}
},
{
"cell_type": "code",
"source": [
"with open(os.environ['HADOOP_HOME']+'/etc/hadoop/core-site.xml', 'w') as file:\n",
" file.write(\"\"\"\n",
" \n",
" fs.defaultFS\n",
" hdfs://localhost:8902/\n",
" \n",
"\"\"\")"
],
"metadata": {
"id": "ed3EKZeuBH--"
},
"execution_count": 51,
"outputs": []
},
{
"cell_type": "code",
"source": [
"!cat $HADOOP_HOME/etc/hadoop/core-site.xml"
],
"metadata": {
"id": "CTzAAUV3eaHg",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "23f60115-c315-47da-b3aa-35f108ff2003"
},
"execution_count": 52,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"\n",
" \n",
" fs.defaultFS\n",
" hdfs://localhost:8902/\n",
" \n",
""
]
}
]
},
{
"cell_type": "code",
"source": [
"!hdfs dfs -ls /"
],
"metadata": {
"id": "9NUz2NQpXHzj",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "f985d802-d777-49d5-f5cc-af7a93d03f02"
},
"execution_count": 53,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Found 2 items\n",
"drwxrwxrwx - root supergroup 0 2024-08-04 19:12 /content\n",
"drwxr-xr-x - root supergroup 0 2024-08-04 19:12 /user\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Let us set the local filesystem as default (`file:///` means the local filesystem `file://` and the extra slash `/` indicates the root folder)."
],
"metadata": {
"id": "eg1FRosv9Pp5"
}
},
{
"cell_type": "code",
"source": [
"with open(os.environ['HADOOP_HOME']+'/etc/hadoop/core-site.xml', 'w') as file:\n",
" file.write(\"\"\"\n",
"\n",
" \n",
" fs.defaultFS\n",
" file:///\n",
" \n",
"\"\"\")"
],
"metadata": {
"id": "Eak0oQdk9Xap"
},
"execution_count": 54,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Now run `!hdfs dfs -ls /` as before. This time we are going to be listing the local filesystem and not the HDFS."
],
"metadata": {
"id": "vzOLLTaP9-3a"
}
},
{
"cell_type": "code",
"source": [
"!hdfs dfs -ls /"
],
"metadata": {
"id": "uftFbN7P-Mfb",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "ccabc915-adb4-4b89-85a4-45083b9904c1"
},
"execution_count": 55,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Found 29 items\n",
"-rwxr-xr-x 1 root root 0 2024-08-04 18:59 /.dockerenv\n",
"-rw-r--r-- 1 root root 17294 2023-11-10 04:55 /NGC-DL-CONTAINER-LICENSE\n",
"drwxr-xr-x - root root 20480 2024-08-01 13:20 /bin\n",
"drwxr-xr-x - root root 4096 2022-04-18 10:28 /boot\n",
"drwxr-xr-x - root root 4096 2024-08-04 19:12 /content\n",
"-rw-r--r-- 1 root root 4332 2023-11-10 04:56 /cuda-keyring_1.0-1_all.deb\n",
"drwxr-xr-x - root root 4096 2024-08-01 13:41 /datalab\n",
"drwxr-xr-x - root root 360 2024-08-04 18:59 /dev\n",
"drwxr-xr-x - root root 4096 2024-08-04 18:59 /etc\n",
"drwxr-xr-x - root root 4096 2022-04-18 10:28 /home\n",
"drwxr-xr-x - root root 4096 2024-08-04 18:59 /kaggle\n",
"drwxr-xr-x - root root 4096 2024-08-01 13:20 /lib\n",
"drwxr-xr-x - root root 4096 2024-08-01 13:16 /lib32\n",
"drwxr-xr-x - root root 4096 2024-08-01 13:16 /lib64\n",
"drwxr-xr-x - root root 4096 2023-10-04 02:08 /libx32\n",
"drwxr-xr-x - root root 4096 2023-10-04 02:08 /media\n",
"drwxr-xr-x - root root 4096 2023-10-04 02:08 /mnt\n",
"drwxr-xr-x - root root 4096 2024-08-01 13:54 /opt\n",
"dr-xr-xr-x - root root 0 2024-08-04 18:59 /proc\n",
"drwxr-xr-x - root root 4096 2024-08-01 13:22 /python-apt\n",
"drwx------ - root root 4096 2024-08-04 19:00 /root\n",
"drwxr-xr-x - root root 4096 2024-08-01 13:15 /run\n",
"drwxr-xr-x - root root 4096 2024-08-04 18:59 /sbin\n",
"drwxr-xr-x - root root 4096 2023-10-04 02:08 /srv\n",
"dr-xr-xr-x - root root 0 2024-08-04 18:59 /sys\n",
"drwxrwxrwt - root root 4096 2024-08-04 19:12 /tmp\n",
"drwxr-xr-x - root root 4096 2024-08-01 13:41 /tools\n",
"drwxr-xr-x - root root 4096 2024-08-01 13:42 /usr\n",
"drwxr-xr-x - root root 4096 2024-08-01 13:41 /var\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"I advise to get used to the fact that Hadoop interprets a file path as HDFS (`hdfs://`) vs. local (`file://`) depending on the setting of the variable `fs.defaultFS`, since this is often a source of confusion."
],
"metadata": {
"id": "EFLRoQ9K-vNo"
}
},
{
"cell_type": "markdown",
"source": [
"## Show status of HDFS with `hdfs dfsadmin`\n",
"\n",
"The command `hdfs dfsadmin` allows to run administration tasks on the Hadoop filesystem."
],
"metadata": {
"id": "-X2Gcpm_rsT3"
}
},
{
"cell_type": "code",
"source": [
"!hdfs dfsadmin -h"
],
"metadata": {
"id": "tVjiDpX6F04B",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "de550d62-4cd9-4aca-c90a-df1e8bb28351"
},
"execution_count": 56,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"h: Unknown command\n",
"Usage: hdfs dfsadmin\n",
"Note: Administrative commands can only be run as the HDFS superuser.\n",
"\t[-report [-live] [-dead] [-decommissioning] [-enteringmaintenance] [-inmaintenance] [-slownodes]]\n",
"\t[-safemode ]\n",
"\t[-saveNamespace [-beforeShutdown]]\n",
"\t[-rollEdits]\n",
"\t[-restoreFailedStorage true|false|check]\n",
"\t[-refreshNodes]\n",
"\t[-setQuota ...]\n",
"\t[-clrQuota ...]\n",
"\t[-setSpaceQuota [-storageType ] ...]\n",
"\t[-clrSpaceQuota [-storageType ] ...]\n",
"\t[-finalizeUpgrade]\n",
"\t[-rollingUpgrade []]\n",
"\t[-upgrade ]\n",
"\t[-refreshServiceAcl]\n",
"\t[-refreshUserToGroupsMappings]\n",
"\t[-refreshSuperUserGroupsConfiguration]\n",
"\t[-refreshCallQueue]\n",
"\t[-refresh [arg1..argn]\n",
"\t[-reconfig ]\n",
"\t[-printTopology]\n",
"\t[-refreshNamenodes datanode_host:ipc_port]\n",
"\t[-getVolumeReport datanode_host:ipc_port]\n",
"\t[-deleteBlockPool datanode_host:ipc_port blockpoolId [force]]\n",
"\t[-setBalancerBandwidth ]\n",
"\t[-getBalancerBandwidth ]\n",
"\t[-fetchImage ]\n",
"\t[-allowSnapshot ]\n",
"\t[-disallowSnapshot ]\n",
"\t[-provisionSnapshotTrash [-all]]\n",
"\t[-shutdownDatanode [upgrade]]\n",
"\t[-evictWriters ]\n",
"\t[-getDatanodeInfo ]\n",
"\t[-metasave filename]\n",
"\t[-triggerBlockReport [-incremental] [-namenode ]]\n",
"\t[-listOpenFiles [-blockingDecommission] [-path ]]\n",
"\t[-help [cmd]]\n",
"\n",
"Generic options supported are:\n",
"-conf specify an application configuration file\n",
"-D define a value for a given property\n",
"-fs specify default filesystem URL to use, overrides 'fs.defaultFS' property from configurations.\n",
"-jt specify a ResourceManager\n",
"-files specify a comma-separated list of files to be copied to the map reduce cluster\n",
"-libjars specify a comma-separated list of jar files to be included in the classpath\n",
"-archives specify a comma-separated list of archives to be unarchived on the compute machines\n",
"\n",
"The general command line syntax is:\n",
"command [genericOptions] [commandOptions]\n",
"\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"The comand `hdfs dfsadmin -report` shows the current status of the Hadoop filesystem. In order to run it on our MiniCluster HDFS we need to pass the option\n",
"\n",
" -fs hdfs://localhost:8902/\n",
"\n",
"\n",
" \n",
"Alternatively, we can configure the default filesystem (the URI of the namenode) in `core-site.xml` (see discussion in [The default filesystem)](#scrollTo=The_default_filesystem)."
],
"metadata": {
"id": "ZAo3uxzrB7ud"
}
},
{
"cell_type": "code",
"source": [
"!hdfs dfsadmin -fs hdfs://localhost:8902/ -report"
],
"metadata": {
"id": "ZEHoUeoxinbv",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "b9610113-c5f3-4d9f-e2f4-9855cfdaba8e"
},
"execution_count": 57,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Configured Capacity: 231316381696 (215.43 GB)\n",
"Present Capacity: 159504314437 (148.55 GB)\n",
"DFS Remaining: 159501991936 (148.55 GB)\n",
"DFS Used: 2322501 (2.21 MB)\n",
"DFS Used%: 0.00%\n",
"Replicated Blocks:\n",
"\tUnder replicated blocks: 0\n",
"\tBlocks with corrupt replicas: 0\n",
"\tMissing blocks: 0\n",
"\tMissing blocks (with replication factor 1): 0\n",
"\tLow redundancy blocks with highest priority to recover: 0\n",
"\tPending deletion blocks: 0\n",
"Erasure Coded Block Groups: \n",
"\tLow redundancy block groups: 0\n",
"\tBlock groups with corrupt internal blocks: 0\n",
"\tMissing block groups: 0\n",
"\tLow redundancy blocks with highest priority to recover: 0\n",
"\tPending deletion blocks: 0\n",
"\n",
"-------------------------------------------------\n",
"Live datanodes (1):\n",
"\n",
"Name: 127.0.0.1:39513 (localhost)\n",
"Hostname: 127.0.0.1\n",
"Decommission Status : Normal\n",
"Configured Capacity: 231316381696 (215.43 GB)\n",
"DFS Used: 2322501 (2.21 MB)\n",
"Non DFS Used: 71778512827 (66.85 GB)\n",
"DFS Remaining: 159501991936 (148.55 GB)\n",
"DFS Used%: 0.00%\n",
"DFS Remaining%: 68.95%\n",
"Configured Cache Capacity: 0 (0 B)\n",
"Cache Used: 0 (0 B)\n",
"Cache Remaining: 0 (0 B)\n",
"Cache Used%: 100.00%\n",
"Cache Remaining%: 0.00%\n",
"Xceivers: 0\n",
"Last contact: Sun Aug 04 19:13:22 UTC 2024\n",
"Last Block Report: Sun Aug 04 19:12:23 UTC 2024\n",
"Num of Blocks: 3\n",
"\n",
"\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"The information displayed by `hdfs dfsadmin` corresponds to what is presented in the [NameNode Web UI](#scrollTo=Start_the_NameNode_Web_UI)."
],
"metadata": {
"id": "_5KPExSrasE9"
}
},
{
"cell_type": "markdown",
"source": [
"# Test MapReduce\n",
"\n",
"Find the MapReduce examples that come with the Hadoop distribution."
],
"metadata": {
"id": "v1sqv4BItwPZ"
}
},
{
"cell_type": "markdown",
"source": [
"## Run the `pi` example"
],
"metadata": {
"id": "VFrcfoxvo7em"
}
},
{
"cell_type": "code",
"source": [
"!find . -name \"*examples*.jar\""
],
"metadata": {
"id": "Y6l_P08ovfZ9",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "9770c765-69ce-4028-eabb-b7e5307d9422"
},
"execution_count": 58,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"./hadoop-3.4.0/share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-3.4.0-test-sources.jar\n",
"./hadoop-3.4.0/share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-3.4.0-sources.jar\n",
"./hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.4.0.jar\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Check if the cluster is still running, if not you will need to restart it! (from the cell cell [Configure the MiniCluster's ports](#scrollTo=Configure_the_MiniCluster_s_ports))."
],
"metadata": {
"id": "15uYXe4hgiZl"
}
},
{
"cell_type": "code",
"source": [
"!lsof -n -i -P +c0 -sTCP:LISTEN -ac java"
],
"metadata": {
"id": "RLf3KzXGglLz",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "729403fd-ce8e-494f-ebfd-09bfd2d1ce9c"
},
"execution_count": 59,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME\n",
"java 3727 root 341u IPv4 119325 0t0 TCP 127.0.0.1:8901 (LISTEN)\n",
"java 3727 root 346u IPv4 120023 0t0 TCP 127.0.0.1:8902 (LISTEN)\n",
"java 3727 root 361u IPv4 120629 0t0 TCP 127.0.0.1:39513 (LISTEN)\n",
"java 3727 root 364u IPv4 120632 0t0 TCP 127.0.0.1:39281 (LISTEN)\n",
"java 3727 root 393u IPv4 122005 0t0 TCP 127.0.0.1:39089 (LISTEN)\n",
"java 3727 root 394u IPv4 122019 0t0 TCP 127.0.0.1:39631 (LISTEN)\n",
"java 3727 root 416u IPv4 125902 0t0 TCP *:8031 (LISTEN)\n",
"java 3727 root 434u IPv4 125459 0t0 TCP *:10033 (LISTEN)\n",
"java 3727 root 444u IPv4 130724 0t0 TCP *:19888 (LISTEN)\n",
"java 3727 root 445u IPv4 130725 0t0 TCP *:8088 (LISTEN)\n",
"java 3727 root 454u IPv4 130963 0t0 TCP 127.0.0.1:8900 (LISTEN)\n",
"java 3727 root 464u IPv4 131073 0t0 TCP *:8033 (LISTEN)\n",
"java 3727 root 484u IPv4 132396 0t0 TCP *:8030 (LISTEN)\n",
"java 3727 root 494u IPv4 132413 0t0 TCP 127.0.0.1:8903 (LISTEN)\n",
"java 3727 root 504u IPv4 132829 0t0 TCP 127.0.0.1:43121 (LISTEN)\n",
"java 3727 root 514u IPv4 131545 0t0 TCP 127.0.0.1:41277 (LISTEN)\n",
"java 3727 root 524u IPv4 131549 0t0 TCP *:40533 (LISTEN)\n",
"java 3727 root 525u IPv4 132834 0t0 TCP 127.0.0.1:40903 (LISTEN)\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Here's ChatGPT 3.5's poem inspired by\n",
"`lsof -n -i -P +c0 -sTCP:LISTEN`:"
],
"metadata": {
"id": "-aMDAVoXiGiF"
}
},
{
"cell_type": "code",
"source": [
"# @title\n",
"from IPython.core.display import HTML\n",
"HTML(\"\"\"\n",
"
\n",
"\n",
"\n",
"\n",
"
\n",
"In the realm of networks, where processes twine, \n",
"A command unfolds, a symphony of lines. \n",
"\"Lsof,\" it whispers, with a mystic hum, \n",
"A dance of flags, each one has its own drum. \n",
"
\n",
"\"-n -i -P,\" the conductor commands, \n",
"Navigate swiftly, across distant lands. \n",
"\"+c0\" echoes softly, a chorus of glee, \n",
"Embrace all processes, as far as eyes can see. \n",
"
\n",
"\"-sTCP:LISTEN,\" a stanza profound, \n",
"Seeking the echoes of ports, a network's sound. \n",
"Processes in repose, in a state so keen, \n",
"A tapestry of LISTEN, a poetic scene. \n",
"
\n",
"In the realm of networks, where processes twine, \n",
"A command unfolds, a symphony of lines. \n",
"\"Lsof,\" it whispers, with a mystic hum, \n",
"A dance of flags, each one has its own drum. \n",
"
\n",
"\"-n -i -P,\" the conductor commands, \n",
"Navigate swiftly, across distant lands. \n",
"\"+c0\" echoes softly, a chorus of glee, \n",
"Embrace all processes, as far as eyes can see. \n",
"
\n",
"\"-sTCP:LISTEN,\" a stanza profound, \n",
"Seeking the echoes of ports, a network's sound. \n",
"Processes in repose, in a state so keen, \n",
"A tapestry of LISTEN, a poetic scene. \n",
"
\n"
]
},
"metadata": {},
"execution_count": 60
}
]
},
{
"cell_type": "markdown",
"source": [
"Apart from this poetic digression, I consider `lsof` an exceptionally valuable command."
],
"metadata": {
"id": "bqwCUH7V-vB9"
}
},
{
"cell_type": "markdown",
"source": [
"Use the following command to get the list of available examples in the jar file."
],
"metadata": {
"id": "seyyW_WskaLB"
}
},
{
"cell_type": "code",
"source": [
"!yarn jar ./hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.4.0.jar"
],
"metadata": {
"id": "MEoD8rs7eV-y",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "d268cbc5-8b0e-40cd-ac6f-b74aec6719a9"
},
"execution_count": 61,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"An example program must be given as the first argument.\n",
"Valid program names are:\n",
" aggregatewordcount: An Aggregate based map/reduce program that counts the words in the input files.\n",
" aggregatewordhist: An Aggregate based map/reduce program that computes the histogram of the words in the input files.\n",
" bbp: A map/reduce program that uses Bailey-Borwein-Plouffe to compute exact digits of Pi.\n",
" dbcount: An example job that count the pageview counts from a database.\n",
" distbbp: A map/reduce program that uses a BBP-type formula to compute exact bits of Pi.\n",
" grep: A map/reduce program that counts the matches of a regex in the input.\n",
" join: A job that effects a join over sorted, equally partitioned datasets\n",
" multifilewc: A job that counts words from several files.\n",
" pentomino: A map/reduce tile laying program to find solutions to pentomino problems.\n",
" pi: A map/reduce program that estimates Pi using a quasi-Monte Carlo method.\n",
" randomtextwriter: A map/reduce program that writes 10GB of random textual data per node.\n",
" randomwriter: A map/reduce program that writes 10GB of random data per node.\n",
" secondarysort: An example defining a secondary sort to the reduce.\n",
" sort: A map/reduce program that sorts the data written by the random writer.\n",
" sudoku: A sudoku solver.\n",
" teragen: Generate data for the terasort\n",
" terasort: Run the terasort\n",
" teravalidate: Checking results of terasort\n",
" wordcount: A map/reduce program that counts the words in the input files.\n",
" wordmean: A map/reduce program that counts the average length of the words in the input files.\n",
" wordmedian: A map/reduce program that counts the median length of the words in the input files.\n",
" wordstandarddeviation: A map/reduce program that counts the standard deviation of the length of the words in the input files.\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Let us run the `pi` example (here we call it without arguments in order to get a usage message) through `yarn`."
],
"metadata": {
"id": "66tQzkuwfOeX"
}
},
{
"cell_type": "code",
"source": [
"!yarn jar ./hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.4.0.jar pi"
],
"metadata": {
"id": "XYumkNgNfSnR",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "24cfebd7-812f-462d-e19c-a03a336de7b5"
},
"execution_count": 62,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Usage: org.apache.hadoop.examples.QuasiMonteCarlo \n",
"Generic options supported are:\n",
"-conf specify an application configuration file\n",
"-D define a value for a given property\n",
"-fs specify default filesystem URL to use, overrides 'fs.defaultFS' property from configurations.\n",
"-jt specify a ResourceManager\n",
"-files specify a comma-separated list of files to be copied to the map reduce cluster\n",
"-libjars specify a comma-separated list of jar files to be included in the classpath\n",
"-archives specify a comma-separated list of archives to be unarchived on the compute machines\n",
"\n",
"The general command line syntax is:\n",
"command [genericOptions] [commandOptions]\n",
"\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"The command takes `[genericOptions]` and `[commandOptions]`.\n",
"\n",
"The command options are:\n",
"\n",
"* `nMaps`, the number of mappers\n",
"* `nSamples`, the number of iterations per mapper\n"
],
"metadata": {
"id": "vTGLRYl9GkGw"
}
},
{
"cell_type": "code",
"source": [
"!yarn jar ./hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.4.0.jar pi \\\n",
" 5 1000"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "PBt5b-JKfiyt",
"outputId": "48171141-f7b0-43c6-ee88-96492ce26293"
},
"execution_count": 63,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Number of Maps = 5\n",
"Samples per Map = 1000\n",
"Wrote input for Map #0\n",
"Wrote input for Map #1\n",
"Wrote input for Map #2\n",
"Wrote input for Map #3\n",
"Wrote input for Map #4\n",
"Starting Job\n",
"2024-08-04 19:13:31,625 INFO impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties\n",
"2024-08-04 19:13:31,799 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).\n",
"2024-08-04 19:13:31,799 INFO impl.MetricsSystemImpl: JobTracker metrics system started\n",
"2024-08-04 19:13:31,979 INFO input.FileInputFormat: Total input files to process : 5\n",
"2024-08-04 19:13:32,000 INFO mapreduce.JobSubmitter: number of splits:5\n",
"2024-08-04 19:13:32,402 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_local2011001580_0001\n",
"2024-08-04 19:13:32,402 INFO mapreduce.JobSubmitter: Executing with tokens: []\n",
"2024-08-04 19:13:32,687 INFO mapreduce.Job: The url to track the job: http://localhost:8080/\n",
"2024-08-04 19:13:32,688 INFO mapreduce.Job: Running job: job_local2011001580_0001\n",
"2024-08-04 19:13:32,699 INFO mapred.LocalJobRunner: OutputCommitter set in config null\n",
"2024-08-04 19:13:32,715 INFO output.PathOutputCommitterFactory: No output committer factory defined, defaulting to FileOutputCommitterFactory\n",
"2024-08-04 19:13:32,717 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2\n",
"2024-08-04 19:13:32,717 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false\n",
"2024-08-04 19:13:32,719 INFO mapred.LocalJobRunner: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter\n",
"2024-08-04 19:13:32,814 INFO mapred.LocalJobRunner: Waiting for map tasks\n",
"2024-08-04 19:13:32,815 INFO mapred.LocalJobRunner: Starting task: attempt_local2011001580_0001_m_000000_0\n",
"2024-08-04 19:13:32,870 INFO output.PathOutputCommitterFactory: No output committer factory defined, defaulting to FileOutputCommitterFactory\n",
"2024-08-04 19:13:32,870 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2\n",
"2024-08-04 19:13:32,871 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false\n",
"2024-08-04 19:13:32,918 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]\n",
"2024-08-04 19:13:32,925 INFO mapred.MapTask: Processing split: file:/content/QuasiMonteCarlo_1722798810530_1260429884/in/part0:0+118\n",
"2024-08-04 19:13:33,034 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)\n",
"2024-08-04 19:13:33,034 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100\n",
"2024-08-04 19:13:33,034 INFO mapred.MapTask: soft limit at 83886080\n",
"2024-08-04 19:13:33,034 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600\n",
"2024-08-04 19:13:33,034 INFO mapred.MapTask: kvstart = 26214396; length = 6553600\n",
"2024-08-04 19:13:33,040 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer\n",
"2024-08-04 19:13:33,065 INFO mapred.LocalJobRunner: \n",
"2024-08-04 19:13:33,066 INFO mapred.MapTask: Starting flush of map output\n",
"2024-08-04 19:13:33,066 INFO mapred.MapTask: Spilling map output\n",
"2024-08-04 19:13:33,066 INFO mapred.MapTask: bufstart = 0; bufend = 18; bufvoid = 104857600\n",
"2024-08-04 19:13:33,066 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600\n",
"2024-08-04 19:13:33,075 INFO mapred.MapTask: Finished spill 0\n",
"2024-08-04 19:13:33,119 INFO mapred.Task: Task:attempt_local2011001580_0001_m_000000_0 is done. And is in the process of committing\n",
"2024-08-04 19:13:33,124 INFO mapred.LocalJobRunner: Generated 1000 samples.\n",
"2024-08-04 19:13:33,124 INFO mapred.Task: Task 'attempt_local2011001580_0001_m_000000_0' done.\n",
"2024-08-04 19:13:33,149 INFO mapred.Task: Final Counters for attempt_local2011001580_0001_m_000000_0: Counters: 17\n",
"\tFile System Counters\n",
"\t\tFILE: Number of bytes read=282496\n",
"\t\tFILE: Number of bytes written=997502\n",
"\t\tFILE: Number of read operations=0\n",
"\t\tFILE: Number of large read operations=0\n",
"\t\tFILE: Number of write operations=0\n",
"\tMap-Reduce Framework\n",
"\t\tMap input records=1\n",
"\t\tMap output records=2\n",
"\t\tMap output bytes=18\n",
"\t\tMap output materialized bytes=28\n",
"\t\tInput split bytes=128\n",
"\t\tCombine input records=0\n",
"\t\tSpilled Records=2\n",
"\t\tFailed Shuffles=0\n",
"\t\tMerged Map outputs=0\n",
"\t\tGC time elapsed (ms)=23\n",
"\t\tTotal committed heap usage (bytes)=354418688\n",
"\tFile Input Format Counters \n",
"\t\tBytes Read=130\n",
"2024-08-04 19:13:33,149 INFO mapred.LocalJobRunner: Finishing task: attempt_local2011001580_0001_m_000000_0\n",
"2024-08-04 19:13:33,150 INFO mapred.LocalJobRunner: Starting task: attempt_local2011001580_0001_m_000001_0\n",
"2024-08-04 19:13:33,152 INFO output.PathOutputCommitterFactory: No output committer factory defined, defaulting to FileOutputCommitterFactory\n",
"2024-08-04 19:13:33,152 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2\n",
"2024-08-04 19:13:33,152 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false\n",
"2024-08-04 19:13:33,153 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]\n",
"2024-08-04 19:13:33,156 INFO mapred.MapTask: Processing split: file:/content/QuasiMonteCarlo_1722798810530_1260429884/in/part3:0+118\n",
"2024-08-04 19:13:33,173 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)\n",
"2024-08-04 19:13:33,173 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100\n",
"2024-08-04 19:13:33,173 INFO mapred.MapTask: soft limit at 83886080\n",
"2024-08-04 19:13:33,173 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600\n",
"2024-08-04 19:13:33,173 INFO mapred.MapTask: kvstart = 26214396; length = 6553600\n",
"2024-08-04 19:13:33,174 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer\n",
"2024-08-04 19:13:33,178 INFO mapred.LocalJobRunner: \n",
"2024-08-04 19:13:33,179 INFO mapred.MapTask: Starting flush of map output\n",
"2024-08-04 19:13:33,179 INFO mapred.MapTask: Spilling map output\n",
"2024-08-04 19:13:33,179 INFO mapred.MapTask: bufstart = 0; bufend = 18; bufvoid = 104857600\n",
"2024-08-04 19:13:33,179 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600\n",
"2024-08-04 19:13:33,181 INFO mapred.MapTask: Finished spill 0\n",
"2024-08-04 19:13:33,185 INFO mapred.Task: Task:attempt_local2011001580_0001_m_000001_0 is done. And is in the process of committing\n",
"2024-08-04 19:13:33,187 INFO mapred.LocalJobRunner: Generated 1000 samples.\n",
"2024-08-04 19:13:33,188 INFO mapred.Task: Task 'attempt_local2011001580_0001_m_000001_0' done.\n",
"2024-08-04 19:13:33,188 INFO mapred.Task: Final Counters for attempt_local2011001580_0001_m_000001_0: Counters: 17\n",
"\tFile System Counters\n",
"\t\tFILE: Number of bytes read=283289\n",
"\t\tFILE: Number of bytes written=997562\n",
"\t\tFILE: Number of read operations=0\n",
"\t\tFILE: Number of large read operations=0\n",
"\t\tFILE: Number of write operations=0\n",
"\tMap-Reduce Framework\n",
"\t\tMap input records=1\n",
"\t\tMap output records=2\n",
"\t\tMap output bytes=18\n",
"\t\tMap output materialized bytes=28\n",
"\t\tInput split bytes=128\n",
"\t\tCombine input records=0\n",
"\t\tSpilled Records=2\n",
"\t\tFailed Shuffles=0\n",
"\t\tMerged Map outputs=0\n",
"\t\tGC time elapsed (ms)=0\n",
"\t\tTotal committed heap usage (bytes)=354418688\n",
"\tFile Input Format Counters \n",
"\t\tBytes Read=130\n",
"2024-08-04 19:13:33,189 INFO mapred.LocalJobRunner: Finishing task: attempt_local2011001580_0001_m_000001_0\n",
"2024-08-04 19:13:33,192 INFO mapred.LocalJobRunner: Starting task: attempt_local2011001580_0001_m_000002_0\n",
"2024-08-04 19:13:33,193 INFO output.PathOutputCommitterFactory: No output committer factory defined, defaulting to FileOutputCommitterFactory\n",
"2024-08-04 19:13:33,193 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2\n",
"2024-08-04 19:13:33,193 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false\n",
"2024-08-04 19:13:33,194 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]\n",
"2024-08-04 19:13:33,196 INFO mapred.MapTask: Processing split: file:/content/QuasiMonteCarlo_1722798810530_1260429884/in/part2:0+118\n",
"2024-08-04 19:13:33,239 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)\n",
"2024-08-04 19:13:33,239 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100\n",
"2024-08-04 19:13:33,239 INFO mapred.MapTask: soft limit at 83886080\n",
"2024-08-04 19:13:33,239 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600\n",
"2024-08-04 19:13:33,240 INFO mapred.MapTask: kvstart = 26214396; length = 6553600\n",
"2024-08-04 19:13:33,242 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer\n",
"2024-08-04 19:13:33,246 INFO mapred.LocalJobRunner: \n",
"2024-08-04 19:13:33,247 INFO mapred.MapTask: Starting flush of map output\n",
"2024-08-04 19:13:33,247 INFO mapred.MapTask: Spilling map output\n",
"2024-08-04 19:13:33,247 INFO mapred.MapTask: bufstart = 0; bufend = 18; bufvoid = 104857600\n",
"2024-08-04 19:13:33,247 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600\n",
"2024-08-04 19:13:33,248 INFO mapred.MapTask: Finished spill 0\n",
"2024-08-04 19:13:33,252 INFO mapred.Task: Task:attempt_local2011001580_0001_m_000002_0 is done. And is in the process of committing\n",
"2024-08-04 19:13:33,256 INFO mapred.LocalJobRunner: Generated 1000 samples.\n",
"2024-08-04 19:13:33,257 INFO mapred.Task: Task 'attempt_local2011001580_0001_m_000002_0' done.\n",
"2024-08-04 19:13:33,257 INFO mapred.Task: Final Counters for attempt_local2011001580_0001_m_000002_0: Counters: 17\n",
"\tFile System Counters\n",
"\t\tFILE: Number of bytes read=284082\n",
"\t\tFILE: Number of bytes written=997622\n",
"\t\tFILE: Number of read operations=0\n",
"\t\tFILE: Number of large read operations=0\n",
"\t\tFILE: Number of write operations=0\n",
"\tMap-Reduce Framework\n",
"\t\tMap input records=1\n",
"\t\tMap output records=2\n",
"\t\tMap output bytes=18\n",
"\t\tMap output materialized bytes=28\n",
"\t\tInput split bytes=128\n",
"\t\tCombine input records=0\n",
"\t\tSpilled Records=2\n",
"\t\tFailed Shuffles=0\n",
"\t\tMerged Map outputs=0\n",
"\t\tGC time elapsed (ms)=16\n",
"\t\tTotal committed heap usage (bytes)=354418688\n",
"\tFile Input Format Counters \n",
"\t\tBytes Read=130\n",
"2024-08-04 19:13:33,258 INFO mapred.LocalJobRunner: Finishing task: attempt_local2011001580_0001_m_000002_0\n",
"2024-08-04 19:13:33,258 INFO mapred.LocalJobRunner: Starting task: attempt_local2011001580_0001_m_000003_0\n",
"2024-08-04 19:13:33,259 INFO output.PathOutputCommitterFactory: No output committer factory defined, defaulting to FileOutputCommitterFactory\n",
"2024-08-04 19:13:33,259 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2\n",
"2024-08-04 19:13:33,259 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false\n",
"2024-08-04 19:13:33,260 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]\n",
"2024-08-04 19:13:33,262 INFO mapred.MapTask: Processing split: file:/content/QuasiMonteCarlo_1722798810530_1260429884/in/part4:0+118\n",
"2024-08-04 19:13:33,392 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)\n",
"2024-08-04 19:13:33,392 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100\n",
"2024-08-04 19:13:33,392 INFO mapred.MapTask: soft limit at 83886080\n",
"2024-08-04 19:13:33,393 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600\n",
"2024-08-04 19:13:33,393 INFO mapred.MapTask: kvstart = 26214396; length = 6553600\n",
"2024-08-04 19:13:33,394 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer\n",
"2024-08-04 19:13:33,397 INFO mapred.LocalJobRunner: \n",
"2024-08-04 19:13:33,397 INFO mapred.MapTask: Starting flush of map output\n",
"2024-08-04 19:13:33,398 INFO mapred.MapTask: Spilling map output\n",
"2024-08-04 19:13:33,398 INFO mapred.MapTask: bufstart = 0; bufend = 18; bufvoid = 104857600\n",
"2024-08-04 19:13:33,398 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600\n",
"2024-08-04 19:13:33,399 INFO mapred.MapTask: Finished spill 0\n",
"2024-08-04 19:13:33,403 INFO mapred.Task: Task:attempt_local2011001580_0001_m_000003_0 is done. And is in the process of committing\n",
"2024-08-04 19:13:33,411 INFO mapred.LocalJobRunner: Generated 1000 samples.\n",
"2024-08-04 19:13:33,415 INFO mapred.Task: Task 'attempt_local2011001580_0001_m_000003_0' done.\n",
"2024-08-04 19:13:33,415 INFO mapred.Task: Final Counters for attempt_local2011001580_0001_m_000003_0: Counters: 17\n",
"\tFile System Counters\n",
"\t\tFILE: Number of bytes read=284875\n",
"\t\tFILE: Number of bytes written=997682\n",
"\t\tFILE: Number of read operations=0\n",
"\t\tFILE: Number of large read operations=0\n",
"\t\tFILE: Number of write operations=0\n",
"\tMap-Reduce Framework\n",
"\t\tMap input records=1\n",
"\t\tMap output records=2\n",
"\t\tMap output bytes=18\n",
"\t\tMap output materialized bytes=28\n",
"\t\tInput split bytes=128\n",
"\t\tCombine input records=0\n",
"\t\tSpilled Records=2\n",
"\t\tFailed Shuffles=0\n",
"\t\tMerged Map outputs=0\n",
"\t\tGC time elapsed (ms)=0\n",
"\t\tTotal committed heap usage (bytes)=354418688\n",
"\tFile Input Format Counters \n",
"\t\tBytes Read=130\n",
"2024-08-04 19:13:33,417 INFO mapred.LocalJobRunner: Finishing task: attempt_local2011001580_0001_m_000003_0\n",
"2024-08-04 19:13:33,417 INFO mapred.LocalJobRunner: Starting task: attempt_local2011001580_0001_m_000004_0\n",
"2024-08-04 19:13:33,422 INFO output.PathOutputCommitterFactory: No output committer factory defined, defaulting to FileOutputCommitterFactory\n",
"2024-08-04 19:13:33,423 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2\n",
"2024-08-04 19:13:33,423 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false\n",
"2024-08-04 19:13:33,423 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]\n",
"2024-08-04 19:13:33,431 INFO mapred.MapTask: Processing split: file:/content/QuasiMonteCarlo_1722798810530_1260429884/in/part1:0+118\n",
"2024-08-04 19:13:33,458 INFO mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)\n",
"2024-08-04 19:13:33,458 INFO mapred.MapTask: mapreduce.task.io.sort.mb: 100\n",
"2024-08-04 19:13:33,458 INFO mapred.MapTask: soft limit at 83886080\n",
"2024-08-04 19:13:33,458 INFO mapred.MapTask: bufstart = 0; bufvoid = 104857600\n",
"2024-08-04 19:13:33,458 INFO mapred.MapTask: kvstart = 26214396; length = 6553600\n",
"2024-08-04 19:13:33,462 INFO mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer\n",
"2024-08-04 19:13:33,465 INFO mapred.LocalJobRunner: \n",
"2024-08-04 19:13:33,474 INFO mapred.MapTask: Starting flush of map output\n",
"2024-08-04 19:13:33,474 INFO mapred.MapTask: Spilling map output\n",
"2024-08-04 19:13:33,474 INFO mapred.MapTask: bufstart = 0; bufend = 18; bufvoid = 104857600\n",
"2024-08-04 19:13:33,474 INFO mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600\n",
"2024-08-04 19:13:33,479 INFO mapred.MapTask: Finished spill 0\n",
"2024-08-04 19:13:33,493 INFO mapred.Task: Task:attempt_local2011001580_0001_m_000004_0 is done. And is in the process of committing\n",
"2024-08-04 19:13:33,517 INFO mapred.LocalJobRunner: Generated 1000 samples.\n",
"2024-08-04 19:13:33,518 INFO mapred.Task: Task 'attempt_local2011001580_0001_m_000004_0' done.\n",
"2024-08-04 19:13:33,519 INFO mapred.Task: Final Counters for attempt_local2011001580_0001_m_000004_0: Counters: 17\n",
"\tFile System Counters\n",
"\t\tFILE: Number of bytes read=285156\n",
"\t\tFILE: Number of bytes written=997742\n",
"\t\tFILE: Number of read operations=0\n",
"\t\tFILE: Number of large read operations=0\n",
"\t\tFILE: Number of write operations=0\n",
"\tMap-Reduce Framework\n",
"\t\tMap input records=1\n",
"\t\tMap output records=2\n",
"\t\tMap output bytes=18\n",
"\t\tMap output materialized bytes=28\n",
"\t\tInput split bytes=128\n",
"\t\tCombine input records=0\n",
"\t\tSpilled Records=2\n",
"\t\tFailed Shuffles=0\n",
"\t\tMerged Map outputs=0\n",
"\t\tGC time elapsed (ms)=8\n",
"\t\tTotal committed heap usage (bytes)=473956352\n",
"\tFile Input Format Counters \n",
"\t\tBytes Read=130\n",
"2024-08-04 19:13:33,520 INFO mapred.LocalJobRunner: Finishing task: attempt_local2011001580_0001_m_000004_0\n",
"2024-08-04 19:13:33,520 INFO mapred.LocalJobRunner: map task executor complete.\n",
"2024-08-04 19:13:33,532 INFO mapred.LocalJobRunner: Waiting for reduce tasks\n",
"2024-08-04 19:13:33,534 INFO mapred.LocalJobRunner: Starting task: attempt_local2011001580_0001_r_000000_0\n",
"2024-08-04 19:13:33,561 INFO output.PathOutputCommitterFactory: No output committer factory defined, defaulting to FileOutputCommitterFactory\n",
"2024-08-04 19:13:33,561 INFO output.FileOutputCommitter: File Output Committer Algorithm version is 2\n",
"2024-08-04 19:13:33,561 INFO output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false\n",
"2024-08-04 19:13:33,562 INFO mapred.Task: Using ResourceCalculatorProcessTree : [ ]\n",
"2024-08-04 19:13:33,567 INFO mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@5046a72e\n",
"2024-08-04 19:13:33,570 WARN impl.MetricsSystemImpl: JobTracker metrics system already initialized!\n",
"2024-08-04 19:13:33,603 INFO reduce.MergeManagerImpl: MergerManager: memoryLimit=2382574336, maxSingleShuffleLimit=595643584, mergeThreshold=1572499072, ioSortFactor=10, memToMemMergeOutputsThreshold=10\n",
"2024-08-04 19:13:33,613 INFO reduce.EventFetcher: attempt_local2011001580_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events\n",
"2024-08-04 19:13:33,688 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local2011001580_0001_m_000004_0 decomp: 24 len: 28 to MEMORY\n",
"2024-08-04 19:13:33,696 INFO mapreduce.Job: Job job_local2011001580_0001 running in uber mode : false\n",
"2024-08-04 19:13:33,696 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local2011001580_0001_m_000004_0\n",
"2024-08-04 19:13:33,696 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->24\n",
"2024-08-04 19:13:33,706 INFO mapreduce.Job: map 100% reduce 0%\n",
"2024-08-04 19:13:33,711 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local2011001580_0001_m_000001_0 decomp: 24 len: 28 to MEMORY\n",
"2024-08-04 19:13:33,716 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local2011001580_0001_m_000001_0\n",
"2024-08-04 19:13:33,716 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 2, commitMemory -> 24, usedMemory ->48\n",
"2024-08-04 19:13:33,722 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local2011001580_0001_m_000002_0 decomp: 24 len: 28 to MEMORY\n",
"2024-08-04 19:13:33,726 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local2011001580_0001_m_000002_0\n",
"2024-08-04 19:13:33,726 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 3, commitMemory -> 48, usedMemory ->72\n",
"2024-08-04 19:13:33,729 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local2011001580_0001_m_000000_0 decomp: 24 len: 28 to MEMORY\n",
"2024-08-04 19:13:33,731 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local2011001580_0001_m_000000_0\n",
"2024-08-04 19:13:33,731 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 4, commitMemory -> 72, usedMemory ->96\n",
"2024-08-04 19:13:33,737 INFO reduce.LocalFetcher: localfetcher#1 about to shuffle output of map attempt_local2011001580_0001_m_000003_0 decomp: 24 len: 28 to MEMORY\n",
"2024-08-04 19:13:33,740 INFO reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_local2011001580_0001_m_000003_0\n",
"2024-08-04 19:13:33,741 INFO reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 5, commitMemory -> 96, usedMemory ->120\n",
"2024-08-04 19:13:33,742 INFO reduce.EventFetcher: EventFetcher is interrupted.. Returning\n",
"2024-08-04 19:13:33,743 INFO mapred.LocalJobRunner: 5 / 5 copied.\n",
"2024-08-04 19:13:33,744 INFO reduce.MergeManagerImpl: finalMerge called with 5 in-memory map-outputs and 0 on-disk map-outputs\n",
"2024-08-04 19:13:33,754 INFO mapred.Merger: Merging 5 sorted segments\n",
"2024-08-04 19:13:33,754 INFO mapred.Merger: Down to the last merge-pass, with 5 segments left of total size: 105 bytes\n",
"2024-08-04 19:13:33,757 INFO reduce.MergeManagerImpl: Merged 5 segments, 120 bytes to disk to satisfy reduce memory limit\n",
"2024-08-04 19:13:33,757 INFO reduce.MergeManagerImpl: Merging 1 files, 116 bytes from disk\n",
"2024-08-04 19:13:33,759 INFO reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce\n",
"2024-08-04 19:13:33,759 INFO mapred.Merger: Merging 1 sorted segments\n",
"2024-08-04 19:13:33,759 INFO mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 109 bytes\n",
"2024-08-04 19:13:33,760 INFO mapred.LocalJobRunner: 5 / 5 copied.\n",
"2024-08-04 19:13:33,765 INFO Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords\n",
"2024-08-04 19:13:33,779 INFO mapred.Task: Task:attempt_local2011001580_0001_r_000000_0 is done. And is in the process of committing\n",
"2024-08-04 19:13:33,783 INFO mapred.LocalJobRunner: 5 / 5 copied.\n",
"2024-08-04 19:13:33,783 INFO mapred.Task: Task attempt_local2011001580_0001_r_000000_0 is allowed to commit now\n",
"2024-08-04 19:13:33,786 INFO output.FileOutputCommitter: Saved output of task 'attempt_local2011001580_0001_r_000000_0' to file:/content/QuasiMonteCarlo_1722798810530_1260429884/out\n",
"2024-08-04 19:13:33,788 INFO mapred.LocalJobRunner: reduce > reduce\n",
"2024-08-04 19:13:33,788 INFO mapred.Task: Task 'attempt_local2011001580_0001_r_000000_0' done.\n",
"2024-08-04 19:13:33,792 INFO mapred.Task: Final Counters for attempt_local2011001580_0001_r_000000_0: Counters: 24\n",
"\tFile System Counters\n",
"\t\tFILE: Number of bytes read=285572\n",
"\t\tFILE: Number of bytes written=998097\n",
"\t\tFILE: Number of read operations=0\n",
"\t\tFILE: Number of large read operations=0\n",
"\t\tFILE: Number of write operations=0\n",
"\tMap-Reduce Framework\n",
"\t\tCombine input records=0\n",
"\t\tCombine output records=0\n",
"\t\tReduce input groups=2\n",
"\t\tReduce shuffle bytes=140\n",
"\t\tReduce input records=10\n",
"\t\tReduce output records=0\n",
"\t\tSpilled Records=10\n",
"\t\tShuffled Maps =5\n",
"\t\tFailed Shuffles=0\n",
"\t\tMerged Map outputs=5\n",
"\t\tGC time elapsed (ms)=0\n",
"\t\tTotal committed heap usage (bytes)=473956352\n",
"\tShuffle Errors\n",
"\t\tBAD_ID=0\n",
"\t\tCONNECTION=0\n",
"\t\tIO_ERROR=0\n",
"\t\tWRONG_LENGTH=0\n",
"\t\tWRONG_MAP=0\n",
"\t\tWRONG_REDUCE=0\n",
"\tFile Output Format Counters \n",
"\t\tBytes Written=109\n",
"2024-08-04 19:13:33,792 INFO mapred.LocalJobRunner: Finishing task: attempt_local2011001580_0001_r_000000_0\n",
"2024-08-04 19:13:33,794 INFO mapred.LocalJobRunner: reduce task executor complete.\n",
"2024-08-04 19:13:34,708 INFO mapreduce.Job: map 100% reduce 100%\n",
"2024-08-04 19:13:34,709 INFO mapreduce.Job: Job job_local2011001580_0001 completed successfully\n",
"2024-08-04 19:13:34,753 INFO mapreduce.Job: Counters: 30\n",
"\tFile System Counters\n",
"\t\tFILE: Number of bytes read=1705470\n",
"\t\tFILE: Number of bytes written=5986207\n",
"\t\tFILE: Number of read operations=0\n",
"\t\tFILE: Number of large read operations=0\n",
"\t\tFILE: Number of write operations=0\n",
"\tMap-Reduce Framework\n",
"\t\tMap input records=5\n",
"\t\tMap output records=10\n",
"\t\tMap output bytes=90\n",
"\t\tMap output materialized bytes=140\n",
"\t\tInput split bytes=640\n",
"\t\tCombine input records=0\n",
"\t\tCombine output records=0\n",
"\t\tReduce input groups=2\n",
"\t\tReduce shuffle bytes=140\n",
"\t\tReduce input records=10\n",
"\t\tReduce output records=0\n",
"\t\tSpilled Records=20\n",
"\t\tShuffled Maps =5\n",
"\t\tFailed Shuffles=0\n",
"\t\tMerged Map outputs=5\n",
"\t\tGC time elapsed (ms)=47\n",
"\t\tTotal committed heap usage (bytes)=2365587456\n",
"\tShuffle Errors\n",
"\t\tBAD_ID=0\n",
"\t\tCONNECTION=0\n",
"\t\tIO_ERROR=0\n",
"\t\tWRONG_LENGTH=0\n",
"\t\tWRONG_MAP=0\n",
"\t\tWRONG_REDUCE=0\n",
"\tFile Input Format Counters \n",
"\t\tBytes Read=650\n",
"\tFile Output Format Counters \n",
"\t\tBytes Written=109\n",
"Job Finished in 3.28 seconds\n",
"Estimated value of Pi is 3.14160000000000000000\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"The job completed successfully, however it did not run on the MiniCluster because we did not specify the MiniCluster's Yarn Resource Manager address.\n",
"\n",
"\n",
"In fact, `yarn application -list` returns no apps (neither running nor finished).\n"
],
"metadata": {
"id": "6foi_r8f-QDH"
}
},
{
"cell_type": "code",
"source": [
"!yarn application -D yarn.resourcemanager.address=localhost:8903 -list -appStates ALL"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "_GIEiotdBZZw",
"outputId": "d72b1287-05f2-44d9-e808-faf2597972cc"
},
"execution_count": 64,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"2024-08-04 19:13:37,808 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at localhost/127.0.0.1:8903\n",
"Total number of applications (application-types: [], states: [NEW, NEW_SAVING, SUBMITTED, ACCEPTED, RUNNING, FINISHED, FAILED, KILLED] and tags: []):0\n",
" Application-Id\t Application-Name\t Application-Type\t User\t Queue\t State\t Final-State\t Progress\t Tracking-URL\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## Local configuration files"
],
"metadata": {
"id": "tVaJQD34VhcL"
}
},
{
"cell_type": "markdown",
"source": [
"In order to be able to submit the job to the MiniCluster with YARN we need to edit three files:\n",
"\n",
"\n",
"* `mapred-site.xml`\n",
"* `core-site.xml`\n",
"* `yarn-site.xml`\n",
"\n",
"(see https://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html#YARN_on_a_Single_Node)\n"
],
"metadata": {
"id": "a9geb1q-8qSE"
}
},
{
"cell_type": "code",
"source": [
"file_mapred_site = os.path.join(os.environ['HADOOP_HOME'],'etc/hadoop/mapred-site.xml')\n",
"file_core_site = os.path.join(os.environ['HADOOP_HOME'],'etc/hadoop/core-site.xml')\n",
"file_yarn_site = os.path.join(os.environ['HADOOP_HOME'],'etc/hadoop/yarn-site.xml')"
],
"metadata": {
"id": "YmW0MsoTH0Bz"
},
"execution_count": 65,
"outputs": []
},
{
"cell_type": "code",
"source": [
"%%bash\n",
"cat > $HADOOP_HOME/'etc/hadoop/mapred-site.xml' << 🐸\n",
"\n",
" \n",
" mapreduce.framework.name\n",
" yarn\n",
" \n",
" \n",
" mapreduce.application.classpath\n",
" ${HADOOP_HOME}/share/hadoop/mapreduce/*:${HADOOP_HOME}/share/hadoop/mapreduce/lib/*\n",
" \n",
"\n",
"🐸"
],
"metadata": {
"id": "Ru0uB16T9uPA"
},
"execution_count": 66,
"outputs": []
},
{
"cell_type": "code",
"source": [
"!cat $HADOOP_HOME/'etc/hadoop/mapred-site.xml'"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "1hKTx-LQXa4e",
"outputId": "40e05181-58f4-4b0b-f617-d78aa7e61221"
},
"execution_count": 67,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"\n",
" \n",
" mapreduce.framework.name\n",
" yarn\n",
" \n",
" \n",
" mapreduce.application.classpath\n",
" /content/hadoop-3.4.0/share/hadoop/mapreduce/*:/content/hadoop-3.4.0/share/hadoop/mapreduce/lib/*\n",
" \n",
"\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Set the default filesystem as the MiniCluster's filesystem in `core-site.xml`. This is necessary in order to allow YARN to save the applications logs to HDFS and this is why we create the directory `/tmp/logs` on HDFS."
],
"metadata": {
"id": "QRICGx9K883j"
}
},
{
"cell_type": "code",
"source": [
"with open(file_core_site, 'w') as file:\n",
" file.write(\"\"\"\n",
"\n",
" \n",
" fs.defaultFS\n",
" hdfs://localhost:8902/\n",
" \n",
"\"\"\")"
],
"metadata": {
"id": "R1n1y3io85As"
},
"execution_count": 68,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Set `yarn.resourcemanager.address=localhost:8903` in `yarn-site.xml`."
],
"metadata": {
"id": "pD8pB4MCGa_5"
}
},
{
"cell_type": "code",
"source": [
"with open(file_yarn_site, 'w') as file:\n",
" file.write(\"\"\"\n",
"\n",
"\n",
" yarn.resourcemanager.address\n",
" localhost:8903\n",
"\n",
"\n",
" yarn.log-aggregation-enable\n",
" true\n",
"\n",
"\"\"\")\n"
],
"metadata": {
"id": "F9sxqwOZOWTl"
},
"execution_count": 69,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"It might be necessary at this point to re-start the cluster."
],
"metadata": {
"id": "5Nwameq-Ce3g"
}
},
{
"cell_type": "code",
"source": [
"process.kill()\n",
"!pkill -f java # kill java processes"
],
"metadata": {
"id": "Yh-c5G_bCd6f"
},
"execution_count": 70,
"outputs": []
},
{
"cell_type": "code",
"source": [
"with open('out.txt', \"w\") as stdout_file, open('err.txt', \"w\") as stderr_file:\n",
" process = subprocess.Popen(\n",
" [\"mapred\", \"minicluster\", \"-format\", \"-jhsport\", \"8900\", \"-nnhttpport\", \"8901\", \"-nnport\", \"8902\", \"-rmport\", \"8903\"],\n",
" stdout=stdout_file,\n",
" stderr=stderr_file\n",
" )"
],
"metadata": {
"id": "oNNl1MzDDd6D"
},
"execution_count": 71,
"outputs": []
},
{
"cell_type": "markdown",
"source": [
"Verify that the MiniCluster is running."
],
"metadata": {
"id": "w5pioEgpXw02"
}
},
{
"cell_type": "code",
"source": [
"for att in range(10):\n",
" with open('err.txt') as myfile:\n",
" if 'Started MiniMRCluster' in myfile.read():\n",
" print('MiniCluster is up and running')\n",
" break\n",
" else:\n",
" time.sleep(2)\n"
],
"metadata": {
"id": "CeOTCmbxdHpm",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "8e9bd4d1-8695-4527-96c7-7459c2ad801b"
},
"execution_count": 72,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"MiniCluster is up and running\n"
]
}
]
},
{
"cell_type": "code",
"source": [
"!lsof -n -i -P +c0 -sTCP:LISTEN -ac java"
],
"metadata": {
"id": "ZPk1XrlgDHlI",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "fce7b461-99cd-4d90-e837-3089c52d7e6a"
},
"execution_count": 73,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME\n",
"java 5488 root 341u IPv4 149683 0t0 TCP 127.0.0.1:8901 (LISTEN)\n",
"java 5488 root 351u IPv4 148854 0t0 TCP 127.0.0.1:8902 (LISTEN)\n",
"java 5488 root 361u IPv4 148862 0t0 TCP 127.0.0.1:36547 (LISTEN)\n",
"java 5488 root 364u IPv4 148865 0t0 TCP 127.0.0.1:40385 (LISTEN)\n",
"java 5488 root 393u IPv4 150500 0t0 TCP 127.0.0.1:42681 (LISTEN)\n",
"java 5488 root 394u IPv4 148903 0t0 TCP 127.0.0.1:34431 (LISTEN)\n",
"java 5488 root 420u IPv4 158000 0t0 TCP *:8031 (LISTEN)\n",
"java 5488 root 434u IPv4 154506 0t0 TCP *:10033 (LISTEN)\n",
"java 5488 root 444u IPv4 157946 0t0 TCP *:19888 (LISTEN)\n",
"java 5488 root 446u IPv4 154605 0t0 TCP *:8088 (LISTEN)\n",
"java 5488 root 454u IPv4 157989 0t0 TCP 127.0.0.1:8900 (LISTEN)\n",
"java 5488 root 464u IPv4 157995 0t0 TCP *:8033 (LISTEN)\n",
"java 5488 root 484u IPv4 158005 0t0 TCP *:8030 (LISTEN)\n",
"java 5488 root 494u IPv4 158009 0t0 TCP 127.0.0.1:8903 (LISTEN)\n",
"java 5488 root 504u IPv4 158797 0t0 TCP 127.0.0.1:34535 (LISTEN)\n",
"java 5488 root 514u IPv4 158027 0t0 TCP 127.0.0.1:34217 (LISTEN)\n",
"java 5488 root 524u IPv4 158801 0t0 TCP *:43847 (LISTEN)\n",
"java 5488 root 525u IPv4 158031 0t0 TCP 127.0.0.1:37361 (LISTEN)\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"Submit the app again. You should see at the very beginning of the output:\n",
"\n",
"```\n",
"Connecting to ResourceManager at localhost/127.0.0.1:8903\n",
"```\n",
"\n",
"This means that YARN has read its configuration file."
],
"metadata": {
"id": "tWIHDWeDFe14"
}
},
{
"cell_type": "markdown",
"source": [
"## Run the `pi` app on the MiniCluster with YARN"
],
"metadata": {
"id": "6wsQK9LlGWf-"
}
},
{
"cell_type": "markdown",
"source": [
"Let us run the job on the MiniCluster. \n",
"\n"
],
"metadata": {
"id": "EP0_RTN1ZuhU"
}
},
{
"cell_type": "code",
"source": [
"!yarn jar ./hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.4.0.jar pi \\\n",
" 5 1000"
],
"metadata": {
"id": "yjO0xISPFc_w",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "fa7be714-b6e8-47f1-af5e-2b802ec534d7"
},
"execution_count": 74,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Number of Maps = 5\n",
"Samples per Map = 1000\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"04-Aug-24 07:13:59 PM - INFO: t=2024-08-04T19:13:59+0000 lvl=info msg=\"join connections\" obj=join id=4944f91c91d7 l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49611\n",
"04-Aug-24 07:14:00 PM - INFO: t=2024-08-04T19:14:00+0000 lvl=info msg=\"join connections\" obj=join id=b6557ad3937b l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49611\n",
"04-Aug-24 07:14:00 PM - INFO: t=2024-08-04T19:14:00+0000 lvl=info msg=\"join connections\" obj=join id=31c3ae28621f l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:00 PM - INFO: t=2024-08-04T19:14:00+0000 lvl=info msg=\"join connections\" obj=join id=5aa288b19d68 l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:00 PM - INFO: t=2024-08-04T19:14:00+0000 lvl=info msg=\"join connections\" obj=join id=ec278b8a4015 l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:00 PM - INFO: t=2024-08-04T19:14:00+0000 lvl=info msg=\"join connections\" obj=join id=c8a82bdeb45c l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:00 PM - INFO: t=2024-08-04T19:14:00+0000 lvl=info msg=\"join connections\" obj=join id=6732850300e3 l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:00 PM - INFO: t=2024-08-04T19:14:00+0000 lvl=info msg=\"join connections\" obj=join id=a6678ebec0fb l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:00 PM - INFO: t=2024-08-04T19:14:00+0000 lvl=info msg=\"join connections\" obj=join id=88e97548a4a8 l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:00 PM - INFO: t=2024-08-04T19:14:00+0000 lvl=info msg=\"join connections\" obj=join id=cd4c21a2f9be l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:00 PM - INFO: t=2024-08-04T19:14:00+0000 lvl=info msg=\"join connections\" obj=join id=a1f21ec754c8 l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:00 PM - INFO: t=2024-08-04T19:14:00+0000 lvl=info msg=\"join connections\" obj=join id=524ebc6a8a13 l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:00 PM - INFO: t=2024-08-04T19:14:00+0000 lvl=info msg=\"join connections\" obj=join id=01a4d73455a4 l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:00 PM - INFO: t=2024-08-04T19:14:00+0000 lvl=info msg=\"join connections\" obj=join id=d2679b39ea53 l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:00 PM - INFO: t=2024-08-04T19:14:00+0000 lvl=info msg=\"join connections\" obj=join id=0ac5e9002df7 l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:00 PM - INFO: t=2024-08-04T19:14:00+0000 lvl=info msg=\"join connections\" obj=join id=4088f300f1b2 l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:01 PM - INFO: t=2024-08-04T19:14:01+0000 lvl=info msg=\"join connections\" obj=join id=2c22c3282878 l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:01 PM - INFO: t=2024-08-04T19:14:01+0000 lvl=info msg=\"join connections\" obj=join id=9b7da82d412c l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:01 PM - INFO: t=2024-08-04T19:14:01+0000 lvl=info msg=\"join connections\" obj=join id=9ad5e9b956c0 l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:01 PM - INFO: t=2024-08-04T19:14:01+0000 lvl=info msg=\"join connections\" obj=join id=2aaca7c3ee6e l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:01 PM - INFO: t=2024-08-04T19:14:01+0000 lvl=info msg=\"join connections\" obj=join id=71230148c91d l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:01 PM - INFO: t=2024-08-04T19:14:01+0000 lvl=info msg=\"join connections\" obj=join id=fdd27abddc4c l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:01 PM - INFO: t=2024-08-04T19:14:01+0000 lvl=info msg=\"join connections\" obj=join id=dd6afb0f3192 l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n",
"04-Aug-24 07:14:01 PM - INFO: t=2024-08-04T19:14:01+0000 lvl=info msg=\"join connections\" obj=join id=d4821357ab57 l=127.0.0.1:8901 r=[2a02:8388:6cc5:e800:70ee:91e:4517:7eea]:49620\n"
]
},
{
"output_type": "stream",
"name": "stdout",
"text": [
"Wrote input for Map #0\n",
"Wrote input for Map #1\n",
"Wrote input for Map #2\n",
"Wrote input for Map #3\n",
"Wrote input for Map #4\n",
"Starting Job\n",
"2024-08-04 19:14:02,329 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at localhost/127.0.0.1:8903\n",
"2024-08-04 19:14:03,303 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/root/.staging/job_1722798834650_0001\n",
"2024-08-04 19:14:03,915 INFO input.FileInputFormat: Total input files to process : 5\n",
"2024-08-04 19:14:04,481 INFO mapreduce.JobSubmitter: number of splits:5\n",
"2024-08-04 19:14:05,484 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1722798834650_0001\n",
"2024-08-04 19:14:05,485 INFO mapreduce.JobSubmitter: Executing with tokens: []\n",
"2024-08-04 19:14:06,493 INFO conf.Configuration: resource-types.xml not found\n",
"2024-08-04 19:14:06,503 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'.\n",
"2024-08-04 19:14:07,764 INFO impl.YarnClientImpl: Submitted application application_1722798834650_0001\n",
"2024-08-04 19:14:07,940 INFO mapreduce.Job: The url to track the job: http://446a375b7fe4:8088/proxy/application_1722798834650_0001/\n",
"2024-08-04 19:14:07,947 INFO mapreduce.Job: Running job: job_1722798834650_0001\n",
"2024-08-04 19:14:21,800 INFO mapreduce.Job: Job job_1722798834650_0001 running in uber mode : false\n",
"2024-08-04 19:14:21,802 INFO mapreduce.Job: map 0% reduce 0%\n",
"2024-08-04 19:14:35,193 INFO mapreduce.Job: map 40% reduce 0%\n",
"2024-08-04 19:14:47,348 INFO mapreduce.Job: map 80% reduce 0%\n",
"2024-08-04 19:14:59,592 INFO mapreduce.Job: map 100% reduce 0%\n",
"2024-08-04 19:15:00,604 INFO mapreduce.Job: map 100% reduce 100%\n",
"2024-08-04 19:15:01,621 INFO mapreduce.Job: Job job_1722798834650_0001 completed successfully\n",
"2024-08-04 19:15:01,756 INFO mapreduce.Job: Counters: 54\n",
"\tFile System Counters\n",
"\t\tFILE: Number of bytes read=116\n",
"\t\tFILE: Number of bytes written=1856793\n",
"\t\tFILE: Number of read operations=0\n",
"\t\tFILE: Number of large read operations=0\n",
"\t\tFILE: Number of write operations=0\n",
"\t\tHDFS: Number of bytes read=1320\n",
"\t\tHDFS: Number of bytes written=215\n",
"\t\tHDFS: Number of read operations=25\n",
"\t\tHDFS: Number of large read operations=0\n",
"\t\tHDFS: Number of write operations=3\n",
"\t\tHDFS: Number of bytes read erasure-coded=0\n",
"\tJob Counters \n",
"\t\tLaunched map tasks=5\n",
"\t\tLaunched reduce tasks=1\n",
"\t\tData-local map tasks=5\n",
"\t\tTotal time spent by all maps in occupied slots (ms)=53665\n",
"\t\tTotal time spent by all reduces in occupied slots (ms)=11104\n",
"\t\tTotal time spent by all map tasks (ms)=53665\n",
"\t\tTotal time spent by all reduce tasks (ms)=11104\n",
"\t\tTotal vcore-milliseconds taken by all map tasks=53665\n",
"\t\tTotal vcore-milliseconds taken by all reduce tasks=11104\n",
"\t\tTotal megabyte-milliseconds taken by all map tasks=54952960\n",
"\t\tTotal megabyte-milliseconds taken by all reduce tasks=11370496\n",
"\tMap-Reduce Framework\n",
"\t\tMap input records=5\n",
"\t\tMap output records=10\n",
"\t\tMap output bytes=90\n",
"\t\tMap output materialized bytes=140\n",
"\t\tInput split bytes=730\n",
"\t\tCombine input records=0\n",
"\t\tCombine output records=0\n",
"\t\tReduce input groups=2\n",
"\t\tReduce shuffle bytes=140\n",
"\t\tReduce input records=10\n",
"\t\tReduce output records=0\n",
"\t\tSpilled Records=20\n",
"\t\tShuffled Maps =5\n",
"\t\tFailed Shuffles=0\n",
"\t\tMerged Map outputs=5\n",
"\t\tGC time elapsed (ms)=505\n",
"\t\tCPU time spent (ms)=5570\n",
"\t\tPhysical memory (bytes) snapshot=1933701120\n",
"\t\tVirtual memory (bytes) snapshot=16332873728\n",
"\t\tTotal committed heap usage (bytes)=1436549120\n",
"\t\tPeak Map Physical memory (bytes)=383688704\n",
"\t\tPeak Map Virtual memory (bytes)=2723983360\n",
"\t\tPeak Reduce Physical memory (bytes)=217210880\n",
"\t\tPeak Reduce Virtual memory (bytes)=2730729472\n",
"\tShuffle Errors\n",
"\t\tBAD_ID=0\n",
"\t\tCONNECTION=0\n",
"\t\tIO_ERROR=0\n",
"\t\tWRONG_LENGTH=0\n",
"\t\tWRONG_MAP=0\n",
"\t\tWRONG_REDUCE=0\n",
"\tFile Input Format Counters \n",
"\t\tBytes Read=590\n",
"\tFile Output Format Counters \n",
"\t\tBytes Written=97\n",
"Job Finished in 59.652 seconds\n",
"Estimated value of Pi is 3.14160000000000000000\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"We can now see the finished app listed in the YARN resource manager (note that this time we do not need to specify the Resource Manager's address with the option `-D yarn.resourcemanager.address=localhost:8903`)."
],
"metadata": {
"id": "H-qDAyhcQOCt"
}
},
{
"cell_type": "code",
"source": [
"!yarn application -list -appStates ALL"
],
"metadata": {
"id": "Ko_hoU7ZQTdD",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "0b4af40b-6baa-4926-8c67-12c0e44a78dd"
},
"execution_count": 75,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"2024-08-04 19:15:05,035 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at localhost/127.0.0.1:8903\n",
"Total number of applications (application-types: [], states: [NEW, NEW_SAVING, SUBMITTED, ACCEPTED, RUNNING, FINISHED, FAILED, KILLED] and tags: []):1\n",
" Application-Id\t Application-Name\t Application-Type\t User\t Queue\t State\t Final-State\t Progress\t Tracking-URL\n",
"application_1722798834650_0001\t QuasiMonteCarlo\t MAPREDUCE\t root\troot.default\t FINISHED\t SUCCEEDED\t 100%\thttp://446a375b7fe4:19888/jobhistory/job/job_1722798834650_0001\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"## Run the `pi` app in the background"
],
"metadata": {
"id": "KJ_DvxISR_ds"
}
},
{
"cell_type": "markdown",
"source": [
"Let us run the `pi` app in the background (as a subprocess) and with more mappers, so that it lasts longer and we are able to monitor its progress with the `yarn` command-line."
],
"metadata": {
"id": "hDgf97RkoP7O"
}
},
{
"cell_type": "code",
"source": [
"with open('job_out.txt', \"w\") as stdout_file, open('job_err.txt', \"w\") as stderr_file:\n",
" process = subprocess.Popen(\n",
" [\"yarn\", \"jar\", \"./hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.4.0.jar\", \"pi\",\n",
" \"-D\", \"localhost:8903\",\n",
" \"50\", \"1000000\"],\n",
" stdout=stdout_file,\n",
" stderr=stderr_file\n",
" )"
],
"metadata": {
"id": "JPGCGVCcoPVh"
},
"execution_count": 76,
"outputs": []
},
{
"cell_type": "code",
"source": [
"time.sleep(10)"
],
"metadata": {
"id": "pYXRgDiRSOZS"
},
"execution_count": 77,
"outputs": []
},
{
"cell_type": "code",
"source": [
"!yarn application -list -appStates ALL"
],
"metadata": {
"id": "X-jC_BxS-rpY",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "f28e58bb-bb08-4433-9baf-ba024ca60229"
},
"execution_count": 78,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"2024-08-04 19:15:21,096 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at localhost/127.0.0.1:8903\n",
"Total number of applications (application-types: [], states: [NEW, NEW_SAVING, SUBMITTED, ACCEPTED, RUNNING, FINISHED, FAILED, KILLED] and tags: []):2\n",
" Application-Id\t Application-Name\t Application-Type\t User\t Queue\t State\t Final-State\t Progress\t Tracking-URL\n",
"application_1722798834650_0001\t QuasiMonteCarlo\t MAPREDUCE\t root\troot.default\t FINISHED\t SUCCEEDED\t 100%\thttp://446a375b7fe4:19888/jobhistory/job/job_1722798834650_0001\n",
"application_1722798834650_0002\t QuasiMonteCarlo\t MAPREDUCE\t root\troot.default\t ACCEPTED\t UNDEFINED\t 0%\t N/A\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"If you do not see the newly submitted application in the YARN queue yet, give it some time and re-run the `yarn application -list` command! \n",
"\n"
],
"metadata": {
"id": "lMbgNm_pSfF1"
}
},
{
"cell_type": "code",
"source": [
"!yarn application -list -appStates ALL"
],
"metadata": {
"id": "Da-sAM1NReSG",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "1ac61fd1-bb7a-4e1a-8284-0414d8458a78"
},
"execution_count": 79,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"2024-08-04 19:15:28,650 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at localhost/127.0.0.1:8903\n",
"Total number of applications (application-types: [], states: [NEW, NEW_SAVING, SUBMITTED, ACCEPTED, RUNNING, FINISHED, FAILED, KILLED] and tags: []):2\n",
" Application-Id\t Application-Name\t Application-Type\t User\t Queue\t State\t Final-State\t Progress\t Tracking-URL\n",
"application_1722798834650_0001\t QuasiMonteCarlo\t MAPREDUCE\t root\troot.default\t FINISHED\t SUCCEEDED\t 100%\thttp://446a375b7fe4:19888/jobhistory/job/job_1722798834650_0001\n",
"application_1722798834650_0002\t QuasiMonteCarlo\t MAPREDUCE\t root\troot.default\t ACCEPTED\t UNDEFINED\t 0%\t N/A\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"You should now see something like this:\n",
"```\n",
"Total number of applications (application-types: [], states: [NEW, NEW_SAVING, SUBMITTED, ACCEPTED, RUNNING, FINISHED, FAILED, KILLED] and tags: []):2\n",
" Application-Id\t Application-Name\t Application-Type\t User\t Queue\t State\t Final-State\t Progress\t Tracking-URL\n",
"application_1707941943926_0001\t QuasiMonteCarlo\t MAPREDUCE\t root\t default\t FINISHED\t SUCCEEDED\t 100%\thttp://c532258dcee8:19888/jobhistory/job/job_1707941943926_0001\n",
"application_1707941943926_0002\t QuasiMonteCarlo\t MAPREDUCE\t root\t default\t RUNNING\t UNDEFINED\t 5.18%\t http://localhost:43121\n",
"```\n",
"\n",
"The application in status `RUNNING` is the most recently submitted."
],
"metadata": {
"id": "zIYhojkaSwdi"
}
},
{
"cell_type": "markdown",
"source": [
"# View the logs of an application"
],
"metadata": {
"id": "ACJng9ePTQct"
}
},
{
"cell_type": "markdown",
"source": [
"To view the logs of a finished application use:\n",
"\n",
"```\n",
"yarn logs -applicationID \n",
"```\n",
"\n",
"For the sake of this demo, we are going to pick the id if the first successfully finished app with the following shell command:"
],
"metadata": {
"id": "rydldlaISPUV"
}
},
{
"cell_type": "code",
"source": [
"!yarn application -list -appStates FINISHED 2>/dev/null|grep SUCCEEDED|tail -1| cut -f1"
],
"metadata": {
"colab": {
"base_uri": "https://localhost:8080/"
},
"id": "zF0cW5d3T3ec",
"outputId": "8eea2a34-025c-4b35-ffdb-feb88376db28"
},
"execution_count": 80,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"application_1722798834650_0001\n"
]
}
]
},
{
"cell_type": "markdown",
"source": [
"View the logs for the selected application id (warning: it's a large file!)."
],
"metadata": {
"id": "PI9GkpQnTzZZ"
}
},
{
"cell_type": "code",
"source": [
"%%bash\n",
"app_id=$(yarn application -list -appStates FINISHED 2>/dev/null|grep SUCCEEDED|tail -1| cut -f1)\n",
"yarn logs -applicationId $app_id"
],
"metadata": {
"id": "9r5WySeyS6cF",
"colab": {
"base_uri": "https://localhost:8080/"
},
"outputId": "3d85bc9a-c112-4179-9cac-0fed36ca483a"
},
"execution_count": 81,
"outputs": [
{
"output_type": "stream",
"name": "stdout",
"text": [
"Container: container_1722798834650_0001_01_000004 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:directory.info\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:2101\n",
"LogContents:\n",
"ls -l:\n",
"total 32\n",
"-rw-r--r-- 1 root root 129 Aug 4 19:14 container_tokens\n",
"-rwx------ 1 root root 964 Aug 4 19:14 default_container_executor_session.sh\n",
"-rwx------ 1 root root 1019 Aug 4 19:14 default_container_executor.sh\n",
"lrwxrwxrwx 1 root root 179 Aug 4 19:14 job.jar -> /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/11/job.jar\n",
"lrwxrwxrwx 1 root root 179 Aug 4 19:14 job.xml -> /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/13/job.xml\n",
"-rwx------ 1 root root 8159 Aug 4 19:14 launch_container.sh\n",
"drwx--x--- 2 root root 4096 Aug 4 19:14 tmp\n",
"find -L . -maxdepth 5 -ls:\n",
" 809807 4 drwx--x--- 3 root root 4096 Aug 4 19:14 .\n",
" 809848 4 -rwx------ 1 root root 1019 Aug 4 19:14 ./default_container_executor.sh\n",
" 809751 260 -r-x------ 1 root root 264361 Aug 4 19:14 ./job.xml\n",
" 809822 4 -rw-r--r-- 1 root root 129 Aug 4 19:14 ./container_tokens\n",
" 809841 4 -rwx------ 1 root root 964 Aug 4 19:14 ./default_container_executor_session.sh\n",
" 809826 8 -rwx------ 1 root root 8159 Aug 4 19:14 ./launch_container.sh\n",
" 809849 4 -rw-r--r-- 1 root root 16 Aug 4 19:14 ./.default_container_executor.sh.crc\n",
" 809840 4 -rw-r--r-- 1 root root 72 Aug 4 19:14 ./.launch_container.sh.crc\n",
" 809745 4 drwx------ 2 root root 4096 Aug 4 19:14 ./job.jar\n",
" 809746 276 -r-x------ 1 root root 281609 Aug 4 19:14 ./job.jar/job.jar\n",
" 809823 4 -rw-r--r-- 1 root root 12 Aug 4 19:14 ./.container_tokens.crc\n",
" 809847 4 -rw-r--r-- 1 root root 16 Aug 4 19:14 ./.default_container_executor_session.sh.crc\n",
" 809821 4 drwx--x--- 2 root root 4096 Aug 4 19:14 ./tmp\n",
"broken symlinks(find -L . -maxdepth 5 -type l -ls):\n",
"\n",
"End of LogType:directory.info\n",
"*******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000004 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:launch_container.sh\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:8159\n",
"LogContents:\n",
"#!/bin/bash\n",
"\n",
"set -o pipefail -e\n",
"export PRELAUNCH_OUT=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000004/prelaunch.out\"\n",
"exec >\"${PRELAUNCH_OUT}\"\n",
"export PRELAUNCH_ERR=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000004/prelaunch.err\"\n",
"exec 2>\"${PRELAUNCH_ERR}\"\n",
"echo \"Setting up env variables\"\n",
"export JAVA_HOME=${JAVA_HOME:-\"/usr/lib/jvm/java-11-openjdk-amd64\"}\n",
"export HADOOP_COMMON_HOME=${HADOOP_COMMON_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_HDFS_HOME=${HADOOP_HDFS_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-\"/content/hadoop-3.4.0/etc/hadoop\"}\n",
"export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_HOME=${HADOOP_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export PATH=${PATH:-\"/content/hadoop-3.4.0/bin:/opt/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/tools/node/bin:/tools/google-cloud-sdk/bin\"}\n",
"export LANG=${LANG:-\"en_US.UTF-8\"}\n",
"export HADOOP_TOKEN_FILE_LOCATION=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000004/container_tokens\"\n",
"export CONTAINER_ID=\"container_1722798834650_0001_01_000004\"\n",
"export NM_PORT=\"34535\"\n",
"export NM_HOST=\"localhost\"\n",
"export NM_HTTP_PORT=\"37361\"\n",
"export LOCAL_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001\"\n",
"export LOCAL_USER_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/\"\n",
"export LOG_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_0/application_1722798834650_0001/container_1722798834650_0001_01_000004,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000004,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000004,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000004\"\n",
"export USER=\"root\"\n",
"export LOGNAME=\"root\"\n",
"export HOME=\"/home/\"\n",
"export PWD=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000004\"\n",
"export LOCALIZATION_COUNTERS=\"0,548046,0,2,4\"\n",
"export JVM_PID=\"$$\"\n",
"export NM_AUX_SERVICE_mapreduce_shuffle=\"AACrRwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=\"\n",
"export STDOUT_LOGFILE_ENV=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000004/stdout\"\n",
"export SHELL=\"/bin/bash\"\n",
"export HADOOP_ROOT_LOGGER=\"INFO,console\"\n",
"export CLASSPATH=\"$PWD:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:/content/hadoop-3.4.0/share/hadoop/mapreduce/*:/content/hadoop-3.4.0/share/hadoop/mapreduce/lib/*:job.jar/*:job.jar/classes/:job.jar/lib/*:$PWD/*\"\n",
"export LD_LIBRARY_PATH=\"$PWD:$HADOOP_COMMON_HOME/lib/native\"\n",
"export STDERR_LOGFILE_ENV=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000004/stderr\"\n",
"export HADOOP_CLIENT_OPTS=\"\"\n",
"export MALLOC_ARENA_MAX=\"4\"\n",
"echo \"Setting up job resources\"\n",
"ln -sf -- \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/13/job.xml\" \"job.xml\"\n",
"ln -sf -- \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/11/job.jar\" \"job.jar\"\n",
"echo \"Copying debugging information\"\n",
"# Creating copy of launch script\n",
"cp \"launch_container.sh\" \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000004/launch_container.sh\"\n",
"chmod 640 \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000004/launch_container.sh\"\n",
"# Determining directory contents\n",
"echo \"ls -l:\" 1>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000004/directory.info\"\n",
"ls -l 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000004/directory.info\"\n",
"echo \"find -L . -maxdepth 5 -ls:\" 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000004/directory.info\"\n",
"find -L . -maxdepth 5 -ls 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000004/directory.info\"\n",
"echo \"broken symlinks(find -L . -maxdepth 5 -type l -ls):\" 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000004/directory.info\"\n",
"find -L . -maxdepth 5 -type l -ls 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000004/directory.info\"\n",
"echo \"Launching container\"\n",
"exec /bin/bash -c \"$JAVA_HOME/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx820m -Djava.io.tmpdir=$PWD/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000004 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.0.1 40041 attempt_1722798834650_0001_m_000002_0 4 1>/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000004/stdout 2>/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000004/stderr \"\n",
"\n",
"End of LogType:launch_container.sh\n",
"************************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000004 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:prelaunch.err\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:0\n",
"LogContents:\n",
"\n",
"End of LogType:prelaunch.err\n",
"******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000004 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:prelaunch.out\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:100\n",
"LogContents:\n",
"Setting up env variables\n",
"Setting up job resources\n",
"Copying debugging information\n",
"Launching container\n",
"\n",
"End of LogType:prelaunch.out\n",
"******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000004 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:stderr\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:0\n",
"LogContents:\n",
"\n",
"End of LogType:stderr\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000004 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:stdout\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:0\n",
"LogContents:\n",
"\n",
"End of LogType:stdout\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000004 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:syslog\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:31510\n",
"LogContents:\n",
"2024-08-04 19:14:39,635 INFO [main] org.apache.hadoop.security.SecurityUtil: Updating Configuration\n",
"2024-08-04 19:14:40,070 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties\n",
"2024-08-04 19:14:40,481 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).\n",
"2024-08-04 19:14:40,481 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started\n",
"2024-08-04 19:14:40,778 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens: [Kind: mapreduce.job, Service: job_1722798834650_0001, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@70f02c32)]\n",
"2024-08-04 19:14:40,910 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.\n",
"2024-08-04 19:14:41,581 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001\n",
"2024-08-04 19:14:43,074 INFO [main] org.apache.hadoop.mapred.YarnChild: \n",
"/************************************************************\n",
"[system properties]\n",
"os.name: Linux\n",
"os.version: 6.1.85+\n",
"java.home: /usr/lib/jvm/java-11-openjdk-amd64\n",
"java.runtime.version: 11.0.24+8-post-Ubuntu-1ubuntu322.04\n",
"java.vendor: Ubuntu\n",
"java.version: 11.0.24\n",
"java.vm.name: OpenJDK 64-Bit Server VM\n",
"java.class.path: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000004:/content/hadoop-3.4.0/etc/hadoop:/content/hadoop-3.4.0/share/hadoop/common/hadoop-kms-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-common-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-registry-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-nfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/j2objc-annotations-1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-shaded-guava-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-beanutils-1.9.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsch-0.1.55.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsp-api-2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-io-2.14.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-all-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-cli-1.5.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/bcprov-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/woodstox-core-5.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-servlet-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-http-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-kqueue-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/token-provider-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-crypto-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-logging-1.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-buffer-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-auth-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/guava-27.0-jre.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-xdr-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-proxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/re2j-1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-memcache-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-util-ajax-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-text-1.10.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/metrics-core-3.2.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-unix-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/avro-1.9.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/reload4j-1.2.22.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-configuration2-2.8.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-math3-3.6.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-daemon-1.0.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-core-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-codec-1.15.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-xml-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-lang3-3.12.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/animal-sniffer-annotations-1.17.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-client-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jettison-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-config-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-server-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-redis-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-asn1-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-identity-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-udt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-ssl-ocsp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/gson-2.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsr305-3.0.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-framework-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-databind-2.12.7.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-stomp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/httpcore-4.4.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-socks-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/zookeeper-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/slf4j-api-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-annotations-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-classes-kqueue-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-webapp-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-admin-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-recipes-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-server-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-pkix-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-xml-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-classes-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-io-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-haproxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-http-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jline-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/checker-qual-2.5.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-client-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-simplekdc-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-core-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-util-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-http2-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-collections-3.2.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jakarta.activation-api-1.2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-classes-macos-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-rxtx-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/zookeeper-jute-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-security-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-net-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/failureaccess-1.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-compress-1.24.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/stax2-api-4.2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-core-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final-linux-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-json-1.20.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/audience-annotations-0.12.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/dnsjava-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/nimbus-jose-jwt-9.31.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-common-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-shaded-protobuf_3_21-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/httpclient-4.5.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jul-to-slf4j-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-mqtt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-kqueue-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final-linux-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-smtp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/snappy-java-1.1.10.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-sctp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-native-client-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-client-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-rbf-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-nfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-rbf-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-native-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/j2objc-annotations-1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-shaded-guava-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-beanutils-1.9.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsch-0.1.55.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-io-2.14.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-all-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-cli-1.5.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/woodstox-core-5.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-servlet-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-http-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/token-provider-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-crypto-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-logging-1.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-buffer-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-auth-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/guava-27.0-jre.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-xdr-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-proxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/re2j-1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-memcache-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-util-ajax-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-text-1.10.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/HikariCP-4.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/metrics-core-3.2.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-unix-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/avro-1.9.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/reload4j-1.2.22.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-configuration2-2.8.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-math3-3.6.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-core-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-codec-1.15.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-xml-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-lang3-3.12.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/animal-sniffer-annotations-1.17.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-client-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jettison-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-config-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-server-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-redis-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-asn1-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-identity-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-udt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-ssl-ocsp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/gson-2.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsr305-3.0.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-framework-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-databind-2.12.7.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-stomp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/httpcore-4.4.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-socks-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/zookeeper-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-annotations-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-classes-kqueue-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-webapp-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-admin-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-recipes-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-server-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-pkix-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-xml-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-classes-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-io-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-haproxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-http-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jline-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/checker-qual-2.5.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-client-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-simplekdc-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-core-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-util-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-http2-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jakarta.activation-api-1.2.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-classes-macos-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-rxtx-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/zookeeper-jute-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-security-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-net-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/failureaccess-1.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-compress-1.24.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/stax2-api-4.2.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-core-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final-linux-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-json-1.20.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/audience-annotations-0.12.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/dnsjava-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/nimbus-jose-jwt-9.31.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-common-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-shaded-protobuf_3_21-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/httpclient-4.5.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-mqtt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final-linux-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-smtp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/snappy-java-1.1.10.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-sctp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-services-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-mawo-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-router-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-registry-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-api-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-globalpolicygenerator-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-services-api-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-tests-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jakarta.xml.bind-api-2.3.2.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-plus-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/objenesis-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.inject-1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jna-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.websocket-client-api-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/bcpkix-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/snakeyaml-2.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/guice-servlet-4.2.3.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/codemodel-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/asm-tree-9.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-client-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-api-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-client-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/bcutil-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/fst-2.50.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax-websocket-client-impl-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.websocket-api-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/guice-4.2.3.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jsonschema2pojo-core-1.0.2.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-common-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-jndi-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/asm-commons-9.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax-websocket-server-impl-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jersey-guice-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jersey-client-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-annotations-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-jaxrs-base-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/mockito-core-2.28.2.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/lib/*:job.jar/job.jar:job.jar/classes/:job.jar/lib/*:/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000004/job.jar\n",
"java.io.tmpdir: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000004/tmp\n",
"user.dir: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000004\n",
"user.name: root\n",
"************************************************************/\n",
"2024-08-04 19:14:43,083 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id\n",
"2024-08-04 19:14:44,501 INFO [main] org.apache.hadoop.mapreduce.lib.output.PathOutputCommitterFactory: No output committer factory defined, defaulting to FileOutputCommitterFactory\n",
"2024-08-04 19:14:44,503 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 2\n",
"2024-08-04 19:14:44,503 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false\n",
"2024-08-04 19:14:44,574 INFO [main] org.apache.hadoop.mapred.Task: Using ResourceCalculatorProcessTree : [ ]\n",
"2024-08-04 19:14:45,146 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: hdfs://localhost:8902/user/root/QuasiMonteCarlo_1722798838891_1650860168/in/part2:0+118\n",
"2024-08-04 19:14:45,350 INFO [main] org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)\n",
"2024-08-04 19:14:45,350 INFO [main] org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100\n",
"2024-08-04 19:14:45,350 INFO [main] org.apache.hadoop.mapred.MapTask: soft limit at 83886080\n",
"2024-08-04 19:14:45,350 INFO [main] org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600\n",
"2024-08-04 19:14:45,350 INFO [main] org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600\n",
"2024-08-04 19:14:45,390 INFO [main] org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer\n",
"2024-08-04 19:14:45,530 INFO [main] org.apache.hadoop.mapred.MapTask: Starting flush of map output\n",
"2024-08-04 19:14:45,530 INFO [main] org.apache.hadoop.mapred.MapTask: Spilling map output\n",
"2024-08-04 19:14:45,530 INFO [main] org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 18; bufvoid = 104857600\n",
"2024-08-04 19:14:45,530 INFO [main] org.apache.hadoop.mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600\n",
"2024-08-04 19:14:45,541 INFO [main] org.apache.hadoop.mapred.MapTask: Finished spill 0\n",
"2024-08-04 19:14:45,602 INFO [main] org.apache.hadoop.mapred.Task: Task:attempt_1722798834650_0001_m_000002_0 is done. And is in the process of committing\n",
"2024-08-04 19:14:45,640 INFO [main] org.apache.hadoop.mapred.Task: Task 'attempt_1722798834650_0001_m_000002_0' done.\n",
"2024-08-04 19:14:45,661 INFO [main] org.apache.hadoop.mapred.Task: Final Counters for attempt_1722798834650_0001_m_000002_0: Counters: 28\n",
"\tFile System Counters\n",
"\t\tFILE: Number of bytes read=0\n",
"\t\tFILE: Number of bytes written=309460\n",
"\t\tFILE: Number of read operations=0\n",
"\t\tFILE: Number of large read operations=0\n",
"\t\tFILE: Number of write operations=0\n",
"\t\tHDFS: Number of bytes read=264\n",
"\t\tHDFS: Number of bytes written=0\n",
"\t\tHDFS: Number of read operations=4\n",
"\t\tHDFS: Number of large read operations=0\n",
"\t\tHDFS: Number of write operations=0\n",
"\t\tHDFS: Number of bytes read erasure-coded=0\n",
"\tMap-Reduce Framework\n",
"\t\tMap input records=1\n",
"\t\tMap output records=2\n",
"\t\tMap output bytes=18\n",
"\t\tMap output materialized bytes=28\n",
"\t\tInput split bytes=146\n",
"\t\tCombine input records=0\n",
"\t\tSpilled Records=2\n",
"\t\tFailed Shuffles=0\n",
"\t\tMerged Map outputs=0\n",
"\t\tGC time elapsed (ms)=74\n",
"\t\tCPU time spent (ms)=920\n",
"\t\tPhysical memory (bytes) snapshot=327950336\n",
"\t\tVirtual memory (bytes) snapshot=2719514624\n",
"\t\tTotal committed heap usage (bytes)=216006656\n",
"\t\tPeak Map Physical memory (bytes)=327950336\n",
"\t\tPeak Map Virtual memory (bytes)=2719514624\n",
"\tFile Input Format Counters \n",
"\t\tBytes Read=118\n",
"2024-08-04 19:14:45,663 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system...\n",
"2024-08-04 19:14:45,663 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped.\n",
"2024-08-04 19:14:45,664 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system shutdown complete.\n",
"\n",
"End of LogType:syslog\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000005 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:directory.info\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:2101\n",
"LogContents:\n",
"ls -l:\n",
"total 32\n",
"-rw-r--r-- 1 root root 129 Aug 4 19:14 container_tokens\n",
"-rwx------ 1 root root 964 Aug 4 19:14 default_container_executor_session.sh\n",
"-rwx------ 1 root root 1019 Aug 4 19:14 default_container_executor.sh\n",
"lrwxrwxrwx 1 root root 179 Aug 4 19:14 job.jar -> /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/11/job.jar\n",
"lrwxrwxrwx 1 root root 179 Aug 4 19:14 job.xml -> /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/13/job.xml\n",
"-rwx------ 1 root root 8159 Aug 4 19:14 launch_container.sh\n",
"drwx--x--- 2 root root 4096 Aug 4 19:14 tmp\n",
"find -L . -maxdepth 5 -ls:\n",
" 809830 4 drwx--x--- 3 root root 4096 Aug 4 19:14 .\n",
" 809860 4 -rwx------ 1 root root 1019 Aug 4 19:14 ./default_container_executor.sh\n",
" 809751 260 -r-x------ 1 root root 264361 Aug 4 19:14 ./job.xml\n",
" 809845 4 -rw-r--r-- 1 root root 129 Aug 4 19:14 ./container_tokens\n",
" 809857 4 -rwx------ 1 root root 964 Aug 4 19:14 ./default_container_executor_session.sh\n",
" 809851 8 -rwx------ 1 root root 8159 Aug 4 19:14 ./launch_container.sh\n",
" 809864 4 -rw-r--r-- 1 root root 16 Aug 4 19:14 ./.default_container_executor.sh.crc\n",
" 809852 4 -rw-r--r-- 1 root root 72 Aug 4 19:14 ./.launch_container.sh.crc\n",
" 809745 4 drwx------ 2 root root 4096 Aug 4 19:14 ./job.jar\n",
" 809746 276 -r-x------ 1 root root 281609 Aug 4 19:14 ./job.jar/job.jar\n",
" 809846 4 -rw-r--r-- 1 root root 12 Aug 4 19:14 ./.container_tokens.crc\n",
" 809858 4 -rw-r--r-- 1 root root 16 Aug 4 19:14 ./.default_container_executor_session.sh.crc\n",
" 809843 4 drwx--x--- 2 root root 4096 Aug 4 19:14 ./tmp\n",
"broken symlinks(find -L . -maxdepth 5 -type l -ls):\n",
"\n",
"End of LogType:directory.info\n",
"*******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000005 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:launch_container.sh\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:8159\n",
"LogContents:\n",
"#!/bin/bash\n",
"\n",
"set -o pipefail -e\n",
"export PRELAUNCH_OUT=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000005/prelaunch.out\"\n",
"exec >\"${PRELAUNCH_OUT}\"\n",
"export PRELAUNCH_ERR=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000005/prelaunch.err\"\n",
"exec 2>\"${PRELAUNCH_ERR}\"\n",
"echo \"Setting up env variables\"\n",
"export JAVA_HOME=${JAVA_HOME:-\"/usr/lib/jvm/java-11-openjdk-amd64\"}\n",
"export HADOOP_COMMON_HOME=${HADOOP_COMMON_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_HDFS_HOME=${HADOOP_HDFS_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-\"/content/hadoop-3.4.0/etc/hadoop\"}\n",
"export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_HOME=${HADOOP_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export PATH=${PATH:-\"/content/hadoop-3.4.0/bin:/opt/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/tools/node/bin:/tools/google-cloud-sdk/bin\"}\n",
"export LANG=${LANG:-\"en_US.UTF-8\"}\n",
"export HADOOP_TOKEN_FILE_LOCATION=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000005/container_tokens\"\n",
"export CONTAINER_ID=\"container_1722798834650_0001_01_000005\"\n",
"export NM_PORT=\"34535\"\n",
"export NM_HOST=\"localhost\"\n",
"export NM_HTTP_PORT=\"37361\"\n",
"export LOCAL_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001\"\n",
"export LOCAL_USER_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/\"\n",
"export LOG_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_0/application_1722798834650_0001/container_1722798834650_0001_01_000005,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000005,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000005,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000005\"\n",
"export USER=\"root\"\n",
"export LOGNAME=\"root\"\n",
"export HOME=\"/home/\"\n",
"export PWD=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000005\"\n",
"export LOCALIZATION_COUNTERS=\"0,548046,0,2,6\"\n",
"export JVM_PID=\"$$\"\n",
"export NM_AUX_SERVICE_mapreduce_shuffle=\"AACrRwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=\"\n",
"export STDOUT_LOGFILE_ENV=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000005/stdout\"\n",
"export SHELL=\"/bin/bash\"\n",
"export HADOOP_ROOT_LOGGER=\"INFO,console\"\n",
"export CLASSPATH=\"$PWD:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:/content/hadoop-3.4.0/share/hadoop/mapreduce/*:/content/hadoop-3.4.0/share/hadoop/mapreduce/lib/*:job.jar/*:job.jar/classes/:job.jar/lib/*:$PWD/*\"\n",
"export LD_LIBRARY_PATH=\"$PWD:$HADOOP_COMMON_HOME/lib/native\"\n",
"export STDERR_LOGFILE_ENV=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000005/stderr\"\n",
"export HADOOP_CLIENT_OPTS=\"\"\n",
"export MALLOC_ARENA_MAX=\"4\"\n",
"echo \"Setting up job resources\"\n",
"ln -sf -- \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/13/job.xml\" \"job.xml\"\n",
"ln -sf -- \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/11/job.jar\" \"job.jar\"\n",
"echo \"Copying debugging information\"\n",
"# Creating copy of launch script\n",
"cp \"launch_container.sh\" \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000005/launch_container.sh\"\n",
"chmod 640 \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000005/launch_container.sh\"\n",
"# Determining directory contents\n",
"echo \"ls -l:\" 1>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000005/directory.info\"\n",
"ls -l 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000005/directory.info\"\n",
"echo \"find -L . -maxdepth 5 -ls:\" 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000005/directory.info\"\n",
"find -L . -maxdepth 5 -ls 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000005/directory.info\"\n",
"echo \"broken symlinks(find -L . -maxdepth 5 -type l -ls):\" 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000005/directory.info\"\n",
"find -L . -maxdepth 5 -type l -ls 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000005/directory.info\"\n",
"echo \"Launching container\"\n",
"exec /bin/bash -c \"$JAVA_HOME/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx820m -Djava.io.tmpdir=$PWD/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000005 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.0.1 40041 attempt_1722798834650_0001_m_000003_0 5 1>/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000005/stdout 2>/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000005/stderr \"\n",
"\n",
"End of LogType:launch_container.sh\n",
"************************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000005 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:prelaunch.err\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:0\n",
"LogContents:\n",
"\n",
"End of LogType:prelaunch.err\n",
"******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000005 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:prelaunch.out\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:100\n",
"LogContents:\n",
"Setting up env variables\n",
"Setting up job resources\n",
"Copying debugging information\n",
"Launching container\n",
"\n",
"End of LogType:prelaunch.out\n",
"******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000005 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:stderr\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:0\n",
"LogContents:\n",
"\n",
"End of LogType:stderr\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000005 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:stdout\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:0\n",
"LogContents:\n",
"\n",
"End of LogType:stdout\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000005 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:syslog\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:31510\n",
"LogContents:\n",
"2024-08-04 19:14:39,501 INFO [main] org.apache.hadoop.security.SecurityUtil: Updating Configuration\n",
"2024-08-04 19:14:39,908 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties\n",
"2024-08-04 19:14:40,230 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).\n",
"2024-08-04 19:14:40,230 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started\n",
"2024-08-04 19:14:40,517 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens: [Kind: mapreduce.job, Service: job_1722798834650_0001, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@70f02c32)]\n",
"2024-08-04 19:14:40,663 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.\n",
"2024-08-04 19:14:41,380 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001\n",
"2024-08-04 19:14:42,874 INFO [main] org.apache.hadoop.mapred.YarnChild: \n",
"/************************************************************\n",
"[system properties]\n",
"os.name: Linux\n",
"os.version: 6.1.85+\n",
"java.home: /usr/lib/jvm/java-11-openjdk-amd64\n",
"java.runtime.version: 11.0.24+8-post-Ubuntu-1ubuntu322.04\n",
"java.vendor: Ubuntu\n",
"java.version: 11.0.24\n",
"java.vm.name: OpenJDK 64-Bit Server VM\n",
"java.class.path: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000005:/content/hadoop-3.4.0/etc/hadoop:/content/hadoop-3.4.0/share/hadoop/common/hadoop-kms-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-common-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-registry-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-nfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/j2objc-annotations-1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-shaded-guava-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-beanutils-1.9.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsch-0.1.55.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsp-api-2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-io-2.14.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-all-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-cli-1.5.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/bcprov-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/woodstox-core-5.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-servlet-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-http-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-kqueue-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/token-provider-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-crypto-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-logging-1.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-buffer-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-auth-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/guava-27.0-jre.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-xdr-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-proxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/re2j-1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-memcache-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-util-ajax-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-text-1.10.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/metrics-core-3.2.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-unix-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/avro-1.9.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/reload4j-1.2.22.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-configuration2-2.8.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-math3-3.6.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-daemon-1.0.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-core-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-codec-1.15.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-xml-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-lang3-3.12.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/animal-sniffer-annotations-1.17.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-client-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jettison-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-config-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-server-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-redis-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-asn1-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-identity-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-udt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-ssl-ocsp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/gson-2.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsr305-3.0.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-framework-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-databind-2.12.7.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-stomp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/httpcore-4.4.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-socks-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/zookeeper-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/slf4j-api-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-annotations-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-classes-kqueue-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-webapp-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-admin-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-recipes-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-server-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-pkix-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-xml-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-classes-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-io-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-haproxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-http-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jline-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/checker-qual-2.5.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-client-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-simplekdc-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-core-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-util-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-http2-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-collections-3.2.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jakarta.activation-api-1.2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-classes-macos-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-rxtx-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/zookeeper-jute-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-security-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-net-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/failureaccess-1.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-compress-1.24.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/stax2-api-4.2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-core-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final-linux-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-json-1.20.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/audience-annotations-0.12.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/dnsjava-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/nimbus-jose-jwt-9.31.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-common-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-shaded-protobuf_3_21-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/httpclient-4.5.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jul-to-slf4j-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-mqtt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-kqueue-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final-linux-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-smtp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/snappy-java-1.1.10.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-sctp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-native-client-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-client-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-rbf-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-nfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-rbf-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-native-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/j2objc-annotations-1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-shaded-guava-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-beanutils-1.9.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsch-0.1.55.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-io-2.14.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-all-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-cli-1.5.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/woodstox-core-5.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-servlet-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-http-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/token-provider-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-crypto-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-logging-1.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-buffer-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-auth-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/guava-27.0-jre.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-xdr-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-proxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/re2j-1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-memcache-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-util-ajax-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-text-1.10.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/HikariCP-4.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/metrics-core-3.2.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-unix-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/avro-1.9.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/reload4j-1.2.22.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-configuration2-2.8.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-math3-3.6.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-core-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-codec-1.15.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-xml-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-lang3-3.12.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/animal-sniffer-annotations-1.17.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-client-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jettison-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-config-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-server-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-redis-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-asn1-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-identity-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-udt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-ssl-ocsp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/gson-2.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsr305-3.0.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-framework-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-databind-2.12.7.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-stomp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/httpcore-4.4.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-socks-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/zookeeper-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-annotations-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-classes-kqueue-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-webapp-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-admin-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-recipes-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-server-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-pkix-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-xml-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-classes-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-io-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-haproxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-http-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jline-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/checker-qual-2.5.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-client-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-simplekdc-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-core-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-util-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-http2-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jakarta.activation-api-1.2.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-classes-macos-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-rxtx-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/zookeeper-jute-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-security-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-net-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/failureaccess-1.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-compress-1.24.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/stax2-api-4.2.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-core-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final-linux-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-json-1.20.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/audience-annotations-0.12.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/dnsjava-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/nimbus-jose-jwt-9.31.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-common-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-shaded-protobuf_3_21-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/httpclient-4.5.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-mqtt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final-linux-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-smtp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/snappy-java-1.1.10.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-sctp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-services-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-mawo-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-router-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-registry-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-api-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-globalpolicygenerator-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-services-api-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-tests-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jakarta.xml.bind-api-2.3.2.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-plus-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/objenesis-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.inject-1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jna-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.websocket-client-api-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/bcpkix-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/snakeyaml-2.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/guice-servlet-4.2.3.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/codemodel-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/asm-tree-9.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-client-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-api-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-client-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/bcutil-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/fst-2.50.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax-websocket-client-impl-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.websocket-api-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/guice-4.2.3.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jsonschema2pojo-core-1.0.2.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-common-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-jndi-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/asm-commons-9.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax-websocket-server-impl-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jersey-guice-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jersey-client-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-annotations-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-jaxrs-base-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/mockito-core-2.28.2.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/lib/*:job.jar/job.jar:job.jar/classes/:job.jar/lib/*:/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000005/job.jar\n",
"java.io.tmpdir: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000005/tmp\n",
"user.dir: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000005\n",
"user.name: root\n",
"************************************************************/\n",
"2024-08-04 19:14:42,875 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id\n",
"2024-08-04 19:14:44,317 INFO [main] org.apache.hadoop.mapreduce.lib.output.PathOutputCommitterFactory: No output committer factory defined, defaulting to FileOutputCommitterFactory\n",
"2024-08-04 19:14:44,319 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 2\n",
"2024-08-04 19:14:44,319 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false\n",
"2024-08-04 19:14:44,410 INFO [main] org.apache.hadoop.mapred.Task: Using ResourceCalculatorProcessTree : [ ]\n",
"2024-08-04 19:14:45,042 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: hdfs://localhost:8902/user/root/QuasiMonteCarlo_1722798838891_1650860168/in/part3:0+118\n",
"2024-08-04 19:14:45,245 INFO [main] org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)\n",
"2024-08-04 19:14:45,245 INFO [main] org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100\n",
"2024-08-04 19:14:45,245 INFO [main] org.apache.hadoop.mapred.MapTask: soft limit at 83886080\n",
"2024-08-04 19:14:45,245 INFO [main] org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600\n",
"2024-08-04 19:14:45,245 INFO [main] org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600\n",
"2024-08-04 19:14:45,261 INFO [main] org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer\n",
"2024-08-04 19:14:45,348 INFO [main] org.apache.hadoop.mapred.MapTask: Starting flush of map output\n",
"2024-08-04 19:14:45,348 INFO [main] org.apache.hadoop.mapred.MapTask: Spilling map output\n",
"2024-08-04 19:14:45,352 INFO [main] org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 18; bufvoid = 104857600\n",
"2024-08-04 19:14:45,352 INFO [main] org.apache.hadoop.mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600\n",
"2024-08-04 19:14:45,372 INFO [main] org.apache.hadoop.mapred.MapTask: Finished spill 0\n",
"2024-08-04 19:14:45,441 INFO [main] org.apache.hadoop.mapred.Task: Task:attempt_1722798834650_0001_m_000003_0 is done. And is in the process of committing\n",
"2024-08-04 19:14:45,493 INFO [main] org.apache.hadoop.mapred.Task: Task 'attempt_1722798834650_0001_m_000003_0' done.\n",
"2024-08-04 19:14:45,517 INFO [main] org.apache.hadoop.mapred.Task: Final Counters for attempt_1722798834650_0001_m_000003_0: Counters: 28\n",
"\tFile System Counters\n",
"\t\tFILE: Number of bytes read=0\n",
"\t\tFILE: Number of bytes written=309460\n",
"\t\tFILE: Number of read operations=0\n",
"\t\tFILE: Number of large read operations=0\n",
"\t\tFILE: Number of write operations=0\n",
"\t\tHDFS: Number of bytes read=264\n",
"\t\tHDFS: Number of bytes written=0\n",
"\t\tHDFS: Number of read operations=4\n",
"\t\tHDFS: Number of large read operations=0\n",
"\t\tHDFS: Number of write operations=0\n",
"\t\tHDFS: Number of bytes read erasure-coded=0\n",
"\tMap-Reduce Framework\n",
"\t\tMap input records=1\n",
"\t\tMap output records=2\n",
"\t\tMap output bytes=18\n",
"\t\tMap output materialized bytes=28\n",
"\t\tInput split bytes=146\n",
"\t\tCombine input records=0\n",
"\t\tSpilled Records=2\n",
"\t\tFailed Shuffles=0\n",
"\t\tMerged Map outputs=0\n",
"\t\tGC time elapsed (ms)=80\n",
"\t\tCPU time spent (ms)=900\n",
"\t\tPhysical memory (bytes) snapshot=311812096\n",
"\t\tVirtual memory (bytes) snapshot=2722275328\n",
"\t\tTotal committed heap usage (bytes)=216006656\n",
"\t\tPeak Map Physical memory (bytes)=311812096\n",
"\t\tPeak Map Virtual memory (bytes)=2722275328\n",
"\tFile Input Format Counters \n",
"\t\tBytes Read=118\n",
"2024-08-04 19:14:45,519 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system...\n",
"2024-08-04 19:14:45,520 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped.\n",
"2024-08-04 19:14:45,520 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system shutdown complete.\n",
"\n",
"End of LogType:syslog\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000002 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:directory.info\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:2101\n",
"LogContents:\n",
"ls -l:\n",
"total 32\n",
"-rw-r--r-- 1 root root 129 Aug 4 19:14 container_tokens\n",
"-rwx------ 1 root root 964 Aug 4 19:14 default_container_executor_session.sh\n",
"-rwx------ 1 root root 1019 Aug 4 19:14 default_container_executor.sh\n",
"lrwxrwxrwx 1 root root 179 Aug 4 19:14 job.jar -> /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/11/job.jar\n",
"lrwxrwxrwx 1 root root 179 Aug 4 19:14 job.xml -> /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/13/job.xml\n",
"-rwx------ 1 root root 8159 Aug 4 19:14 launch_container.sh\n",
"drwx--x--- 2 root root 4096 Aug 4 19:14 tmp\n",
"find -L . -maxdepth 5 -ls:\n",
" 809832 4 drwx--x--- 3 root root 4096 Aug 4 19:14 .\n",
" 809849 4 -rwx------ 1 root root 1019 Aug 4 19:14 ./default_container_executor.sh\n",
" 809751 260 -r-x------ 1 root root 264361 Aug 4 19:14 ./job.xml\n",
" 809840 4 -rw-r--r-- 1 root root 129 Aug 4 19:14 ./container_tokens\n",
" 809847 4 -rwx------ 1 root root 964 Aug 4 19:14 ./default_container_executor_session.sh\n",
" 809845 8 -rwx------ 1 root root 8159 Aug 4 19:14 ./launch_container.sh\n",
" 809851 4 -rw-r--r-- 1 root root 16 Aug 4 19:14 ./.default_container_executor.sh.crc\n",
" 809846 4 -rw-r--r-- 1 root root 72 Aug 4 19:14 ./.launch_container.sh.crc\n",
" 809745 4 drwx------ 2 root root 4096 Aug 4 19:14 ./job.jar\n",
" 809746 276 -r-x------ 1 root root 281609 Aug 4 19:14 ./job.jar/job.jar\n",
" 809841 4 -rw-r--r-- 1 root root 12 Aug 4 19:14 ./.container_tokens.crc\n",
" 809848 4 -rw-r--r-- 1 root root 16 Aug 4 19:14 ./.default_container_executor_session.sh.crc\n",
" 809839 4 drwx--x--- 2 root root 4096 Aug 4 19:14 ./tmp\n",
"broken symlinks(find -L . -maxdepth 5 -type l -ls):\n",
"\n",
"End of LogType:directory.info\n",
"*******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000002 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:launch_container.sh\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:8159\n",
"LogContents:\n",
"#!/bin/bash\n",
"\n",
"set -o pipefail -e\n",
"export PRELAUNCH_OUT=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000002/prelaunch.out\"\n",
"exec >\"${PRELAUNCH_OUT}\"\n",
"export PRELAUNCH_ERR=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000002/prelaunch.err\"\n",
"exec 2>\"${PRELAUNCH_ERR}\"\n",
"echo \"Setting up env variables\"\n",
"export JAVA_HOME=${JAVA_HOME:-\"/usr/lib/jvm/java-11-openjdk-amd64\"}\n",
"export HADOOP_COMMON_HOME=${HADOOP_COMMON_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_HDFS_HOME=${HADOOP_HDFS_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-\"/content/hadoop-3.4.0/etc/hadoop\"}\n",
"export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_HOME=${HADOOP_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export PATH=${PATH:-\"/content/hadoop-3.4.0/bin:/opt/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/tools/node/bin:/tools/google-cloud-sdk/bin\"}\n",
"export LANG=${LANG:-\"en_US.UTF-8\"}\n",
"export HADOOP_TOKEN_FILE_LOCATION=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000002/container_tokens\"\n",
"export CONTAINER_ID=\"container_1722798834650_0001_01_000002\"\n",
"export NM_PORT=\"34535\"\n",
"export NM_HOST=\"localhost\"\n",
"export NM_HTTP_PORT=\"37361\"\n",
"export LOCAL_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001\"\n",
"export LOCAL_USER_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/\"\n",
"export LOG_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_0/application_1722798834650_0001/container_1722798834650_0001_01_000002,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000002,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000002,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000002\"\n",
"export USER=\"root\"\n",
"export LOGNAME=\"root\"\n",
"export HOME=\"/home/\"\n",
"export PWD=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000002\"\n",
"export LOCALIZATION_COUNTERS=\"0,548046,0,2,5\"\n",
"export JVM_PID=\"$$\"\n",
"export NM_AUX_SERVICE_mapreduce_shuffle=\"AACrRwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=\"\n",
"export STDOUT_LOGFILE_ENV=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000002/stdout\"\n",
"export SHELL=\"/bin/bash\"\n",
"export HADOOP_ROOT_LOGGER=\"INFO,console\"\n",
"export CLASSPATH=\"$PWD:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:/content/hadoop-3.4.0/share/hadoop/mapreduce/*:/content/hadoop-3.4.0/share/hadoop/mapreduce/lib/*:job.jar/*:job.jar/classes/:job.jar/lib/*:$PWD/*\"\n",
"export LD_LIBRARY_PATH=\"$PWD:$HADOOP_COMMON_HOME/lib/native\"\n",
"export STDERR_LOGFILE_ENV=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000002/stderr\"\n",
"export HADOOP_CLIENT_OPTS=\"\"\n",
"export MALLOC_ARENA_MAX=\"4\"\n",
"echo \"Setting up job resources\"\n",
"ln -sf -- \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/13/job.xml\" \"job.xml\"\n",
"ln -sf -- \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/11/job.jar\" \"job.jar\"\n",
"echo \"Copying debugging information\"\n",
"# Creating copy of launch script\n",
"cp \"launch_container.sh\" \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000002/launch_container.sh\"\n",
"chmod 640 \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000002/launch_container.sh\"\n",
"# Determining directory contents\n",
"echo \"ls -l:\" 1>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000002/directory.info\"\n",
"ls -l 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000002/directory.info\"\n",
"echo \"find -L . -maxdepth 5 -ls:\" 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000002/directory.info\"\n",
"find -L . -maxdepth 5 -ls 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000002/directory.info\"\n",
"echo \"broken symlinks(find -L . -maxdepth 5 -type l -ls):\" 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000002/directory.info\"\n",
"find -L . -maxdepth 5 -type l -ls 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000002/directory.info\"\n",
"echo \"Launching container\"\n",
"exec /bin/bash -c \"$JAVA_HOME/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx820m -Djava.io.tmpdir=$PWD/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000002 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.0.1 40041 attempt_1722798834650_0001_m_000000_0 2 1>/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000002/stdout 2>/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000002/stderr \"\n",
"\n",
"End of LogType:launch_container.sh\n",
"************************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000002 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:prelaunch.err\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:0\n",
"LogContents:\n",
"\n",
"End of LogType:prelaunch.err\n",
"******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000002 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:prelaunch.out\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:100\n",
"LogContents:\n",
"Setting up env variables\n",
"Setting up job resources\n",
"Copying debugging information\n",
"Launching container\n",
"\n",
"End of LogType:prelaunch.out\n",
"******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000002 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:stderr\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:0\n",
"LogContents:\n",
"\n",
"End of LogType:stderr\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000002 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:stdout\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:0\n",
"LogContents:\n",
"\n",
"End of LogType:stdout\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000002 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:syslog\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:31510\n",
"LogContents:\n",
"2024-08-04 19:14:26,405 INFO [main] org.apache.hadoop.security.SecurityUtil: Updating Configuration\n",
"2024-08-04 19:14:26,656 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties\n",
"2024-08-04 19:14:27,095 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).\n",
"2024-08-04 19:14:27,095 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started\n",
"2024-08-04 19:14:27,421 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens: [Kind: mapreduce.job, Service: job_1722798834650_0001, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@70f02c32)]\n",
"2024-08-04 19:14:27,614 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.\n",
"2024-08-04 19:14:28,578 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001\n",
"2024-08-04 19:14:30,299 INFO [main] org.apache.hadoop.mapred.YarnChild: \n",
"/************************************************************\n",
"[system properties]\n",
"os.name: Linux\n",
"os.version: 6.1.85+\n",
"java.home: /usr/lib/jvm/java-11-openjdk-amd64\n",
"java.runtime.version: 11.0.24+8-post-Ubuntu-1ubuntu322.04\n",
"java.vendor: Ubuntu\n",
"java.version: 11.0.24\n",
"java.vm.name: OpenJDK 64-Bit Server VM\n",
"java.class.path: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000002:/content/hadoop-3.4.0/etc/hadoop:/content/hadoop-3.4.0/share/hadoop/common/hadoop-kms-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-common-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-registry-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-nfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/j2objc-annotations-1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-shaded-guava-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-beanutils-1.9.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsch-0.1.55.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsp-api-2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-io-2.14.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-all-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-cli-1.5.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/bcprov-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/woodstox-core-5.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-servlet-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-http-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-kqueue-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/token-provider-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-crypto-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-logging-1.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-buffer-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-auth-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/guava-27.0-jre.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-xdr-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-proxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/re2j-1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-memcache-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-util-ajax-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-text-1.10.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/metrics-core-3.2.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-unix-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/avro-1.9.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/reload4j-1.2.22.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-configuration2-2.8.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-math3-3.6.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-daemon-1.0.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-core-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-codec-1.15.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-xml-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-lang3-3.12.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/animal-sniffer-annotations-1.17.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-client-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jettison-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-config-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-server-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-redis-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-asn1-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-identity-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-udt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-ssl-ocsp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/gson-2.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsr305-3.0.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-framework-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-databind-2.12.7.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-stomp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/httpcore-4.4.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-socks-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/zookeeper-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/slf4j-api-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-annotations-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-classes-kqueue-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-webapp-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-admin-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-recipes-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-server-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-pkix-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-xml-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-classes-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-io-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-haproxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-http-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jline-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/checker-qual-2.5.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-client-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-simplekdc-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-core-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-util-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-http2-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-collections-3.2.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jakarta.activation-api-1.2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-classes-macos-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-rxtx-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/zookeeper-jute-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-security-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-net-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/failureaccess-1.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-compress-1.24.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/stax2-api-4.2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-core-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final-linux-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-json-1.20.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/audience-annotations-0.12.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/dnsjava-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/nimbus-jose-jwt-9.31.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-common-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-shaded-protobuf_3_21-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/httpclient-4.5.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jul-to-slf4j-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-mqtt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-kqueue-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final-linux-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-smtp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/snappy-java-1.1.10.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-sctp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-native-client-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-client-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-rbf-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-nfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-rbf-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-native-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/j2objc-annotations-1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-shaded-guava-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-beanutils-1.9.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsch-0.1.55.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-io-2.14.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-all-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-cli-1.5.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/woodstox-core-5.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-servlet-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-http-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/token-provider-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-crypto-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-logging-1.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-buffer-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-auth-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/guava-27.0-jre.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-xdr-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-proxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/re2j-1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-memcache-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-util-ajax-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-text-1.10.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/HikariCP-4.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/metrics-core-3.2.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-unix-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/avro-1.9.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/reload4j-1.2.22.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-configuration2-2.8.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-math3-3.6.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-core-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-codec-1.15.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-xml-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-lang3-3.12.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/animal-sniffer-annotations-1.17.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-client-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jettison-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-config-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-server-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-redis-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-asn1-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-identity-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-udt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-ssl-ocsp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/gson-2.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsr305-3.0.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-framework-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-databind-2.12.7.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-stomp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/httpcore-4.4.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-socks-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/zookeeper-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-annotations-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-classes-kqueue-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-webapp-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-admin-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-recipes-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-server-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-pkix-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-xml-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-classes-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-io-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-haproxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-http-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jline-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/checker-qual-2.5.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-client-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-simplekdc-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-core-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-util-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-http2-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jakarta.activation-api-1.2.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-classes-macos-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-rxtx-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/zookeeper-jute-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-security-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-net-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/failureaccess-1.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-compress-1.24.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/stax2-api-4.2.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-core-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final-linux-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-json-1.20.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/audience-annotations-0.12.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/dnsjava-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/nimbus-jose-jwt-9.31.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-common-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-shaded-protobuf_3_21-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/httpclient-4.5.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-mqtt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final-linux-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-smtp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/snappy-java-1.1.10.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-sctp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-services-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-mawo-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-router-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-registry-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-api-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-globalpolicygenerator-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-services-api-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-tests-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jakarta.xml.bind-api-2.3.2.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-plus-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/objenesis-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.inject-1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jna-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.websocket-client-api-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/bcpkix-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/snakeyaml-2.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/guice-servlet-4.2.3.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/codemodel-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/asm-tree-9.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-client-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-api-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-client-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/bcutil-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/fst-2.50.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax-websocket-client-impl-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.websocket-api-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/guice-4.2.3.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jsonschema2pojo-core-1.0.2.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-common-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-jndi-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/asm-commons-9.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax-websocket-server-impl-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jersey-guice-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jersey-client-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-annotations-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-jaxrs-base-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/mockito-core-2.28.2.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/lib/*:job.jar/job.jar:job.jar/classes/:job.jar/lib/*:/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000002/job.jar\n",
"java.io.tmpdir: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000002/tmp\n",
"user.dir: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000002\n",
"user.name: root\n",
"************************************************************/\n",
"2024-08-04 19:14:30,300 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id\n",
"2024-08-04 19:14:31,952 INFO [main] org.apache.hadoop.mapreduce.lib.output.PathOutputCommitterFactory: No output committer factory defined, defaulting to FileOutputCommitterFactory\n",
"2024-08-04 19:14:31,957 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 2\n",
"2024-08-04 19:14:31,958 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false\n",
"2024-08-04 19:14:32,045 INFO [main] org.apache.hadoop.mapred.Task: Using ResourceCalculatorProcessTree : [ ]\n",
"2024-08-04 19:14:32,998 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: hdfs://localhost:8902/user/root/QuasiMonteCarlo_1722798838891_1650860168/in/part0:0+118\n",
"2024-08-04 19:14:33,519 INFO [main] org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)\n",
"2024-08-04 19:14:33,520 INFO [main] org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100\n",
"2024-08-04 19:14:33,520 INFO [main] org.apache.hadoop.mapred.MapTask: soft limit at 83886080\n",
"2024-08-04 19:14:33,520 INFO [main] org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600\n",
"2024-08-04 19:14:33,520 INFO [main] org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600\n",
"2024-08-04 19:14:33,562 INFO [main] org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer\n",
"2024-08-04 19:14:33,974 INFO [main] org.apache.hadoop.mapred.MapTask: Starting flush of map output\n",
"2024-08-04 19:14:33,974 INFO [main] org.apache.hadoop.mapred.MapTask: Spilling map output\n",
"2024-08-04 19:14:33,974 INFO [main] org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 18; bufvoid = 104857600\n",
"2024-08-04 19:14:33,974 INFO [main] org.apache.hadoop.mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600\n",
"2024-08-04 19:14:34,008 INFO [main] org.apache.hadoop.mapred.MapTask: Finished spill 0\n",
"2024-08-04 19:14:34,066 INFO [main] org.apache.hadoop.mapred.Task: Task:attempt_1722798834650_0001_m_000000_0 is done. And is in the process of committing\n",
"2024-08-04 19:14:34,129 INFO [main] org.apache.hadoop.mapred.Task: Task 'attempt_1722798834650_0001_m_000000_0' done.\n",
"2024-08-04 19:14:34,161 INFO [main] org.apache.hadoop.mapred.Task: Final Counters for attempt_1722798834650_0001_m_000000_0: Counters: 28\n",
"\tFile System Counters\n",
"\t\tFILE: Number of bytes read=0\n",
"\t\tFILE: Number of bytes written=309460\n",
"\t\tFILE: Number of read operations=0\n",
"\t\tFILE: Number of large read operations=0\n",
"\t\tFILE: Number of write operations=0\n",
"\t\tHDFS: Number of bytes read=264\n",
"\t\tHDFS: Number of bytes written=0\n",
"\t\tHDFS: Number of read operations=4\n",
"\t\tHDFS: Number of large read operations=0\n",
"\t\tHDFS: Number of write operations=0\n",
"\t\tHDFS: Number of bytes read erasure-coded=0\n",
"\tMap-Reduce Framework\n",
"\t\tMap input records=1\n",
"\t\tMap output records=2\n",
"\t\tMap output bytes=18\n",
"\t\tMap output materialized bytes=28\n",
"\t\tInput split bytes=146\n",
"\t\tCombine input records=0\n",
"\t\tSpilled Records=2\n",
"\t\tFailed Shuffles=0\n",
"\t\tMerged Map outputs=0\n",
"\t\tGC time elapsed (ms)=88\n",
"\t\tCPU time spent (ms)=880\n",
"\t\tPhysical memory (bytes) snapshot=331776000\n",
"\t\tVirtual memory (bytes) snapshot=2719916032\n",
"\t\tTotal committed heap usage (bytes)=216006656\n",
"\t\tPeak Map Physical memory (bytes)=331776000\n",
"\t\tPeak Map Virtual memory (bytes)=2719916032\n",
"\tFile Input Format Counters \n",
"\t\tBytes Read=118\n",
"2024-08-04 19:14:34,163 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system...\n",
"2024-08-04 19:14:34,164 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped.\n",
"2024-08-04 19:14:34,164 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system shutdown complete.\n",
"\n",
"End of LogType:syslog\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000003 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:directory.info\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:2101\n",
"LogContents:\n",
"ls -l:\n",
"total 32\n",
"-rw-r--r-- 1 root root 129 Aug 4 19:14 container_tokens\n",
"-rwx------ 1 root root 964 Aug 4 19:14 default_container_executor_session.sh\n",
"-rwx------ 1 root root 1019 Aug 4 19:14 default_container_executor.sh\n",
"lrwxrwxrwx 1 root root 179 Aug 4 19:14 job.jar -> /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/11/job.jar\n",
"lrwxrwxrwx 1 root root 179 Aug 4 19:14 job.xml -> /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/13/job.xml\n",
"-rwx------ 1 root root 8160 Aug 4 19:14 launch_container.sh\n",
"drwx--x--- 2 root root 4096 Aug 4 19:14 tmp\n",
"find -L . -maxdepth 5 -ls:\n",
" 809809 4 drwx--x--- 3 root root 4096 Aug 4 19:14 .\n",
" 809824 4 -rwx------ 1 root root 1019 Aug 4 19:14 ./default_container_executor.sh\n",
" 809751 260 -r-x------ 1 root root 264361 Aug 4 19:14 ./job.xml\n",
" 809816 4 -rw-r--r-- 1 root root 129 Aug 4 19:14 ./container_tokens\n",
" 809822 4 -rwx------ 1 root root 964 Aug 4 19:14 ./default_container_executor_session.sh\n",
" 809820 8 -rwx------ 1 root root 8160 Aug 4 19:14 ./launch_container.sh\n",
" 809825 4 -rw-r--r-- 1 root root 16 Aug 4 19:14 ./.default_container_executor.sh.crc\n",
" 809821 4 -rw-r--r-- 1 root root 72 Aug 4 19:14 ./.launch_container.sh.crc\n",
" 809745 4 drwx------ 2 root root 4096 Aug 4 19:14 ./job.jar\n",
" 809746 276 -r-x------ 1 root root 281609 Aug 4 19:14 ./job.jar/job.jar\n",
" 809817 4 -rw-r--r-- 1 root root 12 Aug 4 19:14 ./.container_tokens.crc\n",
" 809823 4 -rw-r--r-- 1 root root 16 Aug 4 19:14 ./.default_container_executor_session.sh.crc\n",
" 809815 4 drwx--x--- 2 root root 4096 Aug 4 19:14 ./tmp\n",
"broken symlinks(find -L . -maxdepth 5 -type l -ls):\n",
"\n",
"End of LogType:directory.info\n",
"*******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000003 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:launch_container.sh\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:8160\n",
"LogContents:\n",
"#!/bin/bash\n",
"\n",
"set -o pipefail -e\n",
"export PRELAUNCH_OUT=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000003/prelaunch.out\"\n",
"exec >\"${PRELAUNCH_OUT}\"\n",
"export PRELAUNCH_ERR=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000003/prelaunch.err\"\n",
"exec 2>\"${PRELAUNCH_ERR}\"\n",
"echo \"Setting up env variables\"\n",
"export JAVA_HOME=${JAVA_HOME:-\"/usr/lib/jvm/java-11-openjdk-amd64\"}\n",
"export HADOOP_COMMON_HOME=${HADOOP_COMMON_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_HDFS_HOME=${HADOOP_HDFS_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-\"/content/hadoop-3.4.0/etc/hadoop\"}\n",
"export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_HOME=${HADOOP_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export PATH=${PATH:-\"/content/hadoop-3.4.0/bin:/opt/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/tools/node/bin:/tools/google-cloud-sdk/bin\"}\n",
"export LANG=${LANG:-\"en_US.UTF-8\"}\n",
"export HADOOP_TOKEN_FILE_LOCATION=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000003/container_tokens\"\n",
"export CONTAINER_ID=\"container_1722798834650_0001_01_000003\"\n",
"export NM_PORT=\"34535\"\n",
"export NM_HOST=\"localhost\"\n",
"export NM_HTTP_PORT=\"37361\"\n",
"export LOCAL_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001\"\n",
"export LOCAL_USER_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/\"\n",
"export LOG_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_0/application_1722798834650_0001/container_1722798834650_0001_01_000003,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000003,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000003,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000003\"\n",
"export USER=\"root\"\n",
"export LOGNAME=\"root\"\n",
"export HOME=\"/home/\"\n",
"export PWD=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000003\"\n",
"export LOCALIZATION_COUNTERS=\"0,548046,0,2,26\"\n",
"export JVM_PID=\"$$\"\n",
"export NM_AUX_SERVICE_mapreduce_shuffle=\"AACrRwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=\"\n",
"export STDOUT_LOGFILE_ENV=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000003/stdout\"\n",
"export SHELL=\"/bin/bash\"\n",
"export HADOOP_ROOT_LOGGER=\"INFO,console\"\n",
"export CLASSPATH=\"$PWD:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:/content/hadoop-3.4.0/share/hadoop/mapreduce/*:/content/hadoop-3.4.0/share/hadoop/mapreduce/lib/*:job.jar/*:job.jar/classes/:job.jar/lib/*:$PWD/*\"\n",
"export LD_LIBRARY_PATH=\"$PWD:$HADOOP_COMMON_HOME/lib/native\"\n",
"export STDERR_LOGFILE_ENV=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000003/stderr\"\n",
"export HADOOP_CLIENT_OPTS=\"\"\n",
"export MALLOC_ARENA_MAX=\"4\"\n",
"echo \"Setting up job resources\"\n",
"ln -sf -- \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/13/job.xml\" \"job.xml\"\n",
"ln -sf -- \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/11/job.jar\" \"job.jar\"\n",
"echo \"Copying debugging information\"\n",
"# Creating copy of launch script\n",
"cp \"launch_container.sh\" \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000003/launch_container.sh\"\n",
"chmod 640 \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000003/launch_container.sh\"\n",
"# Determining directory contents\n",
"echo \"ls -l:\" 1>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000003/directory.info\"\n",
"ls -l 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000003/directory.info\"\n",
"echo \"find -L . -maxdepth 5 -ls:\" 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000003/directory.info\"\n",
"find -L . -maxdepth 5 -ls 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000003/directory.info\"\n",
"echo \"broken symlinks(find -L . -maxdepth 5 -type l -ls):\" 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000003/directory.info\"\n",
"find -L . -maxdepth 5 -type l -ls 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000003/directory.info\"\n",
"echo \"Launching container\"\n",
"exec /bin/bash -c \"$JAVA_HOME/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx820m -Djava.io.tmpdir=$PWD/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000003 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.0.1 40041 attempt_1722798834650_0001_m_000001_0 3 1>/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000003/stdout 2>/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000003/stderr \"\n",
"\n",
"End of LogType:launch_container.sh\n",
"************************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000003 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:prelaunch.err\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:0\n",
"LogContents:\n",
"\n",
"End of LogType:prelaunch.err\n",
"******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000003 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:prelaunch.out\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:100\n",
"LogContents:\n",
"Setting up env variables\n",
"Setting up job resources\n",
"Copying debugging information\n",
"Launching container\n",
"\n",
"End of LogType:prelaunch.out\n",
"******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000003 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:stderr\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:0\n",
"LogContents:\n",
"\n",
"End of LogType:stderr\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000003 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:stdout\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:0\n",
"LogContents:\n",
"\n",
"End of LogType:stdout\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000003 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:syslog\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:31510\n",
"LogContents:\n",
"2024-08-04 19:14:26,096 INFO [main] org.apache.hadoop.security.SecurityUtil: Updating Configuration\n",
"2024-08-04 19:14:26,500 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties\n",
"2024-08-04 19:14:26,997 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).\n",
"2024-08-04 19:14:26,998 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started\n",
"2024-08-04 19:14:27,330 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens: [Kind: mapreduce.job, Service: job_1722798834650_0001, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@1fa1cab1)]\n",
"2024-08-04 19:14:27,429 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.\n",
"2024-08-04 19:14:28,423 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001\n",
"2024-08-04 19:14:30,060 INFO [main] org.apache.hadoop.mapred.YarnChild: \n",
"/************************************************************\n",
"[system properties]\n",
"os.name: Linux\n",
"os.version: 6.1.85+\n",
"java.home: /usr/lib/jvm/java-11-openjdk-amd64\n",
"java.runtime.version: 11.0.24+8-post-Ubuntu-1ubuntu322.04\n",
"java.vendor: Ubuntu\n",
"java.version: 11.0.24\n",
"java.vm.name: OpenJDK 64-Bit Server VM\n",
"java.class.path: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000003:/content/hadoop-3.4.0/etc/hadoop:/content/hadoop-3.4.0/share/hadoop/common/hadoop-kms-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-common-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-registry-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-nfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/j2objc-annotations-1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-shaded-guava-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-beanutils-1.9.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsch-0.1.55.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsp-api-2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-io-2.14.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-all-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-cli-1.5.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/bcprov-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/woodstox-core-5.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-servlet-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-http-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-kqueue-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/token-provider-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-crypto-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-logging-1.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-buffer-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-auth-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/guava-27.0-jre.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-xdr-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-proxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/re2j-1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-memcache-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-util-ajax-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-text-1.10.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/metrics-core-3.2.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-unix-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/avro-1.9.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/reload4j-1.2.22.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-configuration2-2.8.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-math3-3.6.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-daemon-1.0.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-core-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-codec-1.15.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-xml-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-lang3-3.12.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/animal-sniffer-annotations-1.17.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-client-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jettison-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-config-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-server-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-redis-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-asn1-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-identity-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-udt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-ssl-ocsp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/gson-2.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsr305-3.0.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-framework-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-databind-2.12.7.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-stomp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/httpcore-4.4.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-socks-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/zookeeper-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/slf4j-api-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-annotations-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-classes-kqueue-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-webapp-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-admin-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-recipes-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-server-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-pkix-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-xml-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-classes-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-io-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-haproxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-http-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jline-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/checker-qual-2.5.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-client-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-simplekdc-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-core-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-util-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-http2-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-collections-3.2.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jakarta.activation-api-1.2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-classes-macos-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-rxtx-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/zookeeper-jute-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-security-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-net-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/failureaccess-1.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-compress-1.24.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/stax2-api-4.2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-core-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final-linux-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-json-1.20.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/audience-annotations-0.12.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/dnsjava-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/nimbus-jose-jwt-9.31.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-common-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-shaded-protobuf_3_21-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/httpclient-4.5.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jul-to-slf4j-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-mqtt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-kqueue-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final-linux-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-smtp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/snappy-java-1.1.10.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-sctp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-native-client-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-client-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-rbf-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-nfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-rbf-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-native-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/j2objc-annotations-1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-shaded-guava-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-beanutils-1.9.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsch-0.1.55.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-io-2.14.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-all-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-cli-1.5.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/woodstox-core-5.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-servlet-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-http-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/token-provider-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-crypto-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-logging-1.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-buffer-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-auth-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/guava-27.0-jre.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-xdr-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-proxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/re2j-1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-memcache-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-util-ajax-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-text-1.10.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/HikariCP-4.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/metrics-core-3.2.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-unix-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/avro-1.9.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/reload4j-1.2.22.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-configuration2-2.8.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-math3-3.6.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-core-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-codec-1.15.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-xml-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-lang3-3.12.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/animal-sniffer-annotations-1.17.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-client-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jettison-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-config-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-server-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-redis-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-asn1-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-identity-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-udt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-ssl-ocsp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/gson-2.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsr305-3.0.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-framework-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-databind-2.12.7.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-stomp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/httpcore-4.4.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-socks-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/zookeeper-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-annotations-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-classes-kqueue-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-webapp-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-admin-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-recipes-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-server-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-pkix-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-xml-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-classes-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-io-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-haproxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-http-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jline-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/checker-qual-2.5.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-client-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-simplekdc-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-core-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-util-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-http2-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jakarta.activation-api-1.2.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-classes-macos-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-rxtx-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/zookeeper-jute-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-security-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-net-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/failureaccess-1.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-compress-1.24.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/stax2-api-4.2.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-core-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final-linux-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-json-1.20.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/audience-annotations-0.12.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/dnsjava-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/nimbus-jose-jwt-9.31.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-common-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-shaded-protobuf_3_21-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/httpclient-4.5.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-mqtt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final-linux-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-smtp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/snappy-java-1.1.10.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-sctp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-services-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-mawo-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-router-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-registry-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-api-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-globalpolicygenerator-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-services-api-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-tests-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jakarta.xml.bind-api-2.3.2.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-plus-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/objenesis-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.inject-1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jna-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.websocket-client-api-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/bcpkix-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/snakeyaml-2.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/guice-servlet-4.2.3.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/codemodel-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/asm-tree-9.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-client-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-api-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-client-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/bcutil-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/fst-2.50.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax-websocket-client-impl-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.websocket-api-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/guice-4.2.3.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jsonschema2pojo-core-1.0.2.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-common-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-jndi-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/asm-commons-9.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax-websocket-server-impl-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jersey-guice-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jersey-client-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-annotations-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-jaxrs-base-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/mockito-core-2.28.2.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/lib/*:job.jar/job.jar:job.jar/classes/:job.jar/lib/*:/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000003/job.jar\n",
"java.io.tmpdir: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000003/tmp\n",
"user.dir: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000003\n",
"user.name: root\n",
"************************************************************/\n",
"2024-08-04 19:14:30,111 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id\n",
"2024-08-04 19:14:31,735 INFO [main] org.apache.hadoop.mapreduce.lib.output.PathOutputCommitterFactory: No output committer factory defined, defaulting to FileOutputCommitterFactory\n",
"2024-08-04 19:14:31,737 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 2\n",
"2024-08-04 19:14:31,737 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false\n",
"2024-08-04 19:14:31,857 INFO [main] org.apache.hadoop.mapred.Task: Using ResourceCalculatorProcessTree : [ ]\n",
"2024-08-04 19:14:32,725 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: hdfs://localhost:8902/user/root/QuasiMonteCarlo_1722798838891_1650860168/in/part1:0+118\n",
"2024-08-04 19:14:33,224 INFO [main] org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)\n",
"2024-08-04 19:14:33,225 INFO [main] org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100\n",
"2024-08-04 19:14:33,225 INFO [main] org.apache.hadoop.mapred.MapTask: soft limit at 83886080\n",
"2024-08-04 19:14:33,225 INFO [main] org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600\n",
"2024-08-04 19:14:33,225 INFO [main] org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600\n",
"2024-08-04 19:14:33,266 INFO [main] org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer\n",
"2024-08-04 19:14:33,436 INFO [main] org.apache.hadoop.mapred.MapTask: Starting flush of map output\n",
"2024-08-04 19:14:33,436 INFO [main] org.apache.hadoop.mapred.MapTask: Spilling map output\n",
"2024-08-04 19:14:33,436 INFO [main] org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 18; bufvoid = 104857600\n",
"2024-08-04 19:14:33,436 INFO [main] org.apache.hadoop.mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600\n",
"2024-08-04 19:14:33,459 INFO [main] org.apache.hadoop.mapred.MapTask: Finished spill 0\n",
"2024-08-04 19:14:33,555 INFO [main] org.apache.hadoop.mapred.Task: Task:attempt_1722798834650_0001_m_000001_0 is done. And is in the process of committing\n",
"2024-08-04 19:14:33,652 INFO [main] org.apache.hadoop.mapred.Task: Task 'attempt_1722798834650_0001_m_000001_0' done.\n",
"2024-08-04 19:14:33,767 INFO [main] org.apache.hadoop.mapred.Task: Final Counters for attempt_1722798834650_0001_m_000001_0: Counters: 28\n",
"\tFile System Counters\n",
"\t\tFILE: Number of bytes read=0\n",
"\t\tFILE: Number of bytes written=309460\n",
"\t\tFILE: Number of read operations=0\n",
"\t\tFILE: Number of large read operations=0\n",
"\t\tFILE: Number of write operations=0\n",
"\t\tHDFS: Number of bytes read=264\n",
"\t\tHDFS: Number of bytes written=0\n",
"\t\tHDFS: Number of read operations=4\n",
"\t\tHDFS: Number of large read operations=0\n",
"\t\tHDFS: Number of write operations=0\n",
"\t\tHDFS: Number of bytes read erasure-coded=0\n",
"\tMap-Reduce Framework\n",
"\t\tMap input records=1\n",
"\t\tMap output records=2\n",
"\t\tMap output bytes=18\n",
"\t\tMap output materialized bytes=28\n",
"\t\tInput split bytes=146\n",
"\t\tCombine input records=0\n",
"\t\tSpilled Records=2\n",
"\t\tFailed Shuffles=0\n",
"\t\tMerged Map outputs=0\n",
"\t\tGC time elapsed (ms)=80\n",
"\t\tCPU time spent (ms)=980\n",
"\t\tPhysical memory (bytes) snapshot=383688704\n",
"\t\tVirtual memory (bytes) snapshot=2723983360\n",
"\t\tTotal committed heap usage (bytes)=298844160\n",
"\t\tPeak Map Physical memory (bytes)=383688704\n",
"\t\tPeak Map Virtual memory (bytes)=2723983360\n",
"\tFile Input Format Counters \n",
"\t\tBytes Read=118\n",
"2024-08-04 19:14:33,771 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system...\n",
"2024-08-04 19:14:33,771 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped.\n",
"2024-08-04 19:14:33,772 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system shutdown complete.\n",
"\n",
"End of LogType:syslog\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000001 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:directory.info\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:2444\n",
"LogContents:\n",
"ls -l:\n",
"total 36\n",
"-rw-r--r-- 1 root root 105 Aug 4 19:14 container_tokens\n",
"-rwx------ 1 root root 964 Aug 4 19:14 default_container_executor_session.sh\n",
"-rwx------ 1 root root 1019 Aug 4 19:14 default_container_executor.sh\n",
"lrwxrwxrwx 1 root root 179 Aug 4 19:14 job.jar -> /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/11/job.jar\n",
"drwxr-xr-x 2 root root 4096 Aug 4 19:14 jobSubmitDir\n",
"lrwxrwxrwx 1 root root 179 Aug 4 19:14 job.xml -> /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/13/job.xml\n",
"-rwx------ 1 root root 8184 Aug 4 19:14 launch_container.sh\n",
"drwx--x--- 2 root root 4096 Aug 4 19:14 tmp\n",
"find -L . -maxdepth 5 -ls:\n",
" 809763 4 drwx--x--- 4 root root 4096 Aug 4 19:14 .\n",
" 809778 4 -rwx------ 1 root root 1019 Aug 4 19:14 ./default_container_executor.sh\n",
" 809751 260 -r-x------ 1 root root 264361 Aug 4 19:14 ./job.xml\n",
" 809772 4 -rw-r--r-- 1 root root 105 Aug 4 19:14 ./container_tokens\n",
" 809776 4 -rwx------ 1 root root 964 Aug 4 19:14 ./default_container_executor_session.sh\n",
" 809774 8 -rwx------ 1 root root 8184 Aug 4 19:14 ./launch_container.sh\n",
" 809779 4 -rw-r--r-- 1 root root 16 Aug 4 19:14 ./.default_container_executor.sh.crc\n",
" 809775 4 -rw-r--r-- 1 root root 72 Aug 4 19:14 ./.launch_container.sh.crc\n",
" 809745 4 drwx------ 2 root root 4096 Aug 4 19:14 ./job.jar\n",
" 809746 276 -r-x------ 1 root root 281609 Aug 4 19:14 ./job.jar/job.jar\n",
" 809773 4 -rw-r--r-- 1 root root 12 Aug 4 19:14 ./.container_tokens.crc\n",
" 809784 4 drwxr-xr-x 2 root root 4096 Aug 4 19:14 ./jobSubmitDir\n",
" 809748 4 -r-x------ 1 root root 737 Aug 4 19:14 ./jobSubmitDir/job.split\n",
" 809738 4 -r-x------ 1 root root 82 Aug 4 19:14 ./jobSubmitDir/job.splitmetainfo\n",
" 809777 4 -rw-r--r-- 1 root root 16 Aug 4 19:14 ./.default_container_executor_session.sh.crc\n",
" 809771 4 drwx--x--- 2 root root 4096 Aug 4 19:14 ./tmp\n",
"broken symlinks(find -L . -maxdepth 5 -type l -ls):\n",
"\n",
"End of LogType:directory.info\n",
"*******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000001 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:launch_container.sh\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:8184\n",
"LogContents:\n",
"#!/bin/bash\n",
"\n",
"set -o pipefail -e\n",
"export PRELAUNCH_OUT=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_0/application_1722798834650_0001/container_1722798834650_0001_01_000001/prelaunch.out\"\n",
"exec >\"${PRELAUNCH_OUT}\"\n",
"export PRELAUNCH_ERR=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_0/application_1722798834650_0001/container_1722798834650_0001_01_000001/prelaunch.err\"\n",
"exec 2>\"${PRELAUNCH_ERR}\"\n",
"echo \"Setting up env variables\"\n",
"export JAVA_HOME=${JAVA_HOME:-\"/usr/lib/jvm/java-11-openjdk-amd64\"}\n",
"export HADOOP_COMMON_HOME=${HADOOP_COMMON_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_HDFS_HOME=${HADOOP_HDFS_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-\"/content/hadoop-3.4.0/etc/hadoop\"}\n",
"export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_HOME=${HADOOP_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export PATH=${PATH:-\"/content/hadoop-3.4.0/bin:/opt/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/tools/node/bin:/tools/google-cloud-sdk/bin\"}\n",
"export LANG=${LANG:-\"en_US.UTF-8\"}\n",
"export HADOOP_TOKEN_FILE_LOCATION=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000001/container_tokens\"\n",
"export CONTAINER_ID=\"container_1722798834650_0001_01_000001\"\n",
"export NM_PORT=\"34535\"\n",
"export NM_HOST=\"localhost\"\n",
"export NM_HTTP_PORT=\"37361\"\n",
"export LOCAL_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001\"\n",
"export LOCAL_USER_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/\"\n",
"export LOG_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_0/application_1722798834650_0001/container_1722798834650_0001_01_000001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000001\"\n",
"export USER=\"root\"\n",
"export LOGNAME=\"root\"\n",
"export HOME=\"/home/\"\n",
"export PWD=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000001\"\n",
"export LOCALIZATION_COUNTERS=\"548893,0,4,0,818\"\n",
"export JVM_PID=\"$$\"\n",
"export NM_AUX_SERVICE_mapreduce_shuffle=\"AACrRwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=\"\n",
"export APPLICATION_WEB_PROXY_BASE=\"/proxy/application_1722798834650_0001\"\n",
"export SHELL=\"/bin/bash\"\n",
"export CLASSPATH=\"$PWD:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:/content/hadoop-3.4.0/share/hadoop/mapreduce/*:/content/hadoop-3.4.0/share/hadoop/mapreduce/lib/*:job.jar/*:job.jar/classes/:job.jar/lib/*:$PWD/*\"\n",
"export APP_SUBMIT_TIME_ENV=\"1722798847160\"\n",
"export LD_LIBRARY_PATH=\"$PWD:$HADOOP_COMMON_HOME/lib/native\"\n",
"export MALLOC_ARENA_MAX=\"4\"\n",
"echo \"Setting up job resources\"\n",
"ln -sf -- \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/13/job.xml\" \"job.xml\"\n",
"mkdir -p jobSubmitDir\n",
"ln -sf -- \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001/filecache/12/job.split\" \"jobSubmitDir/job.split\"\n",
"ln -sf -- \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/11/job.jar\" \"job.jar\"\n",
"mkdir -p jobSubmitDir\n",
"ln -sf -- \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001/filecache/10/job.splitmetainfo\" \"jobSubmitDir/job.splitmetainfo\"\n",
"echo \"Copying debugging information\"\n",
"# Creating copy of launch script\n",
"cp \"launch_container.sh\" \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_0/application_1722798834650_0001/container_1722798834650_0001_01_000001/launch_container.sh\"\n",
"chmod 640 \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_0/application_1722798834650_0001/container_1722798834650_0001_01_000001/launch_container.sh\"\n",
"# Determining directory contents\n",
"echo \"ls -l:\" 1>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_0/application_1722798834650_0001/container_1722798834650_0001_01_000001/directory.info\"\n",
"ls -l 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_0/application_1722798834650_0001/container_1722798834650_0001_01_000001/directory.info\"\n",
"echo \"find -L . -maxdepth 5 -ls:\" 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_0/application_1722798834650_0001/container_1722798834650_0001_01_000001/directory.info\"\n",
"find -L . -maxdepth 5 -ls 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_0/application_1722798834650_0001/container_1722798834650_0001_01_000001/directory.info\"\n",
"echo \"broken symlinks(find -L . -maxdepth 5 -type l -ls):\" 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_0/application_1722798834650_0001/container_1722798834650_0001_01_000001/directory.info\"\n",
"find -L . -maxdepth 5 -type l -ls 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_0/application_1722798834650_0001/container_1722798834650_0001_01_000001/directory.info\"\n",
"echo \"Launching container\"\n",
"exec /bin/bash -c \"$JAVA_HOME/bin/java -Djava.io.tmpdir=$PWD/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_0/application_1722798834650_0001/container_1722798834650_0001_01_000001 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog -Xmx1024m org.apache.hadoop.mapreduce.v2.app.MRAppMaster 1>/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_0/application_1722798834650_0001/container_1722798834650_0001_01_000001/stdout 2>/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_0/application_1722798834650_0001/container_1722798834650_0001_01_000001/stderr \"\n",
"\n",
"End of LogType:launch_container.sh\n",
"************************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000001 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:prelaunch.err\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:0\n",
"LogContents:\n",
"\n",
"End of LogType:prelaunch.err\n",
"******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000001 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:prelaunch.out\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:100\n",
"LogContents:\n",
"Setting up env variables\n",
"Setting up job resources\n",
"Copying debugging information\n",
"Launching container\n",
"\n",
"End of LogType:prelaunch.out\n",
"******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000001 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:stderr\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:2344\n",
"LogContents:\n",
"WARNING: An illegal reflective access operation has occurred\n",
"WARNING: Illegal reflective access by com.google.inject.internal.cglib.core.$ReflectUtils$1 (file:/content/hadoop-3.4.0/share/hadoop/yarn/lib/guice-4.2.3.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)\n",
"WARNING: Please consider reporting this to the maintainers of com.google.inject.internal.cglib.core.$ReflectUtils$1\n",
"WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations\n",
"WARNING: All illegal access operations will be denied in a future release\n",
"Aug 04, 2024 7:14:17 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register\n",
"INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver as a provider class\n",
"Aug 04, 2024 7:14:17 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register\n",
"INFO: Registering org.apache.hadoop.yarn.webapp.GenericExceptionHandler as a provider class\n",
"Aug 04, 2024 7:14:17 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory register\n",
"INFO: Registering org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices as a root resource class\n",
"Aug 04, 2024 7:14:17 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate\n",
"INFO: Initiating Jersey application, version 'Jersey: 1.19.4 05/24/2017 03:20 PM'\n",
"Aug 04, 2024 7:14:17 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider\n",
"INFO: Binding org.apache.hadoop.mapreduce.v2.app.webapp.JAXBContextResolver to GuiceManagedComponentProvider with the scope \"Singleton\"\n",
"Aug 04, 2024 7:14:18 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider\n",
"INFO: Binding org.apache.hadoop.yarn.webapp.GenericExceptionHandler to GuiceManagedComponentProvider with the scope \"Singleton\"\n",
"Aug 04, 2024 7:14:19 PM com.sun.jersey.guice.spi.container.GuiceComponentProviderFactory getComponentProvider\n",
"INFO: Binding org.apache.hadoop.mapreduce.v2.app.webapp.AMWebServices to GuiceManagedComponentProvider with the scope \"PerRequest\"\n",
"log4j:WARN No appenders could be found for logger (org.apache.hadoop.mapreduce.v2.app.MRAppMaster).\n",
"log4j:WARN Please initialize the log4j system properly.\n",
"log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.\n",
"\n",
"End of LogType:stderr\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000001 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:stdout\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:0\n",
"LogContents:\n",
"\n",
"End of LogType:stdout\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000001 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:syslog\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:97284\n",
"LogContents:\n",
"2024-08-04 19:14:12,495 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for application appattempt_1722798834650_0001_000001\n",
"2024-08-04 19:14:12,717 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: \n",
"/************************************************************\n",
"[system properties]\n",
"os.name: Linux\n",
"os.version: 6.1.85+\n",
"java.home: /usr/lib/jvm/java-11-openjdk-amd64\n",
"java.runtime.version: 11.0.24+8-post-Ubuntu-1ubuntu322.04\n",
"java.vendor: Ubuntu\n",
"java.version: 11.0.24\n",
"java.vm.name: OpenJDK 64-Bit Server VM\n",
"java.class.path: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000001:/content/hadoop-3.4.0/etc/hadoop:/content/hadoop-3.4.0/share/hadoop/common/hadoop-kms-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-common-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-registry-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-nfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/j2objc-annotations-1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-shaded-guava-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-beanutils-1.9.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsch-0.1.55.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsp-api-2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-io-2.14.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-all-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-cli-1.5.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/bcprov-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/woodstox-core-5.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-servlet-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-http-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-kqueue-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/token-provider-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-crypto-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-logging-1.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-buffer-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-auth-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/guava-27.0-jre.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-xdr-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-proxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/re2j-1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-memcache-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-util-ajax-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-text-1.10.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/metrics-core-3.2.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-unix-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/avro-1.9.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/reload4j-1.2.22.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-configuration2-2.8.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-math3-3.6.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-daemon-1.0.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-core-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-codec-1.15.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-xml-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-lang3-3.12.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/animal-sniffer-annotations-1.17.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-client-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jettison-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-config-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-server-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-redis-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-asn1-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-identity-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-udt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-ssl-ocsp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/gson-2.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsr305-3.0.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-framework-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-databind-2.12.7.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-stomp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/httpcore-4.4.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-socks-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/zookeeper-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/slf4j-api-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-annotations-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-classes-kqueue-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-webapp-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-admin-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-recipes-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-server-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-pkix-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-xml-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-classes-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-io-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-haproxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-http-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jline-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/checker-qual-2.5.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-client-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-simplekdc-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-core-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-util-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-http2-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-collections-3.2.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jakarta.activation-api-1.2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-classes-macos-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-rxtx-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/zookeeper-jute-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-security-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-net-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/failureaccess-1.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-compress-1.24.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/stax2-api-4.2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-core-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final-linux-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-json-1.20.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/audience-annotations-0.12.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/dnsjava-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/nimbus-jose-jwt-9.31.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-common-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-shaded-protobuf_3_21-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/httpclient-4.5.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jul-to-slf4j-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-mqtt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-kqueue-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final-linux-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-smtp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/snappy-java-1.1.10.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-sctp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-native-client-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-client-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-rbf-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-nfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-rbf-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-native-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/j2objc-annotations-1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-shaded-guava-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-beanutils-1.9.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsch-0.1.55.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-io-2.14.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-all-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-cli-1.5.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/woodstox-core-5.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-servlet-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-http-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/token-provider-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-crypto-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-logging-1.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-buffer-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-auth-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/guava-27.0-jre.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-xdr-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-proxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/re2j-1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-memcache-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-util-ajax-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-text-1.10.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/HikariCP-4.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/metrics-core-3.2.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-unix-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/avro-1.9.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/reload4j-1.2.22.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-configuration2-2.8.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-math3-3.6.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-core-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-codec-1.15.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-xml-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-lang3-3.12.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/animal-sniffer-annotations-1.17.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-client-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jettison-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-config-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-server-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-redis-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-asn1-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-identity-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-udt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-ssl-ocsp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/gson-2.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsr305-3.0.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-framework-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-databind-2.12.7.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-stomp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/httpcore-4.4.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-socks-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/zookeeper-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-annotations-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-classes-kqueue-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-webapp-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-admin-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-recipes-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-server-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-pkix-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-xml-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-classes-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-io-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-haproxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-http-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jline-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/checker-qual-2.5.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-client-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-simplekdc-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-core-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-util-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-http2-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jakarta.activation-api-1.2.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-classes-macos-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-rxtx-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/zookeeper-jute-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-security-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-net-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/failureaccess-1.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-compress-1.24.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/stax2-api-4.2.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-core-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final-linux-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-json-1.20.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/audience-annotations-0.12.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/dnsjava-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/nimbus-jose-jwt-9.31.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-common-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-shaded-protobuf_3_21-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/httpclient-4.5.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-mqtt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final-linux-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-smtp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/snappy-java-1.1.10.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-sctp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-services-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-mawo-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-router-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-registry-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-api-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-globalpolicygenerator-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-services-api-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-tests-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jakarta.xml.bind-api-2.3.2.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-plus-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/objenesis-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.inject-1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jna-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.websocket-client-api-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/bcpkix-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/snakeyaml-2.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/guice-servlet-4.2.3.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/codemodel-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/asm-tree-9.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-client-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-api-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-client-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/bcutil-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/fst-2.50.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax-websocket-client-impl-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.websocket-api-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/guice-4.2.3.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jsonschema2pojo-core-1.0.2.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-common-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-jndi-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/asm-commons-9.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax-websocket-server-impl-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jersey-guice-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jersey-client-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-annotations-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-jaxrs-base-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/mockito-core-2.28.2.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/lib/*:job.jar/job.jar:job.jar/classes/:job.jar/lib/*:/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000001/job.jar\n",
"java.io.tmpdir: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000001/tmp\n",
"user.dir: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000001\n",
"user.name: root\n",
"************************************************************/\n",
"2024-08-04 19:14:12,821 INFO [main] org.apache.hadoop.security.SecurityUtil: Updating Configuration\n",
"2024-08-04 19:14:13,142 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens: [Kind: YARN_AM_RM_TOKEN, Service: , Ident: (appAttemptId { application_id { id: 1 cluster_timestamp: 1722798834650 } attemptId: 1 } keyId: -675888397)]\n",
"2024-08-04 19:14:13,219 INFO [main] org.apache.hadoop.conf.Configuration: resource-types.xml not found\n",
"2024-08-04 19:14:13,222 INFO [main] org.apache.hadoop.yarn.util.resource.ResourceUtils: Unable to find 'resource-types.xml'.\n",
"2024-08-04 19:14:13,246 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred newApiCommitter.\n",
"2024-08-04 19:14:13,249 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in config null\n",
"2024-08-04 19:14:13,350 INFO [main] org.apache.hadoop.mapreduce.lib.output.PathOutputCommitterFactory: No output committer factory defined, defaulting to FileOutputCommitterFactory\n",
"2024-08-04 19:14:13,352 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 2\n",
"2024-08-04 19:14:13,352 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false\n",
"2024-08-04 19:14:14,156 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter is org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter\n",
"2024-08-04 19:14:14,475 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.jobhistory.EventType for class org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler\n",
"2024-08-04 19:14:14,477 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobEventDispatcher\n",
"2024-08-04 19:14:14,478 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskEventDispatcher\n",
"2024-08-04 19:14:14,480 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.TaskAttemptEventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$TaskAttemptEventDispatcher\n",
"2024-08-04 19:14:14,480 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventType for class org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler\n",
"2024-08-04 19:14:14,482 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.speculate.Speculator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$SpeculatorEventDispatcher\n",
"2024-08-04 19:14:14,482 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.rm.ContainerAllocator$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter\n",
"2024-08-04 19:14:14,484 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncher$EventType for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerLauncherRouter\n",
"2024-08-04 19:14:14,532 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://localhost:8902]\n",
"2024-08-04 19:14:14,562 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://localhost:8902]\n",
"2024-08-04 19:14:14,608 INFO [main] org.apache.hadoop.mapreduce.v2.jobhistory.JobHistoryUtils: Default file system [hdfs://localhost:8902]\n",
"2024-08-04 19:14:14,620 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Creating intermediate history logDir: [hdfs://localhost:8902/tmp/hadoop-yarn/staging/history/done_intermediate] + based on conf. Should ideally be created by the JobHistoryServer: yarn.app.mapreduce.am.create-intermediate-jh-base-dir\n",
"2024-08-04 19:14:14,635 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Perms after creating 493, Expected: 1023\n",
"2024-08-04 19:14:14,635 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Explicitly setting permissions to : 1023, rwxrwxrwt\n",
"2024-08-04 19:14:14,659 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Perms after creating 488, Expected: 504\n",
"2024-08-04 19:14:14,659 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Explicitly setting permissions to : 504, rwxrwx---\n",
"2024-08-04 19:14:14,663 INFO [main] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Emitting job history data to the timeline server is not enabled\n",
"2024-08-04 19:14:14,730 INFO [main] org.apache.hadoop.yarn.event.AsyncDispatcher: Registering class org.apache.hadoop.mapreduce.v2.app.job.event.JobFinishEvent$Type for class org.apache.hadoop.mapreduce.v2.app.MRAppMaster$JobFinishEventHandler\n",
"2024-08-04 19:14:15,157 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties\n",
"2024-08-04 19:14:15,287 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).\n",
"2024-08-04 19:14:15,287 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MRAppMaster metrics system started\n",
"2024-08-04 19:14:15,303 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Adding job token for job_1722798834650_0001 to jobTokenSecretManager\n",
"2024-08-04 19:14:15,550 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Not uberizing job_1722798834650_0001 because: not enabled;\n",
"2024-08-04 19:14:15,581 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Input size for job job_1722798834650_0001 = 590. Number of splits = 5\n",
"2024-08-04 19:14:15,583 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Number of reduces for job job_1722798834650_0001 = 1\n",
"2024-08-04 19:14:15,583 INFO [main] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1722798834650_0001Job Transitioned from NEW to INITED\n",
"2024-08-04 19:14:15,585 INFO [main] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: MRAppMaster launching normal, non-uberized, multi-container job job_1722798834650_0001.\n",
"2024-08-04 19:14:15,656 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue, queueCapacity: 100, scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false, ipcFailOver: false.\n",
"2024-08-04 19:14:15,663 INFO [main] org.apache.hadoop.ipc.Server: Listener at 0.0.0.0:34269\n",
"2024-08-04 19:14:15,665 INFO [Socket Reader #1 for port 0] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 0\n",
"2024-08-04 19:14:15,719 INFO [main] org.apache.hadoop.yarn.factories.impl.pb.RpcServerFactoryPBImpl: Adding protocol org.apache.hadoop.mapreduce.v2.api.MRClientProtocolPB to the server\n",
"2024-08-04 19:14:15,720 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting\n",
"2024-08-04 19:14:15,722 INFO [main] org.apache.hadoop.mapreduce.v2.app.client.MRClientService: Instantiated MRClientService at localhost/127.0.0.1:34269\n",
"2024-08-04 19:14:15,723 INFO [IPC Server listener on 0] org.apache.hadoop.ipc.Server: IPC Server listener on 0: starting\n",
"2024-08-04 19:14:15,761 INFO [main] org.eclipse.jetty.util.log: Logging initialized @4949ms to org.eclipse.jetty.util.log.Slf4jLog\n",
"2024-08-04 19:14:15,894 WARN [main] org.apache.hadoop.security.authentication.server.AuthenticationFilter: Unable to initialize FileSignerSecretProvider, falling back to use random secrets. Reason: Could not read signature secret file: /root/hadoop-http-auth-signature-secret\n",
"2024-08-04 19:14:15,954 INFO [main] org.apache.hadoop.http.HttpServer2: Added global filter 'safety' (class=org.apache.hadoop.http.HttpServer2$QuotingInputFilter)\n",
"2024-08-04 19:14:16,010 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context mapreduce\n",
"2024-08-04 19:14:16,011 INFO [main] org.apache.hadoop.http.HttpServer2: Added filter AM_PROXY_FILTER (class=org.apache.hadoop.yarn.server.webproxy.amfilter.AmIpFilter) to context static\n",
"2024-08-04 19:14:16,017 INFO [main] org.apache.hadoop.http.HttpServer2: ASYNC_PROFILER_HOME environment variable and async.profiler.home system property not specified. Disabling /prof endpoint.\n",
"2024-08-04 19:14:16,782 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Registered webapp guice modules\n",
"2024-08-04 19:14:16,784 INFO [main] org.apache.hadoop.http.HttpServer2: Jetty bound to port 38235\n",
"2024-08-04 19:14:16,786 INFO [main] org.eclipse.jetty.server.Server: jetty-9.4.53.v20231009; built: 2023-10-09T12:29:09.265Z; git: 27bde00a0b95a1d5bbee0eae7984f891d2d0f8c9; jvm 11.0.24+8-post-Ubuntu-1ubuntu322.04\n",
"2024-08-04 19:14:16,825 INFO [main] org.eclipse.jetty.server.session: DefaultSessionIdManager workerName=node0\n",
"2024-08-04 19:14:16,826 INFO [main] org.eclipse.jetty.server.session: No SessionScavenger set, using defaults\n",
"2024-08-04 19:14:16,827 INFO [main] org.eclipse.jetty.server.session: node0 Scavenging every 660000ms\n",
"2024-08-04 19:14:16,845 INFO [main] org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.s.ServletContextHandler@48a663e9{static,/static,jar:file:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-common-3.4.0.jar!/webapps/static,AVAILABLE}\n",
"2024-08-04 19:14:19,462 INFO [main] org.eclipse.jetty.server.handler.ContextHandler: Started o.e.j.w.WebAppContext@7a1ddbf1{mapreduce,/,file:///content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000001/tmp/jetty-0_0_0_0-38235-hadoop-yarn-common-3_4_0_jar-_-any-17571332027292415163/webapp/,AVAILABLE}{jar:file:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-common-3.4.0.jar!/webapps/mapreduce}\n",
"2024-08-04 19:14:19,487 INFO [main] org.eclipse.jetty.server.AbstractConnector: Started ServerConnector@58fbd02e{HTTP/1.1, (http/1.1)}{0.0.0.0:38235}\n",
"2024-08-04 19:14:19,489 INFO [main] org.eclipse.jetty.server.Server: Started @8678ms\n",
"2024-08-04 19:14:19,502 INFO [main] org.apache.hadoop.yarn.webapp.WebApps: Web app mapreduce started at 38235\n",
"2024-08-04 19:14:19,543 INFO [main] org.apache.hadoop.ipc.CallQueueManager: Using callQueue: class java.util.concurrent.LinkedBlockingQueue, queueCapacity: 3000, scheduler: class org.apache.hadoop.ipc.DefaultRpcScheduler, ipcBackoff: false, ipcFailOver: false.\n",
"2024-08-04 19:14:19,543 INFO [main] org.apache.hadoop.ipc.Server: Listener at 0.0.0.0:40041\n",
"2024-08-04 19:14:19,544 INFO [Socket Reader #1 for port 0] org.apache.hadoop.ipc.Server: Starting Socket Reader #1 for port 0\n",
"2024-08-04 19:14:19,579 INFO [IPC Server listener on 0] org.apache.hadoop.ipc.Server: IPC Server listener on 0: starting\n",
"2024-08-04 19:14:19,579 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: IPC Server Responder: starting\n",
"2024-08-04 19:14:19,667 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: nodeBlacklistingEnabled:true\n",
"2024-08-04 19:14:19,667 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: maxTaskFailuresPerNode is 3\n",
"2024-08-04 19:14:19,667 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: blacklistDisablePercent is 33\n",
"2024-08-04 19:14:19,678 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: 0% of the mappers will be scheduled using OPPORTUNISTIC containers\n",
"2024-08-04 19:14:19,770 INFO [main] org.apache.hadoop.yarn.client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at /0.0.0.0:8030\n",
"2024-08-04 19:14:20,077 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: maxContainerCapability: \n",
"2024-08-04 19:14:20,077 INFO [main] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: queue: root.default\n",
"2024-08-04 19:14:20,093 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Upper limit on the thread pool size is 500\n",
"2024-08-04 19:14:20,093 INFO [main] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: The thread pool initial size is 10\n",
"2024-08-04 19:14:20,169 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1722798834650_0001Job Transitioned from INITED to SETUP\n",
"2024-08-04 19:14:20,174 INFO [CommitterEvent Processor #0] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_SETUP\n",
"2024-08-04 19:14:20,205 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1722798834650_0001Job Transitioned from SETUP to RUNNING\n",
"2024-08-04 19:14:20,448 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Resource capability of task type MAP is set to \n",
"2024-08-04 19:14:20,523 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1722798834650_0001_m_000000 Task Transitioned from NEW to SCHEDULED\n",
"2024-08-04 19:14:20,530 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1722798834650_0001_m_000001 Task Transitioned from NEW to SCHEDULED\n",
"2024-08-04 19:14:20,536 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1722798834650_0001_m_000002 Task Transitioned from NEW to SCHEDULED\n",
"2024-08-04 19:14:20,536 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1722798834650_0001_m_000003 Task Transitioned from NEW to SCHEDULED\n",
"2024-08-04 19:14:20,536 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1722798834650_0001_m_000004 Task Transitioned from NEW to SCHEDULED\n",
"2024-08-04 19:14:20,551 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Resource capability of task type REDUCE is set to \n",
"2024-08-04 19:14:20,551 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1722798834650_0001_r_000000 Task Transitioned from NEW to SCHEDULED\n",
"2024-08-04 19:14:20,571 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000000_0 TaskAttempt Transitioned from NEW to UNASSIGNED\n",
"2024-08-04 19:14:20,571 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000001_0 TaskAttempt Transitioned from NEW to UNASSIGNED\n",
"2024-08-04 19:14:20,571 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000002_0 TaskAttempt Transitioned from NEW to UNASSIGNED\n",
"2024-08-04 19:14:20,571 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000003_0 TaskAttempt Transitioned from NEW to UNASSIGNED\n",
"2024-08-04 19:14:20,571 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000004_0 TaskAttempt Transitioned from NEW to UNASSIGNED\n",
"2024-08-04 19:14:20,571 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_r_000000_0 TaskAttempt Transitioned from NEW to UNASSIGNED\n",
"2024-08-04 19:14:20,573 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Event Writer setup for JobId: job_1722798834650_0001, File: hdfs://localhost:8902/tmp/hadoop-yarn/staging/root/.staging/job_1722798834650_0001/job_1722798834650_0001_1.jhist\n",
"2024-08-04 19:14:20,584 INFO [Thread-57] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: mapResourceRequest:\n",
"2024-08-04 19:14:20,726 INFO [Thread-57] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: reduceResourceRequest:\n",
"2024-08-04 19:14:21,107 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:1 ScheduledMaps:5 ScheduledReds:0 AssignedMaps:0 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:0 ContRel:0 HostLocal:0 RackLocal:0\n",
"2024-08-04 19:14:21,633 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: applicationId=application_1722798834650_0001: ask=3 release=0 newContainers=0 finishedContainers=0 resourceLimit= knownNMs=1\n",
"2024-08-04 19:14:21,652 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:21,653 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1\n",
"2024-08-04 19:14:22,677 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 2\n",
"2024-08-04 19:14:22,680 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_1722798834650_0001_01_000002 to attempt_1722798834650_0001_m_000000_0\n",
"2024-08-04 19:14:22,686 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_1722798834650_0001_01_000003 to attempt_1722798834650_0001_m_000001_0\n",
"2024-08-04 19:14:22,689 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:22,689 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1\n",
"2024-08-04 19:14:22,689 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:1 ScheduledMaps:3 ScheduledReds:0 AssignedMaps:2 AssignedReds:0 CompletedMaps:0 CompletedReds:0 ContAlloc:2 ContRel:0 HostLocal:2 RackLocal:0\n",
"2024-08-04 19:14:22,802 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: The job-jar file on the remote FS is hdfs://localhost:8902/tmp/hadoop-yarn/staging/root/.staging/job_1722798834650_0001/job.jar\n",
"2024-08-04 19:14:22,813 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: The job-conf file on the remote FS is /tmp/hadoop-yarn/staging/root/.staging/job_1722798834650_0001/job.xml\n",
"2024-08-04 19:14:22,816 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Adding #0 tokens and #1 secret keys for NM use for launching container\n",
"2024-08-04 19:14:22,816 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Size of containertokens_dob is 1\n",
"2024-08-04 19:14:22,816 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Putting shuffle token in serviceData\n",
"2024-08-04 19:14:22,897 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapred.JobConf: Task java-opts do not specify heap size. Setting task attempt jvm max heap size to -Xmx820m\n",
"2024-08-04 19:14:22,901 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000000_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED\n",
"2024-08-04 19:14:22,912 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapred.JobConf: Task java-opts do not specify heap size. Setting task attempt jvm max heap size to -Xmx820m\n",
"2024-08-04 19:14:22,913 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000001_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED\n",
"2024-08-04 19:14:22,915 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_1722798834650_0001_01_000002 taskAttempt attempt_1722798834650_0001_m_000000_0\n",
"2024-08-04 19:14:22,918 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1722798834650_0001_m_000000_0\n",
"2024-08-04 19:14:22,922 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_1722798834650_0001_01_000003 taskAttempt attempt_1722798834650_0001_m_000001_0\n",
"2024-08-04 19:14:22,922 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1722798834650_0001_m_000001_0\n",
"2024-08-04 19:14:23,109 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1722798834650_0001_m_000000_0 : 43847\n",
"2024-08-04 19:14:23,111 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1722798834650_0001_m_000000_0] using containerId: [container_1722798834650_0001_01_000002 on NM: [localhost:34535]\n",
"2024-08-04 19:14:23,116 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000000_0 TaskAttempt Transitioned from ASSIGNED to RUNNING\n",
"2024-08-04 19:14:23,118 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1722798834650_0001_m_000000 Task Transitioned from SCHEDULED to RUNNING\n",
"2024-08-04 19:14:23,144 INFO [ContainerLauncher #1] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1722798834650_0001_m_000001_0 : 43847\n",
"2024-08-04 19:14:23,149 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1722798834650_0001_m_000001_0] using containerId: [container_1722798834650_0001_01_000003 on NM: [localhost:34535]\n",
"2024-08-04 19:14:23,149 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000001_0 TaskAttempt Transitioned from ASSIGNED to RUNNING\n",
"2024-08-04 19:14:23,149 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1722798834650_0001_m_000001 Task Transitioned from SCHEDULED to RUNNING\n",
"2024-08-04 19:14:23,697 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: applicationId=application_1722798834650_0001: ask=3 release=0 newContainers=0 finishedContainers=0 resourceLimit= knownNMs=1\n",
"2024-08-04 19:14:23,698 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:23,698 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1\n",
"2024-08-04 19:14:24,706 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:24,706 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1\n",
"2024-08-04 19:14:25,714 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:25,715 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1\n",
"2024-08-04 19:14:26,723 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:26,723 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1\n",
"2024-08-04 19:14:27,732 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:27,732 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1\n",
"2024-08-04 19:14:27,816 INFO [Socket Reader #1 for port 0] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1722798834650_0001 (auth:SIMPLE) from localhost:34572 / 127.0.0.1:34572\n",
"2024-08-04 19:14:27,918 INFO [IPC Server handler 7 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1722798834650_0001_m_000003 asked for a task\n",
"2024-08-04 19:14:27,918 INFO [IPC Server handler 7 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1722798834650_0001_m_000003 given task: attempt_1722798834650_0001_m_000001_0\n",
"2024-08-04 19:14:28,136 INFO [Socket Reader #1 for port 0] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1722798834650_0001 (auth:SIMPLE) from localhost:34586 / 127.0.0.1:34586\n",
"2024-08-04 19:14:28,222 INFO [IPC Server handler 6 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1722798834650_0001_m_000002 asked for a task\n",
"2024-08-04 19:14:28,222 INFO [IPC Server handler 6 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1722798834650_0001_m_000002 given task: attempt_1722798834650_0001_m_000000_0\n",
"2024-08-04 19:14:28,739 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:28,739 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1\n",
"2024-08-04 19:14:29,744 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:29,744 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1\n",
"2024-08-04 19:14:30,750 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:30,750 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1\n",
"2024-08-04 19:14:31,757 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:31,757 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1\n",
"2024-08-04 19:14:32,763 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:32,764 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold not met. completedMapsForReduceSlowstart 1\n",
"2024-08-04 19:14:33,414 INFO [IPC Server handler 4 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1722798834650_0001_m_000001_0 is : 0.0\n",
"2024-08-04 19:14:33,632 INFO [IPC Server handler 22 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1722798834650_0001_m_000001_0 is : 1.0\n",
"2024-08-04 19:14:33,651 INFO [IPC Server handler 7 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgment from attempt_1722798834650_0001_m_000001_0\n",
"2024-08-04 19:14:33,659 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000001_0 TaskAttempt Transitioned from RUNNING to SUCCESS_FINISHING_CONTAINER\n",
"2024-08-04 19:14:33,736 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1722798834650_0001_m_000001_0\n",
"2024-08-04 19:14:33,741 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1722798834650_0001_m_000001 Task Transitioned from RUNNING to SUCCEEDED\n",
"2024-08-04 19:14:33,766 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 1\n",
"2024-08-04 19:14:33,786 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:33,786 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Reduce slow start threshold reached. Scheduling reduces.\n",
"2024-08-04 19:14:33,786 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: completedMapPercent 0.2 totalResourceLimit: finalMapResourceLimit: finalReduceResourceLimit: netScheduledMapResource: netScheduledReduceResource:\n",
"2024-08-04 19:14:33,786 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:1 ScheduledMaps:3 ScheduledReds:0 AssignedMaps:2 AssignedReds:0 CompletedMaps:1 CompletedReds:0 ContAlloc:2 ContRel:0 HostLocal:2 RackLocal:0\n",
"2024-08-04 19:14:33,969 INFO [IPC Server handler 6 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1722798834650_0001_m_000000_0 is : 0.0\n",
"2024-08-04 19:14:34,103 INFO [IPC Server handler 4 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1722798834650_0001_m_000000_0 is : 1.0\n",
"2024-08-04 19:14:34,111 INFO [IPC Server handler 5 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgment from attempt_1722798834650_0001_m_000000_0\n",
"2024-08-04 19:14:34,112 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000000_0 TaskAttempt Transitioned from RUNNING to SUCCESS_FINISHING_CONTAINER\n",
"2024-08-04 19:14:34,113 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1722798834650_0001_m_000000_0\n",
"2024-08-04 19:14:34,113 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1722798834650_0001_m_000000 Task Transitioned from RUNNING to SUCCEEDED\n",
"2024-08-04 19:14:34,113 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 2\n",
"2024-08-04 19:14:34,787 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:1 ScheduledMaps:3 ScheduledReds:0 AssignedMaps:2 AssignedReds:0 CompletedMaps:2 CompletedReds:0 ContAlloc:2 ContRel:0 HostLocal:2 RackLocal:0\n",
"2024-08-04 19:14:34,814 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_1722798834650_0001_01_000003\n",
"2024-08-04 19:14:34,815 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_1722798834650_0001_01_000002\n",
"2024-08-04 19:14:34,815 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 2\n",
"2024-08-04 19:14:34,815 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1722798834650_0001_m_000001_0: \n",
"2024-08-04 19:14:34,816 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000001_0 TaskAttempt Transitioned from SUCCESS_FINISHING_CONTAINER to SUCCEEDED\n",
"2024-08-04 19:14:34,816 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1722798834650_0001_m_000000_0: \n",
"2024-08-04 19:14:34,816 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000000_0 TaskAttempt Transitioned from SUCCESS_FINISHING_CONTAINER to SUCCEEDED\n",
"2024-08-04 19:14:34,816 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_1722798834650_0001_01_000004 to attempt_1722798834650_0001_m_000002_0\n",
"2024-08-04 19:14:34,817 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_1722798834650_0001_01_000005 to attempt_1722798834650_0001_m_000003_0\n",
"2024-08-04 19:14:34,817 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:34,817 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: completedMapPercent 0.4 totalResourceLimit: finalMapResourceLimit: finalReduceResourceLimit: netScheduledMapResource: netScheduledReduceResource:\n",
"2024-08-04 19:14:34,817 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:1 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:2 AssignedReds:0 CompletedMaps:2 CompletedReds:0 ContAlloc:4 ContRel:0 HostLocal:4 RackLocal:0\n",
"2024-08-04 19:14:34,817 INFO [ContainerLauncher #3] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_COMPLETED for container container_1722798834650_0001_01_000002 taskAttempt attempt_1722798834650_0001_m_000000_0\n",
"2024-08-04 19:14:34,818 INFO [ContainerLauncher #2] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_COMPLETED for container container_1722798834650_0001_01_000003 taskAttempt attempt_1722798834650_0001_m_000001_0\n",
"2024-08-04 19:14:34,821 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapred.JobConf: Task java-opts do not specify heap size. Setting task attempt jvm max heap size to -Xmx820m\n",
"2024-08-04 19:14:34,822 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000002_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED\n",
"2024-08-04 19:14:34,825 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapred.JobConf: Task java-opts do not specify heap size. Setting task attempt jvm max heap size to -Xmx820m\n",
"2024-08-04 19:14:34,825 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000003_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED\n",
"2024-08-04 19:14:34,829 INFO [ContainerLauncher #4] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_1722798834650_0001_01_000004 taskAttempt attempt_1722798834650_0001_m_000002_0\n",
"2024-08-04 19:14:34,829 INFO [ContainerLauncher #4] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1722798834650_0001_m_000002_0\n",
"2024-08-04 19:14:34,835 INFO [ContainerLauncher #5] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_1722798834650_0001_01_000005 taskAttempt attempt_1722798834650_0001_m_000003_0\n",
"2024-08-04 19:14:34,835 INFO [ContainerLauncher #5] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1722798834650_0001_m_000003_0\n",
"2024-08-04 19:14:34,903 INFO [ContainerLauncher #4] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1722798834650_0001_m_000002_0 : 43847\n",
"2024-08-04 19:14:34,904 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1722798834650_0001_m_000002_0] using containerId: [container_1722798834650_0001_01_000004 on NM: [localhost:34535]\n",
"2024-08-04 19:14:34,904 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000002_0 TaskAttempt Transitioned from ASSIGNED to RUNNING\n",
"2024-08-04 19:14:34,904 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1722798834650_0001_m_000002 Task Transitioned from SCHEDULED to RUNNING\n",
"2024-08-04 19:14:34,929 INFO [ContainerLauncher #5] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1722798834650_0001_m_000003_0 : 43847\n",
"2024-08-04 19:14:34,930 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1722798834650_0001_m_000003_0] using containerId: [container_1722798834650_0001_01_000005 on NM: [localhost:34535]\n",
"2024-08-04 19:14:34,930 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000003_0 TaskAttempt Transitioned from ASSIGNED to RUNNING\n",
"2024-08-04 19:14:34,931 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1722798834650_0001_m_000003 Task Transitioned from SCHEDULED to RUNNING\n",
"2024-08-04 19:14:35,827 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: applicationId=application_1722798834650_0001: ask=3 release=0 newContainers=0 finishedContainers=0 resourceLimit= knownNMs=1\n",
"2024-08-04 19:14:35,827 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:35,827 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: completedMapPercent 0.4 totalResourceLimit: finalMapResourceLimit: finalReduceResourceLimit: netScheduledMapResource: netScheduledReduceResource:\n",
"2024-08-04 19:14:36,854 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:36,854 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: completedMapPercent 0.4 totalResourceLimit: finalMapResourceLimit: finalReduceResourceLimit: netScheduledMapResource: netScheduledReduceResource:\n",
"2024-08-04 19:14:37,863 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:37,863 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: completedMapPercent 0.4 totalResourceLimit: finalMapResourceLimit: finalReduceResourceLimit: netScheduledMapResource: netScheduledReduceResource:\n",
"2024-08-04 19:14:38,869 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:38,869 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: completedMapPercent 0.4 totalResourceLimit: finalMapResourceLimit: finalReduceResourceLimit: netScheduledMapResource: netScheduledReduceResource:\n",
"2024-08-04 19:14:39,876 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:39,876 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: completedMapPercent 0.4 totalResourceLimit: finalMapResourceLimit: finalReduceResourceLimit: netScheduledMapResource: netScheduledReduceResource:\n",
"2024-08-04 19:14:40,881 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:40,881 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: completedMapPercent 0.4 totalResourceLimit: finalMapResourceLimit: finalReduceResourceLimit: netScheduledMapResource: netScheduledReduceResource:\n",
"2024-08-04 19:14:41,051 INFO [Socket Reader #1 for port 0] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1722798834650_0001 (auth:SIMPLE) from localhost:34438 / 127.0.0.1:34438\n",
"2024-08-04 19:14:41,087 INFO [IPC Server handler 4 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1722798834650_0001_m_000005 asked for a task\n",
"2024-08-04 19:14:41,087 INFO [IPC Server handler 4 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1722798834650_0001_m_000005 given task: attempt_1722798834650_0001_m_000003_0\n",
"2024-08-04 19:14:41,270 INFO [Socket Reader #1 for port 0] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1722798834650_0001 (auth:SIMPLE) from localhost:34442 / 127.0.0.1:34442\n",
"2024-08-04 19:14:41,315 INFO [IPC Server handler 2 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1722798834650_0001_m_000004 asked for a task\n",
"2024-08-04 19:14:41,315 INFO [IPC Server handler 2 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1722798834650_0001_m_000004 given task: attempt_1722798834650_0001_m_000002_0\n",
"2024-08-04 19:14:41,895 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:41,895 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: completedMapPercent 0.4 totalResourceLimit: finalMapResourceLimit: finalReduceResourceLimit: netScheduledMapResource: netScheduledReduceResource:\n",
"2024-08-04 19:14:42,901 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:42,901 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: completedMapPercent 0.4 totalResourceLimit: finalMapResourceLimit: finalReduceResourceLimit: netScheduledMapResource: netScheduledReduceResource:\n",
"2024-08-04 19:14:43,910 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:43,910 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: completedMapPercent 0.4 totalResourceLimit: finalMapResourceLimit: finalReduceResourceLimit: netScheduledMapResource: netScheduledReduceResource:\n",
"2024-08-04 19:14:44,917 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:44,917 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: completedMapPercent 0.4 totalResourceLimit: finalMapResourceLimit: finalReduceResourceLimit: netScheduledMapResource: netScheduledReduceResource:\n",
"2024-08-04 19:14:45,337 INFO [IPC Server handler 0 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1722798834650_0001_m_000003_0 is : 0.0\n",
"2024-08-04 19:14:45,475 INFO [IPC Server handler 3 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1722798834650_0001_m_000003_0 is : 1.0\n",
"2024-08-04 19:14:45,485 INFO [IPC Server handler 11 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgment from attempt_1722798834650_0001_m_000003_0\n",
"2024-08-04 19:14:45,489 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000003_0 TaskAttempt Transitioned from RUNNING to SUCCESS_FINISHING_CONTAINER\n",
"2024-08-04 19:14:45,489 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1722798834650_0001_m_000003_0\n",
"2024-08-04 19:14:45,489 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1722798834650_0001_m_000003 Task Transitioned from RUNNING to SUCCEEDED\n",
"2024-08-04 19:14:45,489 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 3\n",
"2024-08-04 19:14:45,525 INFO [IPC Server handler 1 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1722798834650_0001_m_000002_0 is : 0.0\n",
"2024-08-04 19:14:45,634 INFO [IPC Server handler 14 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1722798834650_0001_m_000002_0 is : 1.0\n",
"2024-08-04 19:14:45,639 INFO [IPC Server handler 21 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgment from attempt_1722798834650_0001_m_000002_0\n",
"2024-08-04 19:14:45,641 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000002_0 TaskAttempt Transitioned from RUNNING to SUCCESS_FINISHING_CONTAINER\n",
"2024-08-04 19:14:45,647 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1722798834650_0001_m_000002_0\n",
"2024-08-04 19:14:45,647 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1722798834650_0001_m_000002 Task Transitioned from RUNNING to SUCCEEDED\n",
"2024-08-04 19:14:45,647 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 4\n",
"2024-08-04 19:14:45,918 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:1 ScheduledMaps:1 ScheduledReds:0 AssignedMaps:2 AssignedReds:0 CompletedMaps:4 CompletedReds:0 ContAlloc:4 ContRel:0 HostLocal:4 RackLocal:0\n",
"2024-08-04 19:14:45,923 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Recalculating schedule, headroom=\n",
"2024-08-04 19:14:45,923 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: completedMapPercent 0.8 totalResourceLimit: finalMapResourceLimit: finalReduceResourceLimit: netScheduledMapResource: netScheduledReduceResource:\n",
"2024-08-04 19:14:45,923 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Ramping up 1\n",
"2024-08-04 19:14:45,923 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:1 ScheduledReds:1 AssignedMaps:2 AssignedReds:0 CompletedMaps:4 CompletedReds:0 ContAlloc:4 ContRel:0 HostLocal:4 RackLocal:0\n",
"2024-08-04 19:14:46,933 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: applicationId=application_1722798834650_0001: ask=1 release=0 newContainers=1 finishedContainers=2 resourceLimit= knownNMs=1\n",
"2024-08-04 19:14:46,933 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_1722798834650_0001_01_000005\n",
"2024-08-04 19:14:46,933 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_1722798834650_0001_01_000004\n",
"2024-08-04 19:14:46,933 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1\n",
"2024-08-04 19:14:46,933 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1722798834650_0001_m_000003_0: \n",
"2024-08-04 19:14:46,934 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000003_0 TaskAttempt Transitioned from SUCCESS_FINISHING_CONTAINER to SUCCEEDED\n",
"2024-08-04 19:14:46,934 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1722798834650_0001_m_000002_0: \n",
"2024-08-04 19:14:46,934 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_1722798834650_0001_01_000006 to attempt_1722798834650_0001_m_000004_0\n",
"2024-08-04 19:14:46,934 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000002_0 TaskAttempt Transitioned from SUCCESS_FINISHING_CONTAINER to SUCCEEDED\n",
"2024-08-04 19:14:46,934 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:1 AssignedMaps:1 AssignedReds:0 CompletedMaps:4 CompletedReds:0 ContAlloc:5 ContRel:0 HostLocal:5 RackLocal:0\n",
"2024-08-04 19:14:46,937 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapred.JobConf: Task java-opts do not specify heap size. Setting task attempt jvm max heap size to -Xmx820m\n",
"2024-08-04 19:14:46,938 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000004_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED\n",
"2024-08-04 19:14:46,939 INFO [ContainerLauncher #6] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_COMPLETED for container container_1722798834650_0001_01_000005 taskAttempt attempt_1722798834650_0001_m_000003_0\n",
"2024-08-04 19:14:46,942 INFO [ContainerLauncher #7] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_COMPLETED for container container_1722798834650_0001_01_000004 taskAttempt attempt_1722798834650_0001_m_000002_0\n",
"2024-08-04 19:14:46,946 INFO [ContainerLauncher #8] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_1722798834650_0001_01_000006 taskAttempt attempt_1722798834650_0001_m_000004_0\n",
"2024-08-04 19:14:46,946 INFO [ContainerLauncher #8] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1722798834650_0001_m_000004_0\n",
"2024-08-04 19:14:46,985 INFO [ContainerLauncher #8] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1722798834650_0001_m_000004_0 : 43847\n",
"2024-08-04 19:14:46,985 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1722798834650_0001_m_000004_0] using containerId: [container_1722798834650_0001_01_000006 on NM: [localhost:34535]\n",
"2024-08-04 19:14:46,985 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000004_0 TaskAttempt Transitioned from ASSIGNED to RUNNING\n",
"2024-08-04 19:14:46,986 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1722798834650_0001_m_000004 Task Transitioned from SCHEDULED to RUNNING\n",
"2024-08-04 19:14:47,949 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: applicationId=application_1722798834650_0001: ask=3 release=0 newContainers=1 finishedContainers=0 resourceLimit= knownNMs=1\n",
"2024-08-04 19:14:47,949 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated containers 1\n",
"2024-08-04 19:14:47,949 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned to reduce\n",
"2024-08-04 19:14:47,949 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Assigned container container_1722798834650_0001_01_000007 to attempt_1722798834650_0001_r_000000_0\n",
"2024-08-04 19:14:47,949 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:1 CompletedMaps:4 CompletedReds:0 ContAlloc:6 ContRel:0 HostLocal:5 RackLocal:0\n",
"2024-08-04 19:14:47,989 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapred.JobConf: Task java-opts do not specify heap size. Setting task attempt jvm max heap size to -Xmx820m\n",
"2024-08-04 19:14:48,000 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_r_000000_0 TaskAttempt Transitioned from UNASSIGNED to ASSIGNED\n",
"2024-08-04 19:14:48,008 INFO [ContainerLauncher #9] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_REMOTE_LAUNCH for container container_1722798834650_0001_01_000007 taskAttempt attempt_1722798834650_0001_r_000000_0\n",
"2024-08-04 19:14:48,008 INFO [ContainerLauncher #9] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Launching attempt_1722798834650_0001_r_000000_0\n",
"2024-08-04 19:14:48,112 INFO [ContainerLauncher #9] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Shuffle port returned by ContainerManager for attempt_1722798834650_0001_r_000000_0 : 43847\n",
"2024-08-04 19:14:48,112 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: TaskAttempt: [attempt_1722798834650_0001_r_000000_0] using containerId: [container_1722798834650_0001_01_000007 on NM: [localhost:34535]\n",
"2024-08-04 19:14:48,112 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_r_000000_0 TaskAttempt Transitioned from ASSIGNED to RUNNING\n",
"2024-08-04 19:14:48,113 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1722798834650_0001_r_000000 Task Transitioned from SCHEDULED to RUNNING\n",
"2024-08-04 19:14:48,954 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerRequestor: applicationId=application_1722798834650_0001: ask=1 release=0 newContainers=0 finishedContainers=0 resourceLimit= knownNMs=1\n",
"2024-08-04 19:14:53,473 INFO [Socket Reader #1 for port 0] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1722798834650_0001 (auth:SIMPLE) from localhost:58978 / 127.0.0.1:58978\n",
"2024-08-04 19:14:53,530 INFO [IPC Server handler 1 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1722798834650_0001_m_000006 asked for a task\n",
"2024-08-04 19:14:53,530 INFO [IPC Server handler 1 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1722798834650_0001_m_000006 given task: attempt_1722798834650_0001_m_000004_0\n",
"2024-08-04 19:14:54,873 INFO [Socket Reader #1 for port 0] SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for job_1722798834650_0001 (auth:SIMPLE) from localhost:58994 / 127.0.0.1:58994\n",
"2024-08-04 19:14:54,935 INFO [IPC Server handler 6 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID : jvm_1722798834650_0001_r_000007 asked for a task\n",
"2024-08-04 19:14:54,935 INFO [IPC Server handler 6 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: JVM with ID: jvm_1722798834650_0001_r_000007 given task: attempt_1722798834650_0001_r_000000_0\n",
"2024-08-04 19:14:57,673 INFO [IPC Server handler 6 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1722798834650_0001_m_000004_0 is : 0.0\n",
"2024-08-04 19:14:57,820 INFO [IPC Server handler 4 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1722798834650_0001_m_000004_0 is : 1.0\n",
"2024-08-04 19:14:57,841 INFO [IPC Server handler 5 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgment from attempt_1722798834650_0001_m_000004_0\n",
"2024-08-04 19:14:57,843 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000004_0 TaskAttempt Transitioned from RUNNING to SUCCESS_FINISHING_CONTAINER\n",
"2024-08-04 19:14:57,843 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1722798834650_0001_m_000004_0\n",
"2024-08-04 19:14:57,845 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1722798834650_0001_m_000004 Task Transitioned from RUNNING to SUCCEEDED\n",
"2024-08-04 19:14:57,845 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 5\n",
"2024-08-04 19:14:58,031 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Before Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:1 AssignedReds:1 CompletedMaps:5 CompletedReds:0 ContAlloc:6 ContRel:0 HostLocal:5 RackLocal:0\n",
"2024-08-04 19:14:58,052 INFO [IPC Server handler 2 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: MapCompletionEvents request from attempt_1722798834650_0001_r_000000_0. startIndex 0 maxEvents 10000\n",
"2024-08-04 19:14:58,388 INFO [IPC Server handler 3 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1722798834650_0001_r_000000_0 is : 0.0\n",
"2024-08-04 19:14:59,042 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Received completed container container_1722798834650_0001_01_000006\n",
"2024-08-04 19:14:59,042 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: After Scheduling: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:0 AssignedReds:1 CompletedMaps:5 CompletedReds:0 ContAlloc:6 ContRel:0 HostLocal:5 RackLocal:0\n",
"2024-08-04 19:14:59,043 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: Diagnostics report from attempt_1722798834650_0001_m_000004_0: \n",
"2024-08-04 19:14:59,043 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_m_000004_0 TaskAttempt Transitioned from SUCCESS_FINISHING_CONTAINER to SUCCEEDED\n",
"2024-08-04 19:14:59,043 INFO [ContainerLauncher #0] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: Processing the event EventType: CONTAINER_COMPLETED for container container_1722798834650_0001_01_000006 taskAttempt attempt_1722798834650_0001_m_000004_0\n",
"2024-08-04 19:14:59,151 INFO [IPC Server handler 0 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit-pending state update from attempt_1722798834650_0001_r_000000_0\n",
"2024-08-04 19:14:59,154 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_r_000000_0 TaskAttempt Transitioned from RUNNING to COMMIT_PENDING\n",
"2024-08-04 19:14:59,155 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: attempt_1722798834650_0001_r_000000_0 given a go for committing the task output.\n",
"2024-08-04 19:14:59,159 INFO [IPC Server handler 3 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Commit go/no-go request from attempt_1722798834650_0001_r_000000_0\n",
"2024-08-04 19:14:59,159 INFO [IPC Server handler 3 on default port 40041] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Result of canCommit for attempt_1722798834650_0001_r_000000_0:true\n",
"2024-08-04 19:14:59,210 INFO [IPC Server handler 11 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1722798834650_0001_r_000000_0 is : 1.0\n",
"2024-08-04 19:14:59,215 INFO [IPC Server handler 1 on default port 40041] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Done acknowledgment from attempt_1722798834650_0001_r_000000_0\n",
"2024-08-04 19:14:59,218 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_r_000000_0 TaskAttempt Transitioned from COMMIT_PENDING to SUCCESS_FINISHING_CONTAINER\n",
"2024-08-04 19:14:59,221 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: Task succeeded with attempt attempt_1722798834650_0001_r_000000_0\n",
"2024-08-04 19:14:59,222 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskImpl: task_1722798834650_0001_r_000000 Task Transitioned from RUNNING to SUCCEEDED\n",
"2024-08-04 19:14:59,222 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Num completed Tasks: 6\n",
"2024-08-04 19:14:59,223 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1722798834650_0001Job Transitioned from RUNNING to COMMITTING\n",
"2024-08-04 19:14:59,249 INFO [CommitterEvent Processor #1] org.apache.hadoop.mapreduce.v2.app.commit.CommitterEventHandler: Processing the event EventType: JOB_COMMIT\n",
"2024-08-04 19:14:59,341 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Calling handler for JobFinishedEvent \n",
"2024-08-04 19:14:59,342 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: job_1722798834650_0001Job Transitioned from COMMITTING to SUCCEEDED\n",
"2024-08-04 19:14:59,356 INFO [Thread-78] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Job finished cleanly, recording last MRAppMaster retry\n",
"2024-08-04 19:14:59,356 INFO [Thread-78] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify RMCommunicator isAMLastRetry: true\n",
"2024-08-04 19:14:59,356 INFO [Thread-78] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: RMCommunicator notified that shouldUnregistered is: true\n",
"2024-08-04 19:14:59,356 INFO [Thread-78] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Notify JHEH isAMLastRetry: true\n",
"2024-08-04 19:14:59,356 INFO [Thread-78] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: JobHistoryEventHandler notified that forceJobCompletion is true\n",
"2024-08-04 19:14:59,356 INFO [Thread-78] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Calling stop for all the services\n",
"2024-08-04 19:14:59,362 INFO [Thread-78] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopping JobHistoryEventHandler. Size of the outstanding queue size is 0\n",
"2024-08-04 19:14:59,410 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://localhost:8902/tmp/hadoop-yarn/staging/root/.staging/job_1722798834650_0001/job_1722798834650_0001_1.jhist to hdfs://localhost:8902/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1722798834650_0001-1722798847160-root-QuasiMonteCarlo-1722798899334-5-1-SUCCEEDED-root.default-1722798860122.jhist_tmp\n",
"2024-08-04 19:14:59,453 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied from: hdfs://localhost:8902/tmp/hadoop-yarn/staging/root/.staging/job_1722798834650_0001/job_1722798834650_0001_1.jhist to done location: hdfs://localhost:8902/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1722798834650_0001-1722798847160-root-QuasiMonteCarlo-1722798899334-5-1-SUCCEEDED-root.default-1722798860122.jhist_tmp\n",
"2024-08-04 19:14:59,454 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Set historyUrl to http://446a375b7fe4:19888/jobhistory/job/job_1722798834650_0001\n",
"2024-08-04 19:14:59,456 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copying hdfs://localhost:8902/tmp/hadoop-yarn/staging/root/.staging/job_1722798834650_0001/job_1722798834650_0001_1_conf.xml to hdfs://localhost:8902/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1722798834650_0001_conf.xml_tmp\n",
"2024-08-04 19:14:59,508 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Copied from: hdfs://localhost:8902/tmp/hadoop-yarn/staging/root/.staging/job_1722798834650_0001/job_1722798834650_0001_1_conf.xml to done location: hdfs://localhost:8902/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1722798834650_0001_conf.xml_tmp\n",
"2024-08-04 19:14:59,521 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://localhost:8902/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1722798834650_0001.summary_tmp to hdfs://localhost:8902/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1722798834650_0001.summary\n",
"2024-08-04 19:14:59,530 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://localhost:8902/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1722798834650_0001_conf.xml_tmp to hdfs://localhost:8902/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1722798834650_0001_conf.xml\n",
"2024-08-04 19:14:59,536 INFO [eventHandlingThread] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Moved tmp to done: hdfs://localhost:8902/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1722798834650_0001-1722798847160-root-QuasiMonteCarlo-1722798899334-5-1-SUCCEEDED-root.default-1722798860122.jhist_tmp to hdfs://localhost:8902/tmp/hadoop-yarn/staging/history/done_intermediate/root/job_1722798834650_0001-1722798847160-root-QuasiMonteCarlo-1722798899334-5-1-SUCCEEDED-root.default-1722798860122.jhist\n",
"2024-08-04 19:14:59,540 INFO [Thread-78] org.apache.hadoop.mapreduce.jobhistory.JobHistoryEventHandler: Stopped JobHistoryEventHandler. super.stop()\n",
"2024-08-04 19:14:59,541 INFO [Thread-78] org.apache.hadoop.mapreduce.v2.app.launcher.ContainerLauncherImpl: KILLING attempt_1722798834650_0001_r_000000_0\n",
"2024-08-04 19:14:59,582 INFO [AsyncDispatcher event handler] org.apache.hadoop.mapreduce.v2.app.job.impl.TaskAttemptImpl: attempt_1722798834650_0001_r_000000_0 TaskAttempt Transitioned from SUCCESS_FINISHING_CONTAINER to SUCCEEDED\n",
"2024-08-04 19:14:59,589 INFO [Thread-78] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Setting job diagnostics to \n",
"2024-08-04 19:14:59,589 INFO [Thread-78] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: History url is http://446a375b7fe4:19888/jobhistory/job/job_1722798834650_0001\n",
"2024-08-04 19:14:59,638 INFO [Thread-78] org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator: Waiting for application to be successfully unregistered.\n",
"2024-08-04 19:15:00,641 INFO [Thread-78] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Final Stats: PendingReds:0 ScheduledMaps:0 ScheduledReds:0 AssignedMaps:0 AssignedReds:1 CompletedMaps:5 CompletedReds:0 ContAlloc:6 ContRel:0 HostLocal:5 RackLocal:0\n",
"2024-08-04 19:15:00,643 INFO [Thread-78] org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Deleting staging directory hdfs://localhost:8902/ /tmp/hadoop-yarn/staging/root/.staging/job_1722798834650_0001\n",
"2024-08-04 19:15:00,646 INFO [Thread-78] org.apache.hadoop.ipc.Server: Stopping server on 40041\n",
"2024-08-04 19:15:00,647 INFO [IPC Server listener on 0] org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 0\n",
"2024-08-04 19:15:00,649 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: Stopping IPC Server Responder\n",
"2024-08-04 19:15:00,654 INFO [TaskHeartbeatHandler PingChecker] org.apache.hadoop.mapreduce.v2.app.TaskHeartbeatHandler: TaskHeartbeatHandler thread interrupted\n",
"2024-08-04 19:15:00,655 INFO [Ping Checker for TaskAttemptFinishingMonitor] org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: TaskAttemptFinishingMonitor thread interrupted\n",
"2024-08-04 19:15:05,657 INFO [Thread-78] org.apache.hadoop.ipc.Server: Stopping server on 34269\n",
"2024-08-04 19:15:05,658 INFO [IPC Server Responder] org.apache.hadoop.ipc.Server: Stopping IPC Server Responder\n",
"2024-08-04 19:15:05,658 INFO [IPC Server listener on 0] org.apache.hadoop.ipc.Server: Stopping IPC Server listener on 0\n",
"2024-08-04 19:15:05,674 INFO [Thread-78] org.eclipse.jetty.server.handler.ContextHandler: Stopped o.e.j.w.WebAppContext@7a1ddbf1{mapreduce,/,null,STOPPED}{jar:file:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-common-3.4.0.jar!/webapps/mapreduce}\n",
"2024-08-04 19:15:05,691 INFO [Thread-78] org.eclipse.jetty.server.AbstractConnector: Stopped ServerConnector@58fbd02e{HTTP/1.1, (http/1.1)}{0.0.0.0:0}\n",
"2024-08-04 19:15:05,691 INFO [Thread-78] org.eclipse.jetty.server.session: node0 Stopped scavenging\n",
"2024-08-04 19:15:05,692 INFO [Thread-78] org.eclipse.jetty.server.handler.ContextHandler: Stopped o.e.j.s.ServletContextHandler@48a663e9{static,/static,jar:file:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-common-3.4.0.jar!/webapps/static,STOPPED}\n",
"\n",
"End of LogType:syslog\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000006 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:directory.info\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:2101\n",
"LogContents:\n",
"ls -l:\n",
"total 32\n",
"-rw-r--r-- 1 root root 129 Aug 4 19:14 container_tokens\n",
"-rwx------ 1 root root 964 Aug 4 19:14 default_container_executor_session.sh\n",
"-rwx------ 1 root root 1019 Aug 4 19:14 default_container_executor.sh\n",
"lrwxrwxrwx 1 root root 179 Aug 4 19:14 job.jar -> /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/11/job.jar\n",
"lrwxrwxrwx 1 root root 179 Aug 4 19:14 job.xml -> /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/13/job.xml\n",
"-rwx------ 1 root root 8159 Aug 4 19:14 launch_container.sh\n",
"drwx--x--- 2 root root 4096 Aug 4 19:14 tmp\n",
"find -L . -maxdepth 5 -ls:\n",
" 809806 4 drwx--x--- 3 root root 4096 Aug 4 19:14 .\n",
" 809828 4 -rwx------ 1 root root 1019 Aug 4 19:14 ./default_container_executor.sh\n",
" 809751 260 -r-x------ 1 root root 264361 Aug 4 19:14 ./job.xml\n",
" 809822 4 -rw-r--r-- 1 root root 129 Aug 4 19:14 ./container_tokens\n",
" 809826 4 -rwx------ 1 root root 964 Aug 4 19:14 ./default_container_executor_session.sh\n",
" 809824 8 -rwx------ 1 root root 8159 Aug 4 19:14 ./launch_container.sh\n",
" 809829 4 -rw-r--r-- 1 root root 16 Aug 4 19:14 ./.default_container_executor.sh.crc\n",
" 809825 4 -rw-r--r-- 1 root root 72 Aug 4 19:14 ./.launch_container.sh.crc\n",
" 809745 4 drwx------ 2 root root 4096 Aug 4 19:14 ./job.jar\n",
" 809746 276 -r-x------ 1 root root 281609 Aug 4 19:14 ./job.jar/job.jar\n",
" 809823 4 -rw-r--r-- 1 root root 12 Aug 4 19:14 ./.container_tokens.crc\n",
" 809827 4 -rw-r--r-- 1 root root 16 Aug 4 19:14 ./.default_container_executor_session.sh.crc\n",
" 809821 4 drwx--x--- 2 root root 4096 Aug 4 19:14 ./tmp\n",
"broken symlinks(find -L . -maxdepth 5 -type l -ls):\n",
"\n",
"End of LogType:directory.info\n",
"*******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000006 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:launch_container.sh\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:8159\n",
"LogContents:\n",
"#!/bin/bash\n",
"\n",
"set -o pipefail -e\n",
"export PRELAUNCH_OUT=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000006/prelaunch.out\"\n",
"exec >\"${PRELAUNCH_OUT}\"\n",
"export PRELAUNCH_ERR=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000006/prelaunch.err\"\n",
"exec 2>\"${PRELAUNCH_ERR}\"\n",
"echo \"Setting up env variables\"\n",
"export JAVA_HOME=${JAVA_HOME:-\"/usr/lib/jvm/java-11-openjdk-amd64\"}\n",
"export HADOOP_COMMON_HOME=${HADOOP_COMMON_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_HDFS_HOME=${HADOOP_HDFS_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-\"/content/hadoop-3.4.0/etc/hadoop\"}\n",
"export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_HOME=${HADOOP_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export PATH=${PATH:-\"/content/hadoop-3.4.0/bin:/opt/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/tools/node/bin:/tools/google-cloud-sdk/bin\"}\n",
"export LANG=${LANG:-\"en_US.UTF-8\"}\n",
"export HADOOP_TOKEN_FILE_LOCATION=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000006/container_tokens\"\n",
"export CONTAINER_ID=\"container_1722798834650_0001_01_000006\"\n",
"export NM_PORT=\"34535\"\n",
"export NM_HOST=\"localhost\"\n",
"export NM_HTTP_PORT=\"37361\"\n",
"export LOCAL_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001\"\n",
"export LOCAL_USER_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/\"\n",
"export LOG_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_0/application_1722798834650_0001/container_1722798834650_0001_01_000006,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000006,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000006,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000006\"\n",
"export USER=\"root\"\n",
"export LOGNAME=\"root\"\n",
"export HOME=\"/home/\"\n",
"export PWD=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000006\"\n",
"export LOCALIZATION_COUNTERS=\"0,548046,0,2,1\"\n",
"export JVM_PID=\"$$\"\n",
"export NM_AUX_SERVICE_mapreduce_shuffle=\"AACrRwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=\"\n",
"export STDOUT_LOGFILE_ENV=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000006/stdout\"\n",
"export SHELL=\"/bin/bash\"\n",
"export HADOOP_ROOT_LOGGER=\"INFO,console\"\n",
"export CLASSPATH=\"$PWD:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:/content/hadoop-3.4.0/share/hadoop/mapreduce/*:/content/hadoop-3.4.0/share/hadoop/mapreduce/lib/*:job.jar/*:job.jar/classes/:job.jar/lib/*:$PWD/*\"\n",
"export LD_LIBRARY_PATH=\"$PWD:$HADOOP_COMMON_HOME/lib/native\"\n",
"export STDERR_LOGFILE_ENV=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000006/stderr\"\n",
"export HADOOP_CLIENT_OPTS=\"\"\n",
"export MALLOC_ARENA_MAX=\"4\"\n",
"echo \"Setting up job resources\"\n",
"ln -sf -- \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/13/job.xml\" \"job.xml\"\n",
"ln -sf -- \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/11/job.jar\" \"job.jar\"\n",
"echo \"Copying debugging information\"\n",
"# Creating copy of launch script\n",
"cp \"launch_container.sh\" \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000006/launch_container.sh\"\n",
"chmod 640 \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000006/launch_container.sh\"\n",
"# Determining directory contents\n",
"echo \"ls -l:\" 1>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000006/directory.info\"\n",
"ls -l 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000006/directory.info\"\n",
"echo \"find -L . -maxdepth 5 -ls:\" 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000006/directory.info\"\n",
"find -L . -maxdepth 5 -ls 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000006/directory.info\"\n",
"echo \"broken symlinks(find -L . -maxdepth 5 -type l -ls):\" 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000006/directory.info\"\n",
"find -L . -maxdepth 5 -type l -ls 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000006/directory.info\"\n",
"echo \"Launching container\"\n",
"exec /bin/bash -c \"$JAVA_HOME/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx820m -Djava.io.tmpdir=$PWD/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000006 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog org.apache.hadoop.mapred.YarnChild 127.0.0.1 40041 attempt_1722798834650_0001_m_000004_0 6 1>/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000006/stdout 2>/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000006/stderr \"\n",
"\n",
"End of LogType:launch_container.sh\n",
"************************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000006 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:prelaunch.err\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:0\n",
"LogContents:\n",
"\n",
"End of LogType:prelaunch.err\n",
"******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000006 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:prelaunch.out\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:100\n",
"LogContents:\n",
"Setting up env variables\n",
"Setting up job resources\n",
"Copying debugging information\n",
"Launching container\n",
"\n",
"End of LogType:prelaunch.out\n",
"******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000006 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:stderr\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:0\n",
"LogContents:\n",
"\n",
"End of LogType:stderr\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000006 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:stdout\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:0\n",
"LogContents:\n",
"\n",
"End of LogType:stdout\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000006 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:syslog\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:31510\n",
"LogContents:\n",
"2024-08-04 19:14:51,129 INFO [main] org.apache.hadoop.security.SecurityUtil: Updating Configuration\n",
"2024-08-04 19:14:51,786 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties\n",
"2024-08-04 19:14:52,527 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).\n",
"2024-08-04 19:14:52,527 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system started\n",
"2024-08-04 19:14:52,926 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens: [Kind: mapreduce.job, Service: job_1722798834650_0001, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@70f02c32)]\n",
"2024-08-04 19:14:53,089 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.\n",
"2024-08-04 19:14:53,846 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001\n",
"2024-08-04 19:14:55,154 INFO [main] org.apache.hadoop.mapred.YarnChild: \n",
"/************************************************************\n",
"[system properties]\n",
"os.name: Linux\n",
"os.version: 6.1.85+\n",
"java.home: /usr/lib/jvm/java-11-openjdk-amd64\n",
"java.runtime.version: 11.0.24+8-post-Ubuntu-1ubuntu322.04\n",
"java.vendor: Ubuntu\n",
"java.version: 11.0.24\n",
"java.vm.name: OpenJDK 64-Bit Server VM\n",
"java.class.path: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000006:/content/hadoop-3.4.0/etc/hadoop:/content/hadoop-3.4.0/share/hadoop/common/hadoop-kms-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-common-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-registry-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-nfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/j2objc-annotations-1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-shaded-guava-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-beanutils-1.9.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsch-0.1.55.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsp-api-2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-io-2.14.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-all-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-cli-1.5.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/bcprov-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/woodstox-core-5.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-servlet-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-http-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-kqueue-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/token-provider-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-crypto-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-logging-1.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-buffer-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-auth-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/guava-27.0-jre.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-xdr-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-proxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/re2j-1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-memcache-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-util-ajax-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-text-1.10.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/metrics-core-3.2.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-unix-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/avro-1.9.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/reload4j-1.2.22.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-configuration2-2.8.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-math3-3.6.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-daemon-1.0.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-core-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-codec-1.15.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-xml-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-lang3-3.12.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/animal-sniffer-annotations-1.17.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-client-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jettison-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-config-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-server-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-redis-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-asn1-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-identity-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-udt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-ssl-ocsp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/gson-2.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsr305-3.0.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-framework-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-databind-2.12.7.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-stomp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/httpcore-4.4.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-socks-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/zookeeper-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/slf4j-api-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-annotations-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-classes-kqueue-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-webapp-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-admin-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-recipes-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-server-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-pkix-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-xml-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-classes-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-io-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-haproxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-http-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jline-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/checker-qual-2.5.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-client-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-simplekdc-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-core-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-util-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-http2-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-collections-3.2.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jakarta.activation-api-1.2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-classes-macos-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-rxtx-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/zookeeper-jute-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-security-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-net-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/failureaccess-1.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-compress-1.24.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/stax2-api-4.2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-core-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final-linux-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-json-1.20.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/audience-annotations-0.12.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/dnsjava-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/nimbus-jose-jwt-9.31.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-common-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-shaded-protobuf_3_21-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/httpclient-4.5.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jul-to-slf4j-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-mqtt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-kqueue-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final-linux-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-smtp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/snappy-java-1.1.10.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-sctp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-native-client-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-client-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-rbf-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-nfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-rbf-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-native-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/j2objc-annotations-1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-shaded-guava-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-beanutils-1.9.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsch-0.1.55.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-io-2.14.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-all-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-cli-1.5.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/woodstox-core-5.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-servlet-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-http-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/token-provider-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-crypto-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-logging-1.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-buffer-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-auth-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/guava-27.0-jre.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-xdr-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-proxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/re2j-1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-memcache-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-util-ajax-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-text-1.10.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/HikariCP-4.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/metrics-core-3.2.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-unix-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/avro-1.9.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/reload4j-1.2.22.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-configuration2-2.8.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-math3-3.6.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-core-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-codec-1.15.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-xml-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-lang3-3.12.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/animal-sniffer-annotations-1.17.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-client-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jettison-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-config-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-server-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-redis-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-asn1-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-identity-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-udt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-ssl-ocsp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/gson-2.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsr305-3.0.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-framework-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-databind-2.12.7.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-stomp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/httpcore-4.4.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-socks-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/zookeeper-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-annotations-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-classes-kqueue-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-webapp-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-admin-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-recipes-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-server-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-pkix-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-xml-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-classes-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-io-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-haproxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-http-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jline-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/checker-qual-2.5.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-client-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-simplekdc-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-core-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-util-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-http2-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jakarta.activation-api-1.2.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-classes-macos-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-rxtx-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/zookeeper-jute-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-security-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-net-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/failureaccess-1.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-compress-1.24.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/stax2-api-4.2.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-core-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final-linux-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-json-1.20.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/audience-annotations-0.12.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/dnsjava-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/nimbus-jose-jwt-9.31.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-common-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-shaded-protobuf_3_21-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/httpclient-4.5.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-mqtt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final-linux-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-smtp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/snappy-java-1.1.10.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-sctp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-services-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-mawo-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-router-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-registry-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-api-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-globalpolicygenerator-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-services-api-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-tests-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jakarta.xml.bind-api-2.3.2.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-plus-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/objenesis-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.inject-1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jna-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.websocket-client-api-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/bcpkix-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/snakeyaml-2.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/guice-servlet-4.2.3.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/codemodel-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/asm-tree-9.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-client-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-api-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-client-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/bcutil-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/fst-2.50.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax-websocket-client-impl-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.websocket-api-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/guice-4.2.3.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jsonschema2pojo-core-1.0.2.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-common-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-jndi-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/asm-commons-9.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax-websocket-server-impl-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jersey-guice-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jersey-client-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-annotations-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-jaxrs-base-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/mockito-core-2.28.2.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/lib/*:job.jar/job.jar:job.jar/classes/:job.jar/lib/*:/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000006/job.jar\n",
"java.io.tmpdir: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000006/tmp\n",
"user.dir: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000006\n",
"user.name: root\n",
"************************************************************/\n",
"2024-08-04 19:14:55,155 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id\n",
"2024-08-04 19:14:56,696 INFO [main] org.apache.hadoop.mapreduce.lib.output.PathOutputCommitterFactory: No output committer factory defined, defaulting to FileOutputCommitterFactory\n",
"2024-08-04 19:14:56,703 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 2\n",
"2024-08-04 19:14:56,703 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false\n",
"2024-08-04 19:14:56,760 INFO [main] org.apache.hadoop.mapred.Task: Using ResourceCalculatorProcessTree : [ ]\n",
"2024-08-04 19:14:57,364 INFO [main] org.apache.hadoop.mapred.MapTask: Processing split: hdfs://localhost:8902/user/root/QuasiMonteCarlo_1722798838891_1650860168/in/part4:0+118\n",
"2024-08-04 19:14:57,582 INFO [main] org.apache.hadoop.mapred.MapTask: (EQUATOR) 0 kvi 26214396(104857584)\n",
"2024-08-04 19:14:57,583 INFO [main] org.apache.hadoop.mapred.MapTask: mapreduce.task.io.sort.mb: 100\n",
"2024-08-04 19:14:57,583 INFO [main] org.apache.hadoop.mapred.MapTask: soft limit at 83886080\n",
"2024-08-04 19:14:57,583 INFO [main] org.apache.hadoop.mapred.MapTask: bufstart = 0; bufvoid = 104857600\n",
"2024-08-04 19:14:57,583 INFO [main] org.apache.hadoop.mapred.MapTask: kvstart = 26214396; length = 6553600\n",
"2024-08-04 19:14:57,591 INFO [main] org.apache.hadoop.mapred.MapTask: Map output collector class = org.apache.hadoop.mapred.MapTask$MapOutputBuffer\n",
"2024-08-04 19:14:57,683 INFO [main] org.apache.hadoop.mapred.MapTask: Starting flush of map output\n",
"2024-08-04 19:14:57,683 INFO [main] org.apache.hadoop.mapred.MapTask: Spilling map output\n",
"2024-08-04 19:14:57,683 INFO [main] org.apache.hadoop.mapred.MapTask: bufstart = 0; bufend = 18; bufvoid = 104857600\n",
"2024-08-04 19:14:57,683 INFO [main] org.apache.hadoop.mapred.MapTask: kvstart = 26214396(104857584); kvend = 26214392(104857568); length = 5/6553600\n",
"2024-08-04 19:14:57,692 INFO [main] org.apache.hadoop.mapred.MapTask: Finished spill 0\n",
"2024-08-04 19:14:57,787 INFO [main] org.apache.hadoop.mapred.Task: Task:attempt_1722798834650_0001_m_000004_0 is done. And is in the process of committing\n",
"2024-08-04 19:14:57,844 INFO [main] org.apache.hadoop.mapred.Task: Task 'attempt_1722798834650_0001_m_000004_0' done.\n",
"2024-08-04 19:14:57,878 INFO [main] org.apache.hadoop.mapred.Task: Final Counters for attempt_1722798834650_0001_m_000004_0: Counters: 28\n",
"\tFile System Counters\n",
"\t\tFILE: Number of bytes read=0\n",
"\t\tFILE: Number of bytes written=309460\n",
"\t\tFILE: Number of read operations=0\n",
"\t\tFILE: Number of large read operations=0\n",
"\t\tFILE: Number of write operations=0\n",
"\t\tHDFS: Number of bytes read=264\n",
"\t\tHDFS: Number of bytes written=0\n",
"\t\tHDFS: Number of read operations=4\n",
"\t\tHDFS: Number of large read operations=0\n",
"\t\tHDFS: Number of write operations=0\n",
"\t\tHDFS: Number of bytes read erasure-coded=0\n",
"\tMap-Reduce Framework\n",
"\t\tMap input records=1\n",
"\t\tMap output records=2\n",
"\t\tMap output bytes=18\n",
"\t\tMap output materialized bytes=28\n",
"\t\tInput split bytes=146\n",
"\t\tCombine input records=0\n",
"\t\tSpilled Records=2\n",
"\t\tFailed Shuffles=0\n",
"\t\tMerged Map outputs=0\n",
"\t\tGC time elapsed (ms)=91\n",
"\t\tCPU time spent (ms)=840\n",
"\t\tPhysical memory (bytes) snapshot=361263104\n",
"\t\tVirtual memory (bytes) snapshot=2716454912\n",
"\t\tTotal committed heap usage (bytes)=273678336\n",
"\t\tPeak Map Physical memory (bytes)=361263104\n",
"\t\tPeak Map Virtual memory (bytes)=2716454912\n",
"\tFile Input Format Counters \n",
"\t\tBytes Read=118\n",
"2024-08-04 19:14:57,880 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping MapTask metrics system...\n",
"2024-08-04 19:14:57,883 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system stopped.\n",
"2024-08-04 19:14:57,883 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: MapTask metrics system shutdown complete.\n",
"\n",
"End of LogType:syslog\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000007 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:directory.info\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:2101\n",
"LogContents:\n",
"ls -l:\n",
"total 36\n",
"-rw-r--r-- 1 root root 129 Aug 4 19:14 container_tokens\n",
"-rwx------ 1 root root 964 Aug 4 19:14 default_container_executor_session.sh\n",
"-rwx------ 1 root root 1019 Aug 4 19:14 default_container_executor.sh\n",
"lrwxrwxrwx 1 root root 179 Aug 4 19:14 job.jar -> /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/11/job.jar\n",
"lrwxrwxrwx 1 root root 179 Aug 4 19:14 job.xml -> /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/13/job.xml\n",
"-rwx------ 1 root root 8351 Aug 4 19:14 launch_container.sh\n",
"drwx--x--- 2 root root 4096 Aug 4 19:14 tmp\n",
"find -L . -maxdepth 5 -ls:\n",
" 809887 4 drwx--x--- 3 root root 4096 Aug 4 19:14 .\n",
" 809917 4 -rwx------ 1 root root 1019 Aug 4 19:14 ./default_container_executor.sh\n",
" 809751 260 -r-x------ 1 root root 264361 Aug 4 19:14 ./job.xml\n",
" 809911 4 -rw-r--r-- 1 root root 129 Aug 4 19:14 ./container_tokens\n",
" 809915 4 -rwx------ 1 root root 964 Aug 4 19:14 ./default_container_executor_session.sh\n",
" 809913 12 -rwx------ 1 root root 8351 Aug 4 19:14 ./launch_container.sh\n",
" 809918 4 -rw-r--r-- 1 root root 16 Aug 4 19:14 ./.default_container_executor.sh.crc\n",
" 809914 4 -rw-r--r-- 1 root root 76 Aug 4 19:14 ./.launch_container.sh.crc\n",
" 809745 4 drwx------ 2 root root 4096 Aug 4 19:14 ./job.jar\n",
" 809746 276 -r-x------ 1 root root 281609 Aug 4 19:14 ./job.jar/job.jar\n",
" 809912 4 -rw-r--r-- 1 root root 12 Aug 4 19:14 ./.container_tokens.crc\n",
" 809916 4 -rw-r--r-- 1 root root 16 Aug 4 19:14 ./.default_container_executor_session.sh.crc\n",
" 809910 4 drwx--x--- 2 root root 4096 Aug 4 19:14 ./tmp\n",
"broken symlinks(find -L . -maxdepth 5 -type l -ls):\n",
"\n",
"End of LogType:directory.info\n",
"*******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000007 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:launch_container.sh\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:8351\n",
"LogContents:\n",
"#!/bin/bash\n",
"\n",
"set -o pipefail -e\n",
"export PRELAUNCH_OUT=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000007/prelaunch.out\"\n",
"exec >\"${PRELAUNCH_OUT}\"\n",
"export PRELAUNCH_ERR=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000007/prelaunch.err\"\n",
"exec 2>\"${PRELAUNCH_ERR}\"\n",
"echo \"Setting up env variables\"\n",
"export JAVA_HOME=${JAVA_HOME:-\"/usr/lib/jvm/java-11-openjdk-amd64\"}\n",
"export HADOOP_COMMON_HOME=${HADOOP_COMMON_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_HDFS_HOME=${HADOOP_HDFS_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_CONF_DIR=${HADOOP_CONF_DIR:-\"/content/hadoop-3.4.0/etc/hadoop\"}\n",
"export HADOOP_YARN_HOME=${HADOOP_YARN_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export HADOOP_HOME=${HADOOP_HOME:-\"/content/hadoop-3.4.0\"}\n",
"export PATH=${PATH:-\"/content/hadoop-3.4.0/bin:/opt/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/tools/node/bin:/tools/google-cloud-sdk/bin\"}\n",
"export LANG=${LANG:-\"en_US.UTF-8\"}\n",
"export HADOOP_TOKEN_FILE_LOCATION=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000007/container_tokens\"\n",
"export CONTAINER_ID=\"container_1722798834650_0001_01_000007\"\n",
"export NM_PORT=\"34535\"\n",
"export NM_HOST=\"localhost\"\n",
"export NM_HTTP_PORT=\"37361\"\n",
"export LOCAL_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001\"\n",
"export LOCAL_USER_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/\"\n",
"export LOG_DIRS=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_0/application_1722798834650_0001/container_1722798834650_0001_01_000007,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000007,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_2/application_1722798834650_0001/container_1722798834650_0001_01_000007,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_3/application_1722798834650_0001/container_1722798834650_0001_01_000007\"\n",
"export USER=\"root\"\n",
"export LOGNAME=\"root\"\n",
"export HOME=\"/home/\"\n",
"export PWD=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000007\"\n",
"export LOCALIZATION_COUNTERS=\"0,548046,0,2,14\"\n",
"export JVM_PID=\"$$\"\n",
"export NM_AUX_SERVICE_mapreduce_shuffle=\"AACrRwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=\"\n",
"export STDOUT_LOGFILE_ENV=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000007/stdout\"\n",
"export SHELL=\"/bin/bash\"\n",
"export HADOOP_ROOT_LOGGER=\"INFO,console\"\n",
"export CLASSPATH=\"$PWD:$HADOOP_CONF_DIR:$HADOOP_COMMON_HOME/share/hadoop/common/*:$HADOOP_COMMON_HOME/share/hadoop/common/lib/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/*:$HADOOP_HDFS_HOME/share/hadoop/hdfs/lib/*:$HADOOP_YARN_HOME/share/hadoop/yarn/*:$HADOOP_YARN_HOME/share/hadoop/yarn/lib/*:/content/hadoop-3.4.0/share/hadoop/mapreduce/*:/content/hadoop-3.4.0/share/hadoop/mapreduce/lib/*:job.jar/*:job.jar/classes/:job.jar/lib/*:$PWD/*\"\n",
"export LD_LIBRARY_PATH=\"$PWD:$HADOOP_COMMON_HOME/lib/native\"\n",
"export STDERR_LOGFILE_ENV=\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000007/stderr\"\n",
"export HADOOP_CLIENT_OPTS=\"\"\n",
"export MALLOC_ARENA_MAX=\"4\"\n",
"echo \"Setting up job resources\"\n",
"ln -sf -- \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/13/job.xml\" \"job.xml\"\n",
"ln -sf -- \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001/filecache/11/job.jar\" \"job.jar\"\n",
"echo \"Copying debugging information\"\n",
"# Creating copy of launch script\n",
"cp \"launch_container.sh\" \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000007/launch_container.sh\"\n",
"chmod 640 \"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000007/launch_container.sh\"\n",
"# Determining directory contents\n",
"echo \"ls -l:\" 1>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000007/directory.info\"\n",
"ls -l 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000007/directory.info\"\n",
"echo \"find -L . -maxdepth 5 -ls:\" 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000007/directory.info\"\n",
"find -L . -maxdepth 5 -ls 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000007/directory.info\"\n",
"echo \"broken symlinks(find -L . -maxdepth 5 -type l -ls):\" 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000007/directory.info\"\n",
"find -L . -maxdepth 5 -type l -ls 1>>\"/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000007/directory.info\"\n",
"echo \"Launching container\"\n",
"exec /bin/bash -c \"$JAVA_HOME/bin/java -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN -Xmx820m -Djava.io.tmpdir=$PWD/tmp -Dlog4j.configuration=container-log4j.properties -Dyarn.app.container.log.dir=/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000007 -Dyarn.app.container.log.filesize=0 -Dhadoop.root.logger=INFO,CLA -Dhadoop.root.logfile=syslog -Dyarn.app.mapreduce.shuffle.logger=INFO,shuffleCLA -Dyarn.app.mapreduce.shuffle.logfile=syslog.shuffle -Dyarn.app.mapreduce.shuffle.log.filesize=0 -Dyarn.app.mapreduce.shuffle.log.backups=0 org.apache.hadoop.mapred.YarnChild 127.0.0.1 40041 attempt_1722798834650_0001_r_000000_0 7 1>/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000007/stdout 2>/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-logDir-nm-0_1/application_1722798834650_0001/container_1722798834650_0001_01_000007/stderr \"\n",
"\n",
"End of LogType:launch_container.sh\n",
"************************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000007 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:prelaunch.err\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:0\n",
"LogContents:\n",
"\n",
"End of LogType:prelaunch.err\n",
"******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000007 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:prelaunch.out\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:100\n",
"LogContents:\n",
"Setting up env variables\n",
"Setting up job resources\n",
"Copying debugging information\n",
"Launching container\n",
"\n",
"End of LogType:prelaunch.out\n",
"******************************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000007 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:stderr\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:0\n",
"LogContents:\n",
"\n",
"End of LogType:stderr\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000007 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:stdout\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:0\n",
"LogContents:\n",
"\n",
"End of LogType:stdout\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000007 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:syslog\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:31103\n",
"LogContents:\n",
"2024-08-04 19:14:53,552 INFO [main] org.apache.hadoop.security.SecurityUtil: Updating Configuration\n",
"2024-08-04 19:14:53,885 INFO [main] org.apache.hadoop.metrics2.impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties\n",
"2024-08-04 19:14:54,218 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).\n",
"2024-08-04 19:14:54,218 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system started\n",
"2024-08-04 19:14:54,418 INFO [main] org.apache.hadoop.mapred.YarnChild: Executing with tokens: [Kind: mapreduce.job, Service: job_1722798834650_0001, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@1fa1cab1)]\n",
"2024-08-04 19:14:54,585 INFO [main] org.apache.hadoop.mapred.YarnChild: Sleeping for 0ms before retrying again. Got null now.\n",
"2024-08-04 19:14:55,264 INFO [main] org.apache.hadoop.mapred.YarnChild: mapreduce.cluster.local.dir for child: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_0/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_1/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001,/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_3/usercache/root/appcache/application_1722798834650_0001\n",
"2024-08-04 19:14:56,558 INFO [main] org.apache.hadoop.mapred.YarnChild: \n",
"/************************************************************\n",
"[system properties]\n",
"os.name: Linux\n",
"os.version: 6.1.85+\n",
"java.home: /usr/lib/jvm/java-11-openjdk-amd64\n",
"java.runtime.version: 11.0.24+8-post-Ubuntu-1ubuntu322.04\n",
"java.vendor: Ubuntu\n",
"java.version: 11.0.24\n",
"java.vm.name: OpenJDK 64-Bit Server VM\n",
"java.class.path: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000007:/content/hadoop-3.4.0/etc/hadoop:/content/hadoop-3.4.0/share/hadoop/common/hadoop-kms-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-common-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-registry-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-nfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/hadoop-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/j2objc-annotations-1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-shaded-guava-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-beanutils-1.9.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsch-0.1.55.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsp-api-2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-io-2.14.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-all-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-cli-1.5.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/bcprov-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/woodstox-core-5.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-servlet-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-http-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-kqueue-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/token-provider-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-crypto-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-logging-1.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-buffer-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-auth-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/guava-27.0-jre.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-xdr-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-proxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/re2j-1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-memcache-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-util-ajax-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-text-1.10.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/metrics-core-3.2.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-unix-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/avro-1.9.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/reload4j-1.2.22.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-configuration2-2.8.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-math3-3.6.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-daemon-1.0.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-core-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-codec-1.15.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-xml-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-lang3-3.12.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/animal-sniffer-annotations-1.17.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jaxb-api-2.2.11.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-client-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jettison-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-config-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-server-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-redis-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-asn1-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-identity-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-udt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-ssl-ocsp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/gson-2.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsr305-3.0.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-framework-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-databind-2.12.7.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-stomp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/httpcore-4.4.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-socks-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/zookeeper-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/slf4j-api-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-annotations-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jackson-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-classes-kqueue-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-webapp-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-admin-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/curator-recipes-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-server-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerby-pkix-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jsr311-api-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jaxb-impl-2.2.3-1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-xml-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/javax.servlet-api-3.1.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-classes-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-io-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-haproxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-http-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jline-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/checker-qual-2.5.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-client-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-simplekdc-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-core-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-util-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-http2-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-collections-3.2.2.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jakarta.activation-api-1.2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-classes-macos-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-rxtx-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/zookeeper-jute-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jetty-security-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-net-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/failureaccess-1.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/commons-compress-1.24.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/stax2-api-4.2.1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-core-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final-linux-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jersey-json-1.20.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/audience-annotations-0.12.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/dnsjava-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/nimbus-jose-jwt-9.31.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-handler-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/kerb-common-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/hadoop-shaded-protobuf_3_21-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/httpclient-4.5.13.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jul-to-slf4j-1.7.36.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-mqtt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-kqueue-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-native-epoll-4.1.100.Final-linux-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-codec-smtp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/snappy-java-1.1.10.4.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/jcip-annotations-1.0-1.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/common/lib/netty-transport-sctp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-native-client-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-httpfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-client-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-rbf-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-nfs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-rbf-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/hadoop-hdfs-native-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/j2objc-annotations-1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-shaded-guava-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-beanutils-1.9.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsch-0.1.55.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-io-2.14.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-all-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-cli-1.5.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/woodstox-core-5.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-servlet-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/json-simple-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-http-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.100.Final-osx-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/token-provider-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-crypto-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-logging-1.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-buffer-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-auth-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/guava-27.0-jre.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-xdr-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-proxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/re2j-1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-memcache-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-util-ajax-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-text-1.10.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/HikariCP-4.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/metrics-core-3.2.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-unix-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/avro-1.9.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/reload4j-1.2.22.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-configuration2-2.8.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-math3-3.6.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-daemon-1.0.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-core-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-codec-1.15.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-xml-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-lang3-3.12.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/animal-sniffer-annotations-1.17.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jaxb-api-2.2.11.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-client-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jettison-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-config-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-server-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-redis-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-asn1-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-identity-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-udt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-ssl-ocsp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/gson-2.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsr305-3.0.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-framework-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-databind-2.12.7.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-stomp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/httpcore-4.4.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-socks-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/zookeeper-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-annotations-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jackson-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-classes-kqueue-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-webapp-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-admin-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/listenablefuture-9999.0-empty-to-avoid-conflict-with-guava.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/curator-recipes-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-server-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerby-pkix-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jsr311-api-1.1.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jaxb-impl-2.2.3-1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-xml-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/javax.servlet-api-3.1.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-classes-epoll-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-io-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-haproxy-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-http-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jline-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/checker-qual-2.5.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-client-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-simplekdc-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-core-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/leveldbjni-all-1.8.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-util-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-http2-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-collections-3.2.2.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jakarta.activation-api-1.2.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-classes-macos-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-rxtx-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/zookeeper-jute-3.8.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jetty-security-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-net-3.9.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-dns-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/failureaccess-1.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/commons-compress-1.24.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/stax2-api-4.2.1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-core-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final-linux-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jersey-json-1.20.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/audience-annotations-0.12.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/dnsjava-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/nimbus-jose-jwt-9.31.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-util-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-handler-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/kerb-common-2.0.3.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/hadoop-shaded-protobuf_3_21-1.2.0.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/httpclient-4.5.13.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-mqtt-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-kqueue-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-native-epoll-4.1.100.Final-linux-x86_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-codec-smtp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/snappy-java-1.1.10.4.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-common-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/jcip-annotations-1.0-1.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-resolver-dns-native-macos-4.1.100.Final-osx-aarch_64.jar:/content/hadoop-3.4.0/share/hadoop/hdfs/lib/netty-transport-sctp-4.1.100.Final.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-services-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-mawo-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-resourcemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-web-proxy-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-router-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-timeline-pluginstorage-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-sharedcachemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-applications-unmanaged-am-launcher-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-client-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-applicationhistoryservice-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-registry-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-api-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-nodemanager-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-globalpolicygenerator-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-services-api-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/hadoop-yarn-server-tests-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jakarta.xml.bind-api-2.3.2.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-plus-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/objenesis-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-module-jaxb-annotations-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/protobuf-java-2.5.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.inject-1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/swagger-annotations-1.5.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jna-5.2.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.websocket-client-api-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/bcpkix-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/snakeyaml-2.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/commons-lang-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/guice-servlet-4.2.3.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/mssql-jdbc-6.2.1.jre7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/aopalliance-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/codemodel-2.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/asm-tree-9.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-client-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-api-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/ehcache-3.3.1.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-client-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/bcutil-jdk15on-1.70.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/fst-2.50.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax-websocket-client-impl-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax.websocket-api-1.0.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/guice-4.2.3.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jsonschema2pojo-core-1.0.2.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-common-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-jndi-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/asm-commons-9.6.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/javax-websocket-server-impl-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jersey-guice-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jersey-client-1.19.4.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-servlet-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jetty-annotations-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-jaxrs-json-provider-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/jackson-jaxrs-base-2.12.7.jar:/content/hadoop-3.4.0/share/hadoop/yarn/lib/websocket-server-9.4.53.v20231009.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-core-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-uploader-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-shuffle-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-common-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/mockito-core-2.28.2.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-app-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-3.4.0-tests.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-nativetask-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/hadoop-mapreduce-client-hs-plugins-3.4.0.jar:/content/hadoop-3.4.0/share/hadoop/mapreduce/lib/*:job.jar/job.jar:job.jar/classes/:job.jar/lib/*:/content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000007/job.jar\n",
"java.io.tmpdir: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000007/tmp\n",
"user.dir: /content/target/test/data/MiniHadoopClusterManager/yarn-857754/MiniHadoopClusterManager-localDir-nm-0_2/usercache/root/appcache/application_1722798834650_0001/container_1722798834650_0001_01_000007\n",
"user.name: root\n",
"************************************************************/\n",
"2024-08-04 19:14:56,559 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: session.id is deprecated. Instead, use dfs.metrics.session-id\n",
"2024-08-04 19:14:57,910 INFO [main] org.apache.hadoop.mapreduce.lib.output.PathOutputCommitterFactory: No output committer factory defined, defaulting to FileOutputCommitterFactory\n",
"2024-08-04 19:14:57,914 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: File Output Committer Algorithm version is 2\n",
"2024-08-04 19:14:57,915 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: FileOutputCommitter skip cleanup _temporary folders under output directory:false, ignore cleanup failures: false\n",
"2024-08-04 19:14:57,986 INFO [main] org.apache.hadoop.mapred.Task: Using ResourceCalculatorProcessTree : [ ]\n",
"2024-08-04 19:14:58,019 INFO [main] org.apache.hadoop.mapred.ReduceTask: Using ShuffleConsumerPlugin: org.apache.hadoop.mapreduce.task.reduce.Shuffle@889d9e8\n",
"2024-08-04 19:14:58,022 WARN [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system already initialized!\n",
"2024-08-04 19:14:58,788 INFO [main] org.apache.hadoop.conf.Configuration.deprecation: mapred.skip.on is deprecated. Instead, use mapreduce.job.skiprecords\n",
"2024-08-04 19:14:59,138 INFO [main] org.apache.hadoop.mapred.Task: Task:attempt_1722798834650_0001_r_000000_0 is done. And is in the process of committing\n",
"2024-08-04 19:14:59,160 INFO [main] org.apache.hadoop.mapred.Task: Task attempt_1722798834650_0001_r_000000_0 is allowed to commit now\n",
"2024-08-04 19:14:59,189 INFO [main] org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter: Saved output of task 'attempt_1722798834650_0001_r_000000_0' to hdfs://localhost:8902/user/root/QuasiMonteCarlo_1722798838891_1650860168/out\n",
"2024-08-04 19:14:59,217 INFO [main] org.apache.hadoop.mapred.Task: Task 'attempt_1722798834650_0001_r_000000_0' done.\n",
"2024-08-04 19:14:59,241 INFO [main] org.apache.hadoop.mapred.Task: Final Counters for attempt_1722798834650_0001_r_000000_0: Counters: 35\n",
"\tFile System Counters\n",
"\t\tFILE: Number of bytes read=116\n",
"\t\tFILE: Number of bytes written=309493\n",
"\t\tFILE: Number of read operations=0\n",
"\t\tFILE: Number of large read operations=0\n",
"\t\tFILE: Number of write operations=0\n",
"\t\tHDFS: Number of bytes read=0\n",
"\t\tHDFS: Number of bytes written=215\n",
"\t\tHDFS: Number of read operations=5\n",
"\t\tHDFS: Number of large read operations=0\n",
"\t\tHDFS: Number of write operations=3\n",
"\t\tHDFS: Number of bytes read erasure-coded=0\n",
"\tMap-Reduce Framework\n",
"\t\tCombine input records=0\n",
"\t\tCombine output records=0\n",
"\t\tReduce input groups=2\n",
"\t\tReduce shuffle bytes=140\n",
"\t\tReduce input records=10\n",
"\t\tReduce output records=0\n",
"\t\tSpilled Records=10\n",
"\t\tShuffled Maps =5\n",
"\t\tFailed Shuffles=0\n",
"\t\tMerged Map outputs=5\n",
"\t\tGC time elapsed (ms)=92\n",
"\t\tCPU time spent (ms)=1050\n",
"\t\tPhysical memory (bytes) snapshot=217210880\n",
"\t\tVirtual memory (bytes) snapshot=2730729472\n",
"\t\tTotal committed heap usage (bytes)=216006656\n",
"\t\tPeak Reduce Physical memory (bytes)=217210880\n",
"\t\tPeak Reduce Virtual memory (bytes)=2730729472\n",
"\tShuffle Errors\n",
"\t\tBAD_ID=0\n",
"\t\tCONNECTION=0\n",
"\t\tIO_ERROR=0\n",
"\t\tWRONG_LENGTH=0\n",
"\t\tWRONG_MAP=0\n",
"\t\tWRONG_REDUCE=0\n",
"\tFile Output Format Counters \n",
"\t\tBytes Written=97\n",
"2024-08-04 19:14:59,243 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Stopping ReduceTask metrics system...\n",
"2024-08-04 19:14:59,243 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system stopped.\n",
"2024-08-04 19:14:59,243 INFO [main] org.apache.hadoop.metrics2.impl.MetricsSystemImpl: ReduceTask metrics system shutdown complete.\n",
"\n",
"End of LogType:syslog\n",
"***********************************************************************\n",
"\n",
"Container: container_1722798834650_0001_01_000007 on localhost_34535\n",
"LogAggregationType: AGGREGATED\n",
"====================================================================\n",
"LogType:syslog.shuffle\n",
"LogLastModifiedTime:Sun Aug 04 19:15:07 +0000 2024\n",
"LogLength:4984\n",
"LogContents:\n",
"2024-08-04 19:14:58,045 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: MergerManager: memoryLimit=601882624, maxSingleShuffleLimit=150470656, mergeThreshold=397242560, ioSortFactor=10, memToMemMergeOutputsThreshold=10\n",
"2024-08-04 19:14:58,048 INFO [EventFetcher for fetching Map Completion Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_1722798834650_0001_r_000000_0 Thread started: EventFetcher for fetching Map Completion Events\n",
"2024-08-04 19:14:58,069 INFO [EventFetcher for fetching Map Completion Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher: attempt_1722798834650_0001_r_000000_0: Got 5 new map-outputs\n",
"2024-08-04 19:14:58,348 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1722798834650_0001_m_000001_0 decomp: 24 len: 28 to MEMORY\n",
"2024-08-04 19:14:58,359 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_1722798834650_0001_m_000001_0\n",
"2024-08-04 19:14:58,362 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 1, commitMemory -> 0, usedMemory ->24\n",
"2024-08-04 19:14:58,370 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1722798834650_0001_m_000000_0 decomp: 24 len: 28 to MEMORY\n",
"2024-08-04 19:14:58,370 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_1722798834650_0001_m_000000_0\n",
"2024-08-04 19:14:58,372 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 2, commitMemory -> 24, usedMemory ->48\n",
"2024-08-04 19:14:58,374 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1722798834650_0001_m_000003_0 decomp: 24 len: 28 to MEMORY\n",
"2024-08-04 19:14:58,375 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_1722798834650_0001_m_000003_0\n",
"2024-08-04 19:14:58,375 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 3, commitMemory -> 48, usedMemory ->72\n",
"2024-08-04 19:14:58,378 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1722798834650_0001_m_000002_0 decomp: 24 len: 28 to MEMORY\n",
"2024-08-04 19:14:58,378 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_1722798834650_0001_m_000002_0\n",
"2024-08-04 19:14:58,378 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 4, commitMemory -> 72, usedMemory ->96\n",
"2024-08-04 19:14:58,379 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.Fetcher: fetcher#1 about to shuffle output of map attempt_1722798834650_0001_m_000004_0 decomp: 24 len: 28 to MEMORY\n",
"2024-08-04 19:14:58,379 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.InMemoryMapOutput: Read 24 bytes from map-output for attempt_1722798834650_0001_m_000004_0\n",
"2024-08-04 19:14:58,379 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: closeInMemoryFile -> map-output of size: 24, inMemoryMapOutputs.size() -> 5, commitMemory -> 96, usedMemory ->120\n",
"2024-08-04 19:14:58,381 INFO [EventFetcher for fetching Map Completion Events] org.apache.hadoop.mapreduce.task.reduce.EventFetcher: EventFetcher is interrupted.. Returning\n",
"2024-08-04 19:14:58,381 INFO [fetcher#1] org.apache.hadoop.mapreduce.task.reduce.ShuffleSchedulerImpl: localhost:43847 freed by fetcher#1 in 312ms\n",
"2024-08-04 19:14:58,394 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: finalMerge called with 5 in-memory map-outputs and 0 on-disk map-outputs\n",
"2024-08-04 19:14:58,423 INFO [main] org.apache.hadoop.mapred.Merger: Merging 5 sorted segments\n",
"2024-08-04 19:14:58,423 INFO [main] org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 5 segments left of total size: 105 bytes\n",
"2024-08-04 19:14:58,440 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merged 5 segments, 120 bytes to disk to satisfy reduce memory limit\n",
"2024-08-04 19:14:58,447 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 1 files, 116 bytes from disk\n",
"2024-08-04 19:14:58,448 INFO [main] org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl: Merging 0 segments, 0 bytes from memory into reduce\n",
"2024-08-04 19:14:58,448 INFO [main] org.apache.hadoop.mapred.Merger: Merging 1 sorted segments\n",
"2024-08-04 19:14:58,486 INFO [main] org.apache.hadoop.mapred.Merger: Down to the last merge-pass, with 1 segments left of total size: 109 bytes\n",
"\n",
"End of LogType:syslog.shuffle\n",
"*******************************************************************************\n",
"\n"
]
},
{
"output_type": "stream",
"name": "stderr",
"text": [
"2024-08-04 19:15:45,913 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at localhost/127.0.0.1:8903\n"
]
}
]
},
{
"cell_type": "code",
"source": [],
"metadata": {
"id": "tiDceHeBGInO"
},
"execution_count": 81,
"outputs": []
}
]
}
![\"Open](\"https://colab.research.google.com/assets/colab-badge.svg\"){kind=icon}
![](\"https://github.com/groda/big_data/blob/master/logo_bdb.png?raw=true\"){kind=logo}