{ "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "provenance": [], "authorship_tag": "ABX9TyN7KrJ5aFUDzz8ZbDcCHGEo", "include_colab_link": true }, "kernelspec": { "name": "python3", "display_name": "Python 3" }, "language_info": { "name": "python" } }, "cells": [ { "cell_type": "markdown", "metadata": { "id": "view-in-github", "colab_type": "text" }, "source": [ "\"Open" ] }, { "cell_type": "markdown", "source": [ "# Intro\n", "\n", "This notebook demonstrates an example of how to use [the Primavera P6 xer files reader](https://pypi.org/project/xerparser/0.10.3/) library for data preparation.\n", "\n", "\n", "The XER file contains the same data presented in the [notebook](https://nbviewer.org/github/inigmat/exupery/blob/main/Schedule_CPLEX.ipynb) which is shown in the [\"Optimizing Project Planning and Material transportation\"](https://inigmat.github.io/home/2023/11/29/Transport-and-Resources.html) post.\n", "\n", "By obtaining the file via URL link, the required data—task names, durations, precedences, and release dates (intervals between each house's earliest starting date)—is transformed into Python objects.\n" ], "metadata": { "id": "BhG4x-GqoR_3" } }, { "cell_type": "markdown", "source": [ "# Installing xerparser\n" ], "metadata": { "id": "EYEH5p-0oF4Z" } }, { "cell_type": "code", "source": [ "import importlib\n", "\n", "# Check if xerparser is already installed\n", "try:\n", " importlib.import_module('xerparser')\n", " print(\"xerparser is already installed.\")\n", "except ImportError:\n", " try:\n", " # Attempt installation\n", " %pip install xerparser\n", " except Exception as e:\n", " print(\"An error occurred while installing xerparser:\", e)" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "gbiCgG_RUiY0", "outputId": "bb1d4cc0-73ad-410f-ee4c-0fe89d11d7a2" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "xerparser is already installed.\n" ] } ] }, { "cell_type": "markdown", "source": [ "# Getting the XER file" ], "metadata": { "id": "sya1fu_QqTgW" } }, { "cell_type": "code", "source": [ "import requests\n", "\n", "url = \"https://raw.githubusercontent.com/inigmat/exupery/main/files/schedule/MDL4D.xer\"\n", "\n", "try:\n", " response = requests.get(url)\n", " if response.status_code == 200:\n", " print(\"Request successful!\")\n", " else:\n", " print(f\"Failed to retrieve data. Status code: {response.status_code}\")\n", "except requests.exceptions.RequestException as e:\n", " print(f\"An error occurred: {e}\")" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "sjlr83pETrsi", "outputId": "9559e561-2f45-496c-9ab3-a0276502c925" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "Request successful!\n" ] } ] }, { "cell_type": "markdown", "source": [ "# Read the downloaded file with checking for errors" ], "metadata": { "id": "9nLeMn863_fl" } }, { "cell_type": "code", "source": [ "from xerparser import Xer, CorruptXerFile\n", "\n", "try:\n", " xer = Xer(response.text)\n", "except CorruptXerFile as e:\n", " for error in e.errors:\n", " print(error)" ], "metadata": { "id": "-G_0Dr4lJhul" }, "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "source": [ "# Getting the project data" ], "metadata": { "id": "2qKhVOAF37Yl" } }, { "cell_type": "code", "source": [ "project = list(xer.projects.values())[0]" ], "metadata": { "id": "q1zW1dI7fxJT" }, "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "source": [ "We have to get the following data\n", "\n", "```\n", "NbWorkers = 3\n", "NbHouses = 5\n", "\n", "TaskNames = (\"masonry\",\"carpentry\",\"plumbing\",\n", " \"ceiling\",\"roofing\",\"painting\",\n", " \"windows\",\"facade\",\"garden\",\"moving\")\n", "\n", "Duration = [35, 15, 40, 15, 5, 10, 5, 10, 5, 5]\n", "\n", "ReleaseDate = [31, 0, 90, 120, 90]\n", "\n", "Precedences = [(\"masonry\", \"carpentry\"), (\"masonry\", \"plumbing\"), (\"masonry\", \"ceiling\"),\n", " (\"carpentry\", \"roofing\"), (\"ceiling\", \"painting\"), (\"roofing\", \"windows\"),\n", " (\"roofing\", \"facade\"), (\"plumbing\", \"facade\"), (\"roofing\", \"garden\"),\n", " (\"plumbing\", \"garden\"), (\"windows\", \"moving\"), (\"facade\", \"moving\"),\n", " (\"garden\", \"moving\"), (\"painting\", \"moving\")]\n", "```\n", "\n" ], "metadata": { "id": "WwED080mhUzT" } }, { "cell_type": "markdown", "source": [ "# Start with getting the WBS (house numbers)\n", "\n", "We skip the first two levels using WBS_LVL variable" ], "metadata": { "id": "9ECY5LtO4l-F" } }, { "cell_type": "code", "source": [ "WBS_LVL = 2\n", "houses = [obj.name for obj in project.wbs_nodes[WBS_LVL:]]\n", "NbHouses= len(houses)\n", "houses" ], "metadata": { "id": "h_slxPqjRvWH", "colab": { "base_uri": "https://localhost:8080/" }, "outputId": "2db66196-9c1f-4646-e4e4-a9f8791ea424" }, "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "['House 1', 'House 2', 'House 3', 'House 4', 'House 5']" ] }, "metadata": {}, "execution_count": 5 } ] }, { "cell_type": "markdown", "source": [ "# Obtain the labor resources of the project to get the number of workers" ], "metadata": { "id": "mnuxHbRR5mqB" } }, { "cell_type": "code", "source": [ "workers = {}\n", "for res in project.resources:\n", " if res.rsrc_type == \"RT_Labor\":\n", " workers[res.rsrc_id] = res.resource.name\n", "NbWorkers = len(workers)" ], "metadata": { "id": "spxHZUTpareG" }, "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "source": [ "# Prepare pandas dataframe to check out the tasks data such as 'TASK ID','Name','Type', 'WBS ID', 'Duration', 'Successors (ID, Link, Lag)'" ], "metadata": { "id": "RFsILb_753_3" } }, { "cell_type": "code", "source": [ "import pandas as pd\n", "\n", "tasks_df = pd.DataFrame(columns=['TASK ID','Name','Type', 'WBS ID', 'Duration', 'Successors (ID, Link, Lag)'])\n", "\n", "for task in project.tasks:\n", " task_sucs = []\n", " for pred_link in task.successors:\n", " task_suc = pred_link.task.uid\n", " task_link = pred_link.link\n", " task_lag = pred_link.lag\n", " task_sucs.append((task_suc, task_link, task_lag))\n", "\n", " tasks_df = pd.concat([tasks_df, pd.DataFrame(\n", " {'TASK ID': [task.uid],\n", " 'Name': [task.name],\n", " 'Type': [task.type],\n", " 'WBS ID': [task.wbs_id],\n", " 'Duration': [task.duration],\n", " 'Successors (ID, Link, Lag)': [task_sucs],\n", " })], ignore_index=True)" ], "metadata": { "id": "SdzXOiYMmBiK" }, "execution_count": null, "outputs": [] }, { "cell_type": "code", "source": [ "tasks_df.head(15)" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 514 }, "id": "dAkhLXxm5ph6", "outputId": "36199cdb-a44e-4d67-d913-35478941617c" }, "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ " TASK ID Name Type WBS ID Duration \\\n", "0 104836 House 1 Start TaskType.TT_Mile 26152 0 \n", "1 104837 Masonry TaskType.TT_Task 26154 35 \n", "2 104838 Carpentry TaskType.TT_Task 26154 15 \n", "3 104839 Plumbing TaskType.TT_Task 26154 40 \n", "4 104840 Ceiling TaskType.TT_Task 26154 15 \n", "5 104841 Roofing TaskType.TT_Task 26154 5 \n", "6 104842 Painting TaskType.TT_Task 26154 10 \n", "7 104843 Windows TaskType.TT_Task 26154 5 \n", "8 104844 Facade TaskType.TT_Task 26154 10 \n", "9 104845 Garden TaskType.TT_Task 26154 5 \n", "10 104846 Moving TaskType.TT_Rsrc 26154 5 \n", "11 104847 House 2 Start TaskType.TT_Mile 26152 0 \n", "12 104848 Masonry TaskType.TT_Task 26155 35 \n", "13 104849 Carpentry TaskType.TT_Task 26155 15 \n", "14 104850 Plumbing TaskType.TT_Task 26155 40 \n", "\n", " Successors (ID, Link, Lag) \n", "0 [] \n", "1 [(104838, FS, 0), (104839, FS, 0), (104840, FS... \n", "2 [(104841, FS, 0)] \n", "3 [(104844, FS, 0), (104845, FS, 0)] \n", "4 [(104842, FS, 0)] \n", "5 [(104843, FS, 0), (104844, FS, 0), (104845, FS... \n", "6 [(104846, FS, 0)] \n", "7 [(104846, FS, 0)] \n", "8 [(104846, FS, 0)] \n", "9 [(104846, FS, 0)] \n", "10 [(104883, FF, 0)] \n", "11 [] \n", "12 [(104849, FS, 0), (104850, FS, 0), (104851, FS... \n", "13 [(104852, FS, 0)] \n", "14 [(104855, FS, 0), (104856, FS, 0)] " ], "text/html": [ "\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
TASK IDNameTypeWBS IDDurationSuccessors (ID, Link, Lag)
0104836House 1 StartTaskType.TT_Mile261520[]
1104837MasonryTaskType.TT_Task2615435[(104838, FS, 0), (104839, FS, 0), (104840, FS...
2104838CarpentryTaskType.TT_Task2615415[(104841, FS, 0)]
3104839PlumbingTaskType.TT_Task2615440[(104844, FS, 0), (104845, FS, 0)]
4104840CeilingTaskType.TT_Task2615415[(104842, FS, 0)]
5104841RoofingTaskType.TT_Task261545[(104843, FS, 0), (104844, FS, 0), (104845, FS...
6104842PaintingTaskType.TT_Task2615410[(104846, FS, 0)]
7104843WindowsTaskType.TT_Task261545[(104846, FS, 0)]
8104844FacadeTaskType.TT_Task2615410[(104846, FS, 0)]
9104845GardenTaskType.TT_Task261545[(104846, FS, 0)]
10104846MovingTaskType.TT_Rsrc261545[(104883, FF, 0)]
11104847House 2 StartTaskType.TT_Mile261520[]
12104848MasonryTaskType.TT_Task2615535[(104849, FS, 0), (104850, FS, 0), (104851, FS...
13104849CarpentryTaskType.TT_Task2615515[(104852, FS, 0)]
14104850PlumbingTaskType.TT_Task2615540[(104855, FS, 0), (104856, FS, 0)]
\n", "
\n", "
\n", "\n", "
\n", " \n", "\n", " \n", "\n", " \n", "
\n", "\n", "\n", "
\n", " \n", "\n", "\n", "\n", " \n", "
\n", "
\n", "
\n" ], "application/vnd.google.colaboratory.intrinsic+json": { "type": "dataframe", "variable_name": "tasks_df", "repr_error": "'str' object has no attribute 'empty'" } }, "metadata": {}, "execution_count": 8 } ] }, { "cell_type": "markdown", "source": [ "# Get the list of task names. Filter by wbs_id due to the repeating structure of activities on each house" ], "metadata": { "id": "-1uKFFLf6XkD" } }, { "cell_type": "code", "source": [ "def get_task_dict_by_wbs_id(df, wbs_id):\n", " filtered_df = df[df['WBS ID'] == wbs_id]\n", " task_dict = dict(zip(filtered_df['TASK ID'], filtered_df['Name']))\n", " return task_dict\n", "\n", "wbs_id = '26154' # wbs_id of activites on House 1 (taken from the dataframe above)\n", "task_dict = get_task_dict_by_wbs_id(tasks_df, wbs_id)\n", "TaskNames = tuple(task_dict.values())" ], "metadata": { "id": "z7wTzw6lwNgf" }, "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "source": [ "# Get the tasks duration" ], "metadata": { "id": "xroCud946zAJ" } }, { "cell_type": "code", "source": [ "durations = {task_id: tasks_df.loc[tasks_df['TASK ID'] == task_id, 'Duration'].values[0] for task_id in task_dict.keys()}\n", "Duration = list(durations.values())" ], "metadata": { "id": "-2B234fTxuY7" }, "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "source": [ "# Get the precedences of the tasks" ], "metadata": { "id": "jJgBMctH7HBQ" } }, { "cell_type": "code", "source": [ "prec = []\n", "for task in project.tasks:\n", " if task.uid in task_dict:\n", " for pred_link in task.successors:\n", " suc_uid = pred_link.task.uid\n", " if suc_uid in task_dict:\n", " suc_name = pred_link.task.name\n", " prec.append((task.name,suc_name))\n", "Precedences = prec" ], "metadata": { "id": "DPSmJMX1ytUt" }, "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "source": [ "# Get the release dates. Dates ordered by the house number" ], "metadata": { "id": "VD3uCInt7T9f" } }, { "cell_type": "code", "source": [ "lags = {}\n", "for rel in project.relationships:\n", " if str(rel.predecessor.type) == 'TaskType.TT_Mile':\n", " lags[rel.successor.wbs.name] = rel.lag\n", "lags" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "vn_Kgd9W8pND", "outputId": "1154f128-88b1-48b6-f551-6afcccc4f76c" }, "execution_count": null, "outputs": [ { "output_type": "execute_result", "data": { "text/plain": [ "{'House 2': 0, 'House 1': 31, 'House 3': 90, 'House 4': 120, 'House 5': 90}" ] }, "metadata": {}, "execution_count": 12 } ] }, { "cell_type": "code", "source": [ "ReleaseDate_dict = {key: lags.get(key) for key in sorted(lags)}\n", "ReleaseDate = list(ReleaseDate_dict.values())" ], "metadata": { "id": "kTx8RVxqnOZ8" }, "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "source": [ "# Finally obtain the project data" ], "metadata": { "id": "V4e2bEzQ7sgp" } }, { "cell_type": "code", "source": [ "print(\"NbWorkers =\", NbWorkers, \"\\n\")\n", "print(\"NbHouses =\", NbHouses, \"\\n\")\n", "print(\"TaskNames =\", TaskNames, \"\\n\")\n", "print(\"Duration =\", Duration, \"\\n\")\n", "print(\"ReleaseDate =\", ReleaseDate, \"\\n\")\n", "print(\"Precedences =\", Precedences, \"\\n\")" ], "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "p7aKiCvMNF8Z", "outputId": "2d0f3c82-6208-4328-ffd2-c5c30868fd2b" }, "execution_count": null, "outputs": [ { "output_type": "stream", "name": "stdout", "text": [ "NbWorkers = 3 \n", "\n", "NbHouses = 5 \n", "\n", "TaskNames = ('Masonry', 'Carpentry', 'Plumbing', 'Ceiling', 'Roofing', 'Painting', 'Windows', 'Facade', 'Garden', 'Moving') \n", "\n", "Duration = [35, 15, 40, 15, 5, 10, 5, 10, 5, 5] \n", "\n", "ReleaseDate = [31, 0, 90, 120, 90] \n", "\n", "Precedences = [('Masonry', 'Carpentry'), ('Masonry', 'Plumbing'), ('Masonry', 'Ceiling'), ('Carpentry', 'Roofing'), ('Plumbing', 'Facade'), ('Plumbing', 'Garden'), ('Ceiling', 'Painting'), ('Roofing', 'Windows'), ('Roofing', 'Facade'), ('Roofing', 'Garden'), ('Painting', 'Moving'), ('Windows', 'Moving'), ('Facade', 'Moving'), ('Garden', 'Moving')] \n", "\n" ] } ] } ] }