{ "cells": [ { "cell_type": "markdown", "id": "032fbb8b", "metadata": {}, "source": [ "# Copilot adoption journey and ways-of-working analysis\n", "\n", "This end-to-end companion to `copilot-analytics-examples.ipynb` uses a Viva Insights Person Query\n", "to move from basic Copilot metrics to an adoption journey and ways-of-working assessment.\n", "It answers four practical questions:\n", "\n", "1. **Reach:** how many people have Copilot, and how that has changed over time.\n", "2. **Habit:** how many people show sustained use, emerging use, or no use, using Power\n", " and Habitual User status as the notebook's proxy for *stickiness* - durable, repeated\n", " return usage rather than a one-off trial.\n", "3. **Opportunity:** where adoption differs across managers, individual contributors, and functions.\n", "4. **Ways of working:** how sustained use is associated with collaboration load, meetings,\n", " focus, multitasking, and after-hours work.\n", "\n", "The analysis is observational. It describes associations and adoption patterns; it does not\n", "claim that Copilot caused the working-pattern differences. If you need to estimate a causal\n", "effect, see the [Causal Inference in Copilot Analytics](https://microsoft.github.io/viva-insights-sample-code/causal-inference/)\n", "page and the [Copilot Causal Toolkit](https://microsoft.github.io/viva-insights-sample-code/copilot-causal-toolkit/).\n", "\n", "## Which notebook should I use?\n", "\n", "| Notebook | Use it when |\n", "| --- | --- |\n", "| [`copilot-analytics-examples.ipynb`](https://github.com/microsoft/viva-insights-sample-code/blob/main/examples/utility-python/copilot-analytics-examples.ipynb) | You want a guided tour of the core Copilot metrics and the standard **vivainsights** visuals, and you are getting oriented in the data. |\n", "| **This notebook** | You have at least 12 weeks of Copilot data and need an end-to-end adoption assessment: cohorts, habit formation, conversion targeting, and associated ways of working. |\n", "\n", "## Before you run it\n", "\n", "The notebook is organization-agnostic: it makes no assumptions about a specific\n", "organization's structure, headcount, or function names. Set `INPUT_FILE` in the\n", "configuration cell to your own Person Query export, in CSV or Parquet format.\n", "\n", "The export should contain:\n", "\n", "- at least 12 weeks of Copilot activity, so that the rolling habit window can be evaluated;\n", "- the collaboration and work-pattern metrics used by the diagnostic sections. Any metric\n", " that is absent is skipped automatically rather than causing an error; and\n", "- ideally one or more organizational attributes, such as `FunctionType` or `Organization`,\n", " `IsManager` or `SupervisorIndicator`, and `LevelDesignation` or `Level`.\n", "\n", "Segment definitions follow the standard Copilot Usage Segments. For the background on why\n", "the segments are defined the way they are, including the 9-of-12-weeks rationale and what\n", "to do with fewer than three months of data, see the\n", "[Copilot Usage Segments](https://microsoft.github.io/viva-insights-sample-code/copilot-usage-segments/) page.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "3f386f0f", "metadata": { "execution": { "iopub.execute_input": "2026-08-05T14:21:46.454725Z", "iopub.status.busy": "2026-08-05T14:21:46.454436Z", "iopub.status.idle": "2026-08-05T14:21:46.462344Z", "shell.execute_reply": "2026-08-05T14:21:46.461717Z" } }, "outputs": [], "source": [ "from pathlib import Path\n", "\n", "# ---------------------------------------------------------------------------\n", "# Input. Replace this with your own Viva Insights Person Query export.\n", "# Both .csv and .parquet are supported. The export needs at least 12 weeks of\n", "# Copilot activity for the rolling habit window to be meaningful.\n", "# ---------------------------------------------------------------------------\n", "INPUT_FILE = Path(\"person_query.csv\")\n", "OUTPUT_DIR = Path(\"outputs/copilot-adoption-journey\")\n", "\n", "# ---------------------------------------------------------------------------\n", "# Privacy and group-size floors.\n", "# MIN_PRIVACY_N is the hard floor: no group defined by an organizational\n", "# attribute, journey stage, or entry cohort is reported below this size.\n", "# MIN_DISPLAY_N is the higher bar used for group comparisons, so that rankings\n", "# are not driven by very small teams.\n", "# ---------------------------------------------------------------------------\n", "MIN_PRIVACY_N = 5\n", "MIN_DISPLAY_N = 30\n", "\n", "# ---------------------------------------------------------------------------\n", "# Usage-segment parameters. These are the standard 12-week Copilot Usage Segment\n", "# settings, passed explicitly rather than through the \"12w\" preset so that\n", "# changing POWER_THRESHOLD actually changes the segmentation. See\n", "# https://microsoft.github.io/viva-insights-sample-code/copilot-usage-segments/\n", "# ---------------------------------------------------------------------------\n", "SEGMENT_WINDOW_WEEKS = 12 # max_window: length of the rolling window\n", "SEGMENT_HABIT_WEEKS = 9 # width: active weeks required within the window\n", "SEGMENT_ACTION_THRESHOLD = 1 # threshold: actions that make a week \"active\"\n", "POWER_THRESHOLD = 15 # power_thres: mean weekly actions for Power User\n", "RECENT_WEEKS = 4 # trailing window for the process-ratio diagnostics\n", "\n", "# Smallest standardised effect worth calling material. With tens of thousands of\n", "# person-weeks, a coefficient can clear the 95% significance bar while being far too\n", "# small to act on, so the diagnostics below judge results on size as well as\n", "# significance. 0.1 standard deviations is the conventional floor for a small effect.\n", "MATERIAL_EFFECT_SD = 0.1\n", "\n", "# ---------------------------------------------------------------------------\n", "# Organizational attributes. The notebook looks for each attribute in order and\n", "# uses the first one present in the export, so that it works with the different\n", "# attribute sets that different tenants configure. Add your own column names\n", "# here if they differ.\n", "# ---------------------------------------------------------------------------\n", "FUNCTION_COLUMNS = [\"FunctionType\", \"Organization\", \"Department\"]\n", "MANAGER_COLUMNS = [\"IsManager\", \"SupervisorIndicator\", \"ManagerIndicator\"]\n", "LEVEL_COLUMNS = [\"LevelDesignation\", \"Level\"]\n", "LOCATION_COLUMNS = [\"Location\", \"Region\"]\n", "LICENSE_COLUMNS = [\"Copilot_enabled_days\", \"Total_Copilot_enabled_days\"]\n", "\n", "# Functions at or above this Power + Habitual adoption percentage are surfaced as a\n", "# replication playbook. Lower it if no group currently qualifies.\n", "LEADING_FUNCTION_THRESHOLD_PCT = 50.0\n", "\n", "# Optional: location values known to be data artifacts, for example a registered or\n", "# mailing address used as a default rather than a genuine work site. Excluded only\n", "# from the Location cut, so people keep their function and manager rows. Leave this\n", "# empty unless such an artifact has been confirmed in your own data.\n", "LOCATION_EXCLUDE = []\n", "\n", "OUTPUT_DIR.mkdir(parents=True, exist_ok=True)\n", "print(f\"Input: {INPUT_FILE}\")\n", "print(f\"Output: {OUTPUT_DIR.resolve()}\")\n" ] }, { "cell_type": "markdown", "id": "bdbf460c", "metadata": {}, "source": [ "## 1. Method and interpretation guardrails\n", "\n", "- One row should represent one person-week.\n", "- Blank rows are removed before analysis.\n", "- The **Copilot population** is the set of people who have Copilot in a given week, and it\n", " is the denominator for reach and activation. It is *confirmed* from an enabled-days\n", " column when the export has one, and otherwise *estimated* from the presence of Copilot\n", " telemetry. Being in it is not the same as using Copilot, which is measured separately.\n", "- Usage segmentation uses the standard **12-week rolling definition**, documented on the\n", " [Copilot Usage Segments](https://microsoft.github.io/viva-insights-sample-code/copilot-usage-segments/#formal-definitions)\n", " page and implemented by\n", " [`identify_usage_segments()`](https://microsoft.github.io/vivainsights/reference/identify_usage_segments.html).\n", "- The first weeks of any panel have incomplete rolling histories. People who joined the\n", " Copilot population later may not yet have enough weeks to qualify as Habitual or Power\n", " Users. If you have fewer than three months of data, consider the\n", " [4-week variation](https://microsoft.github.io/viva-insights-sample-code/copilot-usage-segments/)\n", " rather than the 12-week definition used here.\n", "- Groups smaller than `MIN_DISPLAY_N` (30 by default) are excluded from group\n", " comparisons, so that rankings are not driven by very small teams. Below that,\n", " `MIN_PRIVACY_N` (5 by default) is a hard floor: no group defined by an organizational\n", " attribute, journey stage, or entry cohort is reported beneath it. Whole-population\n", " totals, such as the count of people in each usage segment, are not group breakdowns\n", " and are reported in full.\n", "- Adjusted comparisons control for function and manager status when those attributes are\n", " available, but cannot remove all role or seniority differences.\n", "- Every collaboration metric used below is optional. Whatever is missing from your export\n", " is reported as unavailable and skipped, rather than causing the notebook to fail." ] }, { "cell_type": "code", "execution_count": null, "id": "ac263663", "metadata": { "execution": { "iopub.execute_input": "2026-08-05T14:21:46.463895Z", "iopub.status.busy": "2026-08-05T14:21:46.463707Z", "iopub.status.idle": "2026-08-05T14:21:53.690019Z", "shell.execute_reply": "2026-08-05T14:21:53.688420Z" } }, "outputs": [], "source": [ "import re\n", "import warnings\n", "\n", "import matplotlib.pyplot as plt\n", "import matplotlib.ticker as mticker\n", "import numpy as np\n", "import pandas as pd\n", "from scipy import stats\n", "import statsmodels.api as sm\n", "import vivainsights as vi\n", "\n", "warnings.filterwarnings(\"ignore\", category=FutureWarning)\n", "pd.set_option(\"display.width\", 180)\n", "pd.set_option(\"display.max_columns\", 80)\n", "\n", "C_NAVY = \"#16324F\"\n", "C_BLUE = \"#2F6B9A\"\n", "C_TEAL = \"#2A7F83\"\n", "C_GOLD = \"#B9892D\"\n", "C_RED = \"#A4473D\"\n", "C_GREY = \"#7A8288\"\n", "C_LIGHT = \"#E8EDF1\"\n", "C_TEXT = \"#202428\"\n", "\n", "plt.rcParams.update({\n", " \"font.family\": [\"Segoe UI\", \"DejaVu Sans\", \"sans-serif\"],\n", " \"font.size\": 10.5,\n", " \"axes.spines.top\": False,\n", " \"axes.spines.right\": False,\n", " \"axes.grid\": True,\n", " \"grid.alpha\": 0.25,\n", " \"figure.facecolor\": \"white\",\n", " \"axes.facecolor\": \"white\",\n", " \"text.color\": C_TEXT,\n", " \"figure.dpi\": 120,\n", "})\n", "\n", "TABLES = {}\n", "FIGURES = {}\n", "\n", "\n", "def save_table(frame, name):\n", " frame.to_csv(OUTPUT_DIR / f\"{name}.csv\", index=False)\n", " TABLES[name] = frame.copy()\n", " return frame\n", "\n", "\n", "def save_figure(fig, name):\n", " fig.savefig(OUTPUT_DIR / f\"{name}.png\", dpi=180, bbox_inches=\"tight\", facecolor=\"white\")\n", " FIGURES[name] = fig\n", " return fig\n", "\n", "\n", "def add_subtitle(ax, text):\n", " \"\"\"Add a small italic subtitle stating the exact metric shown, directly under an\n", " axis title. Charts without an explicit metric named in the title read ambiguously\n", " once separated from the surrounding narrative text (e.g. on a slide); this keeps\n", " the metric definition attached to the chart itself. The offset is in points rather\n", " than axes fractions so that it stays anchored to the title on tall axes.\"\"\"\n", " ax.annotate(text, xy=(0.0, 1.0), xycoords=\"axes fraction\",\n", " xytext=(0, 20), textcoords=\"offset points\",\n", " fontsize=8.5, style=\"italic\", color=C_GREY, ha=\"left\", va=\"bottom\")\n", "\n", "\n", "def pct(num, den):\n", " return np.nan if not den else 100 * num / den\n", "\n", "\n", "def safe_ratio(num, den):\n", " \"\"\"Percentage that returns NaN instead of inf or a divide-by-zero warning.\n", "\n", " Accepts a scalar or a Series for either argument, so that it can be used both for\n", " element-wise ratios and for \"share of total\" calculations.\n", " \"\"\"\n", " num = pd.Series(num) if not np.isscalar(num) else num\n", " if np.isscalar(den):\n", " den = np.nan if not den else den\n", " return 100 * num / den\n", " den = pd.Series(den)\n", " return 100 * num / den.where(den > 0)\n", "\n", "\n", "def lookup(frame, row, column, default=np.nan):\n", " \"\"\"Read frame.loc[row, column], returning `default` when either is absent.\n", "\n", " Segments, metrics, and organizational groups are all optional in this notebook:\n", " a segment can fall below the minimum group size, and a metric can be missing from\n", " the export entirely. Reading through this helper keeps the narrative sections\n", " degrading gracefully instead of raising KeyError partway through a run.\n", " \"\"\"\n", " if frame is None or column not in getattr(frame, \"columns\", []):\n", " return default\n", " if row not in frame.index:\n", " return default\n", " value = frame.loc[row, column]\n", " if isinstance(value, pd.Series):\n", " value = value.iloc[0]\n", " return default if pd.isna(value) else value\n", "\n", "\n", "def fmt(value, spec=\"{:.1f}\", missing=\"not available\"):\n", " \"\"\"Format a number for narrative text, or say so plainly when it is missing.\"\"\"\n", " return missing if value is None or pd.isna(value) else spec.format(value)\n", "\n", "\n", "def first_available(frame, candidates):\n", " \"\"\"Return the first candidate column present in `frame`, otherwise None.\"\"\"\n", " for name in candidates:\n", " if name in frame.columns:\n", " return name\n", " return None\n", "\n", "\n", "def normalise_column(name):\n", " return re.sub(r\"[^0-9A-Za-z]+\", \"_\", str(name)).strip(\"_\")" ] }, { "cell_type": "code", "execution_count": null, "id": "30865ed1", "metadata": { "execution": { "iopub.execute_input": "2026-08-05T14:21:53.692825Z", "iopub.status.busy": "2026-08-05T14:21:53.692373Z", "iopub.status.idle": "2026-08-05T14:21:54.784582Z", "shell.execute_reply": "2026-08-05T14:21:54.783433Z" } }, "outputs": [], "source": [ "if INPUT_FILE.suffix.lower() in {\".parquet\", \".pq\"}:\n", " raw = pd.read_parquet(INPUT_FILE)\n", "else:\n", " # import_query() reads a Person Query CSV and cleans the column names.\n", " raw = vi.import_query(str(INPUT_FILE))\n", "\n", "raw_rows = len(raw)\n", "raw.columns = [normalise_column(c) for c in raw.columns]\n", "\n", "action_columns = [\n", " col for col in raw.columns\n", " if col.startswith(\"Copilot_actions_taken_in\")\n", "]\n", "if \"Total_Copilot_actions_taken\" not in raw.columns and action_columns:\n", " raw[action_columns] = raw[action_columns].fillna(0)\n", " raw[\"Total_Copilot_actions_taken\"] = raw[action_columns].sum(axis=1)\n", "\n", "required = {\"PersonId\", \"MetricDate\", \"Total_Copilot_actions_taken\",\n", " \"Total_Copilot_active_days\"}\n", "missing = sorted(required - set(raw.columns))\n", "if missing:\n", " raise ValueError(\n", " f\"Missing required columns: {missing}. This notebook needs a Person Query \"\n", " \"export containing Copilot activity metrics.\"\n", " )\n", "\n", "df = raw.dropna(subset=[\"PersonId\", \"MetricDate\"]).copy()\n", "df[\"MetricDate\"] = pd.to_datetime(df[\"MetricDate\"], errors=\"coerce\")\n", "df = df.dropna(subset=[\"MetricDate\"])\n", "\n", "# Resolve the organizational attributes from whichever columns this export provides.\n", "attribute_sources = {}\n", "for target, candidates in [\n", " (\"FunctionType\", FUNCTION_COLUMNS),\n", " (\"ManagerSource\", MANAGER_COLUMNS),\n", " (\"LevelDesignation\", LEVEL_COLUMNS),\n", " (\"Location\", LOCATION_COLUMNS),\n", "]:\n", " source = first_available(df, [normalise_column(c) for c in candidates])\n", " attribute_sources[target] = source\n", " if source is None:\n", " df[target] = np.nan\n", " elif source != target:\n", " df[target] = df[source]\n", "\n", "bad_text = {\"\", \"NA\", \"NULL\", \"#N/A\", \"#n/a\", \"N/A\", \"nan\", \"None\"}\n", "for col in [\"FunctionType\", \"LevelDesignation\", \"Location\", \"ManagerSource\"]:\n", " df[col] = df[col].mask(df[col].astype(str).str.strip().isin(bad_text))\n", "\n", "\n", "def normalise_manager_status(value):\n", " if pd.isna(value):\n", " return np.nan\n", " text = str(value).strip().upper()\n", " if text in {\"MANAGER\", \"MANAGER+\", \"PEOPLE MANAGER\", \"YES\", \"TRUE\", \"Y\", \"1\"}:\n", " return \"Manager\"\n", " if text in {\"IC\", \"INDIVIDUAL CONTRIBUTOR\", \"NO\", \"FALSE\", \"N\", \"0\"}:\n", " return \"IC\"\n", " return str(value).strip()\n", "\n", "\n", "df[\"ManagerStatus\"] = df[\"ManagerSource\"].map(normalise_manager_status)\n", "\n", "duplicate_person_weeks = int(df.duplicated([\"PersonId\", \"MetricDate\"]).sum())\n", "if duplicate_person_weeks:\n", " raise ValueError(f\"Found {duplicate_person_weeks:,} duplicate person-week rows.\")\n", "\n", "df = df.sort_values([\"PersonId\", \"MetricDate\"]).reset_index(drop=True)\n", "df[\"copilot_active\"] = df[\"Total_Copilot_actions_taken\"].fillna(0) > 0\n", "# Distinct from `copilot_active`, which is actions-based: this flag drives the\n", "# population and active-use chart series and follows Total_Copilot_active_days.\n", "df[\"copilot_active_days_flag\"] = df[\"Total_Copilot_active_days\"].fillna(0) > 0\n", "\n", "# ---------------------------------------------------------------------------\n", "# The Copilot population: who had Copilot in a given week.\n", "#\n", "# This is the middle tier between the total measured population and the people\n", "# who actually used Copilot that week, and it is an estimate of who holds a\n", "# licence. It is derived one of two ways, in order of preference:\n", "#\n", "# Confirmed: an enabled-days column exists, so a person is in the Copilot\n", "# population in any week with more than zero enabled days.\n", "# Estimated: no enabled-days column exists, so the notebook falls back to the\n", "# presence of Copilot telemetry. Viva Insights reports Copilot\n", "# metrics for licensed people, so a non-null value is good evidence\n", "# of a licence, but it is inference rather than a licence record.\n", "#\n", "# Note that being in the Copilot population is not the same as using Copilot. A\n", "# person with zero actions in a week is still in the population that week; usage\n", "# is measured separately by `copilot_active`. Keeping the two apart is what lets\n", "# activation be read as a rate rather than as growth in reach.\n", "# ---------------------------------------------------------------------------\n", "LICENSE_COL = first_available(df, [normalise_column(c) for c in LICENSE_COLUMNS])\n", "has_license_col = LICENSE_COL is not None\n", "\n", "df[\"copilot_telemetry_present\"] = (\n", " df[\"Total_Copilot_actions_taken\"].notna()\n", " & df[\"Total_Copilot_active_days\"].notna()\n", ")\n", "if has_license_col:\n", " df[LICENSE_COL] = pd.to_numeric(df[LICENSE_COL], errors=\"coerce\")\n", " df[\"in_copilot_population\"] = df[LICENSE_COL].fillna(0) > 0\n", " POPULATION_QUALIFIER = \"Confirmed\"\n", " POPULATION_BASIS = f\"confirmed from '{LICENSE_COL}' > 0\"\n", "else:\n", " df[\"in_copilot_population\"] = df[\"copilot_telemetry_present\"]\n", " POPULATION_QUALIFIER = \"Estimated\"\n", " POPULATION_BASIS = \"estimated from the presence of Copilot telemetry\"\n", "\n", "POPULATION_LABEL = f\"{POPULATION_QUALIFIER} Copilot population\"\n", "# Lower-case variant for mid-sentence use that keeps the product name capitalised.\n", "POPULATION_LABEL_LC = f\"{POPULATION_QUALIFIER.lower()} Copilot population\"\n", "\n", "blank_rows_removed = raw_rows - len(df)\n", "weeks = np.sort(df[\"MetricDate\"].unique())\n", "latest_week = pd.Timestamp(weeks[-1])\n", "first_week = pd.Timestamp(weeks[0])\n", "\n", "if len(weeks) < SEGMENT_WINDOW_WEEKS:\n", " warnings.warn(\n", " f\"Only {len(weeks)} weeks are available, fewer than the {SEGMENT_WINDOW_WEEKS}-week \"\n", " \"rolling window. Habitual and Power User counts will be understated. See \"\n", " \"https://microsoft.github.io/viva-insights-sample-code/copilot-usage-segments/ \"\n", " \"for the 4-week variation designed for shorter panels.\"\n", " )\n", "\n", "# Report-wide metadata used to build informative footnotes: the date range covered, and\n", "# population size broken down by total measured population, Copilot population, and\n", "# actively-using population, all as of the latest observed week.\n", "latest_rows = df[\"MetricDate\"] == latest_week\n", "total_population_n = int(df.loc[latest_rows, \"PersonId\"].nunique())\n", "copilot_population_n = int(\n", " df.loc[latest_rows & df[\"in_copilot_population\"], \"PersonId\"].nunique()\n", ")\n", "active_population_n = int(\n", " df.loc[latest_rows & df[\"copilot_active_days_flag\"], \"PersonId\"].nunique()\n", ")\n", "report_metadata = pd.DataFrame([{\n", " \"analysis_start\": first_week.date().isoformat(),\n", " \"analysis_end\": latest_week.date().isoformat(),\n", " \"weeks_covered\": len(weeks),\n", " \"total_population_n\": total_population_n,\n", " \"copilot_population_n\": copilot_population_n,\n", " \"copilot_population_basis\": POPULATION_BASIS,\n", " \"active_population_n\": active_population_n,\n", " \"license_column_used\": LICENSE_COL if has_license_col else \"\",\n", "}])\n", "save_table(report_metadata, \"00_report_metadata\")\n", "FOOTNOTE_BASE = (\n", " f\"{report_metadata.at[0, 'analysis_start']} to {report_metadata.at[0, 'analysis_end']} \"\n", " f\"({len(weeks)} weeks) | n={total_population_n:,} measured, \"\n", " f\"{copilot_population_n:,} in the {POPULATION_LABEL_LC}\"\n", " + (\"*\" if not has_license_col else \"\")\n", " + f\", {active_population_n:,} used Copilot in the latest week\"\n", ")\n", "print(f\"Footnote base string: {FOOTNOTE_BASE}\\n\")\n", "\n", "print(f\"Rows in file: {raw_rows:,}\")\n", "print(f\"Blank rows removed: {blank_rows_removed:,}\")\n", "print(f\"Analysis rows: {len(df):,}\")\n", "print(f\"People: {df['PersonId'].nunique():,}\")\n", "print(f\"Weeks: {len(weeks)} \"\n", " f\"({first_week.date()} to {latest_week.date()})\")\n", "print(f\"Duplicate person-weeks: {duplicate_person_weeks:,}\")\n", "print(\"\\nOrganizational attributes resolved from this export:\")\n", "for target, source in attribute_sources.items():\n", " label = target if target != \"ManagerSource\" else \"ManagerStatus\"\n", " if source is None:\n", " print(f\" {label:<18} not available\")\n", " else:\n", " coverage = df[target if target != 'ManagerSource' else 'ManagerStatus'].notna().mean()\n", " print(f\" {label:<18} from '{source}' ({coverage * 100:.1f}% populated)\")\n", "print(\n", " f\"\\nCopilot population: {POPULATION_BASIS}\"\n", ")\n" ] }, { "cell_type": "markdown", "id": "469335e5", "metadata": {}, "source": [ "## 2. Reach and activation momentum\n", "\n", "Adoption has two separate parts that are easy to conflate: how many people *have* Copilot,\n", "and how many of them *use* it. This section keeps them apart, using three nested groups\n", "from broadest to narrowest.\n", "\n", "| Group | What it means | How it is measured |\n", "| --- | --- | --- |\n", "| **Total measured population** | Everyone in the Person Query. | Distinct `PersonId` per week. |\n", "| **Copilot population** | People who have Copilot in that week. This is the reach denominator. | Confirmed from an enabled-days column where the export has one, and otherwise estimated from the presence of Copilot telemetry. |\n", "| **Active users** | People who actually used Copilot in that week. | At least one Copilot action. |\n", "\n", "**Why the middle group is an estimate.** Viva Insights reports Copilot metrics for people\n", "who hold a licence, so a person with non-null Copilot values is almost certainly licensed.\n", "That makes telemetry a good stand-in when no licence field is available, but it is\n", "inference from usage metrics rather than a licence record, so the notebook labels it\n", "**Estimated Copilot population** wherever it appears. When an enabled-days column *is*\n", "present, the same group is measured directly and is labelled **Confirmed Copilot\n", "population**. The label printed below tells you which one you are looking at.\n", "\n", "**Being in the Copilot population is not the same as using Copilot.** Someone with zero\n", "actions in a week is still in the population for that week; they are simply not active.\n", "Keeping the two apart is what allows activation to be read as a rate, rather than\n", "mistaking a growing denominator for growing adoption.\n", "\n", "This distinction matters when the middle group grows. A rise can reflect a licence\n", "rollout, a change in query scope, or a change in telemetry completeness, and none of those\n", "is adoption. Growth in reach and growth in usage are worth checking separately, which is\n", "what the three panels below do.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "72f2d919", "metadata": {}, "outputs": [], "source": [ "weekly = (\n", " df.groupby(\"MetricDate\")\n", " .agg(\n", " measured_population=(\"PersonId\", \"nunique\"),\n", " copilot_population=(\"in_copilot_population\", \"sum\"),\n", " telemetry_present=(\"copilot_telemetry_present\", \"sum\"),\n", " active=(\"copilot_active\", \"sum\"),\n", " active_by_days=(\"copilot_active_days_flag\", \"sum\"),\n", " total_actions=(\"Total_Copilot_actions_taken\", \"sum\"),\n", " )\n", " .reset_index()\n", ")\n", "# Reach: what share of the measured population has Copilot.\n", "weekly[\"copilot_population_pct\"] = safe_ratio(\n", " weekly[\"copilot_population\"], weekly[\"measured_population\"]\n", ")\n", "# Activation: what share of the people who have Copilot actually used it. This is the\n", "# rate that matters, because it is not inflated by a growing denominator.\n", "weekly[\"active_pct_of_copilot_population\"] = safe_ratio(\n", " weekly[\"active\"], weekly[\"copilot_population\"]\n", ")\n", "weekly[\"active_pct_of_measured\"] = safe_ratio(\n", " weekly[\"active\"], weekly[\"measured_population\"]\n", ")\n", "weekly[\"actions_per_active\"] = (\n", " weekly[\"total_actions\"] / weekly[\"active\"].where(weekly[\"active\"] > 0)\n", ")\n", "\n", "copilot_pop = df[df[\"in_copilot_population\"]].copy()\n", "first_copilot_week = (\n", " copilot_pop.groupby(\"PersonId\")[\"MetricDate\"].min().rename(\"first_copilot_week\")\n", ")\n", "newly_in_population = (\n", " first_copilot_week.value_counts().sort_index().rename(\"newly_in_population\")\n", ")\n", "weekly = weekly.merge(\n", " newly_in_population, left_on=\"MetricDate\", right_index=True, how=\"left\"\n", ")\n", "weekly[\"newly_in_population\"] = weekly[\"newly_in_population\"].fillna(0).astype(int)\n", "save_table(weekly, \"02_weekly_reach_and_activation\")\n", "\n", "# Three side-by-side single-axis panels rather than a two-panel design with a secondary\n", "# (twin) axis, because a dual-axis chart invites misreading when the two y-scales are\n", "# easy to conflate at a glance.\n", "fig, axes = plt.subplots(1, 3, figsize=(19, 4.8))\n", "\n", "ax = axes[0]\n", "ax.plot(weekly[\"MetricDate\"], weekly[\"measured_population\"], marker=\"o\", lw=2.0,\n", " color=C_GREY, linestyle=\"--\", label=\"Total measured population\")\n", "ax.plot(weekly[\"MetricDate\"], weekly[\"copilot_population\"], marker=\"o\", lw=2.3,\n", " color=C_GOLD,\n", " label=f\"{POPULATION_LABEL}{'' if has_license_col else '*'}\")\n", "ax.plot(weekly[\"MetricDate\"], weekly[\"active\"], marker=\"o\", lw=2.3,\n", " color=C_TEAL, label=\"Used Copilot that week\")\n", "ax.set_ylabel(\"People\")\n", "ax.set_ylim(0, weekly[\"measured_population\"].max() * 1.08)\n", "ax.legend(frameon=False, fontsize=8.5, loc=\"lower left\")\n", "ax.set_title(\"Reach and usage over time\", loc=\"left\", fontweight=\"bold\")\n", "add_subtitle(ax, f\"Distinct PersonId by MetricDate; Copilot population {POPULATION_BASIS}\")\n", "if not has_license_col:\n", " ax.annotate(\n", " \"*No enabled-days column is available, so the Copilot population is estimated\\n\"\n", " \"from the presence of Copilot telemetry rather than read from a licence record.\",\n", " xy=(0.01, -0.32), xycoords=\"axes fraction\", fontsize=7.5, color=C_GREY,\n", " )\n", "\n", "ax = axes[1]\n", "new_after_baseline = weekly[\"newly_in_population\"].copy()\n", "new_after_baseline.iloc[0] = 0\n", "ax.bar(weekly[\"MetricDate\"], new_after_baseline, width=5.2, color=C_GOLD)\n", "ax.set_ylabel(\"People joining\")\n", "ax.set_title(\"People joining the Copilot population\", loc=\"left\", fontweight=\"bold\")\n", "add_subtitle(ax, \"Count of people whose first Copilot week falls in that week\")\n", "ax.grid(axis=\"x\", visible=False)\n", "\n", "ax = axes[2]\n", "ax.plot(weekly[\"MetricDate\"], weekly[\"active_pct_of_copilot_population\"], color=C_BLUE,\n", " marker=\"o\", lw=2.2)\n", "ax.set_ylabel(\"% of Copilot population active\")\n", "ax.set_ylim(0, 105)\n", "ax.yaxis.set_major_formatter(mticker.PercentFormatter())\n", "ax.set_title(\"Activation rate\", loc=\"left\", fontweight=\"bold\")\n", "add_subtitle(ax, \"% of people who have Copilot that used it in the week\")\n", "\n", "for ax in axes:\n", " ax.tick_params(axis=\"x\", rotation=30)\n", "\n", "fig.suptitle(\"Copilot reach and activation\", x=0.01, ha=\"left\", fontsize=15,\n", " fontweight=\"bold\")\n", "fig.tight_layout()\n", "save_figure(fig, \"02_reach_and_activation\")\n", "plt.show()\n", "\n", "first_row, last_row = weekly.iloc[0], weekly.iloc[-1]\n", "largest_additions = weekly.iloc[1:].nlargest(3, \"newly_in_population\")\n", "\n", "reach_change = last_row[\"copilot_population_pct\"] - first_row[\"copilot_population_pct\"]\n", "activation_change = (\n", " last_row[\"active_pct_of_copilot_population\"]\n", " - first_row[\"active_pct_of_copilot_population\"]\n", ")\n", "depth_change_pct = (\n", " 100 * (last_row[\"actions_per_active\"] / first_row[\"actions_per_active\"] - 1)\n", " if first_row[\"actions_per_active\"] else np.nan\n", ")\n", "\n", "print(f\"Copilot population basis: {POPULATION_BASIS}\\n\")\n", "print(\n", " f\"Reach: the {POPULATION_LABEL_LC} moved from \"\n", " f\"{fmt(first_row['copilot_population_pct'])}% to \"\n", " f\"{fmt(last_row['copilot_population_pct'])}% of the measured population \"\n", " f\"({reach_change:+.1f} points).\"\n", ")\n", "print(\n", " f\"Activation: {fmt(first_row['active_pct_of_copilot_population'])}% to \"\n", " f\"{fmt(last_row['active_pct_of_copilot_population'])}% of that population used \"\n", " f\"Copilot in the week ({activation_change:+.1f} points).\"\n", ")\n", "print(\n", " f\"Depth: actions per active user moved from \"\n", " f\"{fmt(first_row['actions_per_active'])} to {fmt(last_row['actions_per_active'])} \"\n", " f\"({fmt(depth_change_pct, '{:+.0f}')}%).\"\n", ")\n", "if has_license_col:\n", " telemetry_gap = last_row[\"telemetry_present\"] - last_row[\"copilot_population\"]\n", " if abs(telemetry_gap) >= 1:\n", " print(\n", " f\"\\nFor reference, {int(last_row['telemetry_present']):,} people have Copilot \"\n", " f\"telemetry in the latest week versus {int(last_row['copilot_population']):,} \"\n", " f\"with '{LICENSE_COL}' > 0, a difference of {int(telemetry_gap):+,}. Licence \"\n", " \"records are used here because they are the more direct measure.\"\n", " )\n", "\n", "if len(largest_additions) and largest_additions.iloc[0][\"newly_in_population\"] > 0:\n", " print(\"\\nLargest weeks for people joining the Copilot population:\")\n", " print(largest_additions[[\"MetricDate\", \"newly_in_population\",\n", " \"copilot_population_pct\",\n", " \"active_pct_of_copilot_population\"]].to_string(\n", " index=False, formatters={\n", " \"copilot_population_pct\": \"{:.1f}%\".format,\n", " \"active_pct_of_copilot_population\": \"{:.1f}%\".format,\n", " }))\n", " print(\n", " \"\\nCheck what drove these weeks before reading them as adoption. A licence \"\n", " \"rollout, a change in query scope, and a change in telemetry completeness all \"\n", " \"look identical here, and none of them is people choosing to use Copilot.\"\n", " )\n" ] }, { "cell_type": "markdown", "id": "e8a4e0de", "metadata": {}, "source": [ "## 3. Adoption journey and habit formation\n", "\n", "**Why this section matters:** a single week of use does not tell you whether Copilot has\n", "become part of someone's routine. Power and Habitual User status is this notebook's proxy\n", "for **stickiness** - evidence of durable, repeated return usage rather than a one-off\n", "trial. Both segments require at least one action in **9 of the trailing 12 weeks**, so\n", "membership cannot be earned by a single busy week; it requires activity spread across\n", "roughly a quarter. Power Users add a volume threshold on top of that consistency, which\n", "separates heavy, embedded use from lighter-but-still-durable habitual use. Together they\n", "answer a question a simple \"used it this week\" metric cannot: has the behaviour persisted\n", "long enough to call it a habit, and is it a return habit rather than a trial?\n", "\n", "The notebook applies the standard **12-week rolling usage-segment definition** described\n", "on the [Copilot Usage Segments](https://microsoft.github.io/viva-insights-sample-code/copilot-usage-segments/#formal-definitions)\n", "page and implemented by\n", "[`identify_usage_segments()`](https://microsoft.github.io/vivainsights/reference/identify_usage_segments.html).\n", "An \"active week\" means the target metric records at least one action.\n", "\n", "- **Power User:** at least one action in 9 or more of the trailing 12 weeks **and**\n", " an average of at least 15 weekly actions over the rolling period.\n", "- **Habitual User:** at least one action in 9 or more of the trailing 12 weeks, but the\n", " rolling weekly average is below the Power User threshold.\n", "- **Novice User:** a rolling average of at least one weekly action, without meeting the\n", " 9-of-12 habit requirement.\n", "- **Low User:** at least one action during the rolling period, but an average below one\n", " weekly action and without meeting the habit requirement.\n", "- **Non-user:** no actions during the rolling period.\n", "\n", "The categories are evaluated in that order, so Power Users are a high-volume subset of\n", "habitual users. Because the package calculates rolling averages from available history,\n", "Novice, Low, and Non-user labels can appear before a person has 12 observed weeks. A person\n", "cannot satisfy the 9-of-12 Habitual or Power requirement without at least nine active weeks.\n", "\n", "> **A note on thresholds.** The `version=\"12w\"` preset ignores any `power_thres` you pass\n", "> and always applies 15. To keep the configuration cell honest, this notebook calls\n", "> `identify_usage_segments()` with `version=None` and supplies `threshold`, `width`,\n", "> `max_window`, and `power_thres` explicitly. That reproduces the standard 12-week\n", "> definition exactly at the default settings, and it means that changing\n", "> `POWER_THRESHOLD` genuinely changes the segmentation rather than only relabelling it.\n", "\n", "For executive interpretation, Power and Habitual are combined as **Power + Habitual\n", "Users**, Novice and Low as **Emerging**, with Non-user retained separately. A plain-English glossary\n", "of every segment, saved as `00_usage_segment_definitions`, is exported below as a\n", "standalone asset so the definitions can be dropped directly into decks and reports." ] }, { "cell_type": "code", "execution_count": null, "id": "29d5c27f", "metadata": {}, "outputs": [], "source": [ "habit_rule = (\n", " f\"Active in at least {SEGMENT_HABIT_WEEKS} of the trailing \"\n", " f\"{SEGMENT_WINDOW_WEEKS} weeks\"\n", ")\n", "active_week_rule = (\n", " f\"a week counts as active at {SEGMENT_ACTION_THRESHOLD}+ Copilot action(s)\"\n", ")\n", "\n", "segment_definitions = pd.DataFrame([\n", " {\n", " \"segment\": \"Power User\",\n", " \"definition\": (\n", " f\"{habit_rule} AND an average of at least {POWER_THRESHOLD} weekly actions \"\n", " f\"over that window ({active_week_rule}).\"\n", " ),\n", " \"what_it_indicates\": (\n", " \"The highest-stickiness usage: frequent return visits at high volume. The \"\n", " \"clearest evidence that Copilot has become embedded in someone's workflow.\"\n", " ),\n", " },\n", " {\n", " \"segment\": \"Habitual User\",\n", " \"definition\": (\n", " f\"{habit_rule}, with a rolling weekly average below the Power User \"\n", " f\"threshold of {POWER_THRESHOLD}.\"\n", " ),\n", " \"what_it_indicates\": (\n", " \"A durable weekly habit has formed even though usage volume is modest, so \"\n", " \"consistency rather than intensity is the signal.\"\n", " ),\n", " },\n", " {\n", " \"segment\": \"Novice User\",\n", " \"definition\": (\n", " f\"A rolling average of at least one weekly action, without meeting the \"\n", " f\"{SEGMENT_HABIT_WEEKS}-of-{SEGMENT_WINDOW_WEEKS} week requirement.\"\n", " ),\n", " \"what_it_indicates\": (\n", " \"Has tried Copilot repeatedly but has not yet formed a consistent weekly \"\n", " \"habit, so this is the main pool for conversion into Power + Habitual use.\"\n", " ),\n", " },\n", " {\n", " \"segment\": \"Low User\",\n", " \"definition\": (\n", " \"At least one action in the rolling period, but a rolling average below one \"\n", " \"weekly action and not meeting the habit requirement.\"\n", " ),\n", " \"what_it_indicates\": \"Sporadic, occasional use only.\",\n", " },\n", " {\n", " \"segment\": \"Non-user\",\n", " \"definition\": (\n", " f\"No Copilot actions recorded during the rolling {SEGMENT_WINDOW_WEEKS}-week \"\n", " \"period.\"\n", " ),\n", " \"what_it_indicates\": \"No observed usage.\",\n", " },\n", "])\n", "save_table(segment_definitions, \"00_usage_segment_definitions\")\n", "print(segment_definitions.to_string(index=False))\n" ] }, { "cell_type": "code", "execution_count": null, "id": "b4ac1baf", "metadata": { "execution": { "iopub.execute_input": "2026-08-05T14:21:55.671963Z", "iopub.status.busy": "2026-08-05T14:21:55.671697Z", "iopub.status.idle": "2026-08-05T14:22:04.901612Z", "shell.execute_reply": "2026-08-05T14:22:04.900865Z" } }, "outputs": [], "source": [ "# Preserve calendar weeks from the person's first week in the Copilot population.\n", "# Dropping null rows would compress the rolling window into the last N observed rows.\n", "seg_input = df.merge(first_copilot_week, on=\"PersonId\", how=\"inner\")\n", "seg_input = seg_input[seg_input[\"MetricDate\"] >= seg_input[\"first_copilot_week\"]].copy()\n", "seg_input[\"Total_Copilot_actions_taken\"] = (\n", " seg_input[\"Total_Copilot_actions_taken\"].fillna(0).astype(float)\n", ")\n", "\n", "# version=None with explicit parameters, so that the configured POWER_THRESHOLD is\n", "# actually applied. At the default settings this reproduces the standard \"12w\"\n", "# definition exactly. See the note in the section above.\n", "segment_kwargs = dict(\n", " metric=\"Total_Copilot_actions_taken\",\n", " version=None,\n", " threshold=SEGMENT_ACTION_THRESHOLD,\n", " width=SEGMENT_HABIT_WEEKS,\n", " max_window=SEGMENT_WINDOW_WEEKS,\n", " power_thres=POWER_THRESHOLD,\n", ")\n", "\n", "seg = vi.identify_usage_segments(seg_input.copy(), return_type=\"data\", **segment_kwargs)\n", "segment_col = next(\n", " col for col in (\"UsageSegments\", f\"UsageSegments_{SEGMENT_WINDOW_WEEKS}w\")\n", " if col in seg.columns\n", ")\n", "seg = seg.rename(columns={segment_col: \"UsageSegment_12w\"})\n", "\n", "latest_segments = (\n", " seg[seg[\"MetricDate\"] == latest_week][[\"PersonId\", \"UsageSegment_12w\"]]\n", " .drop_duplicates(\"PersonId\")\n", ")\n", "journey_map = {\n", " \"Power User\": \"Power + Habitual\",\n", " \"Habitual User\": \"Power + Habitual\",\n", " \"Novice User\": \"Emerging\",\n", " \"Low User\": \"Emerging\",\n", " \"Non-user\": \"Non-user\",\n", "}\n", "latest_segments[\"JourneyStage\"] = latest_segments[\"UsageSegment_12w\"].map(journey_map)\n", "\n", "segment_order = [\"Power User\", \"Habitual User\", \"Novice User\", \"Low User\", \"Non-user\"]\n", "# Power, Habitual, and Novice match vivainsights.identify_usage_segments()'s own default\n", "# plot colours, so the charts here stay visually consistent with the package's native\n", "# chart. The package renders Low User (\"#808080\") and Non-user (\"grey\") as effectively\n", "# the same grey, which is indistinguishable on a slide, so Low User is given a distinct\n", "# gold tone and all five segments remain separable.\n", "segment_color_map = {\n", " \"Power User\": \"#0c336e\",\n", " \"Habitual User\": \"#1c66b0\",\n", " \"Novice User\": \"#80baea\",\n", " \"Low User\": \"#B9892D\",\n", " \"Non-user\": \"#808080\",\n", "}\n", "segment_colors = [segment_color_map[segment] for segment in segment_order]\n", "\n", "# A single, shared colour scheme for the 3-stage journey grouping (Power + Habitual /\n", "# Emerging / Non-user), reused consistently across every chart that shows it.\n", "journey_stage_order = [\"Power + Habitual\", \"Emerging\", \"Non-user\"]\n", "journey_stage_color_map = {\n", " \"Power + Habitual\": segment_color_map[\"Power User\"],\n", " \"Emerging\": segment_color_map[\"Novice User\"],\n", " \"Non-user\": segment_color_map[\"Non-user\"],\n", "}\n", "segment_summary = (\n", " latest_segments[\"UsageSegment_12w\"].value_counts()\n", " .reindex(segment_order, fill_value=0)\n", " .rename_axis(\"segment\").reset_index(name=\"people\")\n", ")\n", "segment_summary[\"pct\"] = safe_ratio(\n", " segment_summary[\"people\"], segment_summary[\"people\"].sum()\n", ")\n", "save_table(segment_summary, \"03_latest_usage_segments\")\n", "\n", "person_journey = (\n", " copilot_pop.groupby(\"PersonId\")\n", " .agg(\n", " first_copilot_week=(\"MetricDate\", \"min\"),\n", " observed_weeks=(\"MetricDate\", \"nunique\"),\n", " active_weeks=(\"copilot_active\", \"sum\"),\n", " total_actions=(\"Total_Copilot_actions_taken\", \"sum\"),\n", " )\n", " .reset_index()\n", " .merge(latest_segments, on=\"PersonId\", how=\"left\")\n", ")\n", "person_journey[\"active_share\"] = (\n", " person_journey[\"active_weeks\"] / person_journey[\"observed_weeks\"]\n", ")\n", "\n", "cohort_latest = (\n", " person_journey.groupby(\"first_copilot_week\")\n", " .agg(\n", " people=(\"PersonId\", \"size\"),\n", " active_latest=(\"PersonId\", lambda ids: int(\n", " df[(df[\"MetricDate\"] == latest_week)\n", " & (df[\"PersonId\"].isin(ids))\n", " & df[\"copilot_active\"]][\"PersonId\"].nunique()\n", " )),\n", " power_habitual_latest=(\"JourneyStage\", lambda s: int((s == \"Power + Habitual\").sum())),\n", " )\n", " .reset_index()\n", ")\n", "cohort_latest[\"active_latest_pct\"] = safe_ratio(\n", " cohort_latest[\"active_latest\"], cohort_latest[\"people\"]\n", ")\n", "cohort_latest[\"power_habitual_latest_pct\"] = safe_ratio(\n", " cohort_latest[\"power_habitual_latest\"], cohort_latest[\"people\"]\n", ")\n", "cohort_latest[\"weeks_available\"] = (\n", " (latest_week - cohort_latest[\"first_copilot_week\"]).dt.days // 7 + 1\n", ")\n", "cohort_latest[\"power_habitual_latest_pct_eligible\"] = cohort_latest[\n", " \"power_habitual_latest_pct\"\n", "].where(cohort_latest[\"weeks_available\"] >= SEGMENT_WINDOW_WEEKS)\n", "cohort_latest = cohort_latest.sort_values(\"first_copilot_week\").reset_index(drop=True)\n", "# Entry cohorts are a group like any other, so apply the hard privacy floor before the\n", "# table is exported. Suppressed cohorts are counted so the total is still reconcilable.\n", "cohort_suppressed = cohort_latest[cohort_latest[\"people\"] < MIN_PRIVACY_N]\n", "cohort_latest = cohort_latest[cohort_latest[\"people\"] >= MIN_PRIVACY_N].reset_index(drop=True)\n", "if len(cohort_suppressed):\n", " print(\n", " f\"Privacy floor: {len(cohort_suppressed)} entry cohort(s) covering \"\n", " f\"{int(cohort_suppressed['people'].sum()):,} people fall below the \"\n", " f\"{MIN_PRIVACY_N}-person floor and are excluded from the cohort table.\"\n", " )\n", "save_table(cohort_latest, \"03_entry_cohort_journey\")\n", "\n", "\n", "# The package-native time-series view and table, as the source-of-truth presentation\n", "# of the segment calculation.\n", "native_segment_table = vi.identify_usage_segments(\n", " seg_input.copy(), return_type=\"table\", **segment_kwargs\n", ").reset_index()\n", "save_table(native_segment_table, \"03_usage_segments_over_time\")\n", "\n", "native_segment_fig = vi.identify_usage_segments(\n", " seg_input.copy(), return_type=\"plot\", **segment_kwargs\n", ")\n", "native_segment_ax = native_segment_fig.axes[0]\n", "native_segment_ax.set_title(\"\", loc=\"center\")\n", "native_segment_ax.set_title(\n", " f\"{SEGMENT_WINDOW_WEEKS}-week Copilot usage segments over time\",\n", " loc=\"left\", fontweight=\"bold\",\n", ")\n", "for annotation in native_segment_ax.texts:\n", " if annotation.get_text().startswith(\"Usage Segments\"):\n", " annotation.set_visible(False)\n", "for container in native_segment_ax.containers:\n", " segment = container.get_label()\n", " if segment in segment_color_map:\n", " for patch in container.patches:\n", " patch.set_facecolor(segment_color_map[segment])\n", " patch.set_edgecolor(\"white\")\n", "add_subtitle(native_segment_ax,\n", " \"vi.identify_usage_segments(): rolling share of PersonId by usage segment\")\n", "native_segment_ax.legend(title=\"Usage Segment\", frameon=True)\n", "native_segment_fig.text(\n", " 0.01, -0.01,\n", " f\"The first {SEGMENT_WINDOW_WEEKS - 1} dates have incomplete \"\n", " f\"{SEGMENT_WINDOW_WEEKS}-week histories. Habitual and Power status still require at \"\n", " f\"least {SEGMENT_HABIT_WEEKS} active weeks; Novice, Low and Non-user can be assigned \"\n", " \"from available history.\",\n", " fontsize=8.5, color=C_GREY,\n", ")\n", "save_figure(native_segment_fig, \"03_usage_segments_over_time\")\n", "plt.show()\n", "\n", "# Entry cohorts answer a different question from the native segment trend:\n", "# whether people who joined the Copilot population recently have had time to\n", "# activate and form a habit.\n", "plot_cohorts = cohort_latest[cohort_latest[\"people\"] >= MIN_DISPLAY_N].copy()\n", "if len(plot_cohorts):\n", " fig, ax = plt.subplots(figsize=(10.5, 4.8))\n", " ax.bar(plot_cohorts[\"first_copilot_week\"], plot_cohorts[\"active_latest_pct\"],\n", " width=5.0, color=C_TEAL, label=\"Active in latest week\")\n", " ax.plot(plot_cohorts[\"first_copilot_week\"],\n", " plot_cohorts[\"power_habitual_latest_pct_eligible\"],\n", " marker=\"o\", lw=2.2, color=journey_stage_color_map[\"Power + Habitual\"],\n", " label=\"Power + Habitual at latest week\")\n", " ax.set_ylim(0, 105)\n", " ax.yaxis.set_major_formatter(mticker.PercentFormatter())\n", " ax.set_title(\"Latest-week activation and habit status by entry cohort\",\n", " loc=\"left\", fontweight=\"bold\")\n", " add_subtitle(ax, \"% of each entry cohort active or Power + Habitual in the latest week\")\n", " ax.set_xlabel(\"First week in the Copilot population\")\n", " ax.set_ylabel(\"% of cohort\")\n", " ax.legend(frameon=False)\n", " ax.tick_params(axis=\"x\", rotation=30)\n", " fig.tight_layout()\n", " save_figure(fig, \"03_entry_cohort_journey\")\n", " plt.show()\n", "else:\n", " print(\n", " f\"No entry cohort reaches the {MIN_DISPLAY_N}-person display floor, so the \"\n", " \"cohort chart is skipped. This is normal when people joined the Copilot \"\n", " \"population gradually.\"\n", " )\n", "\n", "print(segment_summary.to_string(index=False, formatters={\"pct\": \"{:.1f}%\".format}))\n", "\n", "if len(plot_cohorts):\n", " print(f\"\\nEntry cohorts with at least {MIN_DISPLAY_N} people:\")\n", " print(plot_cohorts[[\n", " \"first_copilot_week\", \"people\", \"weeks_available\", \"active_latest_pct\",\n", " \"power_habitual_latest_pct_eligible\",\n", " ]].to_string(index=False, formatters={\n", " \"active_latest_pct\": \"{:.1f}%\".format,\n", " \"power_habitual_latest_pct_eligible\": lambda v: \"not yet eligible\" if pd.isna(v)\n", " else f\"{v:.1f}%\",\n", " }))\n", "\n", " eligible_cohorts = plot_cohorts.dropna(subset=[\"power_habitual_latest_pct_eligible\"])\n", " if len(eligible_cohorts) >= 2:\n", " oldest, newest = eligible_cohorts.iloc[0], eligible_cohorts.iloc[-1]\n", " gap = oldest[\"active_latest_pct\"] - newest[\"active_latest_pct\"]\n", " direction = (\"lower\" if gap > 0 else \"higher\") if abs(gap) >= 1 else \"similar\"\n", " print(\n", " f\"\\nActivation in the newest fully eligible cohort is {direction} than in \"\n", " f\"the earliest ({fmt(newest['active_latest_pct'])}% vs \"\n", " f\"{fmt(oldest['active_latest_pct'])}%).\"\n", " )\n" ] }, { "cell_type": "markdown", "id": "f00dd63d", "metadata": {}, "source": [ "## 4. Where adoption is leading or lagging\n", "\n", "`vi.create_rank()` provides the package-native organizational comparison. Because usage\n", "segments are categorical, the ranking uses three numeric adoption measures:\n", "\n", "- **Power and Habitual Users (%):** the mean of a 0/100 Habitual-or-Power indicator.\n", "- **Active-week share:** the percentage of observed weeks with at least one Copilot action.\n", "- **Average weekly Copilot actions:** usage depth across observed weeks.\n", "\n", "The native dumbbell plot shows the highest and lowest qualifying group for each attribute;\n", "the exported tables retain every qualifying group. Whichever of function, manager status,\n", "and location your export provides is included, at a minimum group size of\n", "`MIN_DISPLAY_N`. Attributes that are missing, or that resolve to a single group, are\n", "skipped automatically." ] }, { "cell_type": "code", "execution_count": null, "id": "e30872ab", "metadata": { "execution": { "iopub.execute_input": "2026-08-05T14:22:04.904267Z", "iopub.status.busy": "2026-08-05T14:22:04.903903Z", "iopub.status.idle": "2026-08-05T14:22:05.806353Z", "shell.execute_reply": "2026-08-05T14:22:05.805321Z" } }, "outputs": [], "source": [ "latest_attributes = (\n", " df[df[\"MetricDate\"] == latest_week][\n", " [\"PersonId\", \"FunctionType\", \"ManagerStatus\", \"Location\", \"LevelDesignation\"]\n", " ]\n", " .drop_duplicates(\"PersonId\")\n", ")\n", "adoption_snapshot = latest_segments.merge(latest_attributes, on=\"PersonId\", how=\"left\")\n", "adoption_snapshot = adoption_snapshot.merge(\n", " df[df[\"MetricDate\"] == latest_week][\n", " [\"PersonId\", \"copilot_active\", \"Total_Copilot_actions_taken\"]\n", " ].drop_duplicates(\"PersonId\"),\n", " on=\"PersonId\", how=\"left\"\n", ")\n", "\n", "\n", "def adoption_by(frame, attribute):\n", " \"\"\"Adoption summary for one organizational attribute, or None when unusable.\"\"\"\n", " if attribute not in frame.columns or frame[attribute].notna().sum() == 0:\n", " return None\n", " summary = (\n", " frame.dropna(subset=[attribute])\n", " .groupby(attribute)\n", " .agg(\n", " people=(\"PersonId\", \"nunique\"),\n", " active=(\"copilot_active\", \"sum\"),\n", " power_habitual=(\"JourneyStage\", lambda s: int((s == \"Power + Habitual\").sum())),\n", " median_actions_active=(\"Total_Copilot_actions_taken\",\n", " lambda s: s[s > 0].median()),\n", " )\n", " .reset_index()\n", " )\n", " # Never report a group below the hard privacy floor.\n", " summary = summary[summary[\"people\"] >= MIN_PRIVACY_N]\n", " if summary.empty:\n", " return None\n", " summary[\"active_pct\"] = safe_ratio(summary[\"active\"], summary[\"people\"])\n", " summary[\"power_habitual_pct\"] = safe_ratio(\n", " summary[\"power_habitual\"], summary[\"people\"]\n", " )\n", " return summary\n", "\n", "\n", "manager_summary = adoption_by(adoption_snapshot, \"ManagerStatus\")\n", "has_manager_split = (\n", " manager_summary is not None and manager_summary[\"ManagerStatus\"].nunique() > 1\n", ")\n", "if manager_summary is not None:\n", " save_table(manager_summary, \"04_manager_adoption\")\n", "\n", "function_summary = adoption_by(adoption_snapshot, \"FunctionType\")\n", "if function_summary is not None:\n", " function_summary = (\n", " function_summary[function_summary[\"people\"] >= MIN_DISPLAY_N]\n", " .sort_values(\"active_pct\", ascending=False)\n", " .reset_index(drop=True)\n", " )\n", " if function_summary.empty:\n", " function_summary = None\n", "if function_summary is not None:\n", " save_table(function_summary, \"04_function_adoption\")\n", "\n", "# Where the Novice Users are concentrated. This identifies practical targets for\n", "# conversion outreach, rather than an abstract conversion rate with no action attached.\n", "novice_by_function = None\n", "if function_summary is not None:\n", " novice_by_function = (\n", " adoption_snapshot[adoption_snapshot[\"UsageSegment_12w\"] == \"Novice User\"]\n", " .dropna(subset=[\"FunctionType\"])\n", " .groupby(\"FunctionType\")[\"PersonId\"].nunique()\n", " .rename(\"novice_people\").reset_index()\n", " .merge(function_summary[[\"FunctionType\", \"people\"]], on=\"FunctionType\", how=\"right\")\n", " )\n", " novice_by_function[\"novice_people\"] = (\n", " novice_by_function[\"novice_people\"].fillna(0).astype(int)\n", " )\n", " novice_by_function[\"novice_pct_of_function\"] = safe_ratio(\n", " novice_by_function[\"novice_people\"], novice_by_function[\"people\"]\n", " )\n", " novice_by_function[\"share_of_all_novices\"] = safe_ratio(\n", " novice_by_function[\"novice_people\"], novice_by_function[\"novice_people\"].sum()\n", " )\n", " novice_by_function = novice_by_function.sort_values(\n", " \"novice_people\", ascending=False\n", " ).reset_index(drop=True)\n", " save_table(novice_by_function, \"04_novice_users_by_function\")\n", "\n", "rank_data = (\n", " person_journey\n", " .merge(latest_attributes[[\"PersonId\", \"FunctionType\", \"ManagerStatus\", \"Location\"]],\n", " on=\"PersonId\", how=\"left\")\n", ")\n", "rank_data[\"MetricDate\"] = latest_week\n", "rank_data[\"Power and Habitual Users (%)\"] = (\n", " rank_data[\"JourneyStage\"] == \"Power + Habitual\"\n", ").astype(float) * 100\n", "rank_data[\"Active_week_share_pct\"] = rank_data[\"active_share\"] * 100\n", "rank_data[\"Average_weekly_Copilot_actions\"] = (\n", " rank_data[\"total_actions\"] / rank_data[\"observed_weeks\"]\n", ")\n", "if LOCATION_EXCLUDE:\n", " rank_data[\"Location\"] = rank_data[\"Location\"].where(\n", " ~rank_data[\"Location\"].isin(LOCATION_EXCLUDE)\n", " )\n", "\n", "# create_rank() needs at least one group clearing MIN_DISPLAY_N, so only pass attributes\n", "# that can actually produce one. This keeps the section working on exports that carry\n", "# only some of the organizational attributes.\n", "rank_hrvars = []\n", "for attribute in [\"FunctionType\", \"ManagerStatus\", \"Location\"]:\n", " values = rank_data[attribute].dropna()\n", " if values.nunique() < 1:\n", " continue\n", " if values.value_counts().max() < MIN_DISPLAY_N:\n", " continue\n", " rank_hrvars.append(attribute)\n", "\n", "rank_all_measures = None\n", "rank_sustained = None\n", "if not rank_hrvars:\n", " print(\n", " \"No organizational attribute in this export has a group of at least \"\n", " f\"{MIN_DISPLAY_N} people, so the create_rank() comparisons are skipped. \"\n", " \"Add FunctionType, IsManager/SupervisorIndicator, or Location to the query \"\n", " \"to enable this section.\"\n", " )\n", "else:\n", " rank_ready = rank_data.dropna(subset=rank_hrvars, how=\"all\").copy()\n", " for attribute in rank_hrvars:\n", " rank_ready[attribute] = rank_ready[attribute].fillna(\"Unknown\")\n", "\n", " measures = {\n", " \"Power and Habitual Users (%)\": \"Power and Habitual Users (%)\",\n", " \"Active_week_share_pct\": \"Active-week share\",\n", " \"Average_weekly_Copilot_actions\": \"Average weekly Copilot actions\",\n", " }\n", " rank_tables = {}\n", " for metric, label in measures.items():\n", " rank_tables[metric] = vi.create_rank(\n", " rank_ready, metric=metric, hrvar=rank_hrvars,\n", " mingroup=MIN_DISPLAY_N, return_type=\"table\",\n", " )\n", " rank_sustained = rank_tables[\"Power and Habitual Users (%)\"]\n", "\n", " # Preserve the complete table-return outputs, not only the extrema shown in\n", " # create_rank(return_type=\"plot\").\n", " rank_all_measures = pd.concat(\n", " [table.assign(measure=measures[metric]) for metric, table in rank_tables.items()],\n", " ignore_index=True,\n", " )[[\"measure\", \"hrvar\", \"attributes\", \"metric\", \"n\"]].rename(columns={\n", " \"hrvar\": \"organizational_attribute\",\n", " \"attributes\": \"group\",\n", " \"metric\": \"value\",\n", " \"n\": \"people\",\n", " })\n", " save_table(rank_all_measures, \"04_create_rank_all_measures\")\n", "\n", " function_rank = rank_sustained[rank_sustained[\"hrvar\"] == \"FunctionType\"]\n", " rank_table_view = pd.concat([\n", " function_rank.head(10),\n", " function_rank.tail(10),\n", " rank_sustained[rank_sustained[\"hrvar\"] != \"FunctionType\"],\n", " ]).drop_duplicates([\"hrvar\", \"attributes\"]).reset_index(drop=True)\n", " rank_table_view = rank_table_view.rename(columns={\n", " \"hrvar\": \"Attribute\",\n", " \"attributes\": \"Group\",\n", " \"metric\": \"Power + Habitual Users (%)\",\n", " \"n\": \"People\",\n", " })\n", " save_table(rank_table_view, \"04_create_rank_leadership_table\")\n", "\n", " display(\n", " rank_table_view.style\n", " .format({\"Power + Habitual Users (%)\": \"{:.1f}%\"})\n", " .background_gradient(\n", " subset=[\"Power + Habitual Users (%)\"], cmap=\"Blues\", vmin=0, vmax=100\n", " )\n", " .hide(axis=\"index\")\n", " .set_caption(\n", " \"create_rank(return_type='table'): Power + Habitual adoption by organizational group\"\n", " )\n", " )\n", "\n", " rank_fig = vi.create_rank(\n", " rank_ready, metric=\"Power and Habitual Users (%)\", hrvar=rank_hrvars,\n", " mingroup=MIN_DISPLAY_N, return_type=\"plot\", figsize=(9.5, 5.2),\n", " )\n", " # The package titles the plot from the metric column name, and that name already\n", " # reads cleanly (\"Power and Habitual Users (%)\"), so no override is needed.\n", " rank_ax = rank_fig.axes[0]\n", " for y_pos, hrvar in enumerate(rank_hrvars):\n", " group_rank = rank_sustained[rank_sustained[\"hrvar\"] == hrvar]\n", " if group_rank.empty:\n", " continue\n", " high, low = group_rank.iloc[0], group_rank.iloc[-1]\n", " rank_ax.annotate(\n", " f\"{low['attributes']} ({low['metric']:.1f}%)\",\n", " (low[\"metric\"], y_pos), xytext=(-6, -18), textcoords=\"offset points\",\n", " ha=\"right\", fontsize=7.5, color=C_RED,\n", " )\n", " rank_ax.annotate(\n", " f\"{high['attributes']} ({high['metric']:.1f}%)\",\n", " (high[\"metric\"], y_pos), xytext=(6, 8), textcoords=\"offset points\",\n", " ha=\"left\", fontsize=7.5, color=journey_stage_color_map[\"Power + Habitual\"],\n", " )\n", " add_subtitle(rank_ax, \"vi.create_rank(): % of PersonId classified Power or Habitual User in latest week, by group\")\n", " save_figure(rank_fig, \"04_adoption_rank\")\n", " plt.show()\n", "\n", "# The manager segment mix shows the full adoption curve rather than only the highest\n", "# and lowest values returned by the native rank plot.\n", "if has_manager_split:\n", " segment_manager_mix = pd.crosstab(\n", " adoption_snapshot[\"ManagerStatus\"],\n", " adoption_snapshot[\"UsageSegment_12w\"],\n", " normalize=\"index\",\n", " ).mul(100).reindex(columns=segment_order, fill_value=0)\n", "\n", " fig, ax = plt.subplots(figsize=(8.5, 4.8))\n", " segment_manager_mix.plot(\n", " kind=\"bar\", stacked=True, ax=ax, color=segment_colors, width=0.65\n", " )\n", " ax.set_ylabel(\"% within manager group\")\n", " ax.yaxis.set_major_formatter(mticker.PercentFormatter())\n", " ax.set_xlabel(\"\")\n", " ax.set_title(\"Usage segment mix by manager status\", loc=\"left\", fontweight=\"bold\")\n", " add_subtitle(ax, \"% of PersonId in each usage segment, by manager status, in the latest week\")\n", " ax.legend(frameon=False, fontsize=8, ncol=3)\n", " ax.tick_params(axis=\"x\", rotation=0)\n", " fig.tight_layout()\n", " save_figure(fig, \"04_manager_segment_mix\")\n", " plt.show()\n", "else:\n", " segment_manager_mix = None\n", " print(\n", " \"Manager status is unavailable or has only one value in this export, so the \"\n", " \"manager comparison is skipped.\"\n", " )\n", "\n", "if rank_all_measures is not None:\n", " print(\"\\nFull create_rank table outputs are exported in 04_create_rank_all_measures.csv.\")\n", " print(\"\\nHighest Power + Habitual adoption groups from create_rank():\")\n", " print(rank_sustained.head(12).to_string(index=False))\n", "\n", "if manager_summary is not None:\n", " print(\"\\nManager and individual contributor adoption:\")\n", " print(manager_summary[[\"ManagerStatus\", \"people\", \"active_pct\", \"power_habitual_pct\",\n", " \"median_actions_active\"]].to_string(index=False, formatters={\n", " \"active_pct\": \"{:.1f}%\".format,\n", " \"power_habitual_pct\": \"{:.1f}%\".format,\n", " \"median_actions_active\": \"{:.1f}\".format,\n", " }))\n", "\n", "if function_summary is not None:\n", " print(f\"\\nFunctions with at least {MIN_DISPLAY_N} people:\")\n", " print(function_summary[[\"FunctionType\", \"people\", \"active_pct\", \"power_habitual_pct\",\n", " \"median_actions_active\"]].to_string(index=False, formatters={\n", " \"active_pct\": \"{:.1f}%\".format,\n", " \"power_habitual_pct\": \"{:.1f}%\".format,\n", " \"median_actions_active\": \"{:.1f}\".format,\n", " }))\n" ] }, { "cell_type": "markdown", "id": "d3ee3600", "metadata": {}, "source": [ "### Where to replicate, and where to target\n", "\n", "Two practical follow-ups to the ranking above.\n", "\n", "**Where to replicate.** Functions that have already crossed a high Power + Habitual\n", "adoption threshold are proof, using this organization's own tooling, policies, and\n", "workload, that habitual use is achievable at scale. Treat them as a replication playbook,\n", "not a league table.\n", "\n", "**Where to target.** Novice Users have tried Copilot repeatedly without settling into a\n", "weekly habit, so they are the population closest to converting. Knowing which functions\n", "hold most of them turns the gap into a specific, addressable audience.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "30cfe049", "metadata": {}, "outputs": [], "source": [ "leading_functions = None\n", "if function_summary is not None:\n", " leading_functions = function_summary[\n", " function_summary[\"power_habitual_pct\"] >= LEADING_FUNCTION_THRESHOLD_PCT\n", " ].sort_values(\"power_habitual_pct\", ascending=False).reset_index(drop=True)\n", " save_table(leading_functions, \"04_leading_functions_playbook\")\n", "\n", "if leading_functions is None:\n", " print(\n", " \"No function attribute is available in this export, so the leading-function \"\n", " \"view is skipped.\"\n", " )\n", "elif len(leading_functions):\n", " fig, ax = plt.subplots(figsize=(9.5, max(3.2, 0.5 * len(leading_functions) + 1.2)))\n", " bars = ax.barh(leading_functions[\"FunctionType\"], leading_functions[\"power_habitual_pct\"],\n", " color=journey_stage_color_map[\"Power + Habitual\"], height=0.6)\n", " ax.invert_yaxis()\n", " ax.axvline(LEADING_FUNCTION_THRESHOLD_PCT, color=C_GREY, lw=1, linestyle=\"--\")\n", " ax.set_xlabel(\"Power + Habitual Users (%)\")\n", " ax.xaxis.set_major_formatter(mticker.PercentFormatter())\n", " ax.set_xlim(0, 105)\n", " ax.set_title(\n", " f\"Functions at or above {LEADING_FUNCTION_THRESHOLD_PCT:.0f}% Power + Habitual \"\n", " \"adoption\", loc=\"left\", fontweight=\"bold\",\n", " )\n", " add_subtitle(ax, f\"% of PersonId classified Power or Habitual User, by function, n >= {MIN_DISPLAY_N} per group\")\n", " for bar, value in zip(bars, leading_functions[\"power_habitual_pct\"]):\n", " ax.annotate(f\"{value:.1f}%\", (bar.get_width(), bar.get_y() + bar.get_height() / 2),\n", " xytext=(6, 0), textcoords=\"offset points\", va=\"center\", fontsize=9)\n", " ax.grid(axis=\"y\", visible=False)\n", " fig.tight_layout()\n", " save_figure(fig, \"04_leading_functions_playbook\")\n", " plt.show()\n", " print(\n", " f\"{len(leading_functions)} function(s) have already crossed \"\n", " f\"{LEADING_FUNCTION_THRESHOLD_PCT:.0f}% Power + Habitual adoption:\"\n", " )\n", " print(leading_functions[[\"FunctionType\", \"people\", \"power_habitual_pct\"]].to_string(\n", " index=False, formatters={\"power_habitual_pct\": \"{:.1f}%\".format}\n", " ))\n", "else:\n", " best = function_summary.sort_values(\"power_habitual_pct\", ascending=False).iloc[0]\n", " print(\n", " f\"No function has yet crossed {LEADING_FUNCTION_THRESHOLD_PCT:.0f}% Power + \"\n", " f\"Habitual adoption. The current leader is {best['FunctionType']} at \"\n", " f\"{fmt(best['power_habitual_pct'])}%. Lower LEADING_FUNCTION_THRESHOLD_PCT in \"\n", " \"the configuration cell to inspect a wider set of groups.\"\n", " )\n" ] }, { "cell_type": "code", "execution_count": null, "id": "cell0014", "metadata": {}, "outputs": [], "source": [ "total_people = int(segment_summary[\"people\"].sum())\n", "novice_n = int(segment_summary.set_index(\"segment\").loc[\"Novice User\", \"people\"])\n", "\n", "# Where the Novice Users actually are: a concrete view for targeted conversion, showing\n", "# both the count of Novice Users per function and what share of the function they are.\n", "top_novice_functions = (\n", " novice_by_function[novice_by_function[\"novice_people\"] > 0].head(10)\n", " if novice_by_function is not None else None\n", ")\n", "if top_novice_functions is not None and len(top_novice_functions):\n", " fig, ax = plt.subplots(figsize=(9.5, max(3.2, 0.5 * len(top_novice_functions) + 1.2)))\n", " bars = ax.barh(top_novice_functions[\"FunctionType\"], top_novice_functions[\"novice_people\"],\n", " color=segment_color_map[\"Novice User\"], height=0.6)\n", " ax.invert_yaxis()\n", " ax.set_xlabel(\"Novice Users (people)\")\n", " ax.set_title(\"Novice Users by function\", loc=\"left\", fontweight=\"bold\")\n", " add_subtitle(ax, f\"Count of PersonId classified Novice User, by function, n >= {MIN_DISPLAY_N} per group\")\n", " ax.set_ylabel(\"\")\n", " for bar, row in zip(bars, top_novice_functions.itertuples()):\n", " ax.annotate(f\"{row.novice_people:,} ({row.novice_pct_of_function:.0f}% of function)\",\n", " (bar.get_width(), bar.get_y() + bar.get_height() / 2),\n", " xytext=(6, 0), textcoords=\"offset points\", va=\"center\", fontsize=8.5)\n", " ax.grid(axis=\"y\", visible=False)\n", " fig.tight_layout()\n", " save_figure(fig, \"04_novice_users_by_function\")\n", " plt.show()\n", "\n", " top_share = top_novice_functions[\"share_of_all_novices\"].head(3).sum()\n", " print(\n", " f\"Novice population: {novice_n:,} people \"\n", " f\"({fmt(pct(novice_n, total_people))}% of the measured population).\\n\"\n", " )\n", " print(\n", " f\"The three largest functions by Novice User count hold \"\n", " f\"{fmt(top_share, '{:.0f}')}% of all Novice Users, so they are the most \"\n", " \"efficient place to start conversion outreach:\"\n", " )\n", " print(top_novice_functions[[\"FunctionType\", \"novice_people\", \"novice_pct_of_function\",\n", " \"share_of_all_novices\"]].to_string(index=False, formatters={\n", " \"novice_pct_of_function\": \"{:.1f}%\".format,\n", " \"share_of_all_novices\": \"{:.1f}%\".format,\n", " }))\n", "else:\n", " print(\n", " f\"Novice population: {novice_n:,} people \"\n", " f\"({fmt(pct(novice_n, total_people))}% of the measured population).\"\n", " )\n", " print(\n", " \"No function attribute is available, or no function has Novice Users above the \"\n", " \"reporting floor, so the targeting view is skipped.\"\n", " )\n" ] }, { "cell_type": "markdown", "id": "cell0015", "metadata": {}, "source": [ "## 5. Collaboration and working-pattern profile by usage segment\n", "\n", "`vi.keymetrics_scan()` is used as the primary descriptive comparison across Power,\n", "Habitual, Novice, Low, and Non-user segments.\n", "\n", "To keep the comparison like-for-like:\n", "\n", "- only people with a full rolling window of observed Copilot weeks are included;\n", "- each metric is first averaged to one row per person over that trailing window;\n", "- an explicit metric list is supplied for reproducibility, and any metric your export\n", " does not contain is dropped from the list rather than causing an error; and\n", "- the minimum segment size is `MIN_DISPLAY_N`, so a segment that is too small to report\n", " is excluded from the heatmap and from the narrative that follows.\n", "\n", "The heatmap is normalized **within each metric row**. Colour indicates which segment is\n", "relatively high or low for that metric, not whether the result is inherently favourable.\n", "\n", "A note on two metric names that are easy to misread:\n", "\n", "- `Collaboration_span` is an hours-based work-session metric, defined by Microsoft as the\n", " number of hours spent in work sessions before, during, and after working hours. It is\n", " relabelled below as **Work session span hours** so that it is not mistaken for network\n", " breadth.\n", "- Collaboration-network metrics such as internal network size, external network size,\n", " diverse ties, and strong ties are only included if your Person Query happens to contain\n", " them. The cell below reports exactly which metrics were found.\n", "\n", "See the [Microsoft Viva Insights metric reference](https://learn.microsoft.com/en-us/viva/insights/advanced/reference/metrics)\n", "for the full definitions.\n" ] }, { "cell_type": "code", "execution_count": null, "id": "cell0016", "metadata": {}, "outputs": [], "source": [ "candidate_scan_metrics = [\n", " \"Collaboration_hours\",\n", " \"Collaboration_span\",\n", " \"Active_connected_hours\",\n", " \"Meetings\",\n", " \"Meeting_hours\",\n", " \"Calls\",\n", " \"Call_hours\",\n", " \"Chats_sent\",\n", " \"Chat_hours\",\n", " \"Emails_sent\",\n", " \"Email_hours\",\n", " \"Multitasking_hours\",\n", " \"Available_to_focus_hours\",\n", " \"Uninterrupted_hours\",\n", " \"Interrupted_hours\",\n", " \"After_hours_collaboration_hours\",\n", " \"Time_with_leadership\",\n", " \"Internal_network_size\",\n", " \"External_network_size\",\n", "]\n", "scan_metrics = [metric for metric in candidate_scan_metrics if metric in df.columns]\n", "missing_scan_metrics = [m for m in candidate_scan_metrics if m not in df.columns]\n", "\n", "mature_ids = set(\n", " person_journey.loc[\n", " person_journey[\"observed_weeks\"] >= SEGMENT_WINDOW_WEEKS, \"PersonId\"\n", " ]\n", ")\n", "trailing_start = latest_week - pd.Timedelta(weeks=SEGMENT_WINDOW_WEEKS - 1)\n", "scan_data = (\n", " df[df[\"PersonId\"].isin(mature_ids) & (df[\"MetricDate\"] >= trailing_start)]\n", " .groupby(\"PersonId\")[scan_metrics]\n", " .mean()\n", " .reset_index()\n", " .merge(latest_segments[[\"PersonId\", \"UsageSegment_12w\"]],\n", " on=\"PersonId\", how=\"inner\")\n", ")\n", "scan_data = scan_data.rename(\n", " columns={\"Collaboration_span\": \"Work_session_span_hours\"}\n", ")\n", "scan_metrics = [\n", " \"Work_session_span_hours\" if metric == \"Collaboration_span\" else metric\n", " for metric in scan_metrics\n", "]\n", "scan_data[\"MetricDate\"] = latest_week\n", "scan_data[\"UsageSegment_12w\"] = pd.Categorical(\n", " scan_data[\"UsageSegment_12w\"], categories=segment_order, ordered=True\n", ")\n", "\n", "segment_scan_table = None\n", "reported_segments = []\n", "if not scan_metrics:\n", " print(\n", " \"None of the collaboration metrics used by this section are present in the \"\n", " \"export, so the key-metrics scan is skipped.\"\n", " )\n", "elif scan_data.empty:\n", " print(\n", " f\"No person has a full {SEGMENT_WINDOW_WEEKS}-week observed history, so the \"\n", " \"like-for-like key-metrics scan is skipped. Re-run with a longer export.\"\n", " )\n", "else:\n", " segment_scan_table = vi.keymetrics_scan(\n", " scan_data,\n", " hrvar=\"UsageSegment_12w\",\n", " mingroup=MIN_DISPLAY_N,\n", " metrics=scan_metrics,\n", " return_type=\"table\",\n", " )\n", " save_table(segment_scan_table, \"05_keymetrics_by_segment\")\n", " reported_segments = list(segment_scan_table[\"UsageSegment_12w\"])\n", "\n", " segment_scan_fig = vi.keymetrics_scan(\n", " scan_data,\n", " hrvar=\"UsageSegment_12w\",\n", " mingroup=MIN_DISPLAY_N,\n", " metrics=scan_metrics,\n", " return_type=\"plot\",\n", " low_color=\"#DCE6EE\",\n", " mid_color=\"#F4E4BD\",\n", " high_color=\"#C86B45\",\n", " textsize=8.5,\n", " plot_row_scaling_factor=0.52,\n", " )\n", " for figure_text in segment_scan_fig.texts:\n", " if figure_text.get_text().startswith(\"Data from\"):\n", " figure_text.set_text(\n", " f\"Person-level weekly averages from {trailing_start.date()} to \"\n", " f\"{latest_week.date()}; people with at least {SEGMENT_WINDOW_WEEKS} \"\n", " \"observed Copilot weeks.\"\n", " )\n", " save_figure(segment_scan_fig, \"05_keymetrics_by_segment\")\n", " plt.show()\n", "\n", " print(f\"Complete-history comparison: {scan_data['PersonId'].nunique():,} people\")\n", " print(f\"Metrics included ({len(scan_metrics)}): {', '.join(scan_metrics)}\")\n", " if missing_scan_metrics:\n", " print(\n", " f\"Metrics not present in this export ({len(missing_scan_metrics)}): \"\n", " f\"{', '.join(missing_scan_metrics)}\"\n", " )\n", " skipped_segments = [s for s in segment_order if s not in reported_segments]\n", " if skipped_segments:\n", " print(\n", " f\"Segments below the {MIN_DISPLAY_N}-person floor and therefore not \"\n", " f\"reported: {', '.join(skipped_segments)}\"\n", " )\n", " print()\n", " print(segment_scan_table.to_string(index=False))\n", "\n", "# Indexed lookup used by the narrative sections below. Reading through lookup() means a\n", "# segment that fell below the reporting floor produces \"not available\" rather than an error.\n", "scan_by_segment = (\n", " segment_scan_table.set_index(\"UsageSegment_12w\")\n", " if segment_scan_table is not None else None\n", ")\n" ] }, { "cell_type": "markdown", "id": "cell0017", "metadata": {}, "source": [ "## 6. Adjusted and process-level diagnostics\n", "\n", "The native key-metrics scan is the primary segment comparison. This section adds two\n", "diagnostics that the package scan does not provide:\n", "\n", "1. **Adjusted differences.** How Power and Habitual Users compare with everyone else on\n", " each collaboration metric, controlling for function and manager status. Results are in\n", " standard deviations, with 95% confidence intervals, so that metrics on different\n", " scales can be read on one axis.\n", "2. **Process indicators.** Median meeting length, after-hours share, meeting multitasking,\n", " and focus realisation, each shown in its own units (minutes, or a percentage) for each\n", " journey stage. These are deliberately not expressed as a percentage difference from the\n", " Non-user median: three of the four indicators are already percentages, and a percentage\n", " change in a percentage is easy to misread. Comparing the actual medians side by side\n", " needs no such translation.\n", "\n", "The two views answer different questions. The adjusted chart asks whether a gap survives\n", "once role is accounted for; the process panels show the plain, unadjusted levels a reader\n", "can sanity-check against their own experience. Read them together.\n", "\n", "A note on size. Significance and importance are not the same thing, and a large person\n", "count makes small differences statistically detectable. The printed output flags which\n", "adjusted differences also reach the `MATERIAL_EFFECT_SD` floor, which Section 7 explains\n", "in full.\n", "\n", "These remain observational associations rather than Copilot effects." ] }, { "cell_type": "code", "execution_count": null, "id": "cell0018", "metadata": {}, "outputs": [], "source": [ "recent_start = latest_week - pd.Timedelta(weeks=RECENT_WEEKS - 1)\n", "recent = copilot_pop[copilot_pop[\"MetricDate\"] >= recent_start].copy()\n", "recent = recent.merge(latest_segments, on=\"PersonId\", how=\"inner\")\n", "\n", "process_source_metrics = [\n", " \"Meetings\", \"Meeting_hours\", \"Collaboration_hours\",\n", " \"After_hours_collaboration_hours\", \"Multitasking_hours\",\n", " \"Available_to_focus_hours\", \"Uninterrupted_hours\",\n", "]\n", "for metric in process_source_metrics:\n", " if metric not in recent.columns:\n", " recent[metric] = np.nan\n", "\n", "\n", "def safe_rate(numerator, denominator, scale):\n", " return np.where(denominator > 0, numerator / denominator * scale, np.nan)\n", "\n", "\n", "recent[\"meeting_length_min\"] = safe_rate(\n", " recent[\"Meeting_hours\"], recent[\"Meetings\"], 60\n", ")\n", "recent[\"after_hours_share_pct\"] = safe_rate(\n", " recent[\"After_hours_collaboration_hours\"], recent[\"Collaboration_hours\"], 100\n", ")\n", "recent[\"meeting_multitask_share_pct\"] = safe_rate(\n", " recent[\"Multitasking_hours\"], recent[\"Meeting_hours\"], 100\n", ")\n", "recent[\"focus_realisation_pct\"] = safe_rate(\n", " recent[\"Uninterrupted_hours\"], recent[\"Available_to_focus_hours\"], 100\n", ")\n", "\n", "ratio_labels = {\n", " \"meeting_length_min\": \"Meeting length (minutes)\",\n", " \"after_hours_share_pct\": \"After-hours share of collaboration\",\n", " \"meeting_multitask_share_pct\": \"Meeting time spent multitasking\",\n", " \"focus_realisation_pct\": \"Available focus time uninterrupted\",\n", "}\n", "# Keep only the ratios this export can actually support.\n", "ratio_labels = {\n", " key: label for key, label in ratio_labels.items()\n", " if recent[key].notna().any()\n", "}\n", "ratio_cols = list(ratio_labels)\n", "\n", "process_summary = None\n", "if ratio_cols:\n", " person_ratios = (\n", " recent.groupby([\"PersonId\", \"JourneyStage\"])[ratio_cols].mean().reset_index()\n", " )\n", " stage_counts = person_ratios[\"JourneyStage\"].value_counts()\n", " reportable_stages = [\n", " stage for stage in journey_stage_order\n", " if stage_counts.get(stage, 0) >= MIN_PRIVACY_N\n", " ]\n", " process_summary = (\n", " person_ratios[person_ratios[\"JourneyStage\"].isin(reportable_stages)]\n", " .groupby(\"JourneyStage\")[ratio_cols].median()\n", " .reindex(journey_stage_order)\n", " .reset_index()\n", " )\n", " save_table(process_summary, \"06_process_ratio_medians\")\n", "else:\n", " print(\n", " \"None of the process ratios can be computed from this export, because the \"\n", " \"underlying meeting, focus, or after-hours metrics are absent.\"\n", " )\n", "\n", "level_metrics = {\n", " \"Collaboration_hours\": \"Collaboration hours\",\n", " \"Meetings\": \"Meetings\",\n", " \"Meeting_hours\": \"Meeting hours\",\n", " \"Chats_sent\": \"Chats sent\",\n", " \"Emails_sent\": \"Emails sent\",\n", " \"Active_connected_hours\": \"Active connected hours\",\n", " \"Multitasking_hours\": \"Multitasking hours\",\n", " \"Available_to_focus_hours\": \"Available-to-focus hours\",\n", " \"Uninterrupted_hours\": \"Uninterrupted hours\",\n", " \"After_hours_collaboration_hours\": \"After-hours collaboration\",\n", "}\n", "level_metrics = {\n", " metric: label for metric, label in level_metrics.items()\n", " if metric in recent.columns and recent[metric].notna().any()\n", "}\n", "\n", "adjusted = None\n", "if not level_metrics:\n", " print(\"No collaboration level metrics are available, so the adjusted model is skipped.\")\n", "else:\n", " person_levels = (\n", " recent.groupby([\"PersonId\", \"JourneyStage\"])[list(level_metrics)].mean().reset_index()\n", " .merge(latest_attributes[[\"PersonId\", \"FunctionType\", \"ManagerStatus\"]],\n", " on=\"PersonId\", how=\"left\")\n", " )\n", " person_levels[\"power_habitual\"] = (\n", " person_levels[\"JourneyStage\"] == \"Power + Habitual\"\n", " ).astype(int)\n", " person_levels[\"FunctionType\"] = person_levels[\"FunctionType\"].fillna(\"Missing\")\n", " person_levels[\"ManagerStatus\"] = person_levels[\"ManagerStatus\"].fillna(\"Unknown\")\n", "\n", " # Only control for attributes that actually vary, otherwise the dummy block is\n", " # perfectly collinear with the intercept.\n", " control_columns = [\n", " column for column in [\"FunctionType\", \"ManagerStatus\"]\n", " if person_levels[column].nunique() > 1\n", " ]\n", " control_note = (\n", " f\"controlling for {' and '.join(control_columns)}\" if control_columns\n", " else \"unadjusted, because no organizational attribute varies in this export\"\n", " )\n", " parts = [person_levels[[\"power_habitual\"]]]\n", " if control_columns:\n", " parts.append(pd.get_dummies(person_levels[control_columns],\n", " drop_first=True, dtype=float))\n", " controls = sm.add_constant(pd.concat(parts, axis=1).astype(float))\n", "\n", " adjusted_rows = []\n", " for metric, label in level_metrics.items():\n", " ok = person_levels[metric].notna()\n", " y = person_levels.loc[ok, metric].astype(float)\n", " if ok.sum() < 2 * MIN_DISPLAY_N or y.std() == 0:\n", " continue\n", " if person_levels.loc[ok, \"power_habitual\"].nunique() < 2:\n", " continue\n", " y_z = (y - y.mean()) / y.std()\n", " model = sm.OLS(y_z, controls.loc[ok]).fit(cov_type=\"HC3\")\n", " beta = float(model.params[\"power_habitual\"])\n", " se = float(model.bse[\"power_habitual\"])\n", " adjusted_rows.append({\n", " \"metric\": metric,\n", " \"label\": label,\n", " \"n\": int(ok.sum()),\n", " \"adjusted_difference_sd\": beta,\n", " \"ci_low\": beta - 1.96 * se,\n", " \"ci_high\": beta + 1.96 * se,\n", " \"p_value\": float(model.pvalues[\"power_habitual\"]),\n", " })\n", "\n", " if not adjusted_rows:\n", " print(\n", " \"No metric had enough variation and sample size to fit the adjusted model, \"\n", " \"so this comparison is skipped.\"\n", " )\n", " else:\n", " adjusted = pd.DataFrame(adjusted_rows).sort_values(\"adjusted_difference_sd\")\n", " save_table(adjusted, \"06_adjusted_working_pattern_associations\")\n", "\n", "ratio_units = {\n", " \"meeting_length_min\": (\"minutes\", \"{:.0f}\"),\n", " \"after_hours_share_pct\": (\"% of collaboration\", \"{:.1f}\"),\n", " \"meeting_multitask_share_pct\": (\"% of meeting time\", \"{:.1f}\"),\n", " \"focus_realisation_pct\": (\"% of available focus time\", \"{:.1f}\"),\n", "}\n", "\n", "if adjusted is not None:\n", " fig, ax = plt.subplots(figsize=(9.5, max(3.4, 0.45 * len(adjusted) + 1.8)))\n", " ypos = np.arange(len(adjusted))\n", " ax.errorbar(\n", " adjusted[\"adjusted_difference_sd\"], ypos,\n", " xerr=[\n", " adjusted[\"adjusted_difference_sd\"] - adjusted[\"ci_low\"],\n", " adjusted[\"ci_high\"] - adjusted[\"adjusted_difference_sd\"],\n", " ],\n", " fmt=\"o\", color=journey_stage_color_map[\"Power + Habitual\"], ecolor=C_GREY,\n", " capsize=3,\n", " )\n", " ax.axvline(0, color=C_TEXT, lw=1)\n", " ax.set_yticks(ypos)\n", " ax.set_yticklabels(adjusted[\"label\"])\n", " ax.set_xlabel(\"Adjusted difference (standard deviations)\")\n", " ax.set_title(\"Power and Habitual Users versus others, adjusted\",\n", " loc=\"left\", fontweight=\"bold\")\n", " add_subtitle(ax, f\"OLS on person-level weekly means, {control_note}; bars are 95% CI\")\n", " ax.grid(axis=\"y\", visible=False)\n", " fig.tight_layout()\n", " save_figure(fig, \"06_adjusted_working_patterns\")\n", " plt.show()\n", "\n", "# Process indicators are shown in their own units, one panel per indicator, rather than\n", "# as a percentage difference from the Non-user median. A relative view of an indicator\n", "# that is itself a percentage produces a percentage of a percentage, which is easy to\n", "# misread; the actual medians are directly comparable and need no explanation.\n", "if process_summary is not None and ratio_cols:\n", " plot_ratios = [\n", " column for column in ratio_cols\n", " if process_summary[column].notna().sum() >= 2\n", " ]\n", " if plot_ratios:\n", " n_panels = len(plot_ratios)\n", " fig, axes = plt.subplots(1, n_panels, figsize=(3.6 * n_panels, 4.3),\n", " squeeze=False)\n", " stage_values = process_summary.set_index(\"JourneyStage\")\n", " stages = [s for s in journey_stage_order if s in stage_values.index]\n", " for ax, column in zip(axes[0], plot_ratios):\n", " unit, number_format = ratio_units.get(column, (\"\", \"{:.1f}\"))\n", " values = [stage_values.loc[stage, column] for stage in stages]\n", " bars = ax.bar(\n", " range(len(stages)), values, width=0.62,\n", " color=[journey_stage_color_map[stage] for stage in stages],\n", " )\n", " ax.set_xticks(range(len(stages)))\n", " ax.set_xticklabels(\n", " [stage.replace(\" + \", \" +\\n\") for stage in stages], fontsize=8.5\n", " )\n", " ax.set_title(ratio_labels[column], loc=\"left\", fontweight=\"bold\", fontsize=10)\n", " ax.set_ylabel(unit, fontsize=8.5)\n", " top = np.nanmax(values) if np.isfinite(np.nanmax(values)) else 1\n", " ax.set_ylim(0, top * 1.22)\n", " ax.grid(axis=\"x\", visible=False)\n", " for bar, value in zip(bars, values):\n", " if pd.notna(value):\n", " ax.annotate(\n", " number_format.format(value),\n", " (bar.get_x() + bar.get_width() / 2, bar.get_height()),\n", " xytext=(0, 4), textcoords=\"offset points\",\n", " ha=\"center\", fontsize=9,\n", " )\n", " fig.suptitle(\n", " f\"Median process indicators by journey stage, last {RECENT_WEEKS} weeks\",\n", " x=0.01, ha=\"left\", fontsize=13, fontweight=\"bold\",\n", " )\n", " fig.text(\n", " 0.01, -0.04,\n", " \"Each panel is a median across people, shown in its own units. These are \"\n", " \"unadjusted group medians, so read them alongside the adjusted chart above.\",\n", " fontsize=8.5, color=C_GREY,\n", " )\n", " fig.tight_layout()\n", " save_figure(fig, \"06_process_indicators\")\n", " plt.show()\n", "\n", "if adjusted is not None:\n", " print(f\"Adjusted Power + Habitual associations ({control_note}):\")\n", " print(adjusted[[\"label\", \"n\", \"adjusted_difference_sd\", \"ci_low\", \"ci_high\",\n", " \"p_value\"]].to_string(index=False, formatters={\n", " \"adjusted_difference_sd\": \"{:+.3f}\".format,\n", " \"ci_low\": \"{:+.3f}\".format,\n", " \"ci_high\": \"{:+.3f}\".format,\n", " \"p_value\": \"{:.3g}\".format,\n", " }))\n", " significant = adjusted[adjusted[\"p_value\"] < 0.05]\n", " material = significant[\n", " significant[\"adjusted_difference_sd\"].abs() >= MATERIAL_EFFECT_SD\n", " ]\n", " print(\n", " f\"\\n{len(significant)} of {len(adjusted)} metrics differ significantly at the \"\n", " f\"5% level after adjustment, and {len(material)} of those reach the \"\n", " f\"{MATERIAL_EFFECT_SD} SD materiality floor. Metrics that do neither are \"\n", " \"reported above rather than dropped, so that null results stay visible.\"\n", " )\n", "\n", "if process_summary is not None:\n", " print(f\"\\nMedian process indicators over the latest {RECENT_WEEKS} weeks:\")\n", " print(process_summary.to_string(index=False))\n", "\n", "# Indexed lookups used by the narrative sections below.\n", "process_by_stage = (\n", " process_summary.set_index(\"JourneyStage\") if process_summary is not None else None\n", ")\n", "adjusted_by_metric = adjusted.set_index(\"metric\") if adjusted is not None else None\n" ] }, { "cell_type": "markdown", "id": "cell0019", "metadata": {}, "source": [ "## 7. Same-person check: what changes in heavier Copilot-use weeks?\n", "\n", "This view compares each person with themselves and removes common week effects. It controls\n", "for stable individual differences such as role propensity, but it still cannot distinguish\n", "Copilot effects from unusually demanding weeks.\n", "\n", "The coefficient is the standard-deviation change in the process indicator associated with\n", "a one-standard-deviation increase in `log(1 + actions)`.\n", "\n", "### How to read this, and when to conclude anything\n", "\n", "Two tests have to be passed before a result here is worth acting on, and they are\n", "different questions.\n", "\n", "**1. Is it distinguishable from zero?** The horizontal bar is the 95% confidence interval.\n", "If any part of it touches the zero line, the data cannot tell the difference between the\n", "observed association and no association at all. It does not matter how far left or right\n", "the dot sits: an interval that crosses zero is a null result, and should be reported as\n", "one.\n", "\n", "**2. Is it big enough to matter?** This is the test that is easy to skip. These models run\n", "on tens of thousands of person-weeks, and with a sample that large almost anything becomes\n", "statistically significant eventually. A coefficient of `-0.01` SD can have a confidence\n", "interval comfortably clear of zero while describing a change far too small to notice, let\n", "alone manage. The shaded band on the chart marks the region below `MATERIAL_EFFECT_SD`\n", "(0.1 standard deviations by default), which is the conventional floor for a small effect.\n", "A result inside that band is real but negligible.\n", "\n", "So, to answer the question directly: conclude that heavier Copilot use is associated with\n", "lower after-hours work only when the **entire** interval sits left of zero **and** the dot\n", "sits outside the shaded band. If the interval crosses zero, the honest description is \"no\n", "detectable association\", however suggestive the direction looks.\n", "\n", "The cell below labels each result against both tests, so the verdict does not depend on\n", "eyeballing the chart.\n", "\n", "**And even then it is not causal.** A material, statistically clear coefficient here still\n", "only says that heavier-use weeks look different from lighter-use weeks for the same person.\n", "Busy weeks plausibly drive both the Copilot usage and the process change. Establishing\n", "direction needs a design built for it, such as the\n", "[event-study and difference-in-differences examples](https://microsoft.github.io/viva-insights-sample-code/copilot/#event-study--difference-in-differences)\n", "or the [Copilot Causal Toolkit](https://microsoft.github.io/viva-insights-sample-code/copilot-causal-toolkit/).\n" ] }, { "cell_type": "code", "execution_count": null, "id": "cell0020", "metadata": {}, "outputs": [], "source": [ "within_data = copilot_pop.copy()\n", "within_data[\"Total_Copilot_actions_taken\"] = (\n", " within_data[\"Total_Copilot_actions_taken\"].fillna(0)\n", ")\n", "for metric in process_source_metrics:\n", " if metric not in within_data.columns:\n", " within_data[metric] = np.nan\n", "within_data[\"meeting_length_min\"] = safe_rate(\n", " within_data[\"Meeting_hours\"], within_data[\"Meetings\"], 60\n", ")\n", "within_data[\"after_hours_share_pct\"] = safe_rate(\n", " within_data[\"After_hours_collaboration_hours\"],\n", " within_data[\"Collaboration_hours\"], 100\n", ")\n", "within_data[\"meeting_multitask_share_pct\"] = safe_rate(\n", " within_data[\"Multitasking_hours\"], within_data[\"Meeting_hours\"], 100\n", ")\n", "within_data[\"focus_realisation_pct\"] = safe_rate(\n", " within_data[\"Uninterrupted_hours\"], within_data[\"Available_to_focus_hours\"], 100\n", ")\n", "\n", "\n", "def two_way_demean(values, person, week, iterations=10):\n", " result = values.astype(float).copy()\n", " for _ in range(iterations):\n", " result = result - result.groupby(person).transform(\"mean\")\n", " result = result - result.groupby(week).transform(\"mean\")\n", " return result\n", "\n", "\n", "within_rows = []\n", "for metric, label in ratio_labels.items():\n", " work = within_data[[\n", " \"PersonId\", \"MetricDate\", \"Total_Copilot_actions_taken\", metric\n", " ]].dropna().copy()\n", " if work.empty:\n", " continue\n", " work[\"log_actions\"] = np.log1p(work[\"Total_Copilot_actions_taken\"])\n", " work[\"x_within\"] = two_way_demean(\n", " work[\"log_actions\"], work[\"PersonId\"], work[\"MetricDate\"]\n", " )\n", " work[\"y_within\"] = two_way_demean(\n", " work[metric], work[\"PersonId\"], work[\"MetricDate\"]\n", " )\n", " x_sd, y_sd = work[\"x_within\"].std(), work[\"y_within\"].std()\n", " if len(work) < 2 * MIN_DISPLAY_N or pd.isna(x_sd) or pd.isna(y_sd) or x_sd == 0 or y_sd == 0:\n", " continue\n", " model = sm.OLS(\n", " work[\"y_within\"] / y_sd,\n", " sm.add_constant(work[\"x_within\"] / x_sd),\n", " ).fit(cov_type=\"cluster\", cov_kwds={\"groups\": work[\"PersonId\"]})\n", " beta = float(model.params[\"x_within\"])\n", " se = float(model.bse[\"x_within\"])\n", " within_rows.append({\n", " \"metric\": metric,\n", " \"label\": label,\n", " \"n_person_weeks\": len(work),\n", " \"beta_sd\": beta,\n", " \"ci_low\": beta - 1.96 * se,\n", " \"ci_high\": beta + 1.96 * se,\n", " \"p_value\": float(model.pvalues[\"x_within\"]),\n", " })\n", "\n", "within_results = None\n", "if not within_rows:\n", " print(\n", " \"No process ratio has enough person-week observations for the same-person \"\n", " \"check, so this section is skipped.\"\n", " )\n", "else:\n", " within_results = pd.DataFrame(within_rows).sort_values(\"beta_sd\")\n", " # Classify each result against both tests: is it distinguishable from zero, and is\n", " # it large enough to matter. The verdict is stored so that the chart, the printed\n", " # narrative, and the executive summary cannot disagree with one another.\n", " crosses_zero = (within_results[\"ci_low\"] <= 0) & (within_results[\"ci_high\"] >= 0)\n", " is_material = within_results[\"beta_sd\"].abs() >= MATERIAL_EFFECT_SD\n", " within_results[\"verdict\"] = np.select(\n", " [crosses_zero, ~crosses_zero & ~is_material],\n", " [\"No detectable association\", \"Detectable but negligible\"],\n", " default=\"Detectable and material\",\n", " )\n", " within_results[\"direction\"] = np.where(\n", " within_results[\"beta_sd\"] > 0, \"higher\", \"lower\"\n", " )\n", " save_table(within_results, \"07_within_person_process_associations\")\n", "\n", " verdict_style = {\n", " \"Detectable and material\": dict(color=C_TEAL, mfc=C_TEAL),\n", " \"Detectable but negligible\": dict(color=C_TEAL, mfc=\"white\"),\n", " \"No detectable association\": dict(color=C_GREY, mfc=\"white\"),\n", " }\n", "\n", " fig, ax = plt.subplots(figsize=(10.5, max(3.0, 0.62 * len(within_results) + 2.4)))\n", " ypos = np.arange(len(within_results))\n", "\n", " # Region of practical negligibility: anything landing inside it is too small to act\n", " # on, whatever its p-value.\n", " ax.axvspan(-MATERIAL_EFFECT_SD, MATERIAL_EFFECT_SD, color=C_LIGHT, zorder=0)\n", " ax.axvline(0, color=C_TEXT, lw=1.2, zorder=1)\n", "\n", " for y, row in zip(ypos, within_results.itertuples()):\n", " style = verdict_style[row.verdict]\n", " ax.errorbar(\n", " row.beta_sd, y,\n", " xerr=[[row.beta_sd - row.ci_low], [row.ci_high - row.beta_sd]],\n", " fmt=\"o\", capsize=4, lw=1.8, markersize=7,\n", " color=style[\"color\"], ecolor=style[\"color\"], markerfacecolor=style[\"mfc\"],\n", " zorder=3,\n", " )\n", "\n", " ax.set_yticks(ypos)\n", " ax.set_yticklabels(within_results[\"label\"])\n", " ax.set_ylim(-0.7, len(within_results) - 0.3)\n", " ax.set_xlabel(\"Same-person, week-adjusted association (standard deviations)\")\n", " ax.set_title(\"Process indicators in heavier Copilot-use weeks\", loc=\"left\",\n", " fontweight=\"bold\")\n", " add_subtitle(ax, \"Change per 1 SD increase in log(1 + weekly Copilot actions), person and week effects removed\")\n", " ax.grid(axis=\"y\", visible=False)\n", "\n", " # Keep the negligibility band visible even when every interval is tiny.\n", " span = max(\n", " float(within_results[\"ci_high\"].abs().max()),\n", " float(within_results[\"ci_low\"].abs().max()),\n", " MATERIAL_EFFECT_SD * 1.6,\n", " )\n", " ax.set_xlim(-span * 1.35, span * 1.35)\n", "\n", " for y, row in zip(ypos, within_results.itertuples()):\n", " ax.annotate(\n", " f\"{row.beta_sd:+.3f} SD [{row.ci_low:+.3f}, {row.ci_high:+.3f}]\",\n", " (ax.get_xlim()[1], y), xytext=(-6, 0), textcoords=\"offset points\",\n", " ha=\"right\", va=\"center\", fontsize=7.5, color=C_GREY,\n", " )\n", "\n", " legend_handles = [\n", " plt.Line2D([], [], marker=\"o\", linestyle=\"none\", markersize=7,\n", " color=style[\"color\"], markerfacecolor=style[\"mfc\"], label=verdict)\n", " for verdict, style in verdict_style.items()\n", " if verdict in set(within_results[\"verdict\"])\n", " ]\n", " legend_handles.append(\n", " plt.Rectangle((0, 0), 1, 1, color=C_LIGHT,\n", " label=f\"Negligible zone (< {MATERIAL_EFFECT_SD} SD)\")\n", " )\n", " ax.legend(handles=legend_handles, frameon=False, fontsize=8,\n", " loc=\"upper left\", bbox_to_anchor=(0, -0.16), ncol=4,\n", " handletextpad=0.5, columnspacing=1.4)\n", " fig.tight_layout()\n", " save_figure(fig, \"07_within_person_process\")\n", " plt.show()\n", "\n", " print(within_results[[\n", " \"label\", \"n_person_weeks\", \"beta_sd\", \"ci_low\", \"ci_high\", \"p_value\", \"verdict\",\n", " ]].to_string(index=False, formatters={\n", " \"beta_sd\": \"{:+.3f}\".format,\n", " \"ci_low\": \"{:+.3f}\".format,\n", " \"ci_high\": \"{:+.3f}\".format,\n", " \"p_value\": \"{:.3g}\".format,\n", " }))\n", "\n", " print(\"\\nWhat each result supports:\")\n", " for row in within_results.itertuples():\n", " if row.verdict == \"No detectable association\":\n", " print(\n", " f\" {row.label}: no detectable association. The 95% interval \"\n", " f\"[{row.ci_low:+.3f}, {row.ci_high:+.3f}] includes zero, so the \"\n", " f\"{row.direction} direction of the point estimate is not supported.\"\n", " )\n", " elif row.verdict == \"Detectable but negligible\":\n", " print(\n", " f\" {row.label}: statistically clear but negligible. The interval \"\n", " f\"excludes zero, yet {row.beta_sd:+.3f} SD sits inside the \"\n", " f\"{MATERIAL_EFFECT_SD} SD negligible zone, which is a difference too \"\n", " \"small to manage against.\"\n", " )\n", " else:\n", " print(\n", " f\" {row.label}: {row.direction} in heavier-use weeks, by \"\n", " f\"{abs(row.beta_sd):.3f} SD [{row.ci_low:+.3f}, {row.ci_high:+.3f}]. \"\n", " \"This clears both the significance and the materiality bar, though it \"\n", " \"remains an association rather than an effect.\"\n", " )\n", "\n", " material_count = int((within_results[\"verdict\"] == \"Detectable and material\").sum())\n", " if material_count == 0:\n", " print(\n", " \"\\nNo indicator clears both bars. On this data, the honest headline is that \"\n", " \"heavier Copilot-use weeks do not look meaningfully different from lighter \"\n", " \"weeks for the same person, and none of these directions should be reported \"\n", " \"as a finding.\"\n", " )\n", " else:\n", " print(\n", " f\"\\n{material_count} of {len(within_results)} indicators clear both the \"\n", " \"significance and materiality bars. Treat the rest as null results.\"\n", " )\n", "\n", "within_by_metric = (\n", " within_results.set_index(\"metric\") if within_results is not None else None\n", ")\n" ] }, { "cell_type": "markdown", "id": "cell0021", "metadata": {}, "source": [ "## 8. Evidence-led story synthesis\n", "\n", "This section translates the analytical outputs into a reusable evidence matrix for\n", "executive reporting. Each insight records the supporting numbers, the interpretation,\n", "and the principal caveat.\n", "\n", "The synthesis deliberately distinguishes **information-exchange intensity** from\n", "information-flow speed or network breadth. A Person Query contains no direct measure of\n", "information velocity, and collaboration-network metrics are only available if your query\n", "included them, so the wording below stays within what the data can support.\n", "\n", "Each insight is assembled from the values computed above rather than written in advance,\n", "so the direction of every statement follows your data. Where an input is missing, the\n", "insight is omitted rather than asserted." ] }, { "cell_type": "code", "execution_count": null, "id": "cell0022", "metadata": {}, "outputs": [], "source": [ "segment_counts = segment_summary.set_index(\"segment\")\n", "\n", "\n", "def describe(value, rising, falling, flat, tolerance=1.0):\n", " \"\"\"Pick wording from the computed value, so the narrative follows the data.\"\"\"\n", " if pd.isna(value):\n", " return flat\n", " if value > tolerance:\n", " return rising\n", " if value < -tolerance:\n", " return falling\n", " return flat\n", "\n", "\n", "def relative_gap(metric, group=\"Power User\", reference=\"Non-user\"):\n", " \"\"\"Percentage difference between two segments, or NaN when either is unreported.\"\"\"\n", " group_value = lookup(scan_by_segment, group, metric)\n", " reference_value = lookup(scan_by_segment, reference, metric)\n", " if pd.isna(group_value) or pd.isna(reference_value) or not reference_value:\n", " return np.nan\n", " return 100 * (group_value / reference_value - 1)\n", "\n", "\n", "gap_metrics = {\n", " \"Collaboration_hours\": \"Collaboration hours\",\n", " \"Work_session_span_hours\": \"Work-session span hours\",\n", " \"Active_connected_hours\": \"Active connected hours\",\n", " \"Meetings\": \"Meetings\",\n", " \"Meeting_hours\": \"Meeting hours\",\n", " \"Calls\": \"Calls\",\n", " \"Chats_sent\": \"Chats sent\",\n", " \"Emails_sent\": \"Emails sent\",\n", " \"Multitasking_hours\": \"Multitasking hours\",\n", " \"Available_to_focus_hours\": \"Available-to-focus hours\",\n", " \"Uninterrupted_hours\": \"Uninterrupted hours\",\n", " \"After_hours_collaboration_hours\": \"After-hours collaboration hours\",\n", " \"Time_with_leadership\": \"Time with leadership\",\n", "}\n", "if scan_by_segment is not None:\n", " gap_metrics = {\n", " metric: label for metric, label in gap_metrics.items()\n", " if metric in scan_by_segment.columns\n", " }\n", " gap_rows = []\n", " for metric, label in gap_metrics.items():\n", " for group in [\"Power User\", \"Habitual User\"]:\n", " gap_rows.append({\n", " \"metric\": metric,\n", " \"label\": label,\n", " \"segment\": group,\n", " \"segment_value\": lookup(scan_by_segment, group, metric),\n", " \"non_user_value\": lookup(scan_by_segment, \"Non-user\", metric),\n", " \"difference_pct\": relative_gap(metric, group=group),\n", " })\n", " segment_gap_table = pd.DataFrame(gap_rows)\n", " save_table(segment_gap_table, \"08_segment_gaps_vs_non_users\")\n", "else:\n", " segment_gap_table = None\n", " gap_metrics = {}\n", "\n", "sustained_n = int(\n", " segment_counts.loc[\"Power User\", \"people\"]\n", " + segment_counts.loc[\"Habitual User\", \"people\"]\n", ")\n", "sustained_pct = pct(sustained_n, int(segment_summary[\"people\"].sum()))\n", "novice_pct = float(segment_counts.loc[\"Novice User\", \"pct\"])\n", "\n", "manager_lookup = (\n", " manager_summary.set_index(\"ManagerStatus\") if manager_summary is not None else None\n", ")\n", "manager_sustained = lookup(manager_lookup, \"Manager\", \"power_habitual_pct\")\n", "ic_sustained = lookup(manager_lookup, \"IC\", \"power_habitual_pct\")\n", "\n", "insights = []\n", "\n", "# 1. Scale: reach growth against depth growth.\n", "reach_delta = last_row[\"copilot_population_pct\"] - first_row[\"copilot_population_pct\"]\n", "depth_delta_pct = (\n", " 100 * (last_row[\"actions_per_active\"] / first_row[\"actions_per_active\"] - 1)\n", " if first_row[\"actions_per_active\"] else np.nan\n", ")\n", "insights.append({\n", " \"theme\": \"Scale\",\n", " \"insight\": (\n", " f\"The {POPULATION_LABEL_LC} \"\n", " + describe(reach_delta, \"expanded\", \"contracted\", \"held steady\")\n", " + \", while depth among active users \"\n", " + describe(depth_delta_pct, \"rose\", \"fell\", \"stayed broadly stable\", tolerance=5)\n", " + \".\"\n", " ),\n", " \"evidence\": (\n", " f\"Reach moved from {fmt(first_row['copilot_population_pct'])}% to \"\n", " f\"{fmt(last_row['copilot_population_pct'])}% of the measured population; \"\n", " f\"actions per active user moved from \"\n", " f\"{fmt(first_row['actions_per_active'])} to \"\n", " f\"{fmt(last_row['actions_per_active'])} ({fmt(depth_delta_pct, '{:+.0f}')}%).\"\n", " ),\n", " \"implication\": describe(\n", " reach_delta - (depth_delta_pct if pd.notna(depth_delta_pct) else 0),\n", " \"The near-term opportunity is activation and habit formation rather than extending reach further.\",\n", " \"Depth is growing faster than reach, so extending reach is the larger remaining opportunity.\",\n", " \"Reach and depth are moving together, so no single lever stands out.\",\n", " tolerance=5,\n", " ),\n", " \"caveat\": (\n", " f\"The Copilot population is {POPULATION_BASIS}.\"\n", " ),\n", "})\n", "\n", "# 2. Habit: the size of the conversion pool.\n", "insights.append({\n", " \"theme\": \"Habit\",\n", " \"insight\": (\n", " f\"The Novice population is \"\n", " + describe(novice_pct - sustained_pct,\n", " \"larger than the sustained-user base, so there is substantial headroom\",\n", " \"smaller than the sustained-user base, so adoption is already consolidating\",\n", " \"comparable to the sustained-user base\")\n", " + \".\"\n", " ),\n", " \"evidence\": (\n", " f\"{fmt(sustained_pct)}% are Power or Habitual Users, while {fmt(novice_pct)}% \"\n", " \"are Novice Users.\"\n", " ),\n", " \"implication\": \"Targeted use-case reinforcement is the lever that moves Novice Users into repeat use.\",\n", " \"caveat\": \"Recent entrants have incomplete rolling histories and may be understated.\",\n", "})\n", "\n", "# 3. Onboarding: only when at least two cohorts clear the display floor.\n", "if len(plot_cohorts) >= 2:\n", " oldest_cohort, newest_cohort = plot_cohorts.iloc[0], plot_cohorts.iloc[-1]\n", " cohort_gap = oldest_cohort[\"active_latest_pct\"] - newest_cohort[\"active_latest_pct\"]\n", " insights.append({\n", " \"theme\": \"Onboarding\",\n", " \"insight\": (\n", " \"People who joined the Copilot population recently activate \"\n", " + describe(cohort_gap, \"more slowly than established cohorts\",\n", " \"faster than established cohorts\",\n", " \"at a similar rate to established cohorts\", tolerance=5)\n", " + \".\"\n", " ),\n", " \"evidence\": (\n", " f\"The earliest qualifying cohort is {fmt(oldest_cohort['active_latest_pct'])}% \"\n", " f\"active in the latest week versus \"\n", " f\"{fmt(newest_cohort['active_latest_pct'])}% for the newest.\"\n", " ),\n", " \"implication\": \"Onboarding is best measured as a cohort conversion journey rather than a one-time launch.\",\n", " \"caveat\": \"The joining date may reflect licensing, query scope, or telemetry availability rather than a rollout.\",\n", " })\n", "\n", "# 4. Leadership: only when the export distinguishes managers from ICs.\n", "if pd.notna(manager_sustained) and pd.notna(ic_sustained):\n", " manager_gap = manager_sustained - ic_sustained\n", " insights.append({\n", " \"theme\": \"Leadership\",\n", " \"insight\": (\n", " \"Sustained adoption is \"\n", " + describe(manager_gap, \"higher among managers\", \"higher among individual contributors\",\n", " \"similar across managers and individual contributors\", tolerance=2)\n", " + \".\"\n", " ),\n", " \"evidence\": (\n", " f\"{fmt(manager_sustained)}% of managers are Power or Habitual Users versus \"\n", " f\"{fmt(ic_sustained)}% of individual contributors.\"\n", " ),\n", " \"implication\": describe(\n", " manager_gap,\n", " \"Managers can sponsor adoption, and individual-contributor use cases need deliberate reinforcement.\",\n", " \"Adoption is bottom-up, so manager enablement is the gap to close.\",\n", " \"Enablement can be designed for one audience rather than split by seniority.\",\n", " tolerance=2,\n", " ),\n", " \"caveat\": \"Manager roles are structurally more collaboration intensive, which confounds the comparison.\",\n", " })\n", "\n", "# 5. Functional variation: only when create_rank produced a function ranking.\n", "if rank_sustained is not None:\n", " function_rank = rank_sustained[rank_sustained[\"hrvar\"] == \"FunctionType\"]\n", " if len(function_rank) >= 2:\n", " top_function, bottom_function = function_rank.iloc[0], function_rank.iloc[-1]\n", " spread = top_function[\"metric\"] - bottom_function[\"metric\"]\n", " insights.append({\n", " \"theme\": \"Functional variation\",\n", " \"insight\": (\n", " \"Power + Habitual adoption \"\n", " + describe(spread, \"varies widely by function\", \"varies widely by function\",\n", " \"is fairly even across functions\", tolerance=10)\n", " + \".\"\n", " ),\n", " \"evidence\": (\n", " f\"{top_function['attributes']} leads qualifying functions at \"\n", " f\"{fmt(top_function['metric'])}% versus {bottom_function['attributes']} \"\n", " f\"at {fmt(bottom_function['metric'])}%, a spread of \"\n", " f\"{fmt(spread, '{:.0f}')} points.\"\n", " ),\n", " \"implication\": describe(\n", " spread,\n", " \"The next enablement wave should be role-specific rather than enterprise-generic.\",\n", " \"The next enablement wave should be role-specific rather than enterprise-generic.\",\n", " \"A single enterprise-wide enablement approach is defensible here.\",\n", " tolerance=10,\n", " ),\n", " \"caveat\": \"These are descriptive group differences, not performance rankings.\",\n", " })\n", "\n", "# 6 to 8. Segment gaps, only for metrics the scan actually reported.\n", "if scan_by_segment is not None:\n", " collab_gap = relative_gap(\"Collaboration_hours\")\n", " if pd.notna(collab_gap):\n", " insights.append({\n", " \"theme\": \"Information exchange\",\n", " \"insight\": (\n", " \"Power Users operate in a \"\n", " + describe(collab_gap, \"more\", \"less\", \"similarly\", tolerance=5)\n", " + \" collaboration-intensive environment than Non-users.\"\n", " ),\n", " \"evidence\": (\n", " f\"Power Users average \"\n", " f\"{fmt(lookup(scan_by_segment, 'Power User', 'Collaboration_hours'))} \"\n", " f\"collaboration hours versus \"\n", " f\"{fmt(lookup(scan_by_segment, 'Non-user', 'Collaboration_hours'))} for \"\n", " f\"Non-users ({fmt(collab_gap, '{:+.0f}')}%).\"\n", " ),\n", " \"implication\": \"Copilot is most embedded where the volume of information exchange is highest.\",\n", " \"caveat\": \"The data measures activity volume, not information-flow speed or quality.\",\n", " })\n", "\n", " span_gap = relative_gap(\"Work_session_span_hours\")\n", " if pd.notna(span_gap):\n", " insights.append({\n", " \"theme\": \"Workday intensity\",\n", " \"insight\": (\n", " \"Higher usage coincides with \"\n", " + describe(span_gap, \"longer\", \"shorter\", \"similar\", tolerance=3)\n", " + \" work-session spans.\"\n", " ),\n", " \"evidence\": (\n", " f\"Power Users average \"\n", " f\"{fmt(lookup(scan_by_segment, 'Power User', 'Work_session_span_hours'))} \"\n", " f\"work-session span hours, {fmt(span_gap, '{:+.0f}')}% versus Non-users.\"\n", " ),\n", " \"implication\": \"Copilot adoption is concentrated in demanding roles and work patterns.\",\n", " \"caveat\": \"Work-session span is an hours metric, not network breadth.\",\n", " })\n", "\n", " multitask_gap = relative_gap(\"Multitasking_hours\")\n", " if pd.notna(multitask_gap):\n", " insights.append({\n", " \"theme\": \"Focus\",\n", " \"insight\": (\n", " \"Multitasking is \"\n", " + describe(multitask_gap, \"higher\", \"lower\", \"comparable\", tolerance=5)\n", " + \" among Power Users than Non-users.\"\n", " ),\n", " \"evidence\": (\n", " f\"Power Users average \"\n", " f\"{fmt(lookup(scan_by_segment, 'Power User', 'Multitasking_hours'))} \"\n", " f\"multitasking hours versus \"\n", " f\"{fmt(lookup(scan_by_segment, 'Non-user', 'Multitasking_hours'))} \"\n", " f\"({fmt(multitask_gap, '{:+.0f}')}%).\"\n", " ),\n", " \"implication\": \"Copilot enablement is best paired with meeting, asynchronous-work, and focus-time practices.\",\n", " \"caveat\": \"High-demand weeks may drive both Copilot use and fragmentation.\",\n", " })\n", "\n", "# 9. After-hours: absolute hours against share of collaboration.\n", "after_hours_gap = relative_gap(\"After_hours_collaboration_hours\")\n", "power_share = lookup(process_by_stage, \"Power + Habitual\", \"after_hours_share_pct\")\n", "non_share = lookup(process_by_stage, \"Non-user\", \"after_hours_share_pct\")\n", "if pd.notna(after_hours_gap) and pd.notna(power_share) and pd.notna(non_share):\n", " share_gap = power_share - non_share\n", " adjusted_after_hours = lookup(\n", " adjusted_by_metric, \"After_hours_collaboration_hours\", \"adjusted_difference_sd\"\n", " )\n", " adjusted_p = lookup(adjusted_by_metric, \"After_hours_collaboration_hours\", \"p_value\")\n", " insights.append({\n", " \"theme\": \"After-hours\",\n", " \"insight\": (\n", " \"After-hours collaboration as a share of total collaboration is \"\n", " + describe(share_gap, \"higher\", \"lower\", \"similar\", tolerance=2)\n", " + \" for sustained users, even though absolute after-hours hours are \"\n", " + describe(after_hours_gap, \"higher\", \"lower\", \"comparable\", tolerance=5)\n", " + \".\"\n", " ),\n", " \"evidence\": (\n", " f\"Absolute after-hours collaboration differs by \"\n", " f\"{fmt(after_hours_gap, '{:+.0f}')}%, while the median after-hours share is \"\n", " f\"{fmt(power_share)}% for Power and Habitual Users versus {fmt(non_share)}% \"\n", " \"for Non-users.\"\n", " ),\n", " \"implication\": describe(\n", " share_gap,\n", " \"The additional load is spilling beyond the normal working pattern and is worth monitoring.\",\n", " \"The additional collaboration load sits inside the normal working pattern.\",\n", " \"The additional collaboration load sits inside the normal working pattern.\",\n", " tolerance=2,\n", " ),\n", " \"caveat\": (\n", " f\"After adjusting for role attributes the difference is \"\n", " f\"{fmt(adjusted_after_hours, '{:+.3f}')} SD (p={fmt(adjusted_p, '{:.2f}')}).\"\n", " if pd.notna(adjusted_after_hours)\n", " else \"The adjusted model did not cover this metric.\"\n", " ),\n", " })\n", "\n", "# 10. Meetings: level difference against the same-person coefficient.\n", "power_meeting_len = lookup(process_by_stage, \"Power + Habitual\", \"meeting_length_min\")\n", "non_meeting_len = lookup(process_by_stage, \"Non-user\", \"meeting_length_min\")\n", "within_meeting = lookup(within_by_metric, \"meeting_length_min\", \"beta_sd\")\n", "within_meeting_verdict = lookup(\n", " within_by_metric, \"meeting_length_min\", \"verdict\", default=None\n", ")\n", "if pd.notna(power_meeting_len) and pd.notna(non_meeting_len):\n", " meeting_gap = power_meeting_len - non_meeting_len\n", " same_person_note = \"\"\n", " if pd.notna(within_meeting) and within_meeting_verdict:\n", " same_person_note = (\n", " f\"; the same-person coefficient is {fmt(within_meeting, '{:+.3f}')} SD \"\n", " f\"({within_meeting_verdict.lower()})\"\n", " )\n", " insights.append({\n", " \"theme\": \"Meetings\",\n", " \"insight\": (\n", " \"Copilot use is associated with \"\n", " + describe(meeting_gap, \"longer\", \"shorter\", \"similar\", tolerance=2)\n", " + \" meetings.\"\n", " ),\n", " \"evidence\": (\n", " f\"Median meeting length is {fmt(power_meeting_len)} minutes for Power and \"\n", " f\"Habitual Users and {fmt(non_meeting_len)} minutes for Non-users\"\n", " f\"{same_person_note}.\"\n", " ),\n", " \"implication\": describe(\n", " -meeting_gap,\n", " \"Meeting recap and asynchronous follow-through may already be compressing meetings.\",\n", " \"Meeting recap and asynchronous follow-through have not yet translated into shorter meetings.\",\n", " \"Meeting recap and asynchronous follow-through have not yet translated into measurable meeting compression.\",\n", " tolerance=2,\n", " ),\n", " \"caveat\": \"Meeting duration is inferred from weekly meeting hours divided by meeting count.\",\n", " })\n", "\n", "insight_evidence = pd.DataFrame(insights)\n", "insight_evidence.insert(0, \"number\", range(1, len(insight_evidence) + 1))\n", "save_table(insight_evidence, \"08_insight_evidence\")\n", "\n", "print(f\"Evidence-led insights generated from this dataset: {len(insight_evidence)}\")\n", "print(insight_evidence[[\"number\", \"theme\", \"insight\", \"evidence\"]].to_string(index=False))\n" ] }, { "cell_type": "markdown", "id": "cell0023", "metadata": {}, "source": [ "## 9. Executive interpretation\n", "\n", "The final cell writes an executive summary grounded only in results calculated above.\n", "It leads with the adoption journey, then separates reassuring signals from risks and\n", "recommended next analyses." ] }, { "cell_type": "code", "execution_count": null, "id": "cell0024", "metadata": {}, "outputs": [], "source": [ "def section(title, lines):\n", " \"\"\"Append a titled block, skipping it entirely when it has no content.\"\"\"\n", " body = [line for line in lines if line]\n", " return ([title] + body + [\"\"]) if body else []\n", "\n", "\n", "summary_lines = [\"COPILOT ADOPTION AND WAYS-OF-WORKING SUMMARY\", \"=\" * 76, \"\"]\n", "\n", "summary_lines += section(\"WHAT THE DATA COVERS\", [\n", " f\" {df['PersonId'].nunique():,} people across {len(weeks)} weeks \"\n", " f\"({first_week.date()} to {latest_week.date()}).\",\n", " f\" Removed {blank_rows_removed:,} blank rows from the source file.\"\n", " if blank_rows_removed else \"\",\n", " f\" The Copilot population is {POPULATION_BASIS}. Being in it means having\",\n", " \" Copilot, which is not the same as using it; usage is counted separately.\",\n", " f\" Usage segments use a {SEGMENT_WINDOW_WEEKS}-week window, \"\n", " f\"{SEGMENT_HABIT_WEEKS} active weeks required, Power User threshold \"\n", " f\"{POWER_THRESHOLD} weekly actions.\",\n", " f\" Only {len(weeks)} weeks are available, fewer than the \"\n", " f\"{SEGMENT_WINDOW_WEEKS}-week window, so habit-based segments are understated.\"\n", " if len(weeks) < SEGMENT_WINDOW_WEEKS else \"\",\n", "])\n", "\n", "adoption_lines = [\n", " f\" Reach: the {POPULATION_LABEL_LC} moved from \"\n", " f\"{fmt(first_row['copilot_population_pct'])}% to \"\n", " f\"{fmt(last_row['copilot_population_pct'])}% of the measured population.\",\n", " f\" Activation: {fmt(first_row['active_pct_of_copilot_population'])}% to \"\n", " f\"{fmt(last_row['active_pct_of_copilot_population'])}% of that population used \"\n", " f\"Copilot in the week, while actions per active user moved from \"\n", " f\"{fmt(first_row['actions_per_active'])} to {fmt(last_row['actions_per_active'])}.\",\n", " f\" At the latest week, {sustained_n:,} people ({fmt(sustained_pct)}%) were \"\n", " f\"Habitual or Power Users under the {SEGMENT_WINDOW_WEEKS}-week definition.\",\n", "]\n", "if len(largest_additions) and largest_additions.iloc[0][\"newly_in_population\"] > 0:\n", " biggest = largest_additions.iloc[0]\n", " adoption_lines.append(\n", " f\" The largest single week for people joining the Copilot population was \"\n", " f\"{int(biggest['newly_in_population']):,} people in the week of \"\n", " f\"{pd.Timestamp(biggest['MetricDate']).date()}. Confirm what drove it before \"\n", " \"reading it as adoption.\"\n", " )\n", "if len(plot_cohorts) >= 2:\n", " oldest_cohort, newest_cohort = plot_cohorts.iloc[0], plot_cohorts.iloc[-1]\n", " adoption_lines.append(\n", " f\" The earliest qualifying cohort is \"\n", " f\"{fmt(oldest_cohort['active_latest_pct'])}% active in the latest week, versus \"\n", " f\"{fmt(newest_cohort['active_latest_pct'])}% for the newest.\"\n", " )\n", "summary_lines += section(\"ADOPTION JOURNEY\", adoption_lines)\n", "\n", "opportunity_lines = []\n", "if pd.notna(manager_sustained) and pd.notna(ic_sustained):\n", " opportunity_lines += [\n", " f\" Power and Habitual User status is {fmt(manager_sustained)}% among managers \"\n", " f\"and {fmt(ic_sustained)}% among individual contributors.\",\n", " \" Read this as both an enablement signal and a confounding warning, because\",\n", " \" managers have heavier collaboration patterns regardless of Copilot use.\",\n", " ]\n", "if function_summary is not None:\n", " opportunity_lines.append(\n", " \" Function-level adoption varies. Use the function table to target onboarding \"\n", " \"and\\n role-specific scenarios, not to rank performance.\"\n", " )\n", "if novice_by_function is not None and novice_by_function[\"novice_people\"].max() > 0:\n", " top_novice = novice_by_function.iloc[0]\n", " opportunity_lines.append(\n", " f\" The largest single pool of Novice Users is {top_novice['FunctionType']} \"\n", " f\"({int(top_novice['novice_people']):,} people, \"\n", " f\"{fmt(top_novice['share_of_all_novices'], '{:.0f}')}% of all Novice Users).\"\n", " )\n", "summary_lines += section(\"WHERE THE OPPORTUNITY IS\", opportunity_lines)\n", "\n", "native_lines = []\n", "if scan_by_segment is not None:\n", " native_lines.append(\n", " f\" The key-metrics scan compares {scan_data['PersonId'].nunique():,} people \"\n", " f\"across {len(scan_metrics)} metrics.\"\n", " )\n", " collab_gap = relative_gap(\"Collaboration_hours\")\n", " if pd.notna(collab_gap):\n", " native_lines.append(\n", " f\" Power Users average \"\n", " f\"{fmt(lookup(scan_by_segment, 'Power User', 'Collaboration_hours'))} \"\n", " f\"collaboration hours versus \"\n", " f\"{fmt(lookup(scan_by_segment, 'Non-user', 'Collaboration_hours'))} for \"\n", " f\"Non-users ({fmt(collab_gap, '{:+.0f}')}%).\"\n", " )\n", "if rank_sustained is not None:\n", " function_rank = rank_sustained[rank_sustained[\"hrvar\"] == \"FunctionType\"]\n", " if len(function_rank):\n", " leader = function_rank.iloc[0]\n", " native_lines.append(\n", " f\" {leader['attributes']} has the highest Power + Habitual adoption among \"\n", " f\"qualifying functions ({fmt(leader['metric'])}%).\"\n", " )\n", "summary_lines += section(\"NATIVE VIVA INSIGHTS VIEWS\", native_lines)\n", "\n", "ways_lines = []\n", "if adjusted is not None:\n", " significant = adjusted[adjusted[\"p_value\"] < 0.05]\n", " higher = significant[significant[\"adjusted_difference_sd\"] > 0]\n", " lower = significant[significant[\"adjusted_difference_sd\"] < 0]\n", " ways_lines.append(f\" Adjusted comparison, {control_note}.\")\n", " if len(higher):\n", " ways_lines.append(\n", " \" Higher for Power and Habitual Users: \"\n", " + \", \".join(higher[\"label\"].tolist()) + \".\"\n", " )\n", " if len(lower):\n", " ways_lines.append(\n", " \" Lower for Power and Habitual Users: \"\n", " + \", \".join(lower[\"label\"].tolist()) + \".\"\n", " )\n", " if len(significant) == 0:\n", " ways_lines.append(\n", " \" No metric differs significantly at the 5% level once role attributes are \"\n", " \"controlled.\"\n", " )\n", " else:\n", " ways_lines.append(\n", " \" This shows where Copilot use is concentrated. It does not establish that\"\n", " )\n", " ways_lines.append(\" Copilot created those working patterns.\")\n", "if process_by_stage is not None:\n", " for column, label, unit in [\n", " (\"meeting_length_min\", \"Median meeting length\", \" minutes\"),\n", " (\"after_hours_share_pct\", \"Median after-hours share of collaboration\", \"%\"),\n", " (\"meeting_multitask_share_pct\", \"Median meeting multitasking share\", \"%\"),\n", " (\"focus_realisation_pct\", \"Median available focus time uninterrupted\", \"%\"),\n", " ]:\n", " power_value = lookup(process_by_stage, \"Power + Habitual\", column)\n", " non_value = lookup(process_by_stage, \"Non-user\", column)\n", " if pd.notna(power_value) and pd.notna(non_value):\n", " ways_lines.append(\n", " f\" {label}: {fmt(power_value)}{unit} for Power and Habitual Users \"\n", " f\"versus {fmt(non_value)}{unit} for Non-users.\"\n", " )\n", "summary_lines += section(\"WAYS OF WORKING\", ways_lines)\n", "\n", "if within_results is not None:\n", " within_lines = [\" Association with a 1 SD increase in log weekly Copilot actions,\",\n", " \" holding the person and the week constant. A result counts only if\",\n", " f\" its interval clears zero and it reaches {MATERIAL_EFFECT_SD} SD:\"]\n", " for row in within_results.itertuples():\n", " if row.verdict == \"Detectable and material\":\n", " note = f\"{row.direction}, material\"\n", " elif row.verdict == \"Detectable but negligible\":\n", " note = \"clears zero but negligible\"\n", " else:\n", " note = \"no detectable association\"\n", " within_lines.append(f\" {row.label}: {row.beta_sd:+.3f} SD ({note}).\")\n", " material_count = int((within_results[\"verdict\"] == \"Detectable and material\").sum())\n", " if material_count == 0:\n", " within_lines.append(\n", " \" No indicator clears both bars, so heavier-use weeks do not look\"\n", " )\n", " within_lines.append(\n", " \" meaningfully different from lighter weeks for the same person.\"\n", " )\n", " within_lines.append(\n", " \" A same-person comparison removes stable individual differences, but it still\"\n", " )\n", " within_lines.append(\n", " \" cannot separate a Copilot effect from an unusually demanding week.\"\n", " )\n", " summary_lines += section(\"SAME-PERSON CHECK\", within_lines)\n", "\n", "replication_lines = []\n", "if leading_functions is not None:\n", " replication_lines.append(\n", " f\" {len(leading_functions)} function(s) already exceed \"\n", " f\"{LEADING_FUNCTION_THRESHOLD_PCT:.0f}% Power + Habitual adoption and are \"\n", " \"candidates for a replication playbook.\"\n", " if len(leading_functions) else\n", " f\" No function has yet crossed {LEADING_FUNCTION_THRESHOLD_PCT:.0f}% Power + \"\n", " \"Habitual adoption.\"\n", " )\n", "summary_lines += section(\"WHERE TO REPLICATE\", replication_lines)\n", "\n", "action_lines = []\n", "if len(largest_additions) and largest_additions.iloc[0][\"newly_in_population\"] > 0:\n", " action_lines.append(\n", " f\" 1. Confirm whether the weeks when people joined the Copilot population \"\n", " f\"(largest in the week of\\n \"\n", " f\"{pd.Timestamp(largest_additions.iloc[0]['MetricDate']).date()}) reflect \"\n", " \"licensing, query scope, telemetry\\n changes, or a deliberate enablement wave.\"\n", " )\n", "action_lines += [\n", " f\" {len(action_lines) + 1}. Prioritise recent joiners and lower-adoption \"\n", " \"groups for role-based\\n onboarding, then track conversion to Habitual or Power \"\n", " \"status over the next window.\",\n", "]\n", "if pd.notna(manager_sustained) and pd.notna(ic_sustained) and manager_sustained > ic_sustained:\n", " action_lines.append(\n", " f\" {len(action_lines) + 1}. Use managers as adoption sponsors, while building \"\n", " \"explicit individual-contributor\\n use cases so that adoption does not remain \"\n", " \"manager-led.\"\n", " )\n", "if within_results is not None or process_by_stage is not None:\n", " action_lines.append(\n", " f\" {len(action_lines) + 1}. Pair Copilot enablement with meeting recap, \"\n", " \"asynchronous updates, and focus-time\\n norms, and monitor multitasking share \"\n", " \"and focus realisation as guardrail metrics.\"\n", " )\n", "action_lines.append(\n", " f\" {len(action_lines) + 1}. Re-run with a longer panel before making longer-horizon \"\n", " \"retention claims. Causal\\n claims need a separate design: see \"\n", " \"https://microsoft.github.io/viva-insights-sample-code/causal-inference/\"\n", ")\n", "summary_lines += section(\"RECOMMENDED ACTIONS\", action_lines)\n", "\n", "summary_lines += [\n", " \"INTERPRETATION LIMIT\",\n", " \" All findings are observational associations. They do not establish that Copilot\",\n", " \" caused changes in collaboration, focus, or wellbeing.\",\n", "]\n", "\n", "summary_text = \"\\n\".join(summary_lines)\n", "print(summary_text)\n", "(OUTPUT_DIR / \"executive_summary.txt\").write_text(summary_text, encoding=\"utf-8\")\n", "\n", "print(f\"\\nWrote {len(TABLES)} tables, {len(FIGURES)} figures, and executive_summary.txt\")\n", "print(f\"Output directory: {OUTPUT_DIR.resolve()}\")\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.14.6" } }, "nbformat": 4, "nbformat_minor": 5 }