{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Create a Table from 3D points\n",
    "\n",
    "Create a 3LC Table directly from row data containing 3D point cloud information using the Mammoth dataset for 3D object analysis.\n",
    "\n",
    "![img](../../images/mammoth.jpg)\n",
    "\n",
    "<!-- Tags: [\"3D\", \"dimensionality-reduction\"] -->\n",
    "\n",
    "3D point cloud data requires specialized handling for spatial coordinates, normals, and additional geometric properties. Direct row-by-row construction allows precise control over 3D data organization and metadata.\n",
    "\n",
    "This notebook demonstrates writing a 3LC Table directly by adding rows from a JSON file containing 3D mammoth data. We use a TableWriter to build up the table with 3D coordinates and associated metadata, creating a Parquet-backed table for efficient 3D visualization and analysis workflows."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Project setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": [
     "parameters"
    ]
   },
   "outputs": [],
   "source": [
    "PROJECT_NAME = \"3LC Tutorials - Mammoth\"\n",
    "DATASET_NAME = \"Mammoth\"\n",
    "TABLE_NAME = \"mammoth-10k\"\n",
    "DATA_PATH = \"../../data\"\n",
    "INSTALL_DEPENDENCIES = True"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if INSTALL_DEPENDENCIES:\n",
    "    %pip --quiet install 3lc"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Download Source Data\n",
    "\n",
    "We will use the 3D mammoth data as an example. This is a popular toy dataset commonly used in the dimensionality\n",
    "reduction literature.\n",
    "\n",
    "The original data can be found in the [PaCMAP](https://github.com/YingfanWang/PaCMAP/blob/master/data/mammoth_3d.json)\n",
    "github repository."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests\n",
    "\n",
    "response = requests.get(\"https://raw.githubusercontent.com/YingfanWang/PaCMAP/master/data/mammoth_3d.json\")\n",
    "input_data = response.json()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The input data is represented as a JSON list of lists, where each sublist is a 3D point:\n",
    "print(type(input_data))\n",
    "print(len(input_data))\n",
    "print(len(input_data[0]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Writing the Table\n",
    "\n",
    "We construct a `TableWriter`, which will determine the URL and the schema of the table we want to write. In our case,\n",
    "the table will contain a single column of 3-vectors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tlc\n",
    "\n",
    "column_name = \"points\"\n",
    "\n",
    "table_writer = tlc.TableWriter(\n",
    "    dataset_name=DATASET_NAME,\n",
    "    project_name=PROJECT_NAME,\n",
    "    table_name=TABLE_NAME,\n",
    "    description=\"A table containing 10,000 3D points of a mammoth\",\n",
    "    if_exists=\"overwrite\",\n",
    "    column_schemas={column_name: tlc.FloatVector3Schema(\"3D Points\")},\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Next we add the data to the table row by row.\n",
    "for point in input_data:\n",
    "    table_writer.add_row({column_name: point})\n",
    "\n",
    "# Finally, we flush the table writer to ensure that all data is written to disk.\n",
    "table = table_writer.finalize()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Inspect the first row of the table:\n",
    "table[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Next Steps\n",
    "\n",
    "The written table can now be viewed in the 3LC Dashboard.\n",
    "\n",
    "Some ideas for further exploration:\n",
    "\n",
    "- Visualize the data in a scatter plot\n",
    "- Apply dimensionality reduction to the data\n",
    "- Segment the data into clusters"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}