{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 🎨 Data Designer 101: Structured Outputs and Jinja Expressions\n", "\n", "[Click here](https://raw.githubusercontent.com/NVIDIA-NeMo/DataDesigner/refs/heads/main/docs/notebooks/2-structured-outputs-and-jinja-expressions.ipynb) to download this notebook to your computer.\n", "#### 📚 What you'll learn\n", "\n", "In this notebook, we will continue our exploration of Data Designer, demonstrating more advanced data generation using structured outputs and Jinja expressions.\n", "\n", "If this is your first time using Data Designer, we recommend starting with the [first notebook](/notebooks/1-the-basics/) in this 101 series." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### đŸ“Ļ Import the essentials\n", "\n", "- The `essentials` module provides quick access to the most commonly used objects.\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "from data_designer.essentials import (\n", " CategorySamplerParams,\n", " DataDesigner,\n", " DataDesignerConfigBuilder,\n", " ExpressionColumnConfig,\n", " InferenceParameters,\n", " LLMStructuredColumnConfig,\n", " ModelConfig,\n", " PersonFromFakerSamplerParams,\n", " SamplerColumnConfig,\n", " SamplerType,\n", " SubcategorySamplerParams,\n", ")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### ⚙ī¸ Initialize the Data Designer interface\n", "\n", "- `DataDesigner` is the main object that is used to interface with the library.\n" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "[16:29:58] [INFO] â™ģī¸ Using default model providers from '/Users/amanoel/.data-designer/model_providers.yaml'\n" ] } ], "source": [ "data_designer_client = DataDesigner()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 🎛ī¸ Define model configurations\n", "\n", "- Each `ModelConfig` defines a model that can be used during the generation process.\n", "\n", "- The \"model alias\" is used to reference the model in the Data Designer config (as we will see below).\n", "\n", "- The \"model provider\" is the external service that hosts the model (see the [model config](/models/default-model-settings.md) docs for more details).\n", "\n", "- By default, we use [build.nvidia.com](https://build.nvidia.com/models) as the model provider.\n" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "# This name is set in the model provider configuration.\n", "MODEL_PROVIDER = \"nvidia\"\n", "\n", "# The model ID is from build.nvidia.com.\n", "MODEL_ID = \"nvidia/nvidia-nemotron-nano-9b-v2\"\n", "\n", "# We choose this alias to be descriptive for our use case.\n", "MODEL_ALIAS = \"nemotron-nano-v2\"\n", "\n", "# This sets reasoning to False for the nemotron-nano-v2 model.\n", "SYSTEM_PROMPT = \"/no_think\"\n", "\n", "model_configs = [\n", " ModelConfig(\n", " alias=MODEL_ALIAS,\n", " model=MODEL_ID,\n", " provider=MODEL_PROVIDER,\n", " inference_parameters=InferenceParameters(\n", " temperature=0.5,\n", " top_p=1.0,\n", " max_tokens=1024,\n", " ),\n", " )\n", "]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 🏗ī¸ Initialize the Data Designer Config Builder\n", "\n", "- The Data Designer config defines the dataset schema and generation process.\n", "\n", "- The config builder provides an intuitive interface for building this configuration.\n", "\n", "- The list of model configs is provided to the builder at initialization.\n" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "config_builder = DataDesignerConfigBuilder(model_configs=model_configs)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 🧑‍🎨 Designing our data\n", "\n", "- We will again create a product review dataset, but this time we will use structured outputs and Jinja expressions.\n", "\n", "- Structured outputs let you specify the exact schema of the data you want to generate.\n", "\n", "- Data Designer supports schemas specified using either json schema or Pydantic data models (recommended).\n", "\n", "
\n", "\n", "We'll define our structured outputs using [Pydantic](https://docs.pydantic.dev/latest/) data models\n", "\n", "> 💡 **Why Pydantic?**\n", ">\n", "> - Pydantic models provide better IDE support and type validation.\n", ">\n", "> - They are more Pythonic than raw JSON schemas.\n", ">\n", "> - They integrate seamlessly with Data Designer's structured output system.\n" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "from decimal import Decimal\n", "from typing import Literal\n", "\n", "from pydantic import BaseModel, Field\n", "\n", "\n", "# We define a Product schema so that the name, description, and price are generated\n", "# in one go, with the types and constraints specified.\n", "class Product(BaseModel):\n", " name: str = Field(description=\"The name of the product\")\n", " description: str = Field(description=\"A description of the product\")\n", " price: Decimal = Field(description=\"The price of the product\", ge=10, le=1000, decimal_places=2)\n", "\n", "\n", "class ProductReview(BaseModel):\n", " rating: int = Field(description=\"The rating of the product\", ge=1, le=5)\n", " customer_mood: Literal[\"irritated\", \"mad\", \"happy\", \"neutral\", \"excited\"] = Field(\n", " description=\"The mood of the customer\"\n", " )\n", " review: str = Field(description=\"A review of the product\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, let's design our product review dataset using a few more tricks compared to the previous notebook." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "[16:29:58] [INFO] ✅ Validation passed\n" ] }, { "data": { "text/html": [ "\n", "\n", "\n", "
DataDesignerConfigBuilder(\n",
       "    sampler_columns: [\n",
       "        "customer",\n",
       "        "product_category",\n",
       "        "product_subcategory",\n",
       "        "target_age_range",\n",
       "        "review_style"\n",
       "    ]\n",
       ")\n",
       "
\n", "\n" ], "text/plain": [ "DataDesignerConfigBuilder(\n", " sampler_columns: [\n", " \"customer\",\n", " \"product_category\",\n", " \"product_subcategory\",\n", " \"target_age_range\",\n", " \"review_style\"\n", " ]\n", ")" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Since we often only want a few attributes from Person objects, we can\n", "# set drop=True in the column config to drop the column from the final dataset.\n", "config_builder.add_column(\n", " SamplerColumnConfig(\n", " name=\"customer\",\n", " sampler_type=SamplerType.PERSON_FROM_FAKER,\n", " params=PersonFromFakerSamplerParams(),\n", " drop=True,\n", " )\n", ")\n", "\n", "config_builder.add_column(\n", " SamplerColumnConfig(\n", " name=\"product_category\",\n", " sampler_type=SamplerType.CATEGORY,\n", " params=CategorySamplerParams(\n", " values=[\n", " \"Electronics\",\n", " \"Clothing\",\n", " \"Home & Kitchen\",\n", " \"Books\",\n", " \"Home Office\",\n", " ],\n", " ),\n", " )\n", ")\n", "\n", "config_builder.add_column(\n", " SamplerColumnConfig(\n", " name=\"product_subcategory\",\n", " sampler_type=SamplerType.SUBCATEGORY,\n", " params=SubcategorySamplerParams(\n", " category=\"product_category\",\n", " values={\n", " \"Electronics\": [\n", " \"Smartphones\",\n", " \"Laptops\",\n", " \"Headphones\",\n", " \"Cameras\",\n", " \"Accessories\",\n", " ],\n", " \"Clothing\": [\n", " \"Men's Clothing\",\n", " \"Women's Clothing\",\n", " \"Winter Coats\",\n", " \"Activewear\",\n", " \"Accessories\",\n", " ],\n", " \"Home & Kitchen\": [\n", " \"Appliances\",\n", " \"Cookware\",\n", " \"Furniture\",\n", " \"Decor\",\n", " \"Organization\",\n", " ],\n", " \"Books\": [\n", " \"Fiction\",\n", " \"Non-Fiction\",\n", " \"Self-Help\",\n", " \"Textbooks\",\n", " \"Classics\",\n", " ],\n", " \"Home Office\": [\n", " \"Desks\",\n", " \"Chairs\",\n", " \"Storage\",\n", " \"Office Supplies\",\n", " \"Lighting\",\n", " ],\n", " },\n", " ),\n", " )\n", ")\n", "\n", "config_builder.add_column(\n", " SamplerColumnConfig(\n", " name=\"target_age_range\",\n", " sampler_type=SamplerType.CATEGORY,\n", " params=CategorySamplerParams(values=[\"18-25\", \"25-35\", \"35-50\", \"50-65\", \"65+\"]),\n", " )\n", ")\n", "\n", "# Sampler columns support conditional params, which are used if the condition is met.\n", "# In this example, we set the review style to rambling if the target age range is 18-25.\n", "# Note conditional parameters are only supported for Sampler column types.\n", "config_builder.add_column(\n", " SamplerColumnConfig(\n", " name=\"review_style\",\n", " sampler_type=SamplerType.CATEGORY,\n", " params=CategorySamplerParams(\n", " values=[\"rambling\", \"brief\", \"detailed\", \"structured with bullet points\"],\n", " weights=[1, 2, 2, 1],\n", " ),\n", " conditional_params={\n", " \"target_age_range == '18-25'\": CategorySamplerParams(values=[\"rambling\"]),\n", " },\n", " )\n", ")\n", "\n", "# Optionally validate that the columns are configured correctly.\n", "config_builder.validate()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, we will use more advanced Jinja expressions to create new columns.\n", "\n", "Jinja expressions let you:\n", "\n", "- Access nested attributes: `{{ customer.first_name }}`\n", "\n", "- Combine values: `{{ customer.first_name }} {{ customer.last_name }}`\n", "\n", "- Use conditional logic: `{% if condition %}...{% endif %}`\n" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "[16:29:58] [INFO] ✅ Validation passed\n" ] }, { "data": { "text/html": [ "\n", "\n", "\n", "
DataDesignerConfigBuilder(\n",
       "    sampler_columns: [\n",
       "        "customer",\n",
       "        "product_category",\n",
       "        "product_subcategory",\n",
       "        "target_age_range",\n",
       "        "review_style"\n",
       "    ]\n",
       "    llm_structured_columns: ['product', 'customer_review']\n",
       "    expression_columns: ['customer_name', 'customer_age']\n",
       ")\n",
       "
\n", "\n" ], "text/plain": [ "DataDesignerConfigBuilder(\n", " sampler_columns: [\n", " \"customer\",\n", " \"product_category\",\n", " \"product_subcategory\",\n", " \"target_age_range\",\n", " \"review_style\"\n", " ]\n", " llm_structured_columns: ['product', 'customer_review']\n", " expression_columns: ['customer_name', 'customer_age']\n", ")" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# We can create new columns using Jinja expressions that reference\n", "# existing columns, including attributes of nested objects.\n", "config_builder.add_column(\n", " ExpressionColumnConfig(name=\"customer_name\", expr=\"{{ customer.first_name }} {{ customer.last_name }}\")\n", ")\n", "\n", "config_builder.add_column(ExpressionColumnConfig(name=\"customer_age\", expr=\"{{ customer.age }}\"))\n", "\n", "config_builder.add_column(\n", " LLMStructuredColumnConfig(\n", " name=\"product\",\n", " prompt=(\n", " \"Create a product in the '{{ product_category }}' category, focusing on products \"\n", " \"related to '{{ product_subcategory }}'. The target age range of the ideal customer is \"\n", " \"{{ target_age_range }} years old. The product should be priced between $10 and $1000.\"\n", " ),\n", " system_prompt=SYSTEM_PROMPT,\n", " output_format=Product,\n", " model_alias=MODEL_ALIAS,\n", " )\n", ")\n", "\n", "# We can even use if/else logic in our Jinja expressions to create more complex prompt patterns.\n", "config_builder.add_column(\n", " LLMStructuredColumnConfig(\n", " name=\"customer_review\",\n", " prompt=(\n", " \"Your task is to write a review for the following product:\\n\\n\"\n", " \"Product Name: {{ product.name }}\\n\"\n", " \"Product Description: {{ product.description }}\\n\"\n", " \"Price: {{ product.price }}\\n\\n\"\n", " \"Imagine your name is {{ customer_name }} and you are from {{ customer.city }}, {{ customer.state }}. \"\n", " \"Write the review in a style that is '{{ review_style }}'.\"\n", " \"{% if target_age_range == '18-25' %}\"\n", " \"Make sure the review is more informal and conversational.\"\n", " \"{% else %}\"\n", " \"Make sure the review is more formal and structured.\"\n", " \"{% endif %}\"\n", " ),\n", " system_prompt=SYSTEM_PROMPT,\n", " output_format=ProductReview,\n", " model_alias=MODEL_ALIAS,\n", " )\n", ")\n", "\n", "config_builder.validate()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 🔁 Iteration is key – preview the dataset!\n", "\n", "1. Use the `preview` method to generate a sample of records quickly.\n", "\n", "2. Inspect the results for quality and format issues.\n", "\n", "3. Adjust column configurations, prompts, or parameters as needed.\n", "\n", "4. Re-run the preview until satisfied.\n" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "[16:29:58] [INFO] 🧐 Preview generation in progress\n", "[16:29:58] [INFO] ✅ Validation passed\n", "[16:29:58] [INFO] ⛓ī¸ Sorting column configs into a Directed Acyclic Graph\n", "[16:29:58] [INFO] đŸŠē Running health checks for models...\n", "[16:29:58] [INFO] |-- 👀 Checking 'nvidia/nvidia-nemotron-nano-9b-v2' in provider named 'nvidia' for model alias 'nemotron-nano-v2'...\n", "[16:30:00] [INFO] |-- ✅ Passed!\n", "[16:30:00] [INFO] 🎲 Preparing samplers to generate 10 records across 5 columns\n", "[16:30:00] [INFO] 🧩 Generating column `customer_name` from expression\n", "[16:30:00] [INFO] 🧩 Generating column `customer_age` from expression\n", "[16:30:00] [INFO] 🗂ī¸ Preparing llm-structured column generation\n", "[16:30:00] [INFO] |-- column name: 'product'\n", "[16:30:00] [INFO] |-- model config:\n", "{\n", " \"alias\": \"nemotron-nano-v2\",\n", " \"model\": \"nvidia/nvidia-nemotron-nano-9b-v2\",\n", " \"inference_parameters\": {\n", " \"temperature\": 0.5,\n", " \"top_p\": 1.0,\n", " \"max_tokens\": 1024,\n", " \"max_parallel_requests\": 4,\n", " \"timeout\": null,\n", " \"extra_body\": null\n", " },\n", " \"provider\": \"nvidia\"\n", "}\n", "[16:30:00] [INFO] 🐙 Processing llm-structured column 'product' with 4 concurrent workers\n", "[16:30:05] [INFO] 🗂ī¸ Preparing llm-structured column generation\n", "[16:30:05] [INFO] |-- column name: 'customer_review'\n", "[16:30:05] [INFO] |-- model config:\n", "{\n", " \"alias\": \"nemotron-nano-v2\",\n", " \"model\": \"nvidia/nvidia-nemotron-nano-9b-v2\",\n", " \"inference_parameters\": {\n", " \"temperature\": 0.5,\n", " \"top_p\": 1.0,\n", " \"max_tokens\": 1024,\n", " \"max_parallel_requests\": 4,\n", " \"timeout\": null,\n", " \"extra_body\": null\n", " },\n", " \"provider\": \"nvidia\"\n", "}\n", "[16:30:05] [INFO] 🐙 Processing llm-structured column 'customer_review' with 4 concurrent workers\n", "[16:30:13] [INFO] 📊 Model usage summary:\n", "{\n", " \"nvidia/nvidia-nemotron-nano-9b-v2\": {\n", " \"token_usage\": {\n", " \"prompt_tokens\": 6304,\n", " \"completion_tokens\": 3167,\n", " \"total_tokens\": 9471\n", " },\n", " \"request_usage\": {\n", " \"successful_requests\": 20,\n", " \"failed_requests\": 0,\n", " \"total_requests\": 20\n", " },\n", " \"tokens_per_second\": 724,\n", " \"requests_per_minute\": 91\n", " }\n", "}\n", "[16:30:13] [INFO] 📐 Measuring dataset column statistics:\n", "[16:30:13] [INFO] |-- 🎲 column: 'customer'\n", "[16:30:13] [INFO] |-- 🎲 column: 'product_category'\n", "[16:30:13] [INFO] |-- 🎲 column: 'product_subcategory'\n", "[16:30:13] [INFO] |-- 🎲 column: 'target_age_range'\n", "[16:30:13] [INFO] |-- 🎲 column: 'review_style'\n", "[16:30:13] [INFO] |-- 🧩 column: 'customer_name'\n", "[16:30:13] [INFO] |-- 🧩 column: 'customer_age'\n", "[16:30:13] [INFO] |-- 🗂ī¸ column: 'product'\n", "[16:30:13] [INFO] |-- 🗂ī¸ column: 'customer_review'\n", "[16:30:13] [INFO] ✅ Preview complete!\n" ] } ], "source": [ "preview = data_designer_client.preview(config_builder)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
                                                                                                                   \n",
       "                                                 Generated Columns                                                 \n",
       "┏━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃ Name                ┃ Value                                                                                     ┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│ customer            │ {                                                                                         │\n",
       "│                     │     'uuid': '02a9fa30-045c-4443-9223-d314546ce65b',                                       │\n",
       "│                     │     'locale': 'en_US',                                                                    │\n",
       "│                     │     'first_name': 'Steven',                                                               │\n",
       "│                     │     'last_name': 'Caldwell',                                                              │\n",
       "│                     │     'middle_name': None,                                                                  │\n",
       "│                     │     'sex': 'Male',                                                                        │\n",
       "│                     │     'street_number': '9869',                                                              │\n",
       "│                     │     'street_name': 'Reynolds Alley',                                                      │\n",
       "│                     │     'city': 'West Nicholas',                                                              │\n",
       "│                     │     'state': 'Arizona',                                                                   │\n",
       "│                     │     'postcode': '05872',                                                                  │\n",
       "│                     │     'age': 71,                                                                            │\n",
       "│                     │     'birth_date': '1954-01-09',                                                           │\n",
       "│                     │     'country': 'Saint Vincent and the Grenadines',                                        │\n",
       "│                     │     'marital_status': 'separated',                                                        │\n",
       "│                     │     'education_level': 'some_college',                                                    │\n",
       "│                     │     'unit': '',                                                                           │\n",
       "│                     │     'occupation': 'Programmer, systems',                                                  │\n",
       "│                     │     'phone_number': '001-301-970-1903x043',                                               │\n",
       "│                     │     'bachelors_field': 'no_degree'                                                        │\n",
       "│                     │ }                                                                                         │\n",
       "├─────────────────────â”ŧ───────────────────────────────────────────────────────────────────────────────────────────┤\n",
       "│ product_category    │ Home Office                                                                               │\n",
       "├─────────────────────â”ŧ───────────────────────────────────────────────────────────────────────────────────────────┤\n",
       "│ product_subcategory │ Storage                                                                                   │\n",
       "├─────────────────────â”ŧ───────────────────────────────────────────────────────────────────────────────────────────┤\n",
       "│ target_age_range    │ 35-50                                                                                     │\n",
       "├─────────────────────â”ŧ───────────────────────────────────────────────────────────────────────────────────────────┤\n",
       "│ review_style        │ brief                                                                                     │\n",
       "├─────────────────────â”ŧ───────────────────────────────────────────────────────────────────────────────────────────┤\n",
       "│ customer_name       │ Steven Caldwell                                                                           │\n",
       "├─────────────────────â”ŧ───────────────────────────────────────────────────────────────────────────────────────────┤\n",
       "│ customer_age        │ 71                                                                                        │\n",
       "├─────────────────────â”ŧ───────────────────────────────────────────────────────────────────────────────────────────┤\n",
       "│ product             │ {                                                                                         │\n",
       "│                     │     'name': 'Ergonomic Under-Desk Storage Cabinet',                                       │\n",
       "│                     │     'description': 'A sleek and durable storage cabinet designed to fit under desks in a  │\n",
       "│                     │ home office setup. It features adjustable shelves, a modern design to match various       │\n",
       "│                     │ decors, and provides ample space for organizing office supplies, documents, and           │\n",
       "│                     │ electronics. Ideal for professionals aged 35-50 looking to maximize space efficiency.',   │\n",
       "│                     │     'price': 149.99                                                                       │\n",
       "│                     │ }                                                                                         │\n",
       "├─────────────────────â”ŧ───────────────────────────────────────────────────────────────────────────────────────────┤\n",
       "│ customer_review     │ {                                                                                         │\n",
       "│                     │     'rating': 5,                                                                          │\n",
       "│                     │     'customer_mood': 'happy',                                                             │\n",
       "│                     │     'review': 'The Ergonomic Under-Desk Storage Cabinet is a well-designed, durable       │\n",
       "│                     │ solution for optimizing space in a home office. Its adjustable shelves and modern         │\n",
       "│                     │ aesthetics make it both functional and visually appealing, particularly suited for        │\n",
       "│                     │ professionals seeking efficiency.'                                                        │\n",
       "│                     │ }                                                                                         │\n",
       "└─────────────────────┴───────────────────────────────────────────────────────────────────────────────────────────┘\n",
       "                                                                                                                   \n",
       "                                                    [index: 0]                                                     \n",
       "
\n" ], "text/plain": [ " \n", "\u001b[3m Generated Columns \u001b[0m\n", "┏━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n", "┃\u001b[1m \u001b[0m\u001b[1mName \u001b[0m\u001b[1m \u001b[0m┃\u001b[1m \u001b[0m\u001b[1mValue \u001b[0m\u001b[1m \u001b[0m┃\n", "┡━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n", "│ customer │ \u001b[1m{\u001b[0m │\n", "│ │ \u001b[32m'uuid'\u001b[0m: \u001b[32m'02a9fa30-045c-4443-9223-d314546ce65b'\u001b[0m, │\n", "│ │ \u001b[32m'locale'\u001b[0m: \u001b[32m'en_US'\u001b[0m, │\n", "│ │ \u001b[32m'first_name'\u001b[0m: \u001b[32m'Steven'\u001b[0m, │\n", "│ │ \u001b[32m'last_name'\u001b[0m: \u001b[32m'Caldwell'\u001b[0m, │\n", "│ │ \u001b[32m'middle_name'\u001b[0m: \u001b[3;35mNone\u001b[0m, │\n", "│ │ \u001b[32m'sex'\u001b[0m: \u001b[32m'Male'\u001b[0m, │\n", "│ │ \u001b[32m'street_number'\u001b[0m: \u001b[32m'9869'\u001b[0m, │\n", "│ │ \u001b[32m'street_name'\u001b[0m: \u001b[32m'Reynolds Alley'\u001b[0m, │\n", "│ │ \u001b[32m'city'\u001b[0m: \u001b[32m'West Nicholas'\u001b[0m, │\n", "│ │ \u001b[32m'state'\u001b[0m: \u001b[32m'Arizona'\u001b[0m, │\n", "│ │ \u001b[32m'postcode'\u001b[0m: \u001b[32m'05872'\u001b[0m, │\n", "│ │ \u001b[32m'age'\u001b[0m: \u001b[1;36m71\u001b[0m, │\n", "│ │ \u001b[32m'birth_date'\u001b[0m: \u001b[32m'1954-01-09'\u001b[0m, │\n", "│ │ \u001b[32m'country'\u001b[0m: \u001b[32m'Saint Vincent and the Grenadines'\u001b[0m, │\n", "│ │ \u001b[32m'marital_status'\u001b[0m: \u001b[32m'separated'\u001b[0m, │\n", "│ │ \u001b[32m'education_level'\u001b[0m: \u001b[32m'some_college'\u001b[0m, │\n", "│ │ \u001b[32m'unit'\u001b[0m: \u001b[32m''\u001b[0m, │\n", "│ │ \u001b[32m'occupation'\u001b[0m: \u001b[32m'Programmer, systems'\u001b[0m, │\n", "│ │ \u001b[32m'phone_number'\u001b[0m: \u001b[32m'001-301-970-1903x043'\u001b[0m, │\n", "│ │ \u001b[32m'bachelors_field'\u001b[0m: \u001b[32m'no_degree'\u001b[0m │\n", "│ │ \u001b[1m}\u001b[0m │\n", "├─────────────────────â”ŧ───────────────────────────────────────────────────────────────────────────────────────────┤\n", "│ product_category │ Home Office │\n", "├─────────────────────â”ŧ───────────────────────────────────────────────────────────────────────────────────────────┤\n", "│ product_subcategory │ Storage │\n", "├─────────────────────â”ŧ───────────────────────────────────────────────────────────────────────────────────────────┤\n", "│ target_age_range │ 35-50 │\n", "├─────────────────────â”ŧ───────────────────────────────────────────────────────────────────────────────────────────┤\n", "│ review_style │ brief │\n", "├─────────────────────â”ŧ───────────────────────────────────────────────────────────────────────────────────────────┤\n", "│ customer_name │ Steven Caldwell │\n", "├─────────────────────â”ŧ───────────────────────────────────────────────────────────────────────────────────────────┤\n", "│ customer_age │ \u001b[1;36m71\u001b[0m │\n", "├─────────────────────â”ŧ───────────────────────────────────────────────────────────────────────────────────────────┤\n", "│ product │ \u001b[1m{\u001b[0m │\n", "│ │ \u001b[32m'name'\u001b[0m: \u001b[32m'Ergonomic Under-Desk Storage Cabinet'\u001b[0m, │\n", "│ │ \u001b[32m'description'\u001b[0m: \u001b[32m'A sleek and durable storage cabinet designed to fit under desks in a \u001b[0m │\n", "│ │ \u001b[32mhome office setup. It features adjustable shelves, a modern design to match various \u001b[0m │\n", "│ │ \u001b[32mdecors, and provides ample space for organizing office supplies, documents, and \u001b[0m │\n", "│ │ \u001b[32melectronics. Ideal for professionals aged 35-50 looking to maximize space efficiency.'\u001b[0m, │\n", "│ │ \u001b[32m'price'\u001b[0m: \u001b[1;36m149.99\u001b[0m │\n", "│ │ \u001b[1m}\u001b[0m │\n", "├─────────────────────â”ŧ───────────────────────────────────────────────────────────────────────────────────────────┤\n", "│ customer_review │ \u001b[1m{\u001b[0m │\n", "│ │ \u001b[32m'rating'\u001b[0m: \u001b[1;36m5\u001b[0m, │\n", "│ │ \u001b[32m'customer_mood'\u001b[0m: \u001b[32m'happy'\u001b[0m, │\n", "│ │ \u001b[32m'review'\u001b[0m: \u001b[32m'The Ergonomic Under-Desk Storage Cabinet is a well-designed, durable \u001b[0m │\n", "│ │ \u001b[32msolution for optimizing space in a home office. Its adjustable shelves and modern \u001b[0m │\n", "│ │ \u001b[32maesthetics make it both functional and visually appealing, particularly suited for \u001b[0m │\n", "│ │ \u001b[32mprofessionals seeking efficiency.'\u001b[0m │\n", "│ │ \u001b[1m}\u001b[0m │\n", "└─────────────────────┴───────────────────────────────────────────────────────────────────────────────────────────┘\n", " \n", " [index: 0] \n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Run this cell multiple times to cycle through the 10 preview records.\n", "preview.display_sample_record()" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
customerproduct_categoryproduct_subcategorytarget_age_rangereview_stylecustomer_namecustomer_ageproductcustomer_review
0{'uuid': '02a9fa30-045c-4443-9223-d314546ce65b...Home OfficeStorage35-50briefSteven Caldwell71{'name': 'Ergonomic Under-Desk Storage Cabinet...{'rating': 5, 'customer_mood': 'happy', 'revie...
1{'uuid': 'c2631f8f-5d17-4a41-b9f1-36297b9b9e8a...Home OfficeStorage50-65detailedRobert May61{'name': 'Ergonomic Wall-Mounted File Cabinet'...{'rating': 5, 'customer_mood': 'happy', 'revie...
2{'uuid': '5fb4e814-4186-4d55-a6ca-edef9c3df344...ElectronicsCameras50-65structured with bullet pointsJennifer Morse21{'name': 'RetroView Digital Camera', 'descript...{'rating': 5, 'customer_mood': 'happy', 'revie...
3{'uuid': '194eb1db-006e-45e8-ad48-48f9b4bf22ed...ElectronicsHeadphones65+structured with bullet pointsDavid Mays24{'name': 'EarComfort Wireless Headphones', 'de...{'rating': 5, 'customer_mood': 'happy', 'revie...
4{'uuid': '727adea0-9197-46e2-a3ea-8b17585e9b6c...Home OfficeLighting50-65ramblingKerri Diaz102{'name': 'Adjustable LED Desk Lamp', 'descript...{'rating': 4, 'customer_mood': 'happy', 'revie...
5{'uuid': '5fc887bf-fdda-4698-8251-be51f9402a4a...ClothingWomen's Clothing35-50structured with bullet pointsDanielle Romero21{'name': 'Elegant Floral Maxi Dress', 'descrip...{'rating': 5, 'customer_mood': 'happy', 'revie...
6{'uuid': '74c81e95-b43f-487d-b31e-7cf429fffbeb...ElectronicsHeadphones35-50briefRyan Martinez48{'name': 'Premium Noise-Cancelling Over-Ear He...{'rating': 5, 'customer_mood': 'happy', 'revie...
7{'uuid': '8bc6fdf7-1ba7-49cb-b06b-2d1e0f994a7e...Home & KitchenDecor50-65ramblingMary Weiss59{'name': 'Timeless Heritage Wall Art Set', 'de...{'rating': 5, 'customer_mood': 'happy', 'revie...
8{'uuid': '94b5f377-8593-40af-81d2-cd5542b6ff33...Home & KitchenDecor35-50detailedNorma Meadows90{'name': 'Modern Geometric Wall Art Set', 'des...{'rating': 5, 'customer_mood': 'happy', 'revie...
9{'uuid': '78b40c52-9f46-4486-a19a-bb9eca51f87d...Home & KitchenFurniture50-65detailedBrian Miller29{'name': 'Ergonomic Memory Foam Reading Chair'...{'rating': 5, 'customer_mood': 'happy', 'revie...
\n", "
" ], "text/plain": [ " customer product_category \\\n", "0 {'uuid': '02a9fa30-045c-4443-9223-d314546ce65b... Home Office \n", "1 {'uuid': 'c2631f8f-5d17-4a41-b9f1-36297b9b9e8a... Home Office \n", "2 {'uuid': '5fb4e814-4186-4d55-a6ca-edef9c3df344... Electronics \n", "3 {'uuid': '194eb1db-006e-45e8-ad48-48f9b4bf22ed... Electronics \n", "4 {'uuid': '727adea0-9197-46e2-a3ea-8b17585e9b6c... Home Office \n", "5 {'uuid': '5fc887bf-fdda-4698-8251-be51f9402a4a... Clothing \n", "6 {'uuid': '74c81e95-b43f-487d-b31e-7cf429fffbeb... Electronics \n", "7 {'uuid': '8bc6fdf7-1ba7-49cb-b06b-2d1e0f994a7e... Home & Kitchen \n", "8 {'uuid': '94b5f377-8593-40af-81d2-cd5542b6ff33... Home & Kitchen \n", "9 {'uuid': '78b40c52-9f46-4486-a19a-bb9eca51f87d... Home & Kitchen \n", "\n", " product_subcategory target_age_range review_style \\\n", "0 Storage 35-50 brief \n", "1 Storage 50-65 detailed \n", "2 Cameras 50-65 structured with bullet points \n", "3 Headphones 65+ structured with bullet points \n", "4 Lighting 50-65 rambling \n", "5 Women's Clothing 35-50 structured with bullet points \n", "6 Headphones 35-50 brief \n", "7 Decor 50-65 rambling \n", "8 Decor 35-50 detailed \n", "9 Furniture 50-65 detailed \n", "\n", " customer_name customer_age \\\n", "0 Steven Caldwell 71 \n", "1 Robert May 61 \n", "2 Jennifer Morse 21 \n", "3 David Mays 24 \n", "4 Kerri Diaz 102 \n", "5 Danielle Romero 21 \n", "6 Ryan Martinez 48 \n", "7 Mary Weiss 59 \n", "8 Norma Meadows 90 \n", "9 Brian Miller 29 \n", "\n", " product \\\n", "0 {'name': 'Ergonomic Under-Desk Storage Cabinet... \n", "1 {'name': 'Ergonomic Wall-Mounted File Cabinet'... \n", "2 {'name': 'RetroView Digital Camera', 'descript... \n", "3 {'name': 'EarComfort Wireless Headphones', 'de... \n", "4 {'name': 'Adjustable LED Desk Lamp', 'descript... \n", "5 {'name': 'Elegant Floral Maxi Dress', 'descrip... \n", "6 {'name': 'Premium Noise-Cancelling Over-Ear He... \n", "7 {'name': 'Timeless Heritage Wall Art Set', 'de... \n", "8 {'name': 'Modern Geometric Wall Art Set', 'des... \n", "9 {'name': 'Ergonomic Memory Foam Reading Chair'... \n", "\n", " customer_review \n", "0 {'rating': 5, 'customer_mood': 'happy', 'revie... \n", "1 {'rating': 5, 'customer_mood': 'happy', 'revie... \n", "2 {'rating': 5, 'customer_mood': 'happy', 'revie... \n", "3 {'rating': 5, 'customer_mood': 'happy', 'revie... \n", "4 {'rating': 4, 'customer_mood': 'happy', 'revie... \n", "5 {'rating': 5, 'customer_mood': 'happy', 'revie... \n", "6 {'rating': 5, 'customer_mood': 'happy', 'revie... \n", "7 {'rating': 5, 'customer_mood': 'happy', 'revie... \n", "8 {'rating': 5, 'customer_mood': 'happy', 'revie... \n", "9 {'rating': 5, 'customer_mood': 'happy', 'revie... " ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# The preview dataset is available as a pandas DataFrame.\n", "preview.dataset" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 📊 Analyze the generated data\n", "\n", "- Data Designer automatically generates a basic statistical analysis of the generated data.\n", "\n", "- This analysis is available via the `analysis` property of generation result objects.\n" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
──────────────────────────────────────── 🎨 Data Designer Dataset Profile ─────────────────────────────────────────\n",
       "\n",
       "                                                                                                                   \n",
       "                                                 Dataset Overview                                                  \n",
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃ number of records               ┃ number of columns               ┃ percent complete records                    ┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│ 10                              │ 9                               │ 100.0%                                      │\n",
       "└─────────────────────────────────┴─────────────────────────────────┴─────────────────────────────────────────────┘\n",
       "                                                                                                                   \n",
       "                                                                                                                   \n",
       "                                                🎲 Sampler Columns                                                 \n",
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃ column name                    ┃       data type ┃            number unique values ┃               sampler type ┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│ customer                       │            dict │                     10 (100.0%) │          person_from_faker │\n",
       "├────────────────────────────────â”ŧ─────────────────â”ŧ─────────────────────────────────â”ŧ────────────────────────────┤\n",
       "│ product_category               │          string │                       4 (40.0%) │                   category │\n",
       "├────────────────────────────────â”ŧ─────────────────â”ŧ─────────────────────────────────â”ŧ────────────────────────────┤\n",
       "│ product_subcategory            │          string │                       7 (70.0%) │                subcategory │\n",
       "├────────────────────────────────â”ŧ─────────────────â”ŧ─────────────────────────────────â”ŧ────────────────────────────┤\n",
       "│ target_age_range               │          string │                       3 (30.0%) │                   category │\n",
       "├────────────────────────────────â”ŧ─────────────────â”ŧ─────────────────────────────────â”ŧ────────────────────────────┤\n",
       "│ review_style                   │          string │                       4 (40.0%) │                   category │\n",
       "└────────────────────────────────┴─────────────────┴─────────────────────────────────┴────────────────────────────┘\n",
       "                                                                                                                   \n",
       "                                                                                                                   \n",
       "                                             🗂ī¸ LLM-Structured Columns                                              \n",
       "┏━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃                       ┃               ┃                            ┃     prompt tokens ┃      completion tokens ┃\n",
       "┃ column name           ┃     data type ┃       number unique values ┃        per record ┃             per record ┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│ product               │          dict │                10 (100.0%) │     268.0 +/- 0.7 │           77.0 +/- 8.7 │\n",
       "├───────────────────────â”ŧ───────────────â”ŧ────────────────────────────â”ŧ───────────────────â”ŧ────────────────────────┤\n",
       "│ customer_review       │          dict │                10 (100.0%) │     321.0 +/- 7.1 │        176.0 +/- 137.6 │\n",
       "└───────────────────────┴───────────────┴────────────────────────────┴───────────────────┴────────────────────────┘\n",
       "                                                                                                                   \n",
       "                                                                                                                   \n",
       "                                               🧩 Expression Columns                                               \n",
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃ column name                       ┃                data type ┃                             number unique values ┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│ customer_name                     │                   string │                                      10 (100.0%) │\n",
       "├───────────────────────────────────â”ŧ──────────────────────────â”ŧ──────────────────────────────────────────────────┤\n",
       "│ customer_age                      │                   string │                                        9 (90.0%) │\n",
       "└───────────────────────────────────┴──────────────────────────┴──────────────────────────────────────────────────┘\n",
       "                                                                                                                   \n",
       "                                                                                                                   \n",
       "╭────────────────────────────────────────────────── Table Notes ──────────────────────────────────────────────────╮\n",
       "│                                                                                                                 │\n",
       "│  1. All token statistics are based on a sample of max(1000, len(dataset)) records.                              │\n",
       "│  2. Tokens are calculated using tiktoken's cl100k_base tokenizer.                                               │\n",
       "│                                                                                                                 │\n",
       "╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯\n",
       "                                                                                                                   \n",
       "───────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n",
       "
\n" ], "text/plain": [ "\u001b[1;38;2;118;185;0m──────────────────────────────────────── \u001b[0m🎨 Data Designer Dataset Profile\u001b[1;38;2;118;185;0m ─────────────────────────────────────────\u001b[0m\n", "\n", " \n", "\u001b[1;38;2;118;185;0m Dataset Overview \u001b[0m\n", "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n", "┃\u001b[1;2m \u001b[0m\u001b[1;2mnumber of records \u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2mnumber of columns \u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2mpercent complete records \u001b[0m\u001b[1;2m \u001b[0m┃\n", "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n", "│ 10 │ 9 │ 100.0% │\n", "└─────────────────────────────────┴─────────────────────────────────┴─────────────────────────────────────────────┘\n", " \n", " \n", "\u001b[1;38;2;118;185;0m 🎲 Sampler Columns \u001b[0m\n", "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n", "┃\u001b[1;2m \u001b[0m\u001b[1;2mcolumn name \u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m data type\u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m number unique values\u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m sampler type\u001b[0m\u001b[1;2m \u001b[0m┃\n", "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n", "│ customer │ dict │ 10 (100.0%) │ person_from_faker │\n", "├────────────────────────────────â”ŧ─────────────────â”ŧ─────────────────────────────────â”ŧ────────────────────────────┤\n", "│ product_category │ string │ 4 (40.0%) │ category │\n", "├────────────────────────────────â”ŧ─────────────────â”ŧ─────────────────────────────────â”ŧ────────────────────────────┤\n", "│ product_subcategory │ string │ 7 (70.0%) │ subcategory │\n", "├────────────────────────────────â”ŧ─────────────────â”ŧ─────────────────────────────────â”ŧ────────────────────────────┤\n", "│ target_age_range │ string │ 3 (30.0%) │ category │\n", "├────────────────────────────────â”ŧ─────────────────â”ŧ─────────────────────────────────â”ŧ────────────────────────────┤\n", "│ review_style │ string │ 4 (40.0%) │ category │\n", "└────────────────────────────────┴─────────────────┴─────────────────────────────────┴────────────────────────────┘\n", " \n", " \n", "\u001b[1;38;2;118;185;0m 🗂ī¸ LLM-Structured Columns \u001b[0m\n", "┏━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━┓\n", "┃\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m prompt tokens\u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m completion tokens\u001b[0m\u001b[1;2m \u001b[0m┃\n", "┃\u001b[1;2m \u001b[0m\u001b[1;2mcolumn name \u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m data type\u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m number unique values\u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m per record\u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m per record\u001b[0m\u001b[1;2m \u001b[0m┃\n", "┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩\n", "│ product │ dict │ 10 (100.0%) │ 268.0 +/- 0.7 │ 77.0 +/- 8.7 │\n", "├───────────────────────â”ŧ───────────────â”ŧ────────────────────────────â”ŧ───────────────────â”ŧ────────────────────────┤\n", "│ customer_review │ dict │ 10 (100.0%) │ 321.0 +/- 7.1 │ 176.0 +/- 137.6 │\n", "└───────────────────────┴───────────────┴────────────────────────────┴───────────────────┴────────────────────────┘\n", " \n", " \n", "\u001b[1;38;2;118;185;0m 🧩 Expression Columns \u001b[0m\n", "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n", "┃\u001b[1;2m \u001b[0m\u001b[1;2mcolumn name \u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m data type\u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m number unique values\u001b[0m\u001b[1;2m \u001b[0m┃\n", "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n", "│ customer_name │ string │ 10 (100.0%) │\n", "├───────────────────────────────────â”ŧ──────────────────────────â”ŧ──────────────────────────────────────────────────┤\n", "│ customer_age │ string │ 9 (90.0%) │\n", "└───────────────────────────────────┴──────────────────────────┴──────────────────────────────────────────────────┘\n", " \n", " \n", "\u001b[2m╭─\u001b[0m\u001b[2m─────────────────────────────────────────────────\u001b[0m\u001b[2m Table Notes \u001b[0m\u001b[2m─────────────────────────────────────────────────\u001b[0m\u001b[2m─╮\u001b[0m\n", "\u001b[2m│\u001b[0m \u001b[2m│\u001b[0m\n", "\u001b[2m│\u001b[0m 1. All token statistics are based on a sample of max(1000, len(dataset)) records. \u001b[2m│\u001b[0m\n", "\u001b[2m│\u001b[0m 2. Tokens are calculated using tiktoken's cl100k_base tokenizer. \u001b[2m│\u001b[0m\n", "\u001b[2m│\u001b[0m \u001b[2m│\u001b[0m\n", "\u001b[2m╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯\u001b[0m\n", " \n", "\u001b[1;38;2;118;185;0m───────────────────────────────────────────────────────────────────────────────────────────────────────────────────\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Print the analysis as a table.\n", "preview.analysis.to_report()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### 🆙 Scale up!\n", "\n", "- Happy with your preview data?\n", "\n", "- Use the `create` method to submit larger Data Designer generation jobs.\n" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "[16:30:13] [INFO] 🎨 Creating Data Designer dataset\n", "[16:30:13] [INFO] ✅ Validation passed\n", "[16:30:13] [INFO] ⛓ī¸ Sorting column configs into a Directed Acyclic Graph\n", "[16:30:13] [INFO] đŸŠē Running health checks for models...\n", "[16:30:13] [INFO] |-- 👀 Checking 'nvidia/nvidia-nemotron-nano-9b-v2' in provider named 'nvidia' for model alias 'nemotron-nano-v2'...\n", "[16:30:14] [INFO] |-- ✅ Passed!\n", "[16:30:14] [INFO] âŗ Processing batch 1 of 1\n", "[16:30:14] [INFO] 🎲 Preparing samplers to generate 20 records across 5 columns\n", "[16:30:14] [INFO] 🧩 Generating column `customer_name` from expression\n", "[16:30:14] [INFO] 🧩 Generating column `customer_age` from expression\n", "[16:30:14] [INFO] 🗂ī¸ Preparing llm-structured column generation\n", "[16:30:14] [INFO] |-- column name: 'product'\n", "[16:30:14] [INFO] |-- model config:\n", "{\n", " \"alias\": \"nemotron-nano-v2\",\n", " \"model\": \"nvidia/nvidia-nemotron-nano-9b-v2\",\n", " \"inference_parameters\": {\n", " \"temperature\": 0.5,\n", " \"top_p\": 1.0,\n", " \"max_tokens\": 1024,\n", " \"max_parallel_requests\": 4,\n", " \"timeout\": null,\n", " \"extra_body\": null\n", " },\n", " \"provider\": \"nvidia\"\n", "}\n", "[16:30:14] [INFO] 🐙 Processing llm-structured column 'product' with 4 concurrent workers\n", "[16:30:22] [INFO] 🗂ī¸ Preparing llm-structured column generation\n", "[16:30:22] [INFO] |-- column name: 'customer_review'\n", "[16:30:22] [INFO] |-- model config:\n", "{\n", " \"alias\": \"nemotron-nano-v2\",\n", " \"model\": \"nvidia/nvidia-nemotron-nano-9b-v2\",\n", " \"inference_parameters\": {\n", " \"temperature\": 0.5,\n", " \"top_p\": 1.0,\n", " \"max_tokens\": 1024,\n", " \"max_parallel_requests\": 4,\n", " \"timeout\": null,\n", " \"extra_body\": null\n", " },\n", " \"provider\": \"nvidia\"\n", "}\n", "[16:30:22] [INFO] 🐙 Processing llm-structured column 'customer_review' with 4 concurrent workers\n", "[16:30:35] [INFO] 📊 Model usage summary:\n", "{\n", " \"nvidia/nvidia-nemotron-nano-9b-v2\": {\n", " \"token_usage\": {\n", " \"prompt_tokens\": 12414,\n", " \"completion_tokens\": 5547,\n", " \"total_tokens\": 17961\n", " },\n", " \"request_usage\": {\n", " \"successful_requests\": 40,\n", " \"failed_requests\": 0,\n", " \"total_requests\": 40\n", " },\n", " \"tokens_per_second\": 876,\n", " \"requests_per_minute\": 117\n", " }\n", "}\n", "[16:30:35] [INFO] 📐 Measuring dataset column statistics:\n", "[16:30:35] [INFO] |-- 🎲 column: 'customer'\n", "[16:30:35] [INFO] |-- 🎲 column: 'product_category'\n", "[16:30:35] [INFO] |-- 🎲 column: 'product_subcategory'\n", "[16:30:35] [INFO] |-- 🎲 column: 'target_age_range'\n", "[16:30:35] [INFO] |-- 🎲 column: 'review_style'\n", "[16:30:35] [INFO] |-- 🧩 column: 'customer_name'\n", "[16:30:35] [INFO] |-- 🧩 column: 'customer_age'\n", "[16:30:35] [INFO] |-- 🗂ī¸ column: 'product'\n", "[16:30:35] [INFO] |-- 🗂ī¸ column: 'customer_review'\n" ] } ], "source": [ "job_results = data_designer_client.create(config_builder, num_records=20)" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
customerproduct_categoryproduct_subcategorytarget_age_rangereview_stylecustomer_namecustomer_ageproductcustomer_review
0{'age': 93, 'bachelors_field': 'stem', 'birth_...BooksClassics35-50structured with bullet pointsAngela Mitchell93{'description': 'A curated selection of classi...{'customer_mood': 'happy', 'rating': 5, 'revie...
1{'age': 45, 'bachelors_field': 'no_degree', 'b...ClothingAccessories25-35detailedAlex Fletcher45{'description': 'A sleek and versatile crossbo...{'customer_mood': 'happy', 'rating': 5, 'revie...
2{'age': 101, 'bachelors_field': 'arts_humaniti...Home OfficeLighting25-35detailedKathryn Stafford101{'description': 'A sleek, adjustable LED desk ...{'customer_mood': 'happy', 'rating': 5, 'revie...
3{'age': 42, 'bachelors_field': 'arts_humanitie...BooksNon-Fiction35-50structured with bullet pointsDaniel Freeman42{'description': 'A non-fiction book exploring ...{'customer_mood': 'happy', 'rating': 5, 'revie...
4{'age': 39, 'bachelors_field': 'education', 'b...Home OfficeDesks50-65briefBrittany Acosta39{'description': 'A sturdy and comfortable desk...{'customer_mood': 'happy', 'rating': 5, 'revie...
\n", "
" ], "text/plain": [ " customer product_category \\\n", "0 {'age': 93, 'bachelors_field': 'stem', 'birth_... Books \n", "1 {'age': 45, 'bachelors_field': 'no_degree', 'b... Clothing \n", "2 {'age': 101, 'bachelors_field': 'arts_humaniti... Home Office \n", "3 {'age': 42, 'bachelors_field': 'arts_humanitie... Books \n", "4 {'age': 39, 'bachelors_field': 'education', 'b... Home Office \n", "\n", " product_subcategory target_age_range review_style \\\n", "0 Classics 35-50 structured with bullet points \n", "1 Accessories 25-35 detailed \n", "2 Lighting 25-35 detailed \n", "3 Non-Fiction 35-50 structured with bullet points \n", "4 Desks 50-65 brief \n", "\n", " customer_name customer_age \\\n", "0 Angela Mitchell 93 \n", "1 Alex Fletcher 45 \n", "2 Kathryn Stafford 101 \n", "3 Daniel Freeman 42 \n", "4 Brittany Acosta 39 \n", "\n", " product \\\n", "0 {'description': 'A curated selection of classi... \n", "1 {'description': 'A sleek and versatile crossbo... \n", "2 {'description': 'A sleek, adjustable LED desk ... \n", "3 {'description': 'A non-fiction book exploring ... \n", "4 {'description': 'A sturdy and comfortable desk... \n", "\n", " customer_review \n", "0 {'customer_mood': 'happy', 'rating': 5, 'revie... \n", "1 {'customer_mood': 'happy', 'rating': 5, 'revie... \n", "2 {'customer_mood': 'happy', 'rating': 5, 'revie... \n", "3 {'customer_mood': 'happy', 'rating': 5, 'revie... \n", "4 {'customer_mood': 'happy', 'rating': 5, 'revie... " ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Load the generated dataset as a pandas DataFrame.\n", "dataset = job_results.load_dataset()\n", "\n", "dataset.head()" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
──────────────────────────────────────── 🎨 Data Designer Dataset Profile ─────────────────────────────────────────\n",
       "\n",
       "                                                                                                                   \n",
       "                                                 Dataset Overview                                                  \n",
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃ number of records               ┃ number of columns               ┃ percent complete records                    ┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│ 20                              │ 9                               │ 100.0%                                      │\n",
       "└─────────────────────────────────┴─────────────────────────────────┴─────────────────────────────────────────────┘\n",
       "                                                                                                                   \n",
       "                                                                                                                   \n",
       "                                                🎲 Sampler Columns                                                 \n",
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃ column name                    ┃       data type ┃            number unique values ┃               sampler type ┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│ customer                       │            dict │                     20 (100.0%) │          person_from_faker │\n",
       "├────────────────────────────────â”ŧ─────────────────â”ŧ─────────────────────────────────â”ŧ────────────────────────────┤\n",
       "│ product_category               │          string │                       5 (25.0%) │                   category │\n",
       "├────────────────────────────────â”ŧ─────────────────â”ŧ─────────────────────────────────â”ŧ────────────────────────────┤\n",
       "│ product_subcategory            │          string │                      11 (55.0%) │                subcategory │\n",
       "├────────────────────────────────â”ŧ─────────────────â”ŧ─────────────────────────────────â”ŧ────────────────────────────┤\n",
       "│ target_age_range               │          string │                       5 (25.0%) │                   category │\n",
       "├────────────────────────────────â”ŧ─────────────────â”ŧ─────────────────────────────────â”ŧ────────────────────────────┤\n",
       "│ review_style                   │          string │                       4 (20.0%) │                   category │\n",
       "└────────────────────────────────┴─────────────────┴─────────────────────────────────┴────────────────────────────┘\n",
       "                                                                                                                   \n",
       "                                                                                                                   \n",
       "                                             🗂ī¸ LLM-Structured Columns                                              \n",
       "┏━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃                       ┃               ┃                            ┃     prompt tokens ┃      completion tokens ┃\n",
       "┃ column name           ┃     data type ┃       number unique values ┃        per record ┃             per record ┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│ product               │          dict │                20 (100.0%) │     268.0 +/- 0.8 │           63.5 +/- 7.5 │\n",
       "├───────────────────────â”ŧ───────────────â”ŧ────────────────────────────â”ŧ───────────────────â”ŧ────────────────────────┤\n",
       "│ customer_review       │          dict │                20 (100.0%) │     308.0 +/- 7.1 │         173.5 +/- 93.8 │\n",
       "└───────────────────────┴───────────────┴────────────────────────────┴───────────────────┴────────────────────────┘\n",
       "                                                                                                                   \n",
       "                                                                                                                   \n",
       "                                               🧩 Expression Columns                                               \n",
       "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n",
       "┃ column name                       ┃                data type ┃                             number unique values ┃\n",
       "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n",
       "│ customer_name                     │                   string │                                      20 (100.0%) │\n",
       "├───────────────────────────────────â”ŧ──────────────────────────â”ŧ──────────────────────────────────────────────────┤\n",
       "│ customer_age                      │                   string │                                       19 (95.0%) │\n",
       "└───────────────────────────────────┴──────────────────────────┴──────────────────────────────────────────────────┘\n",
       "                                                                                                                   \n",
       "                                                                                                                   \n",
       "╭────────────────────────────────────────────────── Table Notes ──────────────────────────────────────────────────╮\n",
       "│                                                                                                                 │\n",
       "│  1. All token statistics are based on a sample of max(1000, len(dataset)) records.                              │\n",
       "│  2. Tokens are calculated using tiktoken's cl100k_base tokenizer.                                               │\n",
       "│                                                                                                                 │\n",
       "╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯\n",
       "                                                                                                                   \n",
       "───────────────────────────────────────────────────────────────────────────────────────────────────────────────────\n",
       "
\n" ], "text/plain": [ "\u001b[1;38;2;118;185;0m──────────────────────────────────────── \u001b[0m🎨 Data Designer Dataset Profile\u001b[1;38;2;118;185;0m ─────────────────────────────────────────\u001b[0m\n", "\n", " \n", "\u001b[1;38;2;118;185;0m Dataset Overview \u001b[0m\n", "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n", "┃\u001b[1;2m \u001b[0m\u001b[1;2mnumber of records \u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2mnumber of columns \u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2mpercent complete records \u001b[0m\u001b[1;2m \u001b[0m┃\n", "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n", "│ 20 │ 9 │ 100.0% │\n", "└─────────────────────────────────┴─────────────────────────────────┴─────────────────────────────────────────────┘\n", " \n", " \n", "\u001b[1;38;2;118;185;0m 🎲 Sampler Columns \u001b[0m\n", "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n", "┃\u001b[1;2m \u001b[0m\u001b[1;2mcolumn name \u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m data type\u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m number unique values\u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m sampler type\u001b[0m\u001b[1;2m \u001b[0m┃\n", "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n", "│ customer │ dict │ 20 (100.0%) │ person_from_faker │\n", "├────────────────────────────────â”ŧ─────────────────â”ŧ─────────────────────────────────â”ŧ────────────────────────────┤\n", "│ product_category │ string │ 5 (25.0%) │ category │\n", "├────────────────────────────────â”ŧ─────────────────â”ŧ─────────────────────────────────â”ŧ────────────────────────────┤\n", "│ product_subcategory │ string │ 11 (55.0%) │ subcategory │\n", "├────────────────────────────────â”ŧ─────────────────â”ŧ─────────────────────────────────â”ŧ────────────────────────────┤\n", "│ target_age_range │ string │ 5 (25.0%) │ category │\n", "├────────────────────────────────â”ŧ─────────────────â”ŧ─────────────────────────────────â”ŧ────────────────────────────┤\n", "│ review_style │ string │ 4 (20.0%) │ category │\n", "└────────────────────────────────┴─────────────────┴─────────────────────────────────┴────────────────────────────┘\n", " \n", " \n", "\u001b[1;38;2;118;185;0m 🗂ī¸ LLM-Structured Columns \u001b[0m\n", "┏━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━┓\n", "┃\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m prompt tokens\u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m completion tokens\u001b[0m\u001b[1;2m \u001b[0m┃\n", "┃\u001b[1;2m \u001b[0m\u001b[1;2mcolumn name \u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m data type\u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m number unique values\u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m per record\u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m per record\u001b[0m\u001b[1;2m \u001b[0m┃\n", "┡━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━┩\n", "│ product │ dict │ 20 (100.0%) │ 268.0 +/- 0.8 │ 63.5 +/- 7.5 │\n", "├───────────────────────â”ŧ───────────────â”ŧ────────────────────────────â”ŧ───────────────────â”ŧ────────────────────────┤\n", "│ customer_review │ dict │ 20 (100.0%) │ 308.0 +/- 7.1 │ 173.5 +/- 93.8 │\n", "└───────────────────────┴───────────────┴────────────────────────────┴───────────────────┴────────────────────────┘\n", " \n", " \n", "\u001b[1;38;2;118;185;0m 🧩 Expression Columns \u001b[0m\n", "┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━â”ŗ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓\n", "┃\u001b[1;2m \u001b[0m\u001b[1;2mcolumn name \u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m data type\u001b[0m\u001b[1;2m \u001b[0m┃\u001b[1;2m \u001b[0m\u001b[1;2m number unique values\u001b[0m\u001b[1;2m \u001b[0m┃\n", "┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩\n", "│ customer_name │ string │ 20 (100.0%) │\n", "├───────────────────────────────────â”ŧ──────────────────────────â”ŧ──────────────────────────────────────────────────┤\n", "│ customer_age │ string │ 19 (95.0%) │\n", "└───────────────────────────────────┴──────────────────────────┴──────────────────────────────────────────────────┘\n", " \n", " \n", "\u001b[2m╭─\u001b[0m\u001b[2m─────────────────────────────────────────────────\u001b[0m\u001b[2m Table Notes \u001b[0m\u001b[2m─────────────────────────────────────────────────\u001b[0m\u001b[2m─╮\u001b[0m\n", "\u001b[2m│\u001b[0m \u001b[2m│\u001b[0m\n", "\u001b[2m│\u001b[0m 1. All token statistics are based on a sample of max(1000, len(dataset)) records. \u001b[2m│\u001b[0m\n", "\u001b[2m│\u001b[0m 2. Tokens are calculated using tiktoken's cl100k_base tokenizer. \u001b[2m│\u001b[0m\n", "\u001b[2m│\u001b[0m \u001b[2m│\u001b[0m\n", "\u001b[2m╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯\u001b[0m\n", " \n", "\u001b[1;38;2;118;185;0m───────────────────────────────────────────────────────────────────────────────────────────────────────────────────\u001b[0m\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Load the analysis results into memory.\n", "analysis = job_results.load_analysis()\n", "\n", "analysis.to_report()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## ⏭ī¸ Next Steps\n", "\n", "Check out the following notebook to learn more about:\n", "\n", "- [Seeding synthetic data generation with an external dataset](/notebooks/3-seeding-with-a-dataset/)\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.9" } }, "nbformat": 4, "nbformat_minor": 4 }