openapi: 3.1.0 info: title: HoneyHive Data Plane API license: name: MIT url: https://opensource.org/licenses/MIT version: 1.7.0 servers: - url: https://api.dp1.us.honeyhive.ai tags: - name: Charts description: | Define and manage saved charts. Charts are visualizations that aggregate metrics over time with bucketing, filters, and groupings. - name: Configurations description: | Manage prompt configurations, i.e. model parameters, message templates, and response settings that can be versioned and deployed without code changes. - name: Datapoints description: | Manage individual records inside datasets, including batch creation and mapping to source events. - name: Datasets description: | Curate collections of datapoints used as test sets for evaluations and experiments. - name: Events description: | Read and write trace events. Events are the spans that capture every step of an AI application's execution. - name: Experiments description: | Run, retrieve, and compare evaluation runs to measure how prompt or configuration changes affect agent performance. - name: Metric Versions description: | Snapshot, list, and deploy versions of a metric's definition so changes can be reviewed and rolled back without losing history. - name: Metrics description: | Define and run evaluators, i.e. automated quality checks that score traces against criteria like accuracy, safety, or correctness. - name: Queues description: | Manage annotation queues for human review of traces, turning expert feedback into labeled datasets. - name: Sessions description: | Group related trace events into sessions, the top-level container for a multi-step or multi-service AI interaction. paths: /session/start: post: summary: Start a new session (deprecated) operationId: startSessionLegacy x-cli-exclude: true x-ts-sdk-exclude: true deprecated: true tags: - Sessions description: | Deprecated. Use `POST /v1/sessions` instead. The legacy route wraps the session object under a `session` key; the v1 route accepts a bare session object. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/LegacyStartSessionRequest' responses: '200': description: Session successfully started content: application/json: schema: $ref: '#/components/schemas/PostSessionStartResponse' /session/{session_id}/traces: post: summary: Add traces to a session (deprecated) operationId: addSessionTracesLegacy x-cli-exclude: true x-ts-sdk-exclude: true deprecated: true tags: - Sessions description: | Deprecated. Use `POST /v1/sessions/{session_id}/events/batch` instead. This route accepts trace events under either `logs` (deprecated) or `events`. Exactly one of the two MUST be present in a request: neither → 400, both → 400. parameters: - name: session_id in: path required: true schema: type: string description: Session ID to add traces to requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/AddSessionTracesRequest' responses: '200': description: Traces added successfully content: application/json: schema: $ref: '#/components/schemas/SessionTracesResponse' '400': description: | Bad request: both `logs` and `events` supplied, or neither, or invalid event payload. /v1/sessions: post: tags: - Sessions operationId: createSession x-cli-name: create x-ts-sdk-name: create summary: Start a new session description: | Start a new session. The request body is a bare session object (no `session` wrapper). The server creates a session event and returns it. **No required properties.** Every field has a server-side fallback. **Auto-generated properties** (provided by the server when omitted): - `session_id` (string, UUID): Server generates a UUIDv4 if omitted or if the supplied value is not a valid UUID. **Optional properties with defaults:** - `event_name` (string): Falls back to `session_name` when not provided; defaults to `"unknown"` if both are absent. - `source` (string): Defaults to `"unknown"`. **Optional properties:** - `session_name` (string): Display name for the session. - `start_time` (number): Session start time as Unix milliseconds. The session normalizer uses `getInt64()` which only accepts numeric types; if a string is passed, the server silently falls back to the current time. - `end_time` (number): Session end time as Unix milliseconds (same numeric-only caveat as `start_time`). - `duration` (number): Session duration in milliseconds. - `config` (object): Configuration associated with the session. - `inputs` (object): Input data for the session. - `outputs` (object): Output data from the session. - `metadata` (object): Arbitrary metadata. - `user_properties` (object): User properties. - `children_ids` (array of strings): IDs of child events. Idempotent on `session_id`: posting twice with the same `session_id` merges metadata/user_properties into the existing session and returns the existing event. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/StartSessionRequest' responses: '200': description: Session successfully started content: application/json: schema: $ref: '#/components/schemas/PostSessionStartResponse' '400': description: Bad request (invalid session data) /v1/sessions/{session_id}/events/batch: post: tags: - Sessions operationId: createSessionEventBatch x-cli-name: create-event-batch x-ts-sdk-name: createEventBatch summary: Add a batch of events to a session description: | Add a batch of events to an existing session. Each event in the batch is stored with `session_id` set from the URL path, overriding any `session_id` in the event body. Each event must include `event_type` (one of `chain`, `model`, `tool`, `session`) and `inputs`. Unknown top-level fields and unknown per-event fields are rejected. Events are processed sequentially. For higher-throughput ingestion across sessions, use `POST /v1/events/batch` instead. parameters: - name: session_id in: path required: true schema: type: string format: uuid description: Session ID to add events to requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/SessionEventBatchRequest' responses: '200': description: Events added successfully content: application/json: schema: $ref: '#/components/schemas/SessionTracesResponse' example: success: true '400': description: Bad request (invalid event data or missing required fields) '500': description: Events partially added /events: post: tags: - Events operationId: createEventLegacy x-cli-exclude: true x-ts-sdk-exclude: true deprecated: true summary: Create a new event (deprecated) description: | Deprecated. Use `POST /v1/events` instead. The legacy route wraps the event object under an `event` key; the v1 route accepts a bare event object. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/LegacyPostEventRequest' responses: '200': description: Event created successfully content: application/json: schema: $ref: '#/components/schemas/PostEventResponse' example: event_id: 7f22137a-6911-4ed3-bc36-110f1dde6b66 success: true '400': description: Bad request (invalid event data or missing required fields) put: tags: - Events operationId: updateEventLegacy x-cli-exclude: true x-ts-sdk-exclude: true deprecated: true summary: Update an event (deprecated) description: | Deprecated. Use `PUT /v1/events/{event_id}` instead. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/LegacyUpdateEventRequest' example: event_id: 7f22137a-6911-4ed3-bc36-110f1dde6b66 metadata: cost: 0.00008 completion_tokens: 23 prompt_tokens: 35 total_tokens: 58 feedback: rating: 5 metrics: num_words: 2 outputs: role: assistant content: Hello world config: template: - role: system content: Hello, {{ name }}! user_properties: user_id: 691b1f94-d38c-4e92-b051-5e03fee9ff86 duration: 42 responses: '200': description: Event updated '400': description: Bad request /v1/events: post: tags: - Events operationId: createEvent x-cli-name: create x-ts-sdk-name: create summary: Create a new event description: | Create a new event (span) within a session trace. The request body is a bare event object (no `event` wrapper). **Required properties:** - `event_type` (string): Must be one of: `chain`, `model`, `tool`, `session`. - `inputs` (object): Input data for the event. **Auto-generated properties** (provided by the server when omitted): - `event_id` (string, UUID): Unique identifier for the event. - `session_id` (string, UUID): Session/trace identifier. - `parent_id` (string, UUID): Parent event ID. Defaults to `session_id`. **Optional properties with defaults:** - `event_name` (string): Name of the event. Defaults to `"unknown"`. - `source` (string): Source of the event (e.g. `sdk-python`). Defaults to `"unknown"`. **Optional properties:** - `config` (object): Configuration data (e.g. model parameters, prompt templates). - `outputs` (object): Output data from the event. - `error` (string or null): Error message if the event failed. - `children_ids` (array of strings): IDs of child events. - `duration` (number): Duration of the event in milliseconds. - `start_time` (number): Unix timestamp in milliseconds for event start. - `end_time` (number): Unix timestamp in milliseconds for event end. - `metadata` (object): Additional metadata (e.g. token counts, cost). - `metrics` (object): Custom metrics. - `feedback` (object): Feedback data (e.g. ratings, ground truth). - `user_properties` (object): User properties associated with the event. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/PostEventRequest' responses: '200': description: Event created successfully content: application/json: schema: $ref: '#/components/schemas/PostEventResponse' example: event_id: 7f22137a-6911-4ed3-bc36-110f1dde6b66 success: true '400': description: Bad request (invalid event data or missing required fields) /v1/events/{event_id}: get: tags: - Events operationId: getEvent x-cli-name: get x-ts-sdk-name: get summary: Get an event by ID description: | Retrieve a single event by its unique identifier. The event is fetched directly from S3/MinIO storage. parameters: - in: path name: event_id required: true schema: type: string format: uuid description: The unique identifier of the event to retrieve responses: '200': description: Event retrieved successfully content: application/json: schema: $ref: '#/components/schemas/GetEventResponse' '400': description: Bad request (invalid event_id) '404': description: Event not found put: tags: - Events operationId: updateEvent x-cli-name: update x-ts-sdk-name: update summary: Update an event description: | Update fields on an existing event. Only the provided fields are modified; omitted fields are left unchanged. Extra fields not listed below are accepted by the server but silently ignored. parameters: - in: path name: event_id required: true schema: type: string format: uuid description: The unique identifier of the event to update requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/UpdateEventRequest' example: metadata: cost: 0.00008 completion_tokens: 23 prompt_tokens: 35 total_tokens: 58 feedback: rating: 5 metrics: num_words: 2 outputs: role: assistant content: Hello world config: template: - role: system content: Hello, {{ name }}! user_properties: user_id: 691b1f94-d38c-4e92-b051-5e03fee9ff86 duration: 42 responses: '200': description: Event updated '400': description: Bad request /v1/events/export: post: tags: - Events operationId: exportEventsLegacy x-cli-exclude: true x-ts-sdk-exclude: true deprecated: true summary: Retrieve events based on filters (deprecated) description: | Deprecated. Use `POST /v1/events/search` instead. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/LegacyExportEventsRequest' responses: '200': description: Success content: application/json: schema: $ref: '#/components/schemas/ExportEventsResponse' /v1/events/search: post: tags: - Events operationId: searchEvents x-cli-name: search x-ts-sdk-name: search summary: Retrieve events based on filters description: | Search events via POST with filtering and pagination. This is the primary method for retrieving events from HoneyHive. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/SearchEventsRequest' responses: '200': description: Success content: application/json: schema: $ref: '#/components/schemas/ExportEventsResponse' /v1/events/batch: post: tags: - Events operationId: createEventBatch x-cli-name: create-batch x-ts-sdk-name: createBatch summary: Create a batch of events description: | Create multiple events in a single request. When `single_session` is true, all events share the same session created from `session_properties`. **Required properties:** - `events` (array of event objects): Each event must include `event_type` (one of `chain`, `model`, `tool`, `session`) and `inputs`. **Optional properties:** - `single_session` (boolean): If true, all events share a single session created from `session_properties`. Defaults to false. - `session_properties` (object): Session metadata used when `single_session` is true. May include `session_name`, `start_time`, `metadata`. Unknown top-level fields and per-event fields are rejected at the SDK boundary; the legacy aliases `is_single_session`, `session`, and per-event `project` are no longer accepted. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/PostEventBatchRequest' responses: '200': description: Events created content: application/json: schema: $ref: '#/components/schemas/PostEventBatchResponse' example: event_ids: - 7f22137a-6911-4ed3-bc36-110f1dde6b66 - 7f22137a-6911-4ed3-bc36-110f1dde6b67 session_id: caf77ace-3417-4da4-944d-f4a0688f3c23 success: true '400': description: Bad request (invalid event data or missing required fields) '500': description: Events partially created /events/model: post: tags: - Events operationId: createModelEventLegacy x-cli-exclude: true x-ts-sdk-exclude: true deprecated: true summary: Create a new model event (deprecated) description: | Deprecated. Use `POST /v1/events` with `event_type="model"` instead. The `ModelEvent` schema carries 14 field-level deprecations covering the model-specific fields the generic event route now expresses (`model`, `messages`, `response`, `provider`, `usage`, `cost`, `hyperparameters`, `template`, `template_inputs`, `tools`, `tool_choice`, `response_format`, `duration`, `error`), plus `project` inherited from `LegacyEvent`. The legacy route continues to serve traffic and remaps these fields into `inputs.*` / `outputs.*` before storage. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/PostModelEventRequest' responses: '200': description: Model event created content: application/json: schema: $ref: '#/components/schemas/PostEventResponse' example: event_id: 7f22137a-6911-4ed3-bc36-110f1dde6b66 success: true /events/batch: post: tags: - Events operationId: createEventBatchLegacy x-cli-exclude: true x-ts-sdk-exclude: true deprecated: true summary: Create a batch of events (deprecated) description: | Deprecated. Use `POST /v1/events/batch` instead. The legacy route accepts the deprecated `is_single_session` and `session` aliases and lets per-event objects carry the deprecated `project` field; the v1 route rejects all three at the SDK boundary. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/LegacyPostEventBatchRequest' responses: '200': description: Events created content: application/json: schema: $ref: '#/components/schemas/PostEventBatchResponse' example: event_ids: - 7f22137a-6911-4ed3-bc36-110f1dde6b66 - 7f22137a-6911-4ed3-bc36-110f1dde6b67 session_id: caf77ace-3417-4da4-944d-f4a0688f3c23 success: true '500': description: Events partially created /events/model/batch: post: tags: - Events operationId: createModelEventBatchLegacy x-cli-exclude: true x-ts-sdk-exclude: true deprecated: true summary: Create a batch of model events (deprecated) description: | Deprecated. Use `POST /v1/events/batch` with `event_type="model"` on each event instead. Migration notes: the top-level array `model_events` becomes `events`; each element must explicitly set `event_type: "model"` (the legacy route sets this server-side); the deprecated top-level aliases `is_single_session` and `session` are not accepted by the v1 route (`PostEventBatchRequest` is `.strict()`); use `single_session` and `session_properties` instead. The legacy route continues to serve traffic and remaps the model-specific fields on each event (`model`, `messages`, `response`, `provider`, `usage`, `cost`, `hyperparameters`, `template`, `template_inputs`, `tools`, `tool_choice`, `response_format`, `duration`, `error`) into `inputs.*` / `outputs.*` before storage. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/PostModelEventBatchRequest' responses: '200': description: Model events created content: application/json: schema: $ref: '#/components/schemas/PostEventBatchResponse' example: event_ids: - 7f22137a-6911-4ed3-bc36-110f1dde6b66 - 7f22137a-6911-4ed3-bc36-110f1dde6b67 success: true '500': description: Model events partially created /v1/charts: get: tags: - Charts operationId: getCharts x-cli-name: list x-ts-sdk-name: list summary: List charts description: Retrieve all charts in the current scope. responses: '200': description: Charts retrieved successfully content: application/json: schema: $ref: '#/components/schemas/GetChartsResponse' post: tags: - Charts operationId: createChart x-cli-name: create x-ts-sdk-name: create summary: Create a new chart requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/CreateChartRequest' responses: '200': description: Chart created successfully content: application/json: schema: $ref: '#/components/schemas/CreateChartResponse' /v1/charts/{chart_id}: get: tags: - Charts operationId: getChart x-cli-name: get x-ts-sdk-name: get summary: Get a chart description: Retrieve a single chart by id. parameters: - in: path name: chart_id required: true schema: type: string description: The unique identifier of the chart to retrieve responses: '200': description: Chart retrieved successfully content: application/json: schema: $ref: '#/components/schemas/GetChartResponse' put: tags: - Charts operationId: updateChart x-cli-name: update x-ts-sdk-name: update summary: Update a chart description: Update a chart's editable fields. Only fields included in the request body are modified. parameters: - in: path name: chart_id required: true schema: type: string description: The unique identifier of the chart to update requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/UpdateChartRequest' responses: '200': description: Chart updated successfully content: application/json: schema: $ref: '#/components/schemas/UpdateChartResponse' delete: tags: - Charts operationId: deleteChart x-cli-name: delete x-ts-sdk-name: delete summary: Delete a chart parameters: - in: path name: chart_id required: true schema: type: string description: The unique identifier of the chart to delete responses: '200': description: Chart deleted successfully content: application/json: schema: $ref: '#/components/schemas/DeleteChartResponse' /v1/metrics: get: tags: - Metrics operationId: getMetrics x-cli-name: list x-ts-sdk-name: list summary: List all metrics parameters: - name: type in: query required: false schema: type: string description: Filter by metric type - name: id in: query required: false schema: type: string description: Filter by specific metric ID responses: '200': description: Metrics retrieved successfully content: application/json: schema: $ref: '#/components/schemas/GetMetricsResponse' post: tags: - Metrics operationId: createMetric x-cli-name: create x-ts-sdk-name: create summary: Create a new metric requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/CreateMetricRequest' responses: '200': description: Metric created successfully content: application/json: schema: $ref: '#/components/schemas/CreateMetricResponse' put: tags: - Metrics operationId: updateMetricLegacy x-cli-exclude: true x-ts-sdk-exclude: true deprecated: true summary: Update an existing metric (deprecated) description: Deprecated. Use `PUT /v1/metrics/{metric_id}` instead. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/LegacyUpdateMetricRequest' responses: '200': description: Metric updated successfully content: application/json: schema: $ref: '#/components/schemas/UpdateMetricResponse' delete: tags: - Metrics operationId: deleteMetricLegacy x-cli-exclude: true x-ts-sdk-exclude: true deprecated: true summary: Delete a metric (deprecated) description: Deprecated. Use `DELETE /v1/metrics/{metric_id}` instead. parameters: - name: metric_id in: query required: true schema: type: string description: Unique identifier of the metric responses: '200': description: Metric deleted successfully content: application/json: schema: $ref: '#/components/schemas/DeleteMetricResponse' /v1/metrics/{metric_id}: put: tags: - Metrics operationId: updateMetric x-cli-name: update x-ts-sdk-name: update summary: Update an existing metric description: Update a metric's editable fields. Only fields included in the request body are modified. parameters: - in: path name: metric_id required: true schema: type: string description: The unique identifier of the metric to update requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/UpdateMetricRequest' responses: '200': description: Metric updated successfully content: application/json: schema: $ref: '#/components/schemas/UpdateMetricResponse' delete: tags: - Metrics operationId: deleteMetric x-cli-name: delete x-ts-sdk-name: delete summary: Delete a metric parameters: - in: path name: metric_id required: true schema: type: string description: The unique identifier of the metric to delete responses: '200': description: Metric deleted successfully content: application/json: schema: $ref: '#/components/schemas/DeleteMetricResponse' /v1/metrics/{metric_id}/versions: get: tags: - Metric Versions operationId: getMetricVersions x-cli-name: list x-ts-sdk-name: list summary: List versions for a metric description: Retrieve all snapshot versions of the metric's definition, ordered oldest-first. Returns the full version history unpaginated. parameters: - in: path name: metric_id required: true schema: type: string description: The unique identifier of the metric whose versions are being listed responses: '200': description: Metric versions retrieved successfully content: application/json: schema: $ref: '#/components/schemas/GetMetricVersionsResponse' '404': description: No metric with the given `metric_id` exists in the caller's scope. post: tags: - Metric Versions operationId: createMetricVersion x-cli-name: create x-ts-sdk-name: create summary: Create a new metric version description: 'Snapshot the supplied metric definition as a new version. By default the version is created as a draft (`deployed: false`); set `deploy_immediately: true` to also make it the live version in the same transaction.' parameters: - in: path name: metric_id required: true schema: type: string description: The unique identifier of the metric to version requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/CreateMetricVersionRequest' responses: '200': description: Metric version created successfully content: application/json: schema: $ref: '#/components/schemas/CreateMetricVersionResponse' '400': description: Invalid `content` payload. Fails cross-field validation (e.g. an `LLM` metric without `model_provider`/`model_name`). '404': description: No metric with the given `metric_id` exists in the caller's scope. /v1/metrics/{metric_id}/versions/{version_name}/deploy: post: tags: - Metric Versions operationId: deployMetricVersion x-cli-name: deploy x-ts-sdk-name: deploy summary: Deploy a specific metric version description: Mark the named version as the live version for the metric, unmarking any previously deployed version. parameters: - in: path name: metric_id required: true schema: type: string description: The unique identifier of the metric - in: path name: version_name required: true schema: type: string description: The name of the version to deploy responses: '200': description: Metric version deployed successfully content: application/json: schema: $ref: '#/components/schemas/DeployMetricVersionResponse' '404': description: No metric with the given `metric_id` exists in the caller's scope, or no version with the given `version_name` exists for that metric. /v1/metrics/run: post: tags: - Metrics operationId: runMetric x-cli-name: run x-ts-sdk-name: run summary: Run a metric evaluation description: Execute a metric on a specific event requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/RunMetricRequest' responses: '200': description: Metric evaluation succeeded content: application/json: schema: $ref: '#/components/schemas/RunMetricResponse' '400': description: Bad request. Either the request body failed validation (malformed metric or event), or the metric could not be evaluated because the request was deficient before any compute ran (the metric requires ground truth the event lacks, or the metric config is invalid). The error `message` describes the specific cause. '422': description: Could not process content. A compute request was dispatched but evaluation failed — e.g. the metric's code raised an error (`errorCode` `execution_error`, `compilation_error`) or the LLM evaluator's template/provider/response failed (`template_render_error`, `llm_response_parse_error`, ...). The error `message` carries the evaluator's detailed failure text. '500': description: Internal error while running the metric. /v1/metrics/run_metric: post: tags: - Metrics operationId: runMetricLegacy deprecated: true x-cli-exclude: true x-ts-sdk-exclude: true summary: Run a metric evaluation (deprecated) description: Deprecated. Use `POST /v1/metrics/run` instead. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/LegacyRunMetricRequest' responses: '200': description: Metric evaluation succeeded content: application/json: schema: $ref: '#/components/schemas/RunMetricResponse' '400': description: Bad request. Either the request body failed validation (malformed metric or event), or the metric could not be evaluated because the request was deficient before any compute ran (missing ground truth, invalid metric config). The error `message` describes the specific cause. '422': description: Could not process content. A compute request was dispatched but evaluation failed (e.g. `execution_error`, `template_render_error`, `llm_response_parse_error`). '500': description: Internal error while running the metric. /v1/datapoints: get: summary: Retrieve a list of datapoints operationId: getDatapoints x-cli-name: list x-ts-sdk-name: list tags: - Datapoints description: | Retrieve datapoints, optionally filtered by a list of datapoint IDs or dataset name. parameters: - name: datapoint_ids in: query required: false schema: type: array items: type: string description: List of datapoint ids to fetch - name: dataset_name in: query required: false schema: type: string description: Name of the dataset to get datapoints from responses: '200': description: Successful response content: application/json: schema: $ref: '#/components/schemas/GetDatapointsResponse' post: summary: Create a new datapoint operationId: createDatapoint x-cli-name: create x-ts-sdk-name: create tags: - Datapoints description: Create a single datapoint with inputs, history, ground truth, and metadata. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/CreateDatapointRequest' responses: '200': description: Datapoint successfully created content: application/json: schema: $ref: '#/components/schemas/CreateDatapointResponse' /v1/datapoints/batch: post: summary: Create multiple datapoints in batch operationId: batchCreateDatapoints x-cli-name: create-batch x-ts-sdk-name: createBatch tags: - Datapoints description: Create multiple datapoints from events using field mappings and optional filters. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/BatchCreateDatapointsRequest' responses: '200': description: Datapoints successfully created in batch content: application/json: schema: $ref: '#/components/schemas/BatchCreateDatapointsResponse' /v1/datapoints/{datapoint_id}: get: summary: Retrieve a specific datapoint operationId: getDatapoint x-cli-name: get x-ts-sdk-name: get tags: - Datapoints description: Get a single datapoint by its unique identifier. parameters: - name: datapoint_id in: path required: true schema: type: string description: Datapoint ID like `65c13dbbd65fb876b7886cdb` responses: '200': content: application/json: schema: $ref: '#/components/schemas/GetDatapointResponse' description: Successful response put: summary: Update a specific datapoint description: Update fields on an existing datapoint. Only the provided fields are modified. parameters: - name: datapoint_id in: path required: true schema: type: string description: ID of datapoint to update operationId: updateDatapoint x-cli-name: update x-ts-sdk-name: update tags: - Datapoints requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/UpdateDatapointRequest' responses: '200': description: Datapoint successfully updated content: application/json: schema: $ref: '#/components/schemas/UpdateDatapointResponse' '400': description: Error updating datapoint delete: summary: Delete a specific datapoint operationId: deleteDatapoint x-cli-name: delete x-ts-sdk-name: delete tags: - Datapoints description: Permanently delete a datapoint by its unique identifier. parameters: - name: datapoint_id in: path required: true schema: type: string description: Datapoint ID like `65c13dbbd65fb876b7886cdb` responses: '200': description: Datapoint successfully deleted content: application/json: schema: $ref: '#/components/schemas/DeleteDatapointResponse' /v1/datasets: get: tags: - Datasets summary: Get datasets operationId: getDatasets x-cli-name: list x-ts-sdk-name: list description: Retrieve datasets, optionally filtered by dataset ID or name. parameters: - in: query name: dataset_id required: false schema: type: string description: Unique dataset ID for filtering specific dataset - in: query name: name required: false schema: type: string description: Dataset name to filter by responses: '200': description: Successful response content: application/json: schema: $ref: '#/components/schemas/GetDatasetsResponse' post: tags: - Datasets operationId: createDataset x-cli-name: create x-ts-sdk-name: create summary: Create a dataset description: Create a new dataset with an optional name, description, and initial set of datapoint IDs. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/CreateDatasetRequest' responses: '200': description: Successful creation content: application/json: schema: $ref: '#/components/schemas/CreateDatasetResponse' put: tags: - Datasets operationId: updateDatasetLegacy x-cli-exclude: true x-ts-sdk-exclude: true deprecated: true summary: Update a dataset (deprecated) description: Deprecated. Use `PUT /v1/datasets/{dataset_id}` instead. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/LegacyUpdateDatasetRequest' responses: '200': description: Successful update content: application/json: schema: $ref: '#/components/schemas/UpdateDatasetResponse' delete: tags: - Datasets operationId: deleteDatasetLegacy x-cli-exclude: true x-ts-sdk-exclude: true deprecated: true summary: Delete a dataset (deprecated) description: Deprecated. Use `DELETE /v1/datasets/{dataset_id}` instead. parameters: - in: query name: dataset_id required: true schema: type: string description: The unique identifier of the dataset to be deleted like `663876ec4611c47f4970f0c3` responses: '200': description: Successful delete content: application/json: schema: $ref: '#/components/schemas/DeleteDatasetResponse' /v1/datasets/{dataset_id}: put: tags: - Datasets operationId: updateDataset x-cli-name: update x-ts-sdk-name: update summary: Update a dataset description: Update a dataset's name, description, or list of datapoint IDs. parameters: - in: path name: dataset_id required: true schema: type: string description: The unique identifier of the dataset to update like `663876ec4611c47f4970f0c3` requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/UpdateDatasetRequest' responses: '200': description: Successful update content: application/json: schema: $ref: '#/components/schemas/UpdateDatasetResponse' delete: tags: - Datasets operationId: deleteDataset x-cli-name: delete x-ts-sdk-name: delete summary: Delete a dataset description: Permanently delete a dataset by its unique identifier. parameters: - in: path name: dataset_id required: true schema: type: string description: The unique identifier of the dataset to be deleted like `663876ec4611c47f4970f0c3` responses: '200': description: Successful delete content: application/json: schema: $ref: '#/components/schemas/DeleteDatasetResponse' /v1/datasets/{dataset_id}/datapoints: post: tags: - Datasets summary: Add datapoints to a dataset operationId: addDatapoints x-cli-name: add-datapoints x-ts-sdk-name: addDatapoints description: | Add new datapoints to an existing dataset. Provide raw data objects and a field mapping that specifies which fields map to inputs, ground truth, and history. parameters: - in: path name: dataset_id required: true schema: type: string description: The unique identifier of the dataset to add datapoints to like `663876ec4611c47f4970f0c3` requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/AddDatapointsToDatasetRequest' responses: '200': description: Successful addition content: application/json: schema: $ref: '#/components/schemas/AddDatapointsResponse' /v1/datasets/{dataset_id}/datapoints/{datapoint_id}: delete: tags: - Datasets summary: Remove a datapoint from a dataset operationId: removeDatapoint x-cli-name: remove-datapoint x-ts-sdk-name: removeDatapoint description: Remove a specific datapoint from a dataset. The datapoint itself is not deleted, only dereferenced from the dataset. parameters: - in: path name: dataset_id required: true schema: type: string description: The unique identifier of the dataset - in: path name: datapoint_id required: true schema: type: string description: The unique identifier of the datapoint to remove responses: '200': description: Datapoint successfully removed from dataset content: application/json: schema: $ref: '#/components/schemas/RemoveDatapointResponse' /v1/datasets/{dataset_id}/{datapoint_id}: delete: tags: - Datasets summary: Remove a datapoint from a dataset (deprecated) operationId: removeDatapointLegacy x-cli-exclude: true x-ts-sdk-exclude: true deprecated: true description: Deprecated. Use `DELETE /v1/datasets/{dataset_id}/datapoints/{datapoint_id}` instead. parameters: - in: path name: dataset_id required: true schema: type: string description: The unique identifier of the dataset - in: path name: datapoint_id required: true schema: type: string description: The unique identifier of the datapoint to remove responses: '200': description: Datapoint successfully removed from dataset content: application/json: schema: $ref: '#/components/schemas/RemoveDatapointResponse' /v1/events/schema: get: summary: Get events schema (deprecated) operationId: getEventsSchemaLegacy deprecated: true x-cli-exclude: true x-ts-sdk-exclude: true tags: - Events description: Deprecated. Use `GET /v1/runs/{run_id}/schema` (single-run) or `GET /v1/runs/schema` (project-wide) instead. parameters: - in: query name: dateRange required: false schema: oneOf: - type: string - $ref: '#/components/schemas/GetEventsSchemaLegacyDateRangeOneOf1' description: Filter by date range - in: query name: evaluation_id required: false schema: type: string description: Filter by evaluation/run ID responses: '200': description: Events schema retrieved successfully content: application/json: schema: $ref: '#/components/schemas/GetEventsSchemaResponse' /v1/runs: post: summary: Create a new evaluation run operationId: createRun x-cli-name: create-run x-ts-sdk-name: createRun tags: - Experiments description: Create a new experiment run to track an evaluation against a dataset. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/PostExperimentRunRequest' responses: '200': description: Successful response content: application/json: schema: $ref: '#/components/schemas/PostExperimentRunResponse' '400': description: Invalid input get: summary: Get a list of evaluation runs operationId: getRuns x-cli-name: list-runs x-ts-sdk-name: listRuns tags: - Experiments description: | List experiment runs with optional filtering by dataset, status, name, date range, and specific run IDs. Results are paginated and sortable. parameters: - in: query name: dataset_id required: false schema: type: string description: Filter by dataset ID - in: query name: page required: false schema: type: integer minimum: 1 default: 1 description: Page number for pagination - in: query name: limit required: false schema: type: integer minimum: 1 maximum: 100 default: 20 description: Number of results per page - in: query name: run_ids required: false schema: type: array items: type: string description: List of specific run IDs to fetch - in: query name: name required: false schema: type: string description: Filter by run name - in: query name: status required: false schema: type: string enum: - pending - completed - failed - cancelled - running description: Filter by run status - in: query name: dateRange required: false schema: oneOf: - type: string - $ref: '#/components/schemas/GetRunsDateRangeOneOf1' description: Filter by date range - in: query name: sort_by required: false schema: type: string enum: - created_at - updated_at - name - status default: created_at description: Field to sort by - in: query name: sort_order required: false schema: type: string enum: - asc - desc default: desc description: Sort order responses: '200': description: Successful response content: application/json: schema: $ref: '#/components/schemas/GetExperimentRunsResponse' '400': description: Error fetching evaluations /v1/runs/schema: get: summary: Get events schema across all experiment runs in a project operationId: getRunsSchema x-cli-name: get-runs-schema x-ts-sdk-name: getRunsSchema tags: - Experiments description: Retrieve the aggregated events schema (fields, datasets, mappings) across all experiment runs in the project. parameters: - in: query name: dateRange required: false schema: oneOf: - type: string - $ref: '#/components/schemas/GetRunsSchemaDateRangeOneOf1' description: Filter by date range responses: '200': description: Events schema retrieved successfully content: application/json: schema: $ref: '#/components/schemas/GetEventsSchemaResponse' '400': description: Error fetching events schema /v1/runs/{run_id}: get: summary: Get details of an evaluation run operationId: getRun x-cli-name: get-run x-ts-sdk-name: getRun tags: - Experiments description: Retrieve the full details of a single experiment run by its run ID. parameters: - in: path name: run_id required: true schema: type: string format: uuid responses: '200': description: Successful response content: application/json: schema: $ref: '#/components/schemas/GetExperimentRunResponse' '400': description: Error fetching evaluation put: summary: Update an evaluation run operationId: updateRun x-cli-name: update-run x-ts-sdk-name: updateRun tags: - Experiments description: Update fields on an existing experiment run such as name, status, metadata, or results. parameters: - in: path name: run_id required: true schema: type: string format: uuid requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/PutExperimentRunRequest' responses: '200': description: Successful response content: application/json: schema: $ref: '#/components/schemas/PutExperimentRunResponse' '400': description: Invalid input delete: summary: Delete an evaluation run operationId: deleteRun x-cli-name: delete-run x-ts-sdk-name: deleteRun tags: - Experiments description: Permanently delete an experiment run by its run ID. parameters: - in: path name: run_id required: true schema: type: string format: uuid responses: '200': description: Successful response content: application/json: schema: $ref: '#/components/schemas/DeleteExperimentRunResponse' '400': description: Error deleting evaluation /v1/runs/{run_id}/schema: get: summary: Get events schema for a single experiment run operationId: getRunSchema x-cli-name: get-run-schema x-ts-sdk-name: getRunSchema tags: - Experiments description: Retrieve the events schema (fields, datasets, mappings) for a single experiment run. parameters: - name: run_id in: path required: true schema: type: string format: uuid description: Experiment run ID (UUIDv4) - in: query name: dateRange required: false schema: oneOf: - type: string - $ref: '#/components/schemas/GetRunSchemaDateRangeOneOf1' description: Filter by date range responses: '200': description: Events schema retrieved successfully content: application/json: schema: $ref: '#/components/schemas/GetEventsSchemaResponse' '400': description: Error fetching events schema /v1/runs/{run_id}/metrics: get: summary: Get event metrics for an experiment run operationId: getExperimentRunMetrics x-cli-name: get-run-metrics x-ts-sdk-name: getRunMetrics tags: - Experiments description: Retrieve event metrics from ClickHouse for a specific experiment run parameters: - name: run_id in: path required: true schema: type: string format: uuid description: Experiment run ID (UUIDv4) - name: dateRange in: query required: false schema: type: string description: Date range filter as JSON string - name: filters in: query required: false schema: oneOf: - type: string - type: array items: type: object description: Optional filters to apply (JSON string or array of filter objects) responses: '200': description: Event metrics retrieved successfully content: application/json: schema: $ref: '#/components/schemas/GetExperimentRunMetricsResponse' '400': description: Error fetching event metrics /v1/runs/{run_id}/summary: get: summary: Retrieve experiment summary operationId: getExperimentSummary x-cli-name: get-summary x-ts-sdk-name: getSummary tags: - Experiments description: 'Compute evaluation summary for an experiment run: pass/fail results, metric aggregations, per-datapoint results, event details, and the experiment run object.' parameters: - name: run_id in: path required: true schema: type: string format: uuid description: Experiment run ID (UUIDv4) - name: aggregate_function in: query required: false schema: type: string enum: - average - min - max - median - p95 - p99 - p90 - sum - count default: average description: Aggregation function to apply to metrics - name: filters in: query required: false schema: oneOf: - type: string - type: array items: type: object description: Optional filters to apply (JSON string or array of filter objects) responses: '200': description: Experiment summary retrieved successfully content: application/json: schema: $ref: '#/components/schemas/GetExperimentRunResultResponse' '400': description: Error processing experiment summary /v1/runs/{run_id}/result: get: summary: Retrieve experiment result (deprecated) operationId: getExperimentResultLegacy deprecated: true x-cli-exclude: true x-ts-sdk-exclude: true tags: - Experiments description: Deprecated. Use `GET /v1/runs/{run_id}/summary` instead. parameters: - name: run_id in: path required: true schema: type: string format: uuid description: Experiment run ID (UUIDv4) - name: aggregate_function in: query required: false schema: type: string enum: - average - min - max - median - p95 - p99 - p90 - sum - count default: average description: Aggregation function to apply to metrics - name: filters in: query required: false schema: oneOf: - type: string - type: array items: type: object description: Optional filters to apply (JSON string or array of filter objects) responses: '200': description: Experiment result retrieved successfully content: application/json: schema: $ref: '#/components/schemas/GetExperimentRunResultResponse' '400': description: Error processing experiment result /v1/runs/{new_run_id}/compare/{old_run_id}: get: summary: Retrieve experiment comparison operationId: getExperimentComparison x-cli-name: compare-runs x-ts-sdk-name: compareRuns tags: - Experiments description: Compare metrics and results between two experiment runs parameters: - name: new_run_id in: path required: true schema: type: string format: uuid description: New experiment run ID to compare (UUIDv4) - name: old_run_id in: path required: true schema: type: string format: uuid description: Old experiment run ID to compare against (UUIDv4) - name: aggregate_function in: query required: false schema: type: string enum: - average - min - max - median - p95 - p99 - p90 - sum - count default: average description: Aggregation function to apply to metrics - name: filters in: query required: false schema: oneOf: - type: string - type: array items: type: object description: Optional filters to apply (JSON string or array of filter objects) responses: '200': description: Experiment comparison retrieved successfully content: application/json: schema: $ref: '#/components/schemas/GetExperimentRunCompareResponse' '400': description: Error processing experiment comparison /v1/runs/{new_run_id}/compare-with/{old_run_id}: get: summary: Retrieve experiment comparison (deprecated) operationId: getExperimentComparisonLegacy deprecated: true x-cli-exclude: true x-ts-sdk-exclude: true tags: - Experiments description: Deprecated. Use `GET /v1/runs/{new_run_id}/compare/{old_run_id}` instead. parameters: - name: new_run_id in: path required: true schema: type: string format: uuid description: New experiment run ID to compare (UUIDv4) - name: old_run_id in: path required: true schema: type: string format: uuid description: Old experiment run ID to compare against (UUIDv4) - name: aggregate_function in: query required: false schema: type: string description: Aggregation function to apply to metrics - name: filters in: query required: false schema: oneOf: - type: string - type: array items: type: object description: Optional filters to apply (JSON string or array of filter objects) responses: '200': description: Experiment comparison retrieved successfully content: application/json: schema: $ref: '#/components/schemas/GetExperimentRunCompareResponse' '400': description: Error processing experiment comparison /v1/runs/{new_run_id}/compare/{old_run_id}/events: get: summary: Compare events between two experiment runs operationId: getExperimentCompareEvents x-cli-name: compare-run-events x-ts-sdk-name: compareRunEvents tags: - Experiments description: Retrieve and compare events between two experiment runs for detailed analysis parameters: - name: new_run_id in: path required: true schema: type: string format: uuid description: New experiment run ID (UUIDv4) - name: old_run_id in: path required: true schema: type: string format: uuid description: Old experiment run ID to compare against (UUIDv4) - name: event_name in: query required: false schema: type: string description: Filter by event name - name: event_type in: query required: false schema: type: string description: Filter by event type - name: filter in: query required: false schema: oneOf: - type: string - type: object description: Additional filter criteria (JSON string or object) - name: limit in: query required: false schema: type: integer minimum: 1 maximum: 1000 default: 1000 description: Maximum number of results - name: page in: query required: false schema: type: integer minimum: 1 default: 1 description: Page number for pagination responses: '200': description: Event comparison retrieved successfully content: application/json: schema: $ref: '#/components/schemas/GetExperimentCompareEventsResponse' '400': description: Error fetching event comparison /v1/runs/compare/events: get: summary: Compare events between two experiment runs (deprecated) operationId: getExperimentCompareEventsLegacy deprecated: true x-cli-exclude: true x-ts-sdk-exclude: true tags: - Experiments description: Deprecated. Use `GET /v1/runs/{new_run_id}/compare/{old_run_id}/events` instead. parameters: - name: run_id_1 in: query required: true schema: type: string format: uuid description: First experiment run ID (UUIDv4) - name: run_id_2 in: query required: true schema: type: string format: uuid description: Second experiment run ID (UUIDv4) - name: event_name in: query required: false schema: type: string description: Filter by event name - name: event_type in: query required: false schema: type: string description: Filter by event type - name: filter in: query required: false schema: oneOf: - type: string - type: object description: Additional filter criteria (JSON string or object) - name: limit in: query required: false schema: type: integer minimum: 1 maximum: 1000 default: 1000 description: Maximum number of results - name: page in: query required: false schema: type: integer minimum: 1 default: 1 description: Page number for pagination responses: '200': description: Event comparison retrieved successfully content: application/json: schema: $ref: '#/components/schemas/GetExperimentCompareEventsResponse' '400': description: Error fetching event comparison /v1/queues: get: tags: - Queues operationId: getQueues x-ts-sdk-exclude: true x-cli-exclude: true summary: List annotation queues description: | List annotation queues for the current project scope, optionally filtered by enabled status. parameters: - name: enabled in: query required: false schema: type: boolean description: Filter by enabled status responses: '200': description: Queues retrieved successfully content: application/json: schema: $ref: '#/components/schemas/GetAnnotationQueuesResponse' post: tags: - Queues operationId: createQueue x-ts-sdk-exclude: true x-cli-exclude: true summary: Create an annotation queue description: | Create a new annotation queue with a name, optional description, filters, and an initial set of event IDs to add. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/CreateAnnotationQueueRequest' responses: '201': description: Queue created successfully content: application/json: schema: $ref: '#/components/schemas/CreateAnnotationQueueResponse' '400': description: Bad request (invalid queue data or missing required fields) /v1/queues/{queue_id}: get: tags: - Queues operationId: getQueue x-ts-sdk-exclude: true x-cli-exclude: true summary: Get an annotation queue description: Retrieve a single annotation queue by its unique identifier. parameters: - name: queue_id in: path required: true schema: type: string description: Annotation queue ID responses: '200': description: Queue retrieved successfully content: application/json: schema: $ref: '#/components/schemas/GetAnnotationQueueByIdResponse' '404': description: Queue not found put: tags: - Queues operationId: updateQueue x-ts-sdk-exclude: true x-cli-exclude: true summary: Update an annotation queue description: | Update fields on an existing annotation queue. Supports updating name, description, filters, enabled status, and adding/removing events. parameters: - name: queue_id in: path required: true schema: type: string description: Annotation queue ID requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/UpdateAnnotationQueueRequest' responses: '200': description: Queue updated successfully content: application/json: schema: $ref: '#/components/schemas/UpdateAnnotationQueueResponse' '404': description: Queue not found delete: tags: - Queues operationId: deleteQueue x-ts-sdk-exclude: true x-cli-exclude: true summary: Delete an annotation queue description: Soft-delete an annotation queue by its unique identifier. parameters: - name: queue_id in: path required: true schema: type: string description: Annotation queue ID responses: '200': description: Queue deleted successfully content: application/json: schema: $ref: '#/components/schemas/DeleteAnnotationQueueResponse' '404': description: Queue not found /v1/configurations: get: summary: Retrieve a list of configurations operationId: getConfigurations x-cli-exclude: true x-ts-sdk-exclude: true deprecated: true tags: - Configurations description: List configurations with optional filtering by name, environment, and tags. parameters: - name: name in: query required: false schema: type: string description: The name of the configuration like `v0` - name: env in: query required: false schema: type: string description: Environment - "dev", "staging" or "prod" - name: tags in: query required: false schema: type: string description: Tags to filter configurations responses: '200': description: Configurations retrieved successfully content: application/json: schema: $ref: '#/components/schemas/GetConfigurationsResponse' post: summary: Create a new configuration operationId: createConfiguration x-cli-exclude: true x-ts-sdk-exclude: true deprecated: true tags: - Configurations description: Create a new LLM or pipeline configuration with provider, parameters, and environment settings. requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/CreateConfigurationRequest' responses: '200': description: Configuration created successfully content: application/json: schema: $ref: '#/components/schemas/CreateConfigurationResponse' /v1/configurations/{configId}: put: summary: Update an existing configuration operationId: updateConfiguration x-cli-exclude: true x-ts-sdk-exclude: true deprecated: true tags: - Configurations description: Update an existing configuration's name, provider, parameters, environment, or tags. parameters: - name: configId in: path required: true schema: type: string description: Configuration ID like `6638187d505c6812e4043f24` requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/UpdateConfigurationRequest' responses: '200': description: Configuration updated successfully content: application/json: schema: $ref: '#/components/schemas/UpdateConfigurationResponse' delete: summary: Delete a configuration operationId: deleteConfiguration x-cli-exclude: true x-ts-sdk-exclude: true deprecated: true tags: - Configurations description: Permanently delete a configuration by its unique identifier. parameters: - name: configId in: path required: true schema: type: string description: Configuration ID like `6638187d505c6812e4043f24` responses: '200': description: Configuration deleted successfully content: application/json: schema: $ref: '#/components/schemas/DeleteConfigurationResponse' components: securitySchemes: BearerAuth: type: http scheme: bearer schemas: SingleFilter: type: object properties: field: type: string operator: type: string enum: - exists - not exists - is - is not - contains - not contains - greater than - less than - after - before value: anyOf: - type: string - type: number - type: boolean - type: 'null' type: type: string enum: - string - number - boolean - datetime required: - field - operator - value - type Pagination: type: object properties: page: type: integer minimum: 1 limit: type: integer minimum: 1 total: type: integer minimum: 0 total_unfiltered: type: integer minimum: 0 total_pages: type: integer minimum: 0 has_next: type: boolean has_prev: type: boolean required: - page - limit - total - total_unfiltered - total_pages - has_next - has_prev CreateChartRequest: type: object properties: name: type: string maxLength: 200 description: Display name for the chart description: type: string description: Description of what the chart shows metric: type: string minLength: 1 description: Name of the metric to visualize func: type: string description: Aggregation function to apply (e.g. sum, avg, median, min, max) groupBy: type: - string - 'null' description: Field to group results by bucketing: type: string enum: - minute - hour - day - week - month default: day description: Time bucket granularity for aggregation dateRange: anyOf: - $ref: '#/components/schemas/RelativeDateRange' - $ref: '#/components/schemas/AbsoluteDateRange' description: Time range to query query: type: array items: $ref: '#/components/schemas/QueryFilter' description: Filters to apply to the chart data owner_id: type: string minLength: 1 description: ID of the user who owns this chart required: - name - metric additionalProperties: false RelativeDateRange: type: object properties: relative: type: string required: - relative additionalProperties: false AbsoluteDateRange: type: object properties: $gte: anyOf: - type: string - type: number $lte: anyOf: - type: string - type: number required: - $gte - $lte QueryFilter: type: object properties: field: type: string description: Name of the field to filter on value: type: - string - 'null' description: Value to compare against type: type: string description: Data type of the field (e.g. string, number) operator: type: string description: Comparison operator (e.g. is, is not, contains, greater than, less than) required: - field - value - type - operator additionalProperties: false CreateChartResponse: type: object properties: success: type: boolean data: $ref: '#/components/schemas/CreateChartResponseData' required: - success - data additionalProperties: false DeleteChartResponse: type: object properties: success: type: boolean message: type: string required: - success - message additionalProperties: false GetChartResponse: type: object properties: success: type: boolean data: $ref: '#/components/schemas/GetChartResponseData' required: - success - data additionalProperties: false GetChartsResponse: type: object properties: success: type: boolean data: type: array items: $ref: '#/components/schemas/GetChartsResponseDataItem' required: - success - data additionalProperties: false UpdateChartRequest: type: object properties: name: type: string maxLength: 200 description: Display name for the chart description: type: string description: Description of what the chart shows metric: type: string description: Name of the metric to visualize func: type: string description: Aggregation function to apply (e.g. sum, avg, median, min, max) groupBy: type: - string - 'null' description: Field to group results by bucketing: type: string enum: - minute - hour - day - week - month description: Time bucket granularity for aggregation dateRange: anyOf: - $ref: '#/components/schemas/RelativeDateRange' - $ref: '#/components/schemas/AbsoluteDateRange' description: Time range to query query: type: array items: $ref: '#/components/schemas/QueryFilter' description: Filters to apply to the chart data owner_id: type: string minLength: 1 description: ID of the user who owns this chart additionalProperties: false UpdateChartResponse: type: object properties: success: type: boolean data: $ref: '#/components/schemas/UpdateChartResponseData' required: - success - data additionalProperties: false SelectedFunction: type: object properties: id: type: string minLength: 1 name: type: string minLength: 1 description: type: string parameters: type: object additionalProperties: {} required: - id - name ResponseFormat: type: object properties: type: type: string enum: - text - json_object required: - type TemplateItem: type: object properties: role: type: string content: type: string required: - role - content ConfigurationParameters: type: object properties: call_type: type: string enum: - chat - completion model: type: string minLength: 1 hyperparameters: type: object additionalProperties: {} responseFormat: $ref: '#/components/schemas/ResponseFormat' selectedFunctions: type: array items: $ref: '#/components/schemas/SelectedFunction' functionCallParams: type: string enum: - none - auto - force forceFunction: type: object additionalProperties: {} template: anyOf: - type: array items: $ref: '#/components/schemas/TemplateItem' - type: string required: - call_type - model ConfigurationItem: type: object properties: id: type: string minLength: 1 name: type: string type: type: string enum: - LLM - pipeline provider: type: string parameters: $ref: '#/components/schemas/ConfigurationParameters' env: type: array items: type: string enum: - dev - staging - prod tags: type: array items: type: string user_properties: type: - object - 'null' additionalProperties: {} created_at: type: string format: date-time updated_at: type: - string - 'null' format: date-time required: - id - name - type - provider - parameters - env - tags - created_at CreateConfigurationRequest: type: object properties: name: type: string maxLength: 200 type: type: string enum: - LLM - pipeline default: LLM provider: type: string minLength: 1 parameters: $ref: '#/components/schemas/ConfigurationParameters' env: type: array items: type: string enum: - dev - staging - prod tags: type: array items: type: string user_properties: type: - object - 'null' additionalProperties: {} required: - name - provider - parameters additionalProperties: false description: Request body for POST /configurations UpdateConfigurationRequest: type: object properties: name: type: string maxLength: 200 type: type: string enum: - LLM - pipeline default: LLM provider: type: string minLength: 1 parameters: $ref: '#/components/schemas/ConfigurationParameters' env: type: array items: type: string enum: - dev - staging - prod tags: type: array items: type: string user_properties: type: - object - 'null' additionalProperties: {} required: - name additionalProperties: false description: Request body for PUT /configurations GetConfigurationsQuery: type: object properties: name: type: string maxLength: 200 description: Filter configurations by name env: type: string description: Filter configurations by environment tags: type: string description: Filter configurations by tags description: Query parameters for GET /configurations CreateConfigurationResponse: type: object properties: acknowledged: type: boolean insertedId: type: string minLength: 1 required: - acknowledged - insertedId description: Response for POST /configurations UpdateConfigurationResponse: type: object properties: acknowledged: type: boolean modifiedCount: type: number upsertedId: type: - string - 'null' upsertedCount: type: number matchedCount: type: number required: - acknowledged - modifiedCount - upsertedId - upsertedCount - matchedCount description: Response for PUT /configurations DeleteConfigurationResponse: type: object properties: acknowledged: type: boolean deletedCount: type: number required: - acknowledged - deletedCount description: Response for DELETE /configurations GetConfigurationsResponse: type: object properties: configurations: type: array items: $ref: '#/components/schemas/ConfigurationItem' required: - configurations description: Response for GET /configurations DatapointMapping: type: object properties: inputs: type: array items: type: string default: [] history: type: array items: type: string default: [] ground_truth: type: array items: type: string default: [] BatchDateRange: type: object properties: $gte: type: string $lte: type: string CheckState: type: object additionalProperties: type: boolean Datapoint: type: object properties: id: type: string minLength: 1 inputs: type: object additionalProperties: {} history: type: array items: type: object additionalProperties: {} ground_truth: type: - object - 'null' additionalProperties: {} metadata: type: - object - 'null' additionalProperties: {} linked_event: anyOf: - type: string - type: 'null' created_at: type: string updated_at: type: string linked_datasets: type: array items: type: string required: - id - inputs - history - ground_truth - metadata - linked_event - created_at GetDatapointsQuery: type: object properties: datapoint_ids: type: array items: type: string minLength: 1 description: List of datapoint IDs to fetch dataset_name: type: string maxLength: 200 description: Dataset name to filter by additionalProperties: false description: Query parameters for GET /datapoints GetDatapointParams: type: object properties: datapoint_id: type: string minLength: 1 description: Unique identifier of the datapoint required: - datapoint_id description: Path parameters for GET /datapoints/{datapoint_id} CreateDatapointRequest: type: object properties: inputs: type: object additionalProperties: {} default: {} history: type: array items: type: object additionalProperties: {} default: [] ground_truth: type: object additionalProperties: {} metadata: type: object additionalProperties: {} linked_event: type: string linked_datasets: type: array items: type: string minLength: 1 default: [] description: Request body for POST /datapoints UpdateDatapointRequest: type: object properties: inputs: type: object additionalProperties: {} history: type: array items: type: object additionalProperties: {} ground_truth: type: object additionalProperties: {} metadata: type: object additionalProperties: {} linked_event: type: string linked_datasets: type: array items: type: string minLength: 1 description: Request body for PUT /datapoints/{datapoint_id} UpdateDatapointParams: type: object properties: datapoint_id: type: string minLength: 1 required: - datapoint_id description: Path parameters for PUT /datapoints/{datapoint_id} DeleteDatapointParams: type: object properties: datapoint_id: type: string minLength: 1 required: - datapoint_id description: Path parameters for DELETE /datapoints/{datapoint_id} BatchCreateDatapointsRequest: type: object properties: events: type: array items: type: string minLength: 1 deprecated: true mapping: $ref: '#/components/schemas/DatapointMapping' deprecated: true filters: anyOf: - type: object additionalProperties: {} - type: array items: type: object additionalProperties: {} dateRange: $ref: '#/components/schemas/BatchDateRange' checkState: $ref: '#/components/schemas/CheckState' selectAll: type: boolean dataset_id: type: string minLength: 1 description: Request body for POST /datapoints/batch GetDatapointsResponse: type: object properties: datapoints: type: array items: $ref: '#/components/schemas/Datapoint' required: - datapoints description: Response for GET /datapoints GetDatapointResponse: type: object properties: datapoint: type: array items: $ref: '#/components/schemas/Datapoint' required: - datapoint description: Response for GET /datapoints/{datapoint_id} CreateDatapointResponse: type: object properties: inserted: type: boolean result: $ref: '#/components/schemas/CreateDatapointResponseResult' required: - inserted - result description: Response for POST /datapoints UpdateDatapointResponse: type: object properties: updated: type: boolean result: $ref: '#/components/schemas/UpdateDatapointResponseResult' required: - updated - result description: Response for PUT /datapoints/{datapoint_id} DeleteDatapointResponse: type: object properties: deleted: type: boolean required: - deleted description: Response for DELETE /datapoints/{datapoint_id} BatchCreateDatapointsResponse: type: object properties: inserted: type: boolean insertedIds: type: array items: type: string minLength: 1 required: - inserted - insertedIds description: Response for POST /datapoints/batch Dataset: type: object properties: id: type: string minLength: 1 name: type: string maxLength: 200 description: type: - string - 'null' datapoints: type: array items: type: string minLength: 1 created_at: type: string updated_at: type: string required: - id - name - datapoints CreateDatasetRequest: type: object properties: name: type: string maxLength: 200 default: Untitled Dataset description: Name of the dataset description: type: string description: Description of the dataset datapoints: type: array items: type: string minLength: 1 default: [] description: Initial datapoint IDs to include description: Request body for POST /datasets UpdateDatasetRequest: type: object properties: name: type: string maxLength: 200 description: New dataset name description: type: string description: New dataset description datapoints: type: array items: type: string minLength: 1 description: Updated list of datapoint IDs additionalProperties: false description: Request body for PUT /datasets/{dataset_id} LegacyUpdateDatasetRequest: type: object properties: dataset_id: type: string minLength: 1 description: Unique identifier of the dataset to update name: type: string maxLength: 200 description: New dataset name description: type: string description: New dataset description datapoints: type: array items: type: string minLength: 1 description: Updated list of datapoint IDs required: - dataset_id description: Request body for PUT /datasets (deprecated — use PUT /datasets/{dataset_id}) deprecated: true DeleteDatasetParams: type: object properties: dataset_id: type: string minLength: 1 description: Unique identifier of the dataset to delete required: - dataset_id additionalProperties: false description: Path parameters for DELETE /datasets/{dataset_id} LegacyDeleteDatasetQuery: type: object properties: dataset_id: type: string minLength: 1 description: Unique identifier of the dataset to delete required: - dataset_id description: Query parameters for DELETE /datasets (deprecated — use DELETE /datasets/{dataset_id}) deprecated: true AddDatapointsToDatasetRequest: type: object properties: data: type: array items: type: object additionalProperties: {} minItems: 1 description: Array of datapoint data objects to add mapping: $ref: '#/components/schemas/DatapointMapping' description: Field mapping for inputs, ground truth, and history required: - data - mapping description: Request body for POST /datasets/{dataset_id}/datapoints LegacyRemoveDatapointFromDatasetParams: type: object properties: dataset_id: type: string minLength: 1 description: Unique identifier of the dataset datapoint_id: type: string minLength: 1 description: Unique identifier of the datapoint to remove required: - dataset_id - datapoint_id description: Path parameters for DELETE /datasets/{dataset_id}/{datapoint_id} (deprecated — use DELETE /datasets/{dataset_id}/datapoints/{datapoint_id}) deprecated: true CreateDatasetResponse: type: object properties: inserted: type: boolean result: $ref: '#/components/schemas/InsertResult' required: - inserted - result description: Response for POST /datasets InsertResult: type: object properties: insertedId: type: string minLength: 1 required: - insertedId UpdateDatasetResponse: type: object properties: result: $ref: '#/components/schemas/Dataset' required: - result description: Response for PUT /datasets GetDatasetsResponse: type: object properties: datasets: type: array items: $ref: '#/components/schemas/Dataset' required: - datasets description: Response for GET /datasets DeleteDatasetResponse: type: object properties: result: $ref: '#/components/schemas/DeleteResult' required: - result description: Response for DELETE /datasets DeleteResult: type: object properties: id: type: string minLength: 1 required: - id AddDatapointsResponse: type: object properties: inserted: type: boolean datapoint_ids: type: array items: type: string minLength: 1 required: - inserted - datapoint_ids description: Response for POST /datasets/{dataset_id}/datapoints RemoveDatapointResponse: type: object properties: dereferenced: type: boolean message: type: string required: - dereferenced - message description: Response for DELETE /datasets/{dataset_id}/datapoints/{datapoint_id} FiltersArray: type: array items: $ref: '#/components/schemas/SingleFilter' EventSearchFilter: type: object properties: field: type: string operator: type: string enum: - exists - not exists - is - is not - contains - not contains - greater than - less than - after - before value: anyOf: - type: string - type: number - type: boolean - type: 'null' type: type: string enum: - string - number - boolean - datetime required: - field - operator - value additionalProperties: false EventSearchFiltersArray: type: array items: $ref: '#/components/schemas/EventSearchFilter' Event: type: object properties: event_id: type: string project_id: type: string tenant: type: string event_name: type: string event_type: type: string metrics: type: object additionalProperties: {} metadata: type: object additionalProperties: {} feedback: $ref: '#/components/schemas/EventFeedback' required: - event_id - project_id - tenant additionalProperties: {} description: Minimal event object used by evaluation and session endpoints; permissive (passthrough) LegacyEvent: type: object properties: project: type: string deprecated: true description: Project name (ignored by server — project is determined from API key scope) project_id: type: string description: Project ID source: type: string description: Source of the event (e.g., sdk-python) event_name: type: string description: Name of the event event_type: type: string enum: - model - tool - chain - session description: Type of event (model, tool, chain, or session) event_id: type: string description: Unique event identifier session_id: type: string description: Session this event belongs to parent_id: type: string description: Parent event ID in the trace hierarchy children_ids: type: array items: type: string description: Child event IDs in the trace hierarchy config: type: object additionalProperties: {} description: Configuration used for this event inputs: type: object additionalProperties: {} description: Input data for the event outputs: type: object additionalProperties: {} description: Output data from the event error: type: - string - 'null' description: Error message if the event failed start_time: type: number description: Event start time as Unix milliseconds end_time: type: number description: Event end time as Unix milliseconds duration: type: number description: Event duration in milliseconds metadata: type: object additionalProperties: {} description: Arbitrary metadata for the event feedback: type: object additionalProperties: {} description: Feedback data associated with the event metrics: type: object additionalProperties: {} description: Metric values computed for the event user_properties: type: object additionalProperties: {} description: User properties associated with the event additionalProperties: {} description: Full event object for legacy event creation endpoints ModelEvent: type: object properties: project: type: string deprecated: true description: Project name (ignored by server — project is determined from API key scope) project_id: type: string description: Project ID source: type: string description: Source of the event (e.g., sdk-python) event_name: type: string description: Name of the event event_type: type: string enum: - model - tool - chain - session description: Type of event (model, tool, chain, or session) event_id: type: string description: Unique event identifier session_id: type: string description: Session this event belongs to parent_id: type: string description: Parent event ID in the trace hierarchy children_ids: type: array items: type: string description: Child event IDs in the trace hierarchy config: type: object additionalProperties: {} description: Configuration used for this event inputs: type: object additionalProperties: {} description: Input data for the event outputs: type: object additionalProperties: {} description: Output data from the event error: type: - string - 'null' deprecated: true description: Legacy alias — remapped to inputs.error for model events start_time: type: number description: Event start time as Unix milliseconds end_time: type: number description: Event end time as Unix milliseconds duration: type: number deprecated: true description: Legacy alias — remapped to inputs.duration for model events metadata: type: object additionalProperties: {} description: Arbitrary metadata for the event feedback: type: object additionalProperties: {} description: Feedback data associated with the event metrics: type: object additionalProperties: {} description: Metric values computed for the event user_properties: type: object additionalProperties: {} description: User properties associated with the event model_name: type: string description: Model identifier (either this or event_name required) model_version: type: string description: Model version string model: type: string deprecated: true description: Legacy alias for model_name messages: type: array items: {} deprecated: true description: Legacy alias — remapped to inputs.messages response: deprecated: true description: Legacy alias — remapped to outputs.response provider: type: string deprecated: true description: Legacy alias — remapped to inputs.provider usage: type: object additionalProperties: {} deprecated: true description: Legacy alias — remapped to inputs.usage cost: deprecated: true description: Legacy alias — remapped to inputs.cost hyperparameters: type: object additionalProperties: {} deprecated: true description: Legacy alias — remapped to inputs.hyperparameters template: deprecated: true description: Legacy alias — remapped to inputs.template template_inputs: deprecated: true description: Legacy alias — remapped to inputs.template_inputs tools: deprecated: true description: Legacy alias — remapped to inputs.tools tool_choice: deprecated: true description: Legacy alias — remapped to inputs.tool_choice response_format: deprecated: true description: Legacy alias — remapped to inputs.response_format additionalProperties: {} description: Model event object with model-specific fields and legacy aliases SessionProperties: type: object properties: session_name: type: string start_time: type: number description: Session start time as Unix milliseconds user_properties: type: object additionalProperties: {} metadata: type: object additionalProperties: {} additionalProperties: {} description: Session properties for batch event creation UpdateEventRequest: type: object properties: metadata: type: object additionalProperties: {} description: Metadata fields to merge into the event feedback: type: object additionalProperties: {} description: Feedback fields to merge into the event metrics: type: object additionalProperties: {} description: Metric values to merge into the event outputs: type: - object - 'null' additionalProperties: {} description: Output object to replace on the event. Must be an object or null; null preserves the existing outputs. Non-object values (strings, arrays, scalars) are rejected. config: type: object additionalProperties: {} description: Configuration fields to merge into the event user_properties: type: object additionalProperties: {} description: User properties to merge into the event duration: type: number description: Event duration in milliseconds end_time: type: number description: Unix timestamp in milliseconds for event end children_ids: type: array items: type: string description: IDs of child events to set (must be non-empty; an empty array is ignored) description: Request body for PUT /v1/events/{event_id} LegacyUpdateEventRequest: type: object properties: event_id: type: string description: Event ID to update metadata: type: object additionalProperties: {} description: Metadata fields to merge into the event feedback: type: object additionalProperties: {} description: Feedback fields to merge into the event metrics: type: object additionalProperties: {} description: Metric values to merge into the event outputs: type: - object - 'null' additionalProperties: {} description: Output object to replace on the event. Must be an object or null; null preserves the existing outputs. Non-object values (strings, arrays, scalars) are rejected. config: type: object additionalProperties: {} description: Configuration fields to merge into the event user_properties: type: object additionalProperties: {} description: User properties to merge into the event duration: type: number description: Event duration in milliseconds end_time: type: number description: Unix timestamp in milliseconds for event end children_ids: type: array items: type: string description: IDs of child events to set (must be non-empty; an empty array is ignored) required: - event_id description: Request to update an existing event (deprecated — use PUT /v1/events/{event_id}) deprecated: true PostEventRequest: type: object properties: project_id: type: string description: Project ID source: type: string description: Source of the event (e.g., sdk-python) event_name: type: string description: Name of the event event_type: type: string enum: - model - tool - chain - session description: Type of event (model, tool, chain, or session) event_id: type: string description: Unique event identifier session_id: type: string description: Session this event belongs to parent_id: type: string description: Parent event ID in the trace hierarchy children_ids: type: array items: type: string description: Child event IDs in the trace hierarchy config: type: object additionalProperties: {} description: Configuration used for this event inputs: type: object additionalProperties: {} description: Input data for the event outputs: type: object additionalProperties: {} description: Output data from the event error: type: - string - 'null' description: Error message if the event failed start_time: type: number description: Event start time as Unix milliseconds end_time: type: number description: Event end time as Unix milliseconds duration: type: number description: Event duration in milliseconds metadata: type: object additionalProperties: {} description: Arbitrary metadata for the event feedback: type: object additionalProperties: {} description: Feedback data associated with the event metrics: type: object additionalProperties: {} description: Metric values computed for the event user_properties: type: object additionalProperties: {} description: User properties associated with the event required: - event_type - inputs additionalProperties: false description: Request body for POST /v1/events (bare event object) LegacyPostEventRequest: type: object properties: event: $ref: '#/components/schemas/LegacyPostEventRequestEvent' required: - event description: Request body for POST /events (deprecated — use POST /v1/events) deprecated: true SearchEventsRequest: type: object properties: filters: $ref: '#/components/schemas/EventSearchFiltersArray' description: Array of filter criteria to apply dateRange: $ref: '#/components/schemas/SearchEventsRequestDateRange' limit: type: integer minimum: 1 maximum: 1000 description: Limit number of results (default 1000, max 1000) page: type: integer minimum: 1 description: Page number of results (default 1) ignore_order: type: boolean description: 'Deprecated: accepted but ignored. Results are always ordered by start_time descending.' evaluation_id: type: string description: Filter by evaluation/experiment run ID additionalProperties: false description: Request body for POST /v1/events/search LegacyExportEventsRequest: type: object properties: filters: $ref: '#/components/schemas/FiltersArray' description: Array of filter criteria to apply dateRange: $ref: '#/components/schemas/LegacyExportEventsRequestDateRange' projections: type: array items: type: string description: Fields to include in the response limit: type: number description: Limit number of results (default 1000, max 7500) page: type: number description: Page number of results (default 1) ignore_order: type: boolean description: If true, skip result ordering for faster queries evaluation_id: type: string description: Filter by evaluation/experiment run ID description: Request body for POST /v1/events/export PostModelEventRequest: type: object properties: model_event: $ref: '#/components/schemas/ModelEvent' required: - model_event description: Request body for POST /events/model PostEventBatchRequest: type: object properties: events: type: array items: $ref: '#/components/schemas/PostEventRequest' description: Array of events to create single_session: type: boolean description: If true, all events share the same session session_properties: $ref: '#/components/schemas/SessionProperties' required: - events additionalProperties: false description: Request body for POST /v1/events/batch LegacyPostEventBatchRequest: type: object properties: events: type: array items: $ref: '#/components/schemas/LegacyEvent' description: Array of events to create single_session: type: boolean description: If true, all events share the same session is_single_session: type: boolean deprecated: true description: Legacy field name for single_session (backward compatibility) session: $ref: '#/components/schemas/SessionProperties' description: Alias for session_properties (backward compatibility) deprecated: true session_properties: $ref: '#/components/schemas/SessionProperties' required: - events description: Request body for POST /events/batch (deprecated — use POST /v1/events/batch) deprecated: true PostModelEventBatchRequest: type: object properties: model_events: type: array items: $ref: '#/components/schemas/ModelEvent' description: Array of model events to create single_session: type: boolean description: If true, all events share the same session is_single_session: type: boolean deprecated: true description: Legacy field name for single_session (backward compatibility) session: $ref: '#/components/schemas/SessionProperties' description: Alias for session_properties (backward compatibility) deprecated: true session_properties: $ref: '#/components/schemas/SessionProperties' required: - model_events description: Request body for POST /events/model/batch GetEventsQuery: type: object properties: dateRange: anyOf: - $ref: '#/components/schemas/RelativeDateRange' - $ref: '#/components/schemas/AbsoluteDateRange' description: Date range filter filters: $ref: '#/components/schemas/FiltersArray' description: Event filters to apply projections: type: array items: type: string description: Fields to include in the response ignore_order: type: boolean description: If true, skip result ordering for faster queries limit: type: number description: Maximum number of results to return page: type: number description: Page number for pagination evaluation_id: type: string description: Filter by evaluation/experiment run ID description: Query parameters for GET /events PostEventResponse: type: object properties: success: type: boolean event_id: type: string required: - success - event_id description: Response after creating an event GetEventsResponse: type: object properties: events: type: array items: {} totalEvents: type: number required: - events - totalEvents description: Response for GET /events GetEventResponse: type: object properties: event: $ref: '#/components/schemas/GetEventResponseEvent' required: - event description: Response for GET /events/:event_id — single event payload ExportEventsResponse: type: object properties: events: type: array items: $ref: '#/components/schemas/LegacyEvent' count: type: number required: - events - count description: Response for POST /v1/events/search and POST /v1/events/export PostEventBatchResponse: type: object properties: event_ids: type: array items: type: string session_id: type: string success: type: boolean required: - event_ids - success description: Response for POST /events/batch PassingRange: type: object properties: min: type: number max: type: number MetricDatapoints: type: object properties: passed: type: array items: type: string description: Datapoint/session IDs that passed this metric failed: type: array items: type: string description: Datapoint/session IDs that failed this metric required: - passed - failed MetricDetail: type: object properties: metric_name: type: string metric_type: type: string enum: - COMPOSITE - PYTHON - LLM - HUMAN - CLIENT_SIDE description: Type of metric evaluator event_name: type: string description: Name of the event this metric was computed on event_type: type: string description: Type of event (model, tool, chain, session) aggregate: type: number description: Aggregated metric value values: type: array items: type: number description: Individual metric values across datapoints passing_range: $ref: '#/components/schemas/PassingRange' datapoints: $ref: '#/components/schemas/MetricDatapoints' required: - metric_name DatapointResult: type: object properties: datapoint_id: type: - string - 'null' session_id: type: string passed: type: boolean metrics: type: array items: {} required: - session_id - passed EventDetail: type: object properties: event_name: type: string event_type: type: string required: - event_name - event_type ExperimentSchemaField: type: object properties: name: type: string event_type: type: string required: - name - event_type ExperimentSchemaMappingEntry: type: object properties: field_name: type: string event_type: type: string required: - field_name - event_type MetricsAggregation: type: object properties: aggregation_function: type: string details: type: array items: $ref: '#/components/schemas/MetricDetail' ExperimentRunObject: type: object properties: id: type: string run_id: type: string name: type: - string - 'null' description: type: - string - 'null' status: type: - string - 'null' metadata: type: object additionalProperties: {} results: type: object additionalProperties: {} metrics: type: object additionalProperties: type: number event_ids: type: array items: type: string configuration: type: object additionalProperties: {} is_active: type: boolean created_at: anyOf: - type: string - type: string format: date-time updated_at: anyOf: - type: string - type: string format: date-time - type: 'null' scope_type: type: string scope_id: type: string dataset_id: type: - string - 'null' dataset_name: type: - string - 'null' required: - id - run_id - metadata - results - event_ids - configuration - is_active - created_at - scope_type - scope_id EventComparisonDetail: type: object properties: event_name: type: string event_type: type: string presence: type: string enum: - old - new - both required: - event_name - event_type - presence MetricComparison: type: object properties: metric_name: type: string metric_type: type: string old_aggregate: type: number new_aggregate: type: number difference: type: number percentage_change: type: number required: - metric_name EventMetricData: type: object properties: event_name: type: string event_type: type: string session_id: type: string metadata: $ref: '#/components/schemas/EventMetricDataMetadata' metrics: type: object additionalProperties: anyOf: - type: number - type: boolean description: Metric name → value, merged from numeric, float, and boolean ClickHouse columns required: - event_name - event_type - session_id - metadata - metrics ComparableEvent: type: object properties: datapoint_id: type: string event_1: type: object additionalProperties: {} description: Full event record from the first run event_2: type: object additionalProperties: {} description: Full event record from the second run required: - datapoint_id - event_1 - event_2 PostExperimentRunRequest: type: object properties: run_id: type: string name: type: string minLength: 1 description: Experiment run display name description: type: string status: type: string enum: - pending - completed - failed - cancelled - running default: pending metadata: type: object additionalProperties: {} results: type: object additionalProperties: {} dataset_id: type: - string - 'null' event_ids: type: array items: type: string default: [] configuration: type: object additionalProperties: {} evaluators: type: array items: {} default: [] session_ids: type: array items: type: string default: [] datapoint_ids: type: array items: type: string minLength: 1 default: [] passing_ranges: type: object additionalProperties: $ref: '#/components/schemas/PassingRange' description: Request body for POST /runs PutExperimentRunRequest: type: object properties: name: type: string minLength: 1 description: Experiment run display name description: type: string status: type: string enum: - pending - completed - failed - cancelled - running metadata: type: object additionalProperties: {} results: type: object additionalProperties: {} event_ids: type: array items: type: string configuration: type: object additionalProperties: {} evaluators: type: array items: {} session_ids: type: array items: type: string datapoint_ids: type: array items: type: string minLength: 1 passing_ranges: type: object additionalProperties: $ref: '#/components/schemas/PassingRange' description: Request body for PUT /runs/{run_id} GetExperimentRunsQuery: type: object properties: dataset_id: type: string minLength: 1 description: Filter by dataset ID page: type: integer minimum: 1 default: 1 description: Page number for pagination limit: type: integer minimum: 1 maximum: 100 default: 20 description: Number of results per page (max 100) run_ids: type: array items: type: string description: Filter by specific run IDs name: type: string description: Filter by run name status: type: string enum: - pending - completed - failed - cancelled - running description: Filter by run status dateRange: anyOf: - type: string - $ref: '#/components/schemas/AbsoluteDateRange' description: Filter by date range sort_by: type: string enum: - created_at - updated_at - name - status default: created_at description: Field to sort results by sort_order: type: string enum: - asc - desc default: desc description: Sort order description: Query parameters for GET /runs GetExperimentRunParams: type: object properties: run_id: type: string required: - run_id description: Path parameters for GET /runs/{run_id} GetExperimentRunMetricsQuery: type: object properties: dateRange: type: string description: Date range filter as JSON string filters: anyOf: - type: string - type: array items: {} description: Filters to apply to metrics description: Query parameters for GET /runs/{run_id}/metrics LegacyGetExperimentRunResultQuery: type: object properties: aggregate_function: type: string default: average description: 'Aggregation function to apply (default: average)' filters: anyOf: - type: string - type: array items: {} description: Filters to apply to results description: Query parameters for GET /runs/{run_id}/result (deprecated — use GET /runs/{run_id}/summary) deprecated: true LegacyGetExperimentRunCompareParams: type: object properties: new_run_id: type: string description: The new run ID to compare old_run_id: type: string description: The old run ID to compare against required: - new_run_id - old_run_id description: Path parameters for GET /runs/{new_run_id}/compare-with/{old_run_id} (deprecated — use GET /runs/{new_run_id}/compare/{old_run_id}) deprecated: true LegacyGetExperimentRunCompareQuery: type: object properties: aggregate_function: type: string default: average description: 'Aggregation function to apply (default: average)' filters: anyOf: - type: string - type: array items: {} description: Filters to apply to comparison description: Query parameters for GET /runs/{new_run_id}/compare-with/{old_run_id} (deprecated — use GET /runs/{new_run_id}/compare/{old_run_id}) deprecated: true LegacyGetExperimentRunCompareEventsQuery: type: object properties: run_id_1: type: string description: First run ID to compare run_id_2: type: string description: Second run ID to compare event_name: type: string description: Filter by event name event_type: type: string description: Filter by event type filter: anyOf: - type: string - type: object additionalProperties: {} description: Additional filter criteria limit: type: integer exclusiveMinimum: 0 maximum: 1000 default: 1000 description: Maximum number of results (max 1000) page: type: integer exclusiveMinimum: 0 default: 1 description: Page number for pagination required: - run_id_1 - run_id_2 description: Query parameters for GET /runs/compare/events (deprecated — use GET /runs/{new_run_id}/compare/{old_run_id}/events) deprecated: true DeleteExperimentRunParams: type: object properties: run_id: type: string required: - run_id description: Path parameters for DELETE /runs/{run_id} LegacyGetEventsSchemaQuery: type: object properties: dateRange: anyOf: - type: string - $ref: '#/components/schemas/AbsoluteDateRange' description: Date range to filter schema by evaluation_id: type: string description: Filter by evaluation/run ID description: Query parameters for GET /events/schema (deprecated — use GET /runs/{run_id}/schema or GET /runs/schema) deprecated: true PostExperimentRunResponse: type: object properties: evaluation: $ref: '#/components/schemas/ExperimentRunObject' run_id: type: string required: - evaluation - run_id description: Response for POST /runs PutExperimentRunResponse: type: object properties: evaluation: $ref: '#/components/schemas/ExperimentRunObject' warning: type: string required: - evaluation description: Response for PUT /runs/{run_id} GetExperimentRunsResponse: type: object properties: evaluations: type: array items: $ref: '#/components/schemas/ExperimentRunObject' pagination: $ref: '#/components/schemas/Pagination' metrics: type: array items: type: string required: - evaluations - pagination - metrics description: Response for GET /runs GetExperimentRunResponse: type: object properties: evaluation: $ref: '#/components/schemas/ExperimentRunObject' required: - evaluation description: Response for GET /runs/{run_id} GetExperimentRunResultResponse: type: object properties: status: type: string success: type: boolean error: type: string passed: type: array items: type: string failed: type: array items: type: string metrics: $ref: '#/components/schemas/MetricsAggregation' datapoints: type: array items: $ref: '#/components/schemas/DatapointResult' event_details: type: array items: $ref: '#/components/schemas/EventDetail' run_object: $ref: '#/components/schemas/ExperimentRunObject' required: - status - success - passed - failed - metrics - datapoints - event_details - run_object description: 'Evaluation summary for an experiment run: pass/fail results, metric aggregations, per-datapoint results, event details, and the experiment run object.' GetExperimentRunCompareResponse: type: object properties: metrics: type: array items: $ref: '#/components/schemas/MetricComparison' commonDatapoints: type: array items: type: string event_details: type: array items: $ref: '#/components/schemas/EventComparisonDetail' old_run: $ref: '#/components/schemas/ExperimentRunObject' new_run: $ref: '#/components/schemas/ExperimentRunObject' required: - metrics - commonDatapoints - event_details - old_run - new_run description: Comparison between two experiment runs including metrics, common datapoints, and event details GetExperimentRunMetricsResponse: type: object properties: events: type: array items: $ref: '#/components/schemas/EventMetricData' totalEvents: type: integer description: Total number of events matching the query required: - events - totalEvents description: Response for GET /runs/{run_id}/metrics GetEventsSchemaResponse: type: object properties: fields: type: array items: $ref: '#/components/schemas/ExperimentSchemaField' datasets: type: array items: type: string mappings: type: object additionalProperties: type: array items: $ref: '#/components/schemas/ExperimentSchemaMappingEntry' required: - fields - datasets - mappings description: Response for GET /events/schema, GET /runs/{run_id}/schema, and GET /runs/schema GetExperimentCompareEventsResponse: type: object properties: events: type: array items: $ref: '#/components/schemas/ComparableEvent' totalEvents: type: integer description: Total number of events matching the comparison query required: - events - totalEvents description: Response for GET /runs/{new_run_id}/compare/{old_run_id}/events DeleteExperimentRunResponse: type: object properties: id: type: string deleted: type: boolean required: - id - deleted description: Response for DELETE /runs/{run_id} MetricItem: type: object properties: name: type: string maxLength: 200 type: type: string enum: - PYTHON - LLM - HUMAN - COMPOSITE criteria: type: string minLength: 1 description: type: - string - 'null' return_type: type: string enum: - float - boolean - string - categorical enabled_in_prod: type: boolean needs_ground_truth: type: boolean sampling_percentage: type: number minimum: 0 maximum: 100 model_provider: type: - string - 'null' model_name: type: - string - 'null' scale: type: - integer - 'null' exclusiveMinimum: 0 threshold: type: - object - 'null' properties: min: type: number max: type: number pass_when: anyOf: - type: boolean - type: number passing_categories: type: array items: type: string maxLength: 200 minItems: 1 additionalProperties: false categories: type: - array - 'null' items: $ref: '#/components/schemas/MetricItemCategoriesItem' minItems: 1 child_metrics: type: - array - 'null' items: $ref: '#/components/schemas/MetricItemChildMetricsItem' minItems: 1 description: Deprecated and ignored. Composite metrics are no longer supported. deprecated: true filters: $ref: '#/components/schemas/MetricItemFilters' id: type: string minLength: 1 created_at: type: string format: date-time updated_at: type: - string - 'null' format: date-time required: - name - type - criteria - description - return_type - enabled_in_prod - needs_ground_truth - sampling_percentage - filters - id - created_at - updated_at additionalProperties: false CreateMetricRequest: type: object properties: name: type: string maxLength: 200 type: type: string enum: - PYTHON - LLM - HUMAN - COMPOSITE criteria: type: string minLength: 1 description: type: string default: '' return_type: type: string enum: - float - boolean - string - categorical default: float enabled_in_prod: type: boolean default: false needs_ground_truth: type: boolean default: false sampling_percentage: type: number minimum: 0 maximum: 100 default: 10 model_provider: type: - string - 'null' model_name: type: - string - 'null' scale: type: - integer - 'null' exclusiveMinimum: 0 threshold: type: - object - 'null' properties: min: type: number max: type: number pass_when: anyOf: - type: boolean - type: number passing_categories: type: array items: type: string maxLength: 200 minItems: 1 additionalProperties: false categories: type: - array - 'null' items: $ref: '#/components/schemas/CreateMetricRequestCategoriesItem' minItems: 1 child_metrics: type: - array - 'null' items: $ref: '#/components/schemas/CreateMetricRequestChildMetricsItem' minItems: 1 description: Deprecated and ignored. Composite metrics are no longer supported. deprecated: true filters: $ref: '#/components/schemas/CreateMetricRequestFilters' required: - name - type - criteria additionalProperties: false description: Request body for POST /metrics UpdateMetricRequest: type: object properties: name: type: string maxLength: 200 type: type: string enum: - PYTHON - LLM - HUMAN - COMPOSITE criteria: type: string minLength: 1 description: type: - string - 'null' return_type: type: string enum: - float - boolean - string - categorical enabled_in_prod: type: boolean needs_ground_truth: type: boolean sampling_percentage: type: number minimum: 0 maximum: 100 model_provider: type: - string - 'null' model_name: type: - string - 'null' scale: type: - integer - 'null' exclusiveMinimum: 0 threshold: $ref: '#/components/schemas/UpdateMetricRequestThreshold' categories: type: array items: $ref: '#/components/schemas/UpdateMetricRequestCategoriesItem' minItems: 1 child_metrics: type: - array - 'null' items: $ref: '#/components/schemas/UpdateMetricRequestChildMetricsItem' minItems: 1 description: Deprecated and ignored. Composite metrics are no longer supported. deprecated: true filters: $ref: '#/components/schemas/UpdateMetricRequestFilters' additionalProperties: false description: Request body for PUT /metrics/{metric_id} LegacyUpdateMetricRequest: type: object properties: name: type: string maxLength: 200 type: type: string enum: - PYTHON - LLM - HUMAN - COMPOSITE criteria: type: string minLength: 1 description: type: - string - 'null' return_type: type: string enum: - float - boolean - string - categorical enabled_in_prod: type: boolean needs_ground_truth: type: boolean sampling_percentage: type: number minimum: 0 maximum: 100 model_provider: type: - string - 'null' model_name: type: - string - 'null' scale: type: - integer - 'null' exclusiveMinimum: 0 threshold: type: - object - 'null' properties: min: type: number max: type: number pass_when: anyOf: - type: boolean - type: number passing_categories: type: array items: type: string maxLength: 200 minItems: 1 additionalProperties: false categories: type: - array - 'null' items: $ref: '#/components/schemas/LegacyUpdateMetricRequestCategoriesItem' minItems: 1 child_metrics: type: - array - 'null' items: $ref: '#/components/schemas/LegacyUpdateMetricRequestChildMetricsItem' minItems: 1 description: Deprecated and ignored. Composite metrics are no longer supported. deprecated: true filters: $ref: '#/components/schemas/LegacyUpdateMetricRequestFilters' id: type: string minLength: 1 required: - id additionalProperties: false description: Request body for PUT /metrics (deprecated — use PUT /metrics/{metric_id}) deprecated: true GetMetricsQuery: type: object properties: type: type: string description: Filter metrics by type id: type: string minLength: 1 description: Filter by metric ID description: Query parameters for GET /metrics RunMetricRequest: type: object properties: metric: $ref: '#/components/schemas/RunMetricRequestMetric' event: $ref: '#/components/schemas/RunMetricRequestEvent' required: - metric - event additionalProperties: false description: Request body for POST /metrics/run LegacyRunMetricRequest: type: object properties: metric: $ref: '#/components/schemas/LegacyRunMetricRequestMetric' event: $ref: '#/components/schemas/LegacyRunMetricRequestEvent' required: - metric - event description: Request body for POST /metrics/run_metric (deprecated — use POST /metrics/run) deprecated: true GetMetricsResponse: type: object properties: metrics: type: array items: $ref: '#/components/schemas/MetricItem' required: - metrics description: Response for GET /metrics CreateMetricResponse: type: object properties: inserted: type: boolean metric_id: type: string minLength: 1 required: - inserted - metric_id description: Response for POST /metrics UpdateMetricResponse: type: object properties: updated: type: boolean required: - updated description: Response for PUT /metrics DeleteMetricResponse: type: object properties: deleted: type: boolean required: - deleted description: Response for DELETE /metrics/{metric_id} RunMetricResponse: type: object properties: success: type: boolean loading: type: boolean result: anyOf: - type: boolean - type: number - type: string - type: 'null' explanation: type: - string - 'null' required: - success - loading - result - explanation description: Response for POST /metrics/run MetricVersionContent: type: object properties: name: type: string maxLength: 200 type: type: string enum: - PYTHON - LLM - HUMAN - COMPOSITE criteria: type: string minLength: 1 description: type: - string - 'null' return_type: type: string enum: - float - boolean - string - categorical enabled_in_prod: type: boolean needs_ground_truth: type: boolean sampling_percentage: type: number minimum: 0 maximum: 100 model_provider: type: - string - 'null' model_name: type: - string - 'null' scale: type: - integer - 'null' exclusiveMinimum: 0 threshold: type: - object - 'null' properties: min: type: number max: type: number pass_when: anyOf: - type: boolean - type: number passing_categories: type: array items: type: string maxLength: 200 minItems: 1 additionalProperties: false categories: type: - array - 'null' items: $ref: '#/components/schemas/MetricVersionContentCategoriesItem' minItems: 1 child_metrics: type: - array - 'null' items: $ref: '#/components/schemas/MetricVersionContentChildMetricsItem' minItems: 1 description: Deprecated and ignored. Composite metrics are no longer supported. deprecated: true filters: $ref: '#/components/schemas/MetricVersionContentFilters' required: - name - type - criteria - description - return_type - enabled_in_prod - needs_ground_truth - sampling_percentage - filters additionalProperties: false description: Versioned snapshot of a metric definition. MetricVersionContentRequest: type: object properties: name: type: string maxLength: 200 type: type: string enum: - PYTHON - LLM - HUMAN - COMPOSITE criteria: type: string minLength: 1 description: type: string default: '' description: Free-form description of the metric. Defaults to an empty string. return_type: type: string enum: - float - boolean - string - categorical default: float description: Return type of the metric. Defaults to `float`. enabled_in_prod: type: boolean default: false description: Whether this version should run against production traffic. Defaults to `false` for non-HUMAN metrics and `true` for HUMAN metrics. needs_ground_truth: type: boolean default: false description: Whether this metric requires ground-truth labels to evaluate. Defaults to `false`. sampling_percentage: type: number minimum: 0 maximum: 100 default: 10 description: Percentage of events the metric should run against, 0–100. Defaults to `10`. model_provider: type: - string - 'null' model_name: type: - string - 'null' scale: type: - integer - 'null' exclusiveMinimum: 0 threshold: type: - object - 'null' properties: min: type: number max: type: number pass_when: anyOf: - type: boolean - type: number passing_categories: type: array items: type: string maxLength: 200 minItems: 1 additionalProperties: false categories: type: - array - 'null' items: $ref: '#/components/schemas/MetricVersionContentRequestCategoriesItem' minItems: 1 child_metrics: type: - array - 'null' items: $ref: '#/components/schemas/MetricVersionContentRequestChildMetricsItem' minItems: 1 description: Deprecated and ignored. Composite metrics are no longer supported. deprecated: true filters: $ref: '#/components/schemas/MetricVersionContentRequestFilters' required: - name - type - criteria additionalProperties: false description: |- Metric definition snapshot accepted by POST /v1/metrics/{metric_id}/versions. Six fields are optional and fall back to server-side defaults when omitted: - `description` → `""` - `return_type` → `"float"` - `enabled_in_prod` → `true` for HUMAN metrics, `false` otherwise - `needs_ground_truth` → `false` - `sampling_percentage` → `10` - `filters` → `{ "filterArray": [] }` MetricVersion: type: object properties: name: type: string full_sha: type: string message: type: string date: type: string format: date-time deployed: type: boolean content: $ref: '#/components/schemas/MetricVersionContent' required: - name - full_sha - message - date - deployed - content additionalProperties: false description: A versioned snapshot of a metric definition. CreateMetricVersionRequest: type: object properties: message: type: string content: $ref: '#/components/schemas/MetricVersionContentRequest' deploy_immediately: type: boolean required: - message - content additionalProperties: false description: Request body for POST /v1/metrics/{metric_id}/versions GetMetricVersionsResponse: type: object properties: success: type: boolean enum: - true data: type: array items: $ref: '#/components/schemas/MetricVersion' required: - success - data additionalProperties: false description: Response for GET /v1/metrics/{metric_id}/versions CreateMetricVersionResponse: type: object properties: success: type: boolean enum: - true data: $ref: '#/components/schemas/MetricVersion' required: - success - data additionalProperties: false description: Response for POST /v1/metrics/{metric_id}/versions DeployMetricVersionResponse: type: object properties: success: type: boolean enum: - true data: $ref: '#/components/schemas/MetricVersion' required: - success - data additionalProperties: false description: Response for POST /v1/metrics/{metric_id}/versions/{version_name}/deploy CreateAnnotationQueueRequest: type: object properties: name: type: string maxLength: 200 description: type: string maxLength: 1000 default: '' filters: $ref: '#/components/schemas/CreateAnnotationQueueRequestFilters' enabled: type: boolean default: true event_ids: type: array items: type: string default: [] required: - name UpdateAnnotationQueueRequest: type: object properties: name: type: string maxLength: 200 description: type: string maxLength: 1000 filters: $ref: '#/components/schemas/UpdateAnnotationQueueRequestFilters' enabled: type: boolean id: type: string minLength: 1 add_event_ids: type: array items: type: string remove_event_ids: type: array items: type: string required: - id CreateAnnotationQueueResponse: type: object properties: queue: $ref: '#/components/schemas/CreateAnnotationQueueResponseQueue' message: type: string required: - queue - message UpdateAnnotationQueueResponse: type: object properties: queue: $ref: '#/components/schemas/UpdateAnnotationQueueResponseQueue' message: type: string required: - queue - message GetAnnotationQueuesResponse: type: object properties: queues: type: array items: $ref: '#/components/schemas/GetAnnotationQueuesResponseQueuesItem' required: - queues GetAnnotationQueueByIdResponse: type: object properties: name: type: string maxLength: 200 description: type: string maxLength: 1000 filters: $ref: '#/components/schemas/GetAnnotationQueueByIdResponseFilters' enabled: type: boolean id: type: string minLength: 1 scope_id: type: string scope_type: type: string enum: - system - controlplane - dataplane - org - workspace - project is_active: type: boolean created_at: type: string format: date-time updated_at: type: - string - 'null' format: date-time required: - name - description - filters - enabled - id - scope_id - scope_type - is_active - created_at - updated_at DeleteAnnotationQueueResponse: type: object properties: message: type: string required: - message PostSessionRequest: type: object properties: event_id: type: string project_id: type: string tenant: type: string event_name: type: string event_type: type: string metrics: type: object additionalProperties: {} metadata: type: object additionalProperties: {} feedback: $ref: '#/components/schemas/PostSessionRequestFeedback' required: - event_id - project_id - tenant additionalProperties: {} description: Minimal event object used by evaluation and session endpoints; permissive (passthrough) LegacyStartSessionRequest: type: object properties: session: $ref: '#/components/schemas/LegacyStartSessionRequestSession' required: - session description: Request body for POST /session/start (deprecated — use POST /v1/sessions) deprecated: true StartSessionRequest: type: object properties: session_id: type: string description: Client-provided session ID (server generates one if omitted) session_name: type: string description: Display name for the session event_name: type: string description: Fallback name if session_name is not provided source: type: string description: Source of the session (e.g., sdk-python) start_time: type: number description: Session start time as Unix milliseconds end_time: type: number description: Session end time as Unix milliseconds duration: type: number description: Session duration in milliseconds config: type: object additionalProperties: {} description: Configuration associated with the session inputs: type: object additionalProperties: {} description: Input data for the session outputs: type: object additionalProperties: {} description: Output data from the session metadata: type: object additionalProperties: {} description: Arbitrary metadata for the session user_properties: type: object additionalProperties: {} description: User properties associated with the session children_ids: type: array items: type: string description: IDs of child events in this session additionalProperties: false description: Request body for POST /v1/sessions (bare session object) AddSessionTracesRequest: type: object properties: logs: type: array items: $ref: '#/components/schemas/LegacyEvent' description: 'Deprecated: use `events` instead. Traces to add as raw event objects.' deprecated: true events: type: array items: $ref: '#/components/schemas/LegacyEvent' description: Traces to add as raw event objects. Replaces `logs`. description: Request body for POST /session/{session_id}/traces (deprecated — use POST /v1/sessions/{session_id}/events/batch). Exactly one of `logs` (deprecated) or `events` must be present. deprecated: true SessionEventBatchRequest: type: object properties: events: type: array items: $ref: '#/components/schemas/PostEventRequest' description: Events to add to the session required: - events additionalProperties: false description: Request body for POST /v1/sessions/{session_id}/events/batch PostSessionStartResponse: type: object properties: event_id: type: string session_id: type: string parent_id: type: - string - 'null' children_ids: type: array items: type: string event_type: type: - string - 'null' event_name: type: - string - 'null' config: {} inputs: {} outputs: {} error: type: - string - 'null' source: type: - string - 'null' duration: type: - number - 'null' user_properties: {} metrics: {} feedback: {} metadata: {} org_id: type: string workspace_id: type: string project_id: type: string start_time: type: - number - 'null' end_time: type: - number - 'null' required: - event_id - session_id - org_id - workspace_id - project_id additionalProperties: {} description: Full session event object returned after starting a new session SessionTracesResponse: type: object properties: success: type: boolean required: - success description: Response from adding traces to a session TODOSchema: type: object properties: message: type: string description: Placeholder - Zod schema not yet implemented required: - message description: Placeholder schema. The response shape for this endpoint is not yet fully specified; refer to the endpoint documentation for the fields it returns. CreateChartResponseData: type: object properties: id: type: string minLength: 1 name: type: string maxLength: 200 description: type: - string - 'null' metric: type: string func: type: - string - 'null' groupBy: type: - string - 'null' bucketing: type: string dateRange: type: - object - 'null' additionalProperties: {} query: type: - array - 'null' items: $ref: '#/components/schemas/QueryFilter' owner_id: type: - string - 'null' minLength: 1 owner_profile: type: - object - 'null' additionalProperties: {} is_active: type: boolean created_at: anyOf: - type: string format: date-time - type: string updated_at: anyOf: - type: string format: date-time - type: string - type: 'null' required: - id - name - description - metric - func - groupBy - bucketing - dateRange - query - owner_id - is_active - created_at - updated_at GetChartResponseData: type: object properties: id: type: string minLength: 1 name: type: string maxLength: 200 description: type: - string - 'null' metric: type: string func: type: - string - 'null' groupBy: type: - string - 'null' bucketing: type: string dateRange: type: - object - 'null' additionalProperties: {} query: type: - array - 'null' items: $ref: '#/components/schemas/QueryFilter' owner_id: type: - string - 'null' minLength: 1 owner_profile: type: - object - 'null' additionalProperties: {} is_active: type: boolean created_at: anyOf: - type: string format: date-time - type: string updated_at: anyOf: - type: string format: date-time - type: string - type: 'null' required: - id - name - description - metric - func - groupBy - bucketing - dateRange - query - owner_id - is_active - created_at - updated_at GetChartsResponseDataItem: type: object properties: id: type: string minLength: 1 name: type: string maxLength: 200 description: type: - string - 'null' metric: type: string func: type: - string - 'null' groupBy: type: - string - 'null' bucketing: type: string dateRange: type: - object - 'null' additionalProperties: {} query: type: - array - 'null' items: $ref: '#/components/schemas/QueryFilter' owner_id: type: - string - 'null' minLength: 1 owner_profile: type: - object - 'null' additionalProperties: {} is_active: type: boolean created_at: anyOf: - type: string format: date-time - type: string updated_at: anyOf: - type: string format: date-time - type: string - type: 'null' required: - id - name - description - metric - func - groupBy - bucketing - dateRange - query - owner_id - is_active - created_at - updated_at UpdateChartResponseData: type: object properties: id: type: string minLength: 1 name: type: string maxLength: 200 description: type: - string - 'null' metric: type: string func: type: - string - 'null' groupBy: type: - string - 'null' bucketing: type: string dateRange: type: - object - 'null' additionalProperties: {} query: type: - array - 'null' items: $ref: '#/components/schemas/QueryFilter' owner_id: type: - string - 'null' minLength: 1 owner_profile: type: - object - 'null' additionalProperties: {} is_active: type: boolean created_at: anyOf: - type: string format: date-time - type: string updated_at: anyOf: - type: string format: date-time - type: string - type: 'null' required: - id - name - description - metric - func - groupBy - bucketing - dateRange - query - owner_id - is_active - created_at - updated_at CreateDatapointResponseResult: type: object properties: insertedIds: type: array items: type: string minLength: 1 required: - insertedIds UpdateDatapointResponseResult: type: object properties: modifiedCount: type: number required: - modifiedCount EventFeedback: type: object properties: ground_truth: {} LegacyPostEventRequestEvent: type: object properties: project: type: string deprecated: true description: Project name (ignored by server — project is determined from API key scope) project_id: type: string description: Project ID source: type: string description: Source of the event (e.g., sdk-python) event_name: type: string description: Name of the event event_type: type: string enum: - model - tool - chain - session event_id: type: string description: Unique event identifier session_id: type: string description: Session this event belongs to parent_id: type: string description: Parent event ID in the trace hierarchy children_ids: type: array items: type: string description: Child event IDs in the trace hierarchy config: type: object additionalProperties: {} description: Configuration used for this event inputs: type: object additionalProperties: {} outputs: type: object additionalProperties: {} description: Output data from the event error: type: - string - 'null' description: Error message if the event failed start_time: type: number description: Event start time as Unix milliseconds end_time: type: number description: Event end time as Unix milliseconds duration: type: number description: Event duration in milliseconds metadata: type: object additionalProperties: {} description: Arbitrary metadata for the event feedback: type: object additionalProperties: {} description: Feedback data associated with the event metrics: type: object additionalProperties: {} description: Metric values computed for the event user_properties: type: object additionalProperties: {} description: User properties associated with the event required: - event_type - inputs additionalProperties: {} description: Full event object for legacy event creation endpoints SearchEventsRequestDateRange: type: object properties: start_time: type: string format: date-time description: ISO 8601 timestamp for start of date range (inclusive) end_time: type: string format: date-time description: ISO 8601 timestamp for end of date range (inclusive) required: - start_time - end_time additionalProperties: false LegacyExportEventsRequestDateRange: type: object properties: $gte: type: string description: ISO String for start of date range $lte: type: string description: ISO String for end of date range required: - $gte - $lte GetEventResponseEvent: type: object properties: project: type: string deprecated: true description: Project name (ignored by server — project is determined from API key scope) project_id: type: string description: Project ID source: type: string description: Source of the event (e.g., sdk-python) event_name: type: string description: Name of the event event_type: type: string enum: - model - tool - chain - session description: Type of event (model, tool, chain, or session) event_id: type: string session_id: type: string description: Session this event belongs to parent_id: type: string description: Parent event ID in the trace hierarchy children_ids: type: array items: type: string description: Child event IDs in the trace hierarchy config: type: object additionalProperties: {} description: Configuration used for this event inputs: type: object additionalProperties: {} description: Input data for the event outputs: type: object additionalProperties: {} description: Output data from the event error: type: - string - 'null' description: Error message if the event failed start_time: type: number description: Event start time as Unix milliseconds end_time: type: number description: Event end time as Unix milliseconds duration: type: number description: Event duration in milliseconds metadata: type: object additionalProperties: {} description: Arbitrary metadata for the event feedback: type: object additionalProperties: {} description: Feedback data associated with the event metrics: type: object additionalProperties: {} description: Metric values computed for the event user_properties: type: object additionalProperties: {} description: User properties associated with the event required: - event_id additionalProperties: {} description: Full event object for legacy event creation endpoints EventMetricDataMetadata: type: object properties: datapoint_id: type: string MetricItemCategoriesItem: type: object properties: category: type: string maxLength: 200 score: type: - number - 'null' required: - category - score additionalProperties: false MetricItemChildMetricsItem: type: object properties: id: type: string minLength: 1 name: type: string maxLength: 200 weight: type: number scale: type: - integer - 'null' exclusiveMinimum: 0 required: - name - weight additionalProperties: false MetricItemFilters: type: object properties: filterArray: $ref: '#/components/schemas/FiltersArray' required: - filterArray additionalProperties: false CreateMetricRequestCategoriesItem: type: object properties: category: type: string maxLength: 200 score: type: - number - 'null' required: - category - score additionalProperties: false CreateMetricRequestChildMetricsItem: type: object properties: id: type: string minLength: 1 name: type: string maxLength: 200 weight: type: number scale: type: - integer - 'null' exclusiveMinimum: 0 required: - name - weight additionalProperties: false CreateMetricRequestFilters: type: object properties: filterArray: $ref: '#/components/schemas/FiltersArray' default: filterArray: [] required: - filterArray additionalProperties: false UpdateMetricRequestThreshold: type: object properties: min: type: number max: type: number pass_when: anyOf: - type: boolean - type: number passing_categories: type: array items: type: string maxLength: 200 minItems: 1 additionalProperties: false UpdateMetricRequestCategoriesItem: type: object properties: category: type: string maxLength: 200 score: type: - number - 'null' required: - category - score additionalProperties: false UpdateMetricRequestChildMetricsItem: type: object properties: id: type: string minLength: 1 name: type: string maxLength: 200 weight: type: number scale: type: - integer - 'null' exclusiveMinimum: 0 required: - name - weight additionalProperties: false UpdateMetricRequestFilters: type: object properties: filterArray: $ref: '#/components/schemas/FiltersArray' required: - filterArray additionalProperties: false LegacyUpdateMetricRequestCategoriesItem: type: object properties: category: type: string maxLength: 200 score: type: - number - 'null' required: - category - score additionalProperties: false LegacyUpdateMetricRequestChildMetricsItem: type: object properties: id: type: string minLength: 1 name: type: string maxLength: 200 weight: type: number scale: type: - integer - 'null' exclusiveMinimum: 0 required: - name - weight additionalProperties: false LegacyUpdateMetricRequestFilters: type: object properties: filterArray: $ref: '#/components/schemas/FiltersArray' required: - filterArray additionalProperties: false RunMetricRequestMetricCategoriesItem: type: object properties: category: type: string maxLength: 200 score: type: - number - 'null' required: - category - score additionalProperties: false RunMetricRequestMetricChildMetricsItem: type: object properties: id: type: string minLength: 1 name: type: string maxLength: 200 weight: type: number scale: type: - integer - 'null' exclusiveMinimum: 0 required: - name - weight additionalProperties: false RunMetricRequestMetricFilters: type: object properties: filterArray: $ref: '#/components/schemas/FiltersArray' default: filterArray: [] required: - filterArray additionalProperties: false RunMetricRequestMetric: type: object properties: name: type: string maxLength: 200 type: type: string enum: - LLM - PYTHON criteria: type: string minLength: 1 description: type: string default: '' return_type: type: string enum: - float - boolean - string - categorical default: float enabled_in_prod: type: boolean default: false needs_ground_truth: type: boolean default: false sampling_percentage: type: number minimum: 0 maximum: 100 default: 10 model_provider: type: - string - 'null' model_name: type: - string - 'null' scale: type: - integer - 'null' exclusiveMinimum: 0 threshold: type: - object - 'null' properties: min: type: number max: type: number pass_when: anyOf: - type: boolean - type: number passing_categories: type: array items: type: string maxLength: 200 minItems: 1 additionalProperties: false categories: type: - array - 'null' items: $ref: '#/components/schemas/RunMetricRequestMetricCategoriesItem' minItems: 1 child_metrics: type: - array - 'null' items: $ref: '#/components/schemas/RunMetricRequestMetricChildMetricsItem' minItems: 1 description: Deprecated and ignored. Composite metrics are no longer supported. deprecated: true filters: $ref: '#/components/schemas/RunMetricRequestMetricFilters' required: - name - type - criteria additionalProperties: false RunMetricRequestEventFeedback: type: object properties: ground_truth: {} additionalProperties: {} RunMetricRequestEvent: type: object properties: event_type: type: string event_name: type: string inputs: type: object additionalProperties: {} outputs: type: object additionalProperties: {} workspace_id: type: string feedback: $ref: '#/components/schemas/RunMetricRequestEventFeedback' additionalProperties: {} LegacyRunMetricRequestMetricCategoriesItem: type: object properties: category: type: string maxLength: 200 score: type: - number - 'null' required: - category - score additionalProperties: false LegacyRunMetricRequestMetricChildMetricsItem: type: object properties: id: type: string minLength: 1 name: type: string maxLength: 200 weight: type: number scale: type: - integer - 'null' exclusiveMinimum: 0 required: - name - weight additionalProperties: false LegacyRunMetricRequestMetricFilters: type: object properties: filterArray: $ref: '#/components/schemas/FiltersArray' default: filterArray: [] required: - filterArray additionalProperties: false LegacyRunMetricRequestMetric: type: object properties: name: type: string maxLength: 200 type: type: string enum: - LLM - PYTHON criteria: type: string minLength: 1 description: type: string default: '' return_type: type: string enum: - float - boolean - string - categorical default: float enabled_in_prod: type: boolean default: false needs_ground_truth: type: boolean default: false sampling_percentage: type: number minimum: 0 maximum: 100 default: 10 model_provider: type: - string - 'null' model_name: type: - string - 'null' scale: type: - integer - 'null' exclusiveMinimum: 0 threshold: type: - object - 'null' properties: min: type: number max: type: number pass_when: anyOf: - type: boolean - type: number passing_categories: type: array items: type: string maxLength: 200 minItems: 1 additionalProperties: false categories: type: - array - 'null' items: $ref: '#/components/schemas/LegacyRunMetricRequestMetricCategoriesItem' minItems: 1 child_metrics: type: - array - 'null' items: $ref: '#/components/schemas/LegacyRunMetricRequestMetricChildMetricsItem' minItems: 1 description: Deprecated and ignored. Composite metrics are no longer supported. deprecated: true filters: $ref: '#/components/schemas/LegacyRunMetricRequestMetricFilters' required: - name - type - criteria additionalProperties: false LegacyRunMetricRequestEventFeedback: type: object properties: ground_truth: {} additionalProperties: {} LegacyRunMetricRequestEvent: type: object properties: event_type: type: string event_name: type: string inputs: type: object additionalProperties: {} outputs: type: object additionalProperties: {} workspace_id: type: string feedback: $ref: '#/components/schemas/LegacyRunMetricRequestEventFeedback' additionalProperties: {} MetricVersionContentCategoriesItem: type: object properties: category: type: string maxLength: 200 score: type: - number - 'null' required: - category - score additionalProperties: false MetricVersionContentChildMetricsItem: type: object properties: id: type: string minLength: 1 name: type: string maxLength: 200 weight: type: number scale: type: - integer - 'null' exclusiveMinimum: 0 required: - name - weight additionalProperties: false MetricVersionContentFilters: type: object properties: filterArray: $ref: '#/components/schemas/FiltersArray' required: - filterArray additionalProperties: false MetricVersionContentRequestCategoriesItem: type: object properties: category: type: string maxLength: 200 score: type: - number - 'null' required: - category - score additionalProperties: false MetricVersionContentRequestChildMetricsItem: type: object properties: id: type: string minLength: 1 name: type: string maxLength: 200 weight: type: number scale: type: - integer - 'null' exclusiveMinimum: 0 required: - name - weight additionalProperties: false MetricVersionContentRequestFilters: type: object properties: filterArray: $ref: '#/components/schemas/FiltersArray' default: filterArray: [] required: - filterArray additionalProperties: false description: 'ETL filter narrowing which events this metric applies to. Defaults to `{ filterArray: [] }` (no filtering).' CreateAnnotationQueueRequestFilters: type: object properties: filterArray: $ref: '#/components/schemas/FiltersArray' default: filterArray: [] required: - filterArray additionalProperties: false UpdateAnnotationQueueRequestFilters: type: object properties: filterArray: $ref: '#/components/schemas/FiltersArray' required: - filterArray additionalProperties: false CreateAnnotationQueueResponseQueueFilters: type: object properties: filterArray: $ref: '#/components/schemas/FiltersArray' required: - filterArray additionalProperties: false CreateAnnotationQueueResponseQueue: type: object properties: name: type: string maxLength: 200 description: type: string maxLength: 1000 filters: $ref: '#/components/schemas/CreateAnnotationQueueResponseQueueFilters' enabled: type: boolean id: type: string minLength: 1 scope_id: type: string scope_type: type: string enum: - system - controlplane - dataplane - org - workspace - project is_active: type: boolean created_at: type: string format: date-time updated_at: type: - string - 'null' format: date-time required: - name - description - filters - enabled - id - scope_id - scope_type - is_active - created_at - updated_at UpdateAnnotationQueueResponseQueueFilters: type: object properties: filterArray: $ref: '#/components/schemas/FiltersArray' required: - filterArray additionalProperties: false UpdateAnnotationQueueResponseQueue: type: object properties: name: type: string maxLength: 200 description: type: string maxLength: 1000 filters: $ref: '#/components/schemas/UpdateAnnotationQueueResponseQueueFilters' enabled: type: boolean id: type: string minLength: 1 scope_id: type: string scope_type: type: string enum: - system - controlplane - dataplane - org - workspace - project is_active: type: boolean created_at: type: string format: date-time updated_at: type: - string - 'null' format: date-time required: - name - description - filters - enabled - id - scope_id - scope_type - is_active - created_at - updated_at GetAnnotationQueuesResponseQueuesItemFilters: type: object properties: filterArray: $ref: '#/components/schemas/FiltersArray' required: - filterArray additionalProperties: false GetAnnotationQueuesResponseQueuesItem: type: object properties: name: type: string maxLength: 200 description: type: string maxLength: 1000 filters: $ref: '#/components/schemas/GetAnnotationQueuesResponseQueuesItemFilters' enabled: type: boolean id: type: string minLength: 1 scope_id: type: string scope_type: type: string enum: - system - controlplane - dataplane - org - workspace - project is_active: type: boolean created_at: type: string format: date-time updated_at: type: - string - 'null' format: date-time required: - name - description - filters - enabled - id - scope_id - scope_type - is_active - created_at - updated_at GetAnnotationQueueByIdResponseFilters: type: object properties: filterArray: $ref: '#/components/schemas/FiltersArray' required: - filterArray additionalProperties: false PostSessionRequestFeedback: type: object properties: ground_truth: {} LegacyStartSessionRequestSession: type: object properties: session_id: type: string description: Client-provided session ID (server generates one if omitted) session_name: type: string description: Display name for the session event_name: type: string description: Fallback name if session_name is not provided source: type: string description: Source of the session (e.g., sdk-python) start_time: type: number description: Session start time as Unix milliseconds end_time: type: number description: Session end time as Unix milliseconds duration: type: number description: Session duration in milliseconds config: type: object additionalProperties: {} description: Configuration associated with the session inputs: type: object additionalProperties: {} description: Input data for the session outputs: type: object additionalProperties: {} description: Output data from the session metadata: type: object additionalProperties: {} description: Arbitrary metadata for the session user_properties: type: object additionalProperties: {} description: User properties associated with the session children_ids: type: array items: type: string description: IDs of child events in this session additionalProperties: {} GetEventsSchemaLegacyDateRangeOneOf1: type: object properties: $gte: oneOf: - type: string - type: number $lte: oneOf: - type: string - type: number GetRunsDateRangeOneOf1: type: object properties: $gte: oneOf: - type: string - type: number $lte: oneOf: - type: string - type: number GetRunsSchemaDateRangeOneOf1: type: object properties: $gte: oneOf: - type: string - type: number $lte: oneOf: - type: string - type: number GetRunSchemaDateRangeOneOf1: type: object properties: $gte: oneOf: - type: string - type: number $lte: oneOf: - type: string - type: number security: - BearerAuth: []