openapi: 3.1.0 info: title: LangSmith access_policies public API description: 'The LangSmith API is used to programmatically create and manage LangSmith resources. ## Host https://api.smith.langchain.com ## Authentication To authenticate with the LangSmith API, set the `X-Api-Key` header to a valid [LangSmith API key](https://docs.langchain.com/langsmith/create-account-api-key#create-an-api-key). ' version: 0.1.0 servers: - url: / tags: - name: public paths: /api/v1/public/{share_token}/run: get: tags: - public summary: Get Shared Run description: Get the shared run. operationId: get_shared_run_api_v1_public__share_token__run_get parameters: - name: share_token in: path required: true schema: type: string format: uuid title: Share Token - name: exclude_s3_stored_attributes in: query required: false schema: type: boolean default: false title: Exclude S3 Stored Attributes responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/RunPublicSchema' '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' /api/v1/public/{share_token}/run/{id}: get: tags: - public summary: Get Shared Run By Id description: Get the shared run. operationId: get_shared_run_by_id_api_v1_public__share_token__run__id__get parameters: - name: id in: path required: true schema: type: string format: uuid title: Id - name: share_token in: path required: true schema: type: string format: uuid title: Share Token - name: exclude_s3_stored_attributes in: query required: false schema: type: boolean default: false title: Exclude S3 Stored Attributes responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/RunPublicSchema' '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' /api/v1/public/{share_token}/runs/query: post: tags: - public summary: Query Shared Runs description: Get run by ids or the shared run if not specifed. operationId: query_shared_runs_api_v1_public__share_token__runs_query_post parameters: - name: share_token in: path required: true schema: type: string format: uuid title: Share Token requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/QueryParamsForPublicRunSchema' responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/ListPublicRunsResponse' '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' /api/v1/public/{share_token}/feedbacks: get: tags: - public summary: Read Shared Feedbacks operationId: read_shared_feedbacks_api_v1_public__share_token__feedbacks_get parameters: - name: share_token in: path required: true schema: type: string format: uuid title: Share Token - name: run in: query required: false schema: anyOf: - type: array items: type: string format: uuid - type: 'null' title: Run - name: key in: query required: false schema: anyOf: - type: array items: type: string - type: 'null' title: Key - name: session in: query required: false schema: anyOf: - type: array items: type: string format: uuid - type: 'null' title: Session - name: source in: query required: false schema: anyOf: - type: array items: $ref: '#/components/schemas/SourceType' - type: 'null' title: Source - name: limit in: query required: false schema: type: integer maximum: 100 minimum: 1 default: 100 title: Limit - name: offset in: query required: false schema: type: integer minimum: 0 default: 0 title: Offset - name: user in: query required: false schema: anyOf: - type: array items: type: string format: uuid - type: 'null' title: User - name: has_comment in: query required: false schema: anyOf: - type: boolean - type: 'null' title: Has Comment - name: has_score in: query required: false schema: anyOf: - type: boolean - type: 'null' title: Has Score - name: level in: query required: false schema: anyOf: - $ref: '#/components/schemas/FeedbackLevel' - type: 'null' title: Level responses: '200': description: Successful Response content: application/json: schema: type: array items: $ref: '#/components/schemas/FeedbackSchema' title: Response Read Shared Feedbacks Api V1 Public Share Token Feedbacks Get '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' /api/v1/public/{share_token}/datasets: get: tags: - public summary: Read Shared Dataset description: Get dataset by ids or the shared dataset if not specifed. operationId: read_shared_dataset_api_v1_public__share_token__datasets_get parameters: - name: share_token in: path required: true schema: type: string format: uuid title: Share Token - name: offset in: query required: false schema: type: integer minimum: 0 default: 0 title: Offset - name: limit in: query required: false schema: type: integer maximum: 100 minimum: 1 default: 100 title: Limit - name: sort_by in: query required: false schema: $ref: '#/components/schemas/SortByDatasetColumn' default: last_session_start_time - name: sort_by_desc in: query required: false schema: type: boolean default: true title: Sort By Desc responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/DatasetPublicSchema' '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' /api/v1/public/{share_token}/examples/count: get: tags: - public summary: Count Shared Examples description: Count all examples by query params operationId: count_shared_examples_api_v1_public__share_token__examples_count_get parameters: - name: share_token in: path required: true schema: type: string format: uuid title: Share Token - name: id in: query required: false schema: anyOf: - type: array items: type: string format: uuid - type: 'null' title: Id - name: as_of in: query required: false schema: anyOf: - type: string format: date-time - type: string description: Only modifications made on or before this time are included. If None, the latest version of the dataset is used. default: latest title: As Of description: Only modifications made on or before this time are included. If None, the latest version of the dataset is used. - name: metadata in: query required: false schema: anyOf: - type: string - type: 'null' title: Metadata - name: filter in: query required: false schema: anyOf: - type: string - type: 'null' title: Filter responses: '200': description: Successful Response content: application/json: schema: type: integer title: Response Count Shared Examples Api V1 Public Share Token Examples Count Get '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' /api/v1/public/{share_token}/examples: get: tags: - public summary: Read Shared Examples description: Get example by ids or the shared example if not specifed. operationId: read_shared_examples_api_v1_public__share_token__examples_get parameters: - name: share_token in: path required: true schema: type: string format: uuid title: Share Token - name: id in: query required: false schema: anyOf: - type: array items: type: string format: uuid - type: 'null' title: Id - name: as_of in: query required: false schema: anyOf: - type: string format: date-time - type: string description: Only modifications made on or before this time are included. If None, the latest version of the dataset is used. default: latest title: As Of description: Only modifications made on or before this time are included. If None, the latest version of the dataset is used. - name: metadata in: query required: false schema: anyOf: - type: string - type: 'null' title: Metadata - name: offset in: query required: false schema: type: integer minimum: 0 default: 0 title: Offset - name: limit in: query required: false schema: type: integer maximum: 100 minimum: 1 default: 100 title: Limit - name: select in: query required: false schema: type: array items: $ref: '#/components/schemas/ExampleSelect' default: - id - created_at - modified_at - name - dataset_id - metadata - inputs - outputs - attachment_urls title: Select - name: filter in: query required: false schema: anyOf: - type: string - type: 'null' title: Filter responses: '200': description: Successful Response content: application/json: schema: type: array items: $ref: '#/components/schemas/Example' title: Response Read Shared Examples Api V1 Public Share Token Examples Get '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' /api/v1/public/{share_token}/datasets/sessions: get: tags: - public summary: Read Shared Dataset Tracer Sessions description: Get projects run on a dataset that has been shared. operationId: read_shared_dataset_tracer_sessions_api_v1_public__share_token__datasets_sessions_get parameters: - name: share_token in: path required: true schema: type: string format: uuid title: Share Token - name: id in: query required: false schema: anyOf: - type: array items: type: string format: uuid - type: 'null' title: Id - name: name in: query required: false schema: anyOf: - type: string - type: 'null' title: Name - name: name_contains in: query required: false schema: anyOf: - type: string - type: 'null' title: Name Contains - name: dataset_version in: query required: false schema: anyOf: - type: string - type: 'null' title: Dataset Version - name: sort_by in: query required: false schema: $ref: '#/components/schemas/SessionSortableColumns' default: start_time - name: sort_by_desc in: query required: false schema: type: boolean default: true title: Sort By Desc - name: sort_by_feedback_key in: query required: false schema: anyOf: - type: string - type: 'null' title: Sort By Feedback Key - name: offset in: query required: false schema: type: integer minimum: 0 default: 0 title: Offset - name: limit in: query required: false schema: type: integer maximum: 100 minimum: 1 default: 100 title: Limit - name: facets in: query required: false schema: type: boolean default: false title: Facets - name: accept in: header required: false schema: anyOf: - type: string - type: 'null' title: Accept responses: '200': description: Successful Response content: application/json: schema: type: array items: $ref: '#/components/schemas/TracerSession' title: Response Read Shared Dataset Tracer Sessions Api V1 Public Share Token Datasets Sessions Get '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' /api/v1/public/datasets/sessions-bulk: get: tags: - public summary: Read Shared Dataset Tracer Sessions Bulk description: Get sessions from multiple datasets using share tokens. operationId: read_shared_dataset_tracer_sessions_bulk_api_v1_public_datasets_sessions_bulk_get parameters: - name: share_tokens in: query required: true schema: type: array items: type: string title: Share Tokens responses: '200': description: Successful Response content: application/json: schema: type: array items: $ref: '#/components/schemas/TracerSession' title: Response Read Shared Dataset Tracer Sessions Bulk Api V1 Public Datasets Sessions Bulk Get '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' /api/v1/public/{share_token}/examples/runs: post: tags: - public summary: Read Shared Dataset Examples With Runs description: Get examples with associated runs from sessions in a dataset that has been shared. operationId: read_shared_dataset_examples_with_runs_api_v1_public__share_token__examples_runs_post parameters: - name: share_token in: path required: true schema: type: string format: uuid title: Share Token requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/QueryExampleSchemaWithRuns' responses: '200': description: Successful Response content: application/json: schema: anyOf: - type: array items: $ref: '#/components/schemas/PublicExampleWithRuns' - type: array items: $ref: '#/components/schemas/ExampleWithRunsCH' title: Response Read Shared Dataset Examples With Runs Api V1 Public Share Token Examples Runs Post '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' /api/v1/public/{share_token}/datasets/runs/delta: post: tags: - public summary: Read Shared Delta description: Fetch the number of regressions/improvements for each example in a dataset, between sessions[0] and sessions[1]. operationId: read_shared_delta_api_v1_public__share_token__datasets_runs_delta_post parameters: - name: share_token in: path required: true schema: type: string format: uuid title: Share Token requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/QueryFeedbackDelta' responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/SessionFeedbackDelta' '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' /api/v1/public/{share_token}/datasets/runs/delta/stream: post: tags: - public summary: Read Shared Delta Stream description: "Stream feedback deltas for multiple feedback keys.\n\nReturns results in chunks as they become available. Each chunk contains\nresults for one or more feedback keys. Errors for individual chunks are\nincluded in the response rather than failing the entire operation.\n\nResponse format (SSE):\n event: data\n data: {\"feedback_deltas\": {\"key1\": {session_id: {...}}, ...}, \"errors\": null}\n\n event: data\n data: {\"feedback_deltas\": {\"key2\": {...}}, \"errors\": null}\n\n event: end" operationId: read_shared_delta_stream_api_v1_public__share_token__datasets_runs_delta_stream_post parameters: - name: share_token in: path required: true schema: type: string format: uuid title: Share Token requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/QueryFeedbackDeltaBatch' responses: '200': description: Successful Response content: application/json: schema: {} '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' /api/v1/public/{share_token}/datasets/runs/query: post: tags: - public summary: Query Shared Dataset Runs description: Get runs in projects run over a dataset that has been shared. operationId: query_shared_dataset_runs_api_v1_public__share_token__datasets_runs_query_post parameters: - name: share_token in: path required: true schema: type: string format: uuid title: Share Token requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/BodyParamsForRunSchema' responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/ListPublicDatasetRunsResponse' '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' /api/v1/public/{share_token}/datasets/runs/generate-query: post: tags: - public summary: Generate Query For Shared Dataset Runs description: Get runs in projects run over a dataset that has been shared. operationId: generate_query_for_shared_dataset_runs_api_v1_public__share_token__datasets_runs_generate_query_post parameters: - name: share_token in: path required: true schema: type: string format: uuid title: Share Token requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/RequestBodyForRunsGenerateQuery' responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/ResponseBodyForRunsGenerateQuery' '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' /api/v1/public/{share_token}/datasets/runs/stats: post: tags: - public summary: Stats Shared Dataset Runs description: Get run stats in projects run over a dataset that has been shared. operationId: stats_shared_dataset_runs_api_v1_public__share_token__datasets_runs_stats_post parameters: - name: share_token in: path required: true schema: type: string format: uuid title: Share Token requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/RunStatsQueryParams' responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/RunStats' '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' /api/v1/public/{share_token}/datasets/runs/{run_id}: get: tags: - public summary: Read Shared Dataset Run description: Get runs in projects run over a dataset that has been shared. operationId: read_shared_dataset_run_api_v1_public__share_token__datasets_runs__run_id__get parameters: - name: run_id in: path required: true schema: type: string format: uuid title: Run Id - name: share_token in: path required: true schema: type: string format: uuid title: Share Token - name: exclude_s3_stored_attributes in: query required: false schema: type: boolean default: false title: Exclude S3 Stored Attributes responses: '200': description: Successful Response content: application/json: schema: $ref: '#/components/schemas/RunPublicDatasetSchema' '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' /api/v1/public/{share_token}/datasets/feedback: get: tags: - public summary: Read Shared Dataset Feedback description: Get feedback for runs in projects run over a dataset that has been shared. operationId: read_shared_dataset_feedback_api_v1_public__share_token__datasets_feedback_get parameters: - name: share_token in: path required: true schema: type: string format: uuid title: Share Token - name: run in: query required: false schema: anyOf: - type: array items: type: string format: uuid - type: 'null' title: Run - name: key in: query required: false schema: anyOf: - type: array items: type: string - type: 'null' title: Key - name: session in: query required: false schema: anyOf: - type: array items: type: string format: uuid - type: 'null' title: Session - name: source in: query required: false schema: anyOf: - type: array items: $ref: '#/components/schemas/SourceType' - type: 'null' title: Source - name: limit in: query required: false schema: type: integer maximum: 100 minimum: 1 default: 100 title: Limit - name: offset in: query required: false schema: type: integer minimum: 0 default: 0 title: Offset - name: user in: query required: false schema: anyOf: - type: array items: type: string format: uuid - type: 'null' title: User - name: has_comment in: query required: false schema: anyOf: - type: boolean - type: 'null' title: Has Comment - name: has_score in: query required: false schema: anyOf: - type: boolean - type: 'null' title: Has Score - name: level in: query required: false schema: anyOf: - $ref: '#/components/schemas/FeedbackLevel' - type: 'null' title: Level responses: '200': description: Successful Response content: application/json: schema: type: array items: $ref: '#/components/schemas/FeedbackSchema' title: Response Read Shared Dataset Feedback Api V1 Public Share Token Datasets Feedback Get '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' /api/v1/public/{share_token}/datasets/comparative: get: tags: - public summary: Read Shared Comparative Experiments description: Get all comparative experiments for a given dataset. operationId: read_shared_comparative_experiments_api_v1_public__share_token__datasets_comparative_get parameters: - name: share_token in: path required: true schema: type: string format: uuid title: Share Token - name: name in: query required: false schema: anyOf: - type: string - type: 'null' title: Name - name: name_contains in: query required: false schema: anyOf: - type: string - type: 'null' title: Name Contains - name: offset in: query required: false schema: type: integer minimum: 0 default: 0 title: Offset - name: limit in: query required: false schema: type: integer maximum: 100 minimum: 1 default: 100 title: Limit - name: sort_by in: query required: false schema: $ref: '#/components/schemas/SortByComparativeExperimentColumn' default: created_at - name: sort_by_desc in: query required: false schema: type: boolean default: true title: Sort By Desc responses: '200': description: Successful Response content: application/json: schema: type: array items: $ref: '#/components/schemas/PublicComparativeExperiment' title: Response Read Shared Comparative Experiments Api V1 Public Share Token Datasets Comparative Get '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' /api/v1/public/schemas/{version}/message.json: get: tags: - public summary: Get Message Json Schema operationId: get_message_json_schema_api_v1_public_schemas__version__message_json_get parameters: - name: version in: path required: true schema: type: string title: Version responses: '200': description: Successful Response content: application/json: schema: {} '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' /api/v1/public/schemas/{version}/tooldef.json: get: tags: - public summary: Get Tool Def Json Schema operationId: get_tool_def_json_schema_api_v1_public_schemas__version__tooldef_json_get parameters: - name: version in: path required: true schema: type: string title: Version responses: '200': description: Successful Response content: application/json: schema: {} '422': description: Validation Error content: application/json: schema: $ref: '#/components/schemas/HTTPValidationError' components: schemas: SourceType: type: string enum: - api - model - app - auto_eval title: SourceType description: Enum for feedback source types. SessionFeedbackDelta: properties: feedback_deltas: additionalProperties: $ref: '#/components/schemas/FeedbackDelta' propertyNames: format: uuid type: object title: Feedback Deltas type: object required: - feedback_deltas title: SessionFeedbackDelta description: List of feedback keys with number of improvements and regressions for each. RunTypeEnum: type: string enum: - tool - chain - llm - retriever - embedding - prompt - parser title: RunTypeEnum description: Enum for run types. Example: properties: outputs: anyOf: - additionalProperties: true type: object - type: 'null' title: Outputs dataset_id: type: string format: uuid title: Dataset Id source_run_id: anyOf: - type: string format: uuid - type: 'null' title: Source Run Id metadata: anyOf: - additionalProperties: true type: object - type: 'null' title: Metadata inputs: additionalProperties: true type: object title: Inputs created_at: type: string format: date-time title: Created At id: type: string format: uuid title: Id name: type: string title: Name modified_at: anyOf: - type: string format: date-time - type: 'null' title: Modified At attachment_urls: anyOf: - additionalProperties: true type: object - type: 'null' title: Attachment Urls type: object required: - dataset_id - inputs - id - name title: Example description: Example schema. RunsGenerateQueryFeedbackKeys: type: string enum: - user_score - user_edited - user_removed - user_opened_run - user_selected_run - results_size - valid_filter title: RunsGenerateQueryFeedbackKeys ExampleSelect: type: string enum: - id - created_at - modified_at - name - dataset_id - source_run_id - metadata - inputs - outputs - attachment_urls title: ExampleSelect DataType: type: string enum: - kv - llm - chat title: DataType description: Enum for dataset data types. ValidationError: properties: loc: items: anyOf: - type: string - type: integer type: array title: Location msg: type: string title: Message type: type: string title: Error Type type: object required: - loc - msg - type title: ValidationError SimpleExperimentInfo: properties: id: type: string format: uuid title: Id name: type: string title: Name type: object required: - id - name title: SimpleExperimentInfo description: Simple experiment info schema for use with comparative experiments RunSchemaComparisonView: properties: name: type: string title: Name inputs: anyOf: - additionalProperties: true type: object - type: 'null' title: Inputs inputs_preview: anyOf: - type: string - type: 'null' title: Inputs Preview run_type: $ref: '#/components/schemas/RunTypeEnum' start_time: type: string format: date-time title: Start Time end_time: anyOf: - type: string format: date-time - type: 'null' title: End Time extra: anyOf: - additionalProperties: true type: object - type: 'null' title: Extra error: anyOf: - type: string - type: 'null' title: Error execution_order: type: integer minimum: 1.0 title: Execution Order default: 1 serialized: anyOf: - additionalProperties: true type: object - type: 'null' title: Serialized outputs: anyOf: - additionalProperties: true type: object - type: 'null' title: Outputs outputs_preview: anyOf: - type: string - type: 'null' title: Outputs Preview parent_run_id: anyOf: - type: string format: uuid - type: 'null' title: Parent Run Id manifest_id: anyOf: - type: string format: uuid - type: 'null' title: Manifest Id manifest_s3_id: anyOf: - type: string format: uuid - type: 'null' title: Manifest S3 Id events: anyOf: - items: additionalProperties: true type: object type: array - type: 'null' title: Events tags: anyOf: - items: type: string type: array - type: 'null' title: Tags inputs_s3_urls: anyOf: - additionalProperties: true type: object - type: 'null' title: Inputs S3 Urls outputs_s3_urls: anyOf: - additionalProperties: true type: object - type: 'null' title: Outputs S3 Urls s3_urls: anyOf: - additionalProperties: true type: object - type: 'null' title: S3 Urls trace_id: type: string format: uuid title: Trace Id dotted_order: anyOf: - type: string - type: 'null' title: Dotted Order trace_min_start_time: anyOf: - type: string format: date-time - type: 'null' title: Trace Min Start Time trace_max_start_time: anyOf: - type: string format: date-time - type: 'null' title: Trace Max Start Time id: type: string format: uuid title: Id session_id: type: string format: uuid title: Session Id reference_example_id: anyOf: - type: string format: uuid - type: 'null' title: Reference Example Id total_tokens: anyOf: - type: integer - type: 'null' title: Total Tokens prompt_tokens: anyOf: - type: integer - type: 'null' title: Prompt Tokens completion_tokens: anyOf: - type: integer - type: 'null' title: Completion Tokens total_cost: anyOf: - type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ - type: 'null' title: Total Cost prompt_cost: anyOf: - type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ - type: 'null' title: Prompt Cost completion_cost: anyOf: - type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ - type: 'null' title: Completion Cost status: type: string title: Status feedback_stats: anyOf: - additionalProperties: additionalProperties: true type: object type: object - type: 'null' title: Feedback Stats app_path: anyOf: - type: string - type: 'null' title: App Path feedbacks: items: $ref: '#/components/schemas/FeedbackSchema' type: array title: Feedbacks type: object required: - name - run_type - trace_id - id - session_id - status title: RunSchemaComparisonView description: Run schema for comparison view. SessionSortableColumns: type: string enum: - name - start_time - last_run_start_time - latency_p50 - latency_p99 - error_rate - feedback - runs_count title: SessionSortableColumns FeedbackDelta: properties: improved_examples: items: type: string format: uuid type: array title: Improved Examples regressed_examples: items: type: string format: uuid type: array title: Regressed Examples type: object required: - improved_examples - regressed_examples title: FeedbackDelta description: Feedback key with number of improvements and regressions. FeedbackSource: properties: type: anyOf: - type: string - type: 'null' title: Type metadata: anyOf: - additionalProperties: true type: object - type: 'null' title: Metadata user_id: anyOf: - type: string format: uuid - type: 'null' title: User Id ls_user_id: anyOf: - type: string format: uuid - type: 'null' title: Ls User Id user_name: anyOf: - type: string - type: 'null' title: User Name type: object title: FeedbackSource description: The feedback source loaded from the database. RequestBodyForRunsGenerateQuery: properties: query: type: string title: Query feedback_keys: items: $ref: '#/components/schemas/RunsGenerateQueryFeedbackKeys' type: array title: Feedback Keys type: object required: - query title: RequestBodyForRunsGenerateQuery RunsFilterDataSourceTypeEnum: type: string enum: - current - historical - lite - root_lite - runs_feedbacks_rmt_wide title: RunsFilterDataSourceTypeEnum description: Enum for run data source types. QueryFeedbackDelta: properties: baseline_session_id: type: string format: uuid title: Baseline Session Id comparison_session_ids: items: type: string format: uuid type: array title: Comparison Session Ids feedback_key: type: string title: Feedback Key filters: anyOf: - additionalProperties: items: type: string type: array propertyNames: format: uuid type: object - type: 'null' title: Filters offset: type: integer minimum: 0.0 title: Offset default: 0 limit: type: integer maximum: 100.0 minimum: 1.0 title: Limit default: 100 comparative_experiment_id: anyOf: - type: string format: uuid - type: 'null' title: Comparative Experiment Id type: object required: - baseline_session_id - comparison_session_ids - feedback_key title: QueryFeedbackDelta SortByDatasetColumn: type: string enum: - name - created_at - last_session_start_time - example_count - session_count - modified_at title: SortByDatasetColumn description: Enum for available dataset columns to sort by. RunPublicDatasetSchema: properties: name: type: string title: Name inputs: anyOf: - additionalProperties: true type: object - type: 'null' title: Inputs inputs_preview: anyOf: - type: string - type: 'null' title: Inputs Preview run_type: $ref: '#/components/schemas/RunTypeEnum' start_time: type: string format: date-time title: Start Time end_time: anyOf: - type: string format: date-time - type: 'null' title: End Time extra: anyOf: - additionalProperties: true type: object - type: 'null' title: Extra error: anyOf: - type: string - type: 'null' title: Error execution_order: type: integer minimum: 1.0 title: Execution Order default: 1 serialized: anyOf: - additionalProperties: true type: object - type: 'null' title: Serialized outputs: anyOf: - additionalProperties: true type: object - type: 'null' title: Outputs outputs_preview: anyOf: - type: string - type: 'null' title: Outputs Preview parent_run_id: anyOf: - type: string format: uuid - type: 'null' title: Parent Run Id manifest_id: anyOf: - type: string format: uuid - type: 'null' title: Manifest Id manifest_s3_id: anyOf: - type: string format: uuid - type: 'null' title: Manifest S3 Id events: anyOf: - items: additionalProperties: true type: object type: array - type: 'null' title: Events tags: anyOf: - items: type: string type: array - type: 'null' title: Tags inputs_s3_urls: anyOf: - additionalProperties: true type: object - type: 'null' title: Inputs S3 Urls outputs_s3_urls: anyOf: - additionalProperties: true type: object - type: 'null' title: Outputs S3 Urls s3_urls: anyOf: - additionalProperties: true type: object - type: 'null' title: S3 Urls trace_id: type: string format: uuid title: Trace Id dotted_order: type: string title: Dotted Order trace_min_start_time: anyOf: - type: string format: date-time - type: 'null' title: Trace Min Start Time trace_max_start_time: anyOf: - type: string format: date-time - type: 'null' title: Trace Max Start Time id: type: string format: uuid title: Id status: type: string title: Status child_run_ids: anyOf: - items: type: string format: uuid type: array - type: 'null' title: Child Run Ids direct_child_run_ids: anyOf: - items: type: string format: uuid type: array - type: 'null' title: Direct Child Run Ids parent_run_ids: anyOf: - items: type: string format: uuid type: array - type: 'null' title: Parent Run Ids feedback_stats: anyOf: - additionalProperties: additionalProperties: true type: object type: object - type: 'null' title: Feedback Stats reference_example_id: anyOf: - type: string format: uuid - type: 'null' title: Reference Example Id total_tokens: type: integer title: Total Tokens default: 0 prompt_tokens: type: integer title: Prompt Tokens default: 0 completion_tokens: type: integer title: Completion Tokens default: 0 prompt_token_details: anyOf: - additionalProperties: type: integer type: object - type: 'null' title: Prompt Token Details completion_token_details: anyOf: - additionalProperties: type: integer type: object - type: 'null' title: Completion Token Details total_cost: anyOf: - type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ - type: 'null' title: Total Cost prompt_cost: anyOf: - type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ - type: 'null' title: Prompt Cost completion_cost: anyOf: - type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ - type: 'null' title: Completion Cost prompt_cost_details: anyOf: - additionalProperties: type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ type: object - type: 'null' title: Prompt Cost Details completion_cost_details: anyOf: - additionalProperties: type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ type: object - type: 'null' title: Completion Cost Details price_model_id: anyOf: - type: string format: uuid - type: 'null' title: Price Model Id first_token_time: anyOf: - type: string format: date-time - type: 'null' title: First Token Time messages: anyOf: - items: additionalProperties: true type: object type: array - type: 'null' title: Messages session_id: type: string format: uuid title: Session Id type: object required: - name - run_type - trace_id - dotted_order - id - status - session_id title: RunPublicDatasetSchema description: Schema for a run in a publicly-shared dataset. RunSelect: type: string enum: - id - name - run_type - start_time - end_time - status - error - extra - events - inputs - inputs_preview - inputs_s3_urls - inputs_or_signed_url - outputs - outputs_preview - outputs_s3_urls - outputs_or_signed_url - s3_urls - error_or_signed_url - events_or_signed_url - extra_or_signed_url - serialized_or_signed_url - parent_run_id - manifest_id - manifest_s3_id - manifest - session_id - serialized - reference_example_id - reference_dataset_id - total_tokens - prompt_tokens - prompt_token_details - completion_tokens - completion_token_details - total_cost - prompt_cost - prompt_cost_details - completion_cost - completion_cost_details - price_model_id - first_token_time - trace_id - dotted_order - last_queued_at - feedback_stats - child_run_ids - parent_run_ids - tags - in_dataset - app_path - share_token - trace_tier - trace_first_received_at - ttl_seconds - trace_upgrade - thread_id - trace_min_max_start_time - messages - inserted_at title: RunSelect description: Enum for available run columns. ResponseBodyForRunsGenerateQuery: properties: filter: type: string title: Filter feedback_urls: additionalProperties: type: string propertyNames: $ref: '#/components/schemas/RunsGenerateQueryFeedbackKeys' type: object title: Feedback Urls type: object required: - filter - feedback_urls title: ResponseBodyForRunsGenerateQuery QueryFeedbackDeltaBatch: properties: baseline_session_id: type: string format: uuid title: Baseline Session Id comparison_session_ids: items: type: string format: uuid type: array maxItems: 10 minItems: 1 title: Comparison Session Ids feedback_keys: items: type: string type: array maxItems: 100 minItems: 1 title: Feedback Keys filters: anyOf: - additionalProperties: items: type: string type: array propertyNames: format: uuid type: object - type: 'null' title: Filters type: object required: - baseline_session_id - comparison_session_ids - feedback_keys title: QueryFeedbackDeltaBatch description: Request schema for batched feedback delta queries with multiple feedback keys. PublicComparativeExperiment: properties: id: type: string format: uuid title: Id name: anyOf: - type: string - type: 'null' title: Name description: anyOf: - type: string - type: 'null' title: Description created_at: type: string format: date-time title: Created At modified_at: type: string format: date-time title: Modified At extra: anyOf: - additionalProperties: true type: object - type: 'null' title: Extra experiments_info: items: $ref: '#/components/schemas/SimpleExperimentInfo' type: array title: Experiments Info feedback_stats: anyOf: - additionalProperties: true type: object - type: 'null' title: Feedback Stats type: object required: - id - created_at - modified_at - experiments_info title: PublicComparativeExperiment description: Publicly-shared ComparativeExperiment schema. RunDateOrder: type: string enum: - asc - desc title: RunDateOrder description: Enum for run start date order. TracerSession: properties: start_time: type: string format: date-time title: Start Time end_time: anyOf: - type: string format: date-time - type: 'null' title: End Time extra: anyOf: - additionalProperties: true type: object - type: 'null' title: Extra name: type: string title: Name description: anyOf: - type: string - type: 'null' title: Description default_dataset_id: anyOf: - type: string format: uuid - type: 'null' title: Default Dataset Id reference_dataset_id: anyOf: - type: string format: uuid - type: 'null' title: Reference Dataset Id trace_tier: anyOf: - $ref: '#/components/schemas/TraceTier' - type: 'null' id: type: string format: uuid title: Id run_count: anyOf: - type: integer - type: 'null' title: Run Count latency_p50: anyOf: - type: number - type: 'null' title: Latency P50 latency_p99: anyOf: - type: number - type: 'null' title: Latency P99 first_token_p50: anyOf: - type: number - type: 'null' title: First Token P50 first_token_p99: anyOf: - type: number - type: 'null' title: First Token P99 total_tokens: anyOf: - type: integer - type: 'null' title: Total Tokens prompt_tokens: anyOf: - type: integer - type: 'null' title: Prompt Tokens completion_tokens: anyOf: - type: integer - type: 'null' title: Completion Tokens total_cost: anyOf: - type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ - type: 'null' title: Total Cost prompt_cost: anyOf: - type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ - type: 'null' title: Prompt Cost completion_cost: anyOf: - type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ - type: 'null' title: Completion Cost tenant_id: type: string format: uuid title: Tenant Id last_run_start_time: anyOf: - type: string format: date-time - type: 'null' title: Last Run Start Time last_run_start_time_live: anyOf: - type: string format: date-time - type: 'null' title: Last Run Start Time Live feedback_stats: anyOf: - additionalProperties: true type: object - type: 'null' title: Feedback Stats session_feedback_stats: anyOf: - additionalProperties: true type: object - type: 'null' title: Session Feedback Stats run_facets: anyOf: - items: additionalProperties: true type: object type: array - type: 'null' title: Run Facets error_rate: anyOf: - type: number - type: 'null' title: Error Rate streaming_rate: anyOf: - type: number - type: 'null' title: Streaming Rate test_run_number: anyOf: - type: integer - type: 'null' title: Test Run Number type: object required: - id - tenant_id title: TracerSession description: TracerSession schema. BodyParamsForRunSchema: properties: id: anyOf: - items: type: string format: uuid type: array - type: 'null' title: Id trace: anyOf: - type: string format: uuid - type: 'null' title: Trace parent_run: anyOf: - type: string format: uuid - type: 'null' title: Parent Run run_type: anyOf: - $ref: '#/components/schemas/RunTypeEnum' - type: 'null' session: anyOf: - items: type: string format: uuid type: array - type: 'null' title: Session reference_example: anyOf: - items: type: string format: uuid type: array - type: 'null' title: Reference Example execution_order: anyOf: - type: integer maximum: 1.0 minimum: 1.0 - type: 'null' title: Execution Order start_time: anyOf: - type: string format: date-time - type: 'null' title: Start Time end_time: anyOf: - type: string format: date-time - type: 'null' title: End Time error: anyOf: - type: boolean - type: 'null' title: Error query: anyOf: - type: string - type: 'null' title: Query filter: anyOf: - type: string - type: 'null' title: Filter trace_filter: anyOf: - type: string - type: 'null' title: Trace Filter tree_filter: anyOf: - type: string - type: 'null' title: Tree Filter is_root: anyOf: - type: boolean - type: 'null' title: Is Root data_source_type: anyOf: - $ref: '#/components/schemas/RunsFilterDataSourceTypeEnum' - type: 'null' skip_pagination: anyOf: - type: boolean - type: 'null' title: Skip Pagination search_filter: anyOf: - type: string - type: 'null' title: Search Filter use_experimental_search: type: boolean title: Use Experimental Search default: false cursor: anyOf: - type: string - type: 'null' title: Cursor limit: type: integer maximum: 100.0 minimum: 1.0 title: Limit default: 100 select: items: $ref: '#/components/schemas/RunSelect' type: array title: Select default: - id - name - run_type - start_time - end_time - status - error - extra - events - inputs - outputs - parent_run_id - manifest_id - manifest_s3_id - manifest - session_id - serialized - reference_example_id - reference_dataset_id - total_tokens - prompt_tokens - prompt_token_details - completion_tokens - completion_token_details - total_cost - prompt_cost - prompt_cost_details - completion_cost - completion_cost_details - price_model_id - first_token_time - trace_id - dotted_order - last_queued_at - feedback_stats - parent_run_ids - tags - in_dataset - app_path - share_token - trace_tier - trace_first_received_at - ttl_seconds - trace_upgrade - thread_id order: $ref: '#/components/schemas/RunDateOrder' default: desc skip_prev_cursor: type: boolean title: Skip Prev Cursor default: false type: object title: BodyParamsForRunSchema description: Query params for run endpoints. RunStatsGroupBy: properties: attribute: type: string enum: - name - run_type - tag - metadata title: Attribute path: anyOf: - type: string - type: 'null' title: Path max_groups: type: integer title: Max Groups default: 5 type: object required: - attribute title: RunStatsGroupBy description: Group by param for run stats. FeedbackLevel: type: string enum: - run - session title: FeedbackLevel description: Enum for feedback levels. ListPublicDatasetRunsResponse: properties: runs: items: $ref: '#/components/schemas/RunPublicDatasetSchema' type: array title: Runs cursors: additionalProperties: anyOf: - type: string - type: 'null' type: object title: Cursors parsed_query: anyOf: - type: string - type: 'null' title: Parsed Query type: object required: - runs - cursors title: ListPublicDatasetRunsResponse RunPublicSchema: properties: name: type: string title: Name inputs: anyOf: - additionalProperties: true type: object - type: 'null' title: Inputs inputs_preview: anyOf: - type: string - type: 'null' title: Inputs Preview run_type: $ref: '#/components/schemas/RunTypeEnum' start_time: type: string format: date-time title: Start Time end_time: anyOf: - type: string format: date-time - type: 'null' title: End Time extra: anyOf: - additionalProperties: true type: object - type: 'null' title: Extra error: anyOf: - type: string - type: 'null' title: Error execution_order: type: integer minimum: 1.0 title: Execution Order default: 1 serialized: anyOf: - additionalProperties: true type: object - type: 'null' title: Serialized outputs: anyOf: - additionalProperties: true type: object - type: 'null' title: Outputs outputs_preview: anyOf: - type: string - type: 'null' title: Outputs Preview parent_run_id: anyOf: - type: string format: uuid - type: 'null' title: Parent Run Id manifest_id: anyOf: - type: string format: uuid - type: 'null' title: Manifest Id manifest_s3_id: anyOf: - type: string format: uuid - type: 'null' title: Manifest S3 Id events: anyOf: - items: additionalProperties: true type: object type: array - type: 'null' title: Events tags: anyOf: - items: type: string type: array - type: 'null' title: Tags inputs_s3_urls: anyOf: - additionalProperties: true type: object - type: 'null' title: Inputs S3 Urls outputs_s3_urls: anyOf: - additionalProperties: true type: object - type: 'null' title: Outputs S3 Urls s3_urls: anyOf: - additionalProperties: true type: object - type: 'null' title: S3 Urls trace_id: type: string format: uuid title: Trace Id dotted_order: type: string title: Dotted Order trace_min_start_time: anyOf: - type: string format: date-time - type: 'null' title: Trace Min Start Time trace_max_start_time: anyOf: - type: string format: date-time - type: 'null' title: Trace Max Start Time id: type: string format: uuid title: Id status: type: string title: Status child_run_ids: anyOf: - items: type: string format: uuid type: array - type: 'null' title: Child Run Ids direct_child_run_ids: anyOf: - items: type: string format: uuid type: array - type: 'null' title: Direct Child Run Ids parent_run_ids: anyOf: - items: type: string format: uuid type: array - type: 'null' title: Parent Run Ids feedback_stats: anyOf: - additionalProperties: additionalProperties: true type: object type: object - type: 'null' title: Feedback Stats reference_example_id: anyOf: - type: string format: uuid - type: 'null' title: Reference Example Id total_tokens: type: integer title: Total Tokens default: 0 prompt_tokens: type: integer title: Prompt Tokens default: 0 completion_tokens: type: integer title: Completion Tokens default: 0 prompt_token_details: anyOf: - additionalProperties: type: integer type: object - type: 'null' title: Prompt Token Details completion_token_details: anyOf: - additionalProperties: type: integer type: object - type: 'null' title: Completion Token Details total_cost: anyOf: - type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ - type: 'null' title: Total Cost prompt_cost: anyOf: - type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ - type: 'null' title: Prompt Cost completion_cost: anyOf: - type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ - type: 'null' title: Completion Cost prompt_cost_details: anyOf: - additionalProperties: type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ type: object - type: 'null' title: Prompt Cost Details completion_cost_details: anyOf: - additionalProperties: type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ type: object - type: 'null' title: Completion Cost Details price_model_id: anyOf: - type: string format: uuid - type: 'null' title: Price Model Id first_token_time: anyOf: - type: string format: date-time - type: 'null' title: First Token Time messages: anyOf: - items: additionalProperties: true type: object type: array - type: 'null' title: Messages type: object required: - name - run_type - trace_id - dotted_order - id - status title: RunPublicSchema RunStatsQueryParams: properties: id: anyOf: - items: type: string format: uuid type: array - type: 'null' title: Id trace: anyOf: - type: string format: uuid - type: 'null' title: Trace parent_run: anyOf: - type: string format: uuid - type: 'null' title: Parent Run run_type: anyOf: - $ref: '#/components/schemas/RunTypeEnum' - type: 'null' session: anyOf: - items: type: string format: uuid type: array - type: 'null' title: Session reference_example: anyOf: - items: type: string format: uuid type: array - type: 'null' title: Reference Example execution_order: anyOf: - type: integer maximum: 1.0 minimum: 1.0 - type: 'null' title: Execution Order start_time: anyOf: - type: string format: date-time - type: 'null' title: Start Time end_time: anyOf: - type: string format: date-time - type: 'null' title: End Time error: anyOf: - type: boolean - type: 'null' title: Error query: anyOf: - type: string - type: 'null' title: Query filter: anyOf: - type: string - type: 'null' title: Filter trace_filter: anyOf: - type: string - type: 'null' title: Trace Filter tree_filter: anyOf: - type: string - type: 'null' title: Tree Filter is_root: anyOf: - type: boolean - type: 'null' title: Is Root data_source_type: anyOf: - $ref: '#/components/schemas/RunsFilterDataSourceTypeEnum' - type: 'null' skip_pagination: anyOf: - type: boolean - type: 'null' title: Skip Pagination search_filter: anyOf: - type: string - type: 'null' title: Search Filter use_experimental_search: type: boolean title: Use Experimental Search default: false group_by: anyOf: - $ref: '#/components/schemas/RunStatsGroupBy' - type: 'null' groups: anyOf: - items: anyOf: - type: string - type: 'null' type: array - type: 'null' title: Groups select: anyOf: - items: $ref: '#/components/schemas/RunStatsSelect' type: array - type: 'null' title: Select type: object title: RunStatsQueryParams description: Query params for run stats. ExampleWithRunsCH: properties: outputs: anyOf: - additionalProperties: true type: object - type: 'null' title: Outputs dataset_id: type: string format: uuid title: Dataset Id source_run_id: anyOf: - type: string format: uuid - type: 'null' title: Source Run Id metadata: anyOf: - additionalProperties: true type: object - type: 'null' title: Metadata inputs: additionalProperties: true type: object title: Inputs created_at: type: string format: date-time title: Created At id: type: string format: uuid title: Id name: type: string title: Name modified_at: anyOf: - type: string format: date-time - type: 'null' title: Modified At attachment_urls: anyOf: - additionalProperties: true type: object - type: 'null' title: Attachment Urls runs: items: $ref: '#/components/schemas/RunSchemaComparisonView' type: array title: Runs type: object required: - dataset_id - inputs - id - name - runs title: ExampleWithRunsCH description: 'Example schema with list of runs from ClickHouse. For non-grouped endpoint (/datasets/{dataset_id}/runs): runs from single session. For grouped endpoint (/datasets/{dataset_id}/group/runs): flat array of runs from all sessions, where each run has a session_id field for frontend to determine column placement.' HTTPValidationError: properties: detail: items: $ref: '#/components/schemas/ValidationError' type: array title: Detail type: object title: HTTPValidationError TraceTier: type: string enum: - longlived - shortlived title: TraceTier PublicExampleWithRuns: properties: outputs: anyOf: - additionalProperties: true type: object - type: 'null' title: Outputs dataset_id: type: string format: uuid title: Dataset Id source_run_id: anyOf: - type: string format: uuid - type: 'null' title: Source Run Id metadata: anyOf: - additionalProperties: true type: object - type: 'null' title: Metadata inputs: additionalProperties: true type: object title: Inputs created_at: type: string format: date-time title: Created At id: type: string format: uuid title: Id name: type: string title: Name modified_at: anyOf: - type: string format: date-time - type: 'null' title: Modified At attachment_urls: anyOf: - additionalProperties: true type: object - type: 'null' title: Attachment Urls runs: items: $ref: '#/components/schemas/RunPublicDatasetSchema' type: array title: Runs type: object required: - dataset_id - inputs - id - name - runs title: PublicExampleWithRuns description: Schema for an example in a publicly-shared dataset with list of runs. SortByComparativeExperimentColumn: type: string enum: - name - created_at title: SortByComparativeExperimentColumn description: Enum for available comparative experiment columns to sort by. RunStats: properties: run_count: anyOf: - type: integer - type: 'null' title: Run Count latency_p50: anyOf: - type: number - type: 'null' title: Latency P50 latency_p99: anyOf: - type: number - type: 'null' title: Latency P99 first_token_p50: anyOf: - type: number - type: 'null' title: First Token P50 first_token_p99: anyOf: - type: number - type: 'null' title: First Token P99 total_tokens: anyOf: - type: integer - type: 'null' title: Total Tokens prompt_tokens: anyOf: - type: integer - type: 'null' title: Prompt Tokens completion_tokens: anyOf: - type: integer - type: 'null' title: Completion Tokens median_tokens: anyOf: - type: integer - type: 'null' title: Median Tokens completion_tokens_p50: anyOf: - type: integer - type: 'null' title: Completion Tokens P50 prompt_tokens_p50: anyOf: - type: integer - type: 'null' title: Prompt Tokens P50 tokens_p99: anyOf: - type: integer - type: 'null' title: Tokens P99 completion_tokens_p99: anyOf: - type: integer - type: 'null' title: Completion Tokens P99 prompt_tokens_p99: anyOf: - type: integer - type: 'null' title: Prompt Tokens P99 last_run_start_time: anyOf: - type: string format: date-time - type: 'null' title: Last Run Start Time feedback_stats: anyOf: - additionalProperties: true type: object - type: 'null' title: Feedback Stats run_facets: anyOf: - items: additionalProperties: true type: object type: array - type: 'null' title: Run Facets error_rate: anyOf: - type: number - type: 'null' title: Error Rate streaming_rate: anyOf: - type: number - type: 'null' title: Streaming Rate total_cost: anyOf: - type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ - type: 'null' title: Total Cost prompt_cost: anyOf: - type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ - type: 'null' title: Prompt Cost completion_cost: anyOf: - type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ - type: 'null' title: Completion Cost cost_p50: anyOf: - type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ - type: 'null' title: Cost P50 cost_p99: anyOf: - type: string pattern: ^(?!^[-+.]*$)[+-]?0*\d*\.?\d*$ - type: 'null' title: Cost P99 prompt_token_details: anyOf: - additionalProperties: true type: object - type: 'null' title: Prompt Token Details completion_token_details: anyOf: - additionalProperties: true type: object - type: 'null' title: Completion Token Details prompt_cost_details: anyOf: - additionalProperties: true type: object - type: 'null' title: Prompt Cost Details completion_cost_details: anyOf: - additionalProperties: true type: object - type: 'null' title: Completion Cost Details type: object title: RunStats QueryExampleSchemaWithRuns: properties: session_ids: items: type: string format: uuid type: array minItems: 1 title: Session Ids offset: type: integer minimum: 0.0 title: Offset default: 0 limit: type: integer minimum: 1.0 title: Limit default: 10 preview: type: boolean title: Preview default: false comparative_experiment_id: anyOf: - type: string format: uuid - type: 'null' title: Comparative Experiment Id sort_params: anyOf: - $ref: '#/components/schemas/SortParamsForRunsComparisonView' - type: 'null' filters: anyOf: - additionalProperties: items: type: string type: array propertyNames: format: uuid type: object - type: 'null' title: Filters example_ids: anyOf: - items: type: string format: uuid type: array maxItems: 1000 - type: 'null' title: Example Ids include_annotator_detail: type: boolean title: Include Annotator Detail default: false type: object required: - session_ids title: QueryExampleSchemaWithRuns DatasetPublicSchema: properties: name: type: string title: Name description: anyOf: - type: string - type: 'null' title: Description created_at: type: string format: date-time title: Created At inputs_schema_definition: anyOf: - additionalProperties: true type: object - type: 'null' title: Inputs Schema Definition outputs_schema_definition: anyOf: - additionalProperties: true type: object - type: 'null' title: Outputs Schema Definition externally_managed: anyOf: - type: boolean - type: 'null' title: Externally Managed default: false transformations: anyOf: - items: $ref: '#/components/schemas/DatasetTransformation' type: array - type: 'null' title: Transformations data_type: anyOf: - $ref: '#/components/schemas/DataType' - type: 'null' default: kv id: type: string format: uuid title: Id example_count: type: integer title: Example Count type: object required: - name - id - example_count title: DatasetPublicSchema description: 'Public schema for datasets. Doesn''t currently include session counts/stats since public test project sharing is not yet shipped' DatasetTransformation: properties: path: items: type: string type: array title: Path transformation_type: $ref: '#/components/schemas/DatasetTransformationType' type: object required: - path - transformation_type title: DatasetTransformation QueryParamsForPublicRunSchema: properties: id: anyOf: - items: type: string format: uuid type: array - type: 'null' title: Id type: object title: QueryParamsForPublicRunSchema description: Query params for public run endpoints. SortParamsForRunsComparisonView: properties: sort_by: type: string title: Sort By sort_order: type: string enum: - ASC - DESC title: Sort Order default: DESC type: object required: - sort_by title: SortParamsForRunsComparisonView DatasetTransformationType: type: string enum: - convert_to_openai_message - convert_to_openai_tool - remove_system_messages - remove_extra_fields - extract_tools_from_run title: DatasetTransformationType description: 'Enum for dataset transformation types. Ordering determines the order in which transformations are applied if there are multiple transformations on the same path.' RunStatsSelect: type: string enum: - run_count - latency_p50 - latency_p99 - latency_avg - first_token_p50 - first_token_p99 - total_tokens - prompt_tokens - completion_tokens - median_tokens - completion_tokens_p50 - prompt_tokens_p50 - tokens_p99 - completion_tokens_p99 - prompt_tokens_p99 - last_run_start_time - feedback_stats - thread_feedback_stats - run_facets - error_rate - streaming_rate - total_cost - prompt_cost - completion_cost - cost_p50 - cost_p99 - session_feedback_stats - all_run_stats - all_token_stats - prompt_token_details - completion_token_details - prompt_cost_details - completion_cost_details title: RunStatsSelect description: Metrics you can select from run stats endpoint. ListPublicRunsResponse: properties: runs: items: $ref: '#/components/schemas/RunPublicSchema' type: array title: Runs cursors: additionalProperties: anyOf: - type: string - type: 'null' type: object title: Cursors parsed_query: anyOf: - type: string - type: 'null' title: Parsed Query type: object required: - runs - cursors title: ListPublicRunsResponse FeedbackSchema: properties: created_at: type: string format: date-time title: Created At modified_at: type: string format: date-time title: Modified At key: type: string title: Key score: anyOf: - type: number - type: integer - type: boolean - type: 'null' title: Score value: anyOf: - type: number - type: integer - type: boolean - type: string - additionalProperties: true type: object - type: 'null' title: Value comment: anyOf: - type: string - type: 'null' title: Comment correction: anyOf: - additionalProperties: true type: object - type: string - type: 'null' title: Correction feedback_group_id: anyOf: - type: string format: uuid - type: 'null' title: Feedback Group Id comparative_experiment_id: anyOf: - type: string format: uuid - type: 'null' title: Comparative Experiment Id run_id: anyOf: - type: string format: uuid - type: 'null' title: Run Id session_id: anyOf: - type: string format: uuid - type: 'null' title: Session Id id: type: string format: uuid title: Id trace_id: anyOf: - type: string format: uuid - type: 'null' title: Trace Id start_time: anyOf: - type: string format: date-time - type: 'null' title: Start Time is_root: type: boolean title: Is Root default: false feedback_source: anyOf: - $ref: '#/components/schemas/FeedbackSource' - type: 'null' extra: anyOf: - additionalProperties: true type: object - type: 'null' title: Extra feedback_thread_id: anyOf: - type: string - type: 'null' title: Feedback Thread Id type: object required: - key - id title: FeedbackSchema description: Schema for getting feedback. securitySchemes: API Key: type: apiKey in: header name: X-API-Key Tenant ID: type: apiKey in: header name: X-Tenant-Id Bearer Auth: type: http description: Bearer tokens are used to authenticate from the UI. Must also specify x-tenant-id or x-organization-id (for org scoped apis). scheme: bearer Organization ID: type: apiKey in: header name: X-Organization-Id