openapi: 3.1.0 # --------------------------- info: title: SemaDB API summary: No fuss vector database for AI description: >- This API provides a way to interact with a vector database, which is a type of database that stores data as high-dimensional vectors. Vectors are mathematical representations of features or attributes, and each vector has a certain number of dimensions, which can range from tens to thousands. Vector databases are used for a variety of tasks, such as similarity search, recommendation systems, and anomaly detection. They are particularly well-suited for tasks involving unstructured data, such as text, images, and audio. version: 2.0.0 termsOfService: https://www.semafind.com/termsofservice contact: name: Semafind url: https://www.semafind.com/contact email: support@semafind.com license: name: Apache 2.0 url: https://www.apache.org/licenses/LICENSE-2.0.html # --------------------------- servers: - url: https://semadb.semafind.com/v2 description: SemaDB Cloud Public Beta # --------------------------- tags: - name: Collection description: >- A collection is a group of points that share the same search space. For example, a collection could be a group of images, where each image is represented as a vector. The collection would be the search space for similarity search, and each image would be a point in that search space. - name: Point description: >- A point is a vector with a unique identifier. It can also contain any JSON serialisable metadata such as an external ID, document title or image URL. # --------------------------- paths: /collections: summary: Endpoint for managing collections description: >- This endpoint covers operations for collections as a whole, such as creating a new collection or listing all collections. post: tags: - Collection summary: Create a new collection description: >- Creates a new collection if it does not already exist. The maximum number of collections per user is restricted based on the plan. Before you can insert and search points, you must create a collection. operationId: CreateCollection requestBody: description: The collection to create required: true content: application/json: schema: $ref: '#/components/schemas/CreateCollectionRequest' examples: SampleCreateCollection: summary: Sample create collection description: A sample collection to create with id mycollection, vector size 2 and euclidean distance. value: id: mycollection indexSchema: vector: type: vectorVamana vectorVamana: vectorSize: 2 distanceMetric: euclidean searchSize: 75 degreeBound: 64 alpha: 1.2 responses: '200': $ref: '#/components/responses/SuccessfulMessageResponse' description: The collection was created '403': $ref: '#/components/responses/ErrorMessageResponse' description: The maximum collecton quota may be reached '409': $ref: '#/components/responses/ErrorMessageResponse' description: The collection already exists get: tags: - Collection summary: List user collections description: >- Returns a list of all collections for the current user. The list is not sorted by any value and the order may change between requests. operationId: ListCollection responses: '200': description: The list of collections content: application/json: schema: $ref: '#/components/schemas/ListCollectionResponse' examples: SampleCollectionList: summary: Sample collection list description: A sample collection list with one collection value: collections: - id: mycollection - id: anothercollection # --------------------------- /collections/{collectionId}: summary: Endpoint for managing a specific collection description: >- These endpoints interact with a single collection. The collection must exist before you can use these endpoints. parameters: - $ref: '#/components/parameters/CollectionId' get: tags: - Collection summary: Get the details of a collection description: >- This endpoint attempts to also list the shards currently available in the collection. Some shards may be temporarily unavailable. In that case, you can retry at a future time. operationId: GetCollection responses: '200': description: Collection with available shard details content: application/json: schema: $ref: '#/components/schemas/GetCollectionResponse' examples: SampleCollection: summary: Sample collection description: A sample collection with shard information value: id: mycollection indexSchema: vector: type: vectorVamana vectorVamana: vectorSize: 2 distanceMetric: euclidean searchSize: 75 degreeBound: 64 alpha: 1.2 shards: - id: fff3a226-b9f8-4375-8dbd-1a240e000705 pointCount: 42 '503': $ref: '#/components/responses/ErrorMessageResponse' description: Some downstream components may be temporarily unavailable delete: tags: - Collection summary: Delete a collection description: >- Deletes a collection and all of its points. This operation is irreversible. If you want to delete only some points, use the bulk delete endpoint. If some shards are temporarily unavailable, the operation will still succeed, but some of the data will be deleted in the future. operationId: DeleteCollection responses: '200': $ref: '#/components/responses/SuccessfulMessageResponse' description: The collection and all of its data was deleted '202': $ref: '#/components/responses/SuccessfulMessageResponse' description: The collection was deleted, but some of the data will be deleted in the future # --------------------------- /collections/{collectionId}/points: summary: Endpoint for bulk managing points in a collection description: >- The bulk points endpoint is more efficient and operates on multiple points at once. It is recommended to use this endpoint for inserting, updating and deleting points. parameters: - $ref: '#/components/parameters/CollectionId' post: tags: - Point summary: Insert new points into the collection description: >- This endpoint assumes all points to be inserted are new points and does not check for duplication. It is important to ensure consistency of the database you do not insert duplicate points. If you are unsure if a point exists, you can leave the id field blank and the database will assign a new id. *For cosine distance, you must normalise the vectors prior to inserting them.* operationId: InsertPoint requestBody: description: Points to insert required: true content: application/json: schema: $ref: '#/components/schemas/InsertPointsRequest' examples: SampleInsertPoints: summary: Sample insert points description: A sample list of points to insert value: points: - vector: [4.2, 2.4] - vector: [1.2, 3.4] foo: bar externalId: 42 - _id: 3fa85f64-5717-4562-b3fc-2c963f66afa6 vector: [3.4, 2.4] externalId: 43 responses: '200': description: >- The operation was successful but you must check the failedRanges to ensure all the points were inserted. If some points were not inserted, you can retry the operation by using the failedRanges depending on the error. For example, if the failedRanges is [[0, 2]], you can retry the operation by sending the first two points again. content: application/json: schema: $ref: '#/components/schemas/InsertPointsResponse' examples: SuccessfulInsertPointsResponse: summary: Fully successful insert points response description: A sample response indicating full success value: message: success failedRanges: [] SamplePartialInsertPointsResponse: summary: Partial success insert points response description: A sample response indicating partial success value: message: partial success failedRanges: - shardId: fff3a226-b9f8-4375-8dbd-1a240e000705 start: 0 end: 2 error: point already exists '403': $ref: '#/components/responses/ErrorMessageResponse' description: Maximum point quota per collection may be reached '503': $ref: '#/components/responses/ErrorMessageResponse' description: Some downstream components may be temporarily unavailable put: tags: - Point summary: Update existing points with new data description: >- This endpoint allows updating point vectors and metadata. It does not allow updating the point id. If you want to update the id, you must delete the point and insert a new point. The points are required to exist before you can update them. You can check the failedPoints to see which points failed to update and potentially why. operationId: UpdatePoint requestBody: description: Points to update required: true content: application/json: schema: $ref: '#/components/schemas/UpdatePointsRequest' examples: SampleUpdatePoints: summary: Sample update points description: A sample list of points to update value: points: - _id: 3fa85f64-5717-4562-b3fc-2c963f66afa6 vector: [4.2, 2.4] wizard: harry responses: '200': description: Updated points content: application/json: schema: $ref: '#/components/schemas/UpdatePointsResponse' examples: SuccessfulUpdatePointsResponse: summary: Successful update points response description: A sample response indicating full success value: message: success failedPoints: [] SamplePartialUpdatePointsResponse: summary: Partial update points response description: A sample response indicating partial success value: message: partial success failedPoints: - id: 3fa85f64-5717-4562-b3fc-2c963f66afa6 error: not found delete: tags: - Point summary: Delete points by id description: >- Bulk delete points based on id. This endpoint does not check if the points exist. If you attempt to delete a point that does not exist, it will be ignored and included in the failedPoints list. operationId: DeletePoint requestBody: description: Point IDs to delete required: true content: application/json: schema: $ref: '#/components/schemas/DeletePointsRequest' examples: SampleDeletePoints: summary: Sample delete points description: A sample list of point ids to delete value: ids: - 3fa85f64-5717-4562-b3fc-2c963f66afa6 responses: '200': description: Deleted points content: application/json: schema: $ref: '#/components/schemas/DeletePointsResponse' examples: SuccessfulDeletePointsResponse: summary: Successful delete points response description: A sample response indicating full success value: message: success failedPoints: [] SamplePartialDeletePointsResponse: summary: Partial delete points response description: A sample response indicating partial success value: message: partial success failedPoints: - id: 3fa85f64-5717-4562-b3fc-2c963f66afa6 error: not found # --------------------------- /collections/{collectionId}/points/search: summary: Search points description: >- This endpoint allows searching for points in a collection. parameters: - $ref: '#/components/parameters/CollectionId' post: tags: - Point summary: Fast index based search description: >- This endpoint allows searching for points in a collection using the index. The search is based on the index schema of the collection. operationId: SearchPoint requestBody: description: Search request required: true content: application/json: schema: $ref: '#/components/schemas/SearchRequest' examples: SampleSearchPoint: summary: Sample search point description: A sample vector to search with default limit value: query: property: vector vectorVamana: vector: [4.2, 2.4] operator: near searchSize: 75 limit: 10 limit: 10 responses: '200': description: Search results ordered by relevance content: application/json: schema: $ref: '#/components/schemas/SearchPointsResponse' examples: SampleSearchPointResponse: summary: Sample search point response description: A sample response with two points value: points: - _id: "faefe2b1-cf85-48db-9621-94b833ee9cc9" _distance: 0 _hybridScore: -0, description: "A product" price: 100 - _id: "d2e3ebb5-149a-46e4-8b3f-cd6c38530da3" _distance: 314402.94, _hybridScore: -314402.94 description: "Another product" price: 200 # --------------------------- components: parameters: CollectionId: name: collectionId in: path description: The unique identifier of the collection required: true example: mycollection schema: $ref: '#/components/schemas/CollectionId' responses: SuccessfulMessageResponse: description: Operation was successful content: application/json: schema: type: object required: - message properties: message: type: string description: A message indicating the result of the operation examples: - message: Operation successful examples: Success: summary: A successful response description: The operation was successful value: message: success ErrorMessageResponse: description: Something unexpected happened content: application/json: schema: type: object required: - error properties: error: type: string description: An error message hopefully describing the problem examples: GeneralError: summary: General error description: Something unexpected happened value: error: Something went wrong schemas: # --------------------------- # Common objects CollectionId: type: string title: Collection Id description: The unique identifier of the collection pattern: "^[a-z0-9]{3,24}$" VectorSize: type: number description: The size of the vectors in the collection minimum: 1 maximum: 4096 DistanceMetric: type: string enum: - euclidean - cosine - dot - hamming - jaccard - haversine Vector: type: array description: A vector with a fixed number of dimensions minItems: 1 maxItems: 4096 items: type: number PointAsObject: type: object description: JSON serialisable point data properties: _id: type: string format: uuid FailedPoints: type: array description: >- A list of points that failed to insert. Each point has an id and an error message. For example, if the error is "not found", the point does not exist in the collection. items: type: object properties: id: type: string format: uuid error: type: string # --------------------------- # Collection endpoint objects CreateCollectionRequest: type: object properties: id: $ref: '#/components/schemas/CollectionId' indexSchema: $ref: '#/components/schemas/IndexSchema' ListCollectionResponse: type: object properties: collections: type: array items: type: object properties: id: $ref: '#/components/schemas/CollectionId' GetCollectionResponse: type: object properties: id: $ref: '#/components/schemas/CollectionId' indexSchema: $ref: '#/components/schemas/IndexSchema' shards: type: array items: type: object properties: id: type: string format: uuid pointCount: type: number # --------------------------- # Points endpoint objects InsertPointsRequest: type: object required: [points] properties: points: type: array maxItems: 10000 items: $ref: '#/components/schemas/PointAsObject' InsertPointsResponse: type: object properties: message: type: string description: A message indicating the result of the operation failedRanges: type: array description: >- A list of ranges of points that failed to insert. Each range has a start and an end index. The end index is exclusive. For example, if the range is [0, 2], the first two points failed to insert. items: type: object properties: shardId: type: string format: uuid start: type: integer end: type: integer error: type: string UpdatePointsRequest: type: object required: [points] properties: points: type: array maxItems: 100 items: $ref: '#/components/schemas/PointAsObject' UpdatePointsResponse: type: object properties: message: type: string description: A message indicating the result of the operation failedPoints: $ref: '#/components/schemas/FailedPoints' DeletePointsRequest: type: object required: [ids] properties: ids: type: array maxItems: 100 items: type: string format: uuid DeletePointsResponse: type: object properties: message: type: string description: A message indicating the result of the operation failedPoints: $ref: '#/components/schemas/FailedPoints' # --------------------------- # Search objects SearchPointsResponse: type: object properties: points: type: array items: $ref: '#/components/schemas/PointAsObject' SearchRequest: type: object required: [query, limit] properties: query: $ref: '#/components/schemas/Query' select: type: array description: >- A list of properties to return in the search results. If not provided, all properties are returned. items: type: string sort: type: array description: >- A list of sort options for the search results. The search results are sorted by the first sort option, then the second, and so on. maxItems: 10 items: $ref: '#/components/schemas/SortOption' offset: type: number description: The number of points to skip in the search results minimum: 0 default: 0 limit: type: number description: Maximum number of points to return minimum: 1 maximum: 100 default: 10 Query: type: object description: >- A query object that can be used to perform search. The query object can contain multiple filters, each with a property and a value. Use _and and _or to combine queries. required: [property] properties: property: type: string vectorFlat: $ref: '#/components/schemas/SearchVectorFlatOptions' vectorVamana: $ref: '#/components/schemas/SearchVectorVamanaOptions' text: $ref: '#/components/schemas/SearchTextOptions' string: $ref: '#/components/schemas/SearchStringOptions' integer: $ref: '#/components/schemas/SearchNumberOptions' float: $ref: '#/components/schemas/SearchNumberOptions' stringArray: $ref: '#/components/schemas/SearchStringArrayOptions' _and: type: array items: $ref: '#/components/schemas/Query' _or: type: array items: $ref: '#/components/schemas/Query' SortOption: type: object description: >- Sort options for search results. The field is the property to sort by and the order is the direction to sort in. required: [property, descending] properties: property: type: string descending: type: boolean default: false SearchVectorVamanaOptions: type: object description: >- Options for searching vectors with Vamana indexing. The larger the search size the longer the search will take. required: [vector, operator, searchSize, limit] properties: vector: $ref: '#/components/schemas/Vector' operator: type: string enum: [near] searchSize: type: number description: >- Determines the scope of the greedy search algorithm. The higher the value, the more exhaustive the search. minimum: 25 maximum: 75 default: 75 limit: type: number description: Maximum number of points to search minimum: 1 maximum: 75 default: 10 filter: $ref: '#/components/schemas/Query' weight: type: number description: >- The weight of the vector search, the higher the value, the more important the vector search is. default: 1 SearchVectorFlatOptions: type: object description: >- Options for searching vectors with flat indexing. required: [vector, operator, limit] properties: vector: $ref: '#/components/schemas/Vector' operator: type: string enum: [near] limit: type: number description: Maximum number of points to search minimum: 1 maximum: 75 default: 10 filter: $ref: '#/components/schemas/Query' weight: type: number description: >- The weight of the vector search, the higher the value, the more important the vector search is. default: 1 SearchTextOptions: type: object description: >- Text search options, the value is the text to search for. The weight determines the hybrid search weighting. required: [value, operator, limit] properties: value: type: string operator: type: string enum: [containsAll, containsAny] limit: type: number description: Maximum number of points to search minimum: 1 maximum: 75 default: 10 filter: $ref: '#/components/schemas/Query' weight: type: number description: >- The weight of the text search, the higher the value, the more important the text search is. default: 1 SearchStringOptions: type: object description: >- Options for searching strings. The operator determines how the search is performed. The value is a string to search for. required: [value, operator] properties: value: type: string operator: type: string enum: [startsWith, equals, notEquals, greaterThan, greaterThanOrEquals, lessThan, lessThanOrEquals, inRange] endValue: type: string SearchNumberOptions: type: object description: >- Options for searching numbers. The operator determines how the search is performed. The value is a number to search for, endValue is used for range queries. required: [value, operator] properties: value: type: number operator: type: string enum: [equals, notEquals, greaterThan, greaterThanOrEquals, lessThan, lessThanOrEquals, inRange] endValue: type: number SearchStringArrayOptions: type: object description: >- Options for searching string arrays. The operator determines how the search is performed. The value is an array of strings to search for. required: [value, operator] properties: value: type: array items: type: string operator: type: string enum: [containsAll, containsAny] # --------------------------- # Index schema objects IndexSchema: type: object description: >- The schema for the collection, each property can be indexed with a different type of index. minProperties: 1 additionalProperties: $ref: '#/components/schemas/IndexSchemaValue' IndexSchemaValue: type: object description: >- Defines what the property is and how it is indexed. properties: type: type: string enum: [vectorFlat, vectorVamana, text, string, stringArray, integer, float] vectorFlat: $ref: '#/components/schemas/IndexVectorFlatParameters' vectorVamana: $ref: '#/components/schemas/IndexVectorVamanaParameters' text: $ref: '#/components/schemas/IndexTextParameters' string: $ref: '#/components/schemas/IndexStringParameters' stringArray: $ref: '#/components/schemas/IndexStringArrayParameters' IndexVectorFlatParameters: type: object description: >- Parameters for flat indexing. Flat indexing is the simplest form of indexing, where the search is exhaustive. required: [vectorSize, distanceMetric] properties: vectorSize: $ref: '#/components/schemas/VectorSize' distanceMetric: $ref: '#/components/schemas/DistanceMetric' quantizer: $ref: '#/components/schemas/Quantizer' IndexVectorVamanaParameters: type: object description: Parameters for Vamana indexing required: [vectorSize, distanceMetric, searchSize, degreeBound, alpha] properties: vectorSize: $ref: '#/components/schemas/VectorSize' distanceMetric: $ref: '#/components/schemas/DistanceMetric' searchSize: type: number description: >- Determines the scope of the greedy search algorithm. The higher the value, the more exhaustive the search. minimum: 25 maximum: 75 default: 75 degreeBound: type: number description: >- Maximum number of edges of a node in the graph. The higher the value, the denser the graph becomes, slower the search but more accurate. minimum: 32 maximum: 64 default: 64 alpha: type: number description: >- Determines how aggressive the edge pruning is. Higher values reduce pruning, lower values make it more aggressive. minimum: 1.1 maximum: 1.5 default: 1.2 quantizer: $ref: '#/components/schemas/Quantizer' IndexTextParameters: type: object description: Parameters for text indexing required: [analyser] properties: analyser: type: string enum: [standard] default: standard IndexStringParameters: type: object description: Parameters for string indexing required: [caseSensitive] properties: caseSensitive: type: boolean description: Whether the string is case sensitive default: false IndexStringArrayParameters: $ref: '#/components/schemas/IndexStringParameters' # --------------------------- # Quantizer objects Quantizer: type: object description: Applied quantizer to the vectors if any required: [type] properties: type: type: string enum: [none, binary, product] binary: $ref: '#/components/schemas/BinaryQuantizerParameters' product: $ref: '#/components/schemas/ProductQuantizerParameters' BinaryQuantizerParameters: type: object description: >- Converts vectors to boolean values. Works effectively only if vector features are already binary or are normally distributed. required: [distanceMetric] minProperties: 2 properties: threshold: type: number description: Optional initial threshold for binary quantization, if not provided, it will be calculated at trigger threshold. triggerThreshold: type: number description: Optional trigger threshold for binary quantization. minimum: 0 maximum: 50000 default: 10000 distanceMetric: type: string description: The distance metric to use for binary quantization after the vectors are encoded enum: [hamming, jaccard] ProductQuantizerParameters: type: object description: >- Uses the product quantization to reduce the memory footprint of the vectors. It may be slower and less accurate. required: [numCentroids, numSubVectors, triggerThreshold] properties: numCentroids: type: number description: >- Number of centroids to quantize to, this is the k* parameter in the paper and is often set to 255 giving 256 centroids (including 0). We are limiting this to maximum of 256 (uint8) to keep the overhead of this process tractable. minimum: 2 maximum: 256 default: 256 numSubVectors: type: number description: >- Number of subvectors / segments / subquantizers to use, this is the m parameter in the paper and is often set to 8. minimum: 2 triggerThreshold: type: number description: >- The trigger threshold is the number of points in the collection that will trigger the quantization process. This is to ensure that the quantization process is only triggered when the collection is large enough to benefit from the memory savings. minimum: 1000 maximum: 10000 default: 10000