openapi: 3.0.0 info: title: Edge Impulse Learn API version: 1.0.0 servers: - url: https://studio.edgeimpulse.com/v1 security: - ApiKeyAuthentication: [] - JWTAuthentication: [] - JWTHttpHeaderAuthentication: [] tags: - name: Learn paths: /api/{projectId}/training/{learnId}/x: get: summary: Download data description: Download the processed data for this learning block. This is data already processed by the signal processing blocks. operationId: getLearnXData tags: - Learn parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/LearnIdParameter' responses: '200': description: Numpy binary file content: application/octet-stream: schema: type: string format: binary /api/{projectId}/training/{learnId}/y: get: summary: Download labels description: Download the labels for this learning block. This is data already processed by the signal processing blocks. Not all blocks support this function. If so, a GenericApiResponse is returned with an error message. operationId: getLearnYData tags: - Learn parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/LearnIdParameter' responses: '200': description: Numpy binary file content: application/octet-stream: schema: type: string format: binary /api/{projectId}/training/anomaly/{learnId}: get: summary: Anomaly information description: Get information about an anomaly block, such as its dependencies. Use the impulse blocks to find the learnId. operationId: getAnomaly tags: - Learn parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/LearnIdParameter' responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/AnomalyConfigResponse' post: summary: Anomaly settings description: Configure the anomaly block, such as its minimum confidence score. Use the impulse blocks to find the learnId. operationId: setAnomaly tags: - Learn parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/LearnIdParameter' requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/SetAnomalyParameterRequest' responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/GenericApiResponse' /api/{projectId}/training/anomaly/{learnId}/metadata: get: summary: Anomaly metadata description: Get metadata about a trained anomaly block. Use the impulse blocks to find the learnId. operationId: getAnomalyMetadata tags: - Learn x-middleware: - AllowsReadOnly parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/LearnIdParameter' responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/AnomalyModelMetadataResponse' /api/{projectId}/training/anomaly/{learnId}/gmm/metadata: get: summary: Anomaly GMM metadata description: Get raw model metadata of the Gaussian mixture model (GMM) for a trained anomaly block. Use the impulse blocks to find the learnId. operationId: getGmmMetadata tags: - Learn x-middleware: - AllowsReadOnly parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/LearnIdParameter' responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/AnomalyGmmMetadataResponse' /api/{projectId}/training/keras/{learnId}: get: summary: Keras information description: Get information about a Keras block, such as its dependencies. Use the impulse blocks to find the learnId. operationId: getKeras tags: - Learn parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/LearnIdParameter' responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/KerasResponse' post: summary: Keras settings description: Configure the Keras block, such as its minimum confidence score. Use the impulse blocks to find the learnId. operationId: setKeras tags: - Learn parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/LearnIdParameter' requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/SetKerasParameterRequest' responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/GenericApiResponse' /api/{projectId}/training/keras/{learnId}/metadata: get: summary: Keras metadata description: Get metadata about a trained Keras block. Use the impulse blocks to find the learnId. operationId: getKerasMetadata tags: - Learn x-middleware: - AllowsReadOnly parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/LearnIdParameter' - $ref: '#/components/parameters/ExcludeLabelsParameter' responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/KerasModelMetadataResponse' /api/{projectId}/training/keras/{learnId}/data-explorer/features: get: summary: Get data explorer features description: t-SNE2 output of the raw dataset using embeddings from this Keras block operationId: getKerasDataExplorerFeatures tags: - Learn x-middleware: - AllowsReadOnly parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/LearnIdParameter' responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/GetDataExplorerFeaturesResponse' /api/{projectId}/training/keras/{learnId}/files: post: summary: Upload Keras files description: Replace Keras block files with the contents of a zip. This is an internal API. x-internal-api: true security: - permissions: - projects:training:keras:write operationId: uploadKerasFiles tags: - Learn parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/LearnIdParameter' requestBody: required: true content: multipart/form-data: schema: $ref: '#/components/schemas/UploadKerasFilesRequest' responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/GenericApiResponse' /api/{projectId}/training/keras/{learnId}/addFiles: post: summary: Add Keras files description: Add Keras block files with the contents of a zip. This is an internal API. x-internal-api: true security: - permissions: - projects:training:keras:write operationId: addKerasFiles tags: - Learn parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/LearnIdParameter' requestBody: required: true content: multipart/form-data: schema: $ref: '#/components/schemas/AddKerasFilesRequest' responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/GenericApiResponse' /api/{projectId}/training/keras/{learnId}/download-export: get: summary: Download Keras export description: Download an exported Keras block - needs to be exported via 'exportKerasBlock' first operationId: downloadKerasExport tags: - Learn parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/LearnIdParameter' responses: '200': description: File content: application/zip: schema: type: string format: binary /api/{projectId}/training/keras/{learnId}/download-data: get: summary: Download Keras data export description: Download the data of an exported Keras block - needs to be exported via 'exportKerasBlockData' first operationId: downloadKerasData tags: - Learn parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/LearnIdParameter' responses: '200': description: File content: application/zip: schema: type: string format: binary /api/{projectId}/learn-data/{learnId}/model/{modelDownloadId}: get: summary: Download trained model description: Download a trained model for a learning block. Depending on the block this can be a TensorFlow model, or the cluster centroids. operationId: downloadLearnModel tags: - Learn x-middleware: - AllowsReadOnly parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/LearnIdParameter' - $ref: '#/components/parameters/ModelDownloadIdParameter' responses: '200': description: File content: application/octet-stream: schema: type: string format: binary /api/{projectId}/training/anomaly/{learnId}/features/get-graph: get: summary: Trained features description: Get a sample of trained features, this extracts a number of samples and their features. operationId: anomalyTrainedFeatures tags: - Learn x-middleware: - AllowsReadOnly parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/LearnIdParameter' - $ref: '#/components/parameters/FeatureAx1Parameter' - $ref: '#/components/parameters/FeatureAx2Parameter' responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/AnomalyTrainedFeaturesResponse' /api/{projectId}/training/anomaly/{learnId}/features/get-graph/classification/{sampleId}: get: summary: Trained features for sample description: Get trained features for a single sample. This runs both the DSP prerequisites and the anomaly classifier. operationId: anomalyTrainedFeaturesPerSample tags: - Learn x-middleware: - AllowsReadOnly parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/LearnIdParameter' - $ref: '#/components/parameters/SampleIdParameter' responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/AnomalyTrainedFeaturesResponse' /api/{projectId}/pretrained-model: get: summary: Get pretrained model description: Receive info back about the earlier uploaded pretrained model (via `uploadPretrainedModel`) input/output tensors. If you want to deploy a pretrained model from the API, see `startDeployPretrainedModelJob`. operationId: getPretrainedModelInfo tags: - Learn parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/OptionalImpulseIdParameter' responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/GetPretrainedModelResponse' /api/{projectId}/pretrained-model/upload: post: summary: Upload a pretrained model description: Upload a pretrained model and receive info back about the input/output tensors. If you want to deploy a pretrained model from the API, see `startDeployPretrainedModelJob`. operationId: uploadPretrainedModel tags: - Learn parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/OptionalImpulseIdParameter' requestBody: required: true content: multipart/form-data: schema: $ref: '#/components/schemas/UploadPretrainedModelRequest' responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/StartJobResponse' /api/{projectId}/pretrained-model/save: post: summary: Save parameters for pretrained model description: Save input / model configuration for a pretrained model. This overrides the current impulse. If you want to deploy a pretrained model from the API, see `startDeployPretrainedModelJob`. operationId: savePretrainedModelParameters tags: - Learn parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/OptionalImpulseIdParameter' requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/SavePretrainedModelRequest' responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/GenericApiResponse' /api/{projectId}/pretrained-model/test: post: summary: Test pretrained model description: Test out a pretrained model (using raw features) - upload first via `uploadPretrainedModel`. If you want to deploy a pretrained model from the API, see `startDeployPretrainedModelJob`. operationId: testPretrainedModel tags: - Learn x-middleware: - AllowsReadOnly parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/OptionalImpulseIdParameter' requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/TestPretrainedModelRequest' responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/TestPretrainedModelResponse' /api/{projectId}/pretrained-model/profile: post: summary: Profile pretrained model description: Returns the latency, RAM and ROM used for the pretrained model - upload first via `uploadPretrainedModel`. This is using the project's selected latency device. Updates are streamed over the websocket API (or can be retrieved through the /stdout endpoint). Use getProfileTfliteJobResult to get the results when the job is completed. tags: - Learn parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/OptionalImpulseIdParameter' operationId: profilePretrainedModel responses: '200': description: OK content: application/json: schema: $ref: '#/components/schemas/StartJobResponse' /api/{projectId}/pretrained-model/download/{pretrainedModelDownloadType}: get: summary: Download pretrained model description: Download a pretrained model file operationId: downloadPretrainedModel tags: - Learn parameters: - $ref: '#/components/parameters/ProjectIdParameter' - $ref: '#/components/parameters/PretrainedModelDownloadParameter' - $ref: '#/components/parameters/OptionalImpulseIdParameter' responses: '200': description: File content: application/octet-stream: schema: type: string format: binary components: schemas: BlockDisplayCategory: description: Category to display this block in the UI. type: string enum: - classical - tao KerasModelVariantEnum: type: string enum: - int8 - float32 - akida AnomalyGmmMetadata: type: object required: - means - covariances - weights properties: means: type: array items: type: array items: type: number description: 2D array of shape (n, m) covariances: type: array items: type: array items: type: array items: type: number description: 3D array of shape (n, m, m) weights: type: array items: type: number description: 1D array of shape (n,) ProfileModelTableMcu: type: object required: - description - supported properties: description: type: string timePerInferenceMs: type: integer memory: type: object properties: tflite: type: object required: - ram - rom properties: ram: type: integer rom: type: integer eon: type: object required: - ram - rom properties: ram: type: integer rom: type: integer eonRamOptimized: type: object required: - ram - rom properties: ram: type: integer rom: type: integer supported: type: boolean mcuSupportError: type: string BlockType: type: string enum: - official - personal - enterprise - pro-or-enterprise - community StartJobResponse: allOf: - $ref: '#/components/schemas/GenericApiResponse' - type: object required: - id properties: id: type: integer description: Job identifier. Status updates will include this identifier. example: 12873488112 DeployPretrainedModelInputImage: type: object required: - inputType properties: inputType: type: string enum: - image inputScaling: $ref: '#/components/schemas/ImageInputScaling' ImpulseInputBlock: type: object required: - id - type - name - title - primaryVersion properties: id: type: integer minimum: 1 description: Identifier for this block. Make sure to up this number when creating a new block, and don't re-use identifiers. If the block hasn't changed, keep the ID as-is. ID must be unique across the project and greather than zero (>0). type: type: string description: Block type (either time-series, image or features) example: time-series enum: - time-series - image - features name: type: string description: Block name, will be used in menus example: Time series title: type: string description: Block title, used in the impulse UI example: Time series windowSizeMs: type: integer description: Size of the sliding window in milliseconds example: 2004 windowIncreaseMs: type: integer description: We use a sliding window to go over the raw data. How many milliseconds to increase the sliding window with for each step. frequencyHz: type: number description: (Input only) Frequency of the input data in Hz example: 60 classificationWindowIncreaseMs: type: integer description: We use a sliding window to go over the raw data. How many milliseconds to increase the sliding window with for each step in classification mode. padZeros: type: boolean description: Whether to zero pad data when a data item is too short imageWidth: type: integer description: Width all images are resized to before training example: 28 imageHeight: type: integer description: Width all images are resized to before training example: 28 resizeMode: type: string description: How to resize images before training example: squash enum: - squash - fit-short - fit-long - crop resizeMethod: type: string description: Resize method to use when resizing images example: squash enum: - lanczos3 - nearest cropAnchor: type: string description: If images are resized using a crop, choose where to anchor the crop example: middle-center enum: - top-left - top-center - top-right - middle-left - middle-center - middle-right - bottom-left - bottom-center - bottom-right description: type: string description: A short description of the block version, displayed in the block versioning UI example: Reduced learning rate and more layers createdBy: type: string description: The system component that created the block version (createImpulse | clone | tuner). Cannot be set via API. example: createImpulse createdAt: type: string format: date-time description: The datetime that the block version was created. Cannot be set via API. datasetSubset: type: object description: Only generate features for samples where (sample_id + datasetSubsetSeed) % datasetSubset) == 0 required: - datasetSubset - datasetSubsetSeed properties: subsetModulo: type: number subsetSeed: type: number AddKerasFilesRequest: type: object required: - zip properties: zip: type: string format: binary ProfileModelTable: type: object required: - variant - lowEndMcu - highEndMcu - highEndMcuPlusAccelerator - mpu - gpuOrMpuAccelerator description: Performance for a range of device types. Note that MPU is referred to as CPU in Studio, as MPU and CPU are treated equivalent for performance estimation. properties: variant: type: string enum: - int8 - float32 lowEndMcu: $ref: '#/components/schemas/ProfileModelTableMcu' highEndMcu: $ref: '#/components/schemas/ProfileModelTableMcu' highEndMcuPlusAccelerator: $ref: '#/components/schemas/ProfileModelTableMcu' mpu: $ref: '#/components/schemas/ProfileModelTableMpu' gpuOrMpuAccelerator: $ref: '#/components/schemas/ProfileModelTableMpu' AnomalyConfig: type: object required: - name - axes - trained - dependencies - selectedAxes - minimumConfidenceRating properties: dependencies: $ref: '#/components/schemas/DependencyData' name: type: string axes: type: array description: Selectable axes for the anomaly detection block items: type: object required: - label - selected - favourite properties: label: type: string selected: type: boolean favourite: type: boolean trained: type: boolean description: Whether the block is trained clusterCount: type: integer description: Number of clusters for K-means, or number of components for GMM (in config) selectedAxes: type: array items: type: integer description: Selected clusters (in config) minimumConfidenceRating: type: number description: Minimum confidence rating for this block, scores above this number will be flagged as anomaly. BlockParameters: description: Training parameters specific to the type of the learn block. Parameters may be adjusted depending on the model defined in the visual layers. Used for our built-in blocks. oneOf: - $ref: '#/components/schemas/BlockParamsVisualAnomalyPatchcore' - $ref: '#/components/schemas/BlockParamsVisualAnomalyGmm' GetDataExplorerFeaturesResponse: allOf: - $ref: '#/components/schemas/GenericApiResponse' - type: object required: - hasFeatures - data properties: hasFeatures: type: boolean data: type: array items: type: object required: - X - y - yLabel properties: X: type: object description: Data by feature index for this window example: '`{ 0: 9.81, 11: 0.32, 22: 0.79 }`' additionalProperties: type: number y: type: integer description: Training label index yLabel: type: string description: Training label string sample: type: object required: - id - name - startMs - endMs - category properties: id: type: number name: type: string startMs: type: number endMs: type: number category: type: string enum: - training - testing inputBlock: $ref: '#/components/schemas/ImpulseInputBlock' AnomalyCapacity: type: string description: Capacity level for visual anomaly detection. Determines which set of default configurations to use. The higher capacity, the higher number of (Gaussian) components, and the more adapted the model becomes to the original distribution enum: - low - medium - high TestPretrainedModelRequest: type: object required: - features - modelInfo properties: features: type: array items: type: number modelInfo: type: object required: - input - model properties: input: discriminator: propertyName: inputType mapping: time-series: '#/components/schemas/DeployPretrainedModelInputTimeSeries' audio: '#/components/schemas/DeployPretrainedModelInputAudio' image: '#/components/schemas/DeployPretrainedModelInputImage' other: '#/components/schemas/DeployPretrainedModelInputOther' oneOf: - $ref: '#/components/schemas/DeployPretrainedModelInputTimeSeries' - $ref: '#/components/schemas/DeployPretrainedModelInputAudio' - $ref: '#/components/schemas/DeployPretrainedModelInputImage' - $ref: '#/components/schemas/DeployPretrainedModelInputOther' model: discriminator: propertyName: modelType mapping: classification: '#/components/schemas/DeployPretrainedModelModelClassification' regression: '#/components/schemas/DeployPretrainedModelModelRegression' object-detection: '#/components/schemas/DeployPretrainedModelModelObjectDetection' oneOf: - $ref: '#/components/schemas/DeployPretrainedModelModelClassification' - $ref: '#/components/schemas/DeployPretrainedModelModelRegression' - $ref: '#/components/schemas/DeployPretrainedModelModelObjectDetection' KerasCustomMetric: type: object required: - name - value properties: name: description: The name of the metric type: string value: description: The value of this metric for this model type type: string DeployPretrainedModelInputOther: type: object required: - inputType properties: inputType: type: string enum: - other DeployPretrainedModelModelClassification: type: object required: - modelType - labels properties: modelType: type: string enum: - classification labels: type: array items: type: string TestPretrainedModelResponse: allOf: - $ref: '#/components/schemas/GenericApiResponse' - type: object properties: result: type: object description: Classification value per label. For a neural network this will be the confidence, for anomalies the anomaly score. additionalProperties: type: number boundingBoxes: type: array items: $ref: '#/components/schemas/BoundingBoxWithScore' SavePretrainedModelRequest: type: object required: - input - model properties: input: discriminator: propertyName: inputType mapping: time-series: '#/components/schemas/DeployPretrainedModelInputTimeSeries' audio: '#/components/schemas/DeployPretrainedModelInputAudio' image: '#/components/schemas/DeployPretrainedModelInputImage' other: '#/components/schemas/DeployPretrainedModelInputOther' oneOf: - $ref: '#/components/schemas/DeployPretrainedModelInputTimeSeries' - $ref: '#/components/schemas/DeployPretrainedModelInputAudio' - $ref: '#/components/schemas/DeployPretrainedModelInputImage' - $ref: '#/components/schemas/DeployPretrainedModelInputOther' model: discriminator: propertyName: modelType mapping: classification: '#/components/schemas/DeployPretrainedModelModelClassification' regression: '#/components/schemas/DeployPretrainedModelModelRegression' object-detection: '#/components/schemas/DeployPretrainedModelModelObjectDetection' oneOf: - $ref: '#/components/schemas/DeployPretrainedModelModelClassification' - $ref: '#/components/schemas/DeployPretrainedModelModelRegression' - $ref: '#/components/schemas/DeployPretrainedModelModelObjectDetection' AdditionalMetric: type: object required: - name - value - fullPrecisionValue properties: name: type: string value: type: string fullPrecisionValue: type: number tooltipText: type: string link: type: string AnomalyTrainedFeaturesResponse: allOf: - $ref: '#/components/schemas/GenericApiResponse' - type: object required: - totalSampleCount - data properties: totalSampleCount: type: integer description: Total number of windows in the data set data: type: array items: type: object required: - X properties: X: type: object description: Data by feature index for this window. Note that this data was scaled by the StandardScaler, use the anomaly metadata to unscale if needed. example: '0': -2.17 '11': 1.21 '22': 0.79 additionalProperties: type: number label: type: number description: Label used for datapoint colorscale in anomaly explorer (for gmm only). Is currently the result of the scoring function. AnomalyConfigResponse: allOf: - $ref: '#/components/schemas/GenericApiResponse' - $ref: '#/components/schemas/AnomalyConfig' ModelEngineShortEnum: type: string enum: - tflite-eon - tflite-eon-ram-optimized - tflite UploadPretrainedModelRequest: type: object required: - modelFile - modelFileName - modelFileType properties: modelFile: type: string format: binary modelFileName: type: string modelFileType: type: string enum: - tflite - onnx - saved_model representativeFeatures: type: string format: binary device: description: MCU used for calculating latency, query `latencyDevices` in `listProject` for a list of supported devices (and use the "mcu" property here). If this is kept empty then we'll show an overview of multiple devices. type: string ProfileModelInfoMemoryDetails: type: object required: - ram - rom - arenaSize properties: ram: type: integer description: Estimated amount of RAM required by the model, measured in bytes rom: type: integer description: Estimated amount of ROM required by the model, measured in bytes arenaSize: type: integer description: Estimated arena size required for model inference, measured in bytes BoundingBoxWithScore: type: object description: This has the _ratio_ for x/y/w/h (so 0..1) required: - label - x - y - width - height - score properties: label: type: string x: type: number y: type: number width: type: number height: type: number score: type: number DependencyData: type: object required: - classes - blockNames - featureCount - sampleCount properties: classes: type: array items: type: string blockNames: type: array items: type: string featureCount: type: integer sampleCount: type: integer AnomalyModelMetadata: type: object required: - created - scale - mean - clusters - axes properties: created: type: string format: date-time description: Date when the model was trained scale: type: array description: 'Scale input for StandardScaler. Values are scaled like this (where `ix` is axis index): `input[ix] = (input[ix] - mean[ix]) / scale[ix];`' items: type: number mean: type: array description: 'Mean input for StandardScaler. Values are scaled like this (where `ix` is axis index): `input[ix] = (input[ix] - mean[ix]) / scale[ix];`' items: type: number clusters: type: array description: Trained K-means clusters items: type: object required: - center - maxError properties: center: type: array description: Center of each cluster (one value per axis) items: type: number maxError: type: number description: Size of the cluster axes: type: array description: Which axes were included during training (by index) example: '`[ 0, 11, 22 ]`' items: type: integer defaultMinimumConfidenceRating: type: number description: Default minimum confidence rating required before tagging as anomaly, based on scores of training data (GMM only). availableModelTypes: type: array description: The types of model that are available items: $ref: '#/components/schemas/KerasModelTypeEnum' recommendedModelType: description: The model type that is recommended for use $ref: '#/components/schemas/KerasModelTypeEnum' modelValidationMetrics: type: array description: Metrics for each of the available model types items: $ref: '#/components/schemas/KerasModelMetadataMetrics' hasTrainedModel: type: boolean SetAnomalyParameterRequest: type: object description: Only fields defined in this object are set properties: minimumConfidenceRating: type: number description: Minimum confidence score, if the anomaly block scores a sample above this threshold it will be flagged as anomaly. KerasVisualLayerType: type: string enum: - dense - conv1d - conv2d - reshape - flatten - dropout - batchNormalization - transfer_mobilenetv2_a35 - transfer_mobilenetv2_a1 - transfer_mobilenetv2_a05 - transfer_mobilenetv2_160_a1 - transfer_mobilenetv2_160_a75 - transfer_mobilenetv2_160_a5 - transfer_mobilenetv2_160_a35 - transfer_mobilenetv1_a25_d100 - transfer_mobilenetv1_a2_d100 - transfer_mobilenetv1_a1_d100 - transfer_kws_mobilenetv1_a1_d100 - transfer_kws_mobilenetv2_a35_d100 - transfer_kws_syntiant_ndp10x - transfer_kws_conv2d_tiny - object_ssd_mobilenet_v2_fpnlite_320x320 - fomo_mobilenet_v2_a01 - fomo_mobilenet_v2_a35 - transfer_organization - transfer_akidanet_imagenet_160_a100 - transfer_akidanet_imagenet_160_a50 - transfer_akidanet_imagenet_160_a25 - transfer_akidanet_imagenet_224_a100 - transfer_akidanet_imagenet_224_a50 - transfer_akidanet_imagenet_224_a25 - fomo_akidanet_a50 - fomo_ad_gmm - fomo_ad_patchcore ObjectDetectionLastLayer: type: string enum: - mobilenet-ssd - fomo - yolov2-akida - yolov5 - yolov5v5-drpai - yolox - yolov7 - tao-retinanet - tao-ssd - tao-yolov3 - tao-yolov4 DeployPretrainedModelModelRegression: type: object required: - modelType properties: modelType: type: string enum: - regression ProfileModelInfo: type: object required: - device - tfliteFileSizeBytes - isSupportedOnMcu properties: device: type: string tfliteFileSizeBytes: type: integer isSupportedOnMcu: type: boolean memory: type: object properties: tflite: $ref: '#/components/schemas/ProfileModelInfoMemoryDetails' eon: $ref: '#/components/schemas/ProfileModelInfoMemoryDetails' eonRamOptimized: $ref: '#/components/schemas/ProfileModelInfoMemoryDetails' timePerInferenceMs: type: integer mcuSupportError: type: string AnomalyGmmMetadataResponse: allOf: - $ref: '#/components/schemas/GenericApiResponse' - $ref: '#/components/schemas/AnomalyGmmMetadata' KerasVisualLayer: type: object required: - type properties: type: $ref: '#/components/schemas/KerasVisualLayerType' neurons: type: integer description: Number of neurons or filters in this layer (only for dense, conv1d, conv2d) or in the final conv2d layer (only for transfer layers) kernelSize: type: integer description: Kernel size for the convolutional layers (only for conv1d, conv2d) dropoutRate: type: number description: Fraction of input units to drop (only for dropout) or in the final layer dropout (only for transfer layers) columns: type: integer description: Number of columns for the reshape operation (only for reshape) stack: type: integer description: Number of convolutional layers before the pooling layer (only for conv1d, conv2d) enabled: type: boolean organizationModelId: type: integer description: Custom transfer learning model ID (when type is set to transfer_organization) ImageInputScaling: description: Normalization that is applied to images. If this is not set then 0..1 is used. "0..1" gives you non-normalized pixels between 0 and 1. "-1..1" gives you non-normalized pixels between -1 and 1. "0..255" gives you non-normalized pixels between 0 and 255. "-128..127" gives you non-normalized pixels between -128 and 127. "torch" first scales pixels between 0 and 1, then applies normalization using the ImageNet dataset (same as `torchvision.transforms.Normalize()`). "bgr-subtract-imagenet-mean" scales to 0..255, reorders pixels to BGR, and subtracts the ImageNet mean from each channel. type: string enum: - 0..1 - -1..1 - -128..127 - 0..255 - torch - bgr-subtract-imagenet-mean DeployPretrainedModelInputAudio: type: object required: - inputType - frequencyHz properties: inputType: type: string enum: - audio frequencyHz: type: number ProfileModelTableMpu: type: object required: - description - supported properties: description: type: string timePerInferenceMs: type: integer rom: type: number supported: type: boolean BlockParamsVisualAnomalyGmm: type: object properties: backbone: type: string description: The backbone to use for feature extraction KerasConfig: type: object required: - dependencies - trained - name - script - minimumConfidenceRating - selectedModelType - mode - trainingCycles - learningRate - defaultBatchSize - visualLayers - augmentationPolicyImage - transferLearningModels - shape - profileInt8 - skipEmbeddingsAndMemory - showAdvancedTrainingSettings - showAugmentationTrainingSettings properties: dependencies: $ref: '#/components/schemas/DependencyData' trained: type: boolean description: Whether the block is trained name: type: string type: $ref: '#/components/schemas/LearnBlockType' script: type: string description: The Keras script. This script might be empty if the mode is visual. minimumConfidenceRating: type: number description: Minimum confidence rating required for the neural network. Scores below this confidence are tagged as uncertain. selectedModelType: description: The model type that is currently selected. $ref: '#/components/schemas/KerasModelTypeEnum' mode: type: string description: The mode (visual or expert) to use for editing this network. enum: - visual - expert visualLayers: type: array description: The visual layers (if in visual mode) for the neural network. This will be an empty array when in expert mode. items: $ref: '#/components/schemas/KerasVisualLayer' trainingCycles: type: integer description: Number of training cycles. If in expert mode this will be 0. learningRate: type: number description: Learning rate (between 0 and 1). If in expert mode this will be 0. batchSize: type: integer description: The batch size used during training. defaultBatchSize: type: integer description: The default batch size if a value is not configured. shape: type: string description: Python-formatted tuple of input axes trainTestSplit: type: number description: Train/test split (between 0 and 1) autoClassWeights: type: boolean description: Whether to automatically balance class weights, use this for skewed datasets. useLearnedOptimizer: type: boolean description: Use learned optimizer and ignore learning rate. augmentationPolicyImage: $ref: '#/components/schemas/AugmentationPolicyImageEnum' augmentationPolicySpectrogram: $ref: '#/components/schemas/AugmentationPolicySpectrogram' transferLearningModels: type: array items: $ref: '#/components/schemas/TransferLearningModel' profileInt8: type: boolean description: Whether to profile the i8 model (might take a very long time) skipEmbeddingsAndMemory: type: boolean description: If set, skips creating embeddings and measuring memory (used in tests) akidaEdgeLearningConfig: $ref: '#/components/schemas/AkidaEdgeLearningConfig' customValidationMetadataKey: type: string description: This metadata key is used to prevent group data leakage between train and validation datasets. showAdvancedTrainingSettings: type: boolean description: Whether the 'Advanced training settings' UI element should be expanded. showAugmentationTrainingSettings: type: boolean description: Whether the 'Augmentation training settings' UI element should be expanded. customParameters: type: object description: Training parameters, this list depends on the list of parameters that the model exposes. additionalProperties: type: string nullable: true anomalyCapacity: description: Capacity level for visual anomaly detection (GMM). Determines which set of default configurations to use. The higher capacity, the higher number of (Gaussian) components, and the more adapted the model becomes to the original distribution $ref: '#/components/schemas/AnomalyCapacity' lastShownModelVariant: $ref: '#/components/schemas/KerasModelVariantEnum' description: Last shown variant on the Keras screen. Used to keep the same view after refreshing. lastShownModelEngine: $ref: '#/components/schemas/ModelEngineShortEnum' description: Last shown model engine on the Keras screen. Used to keep the same view after refreshing. blockParameters: $ref: '#/components/schemas/BlockParameters' description: Training parameters specific to the type of the learn block. Parameters may be adjusted depending on the model defined in the visual layers. Used for our built-in blocks. PretrainedModelTensor: type: object required: - dataType - name - shape properties: dataType: type: string enum: - int8 - uint8 - float32 name: type: string shape: type: array items: type: integer quantizationScale: type: number quantizationZeroPoint: type: number TransferLearningModel: type: object required: - name - shortName - description - hasNeurons - hasDropout - type - author - blockType properties: name: type: string shortName: type: string abbreviatedName: type: string description: type: string hasNeurons: type: boolean hasDropout: type: boolean defaultNeurons: type: integer defaultDropout: type: number defaultLearningRate: type: number defaultTrainingCycles: type: number hasImageAugmentation: type: boolean type: $ref: '#/components/schemas/KerasVisualLayerType' learnBlockType: $ref: '#/components/schemas/LearnBlockType' organizationModelId: type: integer implementationVersion: type: integer repositoryUrl: type: string description: URL to the source code of this custom learn block. author: type: string blockType: $ref: '#/components/schemas/BlockType' customParameters: type: array items: $ref: '#/components/schemas/DSPGroupItem' displayCategory: $ref: '#/components/schemas/BlockDisplayCategory' KerasModelMetadataMetrics: type: object required: - type - loss - confusionMatrix - report - onDevicePerformance - visualization - isSupportedOnMcu - additionalMetrics properties: type: description: The type of model $ref: '#/components/schemas/KerasModelTypeEnum' loss: type: number description: The model's loss on the validation set after training accuracy: type: number description: The model's accuracy on the validation set after training confusionMatrix: type: array example: - - 31 - 1 - 0 - - 2 - 27 - 3 - - 1 - 0 - 39 items: type: array items: type: number report: type: object description: Precision, recall, F1 and support scores onDevicePerformance: type: array items: type: object required: - mcu - name - isDefault - latency - tflite - eon properties: mcu: type: string name: type: string isDefault: type: boolean latency: type: number tflite: type: object required: - ramRequired - romRequired - arenaSize - modelSize properties: ramRequired: type: integer romRequired: type: integer arenaSize: type: integer modelSize: type: integer eon: type: object required: - ramRequired - romRequired - arenaSize - modelSize properties: ramRequired: type: integer romRequired: type: integer arenaSize: type: integer modelSize: type: integer eon_ram_optimized: type: object required: - ramRequired - romRequired - arenaSize - modelSize properties: ramRequired: type: integer romRequired: type: integer arenaSize: type: integer modelSize: type: integer customMetrics: description: Custom, device-specific performance metrics type: array items: $ref: '#/components/schemas/KerasCustomMetric' predictions: type: array items: $ref: '#/components/schemas/ModelPrediction' visualization: type: string enum: - featureExplorer - dataExplorer - none isSupportedOnMcu: type: boolean mcuSupportError: type: string profilingJobId: description: If this is set, then we're still profiling this model. Subscribe to job updates to see when it's done (afterward the metadata will be updated). type: integer profilingJobFailed: description: If this is set, then the profiling job failed (get the status by getting the job logs for 'profilingJobId'). type: boolean additionalMetrics: type: array items: $ref: '#/components/schemas/AdditionalMetric' GetPretrainedModelResponse: allOf: - $ref: '#/components/schemas/GenericApiResponse' - type: object required: - specificDeviceSelected - availableModelTypes properties: specificDeviceSelected: type: boolean description: Whether a specific device was selected for performance profiling availableModelTypes: type: array description: The types of model that are available items: $ref: '#/components/schemas/KerasModelTypeEnum' model: type: object required: - fileName - inputs - outputs properties: fileName: type: string profileInfo: type: object required: - table properties: float32: $ref: '#/components/schemas/ProfileModelInfo' int8: $ref: '#/components/schemas/ProfileModelInfo' table: $ref: '#/components/schemas/ProfileModelTable' inputs: type: array items: $ref: '#/components/schemas/PretrainedModelTensor' outputs: type: array items: $ref: '#/components/schemas/PretrainedModelTensor' profileJobId: type: integer supportsTFLite: type: boolean modelInfo: type: object required: - input - model properties: input: discriminator: propertyName: inputType mapping: time-series: '#/components/schemas/DeployPretrainedModelInputTimeSeries' audio: '#/components/schemas/DeployPretrainedModelInputAudio' image: '#/components/schemas/DeployPretrainedModelInputImage' other: '#/components/schemas/DeployPretrainedModelInputOther' oneOf: - $ref: '#/components/schemas/DeployPretrainedModelInputTimeSeries' - $ref: '#/components/schemas/DeployPretrainedModelInputAudio' - $ref: '#/components/schemas/DeployPretrainedModelInputImage' - $ref: '#/components/schemas/DeployPretrainedModelInputOther' model: discriminator: propertyName: modelType mapping: classification: '#/components/schemas/DeployPretrainedModelModelClassification' regression: '#/components/schemas/DeployPretrainedModelModelRegression' object-detection: '#/components/schemas/DeployPretrainedModelModelObjectDetection' oneOf: - $ref: '#/components/schemas/DeployPretrainedModelModelClassification' - $ref: '#/components/schemas/DeployPretrainedModelModelRegression' - $ref: '#/components/schemas/DeployPretrainedModelModelObjectDetection' BlockParamsVisualAnomalyPatchcore: type: object properties: backbone: type: string description: The backbone to use for feature extraction numLayers: type: integer description: The number of layers in the feature extractor (1-3) poolSize: type: integer description: The pool size for the feature extractor samplingRatio: type: number description: The sampling ratio for the coreset, used for anomaly scoring numNearestNeighbors: type: integer description: The number of nearest neighbors to consider, used for anomaly scoring DSPGroupItem: type: object required: - name - type - param - defaultValue - readonly - shouldShow - required properties: name: type: string example: Scale axes value: type: string defaultValue: type: string type: type: string example: text help: type: string example: Divide axes by this number param: type: string example: scale-axes selectOptions: type: array items: type: object properties: value: type: string selected: type: boolean optionLabel: type: string readonly: type: boolean shouldShow: type: boolean showIf: type: object required: - parameter - operator - value properties: parameter: type: string operator: type: string enum: - eq - neq value: type: string invalidText: type: string section: type: string description: Interface section to render parameter in. enum: - advanced - augmentation - modelProfiling multiline: type: boolean description: Only valid for type "string". Will render a multiline text area. required: type: boolean hint: type: string description: If set, shows a hint below the input. placeholder: type: string description: Sets the placeholder text on the input element (for types "string", "int", "float" and "secret") ModelPrediction: type: object required: - sampleId - startMs - endMs - prediction properties: sampleId: type: integer startMs: type: number endMs: type: number label: type: string prediction: type: string predictionCorrect: type: boolean f1Score: type: number description: Only set for object detection projects anomalyScores: type: array description: Only set for visual anomaly projects. 2D array of shape (n, n) with raw anomaly scores, where n varies based on the image input size and the specific visual anomaly algorithm used. The scores corresponds to each grid cell in the image's spatial matrix. items: type: array items: type: number UploadKerasFilesRequest: type: object required: - zip properties: zip: type: string format: binary AkidaEdgeLearningConfig: type: object required: - enabled properties: enabled: type: boolean description: True if Akida Edge Learning model creation is enabled. Other properties will be ignored if this is false. additionalClasses: type: number description: Number of additional classes that will be added to the Edge Learning model. neuronsPerClass: type: number description: Number of neurons in each class on the last layer in the Edge Learning model. KerasModelMetadataResponse: allOf: - $ref: '#/components/schemas/GenericApiResponse' - $ref: '#/components/schemas/KerasModelMetadata' SetKerasParameterRequest: type: object description: Only fields defined in this object are set properties: mode: type: string description: Whether to use visual or expert mode. enum: - expert - visual minimumConfidenceRating: type: number description: Minimum confidence score, if the neural network scores a sample below this threshold it will be flagged as uncertain. selectedModelType: description: The model type to select, as described in the model metadata call. $ref: '#/components/schemas/KerasModelTypeEnum' script: type: string description: Raw Keras script (only used in expert mode) visualLayers: type: array description: The visual layers for the neural network (only in visual mode). items: $ref: '#/components/schemas/KerasVisualLayer' trainingCycles: type: integer description: Number of training cycles (only in visual mode). learningRate: type: number description: Learning rate (between 0 and 1) (only in visual mode). batchSize: type: number description: Batch size used during training (only in visual mode). trainTestSplit: type: number description: Train/test split (between 0 and 1) autoClassWeights: type: boolean description: Whether to automatically balance class weights, use this for skewed datasets. useLearnedOptimizer: type: boolean description: Use learned optimizer and ignore learning rate. augmentationPolicyImage: $ref: '#/components/schemas/AugmentationPolicyImageEnum' augmentationPolicySpectrogram: $ref: '#/components/schemas/AugmentationPolicySpectrogram' profileInt8: type: boolean description: Whether to profile the i8 model (might take a very long time) skipEmbeddingsAndMemory: type: boolean description: If set, skips creating embeddings and measuring memory (used in tests) akidaEdgeLearningConfig: $ref: '#/components/schemas/AkidaEdgeLearningConfig' customValidationMetadataKey: type: string description: If the 'custom validation split' experiment is enabled, this metadata key is used to prevent group data leakage between train and validation datasets. showAdvancedTrainingSettings: type: boolean description: Whether the 'Advanced training settings' UI element should be expanded. showAugmentationTrainingSettings: type: boolean description: Whether the 'Augmentation training settings' UI element should be expanded. customParameters: type: object description: Training parameters, this list depends on the list of parameters that the model exposes. additionalProperties: type: string nullable: true anomalyCapacity: description: Capacity level for visual anomaly detection. Determines which set of default configurations to use. The higher capacity, the higher number of (Gaussian) components, and the more adapted the model becomes to the original distribution $ref: '#/components/schemas/AnomalyCapacity' lastShownModelVariant: $ref: '#/components/schemas/KerasModelVariantEnum' description: Last shown variant on the Keras screen. Used to keep the same view after refreshing. lastShownModelEngine: $ref: '#/components/schemas/ModelEngineShortEnum' description: Last shown model engine on the Keras screen. Used to keep the same view after refreshing. blockParameters: $ref: '#/components/schemas/BlockParameters' description: Training parameters specific to the type of the learn block. Parameters may be adjusted depending on the model defined in the visual layers. Used for our built-in blocks. AnomalyModelMetadataResponse: allOf: - $ref: '#/components/schemas/GenericApiResponse' - $ref: '#/components/schemas/AnomalyModelMetadata' AugmentationPolicySpectrogram: type: object required: - enabled properties: enabled: type: boolean description: True if spectrogram augmentation is enabled. Other properties will be ignored if this is false. warping: type: boolean description: True if warping along the time axis is enabled. freqMasking: type: string enum: - none - low - high description: The amount of frequency masking to apply. timeMasking: type: string enum: - none - low - high description: The amount of time masking to apply. gaussianNoise: type: string enum: - none - low - high description: The amount of Gaussian noise to add. KerasModelTypeEnum: type: string enum: - int8 - float32 - akida - requiresRetrain KerasModelMetadata: type: object required: - created - layers - classNames - availableModelTypes - recommendedModelType - modelValidationMetrics - hasTrainedModel - mode - imageInputScaling - labels properties: created: type: string format: date-time description: Date when the model was trained layers: type: array description: Layers of the neural network items: $ref: '#/components/schemas/KerasModelLayer' classNames: type: array description: Labels for the output layer items: type: string labels: type: array description: Original labels in the dataset when features were generated, e.g. used to render the feature explorer. items: type: string availableModelTypes: type: array description: The types of model that are available items: $ref: '#/components/schemas/KerasModelTypeEnum' recommendedModelType: description: The model type that is recommended for use $ref: '#/components/schemas/KerasModelTypeEnum' modelValidationMetrics: type: array description: Metrics for each of the available model types items: $ref: '#/components/schemas/KerasModelMetadataMetrics' hasTrainedModel: type: boolean mode: type: string enum: - classification - regression - object-detection - visual-anomaly - anomaly-gmm objectDetectionLastLayer: $ref: '#/components/schemas/ObjectDetectionLastLayer' imageInputScaling: $ref: '#/components/schemas/ImageInputScaling' KerasModelLayer: type: object required: - input - output properties: input: type: object required: - shape - name - type properties: shape: type: integer description: Input size example: 33 name: type: string description: TensorFlow name example: x_input:0 type: type: string description: TensorFlow type example: '' output: type: object required: - shape - name - type properties: shape: type: integer description: Output size example: 20 name: type: string description: TensorFlow name example: dense_1/Relu:0 type: type: string description: TensorFlow type example: '' LearnBlockType: type: string description: The type of learning block (anomaly, keras, keras-transfer-image, keras-transfer-kws, keras-object-detection, keras-regression). Each behaves differently. enum: - anomaly - anomaly-gmm - keras - keras-transfer-image - keras-transfer-kws - keras-object-detection - keras-regression - keras-akida - keras-akida-transfer-image - keras-akida-object-detection - keras-visual-anomaly DeployPretrainedModelModelObjectDetection: type: object required: - modelType - labels - lastLayer - minimumConfidence properties: modelType: type: string enum: - object-detection labels: type: array items: type: string lastLayer: $ref: '#/components/schemas/ObjectDetectionLastLayer' minimumConfidence: description: Threshold for objects (f.e. 0.3) type: number DeployPretrainedModelInputTimeSeries: type: object required: - inputType - frequencyHz - windowLengthMs properties: inputType: type: string enum: - time-series frequencyHz: type: number windowLengthMs: type: integer KerasResponse: allOf: - $ref: '#/components/schemas/GenericApiResponse' - $ref: '#/components/schemas/KerasConfig' GenericApiResponse: type: object required: - success properties: success: type: boolean description: Whether the operation succeeded error: type: string description: Optional error description (set if 'success' was false) AugmentationPolicyImageEnum: type: string description: The data augmentation policy to use with image input enum: - none - all parameters: ProjectIdParameter: name: projectId in: path required: true description: Project ID schema: type: integer SampleIdParameter: name: sampleId in: path required: true description: Sample ID schema: type: integer ModelDownloadIdParameter: name: modelDownloadId in: path required: true description: Model download ID, which can be obtained from the project information schema: type: string OptionalImpulseIdParameter: name: impulseId in: query required: false description: Impulse ID. If this is unset then the default impulse is used. schema: type: integer PretrainedModelDownloadParameter: name: pretrainedModelDownloadType in: path required: true schema: type: string enum: - tflite_float32 - tflite_int8 - onnx - saved_model FeatureAx2Parameter: name: featureAx2 in: query required: true description: Feature axis 2 schema: type: integer ExcludeLabelsParameter: name: excludeLabels in: query required: false description: If set to "true", the "labels" field is left empty (which can be big on e.g. regression projects). schema: type: boolean LearnIdParameter: name: learnId in: path required: true description: Learn Block ID, use the impulse functions to retrieve the ID schema: type: integer FeatureAx1Parameter: name: featureAx1 in: query required: true description: Feature axis 1 schema: type: integer securitySchemes: ApiKeyAuthentication: type: apiKey in: header name: x-api-key JWTAuthentication: type: apiKey in: cookie name: jwt JWTHttpHeaderAuthentication: type: apiKey in: header name: x-jwt-token