openapi: 3.0.0 info: title: Fence OpenAPI Specification admin/user data API version: 0.1.0 description: Access management for Gen3 data commons. Code is available on [GitHub](https://github.com/uc-cdis/fence). termsOfService: http://cdis.uchicago.edu/terms/ contact: email: cdis@uchicago.edu license: name: Apache 2.0 url: http://www.apache.org/licenses/LICENSE-2.0.html servers: - url: https://example.domain/ tags: - name: data description: Generate signed URLs paths: /data/{file_id}: delete: tags: - data summary: delete all locations of a stored data file and remove its record from indexd description: After a user uploads a data file and it is registered in indexd (before or after it is mapped into the graph via metadata submission), this endpoint will delete the file from its storage locations (saved in the record in indexd) and delete the record in indexd. security: - OAuth2: - data responses: 204: description: successful operation 500: description: internal server error; could not delete stored files, or not able to delete indexd record content-type: application/json /data/download/{file_id}: get: tags: - data summary: Create a signed URL for data download given a file_id description: Allow users to get a signed URL for a data file. security: - OAuth2: - user operationId: downloadSignedURL parameters: - name: file_id required: true in: path description: data UUID schema: type: string - name: protocol required: false in: query description: a protocol provided by storage provider, e.g. http, ftp, s3, gs schema: type: string enum: - http - s3 - gs - name: expires_in required: false in: query description: the time (in seconds) in which return url is valid. Must be less than the configured maximum (default is 3600). If it's greater, the configured maximum will be used. schema: type: integer - name: redirect required: false in: query description: if set, then a redirect to the signed url will be sent. Otherwise, json data with the url is returned. schema: type: boolean - name: no_force_sign required: false in: query description: if `no_force_sign=True`, this will request to *not* sign the resulting URL (i.e. just provide the public url without using anonymous signing creds). schema: type: boolean - name: userProject required: false in: query description: a Google Project to bill for accessing data in requester pays buckets in Google Storage. Will override any configured default billing projects. If Fence is configured to automatically attempt to provide the necessary service account(s) billing permission, the provided Google Project will need to have given the Fence admin service account necessary permissions to create custom roles and set Project IAM policies. Please see README for more information. schema: type: string responses: '200': description: successful operation content: '*/*': schema: $ref: '#/components/schemas/SignedURL' '400': description: 'Invalid input: UUID not found or invalid location' '404': description: No location found for this file /ga4gh/drs/v1/objects/{object_id}/access/{access_id}: get: summary: GA4GH DRS Access API to get a URL for fetching bytes. description: Returns a URL that can be used to fetch the bytes of a DrsObject. This method only needs to be called when using an AccessMethod that contains an access_id (e.g., for servers that use signed URLs for fetching object bytes). operationId: GetAccessURL responses: '200': description: The access URL was found successfully. content: '*/*': schema: $ref: '#/components/schemas/AccessURL' '400': description: The request is malformed. '401': description: The request is unauthorized. '404': description: The requested access URL wasn't found '403': description: The requester is not authorized to perform this action. '500': description: An unexpected error occurred. parameters: - name: object_id schema: type: string in: path required: true description: An id of a DrsObject - name: access_id schema: type: string in: path required: true description: An access_id from the access_methods list of a DrsObject tags: - data post: summary: GA4GH DRS Access API to get a URL for fetching bytes. description: Returns a URL that can be used to fetch the bytes of a DrsObject. This method only needs to be called when using an AccessMethod that contains an access_id (e.g., for servers that use signed URLs for fetching object bytes). operationId: PostAccessURL requestBody: content: application/json: schema: $ref: '#/components/schemas/RequestUrlPassport' responses: '200': description: The access URL was found successfully. content: '*/*': schema: $ref: '#/components/schemas/AccessURL' '400': description: The request is malformed. '401': description: The request is unauthorized. '404': description: The requested access URL wasn't found '403': description: The requester is not authorized to perform this action. '500': description: An unexpected error occurred. parameters: - name: object_id schema: type: string in: path required: true description: An id of a DrsObject - name: access_id schema: type: string in: path required: true description: An access_id from the access_methods list of a DrsObject tags: - data /data/upload: post: tags: - data summary: Create a new blank record in indexd to use for uploading a data file less than 5GB. For files larger than 5GB, please use multipart upload presigned url description: 'This is the first step on the API side for the data upload flow. This endpoint causes fence to make a request to indexd to create a new, blank index record, and returns the GUID for this new record and a presigned URL which a client can then use to upload their data file directly to a storage bucket. No parameters are accepted because fence retrieves the username from the current token to send to indexd for the `uploader` field in the new record. This API also supports utilizing the newer authorization within the indexing service by providing an "authz" field. If an "authz" field is provided, the authorization checks are slightly different: your user must have *both* "create" and "write-storage" permission on the resources you are including in the "authz" list. In addition to a different authorization check, when "authz" is provided the "uploader" field in indexd with *not* be populated. Previous authorization check requires a more general, global upload permission: "file_upload" on "/data_file" resource. When "authz" is *not* provided, this endpoint will check for that permission for your user. Accepts a "guid" field in the request body. If "guid" is provided, it checks indexd for an existing record. If not found, it raises a 404. ' security: - OAuth2: - user - data operationId: uploadBlank requestBody: content: application/json: schema: $ref: '#/components/schemas/RequestUploadBlank' responses: 201: description: successful operation; created new record in indexd content: application/json: schema: type: object properties: guid: type: string description: the GUID from the newly created record in indexd url: type: string description: the presigned URL usable for data upload 404: description: Record with not found. /data/upload/{file_id}: get: tags: - data summary: Create a signed URL for data upload specified by file_id description: Allow users to get a signed URL to upload data given the location of preference. security: - OAuth2: - user operationId: uploadSignedURL parameters: - name: file_id required: true in: path description: data UUID schema: type: string - name: protocol required: false in: query description: a protocol provided by storage provider, e.g. http, ftp, s3, gs schema: type: string enum: - http - s3 - name: expires_in required: false in: query description: the time (in seconds) in which return url is valid. Must be less than the configured maximum (default is 3600). If it's greater, the configured maximum will be used. schema: type: integer - name: file_name required: false in: query description: the requested file name in the cloud bucket you will upload to. If not provided, will use the GUID/file_id schema: type: string - name: bucket required: false in: query description: the requested bucket to upload to. If not provided, defaults to the configured DATA_UPLOAD_BUCKET. schema: type: string responses: '200': description: successful operation content: '*/*': schema: $ref: '#/components/schemas/SignedURL' '400': description: 'Invalid input: UUID not found or invalid location' /multipart/init: post: tags: - data summary: Initilize a multipart upload for AWS bucket. It returns an uploadId which can be used for uploading object parts (see /multipart/upload) and completing the upload (see /multipart/complete) description: 'For uploading the big file with the size is larger than 5GB in data upload flow, Fence needs to provide a list of endpoints for supporting multipart upload presigned url This is the first step on the API side for the multipart upload presigned url. This endpoint causes fence to make a request to indexd to create a new, blank index record, and returns the GUID for this new record and an uploadId for multipart upload presigned url. Accepts a "guid" field in the request body. If "guid" is provided, it checks indexd for an existing record. If not found, it raises a 404.' security: - OAuth2: - user - data operationId: multipartUploadInit requestBody: content: application/json: schema: $ref: '#/components/schemas/InitMultipartUpload' responses: 201: description: successful operation; created new blank record in indexd and uploadId for multipart upload content: application/json: schema: type: object properties: guid: type: string description: the GUID from the newly created record in indexd uploadId: type: string description: the uploadId for multipart upload presigned URL usable for data upload 404: description: Record with not found. /multipart/upload: post: tags: - data summary: Upload an object part given uploadId description: This is the second step on the API side for the multipart upload presigned url. The object is chopped into multiple parts with part number start from 1. Each part is submitted to the cloud resource with this endpoint by providing the part number a long with uploadId. It is worth to mention that the minimal supported part size is 5 MB as describe in https://docs.aws.amazon.com/AmazonS3/latest/dev/qfacts.html security: - OAuth2: - user - data operationId: multipartUpload requestBody: content: application/json: schema: $ref: '#/components/schemas/RequestMultipartUpload' responses: 200: description: successful operation content: application/json: schema: type: object properties: presigned_url: type: string description: the presigned URL for uploading part /multipart/complete: post: tags: - data summary: Complete aws multipart upload description: This is the last step for the multipart upload presigned url. All the parts which were submitted need to be combined together. This enpoint takes a list of the part info (partNumber, Etag) and uploadId in order to finish the upload security: - OAuth2: - user - data operationId: multipartUploadComplete requestBody: content: application/json: schema: $ref: '#/components/schemas/CompleteMultipartUpload' responses: 200: description: successful operation /data/buckets: get: tags: - data summary: Get list of buckets and relevant bucket info description: Returns a list of buckets and relevant bucket information provided in the fence-config under AWS_BUCKETS and GS_BUCKETS. It filters out sensitive information about the bucket and only shows information like bucket region. operationId: getBuckets responses: '200': description: successful operation content: '*/*': schema: $ref: '#/components/schemas/BucketInfo' components: schemas: RequestUploadBlank: type: object required: null properties: file_name: type: string required: true description: the file name to use for this upload bucket: type: string required: true description: the requested bucket to upload to. If not provided, defaults to the configured DATA_UPLOAD_BUCKET. guid: type: string required: false description: GUID to be assigned to the object expires_in: type: integer description: optional integer specifying the presigned URL lifetime authz: type: array items: type: string description: requested authorization resources to be set on the resulting indexed record. You must have proper authorization to set this example: guid: 123456abcd file_name: my_file.bam bucket: bucket1 expires_in: 1200 authz: - /programs/A BucketInfo: type: object properties: S3_BUCKETS: type: object description: list and info on S3 buckets configured in the fence config properties: bucket1: type: object properties: region: type: string bucket2: type: object properties: region: type: string GS_BUCKETS: type: object description: list and info on GS buckets configured in the fence config properties: bucket1: type: object properties: region: type: string bucket2: type: object properties: region: type: string AccessURL: type: object required: - url properties: url: type: string description: A fully resolvable URL that can be used to fetch the actual object bytes. RequestUrlPassport: type: object required: true properties: passports: type: array items: type: string required: true description: The list of GA4GH passports containing the authorizations for a given user to access the requested file. NOTE - the key "passports" is configurable in fence via the GA4GH_DRS_POSTED_PASSPORT_FIELD field. The default of "passports" has been provided. SignedURL: type: object properties: url: type: string description: the signed url issued to CompleteMultipartUpload: type: object required: null properties: key: type: string description: the key of the object in the format of GUID/filename uploadId: type: string description: the uploadId for multipart presigned URL upload usable for data upload parts: type: array items: type: object properties: partNumber: type: integer description: The part number ETag: type: string description: the etag that cooresponds to the the object part with the partnumber description: List of part numbers info needed for multipart completion expires_in: type: integer description: optional integer specifying the presigned URL lifetime bucket: type: string required: false description: bucket to upload to RequestMultipartUpload: type: object required: null properties: key: type: string description: the key of the object in the format of GUID/filename uploadId: type: string description: the uploadId for multipart presigned URL upload usable for data upload partNumber: type: integer description: the part number of the part (start from 1) expires_in: type: integer description: optional integer specifying the presigned URL lifetime bucket: type: string required: false description: bucket to upload\ to InitMultipartUpload: type: object required: null properties: file_name: type: string required: true description: the file name to use for this upload guid: type: string required: false description: GUID to be assigned to the object expires_in: type: integer description: optional integer specifying the presigned URL lifetime authz: type: array items: type: string description: requested authorization resources to be set on the resulting indexed record. You must have proper authorization to set this bucket: type: string required: false description: bucket to upload to example: file_name: my_file.bam expires_in: 1200 authz: - /programs/A bucket: bucket-1 securitySchemes: OAuth2: type: oauth2 flows: authorizationCode: authorizationUrl: /oauth/authorize tokenUrl: /oauth/token scopes: user: generic user access