Storing Files

Files can be uploaded to Sphinx using Get Presigned Url For Upload. This endpoint returns a presigned URL from AWS’ S3 you can use to POST files to.

Workflow

  1. GET storage information via Get Presigned Url For Upload.
  2. POST to presignedResponse.url, adding presignedResponse.fields and your file. The file must be last.
  3. POST using path in Create File and Create Dataset.

Addition of a file uses AWS’ native file upload mechanism is used for optimal performance. You can read about the full endpoint here S3 POST Object but usage of this S3 endpoint is not required. Instead you can send a POST request to the presignedResponse.url you receive from Get Presigned Url For Upload. The path field is used as the stagedUploadPath parameter in Create File and Create Dataset.

An example for uploading a file to a stagedUploadPath is below.

1# Create path in storage
2filepath = "your full file path"
3filename = filepath.split("/")[-1]
4response = requests.get(f"https://api.sphinxbio.com/api/storage/url/upload?filename={filename}", headers={"Authorization": f"Bearer {API_KEY}"}).json()
5
6# Upload file to storage
7with open(filepath, "rb") as f:
8    files = {"file": (filepath.split("/")[-1], f)}
9    path = response["presignedResponse"]["url"]
10    requests.post(path, data=response["presignedResponse"]["fields"], files=files)

Note that usage of this file in Create Dataset requires the tracking of a Task. See Get Task.

Built with