Azure Blob Storage

Tusd can store files directly on Azure Blob Storage or other compatible services, e.g. Azurite. The uploaded file is directly transferred to Azure while the user is performing the upload without storing the entire file on disk first.

Configuration

To enable this backend, you must supply the corresponding access credentials using environment variables and specify the container name using -azure-storage, for example:

$ export AZURE_STORAGE_ACCOUNT=xxxxx
$ export AZURE_STORAGE_KEY=xxxxx
$ tusd -azure-storage=my-test-container
[tusd] 2024/02/23 11:34:03.411021 Using Azure endpoint https://xxxxx.blob.core.windows.net.
...

Alternative endpoints

If you want to upload to Azure Blob Storage using a custom endpoint, e.g when using Azurite for local development, you can specify the endpoint using the -azure-endpoint flag. For example:

$ tusd -azure-storage=my-test-container -azure-endpoint=https://my-custom-endpoint.com
[tusd] 2023/02/13 16:15:18.641937 Using Azure endpoint https://my-custom-endpoint.com.
...

Object prefix

If the container is also used to store files from other sources than tusd, it makes sense to use a custom prefix for all object relating to tusd. This can be achieved using the -azure-object-prefix flag. For example, the following configuration will cause the objects to be put under the uploads/ prefix in the bucket:

$ tusd -azure-storage=my-test-container -azure-object-prefix=uploads/

Storage tiers

You can also upload blobs to Azure Blob Storage with a different storage tier than what is set as the default for the storage account. This can be done by using the -azure-blob-access-tier flag. For example:

$ tusd -azure-storage=my-test-container -azure-blob-access-tier=cool

Storage format

Uploads are stored using multiple objects:

  • An informational object with the .info extension holds meta information about the uploads, as described in the section for all storage backends.
  • A file object will contain the uploaded file. Data is appended to the object while the upload is performed.

By default, the objects are stored at the root of the container. For example the objects for the upload ID abcdef123 will be:

  • abcdef123.info: Informational object
  • abcdef123: File object

Testing with Azurite

With Azurite, a local Azure Blob Storage service can be emulated for testing tusd without using the Azure services in the cloud. To get started, please install Azurite (installation instructions) and the Azure CLI (installation instructions). Next, start the local Azurite application:

$ azurite --location ./azurite-data

Azurite provides Blob Storage at http://127.0.0.1:10000 by default and saves the associated data in ./azurite-data. For testing, you can use the well-known storage account devstoreaccount1 and its key.

Next, create a container called mycontainer using the Azure CLI:

$ az storage container create --name mycontainer --connection-string "DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;"

Azurite is now set up, and we can start tusd:

$ AZURE_STORAGE_ACCOUNT=devstoreaccount1 AZURE_STORAGE_KEY=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw== ./tusd -azure-storage=mycontainer -azure-endpoint=http://127.0.0.1:10000

Tusd is then usable at http://localhost:8080/files/ and saves the uploads to the local Azurite instance.