Skip to content

Architecture · By Rivtor · Updated

Upload files from FastAPI to S3-compatible object storage

Keep durable uploads in object storage rather than the API container’s filesystem. Validate the authenticated owner and upload limits, use a server-side S3 client configured for the actual provider, and store an object reference only after handling upload and database failure boundaries.

See the Rivtor deployment workflow →

Choose proxy upload or direct upload

An API can receive the file and send it to object storage, or authorize a short-lived direct upload from the browser. Proxy uploads keep one application-controlled path but consume API bandwidth and request capacity. Direct uploads reduce that transfer burden while requiring careful authorization and object validation.

Do not read an unbounded file into memory merely because a framework accepts multipart input. Set limits at the relevant proxy and application boundaries, restrict accepted types and consider scanning where the product requires it.

Configure the provider explicitly

Rivtor documents S3-compatible R2 credentials under S3_* variables. For an illustrative server-side boto3 client:

import os
import boto3

s3 = boto3.client(
    's3',
    endpoint_url=os.environ['S3_ENDPOINT'],
    region_name=os.environ['S3_REGION'],
    aws_access_key_id=os.environ['S3_ACCESS_KEY_ID'],
    aws_secret_access_key=os.environ['S3_SECRET_ACCESS_KEY'],
)

This fragment constructs a client; it does not upload data or implement a secure endpoint. Install boto3 and the framework’s multipart dependency where needed. Verify the current provider’s supported S3 operations rather than assuming full AWS feature parity.

Design object and database consistency

Choose an application-generated object key scoped to its owner. Never let a browser choose an arbitrary privileged key. Store the original display filename separately and avoid relying on it as a unique storage identity.

An object upload can succeed while the database write fails. Use a pending/completed record or cleanup process so orphaned objects are detectable. For downloads, authorize ownership before returning content or a limited URL. Keep storage credentials out of browser configuration and logs.

Verify persistence and failure paths

Upload a small synthetic file, read it back and compare its size or checksum. Restart the API and confirm it remains available through the authorized application path. Test an oversized file, wrong owner and storage failure in a disposable environment. Review Rivtor data limits; a running API container is not a durable filesystem for uploads.

References