Storage Fine Tuning

This page is for sysadmins who want to tune where OxiCloud places upload data on disk and why the placement matters for throughput and memory. The defaults work; the gains from a tuned layout are significant on busy instances or constrained containers.

The upload lifecycle in 30 seconds

        ┌─── direct (single-PUT / multipart) ───┐
client ─┤                                       │    Streamed DIRECTLY into the
        │                                       ├──► content-addressable store:
        │                                       │    CDC chunking + BLAKE3 +
        │                                       │    dedup checks happen while
        │                                       │    the bytes arrive. No spool
        │                                       │    file, no re-read; chunks
        │                                       │    the store already has are
        │                                       │    never written at all.
        │                                       │
        └─── multi-chunk upload ────────────────┤    Chunk PARTS accumulate on
             (`/api/uploads` /                  │    disk under OXICLOUD_CHUNK_DIR
              `/dav/uploads/...`)               │    until /complete, which
                                                │    streams them (in order)
                                                ▼    through the same CDC path.
                                  ┌─────────────────────────┐
                                  │ STORAGE BACKEND          │
                                  │  • local FS (.blobs/)    │
                                  │  • S3-compatible         │
                                  │  • Azure Blob            │
                                  └─────────────────────────┘

Two practical consequences:

• Direct uploads no longer use a spool directory. Each uploaded byte is written at most once — straight into the blob backend as a CDC chunk. Re-uploads and edited files write only the chunks the store doesn't already have.
• The chunk-session directory sees write-heavy churn during multi-chunk uploads — fast disk (NVMe) and sufficient free space matter more here than on the final storage backend.

Upload size caps — what each one bounds

Three independent caps control how large an upload OxiCloud will accept. Pick them with disk sizing in mind: the chunk directory must be able to hold the worst case (cap × concurrent sessions).

Variable	Default	What it caps	When it fires
OXICLOUD_MAX_UPLOAD_SIZE	10 GB	Whole-file ceiling. Applies to both direct PUT (per-body) and chunked uploads (declared total_size). The absolute upper bound on any single file in OxiCloud.	Chunked: at POST /api/uploads against the JSON-declared total_size, before any chunk is uploaded. Direct PUT: indirectly via OXICLOUD_DIRECT_PUT_MAX_BYTES, which is expected to be ≤ OXICLOUD_MAX_UPLOAD_SIZE.
OXICLOUD_DIRECT_PUT_MAX_BYTES	1 GiB	Non-chunked PUT body. Per-request cap for PUT /webdav/... and PUT /remote.php/dav/files/.../.... Set below OXICLOUD_MAX_UPLOAD_SIZE so larger files are pushed onto the chunked protocol — which is resumable on failure.	During body streaming, as a per-frame accumulator. Excess → 413 with a "use chunked upload" hint.
OXICLOUD_CHUNK_MAX_BYTES	100 MB	Per-chunk body in a chunked-upload session (PATCH /api/uploads/{id} or PUT /remote.php/dav/uploads/.../chunk). Independent of the whole-file cap — a 5 GB file in 100 MB chunks is 50 PATCHes each bounded by this.	During chunk-body streaming. Excess → 413.

Recommendation: prefer chunked uploads for large files

The defaults (OXICLOUD_DIRECT_PUT_MAX_BYTES = 1 GiB, well below OXICLOUD_MAX_UPLOAD_SIZE = 10 GB) are deliberately asymmetric. Files between those two caps can only succeed via the chunked protocol. The reason is resilience: a direct PUT at 95 % of 5 GB that drops loses everything (the partially ingested chunks are reclaimed by GC, but the client must restart from byte 0). The same drop on a chunked upload loses one ~5 MB chunk; the client retries that chunk and continues. NextCloud desktop and the OxiCloud web UI already switch to chunked at ~10 MB (CHUNKED_UPLOAD_THRESHOLD).

Disk sizing

OxiCloud streams bodies frame-by-frame, so RAM is bounded (~10 MB per in-flight upload for the CDC ingest buffers) regardless of the caps. Disk space scales with the caps:

• Direct PUT / multipart: no transient spool. Bytes land directly in the blob backend as deduplicated chunks; worst-case extra disk per upload is the file's own (deduplicated) size — the same space the stored file occupies afterwards.
• Chunked upload: each in-flight session accumulates its chunk parts under OXICLOUD_CHUNK_DIR until /complete streams them into the blob store and the session is cleaned up. Worst case disk per session = file_size (the parts); total = OXICLOUD_MAX_UPLOAD_SIZE × concurrent_chunked_sessions.

Settings	Chunked worst case (5 sessions)	Safe on 4 GB volume?
Defaults: OXICLOUD_MAX_UPLOAD_SIZE=10 GB	50 GB	❌ overflows
OXICLOUD_MAX_UPLOAD_SIZE=500 MB	2.5 GB	✅ fits

Don't put OXICLOUD_CHUNK_DIR on tmpfs

In many container setups the OS temp dir is tmpfs — RAM-backed storage that counts against the cgroup memory limit. A few concurrent multi-GB chunked sessions on tmpfs will wake the OOMKiller long before the uploads finish. Point OXICLOUD_CHUNK_DIR at a real-disk directory in containers.

TL;DR

Variable	Default	Purpose
OXICLOUD_STORAGE_PATH	./storage	Where .blobs/ lives (the canonical content store)
OXICLOUD_CHUNK_DIR	{STORAGE_PATH}/.uploads	Where chunked-upload sessions accumulate

The two rules that matter most:

1. Keep OXICLOUD_CHUNK_DIR off tmpfs (the default in many containers) — chunk parts count against the cgroup memory limit and can trigger OOMKill on multi-GB uploads.
2. NVMe for the chunk dir pays off on deployments with heavy large-file traffic: each chunk PUT writes a file and the progress bitmap, and /complete reads them all back in order.

Where each upload surface writes

Surface	Default destination	Configurable via
REST chunked PUT (PATCH /api/uploads/{id})	{STORAGE_PATH}/.uploads/{upload_id}/chunk_NNNNNN	OXICLOUD_CHUNK_DIR
NextCloud chunked PUT (PUT /dav/uploads/.../chunk)	{STORAGE_PATH}/.uploads/nextcloud/{user}/{upload_id}/{chunk_name}	OXICLOUD_CHUNK_DIR
Direct PUT / multipart / WOPI / chunked /complete	straight into the blob backend (CDC chunks)	OXICLOUD_STORAGE_PATH (local backend)
Final blob storage	{STORAGE_PATH}/.blobs/{ab}/{abc…}.blob	OXICLOUD_STORAGE_PATH

The local blob backend stages each chunk write under {STORAGE_PATH}/.dedup_temp/ and promotes it with an atomic rename(2) — both directories live under OXICLOUD_STORAGE_PATH, so same-filesystem placement (and therefore atomic promotion) is automatic and not separately configurable.

Recommended layouts

Single-disk box (most common)

Defaults are fine:

OXICLOUD_STORAGE_PATH=/var/lib/oxicloud
# OXICLOUD_CHUNK_DIR unset → /var/lib/oxicloud/.uploads

Container with constrained memory

Critical: make sure the chunk dir doesn't sit on tmpfs.

OXICLOUD_STORAGE_PATH=/data
OXICLOUD_CHUNK_DIR=/data/.uploads

Split-disk (NVMe intake + HDD blobs)

OXICLOUD_STORAGE_PATH=/mnt/hdd/oxicloud      # .blobs/ + .dedup_temp/
OXICLOUD_CHUNK_DIR=/mnt/nvme/oxi-chunks

Chunk parts land on NVMe (fast PUTs, fast /complete read-back); the deduplicated chunks are written once to the HDD-backed blob store as /complete streams through them.

Sharing the chunk directory

The REST and NC chunked surfaces can share OXICLOUD_CHUNK_DIR by design. Each writer tags its output so they never interfere:

Writer	On-disk name pattern
REST chunked sessions	oxi-chunk-{uuid}/ — directories with a well-known prefix
NC chunked subtree	nextcloud/{user}/{uuid}/ — under its own root subdir

The 24-hour orphan-session cleanup loop filters strictly on the oxi-chunk- prefix, so it can NEVER delete a non-OxiCloud directory that happens to live alongside chunked sessions.

Quick verification

Boot the server with RUST_LOG=info and the first lines after the banner include:

oxicloud: Upload limits loaded from config max_upload_size_mb=10240 chunk_max_bytes_mb=100

That confirms the upload-cap env vars were read. To confirm directory placement, watch for chunk file creation under your OXICLOUD_CHUNK_DIR (or its default {STORAGE_PATH}/.uploads/) during a chunked upload — ls while a sync is in progress shows the {uuid}/chunk_NNNNNN files appearing in real time.