Cache files to a network volume

​

Requirements

• A network volume attached to your endpoint, mounted at /runpod-volume. The mirror is stored on the volume, so every operation is a safe no-op when no volume is mounted.
• The Runpod Python SDK installed in your worker image.
• An endpoint namespace to scope the mirror. On Serverless this defaults to RUNPOD_ENDPOINT_ID, which Runpod sets automatically.

​

Usage

import runpod
from runpod.serverless import VolumeCache

def handler(job):
    ...

# Hydrate from the volume, download only what's missing, then sync back
with VolumeCache(dirs=["/root/.cache/huggingface"]):
    model = load_model()   # downloads land in the cached directory

runpod.serverless.start({"handler": handler})

vc = VolumeCache(dirs=["/data/models"], namespace="my-model-cache")

vc.hydrate()                 # restore cached files (for example, at startup)
model = load_model()         # populate the cache
vc.sync(background=False)     # persist new files back to the volume, inline

​

Constructor

​

How it works

• Size-bucketed mirror: Cached files live at {volume_path}/.cache/{namespace} on the volume. Files below 256 KiB are packed into a single small.tar archive, which collapses the per-file metadata round-trips that make many small files slow on a network volume. Larger files are copied unpacked into a big/ subdirectory, which preserves their original relative paths so the large-file subtree stays browsable. A versioned manifest.json, written last, records the size and modification time of every cached file and marks the mirror as complete. A mirror without a valid, current manifest is treated as absent, so a sync self-heals.
• Incremental large files, whole-archive small files: Large-file transfers are diffed per file against the manifest, so unchanged files are skipped. The small.tar archive is repacked whole whenever any small file changes, since unpacking and re-diffing many tiny files individually is slower than the volume’s per-file overhead.
• Parallel copy: Large-file transfers run across a thread pool sized by max_workers. The work is I/O-bound, so the default oversubscribes the CPU count.
• Safety: Symbolic links are never followed or copied. Every archive member and every large-file destination is checked to resolve inside one of your configured dirs before it is written, so a mirror entry can’t write outside your cached directories.

​

Limitations

• Concurrent cold-start write amplification: If several workers cold-start at the same time, each may miss the still-empty mirror, download the model, and sync a full copy back. There’s no coordination between concurrent syncs, so the mirror reflects whichever worker synced most recently.
• Background sync on short-lived processes: sync() schedules the copy on a daemon thread. If the process exits without a normal interpreter shutdown (for example, os._exit or SIGKILL), the exit hook never runs and the sync may not complete.
• Orphaned large files aren’t pruned: If you delete or rename a large file locally, its copy under big/ stays on the volume. Hydration is manifest-driven and ignores it, but volume usage grows as you swap model versions.

​

Next steps

• Optimize your endpoints: Combine the volume cache with other strategies to reduce cold starts.
• Cached models: Use Runpod’s platform-level model cache for Hugging Face models.
• Storage options: Compare container disks, network volumes, and S3-compatible storage.