Storage

Flash is now part of the Runpod SDK. This page documents the standalone runpod-flash package, which keeps working but no longer gets new features. For new projects, use the Runpod SDK. To see how Flash commands and imports map to the SDK, see Migrating from Flash.

Flash workers have access to two types of storage: for temporary data and for persistent, sharable data.

Container disk

A container disk provides temporary storage that exists only while a worker is running. Each worker gets its own isolated container disk, with a default size of 64GB for GPU endpoints. You can read and write temporary files to the container disk using standard filesystem operations from within @Endpoint functions. Any file that is not written to a network volume (at /runpod-volume/) is written to the container disk, and will be erased when the worker stops.

Configuring container disk size (GPU-only)

Configure container disk size for GPU endpoints using the template parameter (default: 64GB).

from runpod_flash import Endpoint, GpuType, PodTemplate

@Endpoint(
    name="large-temp-storage",
    gpu=GpuType.NVIDIA_A100_80GB_PCIe,
    template=PodTemplate(containerDiskInGb=100)
)
async def process(data: dict) -> dict:
    # 100GB container disk available
    ...

CPU auto-sizing

CPU endpoints automatically adjust container disk size based on instance limits:

CPU3G and CPU3C instances: vCPU count × 10GB (e.g., 2 vCPU = 20GB)
CPU5C instances: vCPU count × 15GB (e.g., 4 vCPU = 60GB)

If you specify a custom size that exceeds the instance limit, deployment will fail with a validation error.

Network volumes

Network volumes provide persistent storage that survives worker restarts. Each volume is tied to a specific datacenter. Use volumes to share data between endpoint functions or to persist data between runs.

Attaching network volumes

Attach a network volume using the volume parameter. Flash uses the volume name to find an existing volume or create a new one. Specify the datacenter parameter to control where the volume is created:

from runpod_flash import Endpoint, GpuType, DataCenter, NetworkVolume

vol = NetworkVolume(name="model-cache", size=100, datacenter=DataCenter.US_GA_2)

@Endpoint(
    name="persistent-storage",
    gpu=GpuType.NVIDIA_A100_80GB_PCIe,
    datacenter=DataCenter.US_GA_2,
    volume=vol
)
async def process(data: dict) -> dict:
    # Access files at /runpod-volume/
    ...

The size parameter specifies the volume size in GB. Valid values range from 10 to 4096 GB (4 TB). If not specified, size defaults to 100 GB. You can also reference an existing volume by ID:

vol = NetworkVolume(id="vol_abc123")

Multi-datacenter volumes

For endpoints deployed across multiple datacenters, pass a list of volumes (one per datacenter):

from runpod_flash import Endpoint, GpuType, DataCenter, NetworkVolume

volumes = [
    NetworkVolume(name="models-us", size=100, datacenter=DataCenter.US_GA_2),
    NetworkVolume(name="models-eu", size=100, datacenter=DataCenter.EU_RO_1),
]

@Endpoint(
    name="global-inference",
    gpu=GpuType.NVIDIA_A100_80GB_PCIe,
    datacenter=[DataCenter.US_GA_2, DataCenter.EU_RO_1],
    volume=volumes
)
async def process(data: dict) -> dict:
    # Workers in each region access their local volume at /runpod-volume/
    ...

Only one network volume is allowed per datacenter. If you specify multiple volumes in the same datacenter, deployment will fail.

Accessing network volume files

Network volumes mount at /runpod-volume/ and can be accessed like a regular filesystem:

from runpod_flash import Endpoint, GpuType, NetworkVolume

vol = NetworkVolume(name="model-storage")

@Endpoint(
    name="model-server",
    gpu=GpuType.NVIDIA_A100_80GB_PCIe,
    volume=vol,
    dependencies=["torch", "transformers"]
)
async def run_inference(prompt: str) -> dict:
    from transformers import AutoModelForCausalLM, AutoTokenizer

    # Load model from network volume
    # Persists across worker restarts and shared between workers
    model_path = "/runpod-volume/models/llama-7b"
    model = AutoModelForCausalLM.from_pretrained(model_path)
    tokenizer = AutoTokenizer.from_pretrained(model_path)

    # Run inference
    inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
    outputs = model.generate(**inputs, max_length=100)
    text = tokenizer.decode(outputs[0])

    return {"generated_text": text}

Load large models from local disk

If your model is on Hugging Face and your endpoint uses a GPU, try cached models first. They need no network volume or cache code, and you aren’t billed for the download. Set HF_HUB_CACHE to /runpod-volume/huggingface-cache/hub on the endpoint, then attach the model after flash deploy with runpodctl serverless update <endpoint-id> --model-reference https://huggingface.co/<org>/<model>:main. Later deploys keep the reference. Workers wait to start until a host has the model, so compare the whole cold request, including queue time, before you choose between cached models and the volume approach below.

Reading model weights straight from /runpod-volume/ reads them over the network on every cold start. To load from local disk instead, keep the Hugging Face cache on the container disk and wrap the model load in VolumeCache from the Runpod Python SDK. VolumeCache keeps a mirror of the cache directory on the network volume. When a worker starts, it copies the mirror onto the container disk, and after the load it syncs any newly downloaded files back. The first worker downloads the model from Hugging Face, and later cold starts restore it from the volume.

from runpod_flash import Endpoint, GpuType, NetworkVolume

vol = NetworkVolume(name="model-cache", size=100)

@Endpoint(
    name="model-server",
    gpu=GpuType.NVIDIA_A100_80GB_PCIe,
    volume=vol,
    env={"HF_HUB_CACHE": "/opt/hf-cache"},  # Container disk, mirrored by VolumeCache
    dependencies=["torch", "transformers", "runpod>=1.12.0"]
)
class ModelServer:
    def __init__(self):
        from runpod.serverless import VolumeCache
        from transformers import AutoModelForCausalLM, AutoTokenizer

        # Restore the cache from the volume, then sync new files back
        with VolumeCache(dirs=["/opt/hf-cache"]):
            self.model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-7B-Instruct")
            self.tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B-Instruct")

VolumeCache requires runpod 1.12.0 or later. It never fails a job: if any part of the cache fails, the worker logs the error and falls back to a normal cold start, so pass best_effort=False while you set it up. VolumeCache doesn’t copy symbolic links, which the Hugging Face cache uses for its snapshots/ directory, so keep Hugging Face Hub access enabled and let from_pretrained re-link the snapshot on each cold start. Don’t set HF_HUB_OFFLINE=1 on a cache that VolumeCache restores. Size the volume to hold the mirror, which lives under /runpod-volume/.cache/. Size the container disk for the whole model plus headroom too, because the restore and any fallback download both write there. The container disk defaults to 64 GB, so for larger models set template=PodTemplate(containerDiskInGb=...) on the endpoint.

Load-balanced endpoints with storage

from runpod_flash import Endpoint, GpuType, NetworkVolume

vol = NetworkVolume(name="model-storage")

api = Endpoint(
    name="inference-api",
    gpu=GpuType.NVIDIA_GEFORCE_RTX_4090,
    volume=vol,
    workers=(1, 5)
)

@api.post("/generate")
async def generate(prompt: str) -> dict:
    from transformers import AutoModelForCausalLM

    model = AutoModelForCausalLM.from_pretrained("/runpod-volume/models/gpt2")
    # Generate text
    return {"text": "generated"}

@api.get("/models")
async def list_models() -> dict:
    import os
    models = os.listdir("/runpod-volume/models")
    return {"models": models}

Creating and managing network volumes

Network volumes must be created before attaching them to an Endpoint. See Network volumes for detailed instructions.

Get started

Runpod SDK (Beta)

Flash

Serverless

Pods

Public Endpoints

Instant Clusters

Accounts and billing

Organizations (Beta)

Integrations

Hub

Reference

Storage

Container disk

Configuring container disk size (GPU-only)

CPU auto-sizing

Network volumes

Attaching network volumes

Multi-datacenter volumes

Accessing network volume files

Load large models from local disk

Load-balanced endpoints with storage

Creating and managing network volumes

​Container disk

​Configuring container disk size (GPU-only)

​CPU auto-sizing

​Network volumes

​Attaching network volumes

​Multi-datacenter volumes

​Accessing network volume files

​Load large models from local disk

​Load-balanced endpoints with storage

​Creating and managing network volumes

Container disk

Configuring container disk size (GPU-only)

CPU auto-sizing

Network volumes

Attaching network volumes

Multi-datacenter volumes

Accessing network volume files

Load large models from local disk

Load-balanced endpoints with storage

Creating and managing network volumes