Product updates

​

Flash beta: Run Python functions on cloud GPUs

from runpod_flash import Endpoint, GpuType

@Endpoint(
    name="hello-gpu", 
    gpu=GpuType.NVIDIA_GEFORCE_RTX_4090,
    dependencies=["torch"]
) 
async def hello():  # This function runs on Runpod
    import torch
    gpu_name = torch.cuda.get_device_name(0)
    print(f"Hello from your GPU! ({gpu_name})")
    return {"gpu": gpu_name}

asyncio.run(hello())
print("Done!") # This runs locally

• Remote execution: Mark functions with @Endpoint to run on GPUs/CPUs automatically.
• Auto-scaling: Workers scale from 0 to N based on demand.
• Dependency management: Packages install automatically on remote workers.
• Two patterns: Queue-based endpoints for batch work, load-balanced endpoints for REST APIs
• Flash apps: Build production-ready APIs with flash init, flash dev, and flash deploy

​

Flash: Multi-datacenter deployments

​

New Public Endpoints and expanded examples

• Video: SORA 2 and SORA 2 Pro, Kling v2.1 and v2.6 Motion Control, WAN 2.6.
• Image: Seedream 4.0.
• Text: Qwen3 32B, IBM Granite 4.0.
• Audio: Chatterbox Turbo for text-to-speech.

• Vercel AI SDK integration: New @runpod/ai-sdk-provider package for TypeScript projects with streaming, text generation, and image generation support.
• AI coding tools guide: Configure OpenCode, Cursor, and Cline to use Runpod Public Endpoints as your model provider.

• Build a text-to-video pipeline: Chain multiple Public Endpoints to generate videos from text prompts.
• Deploy cached models: Reduce cold start times with model caching.
• Integrate Serverless with web applications: Build a complete image generation app.
• Build a chatbot with Gemma 3: Deploy vLLM with OpenAI API compatibility.
• Run Ollama on Pods: Set up Ollama for LLM inference.
• Build Docker images with Bazel: Containerize your applications.

​

GitHub release rollback GA and load balancing Serverless repos in beta

• GitHub release rollback: Roll back your Serverless endpoint to any previous build from the console. Restore an earlier version when you encounter issues without waiting for a new GitHub release.
• Load balancing Serverless repos (beta): Load balancing endpoints are now available in the Hub. Publish or convert any listing to load balancer type by setting "endpointType": "LB" in your hub.json file, then deploy as a Serverless endpoint or Pod from the Hub page. Maintain a single listing for your model and let users choose their deployment method—autoscaling Serverless or dedicated Pod resources.