Flash beta: Run Python functions on cloud GPUs

Flash is now in public beta. Flash is a Python SDK that lets you run functions on Runpod Serverless GPUs with a single decorator:

Key features

• Remote execution: Mark functions with @Endpoint to run on GPUs/CPUs automatically.
• Auto-scaling: Workers scale from 0 to N based on demand.
• Dependency management: Packages install automatically on remote workers.
• Two patterns: Queue-based endpoints for batch work, load-balanced endpoints for REST APIs
• Flash apps: Build production-ready APIs with flash init, flash run, and flash deploy

Get started:

New Public Endpoints and expanded examples

New Public Endpoints

Expansion of available models across all categories.

• Video: SORA 2 and SORA 2 Pro, Kling v2.1, v2.6 Motion Control, WAN 2.6.
• Image: Seedream 4.0.
• Text: Qwen3 32B, IBM Granite 4.0.
• Audio: Chatterbox Turbo for text-to-speech.

New integrations and guides

• Vercel AI SDK integration: New @runpod/ai-sdk-provider package for TypeScript projects with streaming, text generation, and image generation support.
• AI coding tools guide: Configure OpenCode, Cursor, and Cline to use Runpod Public Endpoints as your model provider.

New tutorials

• Build a text-to-video pipeline: Chain multiple Public Endpoints to generate videos from text prompts.
• Deploy cached models: Reduce cold start times with model caching.
• Integrate Serverless with web applications: Build a complete image generation app.
• Build a chatbot with Gemma 3: Deploy vLLM with OpenAI API compatibility.
• Run Ollama on Pods: Set up Ollama for LLM inference.
• Build Docker images with Bazel: Containerize your applications.

GitHub release rollback GA and load balancing Serverless repos in beta

• GitHub release rollback: Roll back your Serverless endpoint to any previous build from the console. Restore an earlier version when you encounter issues without waiting for a new GitHub release.

• Load balancing Serverless repos (beta): Load balancing endpoints are now available in the Hub. Publish or convert any listing to load balancer type by setting "endpointType": "LB" in your hub.json file, then deploy as a Serverless endpoint or Pod from the Hub page. Maintain a single listing for your model and let users choose their deployment method—autoscaling Serverless or dedicated Pod resources.

Slurm Clusters GA, cached models in beta, and new Public Endpoints available

• Slurm Clusters are now generally available: Deploy production-ready HPC clusters in seconds. These clusters support multi-node performance for distributed training and large-scale simulations with pay-as-you-go billing and no idle costs.
• Cached models are now in beta: Eliminate model download times when starting workers. The system places cached models on host machines before workers start, prioritizing hosts with your model already available for instant startup.
• New Public Endpoints available: WAN 2.5 combines image and audio to create lifelike videos, while Nano Banana merges multiple images for composite creations.

Port labeling, price drops, Runpod Hub, and Tetra beta test

• Port labeling: Name exposed ports in the UI and API to help team members identify services like Jupyter or TensorBoard.
• Price drops: Additional price reductions on popular GPU SKUs to lower training and inference costs.
• Runpod Hub: A curated catalog of one-click endpoints and templates for deploying community projects without starting from scratch.
• Tetra beta test: A Python library for running code on GPU with Runpod. Add a @remote() decorator to functions that need GPU power while the rest of your code runs locally.