Local testing - Runpod Documentation

Testing your handler locally before deploying saves time and helps you catch issues early. The Runpod SDK provides multiple ways to test your handler function without consuming cloud resources.

Basic testing

The simplest way to test your handler is by running it directly with test input.

Inline JSON

Pass test input directly via the command line:

python handler.py --test_input '{"input": {"prompt": "Hello, world!"}}'

This runs your handler with the specified input and displays the output in your terminal.

Test file

For more complex or reusable test inputs, create a test_input.json file in the same directory as your handler:

test_input.json

{
  "input": {
    "prompt": "This is a test input from a JSON file"
  }
}

Run your handler without any arguments:

python main.py

The SDK automatically detects and uses the test_input.json file.

If you provide both a test_input.json file and the --test_input flag, the command-line input takes precedence.

Local API server

For more comprehensive testing, start a local API server that simulates your Serverless endpoint. This lets you send HTTP requests to test your handler as if it were deployed. Start the local server:

python handler.py --rp_serve_api

This starts a FastAPI server on http://localhost:8000.

Send requests to the server

Once your local server is running, send HTTP POST requests from another terminal to test your function:

curl -X POST http://localhost:8000/runsync \
     -H "Content-Type: application/json" \
     -d '{"input": {"prompt": "Hello, world!"}}'

The /run endpoint only returns a fake request ID without executing your code, since async mode requires communication with Runpod’s system. For local testing, use /runsync to execute your handler and get results immediately.

Testing concurrency

To test how your handler performs under parallel execution, use the --rp_api_concurrency flag to set the number of concurrent workers. This command starts your local server with 4 concurrent workers:

python main.py --rp_serve_api --rp_api_concurrency 4

When using --rp_api_concurrency with a value greater than 1, your main file must be named main.py for proper FastAPI integration. If your file has a different name, rename it to main.py before running with multiple workers.

Testing concurrent requests

Send multiple requests simultaneously to test concurrency:

for i in {1..10}; do
    curl -X POST http://localhost:8000/runsync \
         -H "Content-Type: application/json" \
         -d '{"input": {}}' &
done

Handling concurrency in your code

If your handler uses shared state (like global variables), use proper synchronization to avoid race conditions:

import runpod
from threading import Lock

counter = 0
counter_lock = Lock()


def handler(event):
    global counter
    with counter_lock:
        counter += 1
        return {"counter": counter}


runpod.serverless.start({"handler": handler})

Debugging

Log levels

Control the verbosity of console output with the --rp_log_level flag:

python handler.py --rp_serve_api --rp_log_level DEBUG

Available log levels:

ERROR: Only show error messages.
WARN: Show warnings and errors.
INFO: Show general information, warnings, and errors.
DEBUG: Show all messages, including detailed debug information.

Enable the debugger

Use the --rp_debugger flag for detailed troubleshooting:

python handler.py --rp_serve_api --rp_debugger

This enables the Runpod debugger, which provides additional diagnostic information to help you troubleshoot issues.

Server configuration

Customize the local API server with these flags:

Port

Set a custom port (default is 8000):

python handler.py --rp_serve_api --rp_api_port 8080

Host

Set the hostname (default is “localhost”):

python handler.py --rp_serve_api --rp_api_host 0.0.0.0

Setting --rp_api_host to 0.0.0.0 allows connections from other devices on the network. This can be useful for testing but may have security implications.

Flag reference

Here’s a complete reference of all available flags for local testing:

Flag	Description	Default	Example
`--rp_serve_api`	Starts the local API server	N/A	`--rp_serve_api`
`--rp_api_port`	Sets the server port	8000	`--rp_api_port 8080`
`--rp_api_host`	Sets the server hostname	”localhost”	`--rp_api_host 0.0.0.0`
`--rp_api_concurrency`	Sets concurrent workers	1	`--rp_api_concurrency 4`
`--rp_log_level`	Controls log verbosity	INFO	`--rp_log_level DEBUG`
`--rp_debugger`	Enables the debugger	Disabled	`--rp_debugger`
`--test_input`	Provides test input as JSON	N/A	`--test_input '{"input": {}}'`

Combined example

You can combine multiple flags to create a customized local testing environment:

python handler.py --rp_serve_api \
    --rp_api_port 8080 \
    --rp_api_concurrency 4 \
    --rp_log_level DEBUG \
    --rp_debugger

This command:

Starts the local API server on port 8080.
Uses 4 concurrent workers.
Sets the log level to DEBUG for maximum information.
Enables the debugger for troubleshooting.

​Basic testing

​Inline JSON

​Test file

​Local API server

​Send requests to the server

​Testing concurrency

​Testing concurrent requests

​Handling concurrency in your code

​Debugging

​Log levels

​Enable the debugger

​Server configuration

​Port

​Host

​Flag reference

​Combined example

Basic testing

Inline JSON

Test file

Local API server

Send requests to the server

Testing concurrency

Testing concurrent requests

Handling concurrency in your code

Debugging

Log levels

Enable the debugger

Server configuration

Port

Host

Flag reference

Combined example