Skip to main content

Generator Handler

RunPod provides a robust streaming feature that enables users to receive real-time updates on job outputs, mainly when dealing with Language Model tasks. We support two types of streaming generator functions: regular generator and async generator.

import runpod
def generator_handler(job):
for count in range(3):
result = f"This is the {count} generated output."
yield result
"handler": generator_handler, # Required
"return_aggregate_stream": True, # Optional, results available via /run

Return aggregate Stream

By default, when a generator handler is running, the fractional outputs will only be available at the /stream endpoint, if you would also like the outputs to be available from the /run and /runsync endpoints you will need to set return_aggregate_stream to True when starting your handler.