Generator Handler

RunPod provides a robust streaming feature that enables users to receive real-time updates on job outputs, mainly when dealing with Language Model tasks. We support two types of streaming generator functions: regular generator and async generator.

import runpod

def generator_handler(job):
	for count in range(3):
    result = f"This is the {count} generated output."
    yield result 
    
runpod.serverless.start({
  "handler": generator_handler, # Required
  "return_aggregate_stream": True # Optional, results available via /run
})

Return Aggregate Stream

By default, when a generator handler is running, the fractional outputs will only be available at the /stream endpoint, if you would also like the outputs to be available from the /run and /runsync endpoints you will need to set return_aggregate_stream to True when starting your handler.