Using S3 to upload files with serverless
Here's how you can use S3 to upload files with serverless, and get urls to the outputs as files
Uploading to an S3 bucket has 2 major steps
- Writing python code to upload files to S3
- Setting the environment variables for your S3 bucket
Uploading to an S3 bucket with python
-
For the example code, we presume 2 things
- you've installed the runpod python library with
pip install runpod
- you've an image file at
./image.png
in the docker container
- you've installed the runpod python library with
-
Now in your handler function, here's some example code that can upload to an s3 bucket, the code uploads the image
image.png
to an s3 bucket, and returns a url to the imagefrom runpod.serverless.utils import rp_upload import runpod def handler(job): image_url = rp_upload.upload_image(job['id'], "./image.png") return [image_url] runpod.serverless.start({"handler": handler})
-
now package your code in a similar manner as described by the Worker Image Creation and Template Creation steps
Setting the environment variables
-
You can set environment variables for a pod via the template creation/editing interface, the Environment Variables section is in the very bottom
-
Set the following environment variables
-
You've to set the variables
BUCKET_ENDPOINT_URL
BUCKET_ACCESS_KEY_ID
- and
BUCKET_SECRET_ACCESS_KEY
ensure your
BUCKET_ENDPOINT_URL
has the bucket name at the start (for example,https://your-bucket-name.nyc3.digitaloceanspaces.com
orhttps://your-bucket.s3.us-west-004.backblazeb2.com
)
Testing your api out
-
Now when you access your api you should see the image as an output uploaded to s3, here's a sample input
Editors note : your request must contain an input key, and it must be a json item, so ensure you put that, a sample request has been provided belowimport requests endpoint = "https://api.runpod.ai/v2/xxxxxxxxx/run" headers = { "Content-Type": "application/json", "Authorization": "Bearer XXXXXXXXXXXXX" } # Define your inputs # an input value must be present, even if it is unused, # and it must be a json value input_data = { "input": {"inp":"this is an example input"} } response = requests.post(endpoint, json=input_data, headers=headers) json = response.json() # the json will be similar to # {'id': 'e3d2e250-ea81-4074-9838-1c52d006ddcf', 'status': 'IN_QUEUE'}
-
Here's an example output request, with the image in output
response = requests.get("https://api.runpod.ai/v2/xxxxxxxxx/status/" + json['id'], headers=headers) response.json()
Here's an example response, after the request completes
{ 'delayTime': 86588, 'executionTime': 1563, 'id': 'e3d2e250-ea81-4074-9838-1c52d006ddcf', 'output': ['https://your-bucket.s3.us-west-004.backblazeb2.com/your-image.png'], 'status': 'COMPLETED' }
Updated about 1 month ago