To get started with Runpod: This tutorial guides you through the steps necessary to modify your OpenAI Codebase for use with a deployed vLLM Worker on Runpod. You will learn to adjust your code to be compatible with OpenAI’s API, specifically for utilizing Chat Completions, Completions, and Models routes. By the end of this guide, you will have successfully updated your codebase, enabling you to leverage the capabilities of OpenAI’s API on Runpod. To update your codebase, you need to replace the following:
  • Your OpenAI API Key with your Runpod API Key
  • Your OpenAI Serverless Endpoint URL with your Runpod Serverless Endpoint URL
  • Your OpenAI model with your custom LLM model deployed on Runpod

from openai import OpenAI
import os

client = OpenAI(
api_key=os.environ.get("RUNPOD_API_KEY"),
base_url="https://api.runpod.ai/v2/${YOUR_ENDPOINT_ID}/openai/v1",
)

response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "Why is Runpod the best platform?"}],
temperature=0,
max_tokens=100,
)
Congratulations on successfully modifying your OpenAI Codebase for use with your deployed vLLM Worker on Runpod! This tutorial has equipped you with the knowledge to update your code for compatibility with OpenAI’s API and to utilize the full spectrum of features available on the Runpod platform.

​
Next Steps