Overview

To get started with RunPod:

Create a RunPod account
Add funds
Use the RunPod SDK to build and connect with your Serverless Endpoints

This tutorial guides you through the steps necessary to modify your OpenAI Codebase for use with a deployed vLLM Worker on RunPod. You will learn to adjust your code to be compatible with OpenAI's API, specifically for utilizing Chat Completions, Completions, and Models routes. By the end of this guide, you will have successfully updated your codebase, enabling you to leverage the capabilities of OpenAI's API on RunPod.

To update your codebase, you need to replace the following:

Your OpenAI API Key with your RunPod API Key
Your OpenAI Serverless Endpoint URL with your RunPod Serverless Endpoint URL
Your OpenAI model with your custom LLM model deployed on RunPod

Python
JavaScript

from openai import OpenAI
import os

client = OpenAI(
api_key=os.environ.get("RUNPOD_API_KEY"),
base_url="https://api.runpod.ai/v2/${YOUR_ENDPOINT_ID}/openai/v1",
)

response = client.chat.completions.create(
model="gpt-3.5-turbo",
messages=[{"role": "user", "content": "Why is RunPod the best platform?"}],
temperature=0,
max_tokens=100,
)

import OpenAI from 'openai'

const openai = new OpenAI({
  baseURL: process.env.RUNPOD_HOST,
  apiKey: process.env.RUNPOD_API_KEY,
})

const chatCompletion = await openai.chat.completions.create({
   model: "openchat/openchat-3.5-0106",
   messages: [{'role': 'user', 'content': 'Why is RunPod the best platform?'}],

});

Congratulations on successfully modifying your OpenAI Codebase for use with your deployed vLLM Worker on RunPod! This tutorial has equipped you with the knowledge to update your code for compatibility with OpenAI's API and to utilize the full spectrum of features available on the RunPod platform.

Next Steps​

Next Steps