Llama2 13B Chat

Retrieve Results & Status

Note: For information on how to check job status and retrieve results, please refer to our Status Endpoint Documentation.

Streaming Token Outputs

Make a POST request to the /llama2-13b-chat/run API endpoint.
Retrieve the job ID.
Make a GET request to /llama2-13b-chat/stream/{job-id} to retrieve the real-time token output.

Here's a code sample in Python:

import requests
import json
import time

url = f"https://api.runpod.ai/v2/llama2-13b-chat/run"

headers = {
  "Authorization":"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
  "Content-Type": "application/json"
}

prompt = """
Write me an essay about how the french revolution impacted the rest of europe over the 18th century. 
"""

payload = {
  "input": {
    "prompt": prompt,
    "sampling_params": {
      "max_tokens": 1000,
      "n": 1,
      "presence_penalty": 0.2,
      "frequency_penalty": 0.7,
      "temperature": 0.3,
    }
  }
}

response = requests.post(url, headers=headers, json=payload)
response_json = json.loads(response.text)
status_url = f"https://api.runpod.ai/v2/llama2-13b-chat/stream/{response_json['id']}"


for i in range(10):
  time.sleep(1)
  get_status = requests.get(status_url, headers=headers)
  print(get_status.text) 

	# example output from print(get_status.text)
	"""{"status":"IN_PROGRESS","stream":[{"output":{"text":["\nWrite me an essay about how the french revolution impacted the rest of europe over the 18th century. \n\nThe French Revolution, which began in 1789 and lasted for over a decade, had a profound impact on Europe in the late 18th century. The revolution, which was sparked by economic hardship, political corruption, and social inequality, led to the overthrow of the French monarchy and the establishment of a new political order. This essay will examine how the French Revolution impacted the rest of Europe during this period.\nOne of the most significant ways in which the French Revolution impacted"]}},{"output":{"text":["\nWrite me an essay about how the french revolution impacted the rest of europe over the 18th century. \n\nThe French Revolution, which began in 1789 and lasted for over a decade, had a profound impact on Europe in the late 18th century. The revolution, which was sparked by economic hardship, political corruption, and social inequality, led to the overthrow of the French monarchy and the establishment of a new political order. This essay will examine how the French Revolution impacted the rest of Europe during this period.\nOne of the most significant ways in which the French Revolution impacted Europe was"]}},{"output":{"text":["\nWrite me an essay about how the french revolution impacted the rest of europe over the 18th century. \n\nThe French Revolution, which began in 1789 and lasted for over a decade, had a profound impact on Europe in the late 18th century. The revolution, which was sparked by economic hardship, political corruption, and social inequality, led to the overthrow of the French monarchy and the establishment of a new political order. This essay will examine how the French Revolution impacted the rest of Europe during this period.\nOne of the most significant ways in which the French Revolution impacted Europe was through its"]}},{"output":{"text":["\nWrite me an essay about how the french revolution impacted the rest of europe over the 18th century. \n\nThe French Revolution, which began in 1789 and lasted for over a decade, had a profound impact on Europe in the late 18th century. The revolution, which was sparked by economic hardship, political corruption, and social inequality, led to the overthrow of the French monarchy and the establishment of a new political order. This essay will examine how the French Revolution impacted the rest of Europe during this period.\nOne of the most significant ways in which the French Revolution impacted Europe was through its influence on"]}}]}"""

Worker Resources

Source Code Repository