LLM

Generate text using a language model.

POST /llm

LLM

cURL

curl --request POST \
  --url https://dream-gateway.livepeer.cloud/llm \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "messages": [
    {
      "role": "<string>",
      "content": "<string>"
    }
  ],
  "model": "",
  "temperature": 0.7,
  "max_tokens": 256,
  "top_p": 1,
  "top_k": -1,
  "stream": false
}
'

200

{
  "id": "<string>",
  "model": "<string>",
  "created": 123,
  "usage": {
    "prompt_tokens": 123,
    "completion_tokens": 123,
    "total_tokens": 123
  },
  "choices": [
    {
      "index": 123,
      "finish_reason": "",
      "delta": {
        "role": "<string>",
        "content": "<string>"
      },
      "message": {
        "role": "<string>",
        "content": "<string>"
      }
    }
  ]
}

The LLM pipeline is OpenAI API-compatible but does not implement all features of the OpenAI API.

The default Gateway used in this guide is the public Livepeer.cloud Gateway. It is free to use but not intended for production-ready applications. For production-ready applications, consider using the Livepeer Studio Gateway, which requires an API token. Alternatively, you can set up your own Gateway node or partner with one via the ai-video channel on Discord.

Streaming Responses

Ensure your client supports SSE and processes each data: line as it arrives.

By default, the /llm endpoint returns a single JSON response in the OpenAI chat/completions format, as shown in the sidebar.

To receive responses token-by-token, set "stream": true in the request body. The server will then use Server-Sent Events (SSE) to stream output in real time.

Each streamed chunk will look like:

data: {
  "choices": [
    {
      "delta": {
        "content": "...token...",
        "role": "assistant"
      },
      "finish_reason": null
    }
  ]
}

The final chunk will have empty content and "finish_reason": "stop":

data: {
  "choices": [
    {
      "delta": {
        "content": "",
        "role": "assistant"
      },
      "finish_reason": "stop"
    }
  ]
}

Authorizations

Authorization · string · header · required

Bearer authentication header of the form Bearer <token>, where <token> is your auth token.

Body

application/json

messages · LLMMessage · object[] · required

Show child attributes

model · string · default: ""

temperature · number · default: 0.7

max_tokens · integer · default: 256

top_p · number · default: 1

top_k · integer · default: -1

stream · boolean · default: false

Response

200

application/json

Successful Response

id · string · required

model · string · required

created · integer · required

usage · LLMTokenUsage · object · required

Show child attributes

choices · LLMChoice · object[] · required

Show child attributes