OpenAI compatibility

A Veri deployment exposes the OpenAI Chat Completions API at https://api.veri.studio/v1/deployments/{deployment_id}. Anything that already speaks OpenAI works — point base_url at the deployment and pass your Veri API key.

Use the full URL https://api.veri.studio/v1/deployments/{dep.id} as base_url. The OpenAI client appends /chat/completions for you. The dep.endpoint_url field on the SDK response is a relative path that already includes /chat/completions — it’s for direct curl-style calls, not for the OpenAI SDK’s base_url.

What works

Basic chat with one or more messages, returning a completion. Works in:

OpenAI Python SDK
OpenAI Node SDK
LangChain (ChatOpenAI(base_url=...))
LlamaIndex (OpenAI(api_base=...))
Vercel AI SDK
curl with the standard OpenAI request shape
Anything else that targets /chat/completions

from openai import OpenAI

oai = OpenAI(
    base_url=f"https://api.veri.studio/v1/deployments/{dep.id}",
    api_key="vk_your_api_key",
)

response = oai.chat.completions.create(
    model=dep.name,
    messages=[
        {"role": "system", "content": "You are a helpful math tutor."},
        {"role": "user", "content": "What is 12 + 7?"},
    ],
    temperature=0.7,
    max_tokens=256,
)

print(response.choices[0].message.content)
print(response.usage)

What’s NOT supported (yet)

Today’s chat surface is intentionally minimal. If your client passes any of these, they’re either silently ignored or rejected:

Streaming (stream=true) — accepted by the schema but ignored. You always get the full response. Clients that wait for SSE events will hang.
Tool calling (tools, tool_choice) — not supported. Use prompting + structured output parsing instead.
Sampling parameters beyond temperature and max_tokens: top_p, frequency_penalty, presence_penalty, logit_bias are not supported.
Structured outputs (response_format with JSON schema) — not supported.
Vision (image inputs) — not supported.
Audio — not supported.

Streaming and tools are the highest-priority items on the roadmap.

Request shape

{
  "model": "your-deployment-name",
  "messages": [
    {"role": "user", "content": "Hello"}
  ],
  "temperature": 0.7,
  "max_tokens": 256,
  "stream": false
}

The model field in the request is echoed back in the response — it’s not validated against the deployment’s actual model. Pass whatever string you want.

Response shape

{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1715000000,
  "model": "your-deployment-name",
  "choices": [
    {
      "index": 0,
      "message": {"role": "assistant", "content": "Hello back!"},
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 5,
    "total_tokens": 15
  }
}

finish_reason values: stop (normal end), length (hit max_tokens).

Scaled-to-zero deployments

If the deployment scaled to zero, the first request triggers a wake and returns 503 with a Retry-After header and error code deployment_waking while the GPU boots (a few minutes). The OpenAI client’s built-in retries give up well before that, so either retry on 503 yourself until the cold start completes, or pre-warm with veri deployments wake <id> before sending traffic. The Veri SDK’s chat() handles this retry loop for you.

LangChain example

from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage

llm = ChatOpenAI(
    base_url=f"https://api.veri.studio/v1/deployments/{dep.id}",
    api_key="vk_your_api_key",
    model=dep.name,
    temperature=0.7,
)

result = llm.invoke([HumanMessage(content="What is 12 + 7?")])
print(result.content)

LlamaIndex example

from llama_index.llms.openai import OpenAI as LlamaOpenAI

llm = LlamaOpenAI(
    api_base=f"https://api.veri.studio/v1/deployments/{dep.id}",
    api_key="vk_your_api_key",
    model=dep.name,
    temperature=0.7,
)

print(llm.complete("What is 12 + 7?"))

curl

curl https://api.veri.studio/v1/deployments/$DEPLOYMENT_ID/chat/completions \
  -H "Authorization: Bearer vk_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-deployment-name",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

What about embeddings?

Not supported — Veri serves chat completions only, not embeddings. Use a dedicated embeddings provider or run a local model for that surface.

Where to go next

Deployments API

create / chat / stop endpoint reference.

Deployment analytics demo

Planned walkthrough for request, token, latency, and replica metrics.

Hosting & GPU sizing

Pick the right GPU for the model.

Quickstart

First-time deployment in ~10 minutes.

Get started

Training

Deployments

Evaluations

CLI

Volumes

What works

What’s NOT supported (yet)

Request shape

Response shape

Scaled-to-zero deployments

LangChain example

LlamaIndex example

curl

What about embeddings?

Where to go next

Deployments API

Deployment analytics demo

Hosting & GPU sizing

Quickstart

​What works

​What’s NOT supported (yet)

​Request shape

​Response shape

​Scaled-to-zero deployments

​LangChain example

​LlamaIndex example

​curl

​What about embeddings?

​Where to go next

Deployments API

Deployment analytics demo

Hosting & GPU sizing

Quickstart

What works

What’s NOT supported (yet)

Request shape

Response shape

Scaled-to-zero deployments

LangChain example

LlamaIndex example

curl

What about embeddings?

Where to go next