> ## Documentation Index
> Fetch the complete documentation index at: https://docs.veri.studio/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenAI compatibility

> What works with the OpenAI SDK, LangChain, LlamaIndex, etc. — and what doesn't.

A Veri deployment exposes the OpenAI Chat Completions API at `https://api.veri.studio/v1/deployments/{deployment_id}`. Anything that already speaks OpenAI works — point `base_url` at the deployment and pass your Veri API key.

<Note>
  Use the full URL `https://api.veri.studio/v1/deployments/{dep.id}` as `base_url`. The OpenAI client appends `/chat/completions` for you. The `dep.endpoint_url` field on the SDK response is a relative path that already includes `/chat/completions` — it's for direct curl-style calls, not for the OpenAI SDK's `base_url`.
</Note>

## What works

Basic chat with one or more messages, returning a completion. Works in:

* **OpenAI Python SDK**
* **OpenAI Node SDK**
* **LangChain** (`ChatOpenAI(base_url=...)`)
* **LlamaIndex** (`OpenAI(api_base=...)`)
* **Vercel AI SDK**
* **curl** with the standard OpenAI request shape
* Anything else that targets `/chat/completions`

```python theme={null}
from openai import OpenAI

oai = OpenAI(
    base_url=f"https://api.veri.studio/v1/deployments/{dep.id}",
    api_key="vk_your_api_key",
)

response = oai.chat.completions.create(
    model=dep.name,
    messages=[
        {"role": "system", "content": "You are a helpful math tutor."},
        {"role": "user", "content": "What is 12 + 7?"},
    ],
    temperature=0.7,
    max_tokens=256,
)

print(response.choices[0].message.content)
print(response.usage)
```

## What's NOT supported (yet)

<Warning>
  Today's chat surface is intentionally minimal. If your client passes any of these, they're either silently ignored or rejected:

  * **Streaming** (`stream=true`) — accepted by the schema but ignored. You always get the full response. Clients that wait for SSE events will hang.
  * **Tool calling** (`tools`, `tool_choice`) — not supported. Use prompting + structured output parsing instead.
  * **Sampling parameters** beyond `temperature` and `max_tokens`: `top_p`, `frequency_penalty`, `presence_penalty`, `logit_bias` are not supported.
  * **Structured outputs** (`response_format` with JSON schema) — not supported.
  * **Vision** (image inputs) — not supported.
  * **Audio** — not supported.

  Streaming and tools are the highest-priority items on the roadmap.
</Warning>

## Request shape

```json theme={null}
{
  "model": "your-deployment-name",
  "messages": [
    {"role": "user", "content": "Hello"}
  ],
  "temperature": 0.7,
  "max_tokens": 256,
  "stream": false
}
```

The `model` field in the request is **echoed back** in the response — it's not validated against the deployment's actual model. Pass whatever string you want.

## Response shape

```json theme={null}
{
  "id": "chatcmpl-abc123",
  "object": "chat.completion",
  "created": 1715000000,
  "model": "your-deployment-name",
  "choices": [
    {
      "index": 0,
      "message": {"role": "assistant", "content": "Hello back!"},
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 10,
    "completion_tokens": 5,
    "total_tokens": 15
  }
}
```

`finish_reason` values: `stop` (normal end), `length` (hit `max_tokens`).

## Scaled-to-zero deployments

If the deployment [scaled to zero](/cli/deployments#scale-to-zero), the first request triggers a wake and returns `503` with a `Retry-After` header and error code `deployment_waking` while the GPU boots (a few minutes). The OpenAI client's built-in retries give up well before that, so either retry on `503` yourself until the cold start completes, or pre-warm with `veri deployments wake <id>` before sending traffic. The Veri SDK's `chat()` handles this retry loop for you.

## LangChain example

```python theme={null}
from langchain_openai import ChatOpenAI
from langchain_core.messages import HumanMessage

llm = ChatOpenAI(
    base_url=f"https://api.veri.studio/v1/deployments/{dep.id}",
    api_key="vk_your_api_key",
    model=dep.name,
    temperature=0.7,
)

result = llm.invoke([HumanMessage(content="What is 12 + 7?")])
print(result.content)
```

## LlamaIndex example

```python theme={null}
from llama_index.llms.openai import OpenAI as LlamaOpenAI

llm = LlamaOpenAI(
    api_base=f"https://api.veri.studio/v1/deployments/{dep.id}",
    api_key="vk_your_api_key",
    model=dep.name,
    temperature=0.7,
)

print(llm.complete("What is 12 + 7?"))
```

## curl

```bash theme={null}
curl https://api.veri.studio/v1/deployments/$DEPLOYMENT_ID/chat/completions \
  -H "Authorization: Bearer vk_your_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "your-deployment-name",
    "messages": [{"role": "user", "content": "Hello"}]
  }'
```

## What about embeddings?

Not supported — Veri serves chat completions only, not embeddings. Use a dedicated embeddings provider or run a local model for that surface.

## Where to go next

<CardGroup cols={2}>
  <Card title="Deployments API" icon="play" href="/api-reference/introduction">
    `create` / `chat` / `stop` endpoint reference.
  </Card>

  <Card title="Deployment analytics demo" icon="book" href="/demos/deployment-analytics">
    Planned walkthrough for request, token, latency, and replica metrics.
  </Card>

  <Card title="Hosting & GPU sizing" icon="server" href="/deployments/hosting">
    Pick the right GPU for the model.
  </Card>

  <Card title="Quickstart" icon="rocket" href="/deployments">
    First-time deployment in \~10 minutes.
  </Card>
</CardGroup>