> ## Documentation Index > Fetch the complete documentation index at: https://docs.veri.studio/llms.txt > Use this file to discover all available pages before exploring further. # Deployments API > Every endpoint for creating, calling, and managing deployments. Reference for `/v1/deployments/*`. All endpoints require Bearer token auth (`Authorization: Bearer vk_...`). ## Endpoints | Method | Path | Purpose | | ------- | --------------------------------------- | -------------------------------------------------------------------------- | | `POST` | `/v1/deployments` | Create a deployment | | `GET` | `/v1/deployments` | List (paginated, filter by `status`) | | `GET` | `/v1/deployments/{id}` | Get one | | `PATCH` | `/v1/deployments/{id}` | Update scaling settings (replica bounds, concurrency target, idle windows) | | `POST` | `/v1/deployments/{id}/wake` | Pre-warm a `scaled_to_zero` deployment (202, idempotent) | | `POST` | `/v1/deployments/{id}/stop` | Transition to `stopped` | | `POST` | `/v1/deployments/{id}/chat/completions` | OpenAI-compatible chat | | `GET` | `/v1/deployments/{id}/requests` | Request log | | `GET` | `/v1/deployments/{id}/metrics` | Aggregated metrics | ## POST /v1/deployments Create a deployment from a base HF model or a completed training job. **Request schema:** ```json theme={null} { "model": "Qwen/Qwen2.5-0.5B-Instruct", "source": "huggingface", "name": "my-deploy", "gpu": {"gpu_type": "A100-80GB", "gpu_count": 1}, "provider": "aws" } ``` | Field | Required | Notes | | ---------- | -------- | --------------------------------------------------------------------------------------- | | `model` | yes | HF model ID (when `source=huggingface`) or training job ID (when `source=training_job`) | | `source` | yes | `huggingface` or `training_job` | | `name` | yes | Human label; appears in dashboard + as the OpenAI `model` field default | | `gpu` | yes | `{"gpu_type": "...", "gpu_count": 1-8}` | | `provider` | no | Defaults to `aws` (the only serving provider today) | **Validation:** When `source=training_job`, the job must exist, belong to you, and be `completed`. Otherwise `404` or `400`. **Response** (`201`): ```json theme={null} { "object": "deployment", "id": "dep_abc123", "status": "queued", "name": "my-deploy", "model": "Qwen/Qwen2.5-0.5B-Instruct", "source": "huggingface", "endpoint_url": null, "gpu": {"type": "A100-80GB", "count": 1}, "provider": "aws", "total_requests": 0, "error": null, "started_at": null, "stopped_at": null, "created_at": "2026-05-12T...", "updated_at": "2026-05-12T..." } ``` `endpoint_url` populates when status reaches `serving`. ## GET /v1/deployments List your deployments. Filter by `status` (`queued` / `provisioning` / `serving` / `unhealthy` / `scaled_to_zero` / `waking` / `stopped` / `failed`). ```bash theme={null} curl "https://api.veri.studio/v1/deployments?status=serving" \ -H "Authorization: Bearer vk_..." ``` Returns `PaginatedList[DeploymentResponse]`. ## GET /v1/deployments/{id} Single deployment. Same shape as create response, with timestamps populated as it runs. ## PATCH /v1/deployments/{id} Partial update of the scaling settings on a live deployment. Mutable fields: `num_replicas`, `min_replicas`, `max_replicas`, `concurrency_target`, `scale_to_zero_window_seconds`, `idle_delete_after_days`. Anything else (model, gpu, source, engine, provider) is immutable and fails validation. Returns the updated deployment immediately with the new *desired* state; the control plane converges the replica fleet in the background. Compare `ready_replicas` (observed) against `num_replicas` (desired) to watch convergence. ```python theme={null} dep = client.deployments.update(deployment_id, min_replicas=1, max_replicas=4) ``` ## POST /v1/deployments/{id}/wake Pre-warm a `scaled_to_zero` deployment. Returns `202` with the deployment; idempotent (a no-op if it's already up or booting). A chat request to a parked deployment also wakes it: raw HTTP clients receive `503` with a `Retry-After` header and error code `deployment_waking` while the GPU boots, while the Python SDK's `chat()` retries through the wake automatically (see [the chat endpoint](#post-v1deploymentsidchatcompletions)). Fire `wake` early on a predictive signal to hide the cold start entirely. ```python theme={null} client.deployments.wake(deployment_id) ``` ## POST /v1/deployments/{id}/stop Transition to `stopped`. A final billing tick settles the partial-hour spend. Returns the updated deployment. ```python theme={null} final = client.deployments.stop(deployment_id) print(final.status) ``` ## POST /v1/deployments/{id}/chat/completions OpenAI-compatible. See [OpenAI compatibility](/deployments/openai-compat) for the supported subset of fields. **Request:** ```json theme={null} { "model": "my-deploy", "messages": [{"role": "user", "content": "Hello"}], "temperature": 0.7, "max_tokens": 256 } ``` **Response:** ```json theme={null} { "id": "chatcmpl-abc", "object": "chat.completion", "created": 1715000000, "model": "my-deploy", "choices": [ {"index": 0, "message": {"role": "assistant", "content": "..."}, "finish_reason": "stop"} ], "usage": {"prompt_tokens": 10, "completion_tokens": 5, "total_tokens": 15} } ``` ### Waking from zero A chat request to a `scaled_to_zero` deployment triggers its wake and answers `503` with a `Retry-After` header while the GPU boots: ```json theme={null} { "error": { "code": "deployment_waking", "message": "Deployment was scaled to zero and is waking up; retry shortly" } } ``` Raw HTTP clients should retry after `Retry-After` seconds. The Python SDK retries for you: `chat()` sleeps and re-sends until the deployment serves the request or `wake_timeout_s` (default `900`) elapses, calling `on_waking(message)` on each `503` so you can surface progress. Set `wake_timeout_s=0` to fail fast with the `503` instead. ```python theme={null} response = client.deployments.chat( deployment_id, messages=[{"role": "user", "content": "Hi"}], on_waking=lambda msg: print(msg), # optional progress hook wake_timeout_s=900, # 0 = fail fast ) ``` ## GET /v1/deployments/{id}/requests Per-request log: prompt tokens, completion tokens, latency, status code, error (if any). ```bash theme={null} curl ".../v1/deployments/dep_abc/requests?limit=50" -H "Authorization: Bearer vk_..." ``` **Response:** ```json theme={null} { "data": [ { "id": "req_xyz", "deployment_id": "dep_abc", "prompt_tokens": 12, "completion_tokens": 8, "latency_ms": 245.3, "status_code": 200, "error_message": null, "created_at": "2026-05-12T..." } ], "has_more": true } ``` Includes failed requests (`status_code >= 400`) for debugging. ## GET /v1/deployments/{id}/metrics Aggregated counters across all requests: ```json theme={null} { "total_requests": 42, "total_prompt_tokens": 5230, "total_completion_tokens": 1840, "avg_latency_ms": 1240.5, "error_rate": 0.024, "uptime_seconds": 3600.0 } ``` ## State machine ``` queued → provisioning → serving → stopped ↘ ↘ failed failed ↕ unhealthy ``` `unhealthy` is a transient state when health checks fail; it returns to `serving` if the pod recovers. Billing applies during `serving` and `unhealthy` only. ## Worker-only endpoints None — deployments don't have a worker callback API. The serving backend talks to the model server directly via the registered backend interface. ## SDK convenience The SDK wraps the create flow into a `Deployment` dataclass with bound methods: ```python theme={null} dep = client.deployments.create(model="...", source="huggingface", name="...", gpu={...}) dep.wait() # polls until the status settles response = dep.chat([{"role": "user", "content": "Hi"}]) metrics = dep.metrics() recent = dep.requests(limit=20) dep.stop() ``` These all hit the endpoints above. `wait()` returns once the status settles: `serving`, `failed`, `stopped`, or `scaled_to_zero` (a parked deployment won't move again without traffic, so waiting on it would just burn the timeout). `dep.chat()` retries through a wake with the default 15-minute budget; call `client.deployments.chat()` directly to tune `wake_timeout_s` or `on_waking`. ## Related What chat fields are supported. Billing states and idle warning. Pick the right GPU. Planned walkthrough for request, token, latency, and replica metrics.