Create deployment

POST

deployments

Create deployment

curl --request POST \
  --url https://api.veri.studio/v1/deployments \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "model": "<string>",
  "name": "<string>",
  "gpu": {
    "gpu_type": "<string>",
    "gpu_count": 123
  },
  "source": "<string>",
  "provider": "<string>",
  "engine": "<string>",
  "num_replicas": 123,
  "min_replicas": 123,
  "max_replicas": 123,
  "concurrency_target": 123,
  "scale_to_zero_window_seconds": 123,
  "idle_delete_after_days": 123,
  "cache": true,
  "vllm_extra_args": [
    "<string>"
  ],
  "vllm_image": "<string>",
  "startup_profiling": true,
  "trace_bodies": true,
  "trace_retention": "<string>"
}
'

import requests

url = "https://api.veri.studio/v1/deployments"

payload = {
    "model": "<string>",
    "name": "<string>",
    "gpu": {
        "gpu_type": "<string>",
        "gpu_count": 123
    },
    "source": "<string>",
    "provider": "<string>",
    "engine": "<string>",
    "num_replicas": 123,
    "min_replicas": 123,
    "max_replicas": 123,
    "concurrency_target": 123,
    "scale_to_zero_window_seconds": 123,
    "idle_delete_after_days": 123,
    "cache": True,
    "vllm_extra_args": ["<string>"],
    "vllm_image": "<string>",
    "startup_profiling": True,
    "trace_bodies": True,
    "trace_retention": "<string>"
}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({
    model: '<string>',
    name: '<string>',
    gpu: {gpu_type: '<string>', gpu_count: 123},
    source: '<string>',
    provider: '<string>',
    engine: '<string>',
    num_replicas: 123,
    min_replicas: 123,
    max_replicas: 123,
    concurrency_target: 123,
    scale_to_zero_window_seconds: 123,
    idle_delete_after_days: 123,
    cache: true,
    vllm_extra_args: ['<string>'],
    vllm_image: '<string>',
    startup_profiling: true,
    trace_bodies: true,
    trace_retention: '<string>'
  })
};

fetch('https://api.veri.studio/v1/deployments', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

{
  "object": "<string>",
  "id": "<string>",
  "status": "queued",
  "name": "<string>",
  "model": "<string>",
  "source": "<string>",
  "created_at": "2023-11-07T05:31:56Z",
  "updated_at": "2023-11-07T05:31:56Z",
  "source_model_id": "<string>",
  "cache_model_id": "<string>",
  "endpoint_url": "<string>",
  "gpu": {
    "type": "<string>",
    "count": 123
  },
  "provider": "<string>",
  "byoc_repo": "<string>",
  "engine": "<string>",
  "vllm_extra_args": [
    "<string>"
  ],
  "vllm_image": "<string>",
  "startup_profiling": true,
  "trace_bodies": true,
  "trace_retention": "<string>",
  "startup_metrics": "<unknown>",
  "num_replicas": 123,
  "ready_replicas": 123,
  "min_replicas": 123,
  "max_replicas": 123,
  "concurrency_target": 123,
  "scale_to_zero_window_seconds": 123,
  "idle_delete_after_days": 123,
  "cost_per_hour_usd": 123,
  "total_cost_usd": 123,
  "total_requests": 123,
  "error": "<string>",
  "started_at": "2023-11-07T05:31:56Z",
  "stopped_at": "2023-11-07T05:31:56Z"
}

Authorizations

Authorization

string

header

required

API key with the vk_ prefix. Create one from the dashboard.

Body

application/json

model

string

required

name

string

required

gpu

object

required

Show child attributes

source

string

provider

string | null

engine

string | null

Inference engine: "vllm" (default) or "max" (Modular MAX). Omitted => vllm.

num_replicas

integer<int32> | null

VS-341: number of GPU-box replicas to launch. Omitted => 1, which is byte-for-byte today's single-box behavior. Validated 1..=8 at create. Sugar for min_replicas == max_replicas == N (the CLI --replicas flag).

min_replicas

integer<int32> | null

Scaling foundation (§10): replica-count bounds. min == max => fixed N (autoscaling off — there is no boolean flag). Omitted => both default to num_replicas (fixed N). min = 0 arms scale-to-zero (VS-313): the fleet parks after the idle window and wakes on the next request.

max_replicas

integer<int32> | null

concurrency_target

number<double> | null

Per-replica concurrency setpoint the (VS-342) autoscaler targets: desired = ceil(total_ongoing / concurrency_target), clamped to [min_replicas, max_replicas]. Stored as target_ongoing_requests. Omitted => 8 (throughput-biased; lower it for latency-sensitive chat).

scale_to_zero_window_seconds

integer<int32> | null

VS-313: idle seconds before a min_replicas=0 deployment parks (scaled_to_zero). Omitted => 3600. Floor 300 (Fireworks' convention).

idle_delete_after_days

integer<int32> | null

VS-313: optional GC — a deployment parked at zero with no traffic for this many days goes terminal (stopped). Omitted => kept forever.

cache

boolean | null

HF cache-through (source=huggingface only): also save the downloaded snapshot to the caller's model library so later deploys of this repo start from S3 instead of the HF hub. Best-effort — a failed upload never affects this deployment. Omitted => no caching.

vllm_extra_args

string[] | null

VS-370: extra vLLM CLI flags appended verbatim to the engine command (argv list, e.g. ["--speculative-config", "{...}", "--kv-cache-dtype", "fp8"]). vllm engine only. Flags the platform owns (--model, --port, --host, --api-key, --served-model-name) are rejected at create.

vllm_image

string | null

VS-426: explicit vLLM serving image (e.g. "vllm/vllm-openai:v0.25.1") for architectures newer than the platform default vLLM. Official upstream images only (vllm/vllm-openai[-rocm]:). vllm engine only. Forces the docker launcher on every provider, including AWS. Omitted => platform default.

startup_profiling

boolean

Opt-in vLLM startup diagnostics: the serving agent exports VLLM_LOGGING_LEVEL=DEBUG + VLLM_GC_DEBUG=1 to the engine, records startup phase timings, and reports them on the first healthy heartbeat (or in the failure detail with a deeper stderr tail). Default false — diagnostic logging slows inference, so use for diagnosis launches, not steady customer traffic.

trace_bodies

boolean

Request traces: capture this deployment's chat request/response bodies and make them queryable via /v1/deployments/{id}/traces. Default false (privacy: enabling stores end-user prompt/completion content, retained per trace_retention). Requires the traces feature; billed per 1k traces beyond the monthly free allowance.

trace_retention

string | null

Trace retention class: omitted/null = "base" (14-day retention, base rate), "extended" = 400-day retention at the extended rate. Only meaningful with trace_bodies=true.

Response

The created deployment

object

string

required

string

required

status

enum<string>

required

Available options:

queued,

provisioning,

serving,

unhealthy,

stopped,

failed,

scaled_to_zero,

waking

name

string

required

model

string

required

source

string

required

created_at

string<date-time>

required

updated_at

string<date-time>

required

source_model_id

string | null

For a custom_model deployment, the saved model this was created from (provenance). Also set on a source=huggingface deployment that took the transparent cache hit (serving from the caller's cached S3 copy). Null otherwise.

cache_model_id

string | null

HF cache-through: the library model (status importing) this deployment's box is uploading its snapshot into. Null unless the deployment was created with cache=true and the upload is the one this boot owns.

endpoint_url

string | null

gpu

null | object

Pydantic GPUInfo (with type field). Rust uses gpu_type internally but serializes as "type" for JSON parity.

Show child attributes

provider

string | null

byoc_repo

string | null

BYOC (provider='huggingface'): the HF hub repository this endpoint serves under the owner's account. Null for every non-BYOC deployment.

engine

string | null

vllm_extra_args

string[] | null

VS-370: the extra vLLM CLI flags this deployment was created with (argv list). Null unless supplied at create.

vllm_image

string | null

VS-426: the explicit vLLM serving image this deployment was created with. Null unless supplied at create.

startup_profiling

boolean

Whether this deployment was created with startup diagnostics on.

trace_bodies

boolean

Whether request/response bodies are captured as traces.

trace_retention

string | null

Trace retention class ("extended" or null = base). Null unless trace_bodies deployments set it.

startup_metrics

any

Startup phase timings reported by a profiled deployment's first healthy heartbeat (seconds: model_sync_s, engine_start_to_healthy_s, total_s). Null unless startup_profiling was set and the engine reached healthy.

num_replicas

integer<int32>

VS-341: DESIRED replica count (GPU boxes) behind this deployment. The observed count is ready_replicas — the K8s spec/status split; the CLI renders "ready/desired" (e.g. 2/3).

ready_replicas

integer<int32>

Observed: replicas currently serving with a routable endpoint.

min_replicas

integer<int32>

Replica-count bounds. min == max => fixed N (autoscaling off).

max_replicas

integer<int32>

concurrency_target

number<double> | null

Per-replica concurrency setpoint (stored as target_ongoing_requests).

scale_to_zero_window_seconds

integer<int32> | null

VS-313: idle window before a min=0 deployment parks (None = 3600s).

idle_delete_after_days

integer<int32> | null

VS-313: parked-deployment GC in days (None = kept forever).

cost_per_hour_usd

number<double> | null

total_cost_usd

number<double> | null

total_requests

integer<int32>

error

string | null

started_at

string<date-time> | null

stopped_at

string<date-time> | null

Create deployment

curl --request POST \
  --url https://api.veri.studio/v1/deployments \
  --header 'Authorization: Bearer <token>' \
  --header 'Content-Type: application/json' \
  --data '
{
  "model": "<string>",
  "name": "<string>",
  "gpu": {
    "gpu_type": "<string>",
    "gpu_count": 123
  },
  "source": "<string>",
  "provider": "<string>",
  "engine": "<string>",
  "num_replicas": 123,
  "min_replicas": 123,
  "max_replicas": 123,
  "concurrency_target": 123,
  "scale_to_zero_window_seconds": 123,
  "idle_delete_after_days": 123,
  "cache": true,
  "vllm_extra_args": [
    "<string>"
  ],
  "vllm_image": "<string>",
  "startup_profiling": true,
  "trace_bodies": true,
  "trace_retention": "<string>"
}
'

import requests

url = "https://api.veri.studio/v1/deployments"

payload = {
    "model": "<string>",
    "name": "<string>",
    "gpu": {
        "gpu_type": "<string>",
        "gpu_count": 123
    },
    "source": "<string>",
    "provider": "<string>",
    "engine": "<string>",
    "num_replicas": 123,
    "min_replicas": 123,
    "max_replicas": 123,
    "concurrency_target": 123,
    "scale_to_zero_window_seconds": 123,
    "idle_delete_after_days": 123,
    "cache": True,
    "vllm_extra_args": ["<string>"],
    "vllm_image": "<string>",
    "startup_profiling": True,
    "trace_bodies": True,
    "trace_retention": "<string>"
}
headers = {
    "Authorization": "Bearer <token>",
    "Content-Type": "application/json"
}

response = requests.post(url, json=payload, headers=headers)

print(response.text)

const options = {
  method: 'POST',
  headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
  body: JSON.stringify({
    model: '<string>',
    name: '<string>',
    gpu: {gpu_type: '<string>', gpu_count: 123},
    source: '<string>',
    provider: '<string>',
    engine: '<string>',
    num_replicas: 123,
    min_replicas: 123,
    max_replicas: 123,
    concurrency_target: 123,
    scale_to_zero_window_seconds: 123,
    idle_delete_after_days: 123,
    cache: true,
    vllm_extra_args: ['<string>'],
    vllm_image: '<string>',
    startup_profiling: true,
    trace_bodies: true,
    trace_retention: '<string>'
  })
};

fetch('https://api.veri.studio/v1/deployments', options)
  .then(res => res.json())
  .then(res => console.log(res))
  .catch(err => console.error(err));

{
  "object": "<string>",
  "id": "<string>",
  "status": "queued",
  "name": "<string>",
  "model": "<string>",
  "source": "<string>",
  "created_at": "2023-11-07T05:31:56Z",
  "updated_at": "2023-11-07T05:31:56Z",
  "source_model_id": "<string>",
  "cache_model_id": "<string>",
  "endpoint_url": "<string>",
  "gpu": {
    "type": "<string>",
    "count": 123
  },
  "provider": "<string>",
  "byoc_repo": "<string>",
  "engine": "<string>",
  "vllm_extra_args": [
    "<string>"
  ],
  "vllm_image": "<string>",
  "startup_profiling": true,
  "trace_bodies": true,
  "trace_retention": "<string>",
  "startup_metrics": "<unknown>",
  "num_replicas": 123,
  "ready_replicas": 123,
  "min_replicas": 123,
  "max_replicas": 123,
  "concurrency_target": 123,
  "scale_to_zero_window_seconds": 123,
  "idle_delete_after_days": 123,
  "cost_per_hour_usd": 123,
  "total_cost_usd": 123,
  "total_requests": 123,
  "error": "<string>",
  "started_at": "2023-11-07T05:31:56Z",
  "stopped_at": "2023-11-07T05:31:56Z"
}

Overview

Training jobs

Datasets

Deployments

Volumes

Models

Code artifacts

Regions

Billing

Workspace settings

Integrations

API keys

SSH keys

Webhooks

Authorizations

Body

Response