Update deployment settings
Scaling settings converge asynchronously. trace_bodies and trace_retention take effect on the next call, and a PATCH with only those two changes nothing about scaling. Turning capture off pauses the monitors that judge this deployment’s stored bodies until it is back on.
curl --request PATCH \
--url https://api.veri.studio/v1/deployments/{deployment_id} \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data '
{
"num_replicas": 123,
"min_replicas": 123,
"max_replicas": 123,
"concurrency_target": 123,
"scale_to_zero_window_seconds": 123,
"idle_delete_after_days": 123,
"self_heal_attempts": 123,
"trace_bodies": true,
"trace_retention": "<string>"
}
'import requests
url = "https://api.veri.studio/v1/deployments/{deployment_id}"
payload = {
"num_replicas": 123,
"min_replicas": 123,
"max_replicas": 123,
"concurrency_target": 123,
"scale_to_zero_window_seconds": 123,
"idle_delete_after_days": 123,
"self_heal_attempts": 123,
"trace_bodies": True,
"trace_retention": "<string>"
}
headers = {
"Authorization": "Bearer <token>",
"Content-Type": "application/json"
}
response = requests.patch(url, json=payload, headers=headers)
print(response.text)const options = {
method: 'PATCH',
headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
body: JSON.stringify({
num_replicas: 123,
min_replicas: 123,
max_replicas: 123,
concurrency_target: 123,
scale_to_zero_window_seconds: 123,
idle_delete_after_days: 123,
self_heal_attempts: 123,
trace_bodies: true,
trace_retention: '<string>'
})
};
fetch('https://api.veri.studio/v1/deployments/{deployment_id}', options)
.then(res => res.json())
.then(res => console.log(res))
.catch(err => console.error(err));{
"object": "<string>",
"id": "<string>",
"status": "queued",
"name": "<string>",
"model": "<string>",
"source": "<string>",
"created_at": "2023-11-07T05:31:56Z",
"updated_at": "2023-11-07T05:31:56Z",
"source_model_id": "<string>",
"cache_model_id": "<string>",
"endpoint_url": "<string>",
"gpu": {
"type": "<string>",
"count": 123
},
"provider": "<string>",
"byoc_repo": "<string>",
"engine": "<string>",
"vllm_extra_args": [
"<string>"
],
"vllm_image": "<string>",
"startup_profiling": true,
"trace_bodies": true,
"trace_retention": "<string>",
"startup_metrics": "<unknown>",
"num_replicas": 123,
"ready_replicas": 123,
"min_replicas": 123,
"max_replicas": 123,
"concurrency_target": 123,
"scale_to_zero_window_seconds": 123,
"idle_delete_after_days": 123,
"self_heal_attempts": 123,
"cost_per_hour_usd": 123,
"total_cost_usd": 123,
"total_requests": 123,
"error": "<string>",
"failure_code": "<string>",
"last_wake_at": "2023-11-07T05:31:56Z",
"wake_failure_count": 123,
"next_wake_not_before": "2023-11-07T05:31:56Z",
"started_at": "2023-11-07T05:31:56Z",
"stopped_at": "2023-11-07T05:31:56Z"
}Authorizations
Body
PATCH /v1/deployments/{id}: the mutable-field whitelist. Everything else on a deployment (model, source, gpu, engine, provider) is immutable: changing those means creating a new deployment. Sending an immutable field in a PATCH is a validation error instead of a silent no-op.
Desired-state semantics: the PATCH validates + persists the new bounds and returns immediately (desired + observed); the reconciler's scaling pass converges the replica fleet asynchronously. A scaled_to_zero deployment left with min_replicas >= 1 is woken by the PATCH itself (cap-checked); during a first boot the new bounds apply once the deployment serves. An omitted field keeps its stored value; only scale_to_zero_window_seconds and idle_delete_after_days read an explicit null as "clear". The trace fields are not scaling: they take effect on the next call, and a PATCH with only them skips everything above.
Request traces: store this deployment's chat request and response bodies, from the next call on, failed calls included (those free). Requires the traces feature to turn on (never to turn off). Bodies already stored stay until they expire. A PATCH with only trace fields changes nothing about scaling.
Response
The updated deployment (desired state; the reconciler converges the replica fleet asynchronously). A scaled_to_zero deployment left with min_replicas >= 1 is woken by the update (status waking, booting the new floor); during a first boot (provisioning) the new scaling settings apply once it is serving
The URL to call once the deployment serves. Managed deployments
(aws, gcp, digitalocean) get the platform chat URL
{api}/v1/deployments/{id}/chat/completions, authenticated with your
Veri API key; BYOC (huggingface) deployments carry the endpoint URL
under your own HF account. Null until the deployment first serves, and
null while it is parked (scaled_to_zero) until a wake serves again; it
stays set after stop or failure.
Machine-readable reason for the last failure, e.g. watchdog_timeout,
CAPACITY_UNAVAILABLE, replica_lost (the last GPU box was lost
after serving: while 'provisioning' Veri is replacing it, with
'failed' or 'scaled_to_zero' self_heal_attempts was used up or 0), or
the serving agent's oom / model_source_unsupported. Null when no
failure is recorded; cleared with error on an idle park and on wake.
A failed wake re-parks with its code (e.g. wake_boot_failed).
When the most recent wake of this scale-to-zero deployment started: a
chat request to the parked deployment, POST /wake, or a PATCH that
left a parked deployment with min_replicas >= 1. Kept after the
wake serves or the deployment parks again. Null if it never woke.
After a failed wake: the earliest time a chat request triggers the
next wake (backoff of 60s doubling per consecutive failure, up to 1h).
POST /wake and a PATCH that raises min_replicas to 1 or more
bypass it. Null until a wake fails; cleared when a wake reaches
serving. A time in the past means the backoff has elapsed.
curl --request PATCH \
--url https://api.veri.studio/v1/deployments/{deployment_id} \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data '
{
"num_replicas": 123,
"min_replicas": 123,
"max_replicas": 123,
"concurrency_target": 123,
"scale_to_zero_window_seconds": 123,
"idle_delete_after_days": 123,
"self_heal_attempts": 123,
"trace_bodies": true,
"trace_retention": "<string>"
}
'import requests
url = "https://api.veri.studio/v1/deployments/{deployment_id}"
payload = {
"num_replicas": 123,
"min_replicas": 123,
"max_replicas": 123,
"concurrency_target": 123,
"scale_to_zero_window_seconds": 123,
"idle_delete_after_days": 123,
"self_heal_attempts": 123,
"trace_bodies": True,
"trace_retention": "<string>"
}
headers = {
"Authorization": "Bearer <token>",
"Content-Type": "application/json"
}
response = requests.patch(url, json=payload, headers=headers)
print(response.text)const options = {
method: 'PATCH',
headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
body: JSON.stringify({
num_replicas: 123,
min_replicas: 123,
max_replicas: 123,
concurrency_target: 123,
scale_to_zero_window_seconds: 123,
idle_delete_after_days: 123,
self_heal_attempts: 123,
trace_bodies: true,
trace_retention: '<string>'
})
};
fetch('https://api.veri.studio/v1/deployments/{deployment_id}', options)
.then(res => res.json())
.then(res => console.log(res))
.catch(err => console.error(err));{
"object": "<string>",
"id": "<string>",
"status": "queued",
"name": "<string>",
"model": "<string>",
"source": "<string>",
"created_at": "2023-11-07T05:31:56Z",
"updated_at": "2023-11-07T05:31:56Z",
"source_model_id": "<string>",
"cache_model_id": "<string>",
"endpoint_url": "<string>",
"gpu": {
"type": "<string>",
"count": 123
},
"provider": "<string>",
"byoc_repo": "<string>",
"engine": "<string>",
"vllm_extra_args": [
"<string>"
],
"vllm_image": "<string>",
"startup_profiling": true,
"trace_bodies": true,
"trace_retention": "<string>",
"startup_metrics": "<unknown>",
"num_replicas": 123,
"ready_replicas": 123,
"min_replicas": 123,
"max_replicas": 123,
"concurrency_target": 123,
"scale_to_zero_window_seconds": 123,
"idle_delete_after_days": 123,
"self_heal_attempts": 123,
"cost_per_hour_usd": 123,
"total_cost_usd": 123,
"total_requests": 123,
"error": "<string>",
"failure_code": "<string>",
"last_wake_at": "2023-11-07T05:31:56Z",
"wake_failure_count": 123,
"next_wake_not_before": "2023-11-07T05:31:56Z",
"started_at": "2023-11-07T05:31:56Z",
"stopped_at": "2023-11-07T05:31:56Z"
}
