Agent skill (SKILL.md)

---
name: veri
description: Train, deploy, evaluate, and manage language models on Veri, the AI compute platform (api.veri.studio). Covers the veri CLI and REST API for fine-tuning (SFT, DPO, GRPO), harness-in-the-loop RL (train the model inside your own agent), custom training scripts, GPU inference deployments with OpenAI- and Anthropic-compatible endpoints, datasets, inline reward functions, evals (LLM judges, experiments, live-traffic monitors), agent observability, managed agent sessions, volumes, billing, lifecycle webhooks, Prometheus metrics export, and resource cleanup. Use when the user mentions Veri, veri.toml, the veri CLI, api.veri.studio, or wants to fine-tune / RL-train / serve / evaluate a model on rented GPUs, including RL-training the model that powers an existing agent or harness.
---

# Veri

## Setup — do this first

```bash
uv tool install --upgrade 'veri-sdk[mcp]'   # or: pipx install --force 'veri-sdk[mcp]' / pip install -U 'veri-sdk[mcp]'
veri version                    # need sdk_version >= 0.2.45 (this skill's baseline); Python >= 3.10
veri login                      # browser flow; or: veri login --token vk_... (headless/CI)
veri whoami                     # verify: status must NOT be invalid_or_expired_token
veri doctor                     # diagnose install/network if anything fails
```

If `sdk_version` is older than 0.2.45, upgrade before continuing (older CLIs lack commands
this skill uses, and before 0.2.34 `--reward` took a stored reward id instead of a `.py`
path): re-run the install line above. `uv tool upgrade` and `pipx upgrade` never add the
`mcp` extra, and installing one extra alone removes the others, so name every extra you use
in one install: `'veri-sdk[all]'` is every extra. Pip needs Python 3.10+ in a virtualenv; the stock
macOS `python3` is too old.

Veri rents GPUs for model post-training and serving. One CLI (`veri`), one REST API
(`https://api.veri.studio`, Bearer auth with `vk_...` keys), TOML configs. Job, dataset,
deployment and model ids are bare hex strings (no `job_`/`ds_`/`dep_` prefix); evaluators,
experiments and monitors use `evl_`, `exp_`, `mon_`.

- API keys: https://www.veri.studio/settings/api-keys (dashboard). The CLI can mint and revoke keys too (`veri api-keys`).
- Env vars `VERI_API_KEY` / `VERI_API_URL` override the config file.
- Config file: `~/.config/veri/config.toml` (Linux), `~/Library/Application Support/veri/config.toml` (macOS). `veri whoami` prints the active path.
- Global output flags on every command: `--format json|jsonl|csv|table`, `--quiet`.

Docs: https://docs.veri.studio/quickstart · https://docs.veri.studio/cli

## Account limits — plan around these

Concurrency is capped per workspace. Free plan: **4 in-flight training jobs** (8 on Pro),
**2 active deployment replicas** (4 on Pro), 2 unfinished experiments (4 on Pro),
10 agent sessions (25 on Pro; paused ones count), 10 active triggers (25 on Pro). `GET /v1/me`
reports the workspace's `plan`; `GET /v1/plans` lists every plan's limits. Jobs and
replicas are counted weighted, so a 4-node job consumes 4 job slots and a 2-replica
deployment consumes the whole replica budget.

- In-flight jobs = `queued` + `provisioning` + `configuring` + `running`. Terminal jobs never count.
- Active replicas = `queued` + `provisioning` + `serving` + `waking`. A
  `scaled_to_zero` deployment does NOT count, but it starts counting the moment it wakes.
- Enforced at create, at `veri deployments update` when it raises the replica count, and at
  wake (explicit `veri deployments wake` or a chat request to a `scaled_to_zero` deployment):
  a wake with no headroom returns the same 429 and the deployment stays parked.

Hitting a cap returns **HTTP 429 with a `code`**: `CONCURRENT_JOB_LIMIT`,
`ACTIVE_DEPLOYMENT_LIMIT`, `CONCURRENT_EXPERIMENT_LIMIT` or `CONCURRENT_SESSION_LIMIT`:

```json
{"error":{"code":"CONCURRENT_JOB_LIMIT","message":"This workspace has 4 in-flight training jobs and this submit adds 1 (limit 4). Wait for one to finish, or cancel one with `veri jobs cancel <id>`, then resubmit. Contact us to raise this limit."}}
```

**These 429s are NOT transient: do not retry-loop.** Retrying without first freeing
capacity fails every time. Cancel or stop something, then resubmit. The request rate
limiter (120 requests / 60s per key) is the one 429 to wait out: code `RATE_LIMITED`, a
`retry_after_seconds` field and a `Retry-After` header.

"Workspace" in these messages is the workspace your API key belongs to: your personal
workspace, or a team workspace created or joined (by invite) in the dashboard. Each
workspace is its own ownership and billing boundary, and a key created in the dashboard is
scoped to the workspace that was active when it was created. Raising a limit is a "contact us" lever.

## Project conventions

`veri init <name>` scaffolds `configs/{train,deploy}.toml`, `reward.py`, `AGENTS.md`/`CLAUDE.md` and a `veri.toml` project marker (no config reads defaults from it; each config is self-contained, so check its `gpu_type`/`gpu_count` against the GPU table below). Every config starts with `kind = "train"|"deploy"`; `veri run <config.toml>` dispatches on it. Override any key from the CLI: `--set method.learning_rate=2e-6` (TOML-typed; `--set-string` forces string).

Docs (full config schemas + `--set` rules): https://docs.veri.studio/cli/run

## Datasets

JSONL rows with a `prompt` column (string or OpenAI-style message list) and typically `answer`:

```json
{"prompt": [{"role": "user", "content": "What is 12 + 7?"}], "answer": "19"}
```

Chat/ShareGPT/Alpaca/plain-text/prompt-completion/preference (`prompt`,`chosen`,`rejected` for DPO) formats are auto-detected.

```bash
veri datasets check ./train.jsonl                    # validate locally, no upload
veri datasets upload ./train.jsonl --name my-train   # prints the dataset id
veri datasets connect-hf openai/gsm8k --config main --split train --map question=prompt
```

Private/gated HF datasets are not supported. Uploading byte-identical content again returns the existing dataset (same id and original name).

Docs (formats, column mapping): https://docs.veri.studio/training/datasets · https://docs.veri.studio/cli/datasets

## Reward functions (GRPO)

TRL signature, attached to the job inline as Python source text (no separate upload, no reward ids). It runs inside the trainer process with no per-call timeout: keep it fast and deterministic, and avoid network calls.

```python
def reward(completions, answer, **kwargs) -> list[float]:
    # completions: list of strings OR list of message-dict lists
    ...
```

CLI: `--reward ./reward.py` on `veri jobs create` / `veri run-harness`, or `[reward] file = "./reward.py"`
in train.toml (paths resolve against the config file's directory). The CLI AST-lints the file before
submit (parses as Python + defines a function). API/SDK: `reward_source` = the source text; multiple
rewards: `reward_sources` + optional `reward_weights` (equal lengths). Limits: max 8 sources, 256 KiB
each; violations are 400 at submit. Read a job's submitted source back with
`client.training_jobs.reward_source(job_id, index=0)` or on the job's dashboard page.

Docs (contract, debugging): https://docs.veri.studio/training/reward-functions

## Training

Methods: `grpo` (needs a reward source), `sft_text`, `dpo` (these two must omit the reward), `grpo_harness` (needs a reward source; submit via `veri run-harness`, see the harness section below). `grpo_agentic` is NOT runnable (400, being rebuilt).

```toml
# configs/train.toml
kind = "train"
[job]        # name, output_model
[model]      # base = "Qwen/Qwen3-4B"
[dataset]    # id = "ds_..."
[reward]     # file = "./reward.py"   (grpo only)
[method]     # type = "grpo"; learning_rate = 1e-6; max_steps = 100; rollouts_per_prompt = 4
[resources]  # gpu_type = "L4-24GB"; gpu_count = 1; optional region (aws only)
[provider]   # name = "vast"  (optional; default aws. provider under [resources] is ignored)
```

```bash
veri jobs estimate configs/train.toml            # rough AWS price, nothing submitted; GPU shape/provider are checked only on submit
veri jobs estimate configs/train.toml --minutes 90   # price a duration you know
veri jobs create configs/train.toml --follow     # submit + stream logs (Ctrl+C detaches)
veri jobs logs <id> -f                           # tail later
veri jobs get <id> && veri jobs download <id> --output-dir ./checkpoints
veri jobs cancel <id>
```

A submit that fails validation is rejected before anything is created or billed, so submitting is the real check.

Key hyperparameter defaults: grpo: lr 1e-6, rollouts_per_prompt 8, kl_coef 0.0 (omitted = no KL term), max_steps 100 (one prompt per step; num_epochs is not used), max_response_length 2048; sft_text: lr 2e-5, num_epochs 1; dpo: lr 5e-6, beta 0.1. LoRA via `lora_rank`/`lora_alpha`; QLoRA via `load_in_4bit` (NVIDIA only).

Optional: push the trained artifact to the user's Hugging Face account when the job finishes — requires `veri hf set` once (see the Hugging Face section), then add to the train config:

```toml
[huggingface]
repo = "namespace/name"
artifact = "merged"        # or "adapter" (needs method.lora_rank); required
private = true             # default
```

The finished job carries `hf_repo_url`. Submit fails fast (400) if the account isn't connected or `artifact = "adapter"` lacks `lora_rank` — fix, don't retry.

Docs (per-method guides + full hyperparameter tables): https://docs.veri.studio/training · https://docs.veri.studio/training/grpo · https://docs.veri.studio/training/sft · https://docs.veri.studio/training/dpo · https://docs.veri.studio/cli/training

### GPUs and providers (training)

| Provider | GPUs | Status |
| --- | --- | --- |
| `aws` (default) | L4-24GB, A10G-24GB (1x or 4x); A100-80GB, H100-80GB (whole 8-GPU nodes only: gpu_count 8, you pay for all 8) | live |
| `vast` | A100/H100-80GB (1/2/4/8x marketplace) | live |
| `digitalocean` | H100-80GB (1x; 8x is listed but rarely free), H200-141GB (1x) | live |
| `gcp` | A100-80GB, H100-80GB | coming soon (create returns 400) |

Check live prices/availability: `veri gpu list` (add `--provider`/`--region`/`--gpu-type` to switch from the static catalog to live availability), `veri gpu describe <gpu-type>` for specs and node size, `veri gpu compare --gpu-type H100-80GB`, `veri regions list`.

Docs (per-provider constraints): https://docs.veri.studio/training/providers

### Capacity: rough availability and getting unstuck

Every shape comes from a shared pool. Rough availability (it changes by the minute, so check live with `veri gpu list --provider <p>` before picking):

| Shape | Availability |
| --- | --- |
| `aws` L4-24GB / A10G-24GB x1 ($1.05 / $1.31 per GPU-hour) | Plenty (dozens at once): the default for SFT, DPO, GRPO and serving up to about 4B. |
| `aws` L4-24GB / A10G-24GB x4 (96 GB in total) | Usually free (median 1.6 minutes to a box, no capacity misses in October 2026): the first choice for 4 GPUs, and before `digitalocean` H100 x1 for anything that fits in 4 x 24 GB. Many at once can fill Veri's AWS quota (`QUOTA_EXCEEDED`): retry in a few minutes. |
| `aws` A100-80GB / H100-80GB x8 (whole node, $4.46 / $8.94 per GPU-hour) | Rare: expect `CAPACITY_UNAVAILABLE`. |
| `vast` A100-80GB / H100-80GB x1 to x8 (training only, marketplace price) | Usually free; boots slower (5 to 15 min). H100 x2 is the proven harness-RL shape. |
| `digitalocean` H100-80GB x1 ($5.73) | Flaps: often none free for long stretches. |
| `digitalocean` H200-141GB x1 ($5.81, atl1 / nyc2) | Usually free: one GPU with 141 GB (32B in bf16, 70B in fp8). |

`QUOTA_EXCEEDED` means Veri's own account limit on that provider is full (the AWS vCPU limit shared by every L4 and A10G box, or DigitalOcean's few shared droplets): a platform limit, not the user's account. Nothing moves between providers or GPU types automatically. Work down this list:

1. Read the code, not just the status: `veri jobs get <id> --format json` gives `error.code` and `error.message`; for a deployment `veri deployments get <id> --format json` gives `failure_code` and `error` (match the code in any case).
2. `CAPACITY_UNAVAILABLE` (no free box for that shape anywhere it can launch) or `QUOTA_EXCEEDED` (Veri's account limit on that provider is full): nothing booted and nothing was billed. The message ends with the shapes to launch instead, `--provider P --gpu-type G --gpu-count N; ...` (after `Alternatives:` for capacity, after `or use:` for a quota), the cheapest live shapes with at least as much GPU memory. Resubmit with one: `veri deploy <model> --provider P --gpu-type G --gpu-count N --no-cache`, or `veri jobs create <config> --set provider.name=P --gpu-type G --gpu-count N`.
3. Retry the same shape only a few times: wait about 2, then 5, then 10 minutes, at most 3 tries. After `QUOTA_EXCEEDED`, wait a few minutes or use a shape the message names (it only names other providers).
4. Shrink or move: 4x to 1x (`lora_rank` or `load_in_4bit` to fit), AWS 4x L4 to `vast` H100 x2 for harness RL, a big model to `digitalocean` H200 x1, or for AWS shapes the other region (`--region us-west-2` / `us-east-1`, `veri regions list`).
5. Waiting is normal for a while: `queued` → `provisioning` → `configuring` → `running` takes about 5 minutes on AWS and DigitalOcean and 5 to 15 on Vast. Stuck past about 20 minutes: `veri jobs logs <id>`, then `veri jobs cancel <id>` and resubmit on another shape.
6. Never retry unchanged after `config_error` / `dependency_error`, a 400 "not currently enabled" / "coming soon", a 429 limit code, or a 402. `RATE_LIMITED`: wait `retry_after_seconds`. A chat 503 `deployment_waking`: wait `Retry-After`, resend.

### Multi-GPU training (sft_text, dpo)

`gpu_count` above 1 on `sft_text` / `dpo` runs one process per GPU; the plan is automatic and printed at the top of the job log (`plan: dp=4 zero_stage=3 ...`). Knobs under `[method]` use Ultra-Scale Playbook names: `zero_stage` ("auto" | 0 | 2 | 3), `micro_batch_size` (per GPU; the optimizer batch is `micro_batch_size x gpu_count` unless `global_batch_size` is set), `gradient_accumulation_steps`, `global_batch_size`, `gradient_checkpointing`, `precision` ("bf16" | "bf16_mixed" | "fp32"), `cpu_offload`. Rules of thumb: leave `zero_stage` on "auto"; raise `micro_batch_size` until each GPU is busy (at `micro_batch_size = 1` four PCIe GPUs give no speedup, at 8 about 2.4x on L4 x4); set `global_batch_size` to keep training dynamics fixed when changing `gpu_count`; use `zero_stage = 3` plus `cpu_offload = true` for models that do not fit even when sharded. Not on multi-GPU: `use_unsloth`, `load_in_4bit`, `use_liger` (400 at submit). `grpo` and `grpo_harness` keep their own layouts. Docs: https://docs.veri.studio/training/multi-gpu

### When a training job fails

Read `error.code` first, then the log (`veri jobs logs <id>` or `GET /v1/training_jobs/{id}/logs/full`):

| `error.code` | Do this |
| --- | --- |
| `oom` | Lower `micro_batch_size` / `max_seq_length` (or `max_length`, `max_response_length`); on multi-GPU set `zero_stage = 3`, `precision = "bf16"`, `cpu_offload = true`; or use `lora_rank`. The message names the GPU (`rank 2/4 ...`) and prints the plan. |
| `config_error` | Fix the config; never retry the same payload. |
| `dependency_error` | Fix the harness requirements / optimizer opt-in; never retry unchanged. |
| `distributed_error`, `hardware_error` | Re-submit once; if it repeats, report the job id to Veri (hardware is the platform's fault, never the user's). |
| `upload_failed` | Training finished but the final model did not fully reach Veri storage (the message names the files). Re-submit once; if it repeats, report the job id to Veri. |
| `training_stalled` | The log tail carries every GPU's Python stack; lower `micro_batch_size` or the sequence length, then re-submit. |
| `CAPACITY_UNAVAILABLE`, `QUOTA_EXCEEDED` | Never booted, unbilled. Resubmit with a shape the message names (`Alternatives:` / `or use:`), or retry the same shape at most 3 times (see Capacity above). |
| `provisioning_*`, `configuring_timeout`, `heartbeat_timeout` | Re-submit; try another `gpu_type` or region if it repeats. |
| `worker_failed` | The worker went silent for more than 900 s (OOM-killed worker, lost host) or the box ended before it reported; the message names the silence and the last status and step. Resume from the newest checkpoint (`veri jobs resume <id>`) or re-submit; report the job id if it repeats. |
| `provider_reclaimed` (GCP ended the VM early; credited back), `max_run_duration_reached` (GCP 24 h cap; charged) | Resume from the newest checkpoint as a new job: `veri jobs resume <id>` (or `resume_from="<id>@<step>"`). For the 24 h cap also lower `max_steps`. |

### Custom scripts

```bash
veri run-script ./my-run \
  --entrypoint "torchrun --nproc_per_node=\$VERI_NUM_GPUS train.py" \
  --base-image veri/base --gpu-type L4-24GB --gpu-count 1 \
  --requirements ./my-run/requirements.txt
```

Base images: `veri/base`, `veri/base-vllm`, `veri/base-sglang`, `veri/base-claude-agent`. Env injected: `VERI_DATA_DIR`, `VERI_OUTPUT_DIR` (final artifacts, captured on exit 0 only), `VERI_CHECKPOINT_DIR` (write `checkpoint-<step>/` dirs here mid-run; set when checkpointing is on), `VERI_METRICS_FILE`, `VERI_NUM_GPUS`. NVIDIA only (AMD gpu_type → 400). Multi-node (`--num-nodes` ≥ 2) is not enabled yet (400). Exit 0 = completed.

Liveness: the worker keeps the job alive by itself (a status post at least once a minute), so a quiet script is never failed for silence. Progress and checkpoints still come only from the script: append `{"step": n, ...}` JSON lines to `$VERI_METRICS_FILE` (or `veri_sdk.log_metrics`) for the step and curves, and write checkpoints to `$VERI_CHECKPOINT_DIR` (a run that dies keeps only those). A worker silent for more than 900 s (killed, host lost) fails the job as `worker_failed` with "Worker stopped reporting for N s (last status: training, step S)"; "no step reported" means the script never wrote `$VERI_METRICS_FILE`.

Docs (env contract, images, worked examples): https://docs.veri.studio/training/custom-script

### Harness-in-the-loop RL (train the model inside the user's own agent)

GRPO-trains a policy while the user's EXISTING agent code drives the multi-turn
tool-use rollouts, unmodified. Works with any agent that reads the standard env
vars — OpenAI SDK, LangChain, Anthropic SDK. No Veri code inside the agent; each
rollout the platform injects:

- `OPENAI_BASE_URL` / `ANTHROPIC_BASE_URL` → the in-training policy's endpoint
- `VERI_TASK_INPUT` → one dataset row's `prompt` (JSON string); the agent solves it
- `VERI_POLICY_MODEL` → model name to send in requests (other names 404)

The agent runs its normal loop and exits; the reward function (attached to the job
inline) scores the trajectory's FINAL assistant message (extra dataset columns
arrive as lists).

```bash
veri run-harness ./harness \
  --entrypoint "python agent.py" \
  --base-model Qwen/Qwen3-4B \
  --dataset-id ds_... --reward ./reward.py \
  --gpu-type H100-80GB --gpu-count 2 --provider vast \
  --rollouts-per-prompt 4 --max-steps 20 \
  --requirements ./harness/requirements.txt
```

Constraints that WILL bite:

1. `--gpu-count` must be 2-8 on one node (trainer and policy server need disjoint
   GPUs). The CLI default is 4, and a count outside 2-8 fails before the upload.
   GPU 0 trains; the policy server gets the largest power of two <= gpu_count-1 as
   tensor parallelism, so any count 2-8 boots (odd counts leave GPUs idle;
   `vllm_data_parallel_size` reclaims them).
   `H100-80GB x2 --provider vast` is the proven shape.
2. Protocols: OpenAI (default) and Anthropic (`--harness-protocol anthropic`, e.g. the
   Claude Agent SDK) both train with exact token fidelity. Anthropic native `tool_use` /
   `tool_result` blocks are translated both ways; only `image` and `document` content
   blocks are rejected (400).
3. Policy ceiling today: ~7-8B full fine-tune (the trainer runs on one GPU).
4. Every dataset task must be WINNABLE through the agent's own tools: if the tools
   can never surface the ground-truth answer, every rollout scores 0 and that task
   contributes zero learning signal at full GPU price. Before spending, verify a
   sample — run the agent's fetch/tool path offline and check the answer appears.
5. Rollouts run sandboxed with network egress; per-rollout deps come from
   `--requirements`.

Inspect results: the job page shows a per-step/task/trial trajectory archive (full
message history, tool calls, tokens, reward); programmatic:
`client.training_jobs.trajectories(job_id)` + `trajectory_download_url(job_id, tid)`.

Docs (env contract + full worked demo incl. dataset prep and reward):
https://docs.veri.studio/training/harness · https://docs.veri.studio/demos/harness-url-extraction-agent

## Deployments (inference)

A completed job deploys with `veri deploy <job_id>` whether it was a full fine-tune or trained with `lora_rank` (LoRA jobs are served from the merged weights Veri keeps next to the adapter). A LoRA job trained before merged weights were kept returns a 400 that says so; serve that one from Hugging Face instead (`[hf_push]` with `artifact = "merged"` and `private = false` at submit, then `veri deploy <your-hf-repo>`), or retrain.

```bash
veri deploy Qwen/Qwen3-4B --gpu-type L4-24GB --no-cache   # one-step HF deploy
veri deployments create --from-hf Qwen/Qwen3-4B --name qwen-dev \
  --gpu-type L4-24GB --min-replicas 0 --max-replicas 2
veri deployments create --model <job_id> ...     # serve a finished training job
veri deployments create --from-model <model_id> ...   # serve a library model (veri models list)
```

Always pass `--cache` or `--no-cache` to `veri deploy`: without it the command prompts
interactively on a TTY and will hang an agent session. (`--cache` reuses/populates a
server-side copy of the HF snapshot so repeat deploys of the same repo skip the download.)

ALWAYS pass `--gpu-type` explicitly and right-size it (`veri gpu list` for prices; L4-24GB
handles ≤4B models). `veri deploy` defaults to `L4-24GB` x1; `veri deployments create` requires
`--gpu-type`. On AWS, `A100-80GB`/`H100-80GB` serve only as whole 8-GPU nodes (`--gpu-count 8`);
x1 is a 400.

- Engine: `vllm` only (any other value returns 400). Extra vLLM flags: repeat `--vllm-arg=<token>` or `deployment.vllm_extra_args` list in the config; only the accepted tuning flags pass (list: /deployments/api#vllm-extra-args), any other flag returns 400 `invalid_vllm_extra_args`.
- Serving providers live: `aws` (default) and `digitalocean` (1x H100-80GB, 1x H200-141GB: `veri deploy <model> --provider digitalocean --gpu-type H200-141GB --no-cache`; the region is picked for you). Not enabled: `gcp`, `huggingface` (create returns 400 "Serving provider '<name>' is not currently enabled."). `vast` is training-only (serving returns 400). A capacity or quota failure follows the same playbook as training (Capacity above).
- Defaults: gpu_count 1, replicas 1, concurrency_target 8, scale_to_zero_window 3600s (min 300).
- Mutable on a live deployment: replica bounds, concurrency target, idle windows. Model/GPU/provider/engine are immutable.
- Replica budget is 2 by default (see Account limits), so `--max-replicas` above 2 is not servable.
- BILLING: a deployment with min_replicas ≥ 1 bills per-minute until YOU stop it. Use `--min-replicas 0` (scale-to-zero) or `veri deployments stop <id>` when done. Always list deployments after a work session and stop strays.
- `veri deployments stop` is IMMEDIATE, not graceful: every box is terminated and in-flight
  requests are cut. To retire one replica without dropping requests, drain it (below).
- `stop` is also PERMANENT: a stopped deployment cannot be resumed or woken (create a new one).
  It is not a pause; scale-to-zero (`--min-replicas 0`) is. Stopping an already stopped or
  failed deployment is a no-op that returns the row unchanged.

### OpenAI-compatible endpoint

```python
from openai import OpenAI
oai = OpenAI(base_url=f"https://api.veri.studio/v1/deployments/{dep_id}",
             api_key="vk_...")  # client appends /chat/completions
resp = oai.chat.completions.create(model="<deployment-name>", messages=[...], stream=True)
```

- Streaming works (SSE passthrough); the final usage-only chunk arrives only if the client sends `stream_options: {"include_usage": true}`. All other params (`top_p`, `stop`, `seed`, `response_format`, `tools`, ...) pass through to vLLM untouched. On a single-model deployment `model` accepts any string and is rewritten to the deployment's served name; on a multi-adapter fleet (beta, created through the REST API with `serving_mode: "multi_adapter"`) it selects the adapter.
- Tool calling requires the deployment's vLLM to be launched with tool parsing (`--vllm-arg=--enable-auto-tool-choice --vllm-arg=--tool-call-parser --vllm-arg=hermes`).
- A scaled-to-zero deployment returns 503 + `Retry-After` (`deployment_waking`) on first request; retry until warm or pre-warm with `veri deployments wake <id>`. The wake itself can be refused with 429 `ACTIVE_DEPLOYMENT_LIMIT` (see Account limits). Upstream failures (model server unreachable, 5xx/429 from it, or no serving replica) are 503 + `Retry-After` with code `upstream_unavailable`: retry those too.
- Anthropic clients: `POST /v1/deployments/{id}/messages` (+ `/messages/count_tokens`) is the Messages API on the same deployment. No embeddings endpoint.
- Agent observability: send `X-Veri-Agent: <name>` and `X-Veri-Thread-Id: <conversation id>` (both `[A-Za-z0-9._:-]`, 1-64 and 1-128 chars; anything else is a 400) on chat/messages requests. Veri groups requests into agents and conversations: `GET /v1/observability/{overview,agents,conversations}` and `GET /v1/deployments/{id}/threads/{thread_id}`. Store request/response bodies as traces with `--trace-bodies` at create (`veri deployments traces <id>`); monitors need it.

Diagnostics: `veri deployments chat|metrics|requests|bench <id>`.

### Replicas and draining (API only, no CLI)

`GET /v1/deployments/{id}/replicas` lists every replica with live per-box telemetry:
`status`, `requests_running`, `requests_waiting`, `kv_cache_usage`, GPU utilization/memory,
`last_heartbeat_at`. Replica statuses: `queued`, `provisioning`, `loading`, `serving`,
`draining`, `stopped`, `failed`.

`POST /v1/deployments/{id}/replicas/{replica_id}/drain` retires ONE replica gracefully: it
stops receiving new requests immediately, then terminates once in-flight requests finish
(minimum 60s, hard timeout 600s). Returns 202. Default mode shrinks the fleet by one; it is
409 on the last serving replica or when it would breach `min_replicas`. Add `?replace=true`
to swap the box without losing capacity: a replacement launches FIRST while this replica
keeps serving (`replace_requested_at` set), and this replica drains only once the
replacement serves. Both boxes bill during the overlap (one boot). If the replacement fails,
the old replica keeps serving and `replace_error` says why (no automatic retry). One replace
per deployment at a time (409 otherwise); 429 `ACTIVE_DEPLOYMENT_LIMIT` when the workspace
has no room for the extra box. A `loading` replica cannot be drained (409). A draining
replica is still billed until it terminates, so a drain can cost up to 10 extra minutes of
box time.

Docs: https://docs.veri.studio/deployments (params + sizing) · https://docs.veri.studio/deployments/openai-compat (interop) · https://docs.veri.studio/deployments/cost (billing states) · https://docs.veri.studio/cli/deployments (scaling)

## Deleting resources

```bash
veri jobs delete <id>              # terminal jobs only; also deletes stored checkpoints
veri datasets delete <id>
veri deployments delete <id>       # stopped deployments only; also deletes request history
veri models delete <model_id>
veri volumes delete <name> --yes   # --yes is REQUIRED for agents (see below)
veri volumes rm <volume> <path>    # one file inside a volume
```

Deletes are permanent and there is no `--force` / undo. Two things to know before an agent
runs these:

1. `veri volumes delete` is the ONLY one that prompts (it asks you to retype the volume
   name). Without `--yes` it will hang an agent session. Every other delete above executes
   immediately with no confirmation, so confirm with the user yourself first.
2. Ordering constraints are enforced server-side, not by the CLI. A job that a deployment
   still references will not delete; a model with an active deployment returns 409; a
   deployment only deletes once it is stopped AND its final billing settlement has landed
   (a few seconds after stopping). Stop first, then delete.
   A volume that a queued or running job mounts (or reads a dataset from) returns 409 naming the jobs.
   A dataset with an experiment pinned to one of its snapshots returns 409 naming the experiments.
   A model with a running Hugging Face export, and the job it was saved from, return 409 until the export ends.

## Webhooks (API only, no CLI)

Subscribe to lifecycle events instead of polling. `POST /v1/webhooks` with `{"url": ...,
"event_types": [...]}`; omit `event_types` to receive all. Valid values:

```
deployment.serving  deployment.failed  deployment.scaled_to_zero  job.completed  job.failed
model.ready  dataset.ready  dataset.rows_appended  dataset.snapshot_created
experiment.completed  annotation.completed  score.failed
session.started  session.paused  session.ended  session.failed
```

Cancelling a job emits nothing (only `completed`/`failed` fire). URLs must be public HTTPS;
private/internal addresses are rejected at registration and again at every delivery.

Verify every delivery. Each POST carries `webhook-id`, `webhook-timestamp`, and
`webhook-signature: v1,<base64 HMAC-SHA256>` (Standard Webhooks). Sign the exact string
`{webhook-id}.{webhook-timestamp}.{raw body}` using the secret with its `whsec_` prefix
stripped and the remainder base64-DECODED to raw key bytes. Compare against the value after
`v1,` in constant time, and use the raw received body — re-serializing the JSON changes the
signature. `webhook-id` is stable across retries, so use it as the idempotency key.

Retries run 5s, 60s, 10m, 1h, 6h, then every 12h, giving up 72h after the event. 50
consecutive failures auto-disable the endpoint (re-enable with `PATCH {"enabled": true}`).
`POST /v1/webhooks/{id}/test` sends a `webhook.test` event; `GET /v1/webhooks/{id}/deliveries`
returns the 50 most recent attempts with response codes and errors for debugging.

## Metrics export (API only, no CLI)

`GET /v1/metrics/export` returns your deployments in Prometheus text exposition format
(v0.0.4), authenticated with the same `vk_...` bearer key — point a Prometheus, Grafana
Agent, or Datadog OpenMetrics scrape straight at it. Gauges cover deployment status and
desired/ready replicas; per-replica series cover `requests_running`, `requests_waiting`,
`kv_cache_usage_ratio`, GPU utilization and memory, token counters, and
`veri_deployment_replica_metrics_age_seconds` (alert on this to catch a stalled replica).
Only live replicas emit series, and an idle account returns an empty body — that is normal,
not an error. Scrapes share the account's 120-requests-per-60s budget, so keep the interval
at 15s or slower.

## Volumes, models, billing, W&B, Hugging Face

```bash
veri volumes create my-corpus && veri volumes upload my-corpus ./big.jsonl   # >5GiB auto-multipart
veri volumes ls my-corpus                        # storage only: training jobs cannot read volumes yet
veri models save <job_id> --name my-model        # reusable model library; deploy with --from-model
veri models register <job_id> --model acme-bot   # versioned lineage (acme-bot@v6); first version = production
veri models promote acme-bot v6 -m "reason"      # move production (--alias staging); rollback / pin / retention exist
veri datasets create acme-feedback --format preference   # append-only stream; snapshot, then train on acme-feedback@latest
veri billing balance | veri billing burn-rate | veri cost summary
veri wandb set --api-key <key> --project <p>     # training jobs auto-log to W&B
veri hf set                                      # connect a write-scope HF token (prompted, hidden)
veri models push-hf <model_id> --repo ns/name    # push a ready library model to HF (--public, --no-wait)
```

Hugging Face pushes run server-side (nothing downloads locally): connect once with `veri hf set`, then either push at end of training (`[huggingface]` in the train config, see Training) or push any ready library model with `veri models push-hf`. Repos default private. One export per model/repo at a time, two concurrent per account (409 = wait, don't retry-loop).

Docs: https://docs.veri.studio/volumes · https://docs.veri.studio/deployments/custom-models · https://docs.veri.studio/deployments/model-versions · https://docs.veri.studio/training/dataset-streams · https://docs.veri.studio/training/wandb · https://docs.veri.studio/training/huggingface

## Evals and monitors

Evaluators are named, versioned scoring rules: `llm_judge` (a deployment in your workspace
judges), `code` (Python locally, or JavaScript/TypeScript with `--runtime cloud`) or `human`.

```bash
veri evaluators create exact --method code --output-type boolean --output-config '{"pass_when": true}' --source-file ./exact.py
veri evaluators create --file judge.json --judge-deployment <dep_id>   # llm_judge from a spec file
veri experiments run -d <dataset>@latest -t deployment:<dep_id> -e exact -e judge@2   # exit 0 = passed
veri models gate <lineage>      # promotion gate: moving production needs a passing experiment
```

- Decision mode: `"method_config": {"mode": "decision", "prompt": ...}` in the spec makes the
  judge answer one label (`yes`/`no`, a categorical label, or an integer of a numeric scale
  at most 10 wide) and stores that label's probability as the score's `confidence` (0-1).
  No CLI flag; use the spec file or API.
- Target `none` scores outputs already in the dataset (chat rows ending in an assistant
  turn). A local code evaluator runs on your machine with `veri experiments run --local`;
  without it the experiment waits in `awaiting_local`.
- Monitors (API only) score a sampled share of one deployment's live traffic:
  `POST /v1/monitors {"name", "deployment_id", "evaluator_id"}` plus optional
  `sampling_rate` (0-1, default 1), `filter: {"agent": ...}` (an `X-Veri-Agent` value),
  `idle_minutes` (thread-scope evaluators score a conversation after this much idle time;
  1-1440, default 15), `daily_cap` (1-100000, default 1000). The deployment must store
  trace bodies (else 409 `trace_bodies_required`); the evaluator must be `llm_judge` or a
  cloud `code` evaluator with no reference answer. Scores carry `source: monitor` and
  `monitor_id`; a failed evaluation fires `score.failed`.

Docs: https://docs.veri.studio/evals · https://docs.veri.studio/evals/evaluators

## Managed agents (sessions)

Run an agent harness (`claude-code`, `opencode`, `codex`, `langgraph`, `crewai` or `custom`) in a
Veri sandbox, talking to a Veri deployment or your own model endpoint (`base_url`). The REST
routes work from any client.

- Secrets: `PUT /v1/secrets/{name}` (`veri secrets set NAME --value ...`); values never come back.
- Skills: `POST /v1/skills` (`veri skills add <name> --file SKILL.md -d "when to use"`), versioned.
- Environments: `POST /v1/environments` or `veri run environment.toml` (`kind = "environment"`;
  `harness = "claude-code"` (default) or another harness, `[model] deployment_id = "..."` or `base_url` + `secret`, `size`,
  `egress`, `allowed_hosts`, `secrets`, `skills`, `[[mcp_servers]]`, `[permissions]`,
  `idle_timeout_s`). Every edit is a new immutable version; sessions pin theirs.
- Sessions: `POST /v1/sessions {"environment_id", "budget_usd"?}`
  (`veri sessions create <env_id> --budget 5`), then `veri sessions send <id> "<task>" --follow`.
  Transcript: `GET /v1/sessions/{id}/events?after_seq=N` or SSE `GET .../stream`
  (resume with `Last-Event-ID`). `pause` / `resume` / `DELETE` (ends it). Billed per hour
  while the sandbox is awake at the environment size's rate (`rate_usd_per_hour` on the
  session); paused sessions cost nothing; a message to a paused session is 409. At most ten
  non-ended sessions per workspace (429 `CONCURRENT_SESSION_LIMIT`).
- Approvals: an `ask` permission rule (claude-code) parks the turn as `awaiting_approval`;
  answer with `veri sessions approve <session> <approval> allow|deny`.
- Session ops: `veri sessions exec <id> -- <cmd>`, `veri sessions ls` / `cp` (files under
  `/workspace`), `veri sessions attach <id>` (shell; needs the `attach` extra:
  `uv tool install --upgrade 'veri-sdk[all]'`). Preview ports and
  `github_auth` are coming soon (501 / 400 today).
- Triggers: `veri triggers create --environment <env_id> --cron "0 9 * * 1"` or `--webhook github`
  (signed `POST /v1/hooks/{id}`), results to webhooks or `--slack <url>`.

Docs: https://docs.veri.studio/agents/environments · https://docs.veri.studio/agents/sessions · https://docs.veri.studio/agents/triggers · https://docs.veri.studio/agents/session-ops

## REST API essentials

Base `https://api.veri.studio`, header `Authorization: Bearer vk_...`. Public spec: `GET /openapi.json`. Main resources: `/v1/me` (identity), `/v1/training_jobs` (+ `/logs`, `/events`, `/metrics`, `/cancel`, `/model`, `/trajectories`), `/v1/deployments` (+ `/chat/completions`, `/messages`, `/stop`, `/wake`, `/replicas`, `/replicas/{id}/drain`, `/requests`, `/traces`, `/threads/{thread_id}`), `/v1/observability/*`, `/v1/evaluators`, `/v1/experiments`, `/v1/scores`, `/v1/monitors`, `/v1/environments` (+ `/triggers`), `/v1/skills`, `/v1/secrets`, `/v1/sessions` (+ `/exec`, `/files`, `/attach-token`), `/v1/triggers`, `/v1/datasets`, `/v1/volumes`, `/v1/models` (+ `/{id}/push-to-hf`, poll `/v1/hf_exports/{id}`), `/v1/webhooks` (+ `/secret`, `/test`, `/deliveries`), `/v1/metrics/export`, `/v1/settings/integrations/{wandb,huggingface}`, `/v1/billing/*`, `/v1/gpu/catalog`, `/v1/gpu/availability`, `/v1/regions`. DELETE exists for training jobs, datasets, deployments, models, volumes (and volume files). Errors are JSON `{"error": {"message", "code"?}}` (body-parsing 400/415/422 rejections are plain text): 400 invalid/not enabled, 401 bad key, 402 insufficient credit, 404 not found/not owned, 409 still referenced/wrong state, 429 always with a `code` (a cap, or `RATE_LIMITED` + `retry_after_seconds`), 501 not available, 503 + `Retry-After` (`deployment_waking`, `upstream_unavailable`). Lists take `limit` (most also an `after` cursor) and return `has_more`.

MCP alternative: hosted server at `https://api.veri.studio/mcp` (read-only: `/readonly/mcp`, which drops the 12 write and delete tools), or local `veri mcp serve` (needs `veri-sdk[mcp]`). `veri_list_gpus` is the catalog and `veri_gpu_availability` the live view; a failed job's `error_code` is in the job tool's result (both on the hosted server now; local `veri mcp serve` from the next SDK release). On the read-only endpoint the chat tool never wakes a parked deployment: it returns `{status: "scaled_to_zero", choices: [], error: ...}` instead. With writes enabled, chatting to a parked deployment wakes it (GPU boot, billing starts) and the tool returns `{status: "waking", ...}` with a retry hint right away: call it again after the hinted delay rather than waiting on the first call. The stop tool is permanent. Docs: https://docs.veri.studio/api-reference/introduction · https://docs.veri.studio/cli/mcp

## Ground rules for agents

1. Run `veri whoami` before anything; fix auth first if the token is invalid.
2. Estimate before spending: `veri jobs estimate <config>` / `veri cost estimate`. A submit that fails validation creates nothing and costs nothing; GPU time bills real money (402 = out of credit).
3. Never leave deployments running: stop or scale-to-zero after use.
4. Prefer `--format json --quiet` when parsing output.
5. On a 400 that says "not currently enabled" or "coming soon", the provider/feature is off: pick a live alternative, don't retry.
6. On a 429 with a cap code (`CONCURRENT_JOB_LIMIT`, `ACTIVE_DEPLOYMENT_LIMIT`, ...), free capacity first: retrying alone never clears it. On `RATE_LIMITED`, wait `retry_after_seconds`.
7. Deletes are permanent and mostly unprompted. Confirm with the user before deleting anything you did not create in this session.

## When stuck

1. `veri <command> --help` — authoritative for the installed CLI version; `veri doctor` for install/auth/network.
2. The docs: https://docs.veri.studio — key pages: [quickstart](https://docs.veri.studio/quickstart), [CLI overview](https://docs.veri.studio/cli), [full CLI reference + exit codes + env vars](https://docs.veri.studio/cli/reference), [training](https://docs.veri.studio/training), [deployments](https://docs.veri.studio/deployments), [API reference](https://docs.veri.studio/api-reference/introduction).
3. Live API surface: `curl -s https://api.veri.studio/openapi.json` (public, no auth).
4. Read the error body: Veri 400s explain themselves (disabled features say "not currently enabled" or "coming soon"; validation lists supported values).