---
name: Veri
description: Use when training language models with managed or custom training loops, deploying models as OpenAI-compatible inference endpoints, evaluating model performance, managing datasets and reward functions, or orchestrating multi-GPU training jobs. Agents should reach for this skill when users request model training, deployment, evaluation, or infrastructure management tasks.
metadata:
    mintlify-proj: veri
    version: "1.0"
---

# Veri Skill

## Product summary

Veri is a platform for post-training, deploying, and evaluating language models. It provides managed training loops (GRPO, SFT, DPO), custom training script support, OpenAI-compatible inference endpoints, and evaluation infrastructure. The primary entry point is the `veri` CLI (installed via `pip install veri-sdk`), which ships with the Python SDK. Key workflows: training jobs via `veri run configs/train.toml`, deployments via `veri deployments create`, dataset management via `veri datasets`, and volume storage via `veri volumes`. Credentials live at `~/.config/veri/config.toml` (or `$XDG_CONFIG_HOME/veri/config.toml`). Full documentation: https://docs.veri.studio

## When to use

Reach for this skill when:

- **Training**: User wants to fine-tune a model with GRPO, SFT, DPO, or a custom training script. Includes managed training (Veri owns the loop) and bring-your-own-code (you own the loop).
- **Deployments**: User needs to serve a base model or trained checkpoint as an OpenAI-compatible API endpoint with autoscaling, metrics, and cost tracking.
- **Datasets**: User needs to upload training data, connect HuggingFace datasets, or reference data in managed volumes.
- **Reward functions**: User is building GRPO training and needs to score model completions.
- **Multi-GPU/multi-node**: User wants distributed training on provisioned GPUs without managing infrastructure.
- **Observability**: User needs to monitor training metrics, deployment requests, latency, errors, or cost.
- **GPU selection**: User needs to choose the right GPU type, provider (AWS, Hot Aisle, Vast.ai), or region for their workload.

## Quick reference

### CLI commands

| Command | Purpose |
| --- | --- |
| `veri login` | Authenticate with API key |
| `veri run configs/train.toml` | Submit training job from TOML config |
| `veri train --base-model X --gpu-type Y` | Quick training without config file |
| `veri deployments create --model X --gpu-type Y` | Deploy a model |
| `veri deployments chat dep_id` | Test a deployment interactively |
| `veri datasets upload data.jsonl --name X` | Upload training data |
| `veri datasets connect-hf openai/gsm8k --map question=prompt` | Connect HuggingFace dataset |
| `veri rewards upload reward.py --format trl` | Register a reward function |
| `veri volumes create my-corpus` | Create persistent storage |
| `veri volumes upload my-corpus ./data.jsonl` | Upload to volume |
| `veri gpu list` | Check available GPUs and pricing |
| `veri whoami` | Show account, balance, and config path |

### Config file structure

**`configs/train.toml`** (managed training):
```toml
kind = "train"
[job]
name = "my-run"
[model]
base = "Qwen/Qwen2.5-0.5B-Instruct"
[dataset]
id = "ds_abc123"
[reward]
id = "rf_abc"
[method]
type = "grpo"  # or sft_text, dpo, sft_video_gen
learning_rate = 1e-6
max_steps = 100
[resources]
gpu_type = "A100-80GB"
gpu_count = 1
```

**`configs/deploy.toml`**:
```toml
kind = "deploy"
[deployment]
name = "my-endpoint"
model = "job_abc"  # training job id or HF repo
source = "training_job"  # or huggingface
[resources]
gpu_type = "A100-80GB"
gpu_count = 1
[autoscaling]
min_replicas = 1
max_replicas = 1
```

### Dataset JSONL format

```json
{"prompt": [{"role": "user", "content": "What is 2+2?"}], "answer": "4"}
{"prompt": "What is 3+3?", "answer": "6"}
```

Required: `prompt` (string or message list). Optional: `answer` (used by reward functions).

### Reward function signature (TRL format)

```python
def reward(completions, answer, **kwargs) -> list[float]:
    scores = []
    for completion, expected in zip(completions, answer):
        text = completion[-1]["content"] if isinstance(completion, list) else str(completion)
        score = 1.0 if expected in text else 0.0
        scores.append(score)
    return scores
```

Returns one float per completion. Completions are strings or message-dict lists.

### Environment variables

| Variable | Purpose |
| --- | --- |
| `VERI_API_KEY` | API key (takes precedence over config file) |
| `VERI_API_URL` | Custom API endpoint (default: https://api.veri.studio) |
| `VERI_CONFIG_PATH` | Path to config file (default: ~/.config/veri/config.toml) |

### Global CLI flags

| Flag | Effect |
| --- | --- |
| `-o, --format` | Output format: `table` (default), `json`, `jsonl`, `csv` |
| `--json` | Alias for `--format json` |
| `-q, --quiet` | Suppress banners and non-essential output |

## Decision guidance

### When to use managed training vs custom script

| Scenario | Use | Reason |
| --- | --- | --- |
| GRPO, SFT, or DPO with standard hyperparameters | Managed training | Veri owns orchestration; simpler config |
| Custom RL framework (OpenRLHF, verl, slime) | Custom script | You control the loop; bring your own trainer |
| Multi-node distributed training | Custom script | Managed methods are single-node only |
| Bring-your-own training code | Custom script | Full control over entrypoint and dependencies |

### When to use dataset source types

| Source | Use when | Trade-offs |
| --- | --- | --- |
| `upload` (JSONL) | Small, one-off datasets | Simple; no reuse across jobs |
| `hf` (HuggingFace) | Public datasets (GSM8K, etc.) | Fast; cached after first job |
| `volume` | Large datasets, proprietary data, reuse | Upload once, reference many times |

### When to use deployment replicas and autoscaling

| Config | Use when | Cost impact |
| --- | --- | --- |
| `min_replicas: 1, max_replicas: 1` | Always-warm, predictable traffic | Constant billing; no scale-down |
| `min_replicas: 0, max_replicas: 8` | Bursty traffic, cost-sensitive | Scales to zero; cold start on first request (503 + Retry-After) |
| `min_replicas: 2, max_replicas: 4` | Baseline + burst | Minimum floor; scales up on demand |

### GPU sizing for deployments

| Model size | Minimum | Comfortable |
| --- | --- | --- |
| ≤ 1B | A100-80GB ×1 | A100-80GB ×1 |
| 3–8B | A100-80GB ×1 | H100-80GB ×1 |
| 14–34B | H100-80GB ×2 | H100-80GB ×4 |
| 70B+ | H100-80GB ×4 (quantized) | H100-80GB ×8 |

## Workflow

### Training a model (managed GRPO)

1. **Prepare dataset**: Upload JSONL or connect HuggingFace.
   ```bash
   veri datasets upload train.jsonl --name my-data
   # or
   veri datasets connect-hf openai/gsm8k --map question=prompt --map answer=answer
   ```

2. **Write reward function**: Create a `.py` file with `def reward(completions, answer, **kwargs)` returning scores.
   ```bash
   veri rewards upload reward.py --format trl --name my-reward
   ```

3. **Create config**: Write `configs/train.toml` with `kind = "train"`, model, dataset, reward, and hyperparameters.

4. **Dry-run first**: Validate config and see cost estimate.
   ```bash
   veri run configs/train.toml --dry-run
   ```

5. **Submit job**: Launch training.
   ```bash
   veri run configs/train.toml
   ```

6. **Monitor**: Stream logs and metrics.
   ```bash
   veri jobs logs job_abc -f
   ```

7. **Download or deploy**: Get checkpoint or serve directly.
   ```bash
   veri jobs download job_abc --output-dir ./checkpoints
   # or
   veri deployments create --model job_abc --gpu-type A100-80GB
   ```

### Deploying a model

1. **Create config** or use CLI flags:
   ```bash
   veri deployments create \
     --name my-endpoint \
     --model Qwen/Qwen2.5-0.5B-Instruct \
     --gpu-type A100-80GB \
     --gpu-count 1
   ```

2. **Wait for serving**: Check status.
   ```bash
   veri deployments get dep_abc
   ```

3. **Test**: Chat with the deployment.
   ```bash
   veri deployments chat dep_abc
   ```

4. **Call from code**: Use OpenAI SDK or curl.
   ```python
   from openai import OpenAI
   client = OpenAI(
       base_url=f"https://api.veri.studio/v1/deployments/{dep_id}",
       api_key="vk_your_key",
   )
   response = client.chat.completions.create(
       model="my-endpoint",
       messages=[{"role": "user", "content": "Hello"}],
   )
   ```

5. **Monitor**: Check metrics and requests.
   ```bash
   veri deployments metrics dep_abc
   veri deployments requests dep_abc --limit 50
   ```

6. **Stop when done**: Prevent further billing.
   ```bash
   veri deployments stop dep_abc
   ```

### Using volumes for large datasets

1. **Create volume**: One-time setup.
   ```bash
   veri volumes create my-corpus
   ```

2. **Upload data**: Multipart for files > 5 GiB.
   ```bash
   veri volumes upload my-corpus ./train.jsonl
   # or directory:
   veri volumes upload-dir my-corpus ./data --to /data
   ```

3. **Reference in training**: Use `from_volume` helper.
   ```python
   dataset = client.datasets.from_volume(volume="my-corpus", path="/train.jsonl")
   job = client.training_jobs.create(
       base_model="Qwen/Qwen2.5-0.5B-Instruct",
       dataset_id=dataset.id,
       # ...
   )
   ```

4. **Reuse**: Same volume across many jobs without re-uploading.

## Common gotchas

- **Deployment billing never stops on its own**: A deployment with `min_replicas ≥ 1` (the default) keeps billing until you call `stop`. Use `min_replicas: 0` to enable scale-to-zero, or explicitly stop when done.

- **Training job cost accrues from provisioning, not first step**: A job that fails to secure a GPU (e.g., `CAPACITY_UNAVAILABLE`) is not billed. But once the instance boots, cost accrues even before the first training step.

- **Reward functions have no network egress**: External API calls will silently fail or hang. Use only standard library + pre-installed packages in the base image.

- **Multi-node training is custom-script only**: Managed methods (GRPO, SFT, DPO) are single-node. For multi-node, use `veri run-script` with `--num-nodes`.

- **Custom scripts run on NVIDIA GPUs only**: AMD MI300X is not supported for custom scripts; use managed training with `provider="hotaisle"` instead.

- **Volume sync adds startup time**: A 50 GB volume typically takes 30–60s to sync to the worker. Plan for this in job duration estimates.

- **Capacity is per-region and provider**: If a job fails with `CAPACITY_UNAVAILABLE`, try another region (`veri regions list`) or GPU type (`veri gpu list`).

- **Reward function signature must match**: TRL format expects `def reward(completions, answer, **kwargs)` returning a list of floats. Wrong signature rejects at upload with HTTP 422.

- **Dataset dedup is by content hash**: Uploading the same file twice returns the same ID. Connecting the same HF dataset twice with different names creates separate records.

- **Deployments in early access**: GPU capacity for serving is limited. Check `veri gpu list` before planning around a specific GPU. Training is the platform's primary focus.

- **Scale-to-zero cold start returns 503**: When a `min_replicas: 0` deployment wakes, the first request gets a 503 with `Retry-After`. The SDK and CLI handle this; raw HTTP clients must retry.

- **Cancellation is cooperative**: Calling `cancel` on a training job terminates at the next checkpoint boundary, not instantly.

## Verification checklist

Before submitting work:

- [ ] **Dataset is valid**: Run `veri datasets upload --deep-check` or `client.datasets.validate(...)` to catch schema errors early.
- [ ] **Reward function signature matches**: Verify `def reward(completions, answer, **kwargs) -> list[float]` and test locally if possible.
- [ ] **Config has required fields**: `kind`, `[model].base` (or `[deployment].model`), `[resources].gpu_type`, and method-specific fields.
- [ ] **GPU type is available**: Run `veri gpu list` to confirm the requested GPU is live in your region.
- [ ] **Dry-run passes**: Use `veri run --dry-run` to validate config and see cost estimate before submitting.
- [ ] **API key is set**: Confirm `veri whoami` shows your account and balance.
- [ ] **Deployment will be stopped**: If using `min_replicas ≥ 1`, plan to call `veri deployments stop` when done.
- [ ] **Volume is created before referencing**: If using volumes, ensure `veri volumes create` succeeded before submitting a job.
- [ ] **Hyperparameters are reasonable**: Check learning rate, max_steps, rollouts_per_prompt, and max_response_length for your model size.
- [ ] **Output model name is unique**: If saving checkpoints, ensure the name doesn't conflict with existing models.

## Resources

- **Full page navigation**: https://docs.veri.studio/llms.txt — comprehensive list of all documentation pages for agent reference.
- **Training overview**: https://docs.veri.studio/training — choose between managed and custom training.
- **Deployments guide**: https://docs.veri.studio/deployments — serve models with OpenAI-compatible API, autoscaling, and observability.
- **CLI reference**: https://docs.veri.studio/cli — all command groups, flags, and exit codes.

---

> For additional documentation and navigation, see: https://docs.veri.studio/llms.txt