veri datasets

The veri datasets group covers every dataset operation: upload local JSONL, connect a HuggingFace dataset, validate structure, and inspect registered datasets.

Commands

Command	Purpose
`upload <path>`	Upload a JSONL file. Auto-runs structural validation.
`connect-hf <repo>`	Connect a HuggingFace dataset.
`check <path>`	Standalone structural validator.
`preview <path> [--n N]`	Print the first N rows of a local JSONL.
`list`	List registered datasets.
`get <id>`	Show one dataset.
`delete <id>`	Delete a dataset.

Upload a local JSONL

veri datasets upload ./datasets/train.jsonl --name my-train

upload runs structural validation before the network call. If validation fails, nothing is uploaded:

$ veri datasets upload bad.jsonl
Validation failed: 1 issue(s), 0 rows.
  • line 1: invalid JSON (Expecting value)

Use --skip-validation to upload anyway.

The validator checks:

Each non-blank line parses as JSON.
Every row is a JSON object (not a list or scalar).
Row count is > 0.
No duplicate id field values (when an id column exists).
Required columns are present, if specified.

To skip validation:

veri datasets upload bad.jsonl --skip-validation

--deep-check is a reserved flag for tokenizer-dependent checks (length distribution). It is not yet implemented and emits a warning if passed today.

--name defaults to the filename stem. Use --quiet for ID-only output that pipes cleanly:

DS_ID=$(veri datasets upload data.jsonl --quiet)
echo $DS_ID    # ds_abc123

Connect a HuggingFace dataset

No download required; Veri references the HF repo directly.

veri datasets connect-hf openai/gsm8k --config main --split train --name gsm8k-train

GSM8K requires a config name (main or socratic) — datasets with a single config can omit --config. Rename columns at connect time with --map old=new (repeatable):

veri datasets connect-hf openai/gsm8k \
  --config main \
  --split train \
  --map question=prompt \
  --map answer=answer

Flag	Purpose
`--split`	HF split (default `train`).
`--config`	HF dataset config / subset name.
`--map old=new`	Column rename. Repeatable.
`--name`	Display name. Default derived from repo.

Validate without uploading

veri datasets check ./datasets/train.jsonl

Same validator as upload --check, but standalone. Exits 1 on issues:

$ veri datasets check ok.jsonl
OK — 1000 rows.

$ echo $?
0

Use --format json for machine-readable output:

veri datasets check ok.jsonl --format json
# {"ok": true, "row_count": 1000, "errors": []}

Preview rows

veri datasets preview ./datasets/train.jsonl --n 3

Reads the first N JSON rows from a local file. Useful before upload; for registered datasets use veri datasets get <id> followed by the dashboard.

List, get, delete

veri datasets list                              # default 20 rows
veri datasets list --limit 100 --format json
veri datasets get ds_abc123
veri datasets delete ds_abc123

delete is irreversible. The CLI exits 0 and prints the deleted ID; no confirmation prompt.

Exit codes

Code	Meaning
`0`	Success.
`1`	User error (validation failed, missing flag, bad URI).
`2`	API error (server returned 4xx/5xx).
`4`	Auth error (no credentials, expired key).

What’s next

Training

Upload a reward function, submit a job against this dataset.

The run verb

Reference a dataset ID from configs/train.toml.

Get started

Training

Deployments

Evaluations

CLI

Volumes

Commands

Upload a local JSONL

Connect a HuggingFace dataset

Validate without uploading

Preview rows

List, get, delete

Exit codes

What’s next

Training

The run verb

​Commands

​Upload a local JSONL

​Connect a HuggingFace dataset

​Validate without uploading

​Preview rows

​List, get, delete

​Exit codes

​What’s next

Training

The run verb

Commands

Upload a local JSONL

Connect a HuggingFace dataset

Validate without uploading

Preview rows

List, get, delete

Exit codes

What’s next