Create dataset
Two bodies share this route. multipart/form-data (name + file) uploads a one-shot JSONL, which becomes a stream with one snapshot (snap-1). application/json {name, format, auto_snapshot_rows?} creates an EMPTY append-only stream with a locked row format (prompt | preference | completion | chat | eval); append rows with POST /v1/datasets//rows.
curl --request POST \
--url https://api.veri.studio/v1/datasets \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data '
{
"name": "<string>",
"auto_snapshot_rows": 123
}
'import requests
url = "https://api.veri.studio/v1/datasets"
payload = {
"name": "<string>",
"auto_snapshot_rows": 123
}
headers = {
"Authorization": "Bearer <token>",
"Content-Type": "application/json"
}
response = requests.post(url, json=payload, headers=headers)
print(response.text)const options = {
method: 'POST',
headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
body: JSON.stringify({name: '<string>', auto_snapshot_rows: 123})
};
fetch('https://api.veri.studio/v1/datasets', options)
.then(res => res.json())
.then(res => console.log(res))
.catch(err => console.error(err));{
"object": "<string>",
"id": "<string>",
"name": "<string>",
"created_at": "2023-11-07T05:31:56Z",
"source_type": "<string>",
"source_uri": "<string>",
"huggingface_dataset": "<string>",
"num_rows": 123,
"format": "<string>",
"head_row": 123,
"auto_snapshot_rows": 123,
"latest_snapshot_id": "<string>",
"unsnapshotted_rows": 123,
"snapshot_count": 123
}{
"object": "<string>",
"id": "<string>",
"name": "<string>",
"created_at": "2023-11-07T05:31:56Z",
"source_type": "<string>",
"source_uri": "<string>",
"huggingface_dataset": "<string>",
"num_rows": 123,
"format": "<string>",
"head_row": 123,
"auto_snapshot_rows": 123,
"latest_snapshot_id": "<string>",
"unsnapshotted_rows": 123,
"snapshot_count": 123
}Authorizations
API key with the vk_ prefix. Create one from the dashboard.
Body
JSON {name, format} for an empty stream, or a multipart JSONL file upload
JSON body for POST /v1/datasets: an empty stream with a locked format.
Row format every append is validated against.
prompt, preference, completion, chat, eval Optional rule: cut a snapshot automatically once this many rows have landed since the newest snapshot (evaluated by the reconciler).
Response
Multipart upload whose content matches an existing dataset: that dataset
Row format. A stream's locked format (prompt | preference | completion |
chat | eval); an upload records its first row's shape, which adds
chat_no_prompt (chat rows without a prompt, still locked as chat), text,
sharegpt and alpaca (no lock). Job submit refuses GRPO on chat_no_prompt,
text, sharegpt and alpaca rows. Null on Hugging Face / volume sources, and
on a legacy dataset until its first append locks it.
Highest row_id appended so far (0 = empty stream). Includes supersede and tombstone entries, so it is the log length, not the live row count.
"cut a snapshot every N new rows" rule; null = off.
Newest snapshot id ("@snap-N"); populated by GET /v1/datasets/{id}.
Rows appended since the newest snapshot; populated by GET /v1/datasets/{id}.
curl --request POST \
--url https://api.veri.studio/v1/datasets \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data '
{
"name": "<string>",
"auto_snapshot_rows": 123
}
'import requests
url = "https://api.veri.studio/v1/datasets"
payload = {
"name": "<string>",
"auto_snapshot_rows": 123
}
headers = {
"Authorization": "Bearer <token>",
"Content-Type": "application/json"
}
response = requests.post(url, json=payload, headers=headers)
print(response.text)const options = {
method: 'POST',
headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
body: JSON.stringify({name: '<string>', auto_snapshot_rows: 123})
};
fetch('https://api.veri.studio/v1/datasets', options)
.then(res => res.json())
.then(res => console.log(res))
.catch(err => console.error(err));{
"object": "<string>",
"id": "<string>",
"name": "<string>",
"created_at": "2023-11-07T05:31:56Z",
"source_type": "<string>",
"source_uri": "<string>",
"huggingface_dataset": "<string>",
"num_rows": 123,
"format": "<string>",
"head_row": 123,
"auto_snapshot_rows": 123,
"latest_snapshot_id": "<string>",
"unsnapshotted_rows": 123,
"snapshot_count": 123
}{
"object": "<string>",
"id": "<string>",
"name": "<string>",
"created_at": "2023-11-07T05:31:56Z",
"source_type": "<string>",
"source_uri": "<string>",
"huggingface_dataset": "<string>",
"num_rows": 123,
"format": "<string>",
"head_row": 123,
"auto_snapshot_rows": 123,
"latest_snapshot_id": "<string>",
"unsnapshotted_rows": 123,
"snapshot_count": 123
}