Create a monitor
Scores a sampled share of one deployment’s live traffic with one evaluator. A single_turn evaluator scores traced requests as they ship; a thread evaluator scores a thread once it has been idle for idle_minutes (again after it resumes and goes idle again). Scores carry source: monitor and the monitor’s id; a failed evaluation emits the score.failed webhook. The deployment must store trace bodies (the judge reads them).
curl --request POST \
--url https://api.veri.studio/v1/monitors \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data '
{
"name": "<string>",
"deployment_id": "<string>",
"evaluator_id": "<string>",
"evaluator_version": 123,
"sampling_rate": 123,
"filter": {},
"idle_minutes": 123,
"daily_cap": 123,
"enabled": true,
"review_below_confidence": 123,
"review_queue_id": "<string>"
}
'import requests
url = "https://api.veri.studio/v1/monitors"
payload = {
"name": "<string>",
"deployment_id": "<string>",
"evaluator_id": "<string>",
"evaluator_version": 123,
"sampling_rate": 123,
"filter": {},
"idle_minutes": 123,
"daily_cap": 123,
"enabled": True,
"review_below_confidence": 123,
"review_queue_id": "<string>"
}
headers = {
"Authorization": "Bearer <token>",
"Content-Type": "application/json"
}
response = requests.post(url, json=payload, headers=headers)
print(response.text)const options = {
method: 'POST',
headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
body: JSON.stringify({
name: '<string>',
deployment_id: '<string>',
evaluator_id: '<string>',
evaluator_version: 123,
sampling_rate: 123,
filter: {},
idle_minutes: 123,
daily_cap: 123,
enabled: true,
review_below_confidence: 123,
review_queue_id: '<string>'
})
};
fetch('https://api.veri.studio/v1/monitors', options)
.then(res => res.json())
.then(res => console.log(res))
.catch(err => console.error(err));{
"object": "<string>",
"id": "<string>",
"name": "<string>",
"deployment_id": "<string>",
"evaluator_id": "<string>",
"evaluator_name": "<string>",
"scope": "<string>",
"sampling_rate": 123,
"filter": {},
"idle_minutes": 123,
"daily_cap": 123,
"enabled": true,
"stats": {
"scored_today": 123,
"flagged_today": 123,
"scored_7d": 123,
"flagged_7d": 123,
"last_scored_at": "2023-11-07T05:31:56Z",
"human_agreement": {
"n": 123,
"rate": 123
}
},
"created_at": "2023-11-07T05:31:56Z",
"updated_at": "2023-11-07T05:31:56Z",
"evaluator_version": 123,
"review_below_confidence": 123,
"review_queue_id": "<string>"
}Authorizations
API key with the vk_ prefix. Create one from the dashboard.
Body
1-64 lowercase letters, digits, '.', '_' or '-'.
A deployment of the workspace with trace_bodies on.
Evaluator id (evl_...) or name. llm_judge or cloud code; human and local code evaluators, and evaluators that require a reference answer, can't run on live traffic.
Pin a version; null (default) tracks the latest.
0..1 (default 1).
{agent?} (default {}).
1..1440 (default 15).
1..100000 (default 1000).
Confidence routing (set both or neither): scores whose confidence is
below this (0..1) send their target to review_queue_id, an
annotation queue (id or name) reviewing traces (single_turn monitors)
or threads (thread monitors). 400 while Jev is off.
Response
The monitor
One monitor.
Always "monitor".
mon_...
The deployment whose traffic is scored.
single_turn (scores traced requests) | thread (scores idle threads), from the version the monitor runs.
Share of the traffic scored, 0..1.
{agent?}: only requests or threads sent with this X-Veri-Agent.
Thread monitors: minutes without a request before a thread is scored.
Most targets queued per UTC day.
Show child attributes
Show child attributes
The pinned version; null tracks the latest.
Confidence routing: an ok score whose confidence is below this
(0..1) adds its target to review_queue_id; null = off.
The annotation queue low-confidence targets go to.
curl --request POST \
--url https://api.veri.studio/v1/monitors \
--header 'Authorization: Bearer <token>' \
--header 'Content-Type: application/json' \
--data '
{
"name": "<string>",
"deployment_id": "<string>",
"evaluator_id": "<string>",
"evaluator_version": 123,
"sampling_rate": 123,
"filter": {},
"idle_minutes": 123,
"daily_cap": 123,
"enabled": true,
"review_below_confidence": 123,
"review_queue_id": "<string>"
}
'import requests
url = "https://api.veri.studio/v1/monitors"
payload = {
"name": "<string>",
"deployment_id": "<string>",
"evaluator_id": "<string>",
"evaluator_version": 123,
"sampling_rate": 123,
"filter": {},
"idle_minutes": 123,
"daily_cap": 123,
"enabled": True,
"review_below_confidence": 123,
"review_queue_id": "<string>"
}
headers = {
"Authorization": "Bearer <token>",
"Content-Type": "application/json"
}
response = requests.post(url, json=payload, headers=headers)
print(response.text)const options = {
method: 'POST',
headers: {Authorization: 'Bearer <token>', 'Content-Type': 'application/json'},
body: JSON.stringify({
name: '<string>',
deployment_id: '<string>',
evaluator_id: '<string>',
evaluator_version: 123,
sampling_rate: 123,
filter: {},
idle_minutes: 123,
daily_cap: 123,
enabled: true,
review_below_confidence: 123,
review_queue_id: '<string>'
})
};
fetch('https://api.veri.studio/v1/monitors', options)
.then(res => res.json())
.then(res => console.log(res))
.catch(err => console.error(err));{
"object": "<string>",
"id": "<string>",
"name": "<string>",
"deployment_id": "<string>",
"evaluator_id": "<string>",
"evaluator_name": "<string>",
"scope": "<string>",
"sampling_rate": 123,
"filter": {},
"idle_minutes": 123,
"daily_cap": 123,
"enabled": true,
"stats": {
"scored_today": 123,
"flagged_today": 123,
"scored_7d": 123,
"flagged_7d": 123,
"last_scored_at": "2023-11-07T05:31:56Z",
"human_agreement": {
"n": 123,
"rate": 123
}
},
"created_at": "2023-11-07T05:31:56Z",
"updated_at": "2023-11-07T05:31:56Z",
"evaluator_version": 123,
"review_below_confidence": 123,
"review_queue_id": "<string>"
}