> ## Documentation Index
> Fetch the complete documentation index at: https://docs.veri.studio/llms.txt
> Use this file to discover all available pages before exploring further.
# Reward functions
> Score model completions for GRPO training. TRL-format functions, with sandbox limits and validation rules.
A reward function turns a model completion into a number. GRPO uses these scores to compute advantages — completions above the group mean get reinforced, completions below get penalized.
You upload a `.py` file once and reference its `id` from any number of training jobs.
## Format
Veri runs reward functions in the `trl` shape — the format the production GRPO runner uses.
| Format | Signature | When to use |
| ------ | ---------------------------------------------------------- | --------------------------------------------------- |
| `trl` | `def reward(completions, answer, **kwargs) -> list[float]` | Used by TRL GRPOTrainer (today's runner on NVIDIA). |
The validator checks the function signature on upload — wrong shape rejects with HTTP 422.
## TRL format
Receives a batch of completions, returns one score per completion.
```python theme={null}
import re
def reward(completions, answer, **kwargs):
scores = []
for completion, expected in zip(completions, answer):
# Handle chat-format completions (list of message dicts)
text = completion[-1]["content"] if isinstance(completion, list) else str(completion)
format_ok = bool(re.search(r".*?", text, re.DOTALL))
m = re.search(r"(.*?)", text, re.DOTALL)
correct = bool(m and expected and expected.strip() in m.group(1).strip())
scores.append(0.5 * float(format_ok) + 0.5 * float(correct))
return scores
```
* `completions` — list of strings OR list of message-dict lists (depending on how the runner formats prompts)
* `answer` — list of ground-truth strings (from the dataset's `answer` column)
* Return — list of floats, **same length as `completions`**
## Upload
```python SDK theme={null}
reward_fn = client.reward_functions.upload(
"math_reward.py",
format="trl",
name="math-reward",
)
print(reward_fn.id)
```
```bash CLI theme={null}
veri rewards upload math_reward.py --format trl --name math-reward
```
The CLI runs an AST check (file parses + at least one function defined). Use `--skip-validation` to bypass, but the upload will still fail at job submission if the signature is wrong.
## Sandbox + limits
Reward functions run inside the training runner. They have:
* **No network egress** — calls to external APIs will silently fail or hang
* **Standard library + the runner image's pre-installed packages** only — no `pip install` at runtime
* **A timeout** per rollout batch — exceeding it fails the job with `error.code = "training_error"`
For PoC, treat this as "trusted users only." Real isolation (gVisor / Firecracker) is on the Phase 2 roadmap.
## Composing rewards
A common pattern is to weight multiple signals:
```python theme={null}
def reward(completions, answer, **kwargs):
scores = []
for completion, expected in zip(completions, answer):
text = completion[-1]["content"] if isinstance(completion, list) else str(completion)
format_score = 0.3 if has_correct_format(text) else 0.0
correct_score = 0.5 if has_correct_answer(text, expected) else 0.0
length_score = 0.2 * length_penalty(text, target=200)
scores.append(format_score + correct_score + length_score)
return scores
```
Keep each component bounded so the total stays in a stable range — `[0, 1]` is conventional. GRPO normalizes scores within a group, so the absolute scale matters less than the spread.
## Multiple weighted reward functions
Instead of summing signals inside one function, you can upload several reward functions and let GRPO combine them with weights. TRL's GRPOTrainer scores each completion with every function and sums the results, scaled by `reward_weights`. This keeps each signal in its own file (reusable and independently testable) and lets you retune the mix without editing code.
Each function has the standard `trl` signature and returns one score per completion. Upload each file, then pass the ids and matching weights:
```python theme={null}
# correctness.py
def reward(completions, answer, **kwargs):
return [1.0 if c == a else 0.0 for c, a in zip(completions, answer)]
# format.py
import re
def reward(completions, **kwargs):
return [1.0 if re.search(r".*?", str(c), re.DOTALL) else 0.0
for c in completions]
```
```python theme={null}
correctness = client.reward_functions.upload("correctness.py")
format_fn = client.reward_functions.upload("format.py")
job = client.training_jobs.create(
base_model="Qwen/Qwen2.5-0.5B-Instruct",
dataset_id=ds.id,
method="grpo",
reward_function_ids=[correctness.id, format_fn.id],
reward_weights=[1.0, 0.5],
gpu_type="L4-24GB",
gpu_count=1,
)
```
`reward_weights` is optional; omit it to weight every function equally. When provided, its length must match `reward_function_ids` (the SDK and API reject a mismatch). A single `reward_function_id` still works for the one-reward case.
## Reuse across jobs
Once uploaded, a reward function is reusable. The same `id` can drive any number of training jobs.
```python theme={null}
job1 = client.training_jobs.create(
base_model="Qwen/Qwen2.5-0.5B-Instruct",
dataset_id=ds.id,
reward_function_id=reward_fn.id,
# ...
)
job2 = client.training_jobs.create(
base_model="Qwen/Qwen2.5-7B-Instruct",
dataset_id=ds.id,
reward_function_id=reward_fn.id, # same reward
# ...
)
```
## Lifecycle
```python theme={null}
client.reward_functions.list()
client.reward_functions.get(reward_id)
```
Reward functions are deduped by content hash — uploading the same file twice returns the same id. There is no DELETE endpoint today; programmatic delete is on the roadmap.
## Where to go next
What the score actually does inside the training loop.
Wire the reward into `training_jobs.create`.
End-to-end with the example math reward.