Reward functions

A reward function turns a model completion into a number. GRPO uses these scores to compute advantages — completions above the group mean get reinforced, completions below get penalized. You upload a .py file once and reference its id from any number of training jobs.

Format

Veri runs reward functions in the trl shape — the format the production GRPO runner uses.

Format	Signature	When to use
`trl`	`def reward(completions, answer, **kwargs) -> list[float]`	Used by TRL GRPOTrainer (today’s runner on NVIDIA).

The validator checks the function signature on upload — wrong shape rejects with HTTP 422.

TRL format

Receives a batch of completions, returns one score per completion.

import re

def reward(completions, answer, **kwargs):
    scores = []
    for completion, expected in zip(completions, answer):
        # Handle chat-format completions (list of message dicts)
        text = completion[-1]["content"] if isinstance(completion, list) else str(completion)

        format_ok = bool(re.search(r"<answer>.*?</answer>", text, re.DOTALL))

        m = re.search(r"<answer>(.*?)</answer>", text, re.DOTALL)
        correct = bool(m and expected and expected.strip() in m.group(1).strip())

        scores.append(0.5 * float(format_ok) + 0.5 * float(correct))
    return scores

completions — list of strings OR list of message-dict lists (depending on how the runner formats prompts)
answer — list of ground-truth strings (from the dataset’s answer column)
Return — list of floats, same length as completions

Upload

reward_fn = client.reward_functions.upload(
    "math_reward.py",
    format="trl",
    name="math-reward",
)

print(reward_fn.id)

veri rewards upload math_reward.py --format trl --name math-reward

The CLI runs an AST check (file parses + at least one function defined). Use --skip-validation to bypass, but the upload will still fail at job submission if the signature is wrong.

Sandbox + limits

Reward functions run inside the training runner. They have:

No network egress — calls to external APIs will silently fail or hang
Standard library + the runner image’s pre-installed packages only — no pip install at runtime
A timeout per rollout batch — exceeding it fails the job with error.code = "training_error"

For PoC, treat this as “trusted users only.” Real isolation (gVisor / Firecracker) is on the Phase 2 roadmap.

Composing rewards

A common pattern is to weight multiple signals:

def reward(completions, answer, **kwargs):
    scores = []
    for completion, expected in zip(completions, answer):
        text = completion[-1]["content"] if isinstance(completion, list) else str(completion)

        format_score = 0.3 if has_correct_format(text) else 0.0
        correct_score = 0.5 if has_correct_answer(text, expected) else 0.0
        length_score = 0.2 * length_penalty(text, target=200)

        scores.append(format_score + correct_score + length_score)
    return scores

Keep each component bounded so the total stays in a stable range — [0, 1] is conventional. GRPO normalizes scores within a group, so the absolute scale matters less than the spread.

Multiple weighted reward functions

Instead of summing signals inside one function, you can upload several reward functions and let GRPO combine them with weights. TRL’s GRPOTrainer scores each completion with every function and sums the results, scaled by reward_weights. This keeps each signal in its own file (reusable and independently testable) and lets you retune the mix without editing code. Each function has the standard trl signature and returns one score per completion. Upload each file, then pass the ids and matching weights:

# correctness.py
def reward(completions, answer, **kwargs):
    return [1.0 if c == a else 0.0 for c, a in zip(completions, answer)]

# format.py
import re
def reward(completions, **kwargs):
    return [1.0 if re.search(r"<answer>.*?</answer>", str(c), re.DOTALL) else 0.0
            for c in completions]

correctness = client.reward_functions.upload("correctness.py")
format_fn = client.reward_functions.upload("format.py")

job = client.training_jobs.create(
    base_model="Qwen/Qwen2.5-0.5B-Instruct",
    dataset_id=ds.id,
    method="grpo",
    reward_function_ids=[correctness.id, format_fn.id],
    reward_weights=[1.0, 0.5],
    gpu_type="L4-24GB",
    gpu_count=1,
)

reward_weights is optional; omit it to weight every function equally. When provided, its length must match reward_function_ids (the SDK and API reject a mismatch). A single reward_function_id still works for the one-reward case.

Reuse across jobs

Once uploaded, a reward function is reusable. The same id can drive any number of training jobs.

job1 = client.training_jobs.create(
    base_model="Qwen/Qwen2.5-0.5B-Instruct",
    dataset_id=ds.id,
    reward_function_id=reward_fn.id,
    # ...
)
job2 = client.training_jobs.create(
    base_model="Qwen/Qwen2.5-7B-Instruct",
    dataset_id=ds.id,
    reward_function_id=reward_fn.id,  # same reward
    # ...
)

Lifecycle

client.reward_functions.list()
client.reward_functions.get(reward_id)

Reward functions are deduped by content hash — uploading the same file twice returns the same id. There is no DELETE endpoint today; programmatic delete is on the roadmap.

Where to go next

GRPO algorithm

What the score actually does inside the training loop.

Submit a job

Wire the reward into training_jobs.create.

Quickstart

End-to-end with the example math reward.

Get started

Training

Deployments

Evaluations

CLI

Volumes

Format

TRL format

Upload

Sandbox + limits

Composing rewards

Multiple weighted reward functions

Reuse across jobs

Lifecycle

Where to go next

GRPO algorithm

Submit a job

Quickstart

​Format

​TRL format

​Upload

​Sandbox + limits

​Composing rewards

​Multiple weighted reward functions

​Reuse across jobs

​Lifecycle

​Where to go next

GRPO algorithm

Submit a job

Quickstart

Format

TRL format

Upload

Sandbox + limits

Composing rewards

Multiple weighted reward functions

Reuse across jobs

Lifecycle

Where to go next