Reward functions

A reward function turns a model completion into a number. GRPO uses these scores to compute advantages: completions above the group mean get reinforced, completions below get penalized. Rewards are not a separate resource on Veri. You attach the function’s Python source directly to the training job at submit time; the platform stores it with the job and it lives and dies with the job. There is nothing to upload, name, or clean up.

Format

Veri runs reward functions in the trl shape, the format the production GRPO runner uses.

Format	Signature	When to use
`trl`	`def reward(completions, answer, **kwargs) -> list[float]`	Used by TRL GRPOTrainer (today’s runner on NVIDIA).

At submit, the platform checks that each source parses as Python and defines a function; a violation rejects the job with HTTP 400 before anything is billed. Loading and invocation happen in the open-source runtime: see load_reward_function in training_runtime.py (and score_episode in harness_rollout.py for how harness trajectories are scored) if you need to check exactly how your function is called.

TRL format

Receives a batch of completions, returns one score per completion.

import re

def reward(completions, answer, **kwargs):
    scores = []
    for completion, expected in zip(completions, answer):
        # Handle chat-format completions (list of message dicts)
        text = completion[-1]["content"] if isinstance(completion, list) else str(completion)

        format_ok = bool(re.search(r"<answer>.*?</answer>", text, re.DOTALL))

        m = re.search(r"<answer>(.*?)</answer>", text, re.DOTALL)
        correct = bool(m and expected and expected.strip() in m.group(1).strip())

        scores.append(0.5 * float(format_ok) + 0.5 * float(correct))
    return scores

completions — list of strings OR list of message-dict lists (depending on how the runner formats prompts)
answer — list of ground-truth strings (from the dataset’s answer column)
Return — list of floats, same length as completions

Attach it to a job

The reward rides the job submission itself. Pick your surface:

# configs/train.toml
kind = "train"

[reward]
file = "./reward.py"   # path resolves against this config file's directory

job = client.training_jobs.create(
    base_model="Qwen/Qwen3-4B",
    dataset_id=dataset.id,
    method="grpo",
    reward_source=open("math_reward.py").read(),
    output_name="qwen-math",
    gpu_type="L4-24GB",
    gpu_count=1,
)

curl -X POST https://api.veri.studio/v1/training_jobs \
  -H "Authorization: Bearer $VERI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "$(jq -n --rawfile src math_reward.py '{
    base_model: "Qwen/Qwen3-4B",
    dataset_id: "ds_abc123",
    method: "grpo",
    reward_source: $src,
    output_name: "qwen-math",
    gpu: {gpu_type: "L4-24GB", gpu_count: 1}
  }')"

From the CLI, veri jobs create configs/train.toml reads [reward].file, or override it inline with --reward ./reward.py. The CLI runs an AST lint before submitting (the file parses as Python and defines at least one function), so a broken file fails locally instead of at the API. A method=grpo or method=grpo_harness job requires a reward (or environments, which is mutually exclusive with reward sources). sft_text, dpo, sft_video_gen, and custom_script jobs must omit it: the dataset is their training signal. The job response carries reward_count (the number of attached sources; null for reward-free methods). Limits: up to 8 sources per job, 256 KiB each, and every source must define a function. Violations are HTTP 400 at submit.

Sandbox + limits

Reward functions run inside the training runner. They have:

No network egress — calls to external APIs will silently fail or hang
Standard library + the runner image’s pre-installed packages only — no pip install at runtime
A timeout per rollout batch — exceeding it fails the job with error.code = "training_error"

For PoC, treat this as “trusted users only.” Real isolation (gVisor / Firecracker) is on the Phase 2 roadmap.

Composing rewards

A common pattern is to weight multiple signals:

def reward(completions, answer, **kwargs):
    scores = []
    for completion, expected in zip(completions, answer):
        text = completion[-1]["content"] if isinstance(completion, list) else str(completion)

        format_score = 0.3 if has_correct_format(text) else 0.0
        correct_score = 0.5 if has_correct_answer(text, expected) else 0.0
        length_score = 0.2 * length_penalty(text, target=200)

        scores.append(format_score + correct_score + length_score)
    return scores

Keep each component bounded so the total stays in a stable range ([0, 1] is conventional). GRPO normalizes scores within a group, so the absolute scale matters less than the spread.

Multiple weighted reward functions

Instead of summing signals inside one function, you can attach several reward sources and let GRPO combine them with weights. TRL’s GRPOTrainer scores each completion with every function and sums the results, scaled by reward_weights. This keeps each signal in its own file (independently testable) and lets you retune the mix without editing code. Each function has the standard trl signature and returns one score per completion. Pass the sources and matching weights:

# correctness.py
def reward(completions, answer, **kwargs):
    return [1.0 if c == a else 0.0 for c, a in zip(completions, answer)]

# format.py
import re
def reward(completions, **kwargs):
    return [1.0 if re.search(r"<answer>.*?</answer>", str(c), re.DOTALL) else 0.0
            for c in completions]

job = client.training_jobs.create(
    base_model="Qwen/Qwen3-4B",
    dataset_id=ds.id,
    method="grpo",
    reward_sources=[open("correctness.py").read(), open("format.py").read()],
    reward_weights=[1.0, 0.5],
    gpu_type="L4-24GB",
    gpu_count=1,
)

[reward]
files = ["./correctness.py", "./format.py"]
weights = [1.0, 0.5]

reward_weights is optional; omit it to weight every function equally. When provided, its length must match reward_sources (the SDK and API reject a mismatch). A single reward_source still works for the one-reward case.

Viewing a job’s reward

Every job keeps the exact source it was submitted with. Click a training job on the dashboard to view the reward source it was submitted with, or fetch it programmatically:

src = client.training_jobs.reward_source(job_id, index=0)  # index picks the source for multi-reward jobs

Because the source is stored per job, rerunning an old experiment is copy-paste: read the source back, tweak it, submit a new job.

Shaped rewards for harness jobs

Harness-in-the-loop jobs score the trajectory’s final assistant message by default, the deploy-aligned choice, since at inference time the final answer is all that matters. When you need more signal (partial credit for fetching the right page, penalties for invalid tool calls), declare a trajectory parameter and the platform passes the full message history of the rollout’s final turn (protocol-shaped dicts, including tool_use / tool_result content), one history per completion:

def reward(prompts, completions, answer_github, trajectory=None, **kwargs):
    rewards = []
    for i, completion in enumerate(completions):
        score = 1.0 if answer_github[i].lower() in completion.lower() else 0.0
        for message in (trajectory[i] if trajectory else []):
            content = message.get("content")
            if not isinstance(content, list):
                continue
            for block in content:
                if block.get("type") == "tool_use" and "huggingface.co/papers" in str(block.get("input", {})):
                    score += 0.1  # fetched the right page: partial credit
        rewards.append(min(score, 1.0))
    return rewards

Rewards that don’t declare trajectory are called exactly as before; the parameter is opt-in. One sharp edge with chatty agent runtimes (the Claude Agent SDK especially): because the default scores only the final message, instruct your harness to put the answer there (“finish with a last line of exactly ANSWER: <url>”) or a correct mid-trajectory answer can score zero.

Where to go next

GRPO algorithm

What the score actually does inside the training loop.

Submit a job

Wire reward_source into training_jobs.create.

Quickstart

End-to-end with the example math reward.

Get started

Training

Deployments

CLI

Volumes

Webhooks

Format

TRL format

Attach it to a job

Sandbox + limits

Composing rewards

Multiple weighted reward functions

Viewing a job’s reward

Shaped rewards for harness jobs

Where to go next

GRPO algorithm

Submit a job

Quickstart

​Format

​TRL format

​Attach it to a job

​Sandbox + limits

​Composing rewards

​Multiple weighted reward functions

​Viewing a job’s reward

​Shaped rewards for harness jobs

​Where to go next

GRPO algorithm

Submit a job

Quickstart

Format

TRL format

Attach it to a job

Sandbox + limits

Composing rewards

Multiple weighted reward functions

Viewing a job’s reward

Shaped rewards for harness jobs

Where to go next