Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.adaptive-ml.com/llms.txt

Use this file to discover all available pages before exploring further.

Feedback lets you annotate completions with scalar values, boolean ratings, or pairwise preferences. Use feedback for monitoring, evaluation, and as training objectives.

​
Register a feedback key

Before logging feedback, register a key:
adaptive.feedback.register_key(
    key="quality",
    kind="scalar",  # or "bool"
    scoring_type="higher_is_better",
)
ParameterTypeRequiredDescription
keystrYesUnique identifier
kindstrNo"scalar" (default) or "bool"
scoring_typestrNo"higher_is_better" (default) or "lower_is_better"

​
Log metric feedback

Log a rating for a completion using its completion_id from the inference response:
# Get completion_id from inference
response = adaptive.chat.create(
    messages=[{"role": "user", "content": "Hello"}]
)
completion_id = response.choices[0].completion_id

# Log feedback
adaptive.feedback.log_metric(
    value=5,
    completion_id=completion_id,
    feedback_key="quality",
)
Feedback is associated with the completion’s Interaction record.

​
Log preference feedback

Log a pairwise comparison between two completions:
adaptive.feedback.log_preference(
    feedback_key="quality",
    preferred_completion=completion_id_a,
    other_completion=completion_id_b,
)
Use preferences for RLHF/DPO training when you can judge which completion is better but can’t assign absolute scores.See SDK Reference for all feedback methods.