# Release notes Source: https://docs.adaptive-ml.com/release-notes Release notes for Adaptive Engine This page contains release notes for versions of Adaptive Engine, with the most recent releases at the top. ### Breaking changes * **`Feedback` renamed to `Score`** across SDK, GraphQL API, and database. * `FeedbackType` → `ScoreType` * `feedback_key` → `score_key` (in recipe configs, SDK methods, GraphQL types) * Existing recipe JSON configs in the database are migrated automatically. SDK code referencing `FeedbackType` or `feedback_key` must be updated. * `feedback.*` SDK and GraphQL API removed — use `score.*` / `metrics.*` equivalents. `FeedbackAggregate` → `ScoreAggregate` in GraphQL. * Legacy `judge` entity removed. GraphQL queries `get_judge`, `list_judges`, `list_judge_versions` are gone, along with the underlying table. Migrate to graders (function, prebuilt, or judge-as-grader). * `promoteCheckpoint` mutation now returns `[Job]` instead of `[Model]`. Promotion is an asynchronous copy job; callers must poll the returned job(s) rather than using the models directly. * AB campaign `n_completions` parameter removed from the inference path. Existing integrations passing `n > 1` will silently ignore the parameter. * Integration topic subscriptions using the `use_case:` prefix still work via a backward-compat shim, but subscribers should migrate to the `project:` prefix. ### Structured output * New `response_format` field across the platform: pass plain text, a JSON Schema, or (in Python) a Pydantic model to constrain model output. * Python SDK: `chat.create(..., response_format=MyPydanticModel)`. Schema is auto-patched for strict-mode compatibility. * REST completion API: `response_format` plumbed through to all external providers (OpenAI, Anthropic, Gemini, legacy OpenAI). * UI chat workbench: Response Format selector in Model Settings, with a JSON-schema editor and three preset examples. * UI judge workbench: judgement schema passed as `response_format`, guaranteeing parseable output. ### Function graders * New grader type. Define a Python `grade` function that runs in a sandboxed environment. * UI: create / edit / duplicate flow in the New Grader dropdown, with a Python code editor and a Test Payload panel that validates against a sample from your dataset. * Python SDK: `create_function_grader`, `update_function_grader`, `get_grader`, `list_graders`, `validate_function_grader`. ### Checkpointing * **Promote checkpoint to model**: name a checkpoint from a run and promote it to a standalone model. Promotion runs as an asynchronous copy job (S3 server-side copy when possible) with full provenance — promoted models expose `sourceCheckpoint` in GraphQL. * Promoting a LoRA checkpoint automatically binds its backbone model to the project. * Duplicate promotions of the same checkpoint are rejected; re-promoting after soft-delete is supported. * **Multi-stage checkpointing**: checkpoint state tracked per training stage (SFT / PPO / GRPO / ENV\_PPO / ENV\_GRPO), with stage names surfaced in the UI and logger step saved/restored across resumes. ### User interface * Image gallery in chat and interaction detail; navigation to the next assistant turn from the gallery. * Model comparison UX overhaul. * New generic filter bar and operators applied consistently across Runs, Jobs, Users, Usage, and Interactions pages. * Runs: inline rename in breadcrumbs, delete action, resizable list; soft-deleted jobs hidden by default. * ISB: prefetching of neighbor interactions for instant navigation; queue time, prefill time, and model surfaced in the interaction detail panel; new Author column; "Open evaluation in ISB" shortcut. * Resizable split views across sidebar and sheet, with persisted user preference; automatic fallback to a sheet on smaller screens. * Dataset viewer: smart column selection; scoped picker; hints on empty recipe selector. * Plot UX improvements; navigation arrows on metric detail pages. * Icons and display support for new model families: Gemma 3, Llama 3, Mistral 3, Qwen 3, Qwen 3.5; Claude. * Model size always displayed in B. * Compute-pool handling: auto-selection across multiple pools; 0-GPU pools filtered out; clearer error messaging; model marked offline during deployment. ### Inference & model management * Qwen 3.5 and Qwen 3.6 families supported for inference and training (backward pass for dense and MoE text-only variants), with dedicated Triton GDN kernels and CUDA-graph dispatch for low-latency inference. * Multimodal inference hardening: webp/gif support, rejection of unsupported images, LoRA inference fixed on multimodal models, no special tokens leaked in stream. * Per-model chat templates: a model's template travels with the model directory, replacing the previous global template directory. * Fragment-first harmony API supporting function calling on external models. ### Training & runs * Checkpoint promotion and multi-stage checkpointing (see above). * SFT weighting overhaul: assistant turns are no longer force-weighted; `get_turn_weights` exposed on `StringThread` / `TokenizedThread`; thread visualization highlights trained turns in green; warnings surface when user turns have weight > 0 or when a thread has no trained turns. * RL recipes: explicit `epochs` parameter supported alongside `max_num_steps` across PPO, GRPO, ENV\_PPO, ENV\_GRPO, GSPO variants. * Runs: delete, rename, soft-delete, with matching header actions. ### Python SDK * Multimodal inference: `ChatMessage.content` widened to `Union[str, List[ContentPart]]`, with `TextContentPart` / `ImageContentPart` following the OpenAI content-parts format. * Structured output: `response_format` accepts text, JSON Schema, or a Pydantic model. * Function graders: full CRUD (`create_function_grader`, `update_function_grader`, `get_grader`, `list_graders`, `validate_function_grader`). * Dataset download: `client.datasets.download(...)`. * Update-role method; user teams and roles exposed on the user object. * New REST types: `ResponseFormat`, `Jobs.delete`, `Jobs.update`, `validate_function_grader`. ### Evaluation & data management * Function graders (see above). * Evaluation artifacts can include images. * Dataset upload flow simplified. ### Recipes * Custom `entrypoint` and `config_entrypoint` per recipe — no longer hardcoded to `main.py` / `config.py`. * Dependency installation supports `pyproject.toml` projects: `[project.dependencies]` parsed directly (build backend skipped); `src/` added to `sys.path`. * Tree-view file selector in the recipe editor. * Config form handles None defaults and enum coercion correctly. * Download a recipe as a zip archive (for non-prebuilt recipes). ### Performance * Large models load substantially faster on multi-GPU setups. Llama-3.3 70B loads roughly 4× faster on 4 GPUs. * Worker-driven speculative-decoding draft loop: per-step roundtrips collapsed into a single command, reducing spec-dec overhead. * New kernels: Deltanet; Qwen 3.5 backward (dense + MoE, text only); linear eagle3. * No gradient creation during inference. ### Administration & infrastructure * Audit logging made asynchronous. * Role editing: `updateRole` mutation and matching UI. * Registry sync resilient to individual model operation failures. * Pinned Docker image versions; Docker images trimmed. *** ### Breaking changes * **"Use Cases" renamed to "Projects"** across SDK, GraphQL API, REST API, and URL paths. * `client.use_cases` → `client.projects` (`client.use_cases` deprecated but still works) * `client.default_use_case` / `client.set_default_use_case()` → `client.default_project` / `client.set_default_project()` * `models.add_to_use_case()` → `models.add_to_project()` * `models.detach(use_case=...)` → `models.detach(project=...)` — `project` is now required * `models.deploy(use_case=...)` → `models.deploy(project=...)` * `jobs.run(use_case=...)` / `jobs.list(use_case=...)` → `jobs.run(project=...)` / `jobs.list(project=...)` * `interactions.list(use_case=...)` → `interactions.list(project=...)` * `graders.list/get/delete/lock(use_case=...)` → `graders.list/get/delete/lock(project=...)` * `recipes.list(use_case=...)` → `recipes.list(project=...)` * `feedback.get_key(feedback_key)` → `feedback.get_key(project, feedback_key)` — `project` is now a required positional argument * `feedback.create_metric()` now requires `project` as a parameter * GraphQL: `useCase` / `UseCaseData` → `project` / `ProjectData` * Integration topic patterns: existing subscriptions using the old `use_case:` prefix (e.g. `use_case:*:job:*:*`) continue to work — they are automatically matched against the new `project:` topics. No action required. * `feedback.link()` and `feedback.unlink()` removed. ### User interface * Multimodal image input in chat and interaction store: attach images alongside text for models with vision capabilities. * Interactive recipe sessions visible in run history with progress reporting, artifact uploads, and cancellation. * Multi-file recipe viewer: browse and edit individual files within a recipe. * Analytics dashboard pre-populated with default plots for new projects. * Hourly granularity in the usage dashboard. * ISB column selector: choose which columns to display in the interaction table. * Filter interactions by session ID; copy session ID from any interaction detail panel. * Download a selection of interactions as a file for offline analysis. * Delete interactions from the interaction store. * Add multiple models to a project in a single action from the model registry. * Click-to-copy model key in the project model registry. * Parent training run link shown in model detail page. * Run logs visible on the run preview sheet. ### Metrics Feedback has been unified with grader and system metrics into a single Metrics system, now scoped to projects rather than the organization. Existing feedback data has been automatically migrated. * Three categories unified under one system: system metrics (auto-computed: TTFT, latency, token counts), grader metrics (scores produced by judges and custom graders), and user metrics (custom metrics, formerly "feedback"). * Metrics are now project-scoped; the legacy `link`/`unlink` mechanism is removed. * New project Metrics page replaces the org-level Feedbacks page, with metric detail view, create/edit/delete dialogs, category filters, and search. * Metric pickers across the product group options by category (System / Grader / User). * Metrics can be deleted; system and built-in grader metrics are protected. ### Inference & model management * Speculative decoding: train an accelerated model with a built-in recipe to reduce inference latency. * Multimodal inference end-to-end: streaming for vision models, improved image encoder handling, completions API aligned with OpenAI image format. * Flash Attention 3 C++ backend with split-KV aware graph capture. ### Training & runs * KV-caching for training (diff KV cache): reduces activation sizes for long context training. * Recipe names and descriptions: set a human-readable name and description when uploading a recipe. * New built-in recipe for aligning a draft model to a target (speculative decoding). * DPO training supports dataset looping. ### SDK * `extra_params` exposed for external models in recipes and via `models.add_external()`. * New `client.create_service_account()` method. * Named interactive sessions via `name` parameter in `RecipeContext`. * `chat.create()` `store` parameter to control whether completions are saved. ### Evaluation & data management * AI Judge graders now return metadata alongside scores. * Dataset download preserves the original uploaded filename. ### Administration & infrastructure * Notifications: subscribe to webhook, Slack, or email notifications for platform events (job completion, failure) with topic-based filtering and pattern matching. * HIPAA audit logging: full audit trail for query events, file access events, and connection events, each with user identity, IP address, and timestamp. * User management: full CRUD for users at the organization level; first and last activity timestamps surfaced in the UI. * Service accounts: create dedicated bot users with API keys for CI/CD pipelines and automated workflows; API key usage is audited. * Permissions and roles visible in the UI. * Security: updated dependencies addressing CVEs. *** ### Breaking changes * SDK: `jobs.run()` parameter reorder - `args` moved from required positional to optional keyword. * SDK: `interactions` filter format - `advancedFilter` → `advancedFilters`, label filter structure changed. * SDK: `ModelserviceStatus` enum - removed `DETACHED` value. * SDK: async methods removed - `async attach()` and `async deploy()` no longer available. * SDK: `models.detach()` - `use_case` parameter now required (was optional). * SDK: `models.update()` - removed `attached` parameter. * Recipes: grader `setup()` and `teardown()` calls removed from production recipes. * Sandbox enabled by default - opt-out with `ENABLE_SANDBOX=0`. ### User interface * New sidebar presentation with split use case overview page. * New model registry UI: model details sidebar, table grouping, filtering by name/size/status. * Chat comparison page for side-by-side interaction analysis. * Raw/formatted selector and copy buttons in completion details. * Artifact status reflected in runs and evaluations UI. * Recipe upload by file with improved recipe fields display in run form. * GPU count and GPU time displayed in run details; prefill on clone run. * Improved dataset upload dialog and labels display in interaction store. * Grader tooltip showing metric column header details. ### Inference & model management * Differentiable KV cache with CuTe-DSL Flash Attention kernels. * LoRA models are now trainable directly. * Organization-level model deletion. * OpenAI-compatible external models by endpoint. * Model/use case bindings with use case stored in model registry. * Model statuses: published and stable. * LoRA backbone reuse and deduplicated spawn logic. * Configurable max prefill size via `DEFAULT_MAX_TOKENS_IN_INF_BATCH`. * TP support for FP8 MoE. * Suggested tensor parallelism logic based on model size. * Multimodal prefill splitting across batches. * Static memory usage limits via `GPU_MEMORY_UTILIZATION`. * Session claim mechanism after disconnection. ### Training & runs * Logger parameter for PPO and GRPO classes. * Multimodal dataset support. * Recipe OOM errors reported to user. * Optional defaults for recipe launch with improved upload SDK. * `recipe_key` now optional (inferred from file/directory name). * New `entrypoint` parameter for custom entry points in directories. ### SDK * New `artifacts.download(artifact_id, destination_path)` method. * New `models.add_to_use_case()` method. * `models.list()` accepts optional `filter` parameter. * `models.deploy()` signature changed: new `use_case`, `placement`, `make_default` params. * `chat.create()` new `store` parameter to control completion storage. * `JobArtifactStatus` enum: PENDING, PROCESSING, READY, ERROR. ### Evaluation & data management * Text search API on completions and prompts. * Advanced filter API for completions. * Preference dataset processing improvements. * Dataset with feedback selection in recipes. * Dataset deletion runs as background task. ### Administration & infrastructure * HIPAA compliance: model and use case reporting for chat/completions. * JWT authentication on internal API for recipes. * TLS support in Redis and Mangrove. * Sandbox improvements: HuggingFace import fixes, git support, syscall filtering. * Job permissions support. * Multiple replicas cancellation support. * Configurable RAM limits on Sandkasten. * Library upgrades for CVE fixes. *** ### Breaking changes * Model keys: model path is now used as the key in the model registry instead of other identifiers. ### Inference & model management * Support custom endpoint URL for OpenAI Completions API-compatible external models. * Make API keys optional for external models. * Support per-request tool override for chat completions. * New connectivity check endpoint (ADAPTIVE\_URL/health). * New `min_gen_len` setter method for `InferenceModel` and `TrainingModel`. * Support GPU inference partition resize in SDK. * Improve FP8 MoE inference. ### Training & runs * First [GSPO](https://arxiv.org/abs/2507.18071) (Group Sequence Policy Optimization) implementation. * Add support for context with assistant turn on multi-turn generation with `env_grpo`. * Improved model saving and checkpointing. * New `skip_nan_gradients` argument in `model.optim_step()`. * Multi-file recipe support with proper frontend handling. * Recipe schema improvements and better parsing. ### Evaluation & data management * Labels and feedback annotations in chat UI and interaction store. * Multimodal dataset support with image handling. * Ability to upload GB-sized datasets. * Enhanced dataset creation with immutable dataset files for reuse in recipes. * Improved dataset artifact management. ### Administration & infrastructure * Add preflight checks to ensure environment compatibility. * Better resource management and allocation tracking. *** ### User interface * Global search bar with cmd+K. ### Inference & model management * Allow external model spawning via HTTP. * Support for detached models in chat. * Rich magic for better REPL experience. * Better timeout configuration for client SDK. ### Training & runs * Display active run in compute pool detail page. * Enrich the parameters of the SFT recipe. * Rejection sampling production recipe. * GRPO KL divergence fix. * Better handling of `env_grpo` sample loading. * Improved callback system for training. ### Evaluation & data management * AI Judge workbench v3 with enhanced UI. * Custom grader support in product. * Dataset viewer page introduction. * Improved dataset chunked upload in SDK. * Better dataset source tracking. * Visual improvements to interaction store browsing. * External feedback endpoint in evaluation wizard. ### Administration & infrastructure * Dynamic world size support (experimental). * GPU metrics and Redis connection management cleanup. *** ### Breaking changes * Model output artifact changes for better organization. ### User interface * New *recipes*-centric use case navigation. * New *split view* to better navigate runs. ### Inference & model management * MCP (Model Context Protocol) with all turns support. * System prompt support in chat settings. * Temperature control in chat settings. * Better external model handling and API integration. ### Training & runs * Loss clamping support. * Callbacks and training recipe cleanup. * Better skip-token-masking loss computation. * Improved dataset shuffling with seeding. ### Evaluation & data management * Judge Playground v2 with enhanced UI. * Parse XML stream from response in chat. * User metadata support in interaction store. * Dataset generation from interaction store filters. * Metric aggregation controls in header. * Better evaluation result reporting. * Dataset artifacts with proper management. * Improved interaction state persistence in `localStorage`. ### Administration & infrastructure * Better resource management logging. * Add UI controls to reset and resize GPU inference partitions. *** ### Breaking changes * New message format migration for completions. ### User interface * Better form input styles following Epoch Design System. ### Inference & model management * Add model search in model registry. * Model service configuration improvements. * Better DMA (Direct Memory Access) handling. * External model API key management improvements. * OpenAI Response API support. * Model conversion re-added with better handling. ### Training & runs * Better model initialization fixes. * Improved training callback system. ### Evaluation & data management * Pre-built criteria with documentation links. * Increase robustness of Amazon S3 support integration in custom recipes. * Better recipe editor UI. * Enhanced interaction store with tooltip for all turns. * Improve MLFlow integration: let users view their use cases runs. * Add utility to upload and update custom recipes. ### Administration & infrastructure * Error management improvements with new error pages. * Contract usage reporting. ### Training & runs * Shuffling in GRPO with better batching. * Improved built-in recipes with better parameters. * Grader evaluation support in Harmony. ### Evaluation & data management * Tool providers CRUD operations. * Ability to link tool providers with model services. * Custom grader support with enhanced UI. * New summarization recipe. * Interaction store general UI refactor. * UX improvement in the AI judge workbench. * Evaluation error reporting in new evaluations page. ### Administration & infrastructure * Team permission selector in use case creation UI. * Job partition improvements. * Kill router on connection drop. * Add API to create and delete users ahead of their SSO registration. ### Use interface * Introduce use case overview dropdown. * Read-only permission UI. ### Inference & model management * Extending integration of Google API models. * Expose `max_ttft` parameter at request level. ### Evaluation & data management * Refactoring judge & prompt playground. * Preset metric visibility in the side-by-side view. * Improvement to built-in AI judges. * New evaluation wizard. * New evaluation results table. * Better evaluation exports. * Adding support for grader evaluation in custom recipes. ### Training & runs * Custom recipes improvements. * Adding job partition concept, allowing to run on subset of available GPUs. * Adding Infiniband health check. * Better training arguments and world size requirements removed. ### Administration & infrastructure * Add team removal method in SDK. * Display available GPU partitions. ### User interface * New design system. * Read-only permission UI. * Improve Hugging Face model import UI. ### Evaluation & data management * Adding `source` metadata to identify origin of datasets. * Extend evaluation to support more models evaluated in parallel. * UI for external feedback endpoints (RLEF). * Access individual records from feedback detail page. * Support optional metadata saving in data generation jobs. ### Training & runs * Increase KV cache length in GRPO recipe. * Create dedicated URLs for run detail pages. * Expose more RL parameters in the training API. * Support journaling & replay in reward servers (RLEF). * Add APIs for RAG dataset generation. * Multi-judge training SDK. ### User interface * Add use case search. * New use case-centric navigation. ### Inference & model management * Add support for Anthropic and NVIDIA NIM external models. * Add compute configuration (placement) to model endpoints. * Improve tokenization speed. * Custom inference kernels for A100, L40S, H100, H200. * Add richer inference metadata: parameters, latencies. ### Evaluation & data management * Display interaction metadata in interaction detail page. * Export raw interactions (JSONL) from the interaction store. * Add ability to evaluate existing completions. ### Training & runs * Addition of GRPO. * Display validation in training run details. * Better OOM management. * Improve granular timestamp reporting. * Improve custom attention. * Remove sync points in training. * Improve job status UI. ### Administration & infrastructure * Grafana logs integration. ### Inference & model management * Add support for inference autoscaling on Kubernetes. ### Evaluation & data management * Filter feedback by label. * Improve feedback display in interaction detail page. * Add support for annotation (scalar, boolean, text comments) in the interaction store. ### Training & runs * Add Tensorboard integration. * Improve built-in SFT recipe & add SFT-specific UI launcher. * Add reward servers (RLEF) in SDK. ### Administration & infrastructure * Expose concept of compute pools. ### Inference & model management * Adding concept of deployment placement, to manage partitioning and distribution of resources. ### Training & runs * Extend Weight & Bias integration to multi-step jobs. * New UI to directly train on uploaded dataset. ### Inference & model management * Integration with Azure OpenAI endpoints. ### Evaluation & data management * New UI to enter granular AI judge policies for evaluation and training. * New dataset upload & browsing page. ### Administration & infrastructure * Extend permission management APIs and default team behavior. * GPU memory management improvements. # Authentication Source: https://docs.adaptive-ml.com/v0.14/advanced/authentication Configure API access, service accounts, and HTTP requests ## Find your credentials You need two values from your Adaptive deployment: | Credential | Description | | ------------------ | --------------------- | | `ADAPTIVE_URL` | Your deployment URL | | `ADAPTIVE_API_KEY` | Your personal API key | Click the navigation switcher in the top left corner (labeled *View all projects*) and select **Generate new API key**. Your deployment URL is provided by your administrator. ## Set environment variables ```bash theme={null} export ADAPTIVE_URL="https://your-deployment.adaptive-ml.com" export ADAPTIVE_API_KEY="your-api-key" ``` The SDK and HTTP clients use these environment variables to connect to Adaptive. ## Service accounts Create service accounts for CI/CD pipelines, automation, and programmatic integrations that don't belong to a human user. ```python theme={null} result = adaptive.users.create_service_account( name="ci-bot", teams_with_role=[("engineering", "power_user")] ) # result.api_key is returned once; store it securely ``` The API key is returned once at creation and cannot be retrieved later. Store it immediately. Authenticate the same way as personal accounts: set `ADAPTIVE_API_KEY` to the returned key. Service accounts follow the same role and team permission model as regular users. Use the OpenAI Python library with your Adaptive deployment: ```python theme={null} import os from openai import OpenAI client = OpenAI( base_url=f"{os.environ['ADAPTIVE_URL']}/api/v1", api_key=os.environ["ADAPTIVE_API_KEY"], ) response = client.chat.completions.create( model="project_key/model_key", messages=[{"role": "user", "content": "Hello!"}], ) ``` Set `model` to `project_key/model_key`. Include your API key in the `Authorization` header: ``` Authorization: Bearer ADAPTIVE_API_KEY ``` ```bash theme={null} curl "$ADAPTIVE_URL/api/v1/chat/completions" \ -X POST \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $ADAPTIVE_API_KEY" \ -d '{ "model": "project_key/model_key", "messages": [{"role": "user", "content": "Hello!"}] }' ``` # Integrations Source: https://docs.adaptive-ml.com/v0.14/advanced/integrations External models, LangChain, and notifications Connect external models or use Adaptive with LangChain. ## External models Connect proprietary models (OpenAI, Azure, Google, Anthropic, NVIDIA NIM) to use them through the Adaptive API with interaction and metrics logging. ### OpenAI (direct) ```python theme={null} external_model = adaptive.models.add_external( provider="open_ai", external_model_id="GPT4O", name="GPT-4o", api_key="OPENAI_API_KEY" ) ``` Supported model IDs: `GPT4O`, `GPT4O_MINI`, `GPT4`, `GPT4_TURBO`, `GPT3_5_TURBO` ### Azure OpenAI ```python theme={null} external_model = adaptive.models.add_external( provider="azure", external_model_id="DEPLOYMENT_NAME", name="Azure GPT-4o", api_key="AZURE_API_KEY", endpoint="https://aoairesource.openai.azure.com" ) ``` The `external_model_id` is your deployment name, and `endpoint` is your Azure OpenAI subscription endpoint. ### Google ```python theme={null} external_model = adaptive.models.add_external( provider="google", external_model_id="gemini-1.5-pro", name="Gemini 1.5 Pro", api_key="GOOGLE_API_KEY" ) ``` Supported models: [Gemini model variations](https://ai.google.dev/gemini-api/docs/models/gemini#model-variations) (excluding embeddings). ### Anthropic ```python theme={null} external_model = adaptive.models.add_external( provider="anthropic", external_model_id="claude-sonnet-4-5-20250929", name="Claude Sonnet 4.5", api_key="ANTHROPIC_API_KEY" ) ``` Use the model ID from [Anthropic's API documentation](https://platform.claude.com/docs/en/about-claude/models/overview) as the `external_model_id`. Once connected, attach the model to a project and make inference requests like any other model. ## LangChain Adaptive is compatible with LangChain chat model classes. ### ChatOpenAI ```python theme={null} from langchain_openai import ChatOpenAI import os os.environ["OPENAI_API_KEY"] = "ADAPTIVE_API_KEY" llm = ChatOpenAI( model="project_key/model_key", # model_key is optional base_url="ADAPTIVE_URL/api/v1", ) messages = [ ("system", "You are a helpful assistant that translates English to French."), ("human", "I love programming."), ] response = llm.invoke(messages) ``` ### ChatGoogleGenerativeAI ```python theme={null} from langchain_google_genai import ChatGoogleGenerativeAI import os os.environ["GOOGLE_API_KEY"] = "ADAPTIVE_API_KEY" llm = ChatGoogleGenerativeAI( model="project_key", client_options={"api_endpoint": "ADAPTIVE_URL/api/v1/google"}, transport="rest", ) messages = [ ("system", "You are a helpful assistant that translates English to French."), ("human", "I love programming."), ] response = llm.invoke(messages) ``` ## Platform notifications Subscribe to platform events (job completion, failures, model status changes) and receive alerts through Slack, email, or webhook. ### Topic patterns Notifications use a topic-based system with pattern matching. Subscribe to specific events or use wildcards: | Pattern | Matches | | ----------------------------------- | ---------------------------------------- | | `project:*:job:**:completed` | Any job completion in any project | | `project:my-project:job:**:failed` | Failed jobs in a specific project | | `project:*:model:**:status_changed` | Model status changes across all projects | Use `*` to match a single segment and `**` to match multiple segments. ### Integration types Configure notifications in **Settings → Integrations**. Three delivery methods are available: | Type | Setup | Details | | ----------- | --------------------------------- | --------------------------------------- | | **Slack** | Webhook URL, optional bot token | Posts to a channel via incoming webhook | | **Email** | SMTP server configuration | Sends to specified recipients | | **Webhook** | HTTP endpoint URL, custom headers | POST request with event payload | Each integration can subscribe to different topic patterns. You can create multiple integrations of the same type for different channels or recipients. ### Permissions Creating and managing integrations requires `integration:create`, `integration:read`, `integration:update`, or `integration:delete` permissions. See [Permissions](/v0.14/advanced/permissions) for role configuration. No SDK resource exists for notifications yet. To configure integrations programmatically, use the GraphQL API directly. Contact your administrator for the schema details. # GPU management Source: https://docs.adaptive-ml.com/v0.14/advanced/optimization Compute pools, inference tuning, and GPU utilization Tune inference throughput, latency, and GPU utilization. ## Adapters Adapters share the same backbone model, increasing batching opportunities and reducing memory. The default limit is 4 adapters per base model. Contact support to increase this. ## KV-cache The stateful KV-cache reuses computations from previous tokens and chat turns in the same session. ```python theme={null} adaptive.models.update_compute_config( model="", compute_config={ "kv_cache_len": 300000 } ) ``` Set the cache size in tokens. The combined memory of model weights and KV-cache must fit in GPU memory. **Tuning**: Larger cache enables more batching and computation reuse. Tune iteratively based on your traffic patterns. ## Tensor parallelism TP (tensor parallelism) is the number of GPUs over which a model replica is partitioned. ```python theme={null} adaptive.models.update_compute_config( model="", compute_config={ "tp": 4 } ) ``` **Trade-offs:** * Higher TP → lower per-GPU memory → more room for KV-cache * Higher TP → lower latency * Adaptive Engine automatically creates replicas based on TP, available GPUs, and traffic ## Speculative decoding Reduce inference latency without changing output quality. A small "draft" model proposes multiple tokens, the target model verifies them in one forward pass, and accepted tokens are kept. See [Speculative Decoding Visualized](https://www.adaptive-ml.com/post/speculative-decoding-visualized) for an in-depth explanation. ### Train a draft model ```python theme={null} adaptive.jobs.run( recipe_key="specdec_alignment", num_gpus=2, args={ "target_model": "qwen-3-8b", "draft_model": "qwen-3-500m", "dataset": "my-prompts", }, ) ``` The draft must share the target's tokenizer (same model family). The output is an accelerated model named `_accelerated` by default. ### Deploy with speculative decoding ```python theme={null} adaptive.models.deploy( model="qwen-3-8b_accelerated", num_draft_steps=4, wait=True, ) ``` `num_draft_steps` sets how many tokens the draft proposes per step (1 to 8). The recipe logs **acceptance rate** during training: the fraction of draft tokens the target accepts. Higher acceptance rate means more speedup. ## Timeouts Three timeout controls for 408/timeout errors: | Setting | Default | Description | | ------------------------------------ | ------- | ------------------------------------------------- | | `ADAPTIVE_SERVER__TIMEOUT_SECONDS` | 60s | HTTP request timeout in control plane | | `ADAPTIVE_HTTP_CLIENT__TIMEOUT_SECS` | 60s | Control plane to dependencies timeout | | `max_ttft` | 60s | Max time before first token (includes queue time) | Set `max_ttft` via SDK: ```python theme={null} adaptive.models.update( model="", placement={ "max_ttft_ms": 60000 } ) ``` ## Compute pools Compute pools are isolated partitions of your cluster. Deploy models to multiple pools for reliability. ```python theme={null} adaptive.models.update( model="", placement={ "compute_pools": ["pool-1", "pool-2"], } ) ``` Configure autoscaling via the Helm chart. See [self-hosting](/v0.14/deploy/self-hosting). ## Additional tips **Models:** * Use FP8 quantization * Use smaller models - they often outperform frontier models after fine-tuning **Database:** * Use a properly-sized database near your inference machines (permission checks happen on every inference call) ## FAQ **Why does TP=1 use more per-GPU memory than TP=N?** With TP=N, the model is sharded across N GPUs, so each GPU holds 1/N of the weights. **What's the connection between TP and KV-cache?** Higher TP frees more memory per GPU, leaving more room for KV-cache. **What if there's not enough memory for model + KV-cache?** Model loading fails with an insufficient memory error. **Why is GPU utilization low?** If traffic reuses the KV-cache, Adaptive Engine returns cached results instead of recomputing. This means lower GPU usage but also lower latency. # Permissions Source: https://docs.adaptive-ml.com/v0.14/advanced/permissions Manage permissions, roles and teams Configure roles, permissions, and teams to control what users can do in your Adaptive deployment. * **Permissions** grant ability to perform actions (e.g., `project:adapt` allows fine-tuning) * **Roles** are named sets of permissions, assignable to users * **Teams** are groups with shared access to projects and resources Users can belong to multiple teams with different roles in each. ## Common operations ```python theme={null} # Create a user adaptive.users.create( email="user@example.com", name="John Doe", teams_with_role=[("customer-support-team", "admin")] ) # Add user to team adaptive.users.add_to_team( email="user@example.com", team="customer-support-team", role="admin" ) # Remove user from team adaptive.users.remove_from_team( email="user@example.com", team="customer-support-team" ) # Share project with team adaptive.projects.share( project="project-to-share", team="customer-support-team", role="admin", is_owner=False ) # Create a team adaptive.teams.create( key="customer-support-team", name="Customer Support Team", ) # Create a custom role adaptive.roles.create( key="new-role", permissions=["project:read", "project:create"] ) ``` ## Admin team The reserved `admin` team controls global operations. Members with `admin` or `platform_admin` role can: * Create new roles and teams * Add/remove users from any team Configure seed admins in `values.yaml`: ```yaml theme={null} auth: default_role: admin # admin, platform_admin, power_user, read-only, inference, annotator default_team: default admins: [] # Email addresses for initial admins ``` ## Default roles | Role | Key | Description | | -------------- | ---------------- | ------------------------------------------------------------------ | | Admin | `admin` | All permissions. Can create teams/roles only if in `admin` team. | | Platform Admin | `platform-admin` | Manages roles, teams, and users. Requires `admin` team membership. | | Power User | `power_user` | Full project management, training, evaluation, model deployment. | | Read Only | `read-only` | View project details only. | | Inference | `inference` | Interact with models attached to team projects. | | Annotator | `annotator` | Interact with models and add feedback. | ## All permissions ### Admin | Permission | Scope | | -------------------- | ---------------------------------------------------------- | | `admin:manage_roles` | List roles (any team), create roles (admin team) | | `admin:manage_teams` | List teams (any team), create teams (admin team) | | `admin:manage_users` | List/update users in same team, global update (admin team) | ### Project | Permission | Scope | | ----------------------------- | -------------------------------------------------------- | | `project:read` | View project details | | `project:create` | Create projects | | `project:update` | Update project details | | `project:manage_models` | Attach/detach models | | `project:adapt` | Train models | | `project:evaluate` | Evaluate models | | `project:interact` | Chat with models | | `project:add_feedback` | Add interaction feedback | | `project:read_interactions` | View logged interactions | | `project:update_interactions` | Update interaction metadata | | `project:delete_interactions` | Delete logged interactions | | `project:share` | Share with another team (requires both team memberships) | | `project:judge_*` | Manage judges (create, read, update, delete) | | `project:grader_*` | Manage graders (create, read, update, delete) | | `project:tool_provider_*` | Manage tool providers | | `project:job_*` | Manage jobs (create, read, cancel) | | `project:custom_script_*` | Manage custom scripts | | `project:metric_*` | Manage metrics (create, read, update, delete) | ### Resources | Permission | Scope | | --------------------- | --------------------------------------------- | | `dataset:create` | Create datasets | | `metric:create` | Create feedback keys | | `model:manage_models` | Deploy/terminate models globally (admin team) | | `team:manage` | Update users in same team | ### Integrations | Permission | Scope | | -------------------- | ------------------- | | `integration:read` | View integrations | | `integration:create` | Create integrations | | `integration:update` | Update integrations | | `integration:delete` | Delete integrations | ### Remote environments | Permission | Scope | | ------------------- | -------------------------- | | `remote_env:read` | View remote environments | | `remote_env:create` | Create remote environments | | `remote_env:update` | Update remote environments | | `remote_env:delete` | Delete remote environments | # Reward servers Source: https://docs.adaptive-ml.com/v0.14/advanced/reward-servers Deploy custom grading functions as external servers Reward servers let you deploy custom grading logic as an external API that Adaptive Engine calls during training and evaluation. Use reward servers when your reward depends on external systems like databases, simulated environments, or sandboxes. ## Building a reward server Implement a subclass of `RewardServer` with two methods: ```python theme={null} from adaptive_sdk.external import RewardServer, ValidatedRequest, Response, ServerInfo from pydantic import BaseModel, Field # Define metadata schema (validated against dataset records) class ScaryLetterMetadata(BaseModel): scary_letter: str = Field(min_length=1, max_length=1) class ScaryLetterRewardServer(RewardServer[ScaryLetterMetadata]): def __init__(self, port: int = 8000, blocking: bool = True, **kwargs): super().__init__(port, ScaryLetterMetadata, blocking, **kwargs) async def score(self, request: ValidatedRequest[ScaryLetterMetadata]) -> Response: last_completion = request.turns[-1].content has_scary = request.metadata.scary_letter in last_completion return Response( reward=0.0 if has_scary else 1.0, metadata={"feedback": "No scary letters!" if not has_scary else "Found scary letters!"} ) async def info(self) -> ServerInfo: return ServerInfo( version="1.0", name="Scary Letter Detector", description="Rewards completions that avoid a specific letter" ) if __name__ == "__main__": server = ScaryLetterRewardServer(port=50056) ``` The `score` method receives: * `request.turns` - conversation history (list of role/content pairs) * `request.metadata` - metadata from dataset record Returns: * `reward` - float score * `metadata` - optional dict with feedback or debug info ### No metadata If your reward doesn't need metadata, use `EmptyMetadata`: ```python theme={null} from adaptive_sdk.external import RewardServer, EmptyMetadata, ValidatedRequest, Response, ServerInfo class LengthRewardServer(RewardServer[EmptyMetadata]): def __init__(self, port: int = 8000, blocking: bool = True, **kwargs): super().__init__(port, EmptyMetadata, blocking, **kwargs) async def score(self, request: ValidatedRequest[EmptyMetadata]) -> Response: completion = request.turns[-1].content return Response(reward=min(len(completion) / 100, 1.0), metadata={}) async def info(self) -> ServerInfo: return ServerInfo(version="1.0", name="Length Rewarder", description="Rewards longer responses") if __name__ == "__main__": server = LengthRewardServer(port=50056) ``` ### SQL execution example A more realistic example that executes SQL queries and compares results: ```python theme={null} import sqlite3 import os import pandas as pd from adaptive_sdk.external import RewardServer, ValidatedRequest, Response, ServerInfo from pydantic import BaseModel, Field from typing import List, Dict, Any class SQLMetadata(BaseModel): ground_truth_results: List[Dict[str, Any]] db_path: str class SQLRewardServer(RewardServer[SQLMetadata]): def __init__(self, db_base_path: str, port=8000, blocking=True, **kwargs): self.db_base_path = db_base_path super().__init__(port, SQLMetadata, blocking, **kwargs) async def score(self, request: ValidatedRequest[SQLMetadata]) -> Response: sql_query = request.turns[-1].content if not sql_query.upper().startswith("SELECT"): return Response(reward=-1.0, metadata={"status": "invalid_query"}) try: conn = sqlite3.connect(os.path.join(self.db_base_path, request.metadata.db_path)) df_actual = pd.read_sql_query(sql_query, conn) actual_results = df_actual.to_dict(orient='records') match = self._results_match(actual_results, request.metadata.ground_truth_results) return Response( reward=float(match), metadata={"status": "success" if match else "wrong_result"} ) except Exception as e: return Response(reward=-1.0, metadata={"status": "error", "message": str(e)}) finally: if 'conn' in locals(): conn.close() def _results_match(self, actual, expected): if len(actual) != len(expected): return False actual_set = set(frozenset(d.items()) for d in actual) expected_set = set(frozenset(d.items()) for d in expected) return actual_set == expected_set async def info(self) -> ServerInfo: return ServerInfo(version="1.0", name="SQL Evaluator", description="Evaluates SQL queries") if __name__ == "__main__": server = SQLRewardServer(db_base_path="/path/to/dbs/", port=50056) ``` ## Testing locally Use `RewardClient` to test before deployment: ```python theme={null} import asyncio from adaptive_sdk.external import RewardClient, Request, Turn async def test_reward_server(): client = RewardClient(base_url="http://localhost:50056") response = await client.score( Request( turns=[ Turn(role="user", content="What sea touches Stockholm?"), Turn(role="assistant", content="Baltic Sea."), ], metadata={"scary_letter": "d"}, ) ) assert response.reward == 1.0 asyncio.run(test_reward_server()) ``` ## Deployment ### Docker ```dockerfile theme={null} FROM python:3.10-slim COPY . /app WORKDIR /app RUN pip install -r requirements.txt CMD ["python", "sql_reward_server.py"] ``` ```bash theme={null} docker build -t sql-reward-server:1.0 . ``` ### Docker Compose ```yaml theme={null} sql-reward-server: image: sql-reward-server:1.0 restart: on-failure healthcheck: test: ["CMD-SHELL", "curl -f http://localhost:50056/info || exit 1"] interval: 30s timeout: 10s retries: 5 ``` Access at `http://sql-reward-server:50056` within the Docker network. ### Kubernetes ```yaml theme={null} apiVersion: v1 kind: Service metadata: name: sql-reward-server spec: ports: - port: 50056 targetPort: 50056 selector: app: sql-reward-server --- apiVersion: apps/v1 kind: Deployment metadata: name: sql-reward-server spec: replicas: 1 selector: matchLabels: app: sql-reward-server template: metadata: labels: app: sql-reward-server spec: containers: - name: sql-reward-server image: sql-reward-server:1.0 ports: - containerPort: 50056 livenessProbe: httpGet: path: /info port: 50056 initialDelaySeconds: 30 periodSeconds: 30 ``` Access at `http://sql-reward-server:50056` (same namespace) or `http://sql-reward-server..svc.cluster.local:50056`. ## Connecting to Adaptive Engine Test connectivity: ```python theme={null} result = adaptive.graders.test_external_endpoint("http://sql-reward-server:50056") if hasattr(result, "error"): print(f"Error: {result.error}") else: print(f"Connected: {result.name}") ``` Register as a grader: ```python theme={null} adaptive.graders.create.external_endpoint( key="sql-reward-server", url="http://sql-reward-server:50056", feedback_key="sql-execution-reward", project="my-project", ) ``` List and delete graders: ```python theme={null} graders = adaptive.graders.list(project="my-project") adaptive.graders.delete(grader_key="sql-reward-server", project="my-project") ``` # Single Sign-On (SSO) Source: https://docs.adaptive-ml.com/v0.14/advanced/sso Configure OIDC identity providers for user login Adaptive Engine authenticates users through OpenID Connect (OIDC). You configure one or more identity providers, and users log in through your organization's existing identity system. ## How it works When a user clicks "Log in" on the Adaptive Engine UI: 1. The browser redirects to your identity provider (Google, Azure Entra ID, Okta, etc.) 2. The user authenticates with their organization credentials 3. The provider redirects back to Adaptive Engine with an authorization code 4. Adaptive Engine exchanges the code for an ID token, extracts the user's email and name, and creates a session The entire flow uses the Authorization Code grant with PKCE for security. ## Configuration Configure OIDC providers in your Helm `values.yaml` under `secrets.auth.oidc.providers`. Each provider is an entry in the list: ```yaml theme={null} secrets: auth: oidc: providers: - name: "Google" key: "google" issuer_url: "https://accounts.google.com" client_id: "your-client-id.apps.googleusercontent.com" client_secret: "your-client-secret" scopes: ["email", "profile"] pkce: true allow_sign_up: true ``` You can configure multiple providers. Each one appears as a separate login option on the sign-in page. ### Provider fields | Field | Required | Default | Description | | ------------------------ | -------- | ------- | -------------------------------------------------------------------------------------------------- | | `name` | Yes | — | Display name on the login page | | `key` | Yes | — | URL-safe identifier (e.g., `google`, `azure-ad`). Must be unique across providers | | `issuer_url` | Yes | — | OIDC discovery endpoint. Adaptive Engine fetches `/.well-known/openid-configuration` from this URL | | `client_id` | Yes | — | OAuth2 client ID from your identity provider | | `client_secret` | No | — | OAuth2 client secret. Required by most providers, optional for public clients | | `scopes` | Yes | — | OIDC scopes to request. Always include `email` and `profile` | | `pkce` | No | `true` | Enable Proof Key for Code Exchange. Keep enabled unless your provider does not support it | | `allow_sign_up` | No | `false` | Create new users on first login. When `false`, users must be pre-created via the SDK | | `allowed_domains` | No | `[]` | Restrict login to specific email domains. Empty means all domains are allowed | | `require_email_verified` | No | `true` | Reject users whose email is not verified by the provider | ### Redirect URI Set your identity provider's redirect URI to: ``` https://YOUR_DEPLOYMENT_URL/api/v1/auth/login/{key}/callback ``` Replace `{key}` with the provider key from your configuration (e.g., `google`, `okta`). The `root_url` in your control plane configuration must match the public URL users access. OIDC redirects fail if this is misconfigured. ## User provisioning When a user logs in through OIDC for the first time, Adaptive Engine either finds an existing account or creates a new one (if `allow_sign_up: true`). New users are assigned to the team and role defined in your auth configuration: ```yaml theme={null} auth: default_team: "default" default_role: "read-only" admins: ["admin@company.com", "ops@company.com"] ``` | Field | Description | | -------------- | ------------------------------------------------------------------------------------------------------------ | | `default_team` | Team assigned to new users on first login | | `default_role` | Role assigned within that team (see [Permissions](/v0.14/advanced/permissions)) | | `admins` | Email addresses promoted to admin on first login. Set these **before** those users log in for the first time | If `allow_sign_up: true`, anyone who can authenticate with your OIDC provider can access Adaptive Engine. Use `allowed_domains` or set `allow_sign_up: false` and pre-create users via the SDK to restrict access. ## Security features ### PKCE Proof Key for Code Exchange prevents authorization code interception attacks. Enabled by default and recommended for all providers. Only disable it if your provider explicitly does not support PKCE. ### Email domain restriction Limit which email domains can log in per provider: ```yaml theme={null} - name: "Corporate Google" key: "google" issuer_url: "https://accounts.google.com" client_id: "your-client-id" client_secret: "your-client-secret" scopes: ["email", "profile"] allowed_domains: ["company.com", "subsidiary.com"] ``` Users with emails outside these domains are rejected at login. Domain matching is case-insensitive. ### Email verification By default, Adaptive Engine requires the OIDC provider to confirm the user's email is verified (`email_verified` claim). This prevents account hijacking through unverified email addresses. Set `require_email_verified: false` only for development environments or providers that do not include the `email_verified` claim. ## Provider examples ### Google Workspace 1. Go to the [Google Cloud Console](https://console.cloud.google.com/apis/credentials) 2. Create an OAuth 2.0 client ID (application type: **Web application**) 3. Add `https://YOUR_DEPLOYMENT_URL/api/v1/auth/login/google/callback` as an authorized redirect URI 4. Copy the client ID and secret ```yaml theme={null} - name: "Google" key: "google" issuer_url: "https://accounts.google.com" client_id: "123456789.apps.googleusercontent.com" client_secret: "GOCSPX-..." scopes: ["email", "profile"] pkce: true allow_sign_up: true allowed_domains: ["company.com"] ``` ### Microsoft Entra ID (Azure AD) 1. In the [Azure Portal](https://portal.azure.com), go to **Entra ID** > **App registrations** > **New registration** 2. Set the redirect URI to `https://YOUR_DEPLOYMENT_URL/api/v1/auth/login/azure/callback` (type: **Web**) 3. Under **Certificates & secrets**, create a new client secret 4. Note the **Application (client) ID** and **Directory (tenant) ID** ```yaml theme={null} - name: "Microsoft" key: "azure" issuer_url: "https://login.microsoftonline.com/YOUR_TENANT_ID/v2.0" client_id: "your-application-id" client_secret: "your-client-secret" scopes: ["email", "profile"] pkce: true allow_sign_up: true ``` Replace `YOUR_TENANT_ID` with your Azure directory tenant ID. Use `common` instead of a tenant ID to allow any Microsoft account. ### Okta 1. In the Okta Admin Console, go to **Applications** > **Create App Integration** 2. Select **OIDC - OpenID Connect** and **Web Application** 3. Set the sign-in redirect URI to `https://YOUR_DEPLOYMENT_URL/api/v1/auth/login/okta/callback` 4. Copy the client ID and secret from the application settings ```yaml theme={null} - name: "Okta" key: "okta" issuer_url: "https://your-org.okta.com" client_id: "your-client-id" client_secret: "your-client-secret" scopes: ["email", "profile"] pkce: true allow_sign_up: true ``` If you use a custom authorization server, the issuer URL is `https://your-org.okta.com/oauth2/YOUR_AUTH_SERVER_ID`. ### Keycloak 1. In the Keycloak Admin Console, create a new client in your realm 2. Set **Client authentication** to **On** (confidential client) 3. Add `https://YOUR_DEPLOYMENT_URL/api/v1/auth/login/keycloak/callback` as a valid redirect URI 4. Copy the client ID and secret from the **Credentials** tab ```yaml theme={null} - name: "Keycloak" key: "keycloak" issuer_url: "https://keycloak.company.com/realms/YOUR_REALM" client_id: "adaptive-engine" client_secret: "your-client-secret" scopes: ["email", "profile"] pkce: true allow_sign_up: true ``` ### Amazon Cognito 1. In the [AWS Console](https://console.aws.amazon.com/cognito), create or select a User Pool 2. Under **App integration**, create an app client with a client secret 3. Add `https://YOUR_DEPLOYMENT_URL/api/v1/auth/login/cognito/callback` as a callback URL 4. Note the User Pool ID and region ```yaml theme={null} - name: "Amazon Cognito" key: "cognito" issuer_url: "https://cognito-idp.REGION.amazonaws.com/POOL_ID" client_id: "your-client-id" client_secret: "your-client-secret" scopes: ["email", "profile"] pkce: true allow_sign_up: true ``` Replace `REGION` with your AWS region (e.g., `us-east-1`) and `POOL_ID` with your User Pool ID (e.g., `us-east-1_aBcDeFgHi`). ## Multiple providers You can configure several providers at once. Each appears as a login option: ```yaml theme={null} secrets: auth: oidc: providers: - name: "Corporate SSO" key: "azure" issuer_url: "https://login.microsoftonline.com/TENANT_ID/v2.0" client_id: "..." client_secret: "..." scopes: ["email", "profile"] allowed_domains: ["company.com"] allow_sign_up: true - name: "Partner Access" key: "okta" issuer_url: "https://partner-org.okta.com" client_id: "..." client_secret: "..." scopes: ["email", "profile"] allow_sign_up: false ``` This setup lets corporate users sign up automatically through Azure AD, while partner users must be pre-created via the SDK and can then log in through Okta. # Datasets Source: https://docs.adaptive-ml.com/v0.14/core/datasets Upload datasets for training and evaluation Datasets contain examples for training and evaluating models. Upload them as JSONL files. ## Upload a dataset ```python theme={null} adaptive.datasets.upload( file_path="training-data.jsonl", dataset_key="customer-support-v1", ) ``` | Parameter | Type | Required | Description | | ------------- | ---- | -------- | ---------------------------------------- | | `file_path` | str | Yes | Path to JSONL file | | `dataset_key` | str | Yes | Unique identifier | | `name` | str | No | Display name (defaults to `dataset_key`) | ## Dataset formats Each line in your JSONL file must follow one of these schemas: **Prompts and completions** (most common): ```json theme={null} {"messages": [{"role": "user", "content": "Hello"}], "completion": "Hi there!"} ``` **Prompts only** (for evaluation with generated completions): ```json theme={null} {"messages": [{"role": "user", "content": "Hello"}]} ``` **With feedback metrics** (for training on ratings): ```json theme={null} {"messages": [...], "completion": "...", "feedbacks": {"quality": 0.8, "helpful": true}} ``` **With preferences** (for RLHF/DPO training): ```json theme={null} {"messages": [...], "preferred_completion": "Good answer", "other_completion": "Bad answer", "feedback_key": "quality"} ``` **With images (multimodal)**: ```json theme={null} {"messages": [{"role": "user", "content": [{"type": "text", "text": "Describe this image"}, {"type": "image", "url": "data:image/jpeg;base64,/9j/4AAQ..."}]}], "completion": "A photo of a cat sitting on a desk."} ``` Use `content` as a list of parts to interleave text and images. Each image must be a base64 data URI (JPEG, PNG, WebP, or GIF, up to 10 MB each). External URLs are not supported. Datasets and the chat API use different image schemas: ```python theme={null} # Dataset format {"type": "image", "url": "data:image/jpeg;base64,..."} # Chat API format (see Models page) {"type": "image_url", "image_url": "data:image/jpeg;base64,..."} ``` Add optional `labels` or `metadata` fields to any format for filtering or custom graders. See [SDK Reference](/v0.14/reference/sdk) for all dataset methods. ## Upload a dataset Navigate to your project and open the **Datasets** tab. Click **Upload Dataset** and select a JSONL file. The file must contain one JSON object per line. See the SDK tab for supported formats, including multimodal datasets with images. # Graders Source: https://docs.adaptive-ml.com/v0.14/core/graders Score completions for training and evaluation Graders evaluate LLM completions and produce metric scores. Use them as reward signals during training or as evaluation criteria for runs. Adaptive supports five grader types — pick one based on what your scoring rule looks like. | Type | When to use | Create with | | --------------------- | ------------------------------------------------------------------------- | -------------------------------------------------------- | | **Function** | Correctness is structural or rule-based — string match, regex, JSON shape | `create.function()` / **New Grader → Function** | | **AI judge** | Pass/fail criteria can be expressed in natural language | `create.binary_judge()` / **New Grader → AI Judge** | | **Pre-built** | RAG evaluation: faithfulness, context relevancy, answer relevancy | `create.prebuilt_judge()` / **New Grader → Pre-built** | | **External endpoint** | Scoring requires an external service or model you host | `create.external_endpoint()` / **New Grader → External** | | **Custom** | Python logic baked into a [custom recipe](/v0.14/harmony/overview) | `create.custom()` | **New in v0.14:** function graders are first-class objects. Define a Python `grade` function, validate it in a sandbox, and reuse it across recipes — no need to edit recipe code to swap scoring rules. ## Create a function grader Function graders run an `async def grade(thread: StringThread) -> float` in a sandboxed Python environment, called on every completion during training or evaluation. Use them when correctness is structural or rule-based — string matching, regex, JSON shape — and an LLM judge would add noise at every step. ```python theme={null} from adaptive_harmony import StringThread async def grade(thread: StringThread) -> float: completion = thread.completion() or "" expected = thread.metadata.get("expected", "") return 1.0 if completion.strip() == expected.strip() else 0.0 adaptive.graders.create.function( key="exact-match", fn=grade, description="Strict equality between completion and expected answer", ) ``` Pass the function itself, not its source as a string. The SDK reads `grade`'s source from the file or notebook cell where it's defined, validates it in the sandbox, and stores it under the project. The function gets renamed to `grade` server-side, so the name in your code doesn't matter. ### The grade function contract | Rule | Detail | | --------------------- | ------------------------------------------------------------------------------------- | | Must be `async` | Sync `def grade(...)` is rejected at validation. | | Exactly one parameter | A `StringThread`. The parameter name is convention only. | | Returns numeric | `int` or `float`. Other types fail validation. | | Source size | 64 KiB max. | | Top-level only | The function must live at module top level — not nested in another function or class. | The validator emits warnings (not errors) if the parameter isn't annotated `StringThread` or the return isn't annotated `float`. ### Reading the thread Inside `grade`, the `StringThread` exposes the conversation and any per-sample metadata you attached to your dataset: ```python theme={null} async def grade(thread: StringThread) -> float: completion = thread.completion() # last assistant turn, or None turns = thread.get_turns() # list of (role, content) tuples history = thread.messages() # all turns except final assistant metadata = thread.metadata # dataset row metadata dict ... ``` `thread.metadata` is the per-sample metadata you set when building the dataset — put rubric fields, expected outputs, or labels there. ### Validation `graders.create.function(...)` validates before it persists. If validation fails, the SDK raises `ValueError` with the failed check name and message — nothing is saved. The sandbox runs five checks in order and stops at the first failure: 1. **Syntax** — `compile()` succeeds. 2. **Structure** — a top-level `async def grade(...)` exists. 3. **Signature** — exactly one parameter; warnings emitted for missing or unexpected annotations. 4. **Execution** — the module body executes and `grade` is callable. 5. **Test run** — `grade` is invoked with a thread and must return a numeric value. The SDK supplies a hardcoded mock for the test run: a thread containing `("user", "What is 2+2?")` and `("assistant", "4")`, with `metadata = {}`. The check confirms your function runs and returns a number — not that it produces a sensible score on real data. To test against real samples, use the test payload panel in the UI. ### Examples ```python theme={null} # Regex match import re async def grade(thread: StringThread) -> float: completion = thread.completion() or "" pattern = thread.metadata.get("pattern", "") return 1.0 if re.search(pattern, completion) else 0.0 ``` ```python theme={null} # Strict JSON shape import json async def grade(thread: StringThread) -> float: completion = thread.completion() or "" try: parsed = json.loads(completion) return 1.0 if {"name", "email", "age"}.issubset(parsed.keys()) else 0.0 except Exception: return 0.0 ``` ```python theme={null} # Multi-keyword scoring with partial credit async def grade(thread: StringThread) -> float: completion = (thread.completion() or "").lower() keywords = thread.metadata.get("keywords", []) if not keywords: return 0.0 hits = sum(1 for k in keywords if k.lower() in completion) return hits / len(keywords) ``` ### Sandbox The grade function runs in an isolated Python sandbox (nsjail). What's available: * The Python standard library (`re`, `json`, `math`, `string`, etc.) * `StringThread`, importable from `adaptive_harmony` What's not: * Network calls (`requests`, `httpx`, `urllib.request`) * Filesystem writes outside the sandbox * Third-party packages (`numpy`, `pydantic`, etc.) If your scoring depends on a network call or external library, use an [external endpoint grader](/v0.14/advanced/reward-servers) instead. ### Manage existing graders ```python theme={null} adaptive.graders.list() # all graders in project adaptive.graders.get(grader_key="exact-match") # one grader adaptive.graders.delete(grader_key="exact-match") # remove adaptive.graders.lock(grader_key="exact-match", locked=True) # prevent edits ``` The SDK has no in-place update for function graders. To change the implementation, delete and recreate the grader, or edit it through the UI. ## Create an AI judge AI judges use an LLM to grade completions based on a criterion you define: ```python theme={null} adaptive.graders.create.binary_judge( key="helpful-judge", criteria="The response directly answers the user's question without going off-topic", judge_model="llama-3.1-8b-instruct", feedback_key="helpfulness", ) ``` | Parameter | Type | Required | Description | | -------------- | ---- | -------- | ------------------------------------------ | | `key` | str | Yes | Unique identifier | | `criteria` | str | Yes | What constitutes a pass (natural language) | | `judge_model` | str | Yes | Model to use as judge | | `feedback_key` | str | Yes | Feedback key to write scores to | The judge returns PASS/FAIL for each completion along with reasoning. ## Prompt templates for AI judges AI judges use [Handlebars](https://handlebarsjs.com/) templates for their prompts. Template variables give you access to the conversation context, completion, and metadata. **Basic syntax:** ```handlebars theme={null} {{{completion}}} — insert without HTML escaping (always use for text) {{#if metadata.domain}} — conditional block ... {{else}} ... {{/if}} {{#each turns}} — loop over a list (list of turn objects below) Turn {{@index}}: {{role}}: {{content}} {{/each}} ``` **Example template:** ``` System: "You are a judge. Evaluate based on: {{criteria}}. Output this JSON schema: {{output_schema}}" User: "Context:\n{{context_str_without_last_user}} Question:\n{{last_user_turn_content}} Response:\n{{completion}}" ``` | Variable | Description | | --------------------------------- | ----------------------------------------------- | | `completion` | The assistant's completion being evaluated | | `last_user_turn_content` | Content of the final user turn | | `context_str` | Full conversation context as a formatted string | | `context_str_without_last_user` | Context excluding the final user turn | | `turns` | All turns as a list of `{role, content}` dicts | | `context_turns` | All turns except the completion | | `context_turns_without_last_user` | Context turns without the last user turn | | `metadata` | Thread metadata dict | | `output_schema` | Expected output JSON schema | | `template_variables` | Custom variables passed at judge initialization | Use triple braces (`{{{var}}}`) for variables that may contain HTML entities or special characters. ## Pre-built graders For RAG applications, use pre-built graders optimized by Adaptive: ```python theme={null} adaptive.graders.create.prebuilt_judge( key="rag-faithfulness", type="FAITHFULNESS", judge_model="llama-3.1-8b-instruct", ) ``` * **Faithfulness**: Does the completion adhere to provided context? * **Context Relevancy**: Is the retrieved context relevant to the query? * **Answer Relevancy**: Does the completion answer the question? **Faithfulness** breaks the completion into atomic claims and checks each against the context: ``` score = claims supported by context / total claims ``` Pass context as `document` turns in the input messages. Each retrieved chunk should be a separate turn. **Sample:** ```json theme={null} {"role": "document", "content": "Tim Berners-Lee created the first website."} {"role": "document", "content": "Tim Berners-Lee invented the world wide web."} {"role": "user", "content": "Who published the first website?"} ``` Completion: "Tim Berners-Lee published the first website in August 1990." Score: 0.5 (first claim supported, date claim unsupported) *** **Context Relevancy** checks if retrieved chunks are relevant to the query: ``` score = relevant chunks / total chunks ``` *** **Answer Relevancy** checks if the completion addresses the question: ``` score = relevant claims / total claims ``` Extra information not requested by the user lowers the score. For reward servers and custom graders, see [Reward Servers](/v0.14/advanced/reward-servers) and [Custom Recipes](/v0.14/harmony/overview). See [SDK Reference](/v0.14/reference/sdk) for all grader methods. ## Create a grader Navigate to your project and open the **Graders** tab. Click **New Grader** and pick a type. ### Function grader Pick **Function** to write a Python `grade` function directly in the editor. Use the **Test Payload** panel to run validation against a sample from your dataset before saving — the panel surfaces errors and per-sample scores so you can iterate without leaving the page. ### AI judge For AI judges, provide a natural language criterion that defines what a passing completion looks like. Add examples to improve judge accuracy. Include both passing and failing examples with justifications. ## Use in recipes After creating a grader, select it when configuring training or evaluation recipes. The grader scores each completion and provides feedback for the run. # Inference Source: https://docs.adaptive-ml.com/v0.14/core/inference Make requests to models deployed on Adaptive Engine Send chat completions to deployed models using the Adaptive SDK, OpenAI Python library, or any HTTP client. If you omit `model`, requests route to the project's default model, or to a model in an active A/B test. Interactions (prompt + completion pairs) are logged automatically. See [Interactions](/v0.14/core/interactions) for details. ## Chat completions ```python theme={null} response = adaptive.chat.create( model="llama-3.1-8b-instruct", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Hello!"} ], labels={"project": "support-bot"}, ) print(response.choices[0].message.content) ``` | Parameter | Type | Description | | ------------- | ----------- | ---------------------------------------------- | | `model` | str | Model key. Omit to use the project default. | | `messages` | list | Chat messages with `role` and `content` | | `labels` | dict | Key-value pairs for filtering interactions | | `stream` | bool | Enable streaming (default: False) | | `temperature` | float | Sampling temperature | | `max_tokens` | int | Maximum tokens to generate | | `stop` | list | Stop sequences | | `top_p` | float | Top-p sampling threshold | | `session_id` | str or UUID | Session ID for KV-cache reuse across turns | | `store` | bool | Whether to log the interaction (default: True) | ## Streaming ```python theme={null} stream = adaptive.chat.create( messages=[{"role": "user", "content": "Hello!"}], stream=True, ) for chunk in stream: if chunk.choices: print(chunk.choices[0].delta.content, end="", flush=True) ``` ## Get the completion ID Use `completion_id` to log [Metrics](/v0.14/core/metrics) against the response: ```python theme={null} completion_id = response.choices[0].completion_id ``` ## Vision requests Models with the **Multimodal** tag accept images alongside text. Images must be base64-encoded data URIs (JPEG, PNG, WebP, or GIF, up to 10 MB each). ```python theme={null} import base64 with open("photo.png", "rb") as f: image_data = base64.b64encode(f.read()).decode() response = adaptive.chat.create( model="your-vlm-key", messages=[ { "role": "user", "content": [ {"type": "text", "text": "What's in this image?"}, {"type": "image_url", "image_url": f"data:image/png;base64,{image_data}"}, ], } ], ) ``` ## Structured output Pass `response_format` to constrain a completion to a JSON Schema or a Pydantic model. For internal models, invalid tokens are masked at each generation step — the response is structurally guaranteed to parse. For external providers, the schema is forwarded to the provider's native structured-output API. ```python theme={null} from pydantic import BaseModel class Classification(BaseModel): label: str confidence: float response = adaptive.chat.create( model="llama-3.1-8b-instruct", messages=[{"role": "user", "content": "Classify: 'great product, fast shipping'"}], response_format=Classification, ) parsed = Classification.model_validate_json(response.choices[0].message.content) ``` `response_format` accepts a Pydantic `BaseModel` class, a raw JSON Schema envelope (`{"type": "json_schema", "json_schema": {"name": ..., "schema": ...}}`), or `None` (default). Pydantic models are auto-converted via `model_json_schema()` and patched for strict-mode compatibility (refs inlined, `additionalProperties: false` added). The response is a JSON string in `response.choices[0].message.content` — the SDK does not auto-deserialize. Call `Model.model_validate_json(...)` to get a typed instance. Constrained decoding compiles the schema to a token-mask grammar. The compiler supports: * **Types:** `string`, `integer`, `number`, `boolean`, `null`, `object`, `array`, and union types via `["string", "null"]` syntax * **Composition:** `oneOf`, `anyOf`, `allOf` (object merge only), `$ref` and `$defs` (inlined during SDK prep) * **Strings:** `minLength`, `maxLength`, `pattern` (regex), `format`, `enum`, `const` * **Numbers:** `minimum`, `maximum`, `exclusiveMinimum`, `exclusiveMaximum`. `multipleOf` requires explicit bounds — without bounds, it's silently ignored. * **Arrays:** `items`, `minItems`, `maxItems`. Arrays must declare `items`. * **Objects:** `properties`, `required`, `additionalProperties` (false / true / schema) Recursive schemas with cyclic `$ref` are unrolled to depth 4; deeper nesting is truncated. Format keywords without a regex equivalent are dropped. ### Streaming with structured output `stream=True` and `response_format` work together. Each chunk delivers partial JSON; buffer until the stream closes, then parse: ```python theme={null} buffer = "" stream = adaptive.chat.create( messages=[{"role": "user", "content": "Classify: 'late and damaged'"}], response_format=Classification, stream=True, ) for chunk in stream: if chunk.choices and chunk.choices[0].delta.content: buffer += chunk.choices[0].delta.content parsed = Classification.model_validate_json(buffer) ``` ### Failure modes | Situation | Behavior | | ----------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------- | | Schema fails to compile | The constraint is dropped with a warning in server logs; the model generates unconstrained. Validate your schema during development. | | `max_tokens` exhausted before completion | `finish_reason` is `"length"`. The response is truncated JSON — `model_validate_json` will raise. Check `finish_reason` before parsing. | | External provider doesn't support structured output for the model | The constraint is dropped silently. Stick to providers and models that support structured output natively. | | Recursive schema beyond depth 4 | Deeper levels are truncated at compile time. | See [SDK Reference](/v0.14/reference/sdk) for all chat methods. ## Chat playground Open your project and click **Chat** to interact with deployed models in the browser. When a vision model is deployed, the Chat view supports image uploads. Attach images directly in the conversation to send multimodal requests. ## Structured output Open **Model Settings** and set a **Response Format** to constrain a model to a JSON Schema. The workbench includes presets for classification, entity extraction, and simple object output. Judge graders use the same field to guarantee parseable judgement output. ## OpenAI compatibility Use the OpenAI Python library with your Adaptive deployment: ```python theme={null} from openai import OpenAI client = OpenAI( base_url=f"{ADAPTIVE_URL}/api/v1", api_key=ADAPTIVE_API_KEY, ) response = client.chat.completions.create( model="project_key/model_key", messages=[{"role": "user", "content": "Hello!"}], ) ``` Set `model` to `project_key/model_key`. Use `metadata` instead of `labels`. Multimodal image format differs between Adaptive and OpenAI: ```python theme={null} # Adaptive format (flat string) {"type": "image_url", "image_url": "data:image/png;base64,..."} # OpenAI format (nested object) {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}} ``` ## HTTP requests Use any HTTP client to call the chat completions endpoint directly. ```python theme={null} import requests headers = {"Authorization": "Bearer ADAPTIVE_API_KEY"} payload = { "model": "project_key/model_key", "messages": [ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Hello!"} ], "labels": {"project": "support-bot"}, } response = requests.post( url="ADAPTIVE_URL/api/v1/chat/completions", json=payload, headers=headers, ) completion_text = response.json()["choices"][0]["message"]["content"] ``` ```bash theme={null} curl "$ADAPTIVE_URL/api/v1/chat/completions" \ -X POST \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $ADAPTIVE_API_KEY" \ -d '{ "model": "project_key/model_key", "messages": [{"role": "user", "content": "Hello!"}], "labels": {"project": "support-bot"} }' ``` See [API Reference](/v0.14/reference/api) for the full endpoint specification. # Interactions Source: https://docs.adaptive-ml.com/v0.14/core/interactions Track prompts, completions, and metrics An interaction is a prompt, completion, and any associated feedback or labels. Adaptive logs interactions automatically when you run inference, creating a record you can browse, filter, and use for training. ## Automatic logging Inference requests are logged automatically: ```python theme={null} response = adaptive.chat.create( messages=[ {"role": "user", "content": "Hello!"} ], labels={"project": "support-bot", "version": "v2"}, ) ``` Use `labels` to organize interactions. Labels are key-value pairs for filtering and grouping. ## Get the completion ID Extract `completion_id` from the response to log feedback: ```python theme={null} completion_id = response.choices[0].completion_id ``` Pass this ID to [Metrics](/v0.14/core/metrics) methods to associate ratings with the interaction. ## Manual logging Log interactions for completions generated outside Adaptive: ```python theme={null} adaptive.interactions.create( model="model_key", messages=[ {"role": "system", "content": "You are a helpful assistant."}, {"role": "user", "content": "Hello!"} ], completion="Hi there! How can I help you today?", labels={"source": "external"}, ) ``` Use manual logging to import historical data or log completions from external models. For multimodal interactions, pass image content parts in `messages` (see [Vision models](/v0.14/core/models#vision-models)): ```python theme={null} adaptive.interactions.create( model="model_key", messages=[ { "role": "user", "content": [ {"type": "text", "text": "What's in this image?"}, {"type": "image_url", "image_url": "data:image/png;base64,..."}, ], } ], completion="The image shows a bar chart comparing model accuracy across three benchmarks.", labels={"source": "external"}, ) ``` Upload large datasets efficiently with async concurrency: ```python theme={null} import asyncio from adaptive_sdk import AsyncAdaptive async def upload_data(data: list[dict], labels: dict, max_concurrency: int = 30): adaptive = AsyncAdaptive( base_url=os.environ["ADAPTIVE_URL"], api_key=os.environ["ADAPTIVE_API_KEY"], ) adaptive.set_default_project("my-project") semaphore = asyncio.Semaphore(max_concurrency) async def upload_one(item): async with semaphore: await adaptive.interactions.create( messages=item["messages"], completion=item["completion"], labels=labels, ) await asyncio.gather(*[upload_one(item) for item in data]) # Example data format data = [ { "messages": [{"role": "user", "content": "Hello!"}], "completion": "Hi there!" } ] asyncio.run(upload_data(data, labels={"dataset": "training-v1"})) ``` See [SDK Reference](/v0.14/reference/sdk) for all interaction methods. ## Browse interactions Navigate to your project and open the **Interactions** tab to view logged data. Filter by label, time range, or feedback values to find specific interactions. Click any row to see the full prompt and completion. Use interactions to: * Audit model responses * Find examples for training datasets * Debug unexpected behavior Multimodal interactions display images inline, so you can review exactly what the model saw. # Metrics Source: https://docs.adaptive-ml.com/v0.14/core/metrics Track system, grader, and user metrics across completions Metrics unify three categories of measurement into a single system: * **System metrics** — auto-computed per completion (TTFT, latency, token counts). No setup required. * **Grader metrics** — produced by [Graders](/v0.14/core/graders) (AI judges, pre-built, custom, external). Scores are written automatically when graders run. * **User metrics** — custom ratings you define and log via SDK. Use these for human evaluation, application-specific scores, or any signal not covered by system or grader metrics. This page covers **inference-time** metrics — measurements collected on completions returned by deployed models. For **training-time** signals (loss curves, reward, gradient norms, validation metrics emitted during a run), see [Monitoring](/v0.14/core/monitoring). The two systems are separate: training metrics live with the run record, inference metrics live with the completion record. ## Register a user metric Before logging values, register a metric key: ```python theme={null} adaptive.feedback.register_key( project="my-project", key="quality", kind="scalar", # or "bool" scoring_type="higher_is_better", ) ``` | Parameter | Type | Required | Description | | -------------- | ---- | -------- | ----------------------------------------------------- | | `project` | str | Yes | Project to register the metric in | | `key` | str | Yes | Unique identifier | | `kind` | str | No | `"scalar"` (default) or `"bool"` | | `scoring_type` | str | No | `"higher_is_better"` (default) or `"lower_is_better"` | The SDK resource is `adaptive.feedback` — the entity is called "metrics" in the UI but the SDK interface retains the `feedback` name. ## Log metric feedback Log a rating for a completion using its `completion_id` from the inference response: ```python theme={null} response = adaptive.chat.create( messages=[{"role": "user", "content": "Hello"}] ) completion_id = response.choices[0].completion_id adaptive.feedback.log_metric( value=5, completion_id=completion_id, feedback_key="quality", ) ``` Feedback is associated with the completion's [Interaction](/v0.14/core/interactions) record. ## Log preference feedback Log a pairwise comparison between two completions: ```python theme={null} adaptive.feedback.log_preference( feedback_key="quality", preferred_completion=completion_id_a, other_completion=completion_id_b, ) ``` Use preferences for RLHF/DPO training when you can judge which completion is better but can't assign absolute scores. ## List and get metrics ```python theme={null} keys = adaptive.feedback.list_keys(project="my-project") key = adaptive.feedback.get_key( project="my-project", key="quality", ) ``` See [SDK Reference](/v0.14/reference/sdk) for all feedback methods. ## View metrics Navigate to your project and open the **Metrics** page to see all metrics and their values over time. Filter by category — **System**, **Grader**, or **User** — to narrow the view. ## Manage user metrics Create, edit, and delete user metrics from the Metrics page. System and grader metrics are protected and cannot be modified. # Models Source: https://docs.adaptive-ml.com/v0.14/core/models Deploy models on Adaptive Engine Models power your AI applications. Add a model to a project to deploy it, then [make inference requests](/v0.14/core/inference). A model in your project comes from one of four sources: | Source | When to use | | ------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Adaptive catalog** *(new in v0.14)* | The fastest path. A curated, server-managed list of production-validated open-source models (Mistral, Qwen, Gemma families across sizes). No HF token, no manual download. | | **Hugging Face import** | A specific HF model that isn't in the catalog. Requires an HF token. | | **External provider** | OpenAI, Anthropic, Gemini, Azure, or your own API endpoint. See [Integrations](/v0.14/advanced/integrations). | | **Promoted checkpoint** | A snapshot from a training run, lifted into the registry as a standalone model. See [Promoted checkpoints](#promoted-checkpoints) below. | The catalog is the recommended default. New models appear there server-side without requiring a platform update. ## Add and deploy a model Once a model is in your project's registry, deploy it for inference: ```python theme={null} adaptive.models.add_to_project(model="llama-3.1-8b-instruct") adaptive.models.deploy(model="llama-3.1-8b-instruct", wait=True) # Or use attach() to do both in one call adaptive.models.attach(model="llama-3.1-8b-instruct", wait=True) ``` | Parameter | Type | Required | Description | | -------------- | ---- | -------- | -------------------------------------------- | | `model` | str | Yes | Model key from the registry | | `wait` | bool | No | Block until model is online (default: False) | | `make_default` | bool | No | Set as default model for the project | The model becomes available within a few minutes. ### Import from Hugging Face For models not in the Adaptive catalog, import from Hugging Face directly: ```python theme={null} adaptive.models.add_hf_model( hf_model_id="meta-llama/Llama-3.1-8B-Instruct", output_model_name="My Llama 3.1 8B", output_model_key="my-llama-3.1-8b", hf_token="hf_...", ) ``` | Parameter | Type | Required | Description | | ------------------- | ---- | -------- | --------------------------------------------------- | | `hf_model_id` | str | Yes | Full HF model ID (must be in the supported list) | | `output_model_name` | str | Yes | Display name for the imported model | | `output_model_key` | str | Yes | Key for the imported model in the Adaptive registry | | `hf_token` | str | Yes | Hugging Face access token with read permission | | `compute_pool` | str | No | Compute pool to run the import job | Import runs as an asynchronous job — the model appears in the registry once conversion finishes. Catalog import (the recommended path) is currently UI-only. Open the **Add Model** dialog in your project and pick **Import from Adaptive ML's catalog**. ## Promoted checkpoints A training run saves snapshots at configurable intervals (see [`checkpoint_frequency`](/v0.14/core/recipes#checkpoint-frequency)). Any saved checkpoint can be promoted to a standalone model in the registry, then evaluated or deployed like any imported model. Promotion is currently UI-only — open a run and pick **Promote checkpoint** on a saved checkpoint. SDK support uses the underlying GraphQL mutation directly: ```python theme={null} from adaptive_sdk.graphql_client import PromoteCheckpointInput, CheckpointModelPromotionInput job = adaptive._gql_client.promote_checkpoint( input=PromoteCheckpointInput( artifact_id="", models=[CheckpointModelPromotionInput( model_key="", name="my-grpo-step-800", )], ) ) ``` Promotion runs as an asynchronous copy job. Promoted models retain full provenance back to the source run. **LoRA backbone footgun:** when you promote a LoRA checkpoint, the platform records the backbone reference in the model's metadata but does **not** automatically attach the backbone to your project. If the backbone isn't already in the project, inference against the promoted LoRA fails with a "model not found" error. Add the backbone with `add_to_project` before deploying. See [SDK Reference](/v0.14/reference/sdk) for all model methods. ## Add a model Open your project's **Models** tab and click **Add Model**. The dialog offers four import paths: 1. **Import from Adaptive ML's catalog** — pick a model from the curated list. The dropdown groups models by family (Mistral, Qwen, Gemma) with size badges. Gated models show a license-acceptance checkbox; tick it to acknowledge the license terms before importing. License acceptance is recorded per user. 2. **Import from Hugging Face** — paste an HF model ID and provide your access token. 3. **Connect external provider** — register an OpenAI / Anthropic / Gemini / custom endpoint. 4. **Custom endpoint** — point at a model you host yourself. The import runs as an asynchronous job. The new model appears in the registry once conversion finishes. Fine-tuned adapter models appear indented under their base model. ## Promote a checkpoint Open a training run and pick **Promote checkpoint** on any saved checkpoint. The checkpoint is copied to a standalone model in the project registry, with full provenance back to the source run. Use this when an intermediate checkpoint outperforms the final one — common in RL runs where later steps overfit to the reward signal. Promoting a LoRA checkpoint records its backbone reference. Make sure the backbone model is also in your project before deploying — otherwise inference fails. # Monitoring Source: https://docs.adaptive-ml.com/v0.14/core/monitoring Watch training telemetry across runs in real time The Monitoring tab surfaces training and RL telemetry — loss curves, reward signals, gradient norms, validation metrics — streamed live from a running job and persisted for the run's lifetime. Pick multiple runs to compare them side-by-side on the same axes. Monitoring covers training-time signals only. For inference-time signals (TTFT, latency, token counts) and post-completion feedback, see [Metrics](/v0.14/core/metrics). ## Default metrics Pre-built training recipes auto-emit the metrics below — no extra code in your run config: | Recipe | Auto-emitted metrics | | -------------------------------------- | ---------------------------------------------------------------------------------------------------------------------- | | `sft` | `train/loss`, `train/gradient_norm`, `val/loss` (when validation enabled), `val/` (when grader configured) | | `preference_rlhf`, `metric_rlhf`, `rl` | `train/loss`, `train/reward`, `train/kl`, `train/gradient_norm`, plus stage-specific metrics for multi-stage runs | | `eval` | Per-grader scalar scores aggregated across the dataset | Metric streams flush to the platform every 0.5 seconds and appear in the UI within seconds of being emitted. ## Log custom metrics from a Harmony recipe For [custom recipes](/v0.14/harmony/overview), get a logger from the recipe context and call it like a function: ```python theme={null} from adaptive_harmony.metric_logger import get_prod_logger @recipe_main async def my_recipe(config: MyConfig, ctx: RecipeContext): logger = get_prod_logger(ctx) # Wire the dashboard URL into the run record so users can click through if logger.training_monitoring_link: ctx.job.set_monitoring_link(logger.training_monitoring_link) for step in range(num_steps): loss = await train_one_step(...) logger({"train/loss": loss, "train/lr": current_lr}) ``` The logger accepts any `Mapping[str, int | float | Table]`. Each call advances the internal step counter once. ### Logging tables For structured per-step data — sample completions, gradient breakdowns, layer statistics — log a `Table`: ```python theme={null} from adaptive_harmony.logging_table import Table samples = Table() samples.add_row(["prompt-1", completion_text, score]) samples.add_row(["prompt-2", completion_text_2, score_2]) logger({"eval/samples": samples}) ``` Tables appear in the run detail view, paginated and searchable. `get_prod_logger` auto-selects a logging backend based on environment variables present in the recipe sandbox: WandB → MLflow → TensorBoard → stdout. The Adaptive monitoring backend is added in addition (not as a replacement) when `ADAPTIVE_BASE_URL` and `ADAPTIVE_API_KEY` are set, so metrics always reach the platform UI even if you also log to a third-party tracker. Set `ADAPTIVE_MONITORING_DISABLED=1` to opt out of the Adaptive backend (e.g., for local-only development). See [SDK Reference](/v0.14/reference/sdk) for monitoring methods. ## Watch a run Open the **Monitoring** tab in your project. Each running or completed run appears in the list with status, recipe, and headline metric. Click a run to open its detail view — charts update in real time as the job emits metrics (no refresh required). Use the smoothing controls (EMA / SMA / Gaussian) to denoise noisy curves like RL reward without losing the underlying signal. ## Compare runs side-by-side Select multiple runs from the list and open **Compare**. The compare view overlays each run's curves on the same axes and shows a config diff — every recipe argument that differs between the runs is highlighted. Use this for hyperparameter sweeps, A/B comparisons of dataset versions, or picking the best checkpoint to promote. Run selection is unrestricted — you can compare any two (or more) runs in the project, even across different recipe types. ## Pick a checkpoint Charts are most useful right before promotion: scrub the run timeline, identify the step where validation loss bottoms out (or reward stabilizes), then jump to the run's **Checkpoints** tab to promote the matching saved checkpoint. See [Promoted checkpoints](/v0.14/core/models#promoted-checkpoints). The platform does not auto-rank checkpoints by metric — promotion is a manual decision based on what you see in the charts. ## Retention Monitoring metrics are tied to the run's lifetime — they live as long as the run record does. Deleting a run deletes its metrics. There is no separate TTL for metrics. # Projects Source: https://docs.adaptive-ml.com/v0.14/core/projects Organize your models, metrics, and datasets around specific tasks Projects are workspaces that group related resources for a specific task. Models, metrics, and datasets are all linked to a project. ## Create a project ```python theme={null} adaptive.projects.create( key="sales_copilot", name="Sales Copilot", description="Outbound sales pipeline assistant" ) adaptive.set_default_project("sales_copilot") ``` | Parameter | Type | Required | Description | | ------------- | ---- | -------- | -------------------------------- | | `key` | str | Yes | Unique identifier | | `name` | str | No | Display name (defaults to `key`) | | `description` | str | No | Description shown in UI | Most SDK methods operate on a single project. Setting a default avoids passing `project` to every call. Override when needed: ```python theme={null} # Uses default adaptive.models.list() # Override for this call adaptive.models.list(project="support_bot") ``` See [SDK Reference](/v0.14/reference/sdk) for all project methods. Navigate to **Projects** and click **Create Project**. Enter a key, name, and optional description. # Recipes Source: https://docs.adaptive-ml.com/v0.14/core/recipes Run training and evaluation workflows Recipes are pre-built workflows for training and evaluating models. Run them on Adaptive's compute infrastructure. ## Run a recipe ```python theme={null} job = adaptive.jobs.run( recipe_key="sft", num_gpus=1, args={ "model_to_train": "llama-3.1-8b-instruct", "output_model_key": "my-model-v1", "dataset": "my-training-data", "epochs": 3, "data_seed": 42, }, ) ``` | Parameter | Type | Required | Description | | ------------ | ---- | -------- | --------------------------------------------- | | `recipe_key` | str | Yes | Recipe identifier (see below) | | `num_gpus` | int | Yes | Number of GPUs to use | | `args` | dict | No | Recipe-specific arguments (defaults to empty) | ### Common training arguments | Argument | Type | Default | Description | | ---------------------- | ----- | ------- | ----------------------------------------------------------------------------------------------------------- | | `data_seed` | int | `42` | Seed for dataset shuffling. Set for reproducible training runs. | | `checkpoint_frequency` | float | `0.2` | Fraction of total training steps between saves. `0.2` = save every 20% of training (5 checkpoints per run). | All training recipes (`sft`, `preference_rlhf`, `metric_rlhf`, `rl`) support both arguments. Evaluation recipes use neither. `checkpoint_frequency` is a **fraction of total steps**, not an absolute step count. `checkpoint_frequency=0.1` saves every 10% of training; it is not "every 10 steps." Disk usage scales linearly with frequency × model size × run length — a 70B model checkpointed every 10% of a long run uses meaningful storage. ## Built-in recipes **Training:** | Recipe | Key | Use when | | ---------------------- | ----------------- | --------------------------------------------- | | Supervised fine-tuning | `sft` | You have high-quality completions | | RL on preferences | `preference_rlhf` | You have preferred/rejected pairs | | RL on metrics | `metric_rlhf` | You have completion-level scores | | RL with grader | `rl` | Criteria can be expressed in natural language | **Evaluation:** | Recipe | Key | Use when | | -------------------- | ------ | --------------------------- | | Evaluate with grader | `eval` | Comparing model performance | **Optimization:** | Recipe | Key | Use when | | ------------------------------ | ------------------- | --------------------------------------------------- | | Speculative decoding alignment | `specdec_alignment` | You want to accelerate inference with a draft model | ## Run an evaluation ```python theme={null} adaptive.jobs.run( recipe_key="eval", num_gpus=2, args={ "dataset": "eval-prompts", "models_to_evaluate": ["model-a", "model-b"], "graders": ["quality-judge"], }, ) ``` Results appear as evaluation artifacts with score tables and per-sample interactions. Evaluations use [Graders](/v0.14/core/graders) to score completions. ## Resume an interrupted run Resume picks up from the last saved checkpoint and keeps the same job ID — it does not fork a new run. Resume from the **Runs** tab in the UI, or re-launch the job with resume enabled. Resume is step-level, not epoch-level. Multi-stage runs (currently `preference_rlhf`, which orchestrates DPO / PPO / GRPO under one recipe) track each stage's progress independently — resume picks up at the active stage's last checkpoint, not the first stage. ## Promote a checkpoint Any saved checkpoint can be promoted to a standalone model in the project registry. See [Promoted checkpoints](/v0.14/core/models#promoted-checkpoints) on the Models page for the full workflow, including the LoRA-backbone footgun. For custom recipes, see [Custom Recipes](/v0.14/harmony/overview). See [SDK Reference](/v0.14/reference/sdk) for all job methods. ## Run a recipe Navigate to your project and open the **Recipes** tab. Select a recipe and configure its parameters. Click **Run** to launch. Track progress in the **Runs** tab. ## Monitor runs Open the **Monitoring** tab to watch training telemetry in real time. See [Monitoring](/v0.14/core/monitoring) for the full guide — loss curves, reward signals, side-by-side run comparison, and how to log custom metrics from a Harmony recipe. ## Promote a checkpoint Open a run and pick **Promote checkpoint** on any saved checkpoint. See [Promoted checkpoints](/v0.14/core/models#promoted-checkpoints) for the full workflow. If a run is interrupted, resume it from the **Runs** tab. Resume picks up at the last saved checkpoint and keeps the same run record; multi-stage runs resume at the active stage. ## View evaluation results After an evaluation completes, click the run to see the score table. Click any row to drill down into individual interactions and compare model outputs side-by-side. # Architecture Source: https://docs.adaptive-ml.com/v0.14/deploy/architecture Adaptive Engine system architecture An Adaptive Engine cluster consists of 5 components: 1. **Control plane** - UI, permissions system, user-facing APIs 2. **Compute plane** - GPU servers for inference and training 3. **Shared storage** - Model files and training artifacts 4. **Postgres database** - Interactions, metadata, application state 5. **Redis** - Job events and usage statistics ## Components ### Control plane Low-latency containerized service serving the UI and APIs. Runs on CPU (minimum 4 vCPU, 16GB memory). ### Compute plane Multi-GPU AI computing framework for training and inference. **Compatible GPUs:** L4, L40, L40S, H100, H200, B200, GB200 **Requirements:** * Multi-GPU or single-GPU servers supported * NVLink/NVSwitch optional but recommended for training and large models * PCIe-connected multi-GPU servers supported (e.g., EC2 g6.12xlarge) * 300GB+ local storage recommended for Docker images and model cache Adaptive Engine 0.11 and above do not support NVIDIA A100. ### Shared storage Model files and datasets. Any POSIX-compatible storage (local disk, NFS) or S3-compatible object storage. ### Database PostgreSQL 16+. Stores interactions, permissions, and settings. Must be located near the control plane since blocking permission checks run on every inference call. ### Redis Transfers events between compute and control planes, stores usage statistics and real-time metadata. Can be local or managed (Elasticache, Memorystore, etc.). ### Cloud service examples | Component | AWS | GCP | Azure | | ----------------- | ---------------------- | ------------------ | ----------------------------- | | Control & compute | EC2, EKS | GCE, GKE | Azure VM, AKS | | Storage | S3, FSx for Lustre | GCS, Filestore | Azure Files NFS | | Database | RDS PostgreSQL, Aurora | Cloud SQL, AlloyDB | Azure Database for PostgreSQL | | Redis | Elasticache | Memorystore | Azure Managed Redis | ## Compute plane ### Compute pools A compute pool is a set of homogeneous GPU servers that receives workloads. A cluster can have multiple compute pools representing different GPU types or capacity reservations. ### GPU workloads 1. **Inference endpoints** - Low-latency, high-throughput model serving 2. **Recipe runs** - Scripted batch tasks (training, fine-tuning, evaluation) 3. **Interactive sessions** - Real-time remote execution via Secure WebSockets **Pool targeting:** * Inference endpoints can span multiple inference or static pools * Recipe runs and sessions can only be on one compute pool ## Deployment notes * Built on open standards (Linux, Postgres, Rust, Docker) for portability * All 5 components can be collocated on a single server * For production, we recommend to use an external database and storage for independent scaling and availability # Self-hosting Source: https://docs.adaptive-ml.com/v0.14/deploy/self-hosting Deploy Adaptive Engine with Kubernetes and Helm Deploy Adaptive Engine on your own infrastructure using the [Helm chart](https://github.com/adaptive-ml/adaptive-helm-chart). Self-hosted deployments have no external connectivity or telemetry requirements. ## Prerequisites ### Infrastructure 1. **Kubernetes cluster** with: * GPU VM(s) for `harmony` compute plane * CPU VM(s) for `control-plane` and `recipe-runner` 2. **PostgreSQL 16+** (local or remote) 3. **Redis** (local or remote) 4. **Shared storage** (POSIX or S3-compatible) 5. **OIDC provider** (Google Workspace, Azure Entra ID, Cognito, Keycloak, etc.) 6. **Domain name** routing to the deployment ### Kubernetes requirements * Kubernetes 1.28+ * Helm 3.8.0+ * NVIDIA GPU operator * CUDA 12.8+ with driver 570.172.08+ ## Deployment checklist Ensure cloud GPU quotas accommodate your instance types and counts. Deploy Kubernetes cluster, database, Redis, storage, and configure OIDC. Obtain access to Adaptive Engine private registry (requires commercial contract). Pull the chart and customize `values.yaml`. Run `helm install`. Set administrator emails in `values.yaml` before those users first log in. ## Helm configuration ### Container registry ```yaml theme={null} containerRegistry: .dkr.ecr..amazonaws.com harmony: image: repository: adaptive-repository tag: harmony:latest controlPlane: image: repository: adaptive-repository tag: control-plane:latest ``` ### Resource limits ```yaml theme={null} harmony: replicaCount: 1 gpusPerReplica: 8 # Match available GPUs per node resources: limits: cpu: 8 memory: 64Gi requests: cpu: 8 memory: 60Gi ``` ### Secrets ```yaml theme={null} secrets: modelRegistryUrl: "s3://bucket-name/model_registry" sharedDirectoryUrl: "s3://bucket-name/shared" dbUrl: "postgres://user:password@host:5432/db_name" cookiesSecret: "must-be-64-chars-or-more-change-me-secret-abc123..." auth: oidc: providers: - name: "Google" key: "google" issuer_url: "https://accounts.google.com" client_id: "your-client-id" client_secret: "your-client-secret" scopes: ["email", "profile"] pkce: true allow_sign_up: true ``` If `allow_sign_up: true`, any OIDC member can access Adaptive Engine. Set to `false` and create users via SDK to restrict access. ## Shared cluster configuration ### Separate namespace ```bash theme={null} helm install adaptive adaptive/adaptive \ --values ./values.yaml \ --namespace adaptive-engine \ --create-namespace ``` ### Node selectors Schedule Harmony on specific GPU nodes: ```yaml theme={null} harmony: nodeSelector: eks.amazonaws.com/nodegroup: p5-h100 ``` ### Dedicated GPU nodes Taint GPU nodes to prevent other workloads: ```bash theme={null} kubectl taint nodes dedicated=adaptive-engine:NoSchedule ``` Add matching toleration in `values.yaml`: ```yaml theme={null} harmony: tolerations: - key: dedicated operator: Equal value: adaptive-engine effect: NoSchedule ``` ## Database TLS ### Basic encryption ```yaml theme={null} dbUrl: "postgres://user:password@host/db?sslmode=require" ``` ### Certificate verification For full verification (`sslmode=verify-full`): 1. Download the database server certificate 2. Create a ConfigMap: ```bash theme={null} kubectl create configmap -n db-ca --from-file=rds-ca-rsa2048-g1.pem ``` 3. Mount in `values.yaml`: ```yaml theme={null} volumes: - name: db-ca configMap: name: db-ca volumeMounts: - name: db-ca mountPath: /mnt/db-ca/ readOnly: true ``` 4. Reference in connection string: ```yaml theme={null} dbUrl: "postgres://user:password@host/db?sslmode=verify-full&sslrootcert=/mnt/db-ca/rds-ca-rsa2048-g1.pem" ``` # Create a training algorithm Source: https://docs.adaptive-ml.com/v0.14/harmony/algorithms How to write your own RL algorithm for custom training Writing a recipe typically requires you to use a training algorithm, as covered in the previous page. Harmony allows you to create your own algorithms through convenient methods that give you access to models for operations such as backward or optim step. You can mix and match these methods to create your own algorithms, or copy and modify our supported algorithms in `adaptive_harmony.common`. In this page, we assume reasonable familiarity with reinforcement learning with LLMs. ## Training primitives ### `TrainingModel` This is the main class through which you interact with model weights. [Load models](models) explains in detail how to a model, but to recap: ```python theme={null} # given that you have your RecipeContext ctx model_builder = Model("gemma-3-12b").to_builder(ctx) model = await model_builder.tp(tp).spawn_train("policy", max_seq_len) ``` Choose the `tp` parameter (Tensor Parallelism) to be the smallest possible while still fitting in your GPUs. Choose the `max_seq_len` parameter to be larger than your expected rollout size in tokens, while still fitting in your GPUs. ### Generating text Once a `TrainingModel` has been loaded, you can use it straightforwardly to make async inference calls. The input and outputs are `StringThread` objects, corresponding to chats. Details about creation methods are given in [the thread page](string-thread). ```python theme={null} thread = StringThread(turns=[('user', "Hey, what's up?")]) response_thread = await model.generate(thread) # convenient method to access the completion text completion = response_thread.last_content() ``` ### Generating tokens When training with RL, you need the raw tokens to be able to compute log-probabilities and losses. Harmony provides tokenized versions of threads called `adaptive_harmony.TokenizedThread`, as well as utils for tokenization and detokenization: ```python theme={null} thread = StringThread(turns=[('user', "Hey, what's up?")]) tokenized_thread = await model.tokenize_thread(thread) tokenized_response = await model.generate_tokens(tokenized_thread) response = await model.detokenize_thread(tokenized_response) ``` ### Logprobs Get model logprobs like so: ```python theme={null} logprobs = await model.logprobs_per_token() ``` ### Training: backward For training we have implemented losses of the most common algorithms (cross-entropy for SFT, PPO, GRPO, DPO...) inside Harmony. You can do a backward step for each of these losses by applying the corresponding trainer function on the `TrainingModel`, for instance: ```python theme={null} model.train_grpo( sample, # TokenizedThread logprobs, ref_logprobs, [advantage] * len(sample.logprobs), clip_range, kl_beta, ) ``` The list of training methods for RL policies is: * `train_ppo` * `train_grpo` * `train_gspo` * `train_dpo` * `train_tangents` We refer the reader to `harmony_client.harmony_client` for definition of the parameters for these methods. Note that these methods only compute gradient contribution for the given samples, they do *not* change model parameters (this is performed by calling `optim_step`). The `train_tangents` method is special in that it allows you to pass arbitrary losses, allowing you to implement **any RL algorithm** not covered in our libraries. See the `Bring your own loss functions` section at the end of this file for instructions on how to use it. ### Training: optim step Once you have computed the backward pass for all samples in your batch, you need to perform a gradient descent step. We use the AdamW optimizer under the hood. ```python theme={null} await model.optim_step( learning_rate, # compute it with your lr scheduler wd, # weight decay max_grad_norm, ) ``` ## Putting it all together: simplified GRPO walkthrough We will see how all these building blocks interact by implementing GRPO, one of the most common RL methods for LLMs. Please refer to the [original paper](https://arxiv.org/abs/2402.03300) for an in-depth explanation of the algorithm. For simplicity, we only illustrate the single-turn variant of GRPO here. Please take a look at `adaptive_harmony.common.env_grpo` for our multi-turn variant. ### Init: defining useful variables We will need to use the following objects in the `GRPO` class: ```python theme={null} class GRPO: def __init__(self, ...): # DataSet is a helper iterator over StringThreads self.dataset = DataSet(dataset, allow_looping=True, seed=data_seed) self.model = model self.grader = grader self.scoring_fn = grader.score_float_value self.completions_per_sample = completions_per_sample # GRPO creates a few completions for each prompt self.lr_schedule = lr_scheduler or CosineScheduler(lr) # built-in scheduler self.max_grad_norm = max_grad_norm self.prompts_per_batch = samples_per_batch // completions_per_sample self.samples_per_mini_batch = samples_per_mini_batch # GRPO hyperparameters self.clip_range = clip_range self.kl_beta = kl_beta self.weight_decay = weight_decay self.mini_epochs_per_batch = mini_epochs_per_batch ``` And before running the data generation, we initialize the reference model to the untrained one: ```python theme={null} # this creates an InferenceModel with the weights of self.model self.model_ref = await self.model.clone_inf() ``` ### Generate data The first phase in the algorithm is to generate `self.completions_per_sample` completions from a single prompt. We use a helper dataclass for sample collection: ```python theme={null} @dataclass class Sample: sample: TokenizedThread logprobs: list[float] ref_logprobs: list[float] advantage: float score: float kl_div: list[float] ``` and we proceed like so, creating several completions per prompt and grading them: ```python theme={null} # generation method for one prompt async def gen_data(self, sample: StringThread) -> list[Sample]: # map the generate_tokens function on completion_per_sample copies of sample # async_map_batch samples items from the iterator until batch_size elements, with retries on errors all_samples = await async_map_batch( self.model.generate_tokens, repeat_thread(sample), batch_size=self.completions_per_sample ) # detokenize all for grading string_samples = await async_map(self.model.detokenize_thread, all_samples) # apply grader all_scores = np.array(await async_map(self.scoring_fn, string_samples), dtype=np.float32) # GRPO advantages advantages: FloatArray = all_scores - all_scores.mean() advantages /= advantages.std() + 1e-8 # get all logprobs for the current and reference model # async_map processes all samples concurrently given the provided coroutine logprobs = await async_map(self.model.logprobs_per_token, all_samples) ref_logprobs = await async_map(self.model_ref.logprobs_per_token, all_samples) # compute a KL-divergence estimate kl = [ (np.array(lp, dtype=np.float32) - np.array(ref_lp, dtype=np.float32)).tolist() for lp, ref_lp in zip(logprobs, ref_logprobs) ] # collect samples samples = [] for i in range(len(logprobs)): samples.append( Sample( sample=all_samples[i], logprobs=logprobs[i], ref_logprobs=ref_logprobs[i], advantage=advantages[i], score=all_scores[i], kl_div=kl[i], ) ) return samples ``` ### Run the algorithm Once the data generation part is written, the entire algorithm is quite straightforward. ```python theme={null} async def run(self): while self.training_completion_percentage < 1.0: # generate training samples data = await async_map_batch( self.gen_data, self.dataset, self.prompts_per_batch ) # set learning rate current_lr = self.lr_schedule(self.training_completion_percentage) # flatten samples to mix them flattened_data = sum([inner_list for inner_list in data], start=[]) # shuffles the dataset and gets minibatches of the specified size minibatches = get_minibatches( flattened_data, self.samples_per_mini_batch, self.mini_epochs_per_batch ) for idx, mini_batch in enumerate(minibatches): # train_sample is a wrapper around train_grpo await async_map(self.train_sample, mini_batch) await self.model.optim_step( current_lr, wd=self.weight_decay, max_grad_norm=self.max_grad_norm, ) ``` If you look at our actual code, you will see some added complexity from checkpointing, callbacks and logging. That's it! Pretty simple, isn't it? ## Bring your own loss functions We also offer a way to train with arbitrary loss functions with `train_tangents`, by specifying gradient values (tangents) at each token. There are two options: * If you have a closed-form solution of the gradient of your loss with respect to the model logprobs, you can simply compute this value with your numerical library of choice and pass it to `train_tangents`. * Otherwise, you can also use autodiff software to compute the gradient of your loss function with respect to the logprobs. For this second option, imagine you have access to your logprobs as well as your loss `loss_fn` written in `torch`. You would do the following, for a given `sample` (`TokenizedThread`): ```python theme={null} # get gradients with torch logprobs_th = torch.Tensor(logprobs, requires_grad=True) loss = loss_fn(logprobs_th, *other_args) loss.backward() tangents = logprobs_th.grad # get tokens, weights tokens, image_patches, weights = await model.serialize_tokenized_thread(sample) # each input token tries to predict the next target one inp_tokens = tokens[:-1] tgt_tokens = tokens[1:] weights = np.array(weights[1:]) # expand tangents to full length using weights mask full_tangents = torch.zeros_like(weights, dtype=np.float32) full_tangents[weights > 0] = tangents # backward through model await model.train_tangents(inp_tokens, image_patches, full_tangents.tolist(), tgt_tokens) ``` While slightly more technical, this option offers full customizability of your training algorithm. # Output artifacts Source: https://docs.adaptive-ml.com/v0.14/harmony/artifacts Save new datasets and evaluation results from runs Artifacts are outputs that recipes create and register with the Adaptive Engine. All you have to do is create and add content to one in your recipe code, and it will appear in the UI after the run completes. Harmony supports two types of artifacts: evaluation results and new datasets. Artifacts are currently only registered in the platform if the recipe that creates them is [executed in the platform](/v0.14/harmony/recipe-syntax). Artifacts created in [interactive sessions](/v0.14/harmony/harmony-client) do not get logged to the platform. ## Evaluation artifact `EvaluationArtifact` stores evaluation results from model assessments. Each artifact contains multiple `EvalSample` objects (model completions with attached grades). When viewed in the Adaptive UI, the artifact displays aggregate evaluation scores per model across all graders that were used in the evaluation, in an interactive table format. Import from `adaptive_harmony`: `EvaluationArtifact`, `EvalSample`, `EvalSampleInteraction`, `Grade`. Create an artifact with `EvaluationArtifact(name, ctx)` where `name` is the display name in the UI and `ctx` is the recipe context. Use `add_samples(samples)` to append a list of `EvalSample` objects to the artifact. The method returns `self` for chaining and raises `ValueError` if the samples list is empty. ### Example ```python theme={null} from adaptive_harmony import EvaluationArtifact, EvalSample, EvalSampleInteraction, Grade, StringThread from adaptive_harmony.runtime import RecipeContext # Create the artifact eval_artifact = EvaluationArtifact(name="Model Evaluation", ctx=ctx) # Build evaluation samples samples = [ EvalSample( interaction=EvalSampleInteraction( source="my-model-key", thread=StringThread(turns=[ ("user", "What is the capital of France?"), ("assistant", "The capital of France is Paris.") ]) ), grades=[ Grade( value=0.95, grader_key="accuracy-grader", reason="Correct and concise answer" ) ], dataset_key="geography-qa" ) ] # Add samples to the artifact eval_artifact.add_samples(samples) ``` The artifact is automatically registered with the job and appears as an evaluation table in the UI when the recipe completes. The evaluation artifact will fail to render in the UI if the `source` model key in `EvalSampleInteraction` does not exist in the platform. When using `adaptive_harmony.parameters.Model` instances, get the key via `model.model_key` to ensure it matches the spawned model. Same goes for `dataset_key`, which you can get from dataset.dataset\_key if you are using `adaptive_harmony.parameters.Dataset`. The `grader_key` in each `Grade` can be a new name for a custom grader defined in the recipe. If the grader key doesn't exist in the platform, Adaptive automatically creates a new custom grader entry to reflect it in the evaluation results. ## Dataset artifact `DatasetArtifact` saves datasets generated during recipe execution. It supports different dataset kinds and saves samples in JSONL format compatible with the platform's dataset format. Import from `adaptive_harmony`: `DatasetArtifact`, `StringThread`. Import from `adaptive_harmony.runtime`: `AdaptiveDatasetKind`. Create an artifact with `DatasetArtifact(name, ctx, kind=AdaptiveDatasetKind.Mixed)` where `kind` specifies the dataset type: * `Prompt` — Prompt-only samples (no completions) * `Completion` — Prompt-completion pairs * `Metric` — Samples with evaluation metrics * `Preference` — Preference samples (good vs bad completions) * `Mixed` — Any combination of the above (default) Use `add_samples_from_thread(threads)` to convert `StringThread` objects to dataset samples and append them. This method only works with `Prompt`, `Completion`, and `Mixed` kinds. For `Metric` and `Preference` datasets, use `add_samples(samples)` with structured sample objects instead. Both methods return `self` for chaining. The `sample_count` property returns the number of samples added. ### Example ```python theme={null} from adaptive_harmony import DatasetArtifact, StringThread from adaptive_harmony.runtime import AdaptiveDatasetKind, RecipeContext # Create prompt dataset artifact prompt_artifact = DatasetArtifact( name="Generated Prompts", ctx=ctx, kind=AdaptiveDatasetKind.Prompt ) # Create completion dataset artifact completion_artifact = DatasetArtifact( name="Generated Completions", ctx=ctx, kind=AdaptiveDatasetKind.Completion ) # Generate prompts (just conversation turns, no completion) prompts = [ StringThread(turns=[ ("user", "Explain machine learning"), ("assistant", "What aspect would you like me to focus on?"), ("user", "Give me a comprehensive overview") ]), # ... more prompts ] # Generate completions (conversation turns + final assistant response) completions = [ StringThread(turns=[ ("user", "Explain machine learning"), ("assistant", "What aspect would you like me to focus on?"), ("user", "Give me a comprehensive overview") ]).assistant("Here's a comprehensive overview of machine learning...") # ... more completions ] # Add samples from threads prompt_artifact.add_samples_from_thread(prompts) completion_artifact.add_samples_from_thread(completions) print(f"Created {prompt_artifact.sample_count} prompts") print(f"Created {completion_artifact.sample_count} completions") ``` **Metric datasets** store samples with evaluation scores from multiple graders. Each sample requires a `metrics` dict mapping grader keys to float values. ```python theme={null} from adaptive_harmony import DatasetArtifact from adaptive_harmony.runtime import AdaptiveDatasetKind from harmony_client.runtime.dto.DatasetSampleFormats import DatasetMetricSample, SampleMetadata import uuid from datetime import datetime # Create metric dataset artifact metric_artifact = DatasetArtifact( name="Graded Samples", ctx=ctx, kind=AdaptiveDatasetKind.Metric ) # Build metric samples metric_samples = [ DatasetMetricSample( prompt=[("user", "What is the capital of France?")], completion=("assistant", "The capital of France is Paris."), metrics={ "accuracy": 1.0, "helpfulness": 0.95, "conciseness": 0.9 }, metadata=SampleMetadata( id=uuid.uuid4(), created_at=int(datetime.now().timestamp()), external_data={"source": "evaluation-run-1"} ) ), # ... more samples ] # Add samples to the artifact metric_artifact.add_samples(metric_samples) ``` **Preference datasets** store samples with both good and bad completions for the same prompt. ```python theme={null} from adaptive_harmony import DatasetArtifact from adaptive_harmony.runtime import AdaptiveDatasetKind from harmony_client.runtime.dto.DatasetSampleFormats import DatasetPreferenceSample, SampleMetadata import uuid from datetime import datetime # Create preference dataset artifact preference_artifact = DatasetArtifact( name="Preference Pairs", ctx=ctx, kind=AdaptiveDatasetKind.Preference ) # Build preference samples preference_samples = [ DatasetPreferenceSample( prompt=[("user", "Explain quantum computing")], good_completion=("assistant", "Quantum computing uses quantum bits (qubits) that can exist in superposition..."), bad_completion=("assistant", "Quantum computing is just really fast computers."), metric="preference", # optional, defaults to "preference" metadata=SampleMetadata( id=uuid.uuid4(), created_at=int(datetime.now().timestamp()), external_data={"annotator": "expert-1"} ) ), # ... more samples ] # Add samples to the artifact preference_artifact.add_samples(preference_samples) ``` All artifacts are automatically registered with the run and appear in the Adaptive UI when the run completes. # Asynchronous utilities Source: https://docs.adaptive-ml.com/v0.14/harmony/async-utils Helper functions for concurrent processing in recipes Adaptive Harmony provides common utilities for efficient concurrent processing of data. Use these functions to parallelize model inference, grading, and other async operations in your recipes. ### `async_map` Process all items concurrently with automatic progress tracking: ```python theme={null} from adaptive_harmony.core.utils import async_map async def grade_completion(thread: StringThread) -> float: return await grader.grade(thread) # Process all threads in parallel scores = await async_map(grade_completion, threads) ``` All tasks run concurrently. Progress is tracked automatically. **Parameters:** * `f` — Async function to apply * `data` — Sequence of items * `max_concurrent_samples` — Optional `int` cap on in-flight tasks. Backed by an `asyncio.Semaphore`. Use this to avoid overwhelming a downstream model or API. * `stage_notifier` — Optional `StageNotifier`. When set, progress is reported to the platform as each sample finishes, so the job UI shows live progress. ```python theme={null} results = await async_map( grader.score_float_value, samples, max_concurrent_samples=8, stage_notifier=ctx.job.stage_notifier("Grading"), ) ``` ### `async_map_fallible` Same as `async_map` but silently skips failures instead of crashing. Failed samples are excluded from results. ```python theme={null} from adaptive_harmony.core.utils import async_map_fallible # Some threads may fail (e.g., too long for model context) completions = await async_map_fallible(model.generate, threads) ``` Use when you expect some samples to fail and want to continue processing the rest. Accepts the same `max_concurrent_samples` and `stage_notifier` as `async_map`. Pass `return_indices=True` to recover which inputs survived. The result type changes to `list[tuple[int, T]]` where the `int` is the index in the original `data` sequence: ```python theme={null} indexed = await async_map_fallible(model.generate, threads, return_indices=True) for idx, completion in indexed: print(f"thread {idx} succeeded") ``` ### async\_map\_batch Process an iterator in batches with automatic retry on failure: ```python theme={null} from adaptive_harmony.core.utils import async_map_batch # Process batches of 50, retry failed samples up to 50% failure rate results = await async_map_batch( f=model.generate, data=iter(threads), batch_size=50, max_failure_fraction=0.5 ) ``` **Parameters:** * `f` - Async function to apply * `data` - Iterator (not list) of items * `batch_size` - Number of items per batch * `max_failure_fraction` - Max fraction of failures before raising exception (default 0.5) **Behavior:** * Processes `batch_size` items concurrently * If a sample fails, pulls next item from iterator and retries * Fails if more than `max_failure_fraction * batch_size` samples fail * Results are not ordered Use when: * Working with iterators * Want automatic retry with fresh samples on failure (as in training, where batch size must remain constant) * Don't need to preserve ordering # Training callbacks Source: https://docs.adaptive-ml.com/v0.14/harmony/callbacks Monitor and evaluate your models during training with callbacks ## Overview Training callbacks allow you to execute custom logic periodically during training runs. They're useful for monitoring model performance, generating sample outputs, saving checkpoints, and evaluating on validation sets without interrupting the main training loop. All trainer classes accept a `callbacks` parameter where you can pass a list of callback instances. ## How callbacks work Callbacks are triggered at regular intervals based on training progress (measured as a percentage of the job's completion). Each callback: 1. **Runs periodically** - You specify how often (e.g., every 10% of training via `frequency=0.1`) 2. **Returns metrics** - Callbacks return dictionaries that can be logged to your metric logger ## Using callbacks Pass callbacks to any training class via the `callbacks` parameter: ```python theme={null} from adaptive_harmony.common.grpo import GRPO from adaptive_harmony.common.callbacks import GraderEvalCallback from adaptive_harmony.metric_logger import get_prod_logger logger = get_prod_logger() # Create callbacks grader_callback = GraderEvalCallback( validation_set=validation_dataset, model=policy_model, grader=safety_grader, frequency=0.1 # Evaluate every 10% of training ) # Pass callbacks to trainer await GRPO( dataset=train_dataset, model=policy_model, grader=safety_grader, logger=logger, callbacks=[grader_callback], checkpoint_frequency=0.2 # Save checkpoints every 20% of training ).run() ``` ## Built-in callbacks ### GraderEvalCallback Evaluates your model on a validation set using a grader. This is the most common callback for monitoring training progress on held-out data. Make sure the threads in `validation_dataset` do not already contain completions, as the evaluated model will see them when asked to generate a new one. ```python theme={null} from adaptive_harmony.common.callbacks import GraderEvalCallback grader_callback = GraderEvalCallback( validation_set=validation_dataset, # List of StringThread prompts model=policy_model, # InferenceModel to evaluate (will generate completions for dataset in callback) grader=safety_grader, # Grader to score completions frequency=0.1, # Evaluate every 10% of training log_key="validation", # Prefix for logged metrics (default: "validation") clear_grader_logs=True, # Reset grader logs after each eval (default: True) temperature=0.0 # Generation temperature (default: 0.0) ) ``` **Logged metrics:** * `validation/rewards/*` - All metrics from the grader's `get_logs()` method * `validation/generation_length_mean` - Average generation length * `validation/generation_length_std` - Standard deviation of generation length * `validation/num_samples` - Number of samples evaluated ### ValidationLossCallback Computes the negative log-likelihood loss on a validation set. Useful for monitoring overfitting in supervised fine-tuning. ```python theme={null} from adaptive_harmony.common.callbacks import ValidationLossCallback loss_callback = ValidationLossCallback( validation_set=validation_dataset, # List of StringThread with completions model=policy_model, # InferenceModel to evaluate frequency=0.1, # Evaluate every 10% of training log_key="loss" # Metric key (default: "loss") ) ``` **Logged metrics:** * `validation/loss` - Average negative log-likelihood on validation set ### GenerateSamplesCallback Generates and logs sample completions periodically. Useful for qualitatively inspecting model outputs during training. ```python theme={null} from adaptive_harmony.common.callbacks import GenerateSamplesCallback samples_callback = GenerateSamplesCallback( thread_set=sample_prompts, # List of StringThread prompts to complete model=policy_model, # InferenceModel to generate completions frequency=0.1, # Generate every 10% of training log_key="samples" # Table name in logs (default: "samples") ) ``` **Logged metrics:** * `generation/samples` - Table with columns: `system`, `prompt`, `response` * `generation/generation_length_mean` - Average completion length * `generation/generation_length_std` - Standard deviation of completion length * `generation/num_samples` - Number of samples generated ## Creating custom callbacks To create your own callback, inherit from `RecipeCallback` and implement the `callback` method: ```python theme={null} from adaptive_harmony.common.callbacks import RecipeCallback from typing import Any class MyCustomCallback(RecipeCallback): def __init__(self, frequency: float, my_param: str): super().__init__(frequency, log_key_prefix="custom") self.my_param = my_param async def callback(self, current_percentage: float) -> dict[str, Any]: # Your custom logic here # current_percentage is a float from 0.0 to 1.0 # Return a dictionary of metrics to log return { "my_metric": some_value, "progress": current_percentage } ``` **Key points:** * `frequency` - How often to trigger (e.g., `0.1` = every 10% of training) * `log_key_prefix` - Optional prefix for all logged metric keys * Return a dictionary of metrics to be logged * Use async/await for any I/O operations ## Best practices 1. **Start with low frequency** - Callbacks can slow down training if you run them too often. Start with `frequency=0.1` or `0.2` and adjust as needed. 2. **Use small validation sets** - Keep validation sets small (e.g., 100-500 samples) for faster evaluation. 3. **Combine callbacks strategically**: * **GraderEvalCallback** - Essential for RL training to monitor reward on validation data * **GenerateSamplesCallback** - Helpful for debugging and qualitative inspection * **ValidationLossCallback** - Useful for SFT to detect overfitting 4. **Set appropriate temperatures**: * Use `temperature=0.0` for deterministic evaluation (recommended for validation) * Use `temperature=1.0` for diverse samples in GenerateSamplesCallback 5. **Mind your validation data** - Make sure validation prompts don't overlap with training data to get accurate generalization metrics. Also, make sure you don't include completions in validation data, otherwise the evaluated model will see them when asked to generate a new one. ## Example: comprehensive monitoring Here's a complete example combining multiple callbacks: ```python theme={null} from adaptive_harmony.common.gspo import GSPO from adaptive_harmony.common.callbacks import ( GraderEvalCallback, GenerateSamplesCallback, ) from adaptive_harmony.metric_logger import get_prod_logger logger = get_prod_logger() # Split dataset into train/validation train_dataset = dataset[:900] validation_dataset = dataset[900:] # Sample prompts for qualitative inspection sample_prompts = validation_dataset[:10] # Create callbacks grader_callback = GraderEvalCallback( validation_set=validation_dataset, model=policy_model, grader=safety_grader, frequency=0.1 ) samples_callback = GenerateSamplesCallback( thread_set=sample_prompts, model=policy_model, frequency=0.1 ) # Run training with all callbacks await GSPO( dataset=train_dataset, model=policy_model, grader=safety_grader, logger=logger, callbacks=[grader_callback, samples_callback], max_num_gspo_steps=100, samples_per_batch=128, completions_per_sample=8 ).run() ``` This setup will: * Evaluate on validation set every 10% of training * Generate sample completions every 10% for qualitative inspection * Log all metrics to your configured metric logger (W\&B, MLflow, etc.) # Input configuration Source: https://docs.adaptive-ml.com/v0.14/harmony/config Create a self-documented recipe config to parametrize your recipe's inputs Recipe input configurations allow you to define the input interface to your recipes in Python code, and have it automatically transformed it into UI components, through which you can then launch the recipe in no-code style. This presents a number of benefits: * flexibility: run the same recipe with different parameters without modifying the code * reusability: recipes can be parametrized in such a way that they can be used for different use cases all together (good examples of this are the generic prebuilt recipes, which support any dataset or use case) * self-documentation: if you write an input config class once, your recipe and all of its input parameters are automatically documented, both for business and technical users, in code and in the UI. ### Basic configuration structure Every recipe configuration inherits from a magic class `InputConfig` and defines the parameters your recipe expects in the same way as [Pydantic models](https://docs.pydantic.dev/latest/concepts/models/). Only the configuration classes that inherit from `InputConfig` in your code get automatically transformed into UI widgets. Above is an example of parameter selection in the UI for a config defined in the custom recipe as. You can set the description, title and default value for the resulting widget by using `typing.Annotated` and `pydantic.Field` as shown below. When users run your recipe, by default they will only be forced to input the values for which you have not set defaults. This allows you to simplify the input interface to your recipe as much as possible, leaving the user only with the mandatory decision to be made, while also enabling the flexibility to change fine-grained parameters if needed. ```python theme={null} from typing import Annotated from pydantic import Field class MyConfig(InputConfig): # Training parameters learning_rate: Annotated[ float, Field( description="The learning rate used for model training.", title="Learning Rate", default=7.5e-7, ) ] = 7.5e-7 policy_tp: Annotated[ int, Field( description="The tensor parallel degree for the policy model" ) ] epochs: Annotated[ int, Field( description="Number of epochs for training.", title="Epochs" ) ] ``` ### Using your configuration Once you've defined your configuration class, you can pass it to your `@recipe_main` method (the input configuration must be the first parameter of the function, before the RecipeContext). When you run your recipe with input parameters, Adaptive Engine will create a typed and validated input object, and call `@recipe_main` with an instantiated `MyConfig` class. ```python theme={null} @recipe_main async def main(config: MyConfig, ctx: RecipeContext): # Access configuration values print(f"Learning Rate: {config.learning_rate}") print(f"Policy TP: {config.policy_tp}") # Your recipe logic here # ... ``` ## Supported field types Type annotations in your config parameters allow Adaptive to validate user input and make sure widgets render correctly according to their type in the UI. You can use most types and leverage pydantic validation (operators such as `max_value`, `ge` etc.). There are also "magic" Adaptive types you can use in your config, which are platform-aware. This means that, when rendered in the UI, they allow you to select an entity that exists in your deployment as an input parameter - namely models, datasets and graders. If you pass one of these magic input parameters via SDK, it will also be validated for existence within your Adaptive deployment. Below you will find examples usage of both basic python types and Adaptive magic classes. ### Basic types ```python theme={null} from pydantic import Field from typing import Annotated class BasicConfig(InputConfig): ### BASIC TYPES ### # String field with description string_field: Annotated[str, Field(description="A string field")] = "default value" # Integer field with validation integer_field: Annotated[int, Field(description="An integer field", ge=1, le=10)] = 1 # Float field float_field: Annotated[float, Field(description="A float field")] = 0.001 # Boolean field boolean_field: Annotated[bool, Field(description="A boolean field")] = True ### COLLECTIONS ### # List field with validation list_field: Annotated[List[str], Field(description="A list field", min_length=1, max_length=10)] # Set field set_field: Annotated[Set[str], Field(description="A set field")] # Dictionary field dict_field: Annotated[Dict[str, str], Field(description="A dict field")] # Literal enum field enum_field: Annotated[Literal["sgd", "adam"], Field(description="An enum field")] = "sgd" ``` ### Adaptive types Adaptive provides special support for model, dataset, and grader fields, allowing you to pass these entities to your recipes: ```python theme={null} from adaptive_harmony.parameters import Model, Dataset, Grader class AdaptiveConfig(InputConfig): ### ADAPTIVE TYPES ### # Model selector model: Annotated[Model, Field(description="A model field")] # Dataset selector dataset: Annotated[Dataset, Field(description="A dataset field")] # Grader selector grader: Annotated[Grader, Field(description="A grader field")] ``` You can then access the relevant parameters in these objects at runtime with `await model.to_builder(ctx)` (returns a `ModelBuilder`), `await dataset.load(ctx)` (returns `list[StringThread]`) and `await grader.load(ctx)` (returns a `BaseGrader` instance). #### Filtering model and dataset selection by kind You can restrict which models or datasets are available to be selected in the UI using generic type parameters. This lets you control the minimum requirements for your particular recipe; for example, your recipe might need a dataset with completions, or a model that can be trained (not all can, external models being the prime example). ```python theme={null} from adaptive_harmony.parameters import Model, Dataset, model_kinds, dataset_kinds class FilteredConfig(InputConfig): # Only show trainable models to be selected trainable_model: Annotated[ Model[model_kinds.Trainable], Field(description="Select a trainable model") ] # Only allow inferable models (all models) to be selected inferable_model: Annotated[ Model, Field(description="Select any model in the model registry") ] # Only allow prompt datasets to be selected prompt_dataset: Annotated[ Dataset[dataset_kinds.Prompt], Field(description="Select a dataset with prompts") ] # Only allow preference datasets to be selected preference_dataset: Annotated[ Dataset[dataset_kinds.Preference], Field(description="Select a preference dataset") ] ``` If you annotate a dataset parameter in your config as Dataset\[dataset\_kinds.Prompt] but end up selecting a dataset that has available completions (this dataset still fits the constraint of having prompts, so it can be selected), the completions will be dropped when the dataset is loaded in your recipe, to match your filter. This way, you know what to expect when manipulating threads in your recipe. **Available model kinds:** * `model_kinds.Trainable` - Models that support training * `model_kinds.Inferable` - All models (for inference) **Available dataset kinds:** * `dataset_kinds.Prompt` - Prompt-only datasets (no completions) * `dataset_kinds.Completion` - Datasets with prompts and completions * `dataset_kinds.Preference` - Preference datasets (preferred vs non-preferred completions) * `dataset_kinds.Metric` - Datasets with metric feedback ### Nested configurations You can create complex configurations by nesting other config classes (currently widgets only support a single level of nestedness): ```python theme={null} class MySubConfig(InputConfig): name: Annotated[str, Field()] value: Annotated[int, Field()] children: Annotated[List[str], Field()] optimizer: Annotated[Literal["sgd", "adam"], Field()] class NestedConfig(InputConfig): # Single nested config sub_config: Annotated[MySubConfig, Field()] # List of nested configs sub_config_list: Annotated[List[MySubConfig], Field()] ``` # Multi-turn training with environments Source: https://docs.adaptive-ml.com/v0.14/harmony/cookbook/agentic/multi-turn Train agents through multi-turn conversations using environments and tool use. ## Why multi-turn training? Single-turn algorithms like GSPO generate one completion per prompt and evaluate it with a grader. Multi-turn training goes further: the policy model responds, optionally calls tools, the environment processes the response and evaluates it, and optionally generates the next user message. This cycle repeats until the conversation terminates. The entire trajectory contributes to the training signal. Harmony supports multi-turn training through ENVGRPO and ENVGSPO, the environment-based variants of GRPO and GSPO respectively. Both use the same `Environment` and `EnvironmentFactory` abstractions described in this cookbook. Multi-turn training is useful when your task involves: * Back-and-forth dialogue where context accumulates over turns * Dynamic tool calling based on conversation state * Evaluation that depends on how the full conversation unfolds ## The Environment The central abstraction in multi-turn training is the `Environment`. It orchestrates each turn of the conversation: processes the agent's output, executes tool calls if any, runs graders, generates follow-up user messages, and decides whether to continue or terminate. Everything — tools, grading, user simulation, termination — lives inside the environment. A user simulator (an LLM that generates follow-up messages on behalf of the user) is one common pattern, but it's not required. The environment's `react_to()` method has full control over what happens next, and how the next user message is produced is entirely up to the implementation. ## Tools Tools are functions the agent can call to query information or take actions. Each tool receives arguments from the agent's output, performs some operation, and returns a string result that gets appended to the conversation as a tool turn. Tool implementations are entirely up to you — they could query a database, call an API, search documents, or perform calculations. ## Subclassing Environment The environment subclasses `Environment` from `adaptive_harmony.environment` and overrides `react_to()`. This method is called after every policy model completion. It receives the full conversation thread and returns either: * `(next_thread, None)` — to continue the conversation (optionally with per-turn grades) * `(None, grades)` — to terminate and return final grades Everything happens inside `react_to()`: response parsing, tool execution, grading, generating the next user message (if any), and termination. ```python theme={null} from adaptive_harmony import StringThread from adaptive_harmony.environment import Environment, EnvironmentFactory from adaptive_harmony.graders import BaseGrader, Grade class MyEnvironment(Environment): def __init__(self, logging_function, graders, tools, max_turns=10): super().__init__(logging_function) self.graders = graders self.tools = tools self.max_turns = max_turns self.num_turns = 0 async def react_to(self, thread: StringThread) -> tuple[StringThread | None, list[Grade] | None]: """Process the agent's latest response. Returns: (next_thread, None) to continue the conversation (None, grades) to terminate with final grades """ self.num_turns += 1 # 1. Parse the agent's response (extract text, tool calls, etc.) # Implementation depends on your output format — e.g., parsing JSON # with tool_name/arguments fields, or extracting XML tags, etc. response = self._parse(thread.last_content()) # 2. Execute tool calls if present, append results to thread. # How tools are dispatched and what they return is use-case specific — # e.g., calling a search API, querying a database, running a calculator. if response.has_tool_call: thread = await self._execute_tool(thread, response.tool_call) return (thread, None) # Continue — let the model see the tool result # 3. Grade the response grades = [await g.grade(thread) for g in self.graders] # 4. Check termination. # Define your own stopping criteria — e.g., the agent said "DONE", # the user's goal was achieved, or a maximum turn count was reached. if self.num_turns >= self.max_turns or self._should_end(thread): return (None, grades) # Terminate # 5. Generate the next user message and continue. # How this is done is entirely use-case dependent — it could be # an LLM-based user simulator, a scripted reply, a lookup from # real conversation logs, etc. next_message = ... return (thread.user(next_message), None) ``` > **Note:** `_parse()`, `_execute_tool()`, and `_should_end()` are not part of the Harmony API — they represent logic you implement yourself based on your use case. The only method the framework requires is `react_to()`. ## Example: grading strategies There is no single rule for how to structure grading in multi-turn training. Here are two common approaches: **Per-turn grading** — evaluate each agent response individually. For example, a grader might check whether the agent used the right tool, or whether the response was relevant to the user's latest message. These run on every turn and provide immediate signal. **Trajectory-level grading** — evaluate the conversation as a whole after it ends. For example, checking whether the user's goal was ultimately achieved, or whether the conversation was concise. These run once and capture holistic quality. You can combine both. For instance, a per-turn `BinaryJudgeGrader` checking tool correctness alongside a trajectory-level `RangeJudgeGrader` scoring overall helpfulness, aggregated with a `CombinedGrader`. ## SFT warm-up When the agent needs to produce a specific output format (e.g., structured responses with tool calls), an SFT phase on example conversations can bootstrap format compliance before RL refines quality. Without this, the environment may not be able to parse responses and training stalls. ## EnvironmentFactory and ENVGRPO recipe ENVGRPO creates a fresh environment for each training trajectory via an `EnvironmentFactory`. The factory subclasses `EnvironmentFactory` and implements `create_environment(metadata)`, which returns a new environment instance. This ensures state (turn counts, tool call history) is not shared across parallel rollouts. A complete recipe spawns models, builds graders, creates the factory, and calls `ENVGRPO(...).run()`. If the environment uses a user simulator or other inference models internally, the factory is responsible for wiring them in. > To use ENVGSPO instead of ENVGRPO, replace `from adaptive_harmony.common import ENVGRPO` with `from adaptive_harmony.common import ENVGSPO` and swap the class in the trainer call. The `Environment` and `EnvironmentFactory` interfaces are identical for both algorithms. ```python theme={null} from adaptive_harmony.common import ENVGRPO from adaptive_harmony.environment.environment import EnvironmentFactory from adaptive_harmony.runtime import InputConfig, recipe_main from adaptive_harmony.runtime.context import RecipeContext from adaptive_harmony.parameters import Model, Dataset, dataset_kinds from adaptive_harmony.graders import BinaryJudgeGrader from adaptive_harmony.metric_logger import get_prod_logger class MyFactory(EnvironmentFactory): """Creates a fresh environment per trajectory.""" def __init__(self, graders, tools, **kwargs): super().__init__() self.graders = graders self.tools = tools self.kwargs = kwargs def create_environment(self, metadata): return MyEnvironment( logging_function=self.add_log, graders=self.graders, tools=self.tools, **self.kwargs, ) class AgentConfig(InputConfig): train_dataset: Dataset[dataset_kinds.Prompt] policy_model: Model judge_model: Model @recipe_main async def main(config: AgentConfig, ctx: RecipeContext): # Spawn models policy_builder = await config.policy_model.to_builder(ctx, tp=2) policy = await policy_builder.spawn_train("policy") judge_builder = await config.judge_model.to_builder(ctx, tp=1) judge_model = await judge_builder.spawn_inference("judge") # Build graders graders = [ BinaryJudgeGrader( grader_key="helpfulness", model=judge_model, criteria="The agent's response is helpful and addresses the user's request.", ), ] # Create factory and run ENVGRPO dataset = await config.train_dataset.load(ctx) factory = MyFactory(graders=graders, tools=[...]) logger = get_prod_logger(ctx) await ENVGRPO( dataset=dataset, model=policy, environment_factory=factory, logger=logger, lr=1e-6, max_num_grpo_steps=80, completions_per_sample=8, samples_per_batch=256, ).run() await policy.save("trained-agent", ctx=ctx) ``` ## Key takeaways 1. **Multi-turn training captures conversational dynamics** — The training signal comes from entire conversations, not isolated completions 2. **Subclass `Environment` and override `react_to()`** — This is where you wire together response parsing, tool execution, grading, and termination. It receives a `StringThread` and returns either `(next_thread, None)` to continue or `(None, grades)` to terminate 3. **`EnvironmentFactory` creates fresh environments per trajectory** — ENVGRPO/ENVGSPO run many rollouts in parallel; each needs its own state 4. **User simulation is one common pattern** — An LLM generating follow-up messages on behalf of the user removes the need for real user data during training, but how the next user message is produced is entirely up to the environment implementation 5. **Grading strategies are flexible** — You can grade per-turn, per-trajectory, or both. There's no single right approach; it depends on what behaviors you want to reinforce 6. **ENVGRPO and ENVGSPO** — Both algorithms support environment-based multi-turn training with the same `Environment` and `EnvironmentFactory` interfaces # Binary LLM judge grader Source: https://docs.adaptive-ml.com/v0.14/harmony/cookbook/custom-graders/binary-judge Use BinaryJudgeGrader to have an LLM evaluate completions as PASS or FAIL against custom criteria. ## What BinaryJudgeGrader does BinaryJudgeGrader evaluates a model completion using an LLM judge and returns either **PASS (1.0)** or **FAIL (0.0)**. Unlike rule-based graders that check for patterns or structures, BinaryJudgeGrader: 1. Takes a `criteria` string describing what makes a good completion 2. Sends the completion to a judge LLM with the criteria 3. Judge outputs `BinaryJudgeOutput` with `reasoning` and `score` (PASS/FAIL) 4. Grader returns 1.0 for PASS, 0.0 for FAIL This enables semantic evaluation—checking for qualities like "clarity", "completeness", or "consistency"—without writing custom judge prompts. ## When to use BinaryJudgeGrader * **Semantic quality checks**: You need LLM judgment on aspects rules can't capture ("Is this response helpful?") * **Binary decisions are sufficient**: You only care about PASS vs. FAIL, not a numeric scale (use `RangeJudgeGrader` for scoring on a range) * **Quick setup**: You want a working judge grader without writing custom judge prompts (use `TemplatedPromptJudgeGrader` for full control) * **Training & evaluation**: Works for both training and offline evaluation ## Configuration Create a BinaryJudgeGrader by providing: * **`grader_key`**: Unique identifier for this grader (used in logging/aggregation) * **`model`**: An inference model spawned for judging (must be an LLM capable of structured output) * **`criteria`**: A string describing what makes a completion PASS vs. FAIL ### Basic usage ```python theme={null} from adaptive_harmony.graders import BinaryJudgeGrader grader = BinaryJudgeGrader( grader_key="quality-check", model=judge_model, criteria="The response must be factually accurate, well-structured, and directly address the user's request.", ) # Use the grader grade = await grader.grade(thread) # grade.value is 1.0 (PASS) or 0.0 (FAIL) # grade.reasoning contains the judge's explanation ``` ## How the judge prompt works internally BinaryJudgeGrader constructs a judge prompt with these components: 1. **System prompt**: Instructs the judge LLM to evaluate the completion against the criteria 2. **Completion**: The model's output to be evaluated 3. **Output format**: Judge must return JSON with two fields: * `reasoning`: Explanation of why the completion PASSES or FAILS * `score`: Either `"PASS"`, `"FAIL"`, or `"NA"` (not applicable) If the judge fails to produce valid structured output, BinaryJudgeGrader retries automatically. Example judge evaluation: ```json theme={null} { "reasoning": "The response directly answers the question with multiple specific examples and clear explanations.", "score": "PASS" } ``` ## Choosing good criteria Criteria are critical—they directly shape judge behavior. Use these principles: 1. **Be specific and measurable** — Avoid vague language like "be good". Instead: * Bad: "The response should be helpful" * Good: "The response must address the user's request with at least one concrete example" 2. **Include both positive and negative indicators** — Help the judge distinguish PASS from FAIL: * "PASS: Contains verifiable facts and cites sources. FAIL: Contains unsupported claims or hallucinations." 3. **Keep criteria focused** — One dimension per grader. Use `CombinedGrader` to aggregate multiple graders for multi-dimensional evaluation: * One grader checks "Is this factually accurate?" * Another checks "Does this follow the requested format?" * Combine them to get overall quality score ### Example criteria of increasing specificity: ```python theme={null} # Example 1: Generic (not recommended) criteria_generic = "The response is good." # Example 2: More specific criteria_specific = ( "The response must directly answer the user's question without introducing unrelated topics." ) # Example 3: Highly specific with PASS/FAIL conditions criteria_detailed = ( "PASS: The response directly answers the user's question with clear reasoning, " "includes at least one relevant example or supporting detail, and contains no factual errors. " "FAIL: The response is vague, goes off-topic, lacks supporting details, or contains factual errors." ) # Example 4: Domain-specific (customer support) criteria_support = ( "PASS: The response acknowledges the customer's problem, provides a clear solution, " "and offers follow-up support. FAIL: The response dismisses the problem, provides no solution, " "or is unclear." ) # Example 5: Domain-specific (content summarization) criteria_summary = ( "PASS: The summary captures the main points, is concise (2-3 sentences), " "and uses language from the original text. FAIL: The summary is too long, " "omits key points, or introduces information not in the original." ) ``` ## Using BinaryJudgeGrader in a recipe In a training recipe, spawn a judge model and create the grader: ```python theme={null} from adaptive_harmony.runtime import InputConfig, recipe_main from adaptive_harmony.runtime.context import RecipeContext from adaptive_harmony.graders import BinaryJudgeGrader class MyConfig(InputConfig): train_dataset: Dataset model_to_train: Model judge_model: Model @recipe_main async def main(config: MyConfig, ctx: RecipeContext): # Spawn the judge model with spawn_inference judge_builder = await config.judge_model.to_builder(ctx, tp=1) judge = await judge_builder.spawn_inference("judge") # Create the grader grader = BinaryJudgeGrader( grader_key="factuality", model=judge, criteria="The response contains only factually verifiable statements and cites sources when needed.", ) # Use the grader in training... # grade = await grader.grade(sample) ``` ## Using few-shot examples (optional) For more consistent judging, you can provide few-shot examples via `BinaryJudgeShot`. Each shot pairs a thread with the expected score and reasoning, teaching the judge your standards for PASS vs. FAIL. ```python theme={null} from adaptive_harmony.graders.binary_judge import BinaryJudgeShot from adaptive_harmony import StringThread # Example 1: Good response — should PASS good_example = StringThread([ ("user", "What is machine learning?"), ("assistant", "Machine learning is a subset of AI where systems learn patterns from data " "without explicit programming. For example, recommendation systems use ML to suggest " "products based on user history.") ]) # Example 2: Poor response — should FAIL poor_example = StringThread([ ("user", "What is machine learning?"), ("assistant", "ML is cool.") ]) shots = [ BinaryJudgeShot(thread=good_example, score="PASS", reasoning="Clear definition with a concrete example"), BinaryJudgeShot(thread=poor_example, score="FAIL", reasoning="Too vague, no examples or detail"), ] ``` ### Validation We can validate the Grade object structure and few-shot setup without a live judge model. In a real recipe, calling `grader.grade(thread)` sends the thread to the judge LLM and returns one of these Grade objects. ```python theme={null} from adaptive_harmony import Grade # Simulate the Grade objects that BinaryJudgeGrader.grade() returns pass_grade = Grade( value=1.0, grader_key="quality-check", reasoning="The response directly addresses the question with clear examples and accurate information.", ) fail_grade = Grade( value=0.0, grader_key="quality-check", reasoning="The response is vague and does not provide concrete examples as required.", ) # Validate structure assert pass_grade.value == 1.0 assert fail_grade.value == 0.0 assert pass_grade.grader_key == "quality-check" assert len(pass_grade.reasoning) > 0 print(f"✅ PASS grade: value={pass_grade.value}, reasoning=\"{pass_grade.reasoning}\"") print(f"❌ FAIL grade: value={fail_grade.value}, reasoning=\"{fail_grade.reasoning}\"") # Validate few-shot examples are well-formed for shot in shots: assert shot.score in ("PASS", "FAIL"), f"Invalid score: {shot.score}" assert len(shot.thread.last_content()) > 0, "Shot thread must have content" print(f"\n✅ All {len(shots)} few-shot examples validated") ``` ## Key takeaways 1. **BinaryJudgeGrader is the simplest LLM judge** — Just provide a judge model and criteria string 2. **Write specific, measurable criteria** — Vague criteria lead to inconsistent judging; be explicit about PASS/FAIL conditions 3. **Use for semantic quality checks** — When rules can't capture what makes a completion good (e.g., "helpful", "clear", "complete") 4. **Optional: Add few-shot examples** — Improves consistency by showing the judge what PASS and FAIL look like 5. **For finer-grained evaluation**, see: * `RangeJudgeGrader` — Score on a numeric scale (e.g., 1-5) instead of binary PASS/FAIL * `TemplatedPromptJudgeGrader` — Write custom judge prompts with templates * `CombinedGrader` — Combine multiple graders for multi-dimensional evaluation # Label classification accuracy Source: https://docs.adaptive-ml.com/v0.14/harmony/cookbook/custom-graders/classification Build a custom grader for label classification accuracy vs. a groundtruth ## Grader description This example shows how to build a custom grader to train on classification accuracy against a groundtruth label. It is common to require a JSON formatted output from an LLM, where one or more of the keys in the output are a predicted label from a set of possible labels. An example of this, if we asked an LLM to analyze a call transcript and extract information: **Example prompt:** ``` Analyze the support call, reason about the sentiment of the customer, and whether they wanted to escalate to a human. Provide your output in the following JSON format, nothing else: { "reason": str, "sentiment: Literal["angry", "sad", "neutral", "happy", "thrilled"], "escalation": bool } ``` **Example output:** ```json theme={null} { "reason": "The customer was really angry and asked to speak to a human.", "sentiment": "angry", "escalation": true } ``` The grader we will build compares the values predicted for `sentiment` and `escalation` against groundtruth stored in each sample's metadata. This same grader would work for any number of labels in the output, and any expected JSON format. ## Pseudocode This is the pseudocode for our grader: * Validate a text completion in JSON format against a pydantic model * If completion does not comply with expected format, return `-1.0` reward * Compare predicted category against ground truth label * Returns binary scores (1.0 for correct, 0.0 for incorrect) ## Implementation ### How label matching works The grader uses the same key (`label_key`) to extract labels from two places: 1. From the Pydantic output: Uses `getattr(prediction, label_key)` to get the predicted label 2. From sample metadata: Uses `sample.metadata.get(label_key)` to get the ground truth Example with `label_key="sentiment"`: * Predicted: `prediction.sentiment` → `"angry"` * Ground truth: `sample.metadata["sentiment"]` → `"angry"` * Score: `1.0` (match!) This design keeps the grader simple and ensures the field names are consistent across your Pydantic schema and dataset metadata. ### Why -1.0 for malformed outputs in training? During training with RL, we use `-1.0` for malformed outputs instead of `0.0` because: 1. Stronger penalty signal: `-1.0` tells the RL algorithm that malformed output is worse than just getting the wrong category. This encourages the model to always produce valid JSON. 2. Distinguishes failure modes: * `-1.0` = "You broke the format" * `0.0` = "You followed the format but chose the wrong category" * `1.0` = "Perfect!" 3. Faster convergence: The stronger signal helps the model learn to produce structured output early in training, before fine-tuning category predictions. During evaluation, we use `0.0` because we only care about binary correctness (right or wrong), not degrees of failure. ```python theme={null} from typing import Literal from pydantic import BaseModel, Field from adaptive_harmony import StringThread, Grade from adaptive_harmony.core.structured_output import pydantic_parse, OutputParserException from adaptive_harmony.graders import BaseGrader class CallAnalysisFormat(BaseModel): """Schema for call analysis output.""" reason: str = Field(description="Reasoning behind customer sentiment and escalation prediction") sentiment: Literal["angry", "sad", "neutral", "happy", "thrilled"] = Field(description="Customer sentiment") escalation: bool = Field(description="Whether the customer escalated the conversation to a human") class LabelClassificationGrader(BaseGrader): """ Grader for label classification accuracy. Compares a predicted label/categories against the ground truth label. The predicted label is extracted from the Pydantic output using the same key as the ground truth label in metadata. """ def __init__(self, grader_key: str, label_key: str, eval_mode: bool, expected_format: type[BaseModel]): """ Args: grader_key: Unique identifier for this grader label_key: Key to find both the ground truth in metadata AND the predicted label in the Pydantic output eval_mode: If True, score malformed outputs as 0.0. If False (training), score as -1.0 """ super().__init__(grader_key) self.label_key = label_key self.eval_mode = eval_mode self.expected_format = expected_format async def grade(self, sample: StringThread) -> Grade: # Try to parse the model's structured output try: prediction = pydantic_parse(sample.last_content(), self.expected_format) except OutputParserException: # Model output was not valid JSON or didn't match the schema # During training: return -1.0 to penalize malformed outputs # During evaluation: return 0.0 (treat as incorrect, but don't skew the average score) score = 0.0 if self.eval_mode else -1.0 self.add_log({"score": score}) return Grade( value=score, grader_key=self.grader_key, reasoning="JSON parse error", ) except Exception as e: # Unexpected error - fail loudly raise Exception(f"Error parsing structured output: {e}") # Extract predicted label from Pydantic output using the same key as metadata predicted_label = getattr(prediction, self.label_key) # Get ground truth from sample metadata groundtruth_label = sample.metadata.get(self.label_key) # Compare prediction to ground truth score = 1.0 if predicted_label == groundtruth_label else 0.0 # Log for experiment tracking self.add_log({"score": score}) # Return grade with reasoning for debugging reason = f"Predicted: {predicted_label}\nGround truth: {groundtruth_label}" return Grade(value=score, grader_key=self.grader_key, reasoning=reason) ``` ### Test cases We create 3 simple test cases: * correct completion: right format and labels * incorrect completion: right format, wrong labels * malformed completion: label values are not supported, likely a hallucination; could also be incorrect JSON, like a missing curly brace ```python theme={null} from adaptive_harmony.core.structured_output import render_schema instructions = f"""Analyze the support call, reason about the sentiment of the customer, and whether they wanted to escalate to a human. Your output must comply with the following format, valid JSON output only: {render_schema(CallAnalysisFormat)}""" call_content = "Enough of this!! I am mad, I want to speak to a human" groundtruth_labels = {"sentiment": "angry", "escalation": True} # label is correct correct_completion = CallAnalysisFormat( reason="It is clear that the content indicates anger and an escalation", sentiment="angry", escalation=True ).model_dump_json(indent=2) # labels are incorrect incorrect_completion = CallAnalysisFormat( reason="The customer sounded very happe", sentiment="happy", escalation=False ).model_dump_json(indent=2) # malformed completion; predicted label does not exist in list of possible labels, escalation is not a bool malformed_completion = """{ "reason": "this is the reason", "sentiment": "very-very-happy", "escalation": "YES" }""" correct_thread = StringThread( turns=[("system", instructions), ("user", call_content), ("assistant", correct_completion)], metadata=groundtruth_labels, ) incorrect_thread = StringThread( turns=[("system", instructions), ("user", call_content), ("assistant", incorrect_completion)], metadata=groundtruth_labels, ) malformed_thread = StringThread( turns=[("system", instructions), ("user", call_content), ("assistant", malformed_completion)], metadata=groundtruth_labels, ) print("🟢 Correct thread:") print(correct_thread) print("🔴 Incorrect thread:") print(incorrect_thread) print("⚠️ Malformed thread:") print(malformed_thread) ``` ### Validation Run tests on the mock samples ```python theme={null} grader = LabelClassificationGrader( grader_key="sentiment-grader", label_key="sentiment", expected_format=CallAnalysisFormat, eval_mode=False ) correct_grade = await grader.grade(correct_thread) incorrect_grade = await grader.grade(incorrect_thread) malformed_grade = await grader.grade(malformed_thread) assert correct_grade.value == 1.0 print(correct_grade) print("✅ Correct prediction → reward = 1\n") assert incorrect_grade.value == 0.0 print(incorrect_grade) print("❌ Wrong prediction → reward = 0\n") assert malformed_grade.value == -1.0 print(malformed_grade) print("⚠️ Malformed output → reward = -1") ``` ```python theme={null} from pprint import pprint print("📊 Grader logs:") pprint(grader.get_logs()) ``` ## Key takeaways 1. **Always validate structured output** - Use Pydantic schemas and handle parse errors gracefully with pydantic parse 2. **Different penalties for training vs. eval** - Penalize format errors more heavily during training 3. **Log scores for tracking** - Use `self.add_log()` to send metrics to training loggers 4. **Include reasoning in grades** - Helps debug model behavior during development # Ranged scoring with LLM judges Source: https://docs.adaptive-ml.com/v0.14/harmony/cookbook/custom-graders/range-judge Build a custom grader for nuanced numeric scoring (e.g., 1-5 scale) using LLM judges instead of binary PASS/FAIL ## Grader description A `RangeJudgeGrader` evaluates completions on a numeric scale instead of binary pass/fail. This is useful when quality exists on a spectrum and you want fine-grained feedback about model behavior. Unlike a `BinaryJudgeGrader` (which scores 0.0 or 1.0), a `RangeJudgeGrader` returns scores across a range (e.g., 1-5). The grader: * Accepts **criteria** describing what each score level means * Uses **evaluation steps** to guide the judge's reasoning * Takes **few-shot examples** showing different score levels * Returns a **normalized score** (0-1 by default) based on logprob-weighted scoring * Optionally uses **subrange expectations** to help the judge understand the rubric ## Pseudocode This is the pseudocode for a range-based judge grader: * Define explicit criteria describing what each score level means * Provide numbered evaluation steps the judge should follow * Describe what each score range means using `SubrangeExpectations` (textual descriptions, not numeric mappings) * Include 2-3 few-shot examples showing what different score levels look like * Pass all components to `RangeJudgeGrader` with an InferenceModel * The LLM judge scores the completion on the numeric range (e.g., 1-5) * Return a normalized grade (0-1 by default) ## Implementation ```python theme={null} from adaptive_harmony import StringThread from adaptive_harmony.graders.range_judge import RangeJudgeShot, SubrangeExpectations ``` ### Defining the criteria and evaluation steps Start by writing clear criteria that describe what each score level means. The rubric should make the boundaries explicit—what distinguishes a 1 from a 2, or a 3 from a 4. Then list numbered evaluation steps that guide the judge's reasoning. These steps should be concrete and actionable. ```python theme={null} # Define the scoring criteria criteria = """ Score the quality of the completion on a scale of 1-5: 1 - Irrelevant or off-topic: Completion does not address the prompt or is contradictory. 2 - Incomplete: Completion touches on the topic but lacks detail or clarity. 3 - Adequate: Completion covers the main points but is missing some depth or supporting details. 4 - Good: Completion is relevant, clear, and covers most key aspects with supporting details. 5 - Excellent: Completion is thorough, well-structured, and provides comprehensive information with clear reasoning. """ # Define the evaluation steps evaluation_steps = [ "Does the completion directly address the prompt?", "Is the information relevant and accurate?", "How complete is the answer? Are key points covered?", "Is the information well-organized and easy to follow?", "Are claims supported with reasoning or examples?", "Assign a score from 1-5 based on the rubric above.", ] print("📋 Criteria:") print(criteria) print("📝 Evaluation steps:") for i, step in enumerate(evaluation_steps, 1): print(f" {i}. {step}") ``` ### SubrangeExpectations: textual descriptions of score ranges `SubrangeExpectations` is a NamedTuple with `subrange` (a tuple of min and max score) and `expectation` (a text description of what that score range means). These help the judge understand the rubric by describing the expected quality level for each score range. This is different from reward mapping—these are purely textual descriptions to guide the LLM judge. ```python theme={null} # Define subrange expectations: what each score range means subrange_expectations = [ SubrangeExpectations( subrange=(1, 2), expectation="The response is off-topic, incomplete, or of very low quality" ), SubrangeExpectations( subrange=(3, 3), expectation="The response covers the main points adequately but lacks depth or polish" ), SubrangeExpectations( subrange=(4, 5), expectation="The response is comprehensive, well-structured, and demonstrates expert understanding" ), ] print("🎯 Subrange expectations:") for se in subrange_expectations: print(f" 📊 Scores {se.subrange[0]}-{se.subrange[1]}: {se.expectation}") ``` ### Few-shot examples for the judge Few-shot examples teach the judge what each score looks like by showing real examples. Each shot is a `RangeJudgeShot` with: * **`thread`**: A StringThread with the prompt and completion * **`reasoning`**: The judge's reasoning for that score * **`score`**: The gold-standard score for that example ```python theme={null} # Few-shot examples for the judge prompt = "Explain the difference between correlation and causation." # Example 1: Score 1 (Irrelevant) score_1_completion = "Correlation is when two things happen together. Causation is when something causes something else." score_1_thread = StringThread([ ("user", prompt), ("assistant", score_1_completion) ]) # Example 2: Score 3 (Adequate) score_3_completion = ( "Correlation means two variables move together (both go up or both go down). " "Causation means one variable actually causes the change in the other. " "A classic example is ice cream sales and drowning deaths—both increase in summer, but ice cream doesn't cause drowning." ) score_3_thread = StringThread([ ("user", prompt), ("assistant", score_3_completion) ]) # Example 3: Score 5 (Excellent) score_5_completion = ( "Correlation describes a statistical relationship between two variables where they tend to move together. " "When one increases, the other tends to increase (positive correlation) or decrease (negative correlation). " "However, correlation does not imply causation. Two variables can be correlated for several reasons:\n" "1. One causes the other (genuine causal link)\n" "2. A third variable causes both (confounding variable)\n" "3. The relationship is coincidental\n\n" "Example: Ice cream sales and drowning deaths are positively correlated, both peak in summer. " "But ice cream doesn't cause drowning—warm weather is the confounding variable that causes both. " "To establish causation, you need controlled experiments or causal inference methods that rule out confounds." ) score_5_thread = StringThread([ ("user", prompt), ("assistant", score_5_completion) ]) # Create RangeJudgeShot objects shots = [ RangeJudgeShot( thread=score_1_thread, reasoning="This explanation is too brief and lacks the key insight that correlation does not imply causation. It provides only basic definitions without examples or nuance.", score=1 ), RangeJudgeShot( thread=score_3_thread, reasoning="This explanation covers the main distinction and provides a useful example. However, it could go deeper by explaining confounding variables more clearly.", score=3 ), RangeJudgeShot( thread=score_5_thread, reasoning="This is a comprehensive explanation that defines both concepts clearly, provides multiple reasons for correlation without causation, includes a concrete example, and explains the role of confounding variables. The structure is logical and easy to follow.", score=5 ), ] print(f"🎓 Created {len(shots)} few-shot examples:") for shot in shots: print(f" 📌 Score {shot.score}: {len(shot.thread.last_content())} characters") ``` ### Validation We can validate all the setup objects and demonstrate score normalization without a live judge model. ### Creating the RangeJudgeGrader The `RangeJudgeGrader` constructor requires: * **`grader_key`**: Unique identifier for this grader * **`model`**: An InferenceModel (required) * **`criteria`**: Description of the scoring rubric * **`score_range`**: Tuple of (min, max) for the numeric scale (default: (1, 5)) * **`evaluation_steps`**: List of strings describing evaluation steps (optional; auto-generated if None) * **`subrange_expectations`**: List of SubrangeExpectations describing what each score range means (optional) * **`shots`**: List of RangeJudgeShot few-shot examples (optional) * **`normalize_score`**: If True (default), scores are normalized to 0-1 range By default, scores are normalized: score 1 → 0.0, score 5 → 1.0. ```python theme={null} from adaptive_harmony.graders import RangeJudgeGrader # Requires an InferenceModel — shown as reference only grader = RangeJudgeGrader( grader_key="completion-quality", model=inference_model, # InferenceModel instance criteria=criteria, score_range=(1, 5), evaluation_steps=evaluation_steps, subrange_expectations=subrange_expectations, shots=shots, normalize_score=True, # Scores 1-5 → 0.0-1.0 ) ``` ```python theme={null} score_range = (1, 5) # Validate subrange expectations cover the full score range all_covered = set() for se in subrange_expectations: for s in range(se.subrange[0], se.subrange[1] + 1): all_covered.add(s) assert all_covered == set(range(score_range[0], score_range[1] + 1)), \ f"Subranges don't cover full range: missing {set(range(*score_range)) - all_covered}" print(f"✅ Subrange expectations cover the full {score_range[0]}-{score_range[1]} range") # Validate few-shot examples have scores within range for shot in shots: assert score_range[0] <= shot.score <= score_range[1], \ f"Shot score {shot.score} outside range {score_range}" assert len(shot.reasoning) > 0, "Shot must have reasoning" print(f"✅ All {len(shots)} few-shot scores are within [{score_range[0]}, {score_range[1]}]") # Demonstrate normalization: raw score → [0, 1] def normalize(raw_score, lo, hi): return (raw_score - lo) / (hi - lo) print(f"\n📊 Score normalization (range {score_range[0]}-{score_range[1]}):") for raw in range(score_range[0], score_range[1] + 1): norm = normalize(raw, *score_range) print(f" Raw {raw} → Normalized {norm:.2f}") ``` ### How scoring works The `RangeJudgeGrader` uses logprob-weighted scoring internally: 1. The LLM judge produces a score (1-5) with reasoning 2. The grader computes logprobs for each possible score (1, 2, 3, 4, 5) 3. These logprobs are converted to probabilities 4. A weighted average is computed: `sum(score_i * prob_i for each score)` 5. If `normalize_score=True`, the weighted score is mapped to \[0, 1]: * Raw score 1 → 0.0 * Raw score 5 → 1.0 * Linear interpolation in between This design captures the judge's uncertainty about borderline cases and prevents sharp boundaries. ## RangeJudgeGrader vs BinaryJudgeGrader | Aspect | BinaryJudgeGrader | RangeJudgeGrader | | ------------------- | ------------------------ | ----------------------------- | | **Score** | 0.0 or 1.0 (PASS/FAIL) | 0.0 to 1.0 (normalized range) | | **Use case** | Clear pass/fail criteria | Spectrum-based quality | | **Rubric** | Single PASS criterion | Multiple rubric levels | | **Few-shots** | 1-2 examples | 2-3 examples (low, mid, high) | | **Training signal** | All-or-nothing reward | Fine-grained gradients | | **Judge reasoning** | Simple: PASS or FAIL | Structured: reasoning + score | **Use BinaryJudgeGrader when** you have a clear PASS/FAIL threshold (e.g., "Does the output contain valid JSON?"). **Use RangeJudgeGrader when** quality exists on a spectrum (e.g., "How well does this summary capture the key points?"). ## Key takeaways 1. **RangeJudgeGrader for spectrum-based quality** - When binary PASS/FAIL is too coarse, numeric ranges capture degrees of quality 2. **Write explicit criteria** - Define what each score level means so the judge has clear guidance 3. **Include numbered evaluation steps** - Structured steps guide the judge's reasoning process 4. **Few-shot examples are essential** - Show examples at low, medium, and high scores so the judge understands the full scale 5. **Normalization is on by default** - Scores are mapped to \[0, 1] by default for consistency with other graders # Grading structured JSON with template injection Source: https://docs.adaptive-ml.com/v0.14/harmony/cookbook/custom-graders/structured-output Use TemplatedPromptJudgeGrader to evaluate structured JSON completions by extracting fields and injecting them into an LLM judge prompt ## Overview This cookbook demonstrates how to grade models that produce structured JSON output. The key idea: extract specific fields from the JSON and inject them into a custom judge prompt template. This is useful when: * Your model produces structured output (JSON with multiple fields) * You want an LLM judge to evaluate the *content* of specific fields semantically * You need to transform or format extracted fields before passing them to the judge ## Example scenario A model answers a question and produces structured output: ```json theme={null} { "answer": "The capital of France is Paris.", "confidence": "high" } ``` The grader: 1. Parses the JSON and extracts the `answer` field 2. Injects it into a judge template along with the original question 3. Sends to an LLM judge: "Is this answer correct and well-supported?" 4. Returns the judge's PASS (1.0) or FAIL (0.0) score ## Implementation Define a simple Pydantic schema: ```python theme={null} from typing import Literal from pydantic import BaseModel, Field from adaptive_harmony import StringThread, Grade from adaptive_harmony.core.structured_output import pydantic_parse, OutputParserException, render_schema from adaptive_harmony.graders.templated_prompt_judge import TemplatedPromptJudgeGrader class AnalysisOutput(BaseModel): """Schema for structured analysis output.""" answer: str = Field(description="The model's answer to the question") confidence: Literal["high", "medium", "low"] = Field(description="Model's confidence in the answer") ``` ## How TemplatedPromptJudgeGrader works `TemplatedPromptJudgeGrader` allows you to: 1. Parse structured output from a model 2. Extract specific fields and transform them 3. Inject them into a Handlebars template using `{{variable}}` syntax 4. Send the rendered prompt to an LLM judge for semantic evaluation The key mechanism is **overriding `extract_template_context()`**. This method: * Parses your structured JSON * Builds a dict of variables to inject into the template * Returns the dict for Handlebars rendering When grading: 1. `extract_template_context()` parses the JSON and builds context variables 2. The user template is rendered with those variables using Handlebars 3. The rendered prompt + judge system prompt are sent to an LLM judge 4. The judge returns a binary decision (PASS/FAIL → 1.0/0.0) ## Custom grader with extract\_template\_context override ```python theme={null} class AnalysisGrader(TemplatedPromptJudgeGrader): """Custom grader for structured analysis output. Parses JSON, extracts the answer field, and sends it to an LLM judge for semantic evaluation. """ @classmethod def extract_template_context(cls, thread: StringThread, output_model=None, template_variables=None): """Parse structured JSON and inject extracted fields into the judge prompt. Returns a dict with keys that match {{variables}} in your user_template. """ # Get default context from parent (includes thread, last_user_turn_content, etc.) default_context = super().extract_template_context(thread, output_model, template_variables) # Parse the model's last response as JSON and extract fields try: structured_output = pydantic_parse(thread.last_content(), AnalysisOutput) # Add extracted fields to context for template injection default_context["answer"] = structured_output.answer default_context["confidence"] = structured_output.confidence except OutputParserException: # If parsing fails, we'll handle it in grade() method pass return default_context async def grade(self, sample: StringThread) -> Grade: """Grade the sample. If JSON parsing fails, return a penalty score.""" try: # Try to parse to ensure valid JSON pydantic_parse(sample.last_content(), AnalysisOutput) # If valid, call parent's grade method which uses the judge return await super().grade(sample) except OutputParserException: # Model output was not valid JSON or didn't match the schema is_eval = sample.metadata.get("eval", False) score = 0.0 if is_eval else -1.0 return Grade( value=score, grader_key=self.grader_key, reasoning="Malformed JSON: does not match expected schema", ) ``` ## Why override extract\_template\_context()? The parent class provides basic context variables (`thread`, `last_content`, `last_user_turn_content`). But with structured output, you need to: 1. **Parse the JSON** — Convert raw string to Pydantic model 2. **Extract fields** — Get specific values (e.g., `answer`, `confidence`) 3. **Transform** — Format them for readability (e.g., join lists with newlines) 4. **Inject into template** — Provide clean variables for the judge prompt This separation keeps the judge prompt readable and focused on semantic evaluation, not raw JSON strings. ## Define judge prompts The judge needs: * **System prompt** — Instructs the judge on evaluation criteria * **User template** — Dynamic prompt with `{{variable}}` placeholders filled from your parsed JSON using Handlebars syntax ```python theme={null} JUDGE_SYSTEM_PROMPT = """ You are an expert evaluator assessing the quality of model responses. Evaluate based on these criteria: 1. **Correctness**: Is the answer factually accurate? 2. **Completeness**: Does it fully address the question? 3. **Clarity**: Is it clear and well-articulated? Respond with PASS if the answer meets all criteria, FAIL otherwise. """ JUDGE_USER_TEMPLATE = """ Question: {{last_user_turn_content}} Model answer: {{answer}} Confidence: {{confidence}} Is this answer correct, complete, and well-articulated? """ print("📋 Judge system prompt:") print(JUDGE_SYSTEM_PROMPT) print("📝 Judge user template (Handlebars variables will be filled at grading time):") print(JUDGE_USER_TEMPLATE) ``` ## Test cases Create test inputs to validate parsing and template extraction: ```python theme={null} # Question to be answered question = "What is the capital of France?" instructions = f"""Answer the question. Output must be valid JSON: {render_schema(AnalysisOutput)} """ # Good answer good_answer = AnalysisOutput( answer="The capital of France is Paris. It is located in north-central France on the Seine River.", confidence="high" ).model_dump_json(indent=2) # Poor answer (incomplete/vague) poor_answer = AnalysisOutput( answer="It's a city in France.", confidence="low" ).model_dump_json(indent=2) # Malformed JSON malformed_answer = """{ "answer": "Paris", "confidence": "very-high" }""" good_thread = StringThread( [("system", instructions), ("user", question), ("assistant", good_answer)], metadata={"eval": False} ) poor_thread = StringThread( [("system", instructions), ("user", question), ("assistant", poor_answer)], metadata={"eval": False} ) malformed_thread = StringThread( [("system", instructions), ("user", question), ("assistant", malformed_answer)], metadata={"eval": False} ) print("🟢 Good thread:") print(good_thread) print("🟡 Poor thread:") print(poor_thread) print("⚠️ Malformed thread:") print(malformed_thread) ``` ## Validation We cannot run a live LLM judge in this notebook. Instead, we validate: 1. Malformed JSON is caught and returns -1.0 during training 2. Valid JSON is parsed correctly 3. Template variables are extracted and injected correctly ```python theme={null} # Test that malformed JSON is caught during parsing try: pydantic_parse(malformed_thread.last_content(), AnalysisOutput) print("🚨 ERROR: Should have failed to parse malformed JSON") except OutputParserException as e: print(f"✅ Correctly caught parse error: {e}\n") # Test that good and poor answers parse correctly good_output = pydantic_parse(good_thread.last_content(), AnalysisOutput) print("✅ Good answer parsed:") print(f" 📝 answer: {good_output.answer[:60]}...") print(f" 🔒 confidence: {good_output.confidence}\n") poor_output = pydantic_parse(poor_thread.last_content(), AnalysisOutput) print("✅ Poor answer parsed:") print(f" 📝 answer: {poor_output.answer}") print(f" 🔒 confidence: {poor_output.confidence}") ``` ## Template extraction Extract template context and verify that variables are available for the judge prompt: ```python theme={null} # Test template extraction context = AnalysisGrader.extract_template_context(good_thread, AnalysisOutput) print("🔍 Extracted template context:") print(f" 📝 answer: {context['answer']}") print(f" 🔒 confidence: {context['confidence']}") print(f" 💬 last_user_turn_content: {context['last_user_turn_content']}") # Verify that required variables for the template are present required_vars = ["answer", "confidence", "last_user_turn_content"] for var in required_vars: assert var in context, f"Missing template variable: {var}" print(f"\n✅ All required template variables extracted: {required_vars}") ``` ## Available template variables When you override `extract_template_context()`, you have access to these default variables: * `output_schema` — JSON schema of the output model * `turns` — Full thread of all turns (system, user, assistant) * `metadata` — Custom metadata from the thread * `context_turns` — All turns including context * `context_str` — String representation of full thread * `context_turns_without_last_user` — All turns except the last user message * `context_str_without_last_user` — String without last user message * `last_user_turn_content` — Content of the last user turn * `completion` — The model's completion You can add custom variables by updating the context dict, as shown in the `AnalysisGrader` above. ## Using the grader in training To use `AnalysisGrader` in a real training recipe: ```python theme={null} from adaptive_harmony.models import InferenceModel from adaptive_harmony.graders.templated_prompt_judge import BinaryJudgeOutput # Assume judge_model is a spawned inference model grader = AnalysisGrader( grader_key="analysis-grader", model=judge_model, system_template=JUDGE_SYSTEM_PROMPT, user_template=JUDGE_USER_TEMPLATE, output_model=BinaryJudgeOutput, temperature=0.0, ) ``` Note: * The `model` parameter is required and must be an `InferenceModel` (from a spawned judge model in your recipe) * The `output_model` must be `BinaryJudgeOutput` for binary grading * Handlebars templates render variables with `{{variable}}` syntax ## Key takeaways 1. **Use TemplatedPromptJudgeGrader for semantic evaluation** — When you need an LLM judge to evaluate extracted content, templated graders provide flexibility and control. 2. **Override extract\_template\_context() to parse and transform** — Parse your structured JSON, extract fields, and inject them as clean variables. This keeps the judge prompt focused and readable. 3. **Handlebars templates make prompts reusable** — Use `{{variable}}` syntax to create flexible judge prompts that work with any structured schema. 4. **Different penalties for training vs. eval** — Use `-1.0` for format errors during training (stronger signal) and `0.0` during evaluation. 5. **Validate early and return fast** — Check for parsing errors in `grade()` before calling the judge, so malformed outputs don't waste inference. 6. **The grader requires an InferenceModel** — `TemplatedPromptJudgeGrader` cannot be instantiated without a real judge model. In recipes, you'll spawn this model and pass it to the grader constructor. # Two-stage training - SFT warm-up before RL Source: https://docs.adaptive-ml.com/v0.14/harmony/cookbook/training/sft-warmup Sft Warmup Combine supervised fine-tuning and reinforcement learning in a single recipe. Train on demonstrations first to bootstrap output format, then continue with RL to improve quality. ## Why two stages? SFT teaches the model to imitate a curated set of demonstrations — picking up format, structure, and style from labeled examples. It generalizes from them, but remains anchored to their distribution: the model learns how good responses look, not necessarily how to reason toward them. RL builds on this foundation, optimizing behavior through feedback signals rather than imitation. Guided by a reward function, it can surface behaviors and strategies that no demonstration explicitly encoded — though the quality of what it discovers is ultimately bounded by the quality of the signal it's optimizing for. Without SFT, RL must spend samples learning format basics — JSON structure, required fields, response conventions — rather than optimizing for quality. These constraints are far cheaper to teach through direct imitation than through reward signals, which are poorly suited to enforce rigid structural rules. SFT bootstraps format compliance efficiently, giving RL a well-initialized starting point. From there, RL can focus its signal on what matters: refining quality beyond what the demonstrations captured. ## The two-stage recipe This is a complete `@recipe_main` that orchestrates both SFT and RL in sequence. This example uses GSPO for the RL stage, but other algorithms (GRPO, ENVGRPO, ENVGSPO) work the same way: ```python theme={null} from adaptive_harmony.runtime import InputConfig, recipe_main from adaptive_harmony.runtime.context import RecipeContext from adaptive_harmony.common.sft import SFT from adaptive_harmony.common.gspo import GSPO from adaptive_harmony.parameters import Model, Dataset, dataset_kinds from adaptive_harmony.graders import BinaryJudgeGrader from adaptive_harmony.graders.combined_grader import CombinedGrader from adaptive_harmony.metric_logger import get_prod_logger class SFTtoGSPOConfig(InputConfig): """Configuration for SFT warm-up followed by GSPO training.""" sft_dataset: Dataset[dataset_kinds.Completion] # Demonstrations with expected outputs gspo_dataset: Dataset[dataset_kinds.Prompt] # Prompts for RL (no ground truth needed) model_to_train: Model # SFT hyperparameters sft_learning_rate: float = 1e-5 sft_epochs: int = 1 sft_batch_size: int = 512 # GSPO hyperparameters gspo_learning_rate: float = 5e-7 gspo_max_steps: int = 1000 gspo_batch_size: int = 256 @recipe_main async def sft_to_gspo(config: SFTtoGSPOConfig, ctx: RecipeContext): """Run SFT warm-up, then continue with GSPO for quality improvement.""" # --- Model setup --- # Both stages train the same model sequentially: SFT initializes it, # then GSPO continues from where SFT left off. We use spawn_train() # because we need gradient updates (spawn_inference is read-only). model_builder = await config.model_to_train.to_builder(ctx, tp=1, kv_cache_len=131072) model = await model_builder.spawn_train( "sft_then_gspo", max_batch_size=10_000, ) logger = get_prod_logger(ctx=ctx) # === STAGE 1: SFT warm-up === # SFT uses a Completion dataset — each sample contains a full conversation # with the expected assistant response. The model learns to reproduce these # demonstrations via supervised cross-entropy loss. sft_dataset = await config.sft_dataset.load(ctx) sft_trainer = SFT( dataset=sft_dataset, model=model, logger=logger, lr=config.sft_learning_rate, epochs=config.sft_epochs, # Usually 1-3; more risks overfitting samples_per_batch=config.sft_batch_size, max_grad_norm=1.0, weight_decay=0.01, ) await sft_trainer.run() # At this point, `model` has been updated in-place with SFT weights. # We continue training the same model instance in the next stage. # === STAGE 2: GSPO for quality improvement === # GSPO uses a Prompt dataset — each sample is a prompt only, no expected # output. The algorithm generates completions, grades them, and learns # from the reward signal. gspo_dataset = await config.gspo_dataset.load(ctx) # Spawn a separate model for judging. This uses spawn_inference() (not # spawn_train) because we only need forward passes — no gradients. judge_model_builder = await config.model_to_train.to_builder(ctx, tp=1, kv_cache_len=131072) judge_model = await judge_model_builder.spawn_inference("judge") # Set up a grader to score completions. BinaryJudgeGrader sends each # completion to the judge LLM, which scores it PASS (1.0) or FAIL (0.0). grader = BinaryJudgeGrader( grader_key="output-quality", criteria="Is the output high-quality, well-formed, and correct?", model=judge_model, ) # CombinedGrader aggregates one or more graders into a single score. # Even with one grader, the training algorithms expect a CombinedGrader. combined_grader = CombinedGrader( grader_key="combined", graders=[grader], ) gspo_trainer = GSPO( dataset=gspo_dataset, model=model, # Same model instance, now SFT-warm-started grader=combined_grader, logger=logger, lr=config.gspo_learning_rate, # Typically lower than SFT LR samples_per_batch=config.gspo_batch_size, samples_per_mini_batch=64, max_grad_norm=1.0, clip_range=0.2, kl_beta=0.1, max_num_gspo_steps=config.gspo_max_steps, completions_per_sample=4, # Generate 4 completions per prompt for ranking ) await gspo_trainer.run() # Save the final model — includes both SFT and GSPO training await model.save(model_name="sft-then-gspo-final", ctx=ctx) ``` ## SFT vs. RL: Complementary objectives **SFT (Supervised Fine-Tuning)** * Learns to imitate labeled demonstrations — anchored to the distribution of examples * Uses supervised loss (typically cross-entropy) * Converges quickly: few epochs on a small demonstration set (early stopping avoids overfitting) * Best for: bootstrapping output format, structure, or style **RL (e.g., GSPO, GRPO, PPO)** * Learns from feedback signals — grader scores, preference pairs, or other reward signals depending on the algorithm * Can discover behaviors beyond the demonstrations * Step-based; runs for many steps driven by online rollout generation, not dataset size * Best for: optimizing quality, refining output, and improving beyond what demonstrations capture Together, they form a complete pipeline: imitate first, then improve. ## When to skip SFT * **Base model already formats correctly**: If the vast majority of generations produce valid output, skip SFT and start with RL directly. * **No demonstration data available**: SFT requires labeled examples. If you only have unlabeled prompts, start with RL directly. * **Output is simple**: Plain text, single categories, or yes/no don't need SFT. Complex, multi-field structured output benefits most. * **Graders are lenient on format**: If graders score semantically rather than penalizing parsing errors, malformed output is less costly. ## Key takeaways 1. **SFT imitates, RL improves** — SFT is anchored to its demonstrations; RL optimizes beyond them through reward signals 2. **SFT is efficient** — A small demonstration set and few epochs converge quickly; stop early to avoid overfitting 3. **Dataset format depends on the algorithm** — SFT needs full conversations with expected outputs; GSPO and GRPO use prompt-only datasets; DPO uses preference pairs 4. **Consider skipping SFT** — If the base model already formats correctly, has no available demonstrations, or produces simple outputs, start directly with RL # Load datasets and StringThread Source: https://docs.adaptive-ml.com/v0.14/harmony/datasets How to load Adaptive datasets in your recipes You can import datasets previously [uploaded to Adaptive](/v0.14/core/datasets) directly within your recipes. Datasets stored on Adaptive can be loaded by specifying a parameter of type `Dataset` from `adaptive_harmony.parameters` in your recipe's `InputConfig` [class](/v0.14/harmony/config). You can then load it in your recipe as a list of `StringThread` objects by calling `await dataset.load(ctx)`. ## Load a dataset First, define your dataset in your recipe's input config: ```python theme={null} from adaptive_harmony.runtime import InputConfig from adaptive_harmony.parameters import Dataset class MyConfig(InputConfig): dataset: Dataset ``` To load a dataset from Adaptive, you can use the `load` method on the dataset: ```python theme={null} async def my_recipe(config: MyConfig, ctx: RecipeContext): dataset = await config.dataset.load(ctx) ``` This utility can also load local files structured in the Adaptive-supported [format](/v0.14/core/datasets), which you can leverage if you are testing a recipe [locally](/v0.14/harmony/harmony-client). Load a local dataset with `Dataset(dataset_key="local-file", local_file_path="your_file.jsonl")`. ## StringThread object The atomic element of any dataset in the `adaptive_harmony` codebase is a `StringThread`, which is a Rust backed object exposed in Python. A `StringThread` simply contains all the messages in a thread of conversation, along with any metadata associated with that thread (such as metric feedback, ground truth labels or any other metadata). `StringThread` exposes a few helpful methods: ```python theme={null} from adaptive_harmony import StringThread thread = StringThread( turns=[ ("user", "Hello, who are you?"), ("assistant", "I am a large language model. How can I help you today?"), ] ) thread_with_metadata = StringThread.with_metadata( turns=[ ("user", "Hello, who are you?"), ("assistant", "I am a large language model. How can I help you today?"), ], metadata={"label": "polite"} ) # Return all turns of thread as NamedTuples all_turns = thread.get_turns() all_turns[0][0] == all_turns[0].role # Returns a list of tuples, each containing a role and a message # Does not include the completion, if there is one (last turn with `assistant` role) messages = thread.messages() assert messages[-1][0] != "assistant" # Returns the string content of the last message in the thread if its role is `assistant` # Otherwise, returns None completion = thread.completion() new_thread = StringThread([]) new_thread = new_thread.system("You are a helpful bot.") thread = thread.user("My name is John.") # Returns a new thread with a new user message added to the end thread = thread.assistant("Nice to meet you, John!") # Returns a new thread with a new assistant message added to the end ``` ## Loading from Hugging Face You can also load datasets directly from Hugging Face in your recipe. `adaptive_harmony` exposes helper methods to convert arbitrary datasets into a list of `StringThread` objects by allowing you to specify the column in the original dataset that contains chat messages. ```python theme={null} from adaptive_harmony.core.dataset import load_from_hf, convert_sample_dict def load_hf_dataset(): # Helper function to convert HF dataset to Adaptive StringThread convert_sample_fn = convert_sample_dict( turns_key="messages", role_key="role", content_key="content" ) # Load the dataset dataset = load_from_hf( "HuggingFaceH4/ultrachat_200k", "train_sft", convert_sample_fn ) return dataset ``` # Create an evaluation recipe Source: https://docs.adaptive-ml.com/v0.14/harmony/eval Write your own recipe for model evaluation, and log eval artifacts to the product Writing an evaluation recipe requires three main components: 2. **A dataset** - Data to evaluate the models on 3. **Models to evaluate** - The models you want to assess 4. **Graders** - Functions that will score the model outputs An Evaluation recipe should produce an `EvaluationArtifact`, which will be populated with several `EvalSample` (evaluated model's completions with attached grades). When you visit the Adaptive UI, and `EvaluationArtifact` will show aggregate evaluation scores per model on all Graders selected/created in the evaluation. ## Evaluation recipe structure In the previous sections, we omit wrapping all methods in the main function decorated with @recipe\_main (see context [here](/v0.14/harmony/recipe-syntax)) for readability. Assume all the following code is in the main function. Also, check [Parametrize recipe inputs](/v0.14/harmony/config) to learn how to build the recipe `config` with datasets, models and graders we refer to in the next steps. ### 1. Load a dataset ```python theme={null} # Load from Adaptive dataset = await config.dataset.load(ctx) ``` ### 2. Spawn models and generate completions To evaluate your models, you'll generate a list of `EvalSampleInteraction` by spawning each model, running inference to obtain completions, and collecting the results. Use the `spawn_inference` method to create an inference instance for each model. After generating completions, deallocate the model to optimize memory usage. ```python theme={null} from adaptive_harmony import EvalSampleInteraction, Grade from adaptive_harmony.core.utils import batch_process_fallible # Register stages for progress tracking models_list = list(config.models_to_evaluate) stage_names = [f"generate_model_{i}" for i in range(len(models_list))] ctx.job.register_stages(stage_names) interactions: list[EvalSampleInteraction] = [] # models_to_evaluate is a Set[Model] for i, model_param in enumerate(models_list): model_builder = await model_param.to_builder(ctx) model = await model_builder.spawn_inference(model_param.model_key) # Generate completions with progress reporting result_pairs = await batch_process_fallible( model.generate, dataset, batch_size=100, stage_notifier=ctx.job.stage_notifier(stage_names[i]) ) completions = [result for idx, result in result_pairs] interactions.extend( [ EvalSampleInteraction( thread=thread, source=model_param.model_key, ) for thread in completions ] ) # remove the model from GPU await model.dealloc() ``` The [`batch_process_fallible`](/v0.14/harmony/async-utils) utility function concurrently processes all samples with automatic progress reporting and skips completions with errors (when for example the sample is too big for the model's maximum sequence length). ### 3. Spawn graders and grade completions To evaluate model outputs, we need to create/setup graders, and use them to grade each model's completions. Each Grader produces a [Grade](/v0.14/harmony/graders#grade-object) for each sample. ```python theme={null} from adaptive_harmony import EvalSample # Create graders from config graders = [await grader.load(ctx) for grader in config.graders] # Define eval samples with empty grades eval_samples = [ EvalSample( interaction=interaction, grades=[], dataset_key=config.dataset.dataset_key, ) for interaction in interactions ] # Register grading stages for progress tracking grading_stage_names = [f"grade_{i}" for i in range(len(graders))] ctx.job.register_stages(grading_stage_names) # Grade eval samples with progress reporting for i, grader in enumerate(graders): threads = [sample.interaction.thread for sample in eval_samples] grade_pairs = await batch_process_fallible( grader.grade, threads, batch_size=100, stage_notifier=ctx.job.stage_notifier(grading_stage_names[i]) ) # Create index-to-grade mapping for successful grades grade_map = {idx: grade for idx, grade in grade_pairs} # Attach grades to eval samples (skip failed ones) for i, eval_sample in enumerate(eval_samples): if i in grade_map: eval_sample.grades.append(grade_map[i]) ``` If you defined a [custom Recipe Grader](/v0.14/harmony/graders) in this eval recipe, you could add it to the `graders` list above, have its grades added to `eval_samples` and later shown in the UI. If the `grader_key` you hardcode for you recipe grader does not exist yet in the platform, it will be automatically created and reported in the evaluation results; no need to pre-create the recipe grader. ### 4. Output an EvaluationArtifact Now that all samples are graded, you can create an `EvaluationArtifact` and populate it with your `EvalSamples`. You do not need to push this to Adaptive with a specific method; the Adaptive Engine automatically detects that an `EvaluationArtifact` was written in the course of the recipe's execution, registering and displaying it in the UI once the run is complete. Read more about Artifacts in [Run output artifacts](/v0.14/harmony/artifacts). ```python theme={null} # Create evaluation artifact eval_artifact = EvaluationArtifact(name="Evaluation Artifact", ctx=ctx) eval_artifact.add_samples(eval_samples) ``` ## Complete example Here's a complete example combining all the concepts: ```python theme={null} from pydantic import Field from typing import Annotated from loguru import logger from adaptive_harmony import StringThread, EvalSample, EvalSampleInteraction, EvaluationArtifact from adaptive_harmony.runtime import InputConfig, RecipeContext, recipe_main from adaptive_harmony.parameters import Model, Dataset, Grader from adaptive_harmony.core.utils import batch_process_fallible class EvalConfig(InputConfig): dataset: Annotated[Dataset, Field(description="Dataset to evaluate on")] model_1: Annotated[Model, Field(description="First model to evaluate")] model_2: Annotated[Model, Field(description="Second model to evaluate")] tp_1: Annotated[int, Field(description="Tensor parallel degree for model 1", default=1)] tp_2: Annotated[int, Field(description="Tensor parallel degree for model 2", default=1)] graders: Annotated[ list[Grader], Field(description="List of graders to evaluate models with"), ] @recipe_main async def complete_evaluation_recipe(config: EvalConfig, ctx: RecipeContext): client = ctx.client # 1. Load dataset logger.info("Loading dataset...") dataset = await config.dataset.load(ctx) logger.info(f"Loaded {len(dataset)} samples") # 2. Spawn models and run completions logger.info("Running inference for both models...") # Register stages for progress tracking ctx.job.register_stages(["generate_model_1", "generate_model_2"]) model_1_builder = await config.model_1.to_builder(ctx, tp=config.tp_1) model_1 = await model_1_builder.spawn_inference("evaluated_model_1") model_2_builder = await config.model_2.to_builder(ctx, tp=config.tp_2) model_2 = await model_2_builder.spawn_inference("evaluated_model_2") # Generate completions with progress reporting stage_notifier_1 = ctx.job.stage_notifier("generate_model_1") result_pairs_1 = await batch_process_fallible( model_1.generate, dataset, batch_size=100, stage_notifier=stage_notifier_1 ) completions_1 = [result for idx, result in result_pairs_1] stage_notifier_2 = ctx.job.stage_notifier("generate_model_2") result_pairs_2 = await batch_process_fallible( model_2.generate, dataset, batch_size=100, stage_notifier=stage_notifier_2 ) completions_2 = [result for idx, result in result_pairs_2] logger.info(f"Generated {len(completions_1)} completions for model 1") logger.info(f"Generated {len(completions_2)} completions for model 2") # Deallocate models to free GPU resources await model_1.dealloc() await model_2.dealloc() # 3. Spawn graders and grade completions logger.info("Setting up graders...") graders = [await grader.load(ctx) for grader in config.graders] # Create interactions for both models interactions = [] for completion in completions_1: interactions.append(EvalSampleInteraction(thread=completion, source="model_1")) for completion in completions_2: interactions.append(EvalSampleInteraction(thread=completion, source="model_2")) # Create eval samples eval_samples = [ EvalSample( interaction=interaction, grades=[], dataset_key=config.dataset.dataset_key, ) for interaction in interactions ] # Register grading stages for progress tracking grading_stage_names = [f"grade_{i}" for i in range(len(graders))] ctx.job.register_stages(grading_stage_names) # Grade all samples with progress reporting for i, grader in enumerate(graders): logger.info(f"Running {grader.grader_key} grader...") stage_notifier = ctx.job.stage_notifier(grading_stage_names[i]) threads = [sample.interaction.thread for sample in eval_samples] grade_pairs = await batch_process_fallible( grader.grade, threads, batch_size=100, stage_notifier=stage_notifier ) # Create index-to-grade mapping for successful grades grade_map = {idx: grade for idx, grade in grade_pairs} # Attach grades to eval samples (skip failed ones) for i, eval_sample in enumerate(eval_samples): if i in grade_map: eval_sample.grades.append(grade_map[i]) # 4. Create evaluation artifact eval_artifact = EvaluationArtifact(name="Evaluation Artifact", ctx=ctx) eval_artifact.add_samples(eval_samples) logger.info(f"Evaluation completed! Created {len(eval_samples)} eval samples") ``` ## Key components explained ### Evaluation samples Each evaluation sample contains: * **Interaction**: The model's response and source model key * **Grades**: Grades from all graders * **Dataset ID**: Reference to the source dataset ### Graders Graders can be: * **AI Judge Graders**: Use LLMs to score responses * **Rule-based Graders**: Apply predefined criteria * **Combined Graders**: Aggregate multiple grader scores ### Evaluation artifact The `EvaluationArtifact` class: * Collects all evaluation results * Provides methods to analyze and export results * Integrates with the Adaptive platform for result tracking ## Best practices 1. **Model Management**: Always deallocate models after use to free resources 2. **Error Handling**: Use proper error handling for robust evaluation 3. **Progress Tracking**: Provide clear logging for debugging and monitoring 4. **Resource Cleanup**: Properly clean up graders and models # Custom recipe Graders Source: https://docs.adaptive-ml.com/v0.14/harmony/graders How to write custom recipe graders Graders are systems that evaluate generated completions based on any custom heuristic or criteria. Graders in `adaptive_harmony` are designed to be reusable for both evaluations and training - providing a reward for training, or a numerical score with an optional reason for evaluation. ## Define your own recipe grader To create a custom recipe grader, you must inherit from the `BaseGrader` class and implement the `grade` method. `__init__` is not mandatory, implement it only if you need custom state in your custom grader instances. Following is a simple example of a Grader that uses an LLM as a judge: ```python theme={null} from adaptive_harmony import StringThread, HarmonyClient, InferenceModel from adaptive_harmony.graders import BaseGrader, Grade class MyCustomGrader(BaseGrader): def __init__(self, grader_key: str, model: InferenceModel): """ Initialize the grader with parameters like model keys, API endpoints, or other configuration. """ super().__init__(grader_key) self.grader_key = grader_key self.model = model async def grade(self, sample: StringThread) -> Grade: """ The main function that returns the grade. Must return a `Grade` object. """ completion = sample.last_content() # you could judge the sample with the spawned LLM here # judge_thread: StringThread = format_completion_into_judge_prompt(completion) # self.model.generate(judge_prompt) score = 1 reason = "Custom scoring logic which could come from LLM" return Grade(value=score, grader_key=self.grader_key, reasoning=reason) ``` A simple string grader\_key is mandatory for any recipe grader you build; you will understand why when you read [Create an evaluation recipe](/v0.14/harmony/eval). The grader\_key is the logical name that is displayed in the UI to report evaluation scores for a grader that lives in your recipe. ## Quick implementation from a simple function Often times, you might want to implement a grader as a simple Python function; an example of this would be checking a substring in the generated completion against some ground truth label (this would be the case if you'd asked your model to predict some category in a structured way, for example `predicted_category`). A function such as the one described above would require no state. To make it easier to implement such a Grader without the boilerplate overhead, you can use the `BaseGrader.from_function()` method. The only requirement is that the function is asynchronous. Below is an example where: 1. a completion (which we expect to be a valid JSON object) is validated for adherence against a desired Pydantic Model, which includes reasoning and a category label prediction 2. the predicted category label is compared against the ground truth, which is attached in the sample's metadata 3. a reward is then returned as follows: `-1.0` if the model did not respect the output format, `0.0` if the format was respected but the predicted category was wrong, and `1.0` if both the format and the predicted label were correct ```python theme={null} from typing import Literal from pydantic import BaseModel from adaptive_harmony import StringThread from adaptive_harmony.graders import BaseGrader from adaptive_harmony.core.structured_output import pydantic_parse, OutputParserException class OutputExpectation(BaseModel): reasoning: str category: Literal["A", "B", "C", "D", "E"] async def grade_fn(sample: StringThread) -> float: try: structured_output = pydantic_parse(sample.last_content(), OutputExpectation) except OutputParserException: return -1.0 ground_truth = sample.metadata.get("category") if structured_output.category == ground_truth: return 1.0 else: return 0.0 # Creates a grader with default logging grader = BaseGrader.from_function("label-grader", grade_fn) ``` ## Grade object The `Grade` object contains: * **`value`**: The numerical score (float) * **`grader_key`**: Identifier for proper logging in the app for evaluations * **`reasoning`**: Optional field to provide a reasoning to back the score, which will be displayed as interaction metadata in the app if you save this grade in an `EvaluationArtifact` ```python theme={null} from adaptive_harmony import Grade grade = Grade( value=0.95, grader_key="my-custom-grader", reasoning="Response demonstrates excellent factual accuracy and relevance" ) ``` ## Combine grades All [trainer classes](/v0.14/harmony/training#trainers) take a single Grader as the reward function for training. This is because the underlying RL algorithms require a single scalar to optimize towards. When optimizing on multiple graders in a single training run, you must combine their rewards into a single float reward. You can trivially achieve this by using the `CombinedGrader`. You can also (optionally) weigh the contributions of different graders as desired. ```python Use the CombinedGrader theme={null} from adaptive_harmony.graders.combined_grader import CombinedGrader combined_grader = CombinedGrader( grader_key="combined", graders=[grader_1, grader_2, grader_3], weights=[1.0, 2.0, 1.0] ) # Use the combined scorer grade = await combined_grade.grade(thread) print(grade) # mean grade from all graders ``` ## Logging and aggregation Graders provide built-in logging to track metrics across all graded samples. This is useful for monitoring average rewards during RL training or analyzing grader behavior during evaluation. ### How logging works **Step 1: Log data in `grade()`** Use `self.add_log(dict)` to record information for each sample: ```python theme={null} async def grade(self, sample: StringThread) -> Grade: prediction = parse_output(sample.last_content()) score = compute_score(prediction) # Log data for this sample self.add_log({"score": score, "category": prediction.category}) return Grade(value=score, grader_key=self.grader_key) ``` **Step 2: Aggregate logs with `get_logs()`** Call `self.get_logs()` to get aggregated statistics. The default implementation computes mean, std, min, max, and count for any `"score"` keys logged: ```python theme={null} # After grading multiple samples logs = grader.get_logs(clear=True) # clear=True resets log buffer print(logs) # { # "score/mean": 0.85, # "score/std": 0.12, # "score/min": 0.5, # "score/max": 1.0, # "score/count": 100 # } ``` **Understanding `clear=True`:** When `clear=True`, the grader's internal log buffer (`self._logs`) is reset after computing aggregations. This ensures the next `get_logs()` call only aggregates logs added **since the last call**. ```python theme={null} # Training loop example for epoch in range(3): for batch in batches: for sample in batch: grade = await grader.grade(sample) # Logs accumulate # Get stats for this epoch only epoch_logs = grader.get_logs(clear=True) # Aggregates + clears buffer logger.log_metrics(epoch_logs, step=epoch) # Next epoch starts with empty buffer ``` Without `clear=True`, logs accumulate indefinitely, causing `get_logs()` to aggregate over **all samples ever graded**. Use `clear=True` when you want epoch-level or batch-level statistics. ### Custom logging with tables Override `get_logs()` to add custom metrics or create tables for detailed inspection. Tables are rendered as interactive HTML in W\&B/MLflow. **Example: Log malformed outputs in a table** ```python theme={null} from adaptive_harmony.logging_table import Table class MyGrader(BaseGrader): async def grade(self, sample: StringThread) -> Grade: try: prediction = pydantic_parse(sample.last_content(), MySchema) score = 1.0 if prediction.category == sample.metadata["category"] else 0.0 self.add_log({ "score": score, "valid_format": True, "completion": sample.last_content(), "predicted": prediction.category, "ground_truth": sample.metadata["category"] }) except OutputParserException: score = -1.0 self.add_log({ "score": score, "valid_format": False, "completion": sample.last_content(), "predicted": None, "ground_truth": sample.metadata["category"] }) return Grade(value=score, grader_key=self.grader_key) def get_logs(self, clear: bool = False) -> dict[str, float | Table]: # Get default statistics (mean, std, etc.) logs = super().get_logs(clear=False) # Filter logs for malformed samples malformed = [log for log in self._logs if not log.get("valid_format", True)] # Add custom metrics logs["score/malformed_count"] = len(malformed) logs["score/malformed_rate"] = len(malformed) / len(self._logs) if self._logs else 0 # Create a table of malformed samples if malformed: malformed_table = ( Table() .add_column("Completion", [log["completion"][:100] + "..." for log in malformed]) .add_column("Ground Truth", [log["ground_truth"] for log in malformed]) ) logs["score/malformed_samples"] = malformed_table if clear: self.clear_logs() return logs ``` ### Creating tables The `Table` class provides a fluent API for building tables: ```python theme={null} from adaptive_harmony.logging_table import Table # Build table column-by-column table = ( Table() .add_column("Category", ["A", "B", "C"]) .add_column("Score", [0.9, 0.7, 0.85]) .add_column("Reason", ["Good", "Needs work", "Excellent"]) ) # Or build row-by-row table = Table(initial_headers=["Category", "Score", "Reason"]) table.add_row(["A", 0.9, "Good"]) table.add_row(["B", 0.7, "Needs work"]) ``` Tables support strings and floats. They render as styled HTML in experiment tracking dashboards. ### Helper: `get_sample_tables()` The base class provides `get_sample_tables()` to quickly create tables of scored and failed samples: ```python theme={null} from adaptive_harmony.graders.utils import SuccessJudgeLog, FailedJudgeLog class MyGrader(BaseGrader): async def grade(self, sample: StringThread) -> Grade: try: score = compute_score(sample) self.add_log({ "score": score, "prompt": sample.get_turns()[0].content, "reasoning": "Some reasoning" }) return Grade(value=score, grader_key=self.grader_key) except Exception as e: self.add_log({ "error": str(e), "prompt": sample.get_turns()[0].content }) return Grade(value=0.0, grader_key=self.grader_key) def get_logs(self, clear: bool = False) -> dict[str, float | Table]: logs = super().get_logs(clear=False) # Separate successful and failed samples successful = [log for log in self._logs if "score" in log and "error" not in log] failed = [log for log in self._logs if "error" in log] # Use helper to create tables table_logs = self.get_sample_tables(successful, failed) logs.update(table_logs) if clear: self.clear_logs() return logs ``` This creates: * `score/scored_samples` - Table with Prompt, Reasoning, Score columns * `score/unscored_samples` - Table with Prompt, Error columns (if failures exist) * `score/scored_samples_count` - Count of successful samples * `score/unscored_samples_count` - Count of failed samples **Log format required:** * Successful logs: `{"score": float, "prompt": str, "reasoning": str (optional)}` * Failed logs: `{"error": str, "prompt": str (optional)}` ### When to use custom logging **Use default logging when:** * You only need aggregated statistics (mean, std, etc.) * Logging `{"score": value}` is sufficient **Use `get_sample_tables()` when:** * You want quick tables of scored/failed samples * Your logs match the expected format (prompt, reasoning, score, error) **Override `get_logs()` when:** * You want custom metrics (e.g., false positive rate, category-wise accuracy) * You need tables with custom columns or groupings * You want to track additional aggregations beyond score statistics # Harmony client and local testing Source: https://docs.adaptive-ml.com/v0.14/harmony/harmony-client Connect directly to Adaptive Engine locally with a Harmony client When you [start a run](/v0.14/core/recipes) for a recipe in Adaptive platform, that process is executed in the same infrastructure that hosts Adaptive Engine. This is the appropriate way to execute long running processes like training or evaluation recipes. However, when you first start writing a custom recipe or a [grader](/v0.14/harmony/graders), you might want to experiment and debug your logic locally before you neatly wrap the recipe in the appropriate [launch syntax](/v0.14/harmony/recipe-syntax) and [parametrize its inputs](/v0.14/harmony/config). Thankfully, `adaptive_harmony` allows you to establish a direct connection via secure websockets between your local environment and the compute plane of your Adaptive Engines' deployment. When you instantiate a `HarmonyClient`, a desired number of GPU's is allocated directly to you as an interactive session. These GPU's are freed to run other workloads or interactive sessions as soon as the local python process holding that client in memory is killed. ```python theme={null} import asyncio from adaptive_harmony import get_client client = asyncio.run( get_client( addr="wss://YOUR_ADAPTIVE_DEPLOYMENT_URL.com", num_gpus=2, api_key="YOUR_ADAPTIVE_API_KEY", use_case="my-use-case" # must exist in your Adaptive deployment ) ) ``` If you use `adaptive_harmony` in a Jupyter Notebook, you can directly `await` async methods like get\_client in a Jupyter cell (i.e. `await get_client()`), no need to use asyncio. A harmony client effectively serves as an LLM API backed by real compute resources. You can locally call methods on this client which in reality get executed in the Adaptive compute plane, and return python objects back to your environment as results. For example, if you run a `spawn` method to spawn a model on GPU, you'll get back a new handle in Python to a remote model, such as `TrainingModel` or `InferenceModel`, which you can also call methods on (such as `.generate()`, `.train_grpo()`, `.optim_step()`, etc.). This create a hybrid development environment, where you can step through python recipe code locally in your IDE, but have powerful compute resources execute the methods that require them. As you will see in [Recipe syntax](/v0.14/harmony/recipe-syntax), when you structure a custom recipe into the required format for execution in the Adaptive Engine, a client is created automatically at recipe launch and passed to the recipe in the `RecipeContext` (`ctx`). # Prerequisites and setup Source: https://docs.adaptive-ml.com/v0.14/harmony/install Installing the adaptive_harmony python library The only prerequisite to start writing custom recipes is to install `adaptive_harmony`, which is publicly hosted on [Pypi](https://pypi.org/). ```bash theme={null} pip install adaptive-harmony ``` This library allows you to get a [direct connection](/v0.14/harmony/harmony-client) from your local environment to your Adaptive Engine deployment's compute plane, enabling you to write recipes in your local environment and execute them remotely line-by-line seamlessly. # Log run metrics Source: https://docs.adaptive-ml.com/v0.14/harmony/logging Track training metrics with W&B, MLflow, TensorBoard, or console output ## Overview All training classes in `adaptive_harmony` support logging metrics to external tracking services. Metrics include training loss, validation scores, grader evaluations, generation statistics, and any custom metrics from callbacks. ## Available loggers Adaptive Harmony supports four logging backends: ### WandbLogger (Weights & Biases) Integration with [Weights & Biases](https://wandb.ai/) for experiment tracking and visualization. ```python theme={null} from adaptive_harmony.metric_logger import WandbLogger logger = WandbLogger( project_name="my-project", # W&B project name run_name="safety-training-v1", # Name for this specific run entity="my-team" # W&B team/username (optional) ) ``` **Installation:** ```bash theme={null} pip install adaptive-harmony[wandb] ``` **Features:** * Real-time metric visualization * Table logging for sample completions and evaluations * Hyperparameter tracking * Direct link to W\&B dashboard via `logger.training_monitoring_link` **Setup:** * Set your W\&B API key: `export WANDB_API_KEY=your_key_here` * Or login via `wandb login` ### MLFlowLogger Integration with [MLflow](https://mlflow.org/) for experiment tracking. ```python theme={null} from adaptive_harmony.metric_logger import MLFlowLogger logger = MLFlowLogger( project_name="my-project", # MLflow experiment name run_name="safety-training-v1", # Run name within experiment tracking_uri="http://mlflow:5000", # MLflow tracking server URI monitoring_link="http://mlflow:5000/experiments/1", # Optional dashboard link experiment_tags={"team": "research"}, # Optional experiment-level tags run_tags={"version": "v1.0"} # Optional run-level tags ) ``` **Installation:** ```bash theme={null} pip install adaptive-harmony[mlflow] ``` **Features:** * Metric and parameter logging * Table logging (exported as JSON artifacts) * Integration with MLflow UI * Support for custom tags and metadata **Setup:** * Deploy an MLflow tracking server (can be deployed alongside Adaptive, check the [Adaptive Helm Chart](https://github.com/adaptive-ml/adaptive-helm-chart/blob/main/charts/adaptive/values.yaml)) * Set `tracking_uri` to your server URL ### TBMetricsLogger (TensorBoard) Integration with [TensorBoard](https://www.tensorflow.org/tensorboard) for visualization. ```python theme={null} from adaptive_harmony.metric_logger import TBMetricsLogger logger = TBMetricsLogger( run_name="safety-training-v1", # Run identifier logging_dir="/path/to/tensorboard/logs", # Directory for TensorBoard logs monitoring_link="http://tensorboard:6006" # Optional TensorBoard server URL ) ``` **Installation:** ```bash theme={null} pip install adaptive-harmony[tensorboard] ``` **Features:** * Scalar metric tracking * Table logging (exported as HTML files with auto-generated index) * Text logging for string metrics **Setup:** * Specify a directory for logs * View with: `tensorboard --logdir=/path/to/tensorboard/logs` ### StdoutLogger Simple console output logger. Prints metrics to stdout using rich formatting. No external dependencies required. ```python theme={null} from adaptive_harmony.metric_logger import StdoutLogger logger = StdoutLogger() ``` **Features:** * Pretty-printed output to console * No setup required * Useful for local development and debugging ## Using globally configured logger An external logging solution can be deployed globally for all runs in an Adaptive deployment to use. This is accomplished with environment variables defined in the [Adaptive Helm Chart](https://github.com/adaptive-ml/adaptive-helm-chart/blob/main/charts/adaptive/values.yaml). The `get_prod_logger()` function automatically returns the appropriate logger based on your environment configuration, with an active run. Runs are automatically organized in a similar fashion as Adaptive runs, use-case first, then run name. This is the **recommended approach** for production recipes running on Adaptive. ```python theme={null} from adaptive_harmony.metric_logger import get_prod_logger logger = get_prod_logger() ``` **How it works:** The function checks for logger configuration in the following priority order: 1. **Weights & Biases** - If `WANDB_API_KEY` environment variable is set 2. **MLflow** - If `MLFLOW_TRACKING_URI` environment variable is set 3. **TensorBoard** - If `TENSORBOARD_LOGGING_DIR` environment variable is set 4. **StdoutLogger** - Fallback if none of the above are configured **Automatic configuration:** When Adaptive runs your recipe, it automatically sets environment variables for: * `HARMONY_JOB_ID` - Unique identifier for this run * `HARMONY_USE_CASE` - Use case name * `HARMONY_RECIPE_NAME` - Recipe name * `HARMONY_JOB_NAME` - Job name * Plus any logger-specific variables you've configured in your deployment **Example:** ```python theme={null} from adaptive_harmony.common.gspo import GSPO from adaptive_harmony.metric_logger import get_prod_logger logger = get_prod_logger() # Automatically selects configured logger await GSPO( dataset=train_dataset, model=policy_model, grader=safety_grader, logger=logger, max_num_gspo_steps=100, samples_per_batch=128, completions_per_sample=8 ).run() ``` ## State persistence across resumes Loggers expose `get_state()` and `set_state()` that the trainer uses to snapshot logger state into the checkpoint. On resume from a checkpoint, the step counter is restored — so a job that crashed at step 12,000 and resumes does not start logging from step 0. This works for `WandbLogger`, `MLFlowLogger`, `TBMetricsLogger`, `AdaptiveLogger`, `StdoutLogger`, and `CompositeLogger`. There is nothing to configure: it happens automatically as part of [Resuming from checkpoints](/v0.14/harmony/training#resuming-from-checkpoints). ## What gets logged Training classes automatically log: **Scalar metrics:** * `loss/train` - Training loss * `loss/grad_norm` - Gradient norm * `rewards/*` - Grader scores and statistics * `kl_divergence` - KL divergence from reference model (RL methods) * `learning_rate` - Current learning rate **From callbacks:** * `validation/rewards/*` - Validation grader scores (GraderEvalCallback) * `validation/loss` - Validation loss (ValidationLossCallback) * `generation/*` - Sample generation metrics (GenerateSamplesCallback) * Any custom metrics from your callbacks **Tables:** * Sample completions with prompts and responses * Grader evaluations with reasoning * Validation results **Hyperparameters:** * All trainer parameters (learning rate, batch size, etc.) * Model configuration * Recipe config parameters Training classes automatically link their logger's monitoring dashboard to Adaptive's progress reporting UI. When users view your run's progress in Adaptive, they can click through to view detailed metrics in the external logging solution you have set up globally for your Adaptive deployment. See [Progress reporting](/v0.14/harmony/progress) for more details. ## Logging custom metrics You can log custom metrics from your training code by calling the logger directly: ```python theme={null} logger({ "custom_metric/accuracy": 0.95, "custom_metric/f1_score": 0.87, "custom_metric/message": "Training progressing well" }) ``` Or from custom callbacks by returning a dictionary from the `callback` method (see [Training Callbacks](/v0.14/harmony/callbacks)). ## Best practices 1. **Use get\_prod\_logger() for recipes** - Let Adaptive handle logger selection and configuration automatically 2. **Use specific loggers for local development** - When developing locally, you might want to use a specific logger: ```python theme={null} logger = WandbLogger("dev-project", "local-test") ``` 3. **Configure one logger** - Only set environment variables for one logger when deploying Adaptive to avoid confusion 4. **Check monitoring links** - Access the dashboard URL via: ```python theme={null} print(f"View training at: {logger.training_monitoring_link}") ``` 5. **Close loggers when done** - Training classes handle this automatically, but if you're using loggers manually: ```python theme={null} try: # Training code pass finally: logger.close() # Ensure logs are flushed ``` # Load models Source: https://docs.adaptive-ml.com/v0.14/harmony/models Loading models in your recipes ## Loading a model To load a model in a custom recipe, use the `Model` class from `adaptive_harmony.parameters`. Call `await model.to_builder(ctx)` to get a `ModelBuilder` object, on which you can call several methods to configure how your model will be spawned. For example, use `builder.with_adapter()` to enable lightweight adapter training instead of full parameter fine-tuning (only use this if you are training the model). ```python theme={null} from adaptive_harmony.runtime import recipe_main, RecipeContext from adaptive_harmony.parameters import Model @recipe_main async def main(ctx: RecipeContext): # Full parameter Llama 3.1 8B policy_builder = await Model(model_key="llama-3.1-8b-instruct").to_builder( ctx, tp=1, kv_cache_len=100_000, tokens_to_generate=2048, ) # Adapter based Qwen 3 32B reward_builder = await Model(model_key="qwen3-32b").to_builder( ctx, tp=1, kv_cache_len=100_000, tokens_to_generate=2048, ) reward_builder = reward_builder.with_adapter() ``` ### Spawn methods ModelBuilder also exposes `spawn_train` and `spawn_inference` methods. Adaptive Engine unifies training and inference. Instead of requiring different frameworks/runtimes for training and inference, you can simply spawn models meant for training or inference with `spawn_train` and `spawn_inference`. A model spawned with `spawn_train` will require more GPU memory upfront, since Adaptive makes sure that enough memory is available at spawn time to fit the required `max_batch_size` during training (model activations, optimizer state, etc...). If you are not training a given model in your recipe (you are spawning a judge model for example), make sure to always spawn it with `spawn_inference` to reduce GPU memory pressure. The `max_batch_size` parameter defines the maximum number of tokens that can be allocated in a single training batch - i.e a mini batch that is processed by the model in parallel, the batch size corresponding to each optimization step is user-defined and independent from this parameter. It also limits the maximum sequence length that the model is able to train on. In the worst case scenario, for a dataset of samples with length =\~ to `max_batch_size`, the model will train on a single sample at a time. Any sequences larger than `max_batch_size` are simply dropped in the training classes, which also reconcile the desired optimization step batch size in # of samples. `spawn_train` returns a `TrainingModel`, and `spawn_inference` returns an `InferenceModel`; they are both async methods. ```python theme={null} from adaptive_harmony.runtime import recipe_main, RecipeContext from adaptive_harmony.parameters import Model @recipe_main async def main(ctx: RecipeContext): # Spawn for training train_builder = await Model(model_key="llama-3.1-8b-instruct").to_builder( ctx, tp=1, kv_cache_len=100_000, tokens_to_generate=2048, ) train_model = await train_builder.spawn_train(name="train_model", max_batch_size=4096) # Spawn for inference inference_builder = await Model(model_key="llama-3.1-8b-instruct").to_builder( ctx, tp=1, kv_cache_len=100_000, tokens_to_generate=2048, ) inference_model = await inference_builder.spawn_inference(name="inf_model") ``` ### Tensor parallelism (tp) Tensor parallelism (`tp`) determines how many GPUs a model is split across during execution. Choosing the right `tp` value depends on your model size and available hardware: larger models typically require higher `tp` to fit into memory, while smaller models may run efficiently with `tp=1`. Also, as explained above, a given model that fits on 2 GPUs with `tp=2` that was spawned with `inference_spawn` might not fit if it is spawned with `spawn_train`. Always ensure that the number you set for `tp` matches the number of devices you want to use and is supported by your infrastructure. ## Passing model parameters as config input In custom recipes, you can pass models in the recipe config using the `Model` class from `adaptive_harmony.parameters`. Adaptive will validate that the user-configured parameter for `model_to_train` below is a valid model key in Adaptive Engine. ### Using default deployment parameters When you call `await model.to_builder(ctx)`, the model is configured with the **default deployment parameters** as set in the Adaptive platform (KV cache length, tensor parallelism, tokens to generate). You can change these parameters globally for a model by visiting its model details page (click the model in the organizational model registry page), and editing the "Inference Configuration" setting on the right-hand menu. Often, inference defaults will not make sense for the more memory-intensive training regime, so any default can be overridden by passing parameters directly to `to_builder()`: **Overridable parameters:** * `tp` - Tensor parallelism (number of GPUs to split the model across) * `kv_cache_len` - KV cache length for the model * `tokens_to_generate` - Maximum tokens to generate per completion ```python theme={null} from typing import Annotated from adaptive_harmony.runtime import InputConfig, recipe_main, RecipeContext from adaptive_harmony.parameters import Model class MyConfig(InputConfig): model_to_train: Annotated[Model] tp: Annotated[int] max_seq_len: Annotated[int] train_adapter: Annotated[bool] @recipe_main async def main(config: MyConfig, ctx: RecipeContext): # Override default deployment parameters by passing them to to_builder() model_builder = await config.model_to_train.to_builder( ctx, tp=config.tp, # Override tensor parallelism kv_cache_len=100_000, # Override KV cache length tokens_to_generate=2048 # Override max tokens to generate ) if config.train_adapter: model_builder = model_builder.with_adapter() await model_builder.spawn_train("model_trained", config.max_seq_len) ``` ## Spawning in Jupyter notebooks When developing interactively in Jupyter notebooks, you can spawn models directly using `client.model()` without creating a full `RecipeContext`. This is convenient for quick experimentation and prototyping. First, create a client using `get_client`: ```python theme={null} from adaptive_harmony import get_client # Create a client directly client = await get_client( addr="wss://YOUR_ADAPTIVE_DEPLOYMENT_URL", num_gpus=2, api_key="YOUR_API_KEY", use_case="your-use-case", ) # Spawn a model using client.model() policy_builder = client.model( path="llama-3.1-8b-instruct", kv_cache_len=100_000, tokens_to_generate=2048, ).tp(1) policy_model = await policy_builder.spawn_train(name="policy", max_batch_size=4096) ``` Notice that, in recipe scripts, `ctx.client` provides access to the same client object. The `Model().to_builder()` approach shown above is the recommended pattern for recipe scripts as it integrates with the platform's inference parameters configuration system. # Overview Source: https://docs.adaptive-ml.com/v0.14/harmony/overview Build custom training and evaluation workflows Write custom Python recipes for training, evaluation, and other AI workflows using `adaptive_harmony`. ## Getting started Install adaptive-harmony Connect to compute plane for interactive development Structure and entry points Define recipe parameters ## Building recipes Load and spawn models Load datasets and StringThread Custom reward functions Load graders from Adaptive ## Training & evaluation SFT, PPO, GRPO, DPO algorithms Evaluate models and create artifacts JSON and constrained generation Track recipe progress in UI ## Quick reference Upload a recipe: ```python theme={null} adaptive.recipes.upload( path="/path/to/recipe.py", # or directory recipe_key="my-recipe", name="My Training Recipe" ) ``` When uploading a directory, the entry point must be `main.py` and all imports must be relative (e.g., `from .utils import helper`). Run a recipe: ```python theme={null} job = adaptive.jobs.run( recipe_key="my-recipe", num_gpus=2, args={"model": "llama-3.1-8b", "dataset": "my-data"} ) ``` # Load Graders Source: https://docs.adaptive-ml.com/v0.14/harmony/platform-graders Load existing Graders within your recipe You can load [Graders](/v0.14/core/graders) that have been previously created in the platform automatically using the `Grader` class from `adaptive_harmony.parameters` (excluding Recipe Graders, which only exist as code within your custom recipe). ## Include grader in InputConfig In order to do this, you must only make sure that one or more `Grader` parameters can be passed to your recipe as input: ```python theme={null} from pydantic import Field from typing import Annotated, Set from adaptive_harmony.runtime import InputConfig from adaptive_harmony.parameters import Grader class GraderRecipeConfig(InputConfig): graders: Annotated[ Set[Grader], Field( description="The graders to load.", title="Graders", ), ] ``` The `Grader` object will contain all the configuration necessary to instantiate a grader in code, with the exact same properties you've defined in the Adaptive platform. ## Load with Grader.load() You can then automatically load and setup your graders with `await grader.load(ctx)`: ```python theme={null} from adaptive_harmony.runtime import RecipeContext, recipe_main @recipe_main async def main(config: GraderRecipeConfig, ctx: RecipeContext): graders = [] for grader_param in config.graders: grader = await grader_param.load(ctx) graders.append(grader) ``` Now the Graders are ready to use for training, evaluation, or anything else you find use for! # Progress reporting Source: https://docs.adaptive-ml.com/v0.14/harmony/progress Report the progress of your recipe to Adaptive ## Overview Progress reporting allows you to track and display the progress of your recipe's execution in Adaptive Engine. This is useful for long-running recipes where you want to monitor how much work has been completed and how much remains. Progress reporting consists of two main components: 1. **Define stages**: Define logical phases of your recipe execution up-front 2. **Update progress**: Report completion status for each stage ## Defining stages Use `ctx.job.register_stages()` to define the stages of your recipe: ```python theme={null} # Register recipe stages from adaptive_harmony.runtime import recipe_main, RecipeContext @recipe_main async def main(ctx: RecipeContext): stages = ["Generating completions", "Evaluating completions"] ctx.job.register_stages(stages) ``` Stages are displayed in the order they are registered and represent logical phases of your recipe execution. ## Reporting progress ### Generic notifier Adaptive Engine provides a simple interface to report progress, which you can access via `RecipeContext` (generally shown as the `ctx` input parameter): ```python theme={null} # for training progress report stage = "Generating grades" total = 1000 current = 50 ctx.job.report_progress(stage, total, current) ``` This will update the Adaptive run details UI to show you are now at 5% completion on the "Generating grades" stage (50/1000 are done). ### Stage-specific notifier If you don't want to constantly pass a "stage" parameter, you can create a `StageNotifier`, which is a copy of `ctx.job` that holds a default stage. All[Training classes](/v0.14/harmony/training#trainers) take in an optional `stage_notifier` parameter, which - as the name indicates - must be a stage-bounded notifier. This means you can follow the progress of training in # training samples processed from the Adaptive UI without having to edit the Trainer classes. ```python theme={null} from adaptive_harmony.common import GRPO stage_notifier = ctx.job.stage_notifier(stage="GRPO Training") trainer = GRPO( ..., stage_notifier=stage_notifier, ... ) ``` ### Single event notifications Finally, it's frequent that you just want to report that something happened in your recipe, not necessarily that "# X out of a total # Y" samples/steps have been processed. A good example is reporting that a model has been spawned, which is a milestone in the recipe, but is a single action. For these cases, to decrease code verbosity you can use the `SimpleProgressNotifier` progress manager: ```python theme={null} from adaptive_harmony.runtime import SimpleProgressNotifier from adaptive_harmony.parameters import Model async with SimpleProgressNotifier(ctx.job.stage_notifier("Spawning judge model")): judge_builder = await Model(model_key="llama-3.3-70b").to_builder(ctx) judge = await judge_builder.spawn_inference("judge") ``` `SimpleProgressNotifier` works in both synchronous and asynchronous contexts. ## Linking training monitoring dashboards When using a metric logger (W\&B, MLflow, TensorBoard), you can link the logger's monitoring dashboard directly to your progress stages. This allows users to click through from the Adaptive UI to view detailed training metrics. ```python theme={null} from adaptive_harmony.common.grpo import GRPO from adaptive_harmony.metric_logger import get_prod_logger logger = get_prod_logger() # The logger's monitoring link is automatically attached to progress reporting trainer = GRPO( dataset=train_dataset, model=policy_model, grader=safety_grader, logger=logger, stage_notifier=ctx.job.stage_notifier("GRPO Training") ) await trainer.run() ``` All training classes automatically link their logger's `training_monitoring_link` to their stage notifier. This means when users view the run's progress in Adaptive, they can click a link to view detailed metrics in W\&B, MLflow, or TensorBoard. ### Manual monitoring link configuration You can also manually set or override monitoring links in two ways: from adaptive\_harmony.metric\_logger import get\_prod\_logger logger = get\_prod\_logger() **1. Set a global monitoring link for all reported stages of a job:** ```python theme={null} # Set monitoring link for the entire job (get it from your logger) ctx.job.set_monitoring_link(logger.training_monitoring_link) # All subsequent progress reports will use this link ctx.job.report_progress(stage="Training", total=100, current=50) ``` **2. Pass a monitoring link to a specific progress report:** ```python theme={null} # Override the monitoring link for a specific stage ctx.job.report_progress( stage="Training", total=100, current=50, monitoring_link=logger.training_monitoring_link ) ``` This is useful when: * You're not using a training class but still want to link to external dashboards * You want different stages to link to different monitoring dashboards * You're using a custom monitoring solution not covered by the built-in loggers See [Log job metrics](/v0.14/harmony/logging) to learn about configuring metric loggers and accessing monitoring dashboards. ## Visualizing progress on the Adaptive UI Once you have define your progress reporting strategy, uploaded and launched your recipe, the stages and progress will be displayed as below in the adaptive Engine: The example above illustrate the reporting defined with the code below: ```python theme={null} from adaptive_harmony.runtime import recipe_main, RecipeContext @recipe_main async def recipe(config: EvalConfig, ctx: RecipeContext): # load dataset... # spawn models... # Register recipe stages inference_stage, eval_stage = "Generating completions", "Evaluating completions" stages = [inference_stage, eval_stage] ctx.job.register_stages(stages) batch_size = 20 # batch method not implemented here inference_batches = batch(dataset, batch_size=batch_size) completions = [] for i, batch in inference_batches: completions.extend([model.generate(thread) for thread in batch]) ctx.job.report_progress(inference_stage, len(dataset), i * batch_size + len(batch)) eval_batches = batch(completions, batch_size=batch_size) grades = [] for i, batch in eval_batches: grades.extend([grader.grade(thread) for thread in batch]) ctx.job.report_progress( eval_stage, len(completions), i * batch_size + len(batch) ) print("Recipe finished") ``` # Recipe syntax Source: https://docs.adaptive-ml.com/v0.14/harmony/recipe-syntax Learn how to write a simple custom recipe In this page, you will learn how to write a simple custom recipe, from model loading to training. You can set up your development environment by installing `adaptive_harmony` first: ```bash theme={null} pip install adaptive-harmony ``` ## Step by step guide In this recipe, we train a model on completion safety feedback judged by Llama 3.3 70B as a grader, using GRPO. Custom recipes are written as single python files. You can store it anywhere you want in your codebase. Let's create a recipe `my_custom_recipe.py` ```bash theme={null} touch my_custom_recipe.py ``` Fill it with this recipe skeleton: ```python my_custom_recipe.py theme={null} from adaptive_harmony.runtime import recipe_main, RecipeContext @recipe_main async def main(ctx: RecipeContext): print("Hello, world!") ``` The decorator `@recipe_main` defines a single async function in the file as the main entrypoint that Adaptive Engine should run when the recipe is launched. This decorator is required in order to upload a recipe to Adaptive. When you first start writing a recipe, in order to more easily run and debug it locally, you can manually create a `RecipeContext` object. When you upload your recipe to Adaptive, the `RecipeContext` is automatically injected by the platform recipe runner, with the correct permissions and use case-related configuration. ```python my_custom_recipe.py theme={null} from adaptive_harmony.runtime import recipe_main, RecipeContext from harmony_client.runtime.context import RecipeConfig @recipe_main async def main(ctx: RecipeContext): print("Hello, world!") if __name__=="__main__": import asyncio async def run(): # Create RecipeContext manually for local testing config = RecipeConfig( harmony_url="wss://YOUR_ADAPTIVE_DEPLOYMENT_URL", num_gpus=2, api_key="YOUR_ADAPTIVE_API_KEY", use_case="your-use-case", # must exist in your Adaptive deployment ) ctx = await RecipeContext.from_config(config) await main(ctx) asyncio.run(run()) ``` Once you upload and run your recipe on the platform, your `main()` recipe entrypoint method is executed directly, so the final block used for local testing will not run. We begin by spawning the policy model using the `Model` parameter type. ```python theme={null} from adaptive_harmony.parameters import Model # Spawn model for training policy_builder = await Model(model_key="llama-3.2-1b").to_builder(ctx, tp=2) policy_model = await policy_builder.spawn_train(name="policy", max_batch_size=4096) ``` You can specify deployment parameters like tensor parallelism directly in the `to_builder()` call. We load a dataset from the Hugging Face Dataset Hub as an example. The helper functions facilitate converting a Hugging Face dataset to a list of `StringThread`, the format for chat messages + metadata used throughout Adaptive recipes. See [Loading datasets and StringThread](/v0.14/harmony/datasets) to find out how to load a dataset that has been uploaded to Adaptive. ```python theme={null} from adaptive_harmony import StringThread from adaptive_harmony.core.dataset import load_from_hf, convert_sample_dict # Helper function to convert the HF dataset to an Adaptive StringThread convert_sample_fn = convert_sample_dict(turns_key="messages", role_key="role", content_key="content") # Load the dataset dataset = load_from_hf("HuggingFaceH4/ultrachat_200k", "train_sft", convert_sample_fn) # Split into training and validation sets train_dataset = dataset[:900] validation_dataset = dataset[900:1000] ``` We then define the grader that will be used for feedback during training. The grader requires a spawned inference model. ```python theme={null} from adaptive_harmony.graders.binary_judge import BinaryJudgeGrader # Spawn a judge model for the grader judge_builder = await Model(model_key="llama-3.3-70b").to_builder(ctx, tp=2) judge_model = await judge_builder.spawn_inference(name="safety_judge") # Define a grader with the spawned model criteria = "The answer must not be harmful, offensive, or use inappropriate language" safety_grader = BinaryJudgeGrader( grader_key="safety", model=judge_model, criteria=criteria, ) ``` Finally, we pass our model, grader, and parameters to the GRPO trainer. We also add a `GraderEvalCallback` to monitor performance on the validation set during training. ```python theme={null} from adaptive_harmony.metric_logger import WandbLogger from adaptive_harmony.common.grpo import GRPO from adaptive_harmony.common.callbacks import GraderEvalCallback # Define a logger logger = WandbLogger("safety_grpo", "my_first_custom_recipe", "adaptive-ml") # Create a validation callback to monitor training progress validation_callback = GraderEvalCallback( validation_set=validation_dataset, model=policy_model, grader=safety_grader, frequency=0.1 # Evaluate every 10% of training ) # Run GRPO training await GRPO( dataset=train_dataset, model=policy_model, grader=safety_grader, logger=logger, callbacks=[validation_callback], max_num_grpo_steps=100, samples_per_batch=512, samples_per_mini_batch=64, completions_per_sample=4, kl_beta=0.01, ).run() ``` See [Training Callbacks](/v0.14/harmony/callbacks) for more information about running arbitrary code during, including checkpoint saving, sample generation, and validation loss tracking. ## Full recipe ```python my_custom_recipe.py theme={null} from adaptive_harmony.runtime import recipe_main, RecipeContext from adaptive_harmony.parameters import Model from adaptive_harmony.core.dataset import load_from_hf, convert_sample_dict from adaptive_harmony.graders.binary_judge import BinaryJudgeGrader from adaptive_harmony.metric_logger import WandbLogger from adaptive_harmony.common.grpo import GRPO from adaptive_harmony.common.callbacks import GraderEvalCallback @recipe_main async def main(ctx: RecipeContext): # Spawn policy model policy_builder = await Model(model_key="llama-3.2-1b").to_builder(ctx, tp=2) policy_model = await policy_builder.spawn_train(name="policy", max_batch_size=4096) # Helper function to convert the HF dataset to an Adaptive StringThread convert_sample_fn = convert_sample_dict(turns_key="messages", role_key="role", content_key="content") # Load the dataset and split into train/validation dataset = load_from_hf("HuggingFaceH4/ultrachat_200k", "train_sft", convert_sample_fn) train_dataset = dataset[:900] validation_dataset = dataset[900:1000] # Spawn judge model for grader judge_builder = await Model(model_key="llama-3.3-70b").to_builder(ctx, tp=2) judge_model = await judge_builder.spawn_inference(name="safety_judge") # Define a grader with the spawned model criteria = "Evaluate if the answer is harmful, offensive, or use inappropriate language" safety_grader = BinaryJudgeGrader( grader_key="safety", model=judge_model, criteria=criteria, ) # Define a logger logger = WandbLogger("safety_grpo", "my_first_custom_recipe", "adaptive-ml") # Create a validation callback validation_callback = GraderEvalCallback( validation_set=validation_dataset, model=policy_model, grader=safety_grader, frequency=0.1 ) # Run GRPO training await GRPO( dataset=train_dataset, model=policy_model, grader=safety_grader, logger=logger, callbacks=[validation_callback], max_num_grpo_steps=100, samples_per_batch=512, samples_per_mini_batch=64, completions_per_sample=4, kl_beta=0.01, ).run() ``` # Release notes Source: https://docs.adaptive-ml.com/v0.14/harmony/release-notes Release notes for adaptive-harmony Depends on `harmony-client~=1.1`. ### Breaking changes **Training length must be specified explicitly** PPO, GRPO, GSPO, and ENVPPO now require exactly one of `epochs` or the corresponding `max_num_*_steps` to be set. Recipes that previously relied on a default training length raise `AssertionError`. ```python theme={null} # Before await GRPO(dataset=dataset, model=model, grader=grader).run() # After — pick one await GRPO(dataset=dataset, model=model, grader=grader, epochs=1.0).run() await GRPO(dataset=dataset, model=model, grader=grader, max_num_grpo_steps=100).run() ``` `epochs` is a float counting dataset passes; `max_num_*_steps` is an integer counting optimizer steps. **Environments must return a list of grades** Returning a single `Grade` instead of `list[Grade]` from `react_to()` now raises `ValueError`. Wrap single grades in a list. ```python theme={null} # Before return next_thread, Grade(value=1.0, grader_key="env") # After return next_thread, [Grade(value=1.0, grader_key="env")] ``` **Removed `batch_process_fallible`** Use `async_map_batch` instead. `async_map_batch` works on an iterator, processes a fixed-size batch concurrently, and refills the batch from the iterator when samples fail. **WandB runs now keyed by project ID** `WandbLogger` initialised by `get_prod_logger()` now passes the project ID (a stable identifier) as the WandB project, instead of the human-readable project name. Existing WandB dashboards filtering on the old project name lose continuity. Update filters to use the project ID, or set `project_name` explicitly when instantiating `WandbLogger` directly. ### What's new **Structured output via `response_model`** `InferenceModel.generate()` and `InferenceModel.generate_tokens()` now accept a Pydantic model class. They return a `(result, parsed)` tuple where `parsed` is an instance of your model. ```python theme={null} from pydantic import BaseModel class Verdict(BaseModel): reasoning: str pass_: bool result, parsed = await model.generate(thread, Verdict) print(parsed.pass_) ``` This wraps `constrained_decoding` and Pydantic parsing, so JSON output is guaranteed to validate against the schema. JSON Schemas are emitted with `additionalProperties: false` for compatibility with OpenAI strict mode. **Function graders loadable from source code** `BaseGrader.from_source_code(grader_key, source_code)` compiles a string of Python that defines an async `grade` function and returns a grader. `BaseGrader.from_config()` now handles `Function` grader entries from the platform, calling `from_source_code` under the hood. Function graders only run inside sandboxed recipe jobs submitted via the platform — outside the sandbox the source code is not available and `from_config` raises a descriptive error. **Train by epochs in RL recipes** `PPO`, `GRPO`, `GSPO`, and `ENVPPO` accept `epochs: float | None` to specify training length in dataset passes. Pairs with the breaking change above (exactly one of `epochs` or `max_num_*_steps` must be set). ```python theme={null} await GRPO( dataset=prompts, model=policy_model, grader=grader, epochs=2.0, # two passes over the dataset ).run() ``` **DPO with arbitrary loss functions** A new `train_dpo_tangents` joins `train_sft_tangents` and `train_grpo_tangents` for tangent-based DPO training. ```python theme={null} from adaptive_harmony.core.losses import train_dpo_tangents metrics = await train_dpo_tangents( model, pos_thread, neg_thread, ref_logprobs_pos, ref_logprobs_neg, beta=0.1, ) ``` See the [Bring your own loss functions](/v0.14/harmony/algorithms#bring-your-own-loss-functions) section for the broader pattern. **Logger step persisted across resumes** All loggers (`WandbLogger`, `MLFlowLogger`, `TBMetricsLogger`, `AdaptiveLogger`, `StdoutLogger`, `CompositeLogger`) now expose `get_state()` / `set_state()`. The trainer saves logger state into the checkpoint and restores it on resume, so step counters no longer reset to 0 mid-run. **Multi-stage checkpointing** Recipes that chain multiple trainers (for example, SFT followed by GSPO) now checkpoint each stage independently. When a job is resumed, every stage picks up where it left off — earlier stages skip, the active stage restarts from its latest checkpoint, later stages run fresh. No code changes required; the runtime tracks the stage counter for you. **Concurrency control on `async_map`** `async_map` and `async_map_fallible` accept `max_concurrent_samples: int | None = None` to bound parallelism with a semaphore, and `stage_notifier: StageNotifier | None = None` to report per-sample progress to the platform. ```python theme={null} results = await async_map( grader.score_float_value, samples, max_concurrent_samples=8, stage_notifier=ctx.job.stage_notifier("Grading"), ) ``` `async_map_fallible` adds a `return_indices=True` overload that returns `list[tuple[int, T]]`, so you can map results back to the original input indices. **Rich progress bars for model spawning** `spawn_train()` and `spawn_inference()` now render progress with `rich.progress.Progress` instead of `tqdm`. Visible behaviour change in interactive sessions. ### Bug fixes * **OpenAI strict-mode schemas now validate.** `additionalProperties: false` is added recursively to all object schemas, including those reached via `$defs`, `definitions`, `anyOf`/`oneOf`/`allOf`, and `items`. * **Repeated samples in GRPO/GSPO deepcopy metadata.** Sample metadata is no longer shared across grouped completions, preventing accidental mutation across the group. * **Thread weighting in dataset loading.** SFT and RLHF recipes had a bug where some assistant turns received incorrect weights. Loading and recipe weighting now agree. * **MLflow experiment tag for project lookup.** The MLflow run tag used for cross-system project lookup was wrong; it now matches what the platform expects. * **Improved user-error messages in trainer init.** Hyperparameter mismatches and missing inputs raise more specific messages instead of opaque assertion errors. Depends on `harmony-client~=1.0`. ### Breaking changes **"Use Cases" renamed to "Projects"** Anywhere `use_case` was used is now converted to `project`, namely in `get_client` and `RecipeConfig`. **Environment API rewrite** `Environment.react_to()` now returns a StringThread and a list of Grades, rather than a single next turn and a single reward float. ```python theme={null} # Before (v0.1.x) async def react_to(self, thread: StringThread) -> list[tuple[str, str]] | TrajectoryScore: ... # After (v1.0.0) async def react_to(self, thread: StringThread) -> tuple[StringThread | None, list[Grade] | None]: ... ``` * Return `(next_thread, grades)` on each turn. Return `(None, grades)` to terminate the trajectory. * When `next_thread` is not a prefix of the current thread, the environment starts a new conversation segment within the same trajectory. This enables multi-turn training with arbitrary state transitions. `Environment.bootstrap_prompt()` is renamed to `Environment.initialize_state()`. **IgnoreScoreException renamed to IgnoreGradeException** All grader code that catches or raises this exception must update the class name. ```python theme={null} # Before from adaptive_harmony.graders import IgnoreScoreException # After from adaptive_harmony.graders import IgnoreGradeException ``` The exception now carries a detailed default message explaining when ignoring a grade is appropriate. The `message` parameter type changed from `str` to `str | None` (default `None`). **CheckpointCallback deprecated** `CheckpointCallback` is deprecated. Use trainer-level checkpointing (`checkpoint_frequency` parameter) instead. Using both may produce inconsistent checkpoint states. **Rich progress display off by default** The training progress display is now off by default. Set environment variable `ENABLE_RICH_PROGRESS=1` to enable it. The previous `DISABLE_RICH_PROGRESS` environment variable is no longer recognized. **Experiment tag key renamed** The metric logger tag `adaptive.use_case_id` is renamed to `adaptive.project_id`. Update any code or dashboards that filter on this tag. ### What's new **Arbitrary state transition support in environment-based trainers** Environment-based trainers (PPO, GRPO, GSPO) can now handle environments that produce multiple conversation threads per trajectory. Future rewards are "propagated" back, with each turn's advantage derived from the (normalized) future cumulative reward as described in section 4.1.3 of [DeepSeekMath](https://arxiv.org/pdf/2402.03300). **ENVPPO: multi-turn PPO** A new trainer at `adaptive_harmony.common.env_ppo.ENVPPO` supports PPO training within multi-turn environments. It uses a separate value model for advantage estimation via GAE (Generalized Advantage Estimation), with independent learning rates and schedulers for the policy and value models. ```python theme={null} from adaptive_harmony.common.env_ppo import ENVPPO await ENVPPO( dataset=prompts, model=policy_model, value_model=value_model, environment_factory=env_factory, lr_policy=0.75e-6, lr_value=1e-6, max_num_ppo_steps=100, gae_lambda=0.95, gae_gamma=1.0, value_only_fraction=0.25, ).run() ``` Key parameters: * **`value_only_fraction`**: Fraction of training steps where only the value model updates (default: `0.25`). The policy model stays frozen during this warmup. * **`lr_policy`** / **`lr_value`**: Independent learning rates for the policy and value models. * **`gae_lambda`** / **`gae_gamma`**: GAE parameters for advantage estimation (defaults: `0.95` and `1.0`). **Tangent-based loss computation** SFT, GRPO, ENVGRPO, and ENVGSPO recipes now support `use_tangents=True`, which computes gradients through an explicit loss function (`adaptive_harmony.core.losses`) instead of relying on built-in training methods. This exposes per-step training metrics. ```python theme={null} from adaptive_harmony.common.sft import SFT await SFT( dataset=dataset, model=model, use_tangents=True, logger=logger, ).run() ``` When enabled, the logger receives additional `policy/*` metrics: * **SFT**: `policy/loss`, `policy/num_tokens` * **GRPO/ENVGRPO/ENVGSPO**: `policy/pg_clipfrac`, `policy/inner_kl_div`, `policy/Dkl` **Dataset seeding** All training recipes now accept a `data_seed` parameter (default: `42`) that controls dataset shuffling. This ensures reproducible training runs across restarts. ```python theme={null} await GRPO( dataset=dataset, model=model, grader=grader, data_seed=42, # Reproducible shuffle ).run() ``` **Optional system prompt in TemplatedPromptJudge** The `system_template` parameter in `TemplatedPromptJudge` now accepts `None`. When `None`, the judge thread contains only a user turn. **Dataset weight validation for SFT** SFT training now checks dataset turn weights before training starts and logs warnings for: * User turns with non-zero weight (you may be unintentionally training on user messages) * Assistant turns with zero weight (these turns do not contribute to training) **System prompt override utility** A new `override_system_prompt()` function replaces or adds a system prompt on every thread in a dataset: ```python theme={null} from adaptive_harmony.core.utils import override_system_prompt dataset = override_system_prompt(dataset, "You are a helpful assistant.") ``` **Image utilities** New functions in `adaptive_harmony.core.image_utils`: * `pil_to_base64()`: converts a PIL image to a base64 string with optional resizing and grayscale conversion * `image_to_base64()`: converts an image file path to a base64 string **Weight-aware display** `StringThread` and `TokenizedThread` repr output now highlights trained turns (weight > 0) in green and untrained turns in blue. The HTML export (`string_thread_to_html`) applies matching color coding and green borders on images in trained turns. **Improved error messages** * `async_map_batch` now raises a `RuntimeError` with a clear message when the batch failure rate exceeds the threshold, including the allowed failure percentage and the last error thrown. * `async_map_fallible` now logs exceptions before suppressing them. ### Bug fixes * **External dataset feedback values are no longer silently dropped.** When parsing datasets with a `feedbacks` field and a configured `feedback_key`, the reward value is now correctly extracted into the thread metadata. * **Progress percentage no longer exceeds 100%.** When `async_map_batch` retries failed samples, the processed count could exceed the total. Progress reporting now clamps to the total and logs a warning. * **Failed trajectories in ENVGRPO/ENVGSPO no longer discard the entire group.** Trajectory generation within a group switched from `async_map` to `async_map_batch`. If a trajectory fails, a new one is attempted instead of losing all completions for that prompt. Group normalization runs on the successful trajectories. # Threads Source: https://docs.adaptive-ml.com/v0.14/harmony/string-thread The core data structure for training and inference in Harmony recipes `StringThread` is the atomic data element in `adaptive_harmony`. A thread is a sequence of turns (role + content), combined with turn weights for training and optional metadata (metric feedback, ground truth labels, or any custom key-value pairs). ## Create a StringThread ```python theme={null} from adaptive_harmony import StringThread thread = StringThread( turns=[ ("user", "Hello, who are you?"), ("assistant", "I am a large language model. How can I help you today?"), ] ) thread_with_metadata = StringThread( turns=[ ("user", "Hello, who are you?"), ("assistant", "I am a large language model. How can I help you today?"), ], metadata={"label": "polite"} ) ``` ### Builder methods Each method returns a new `StringThread` with the turn appended: ```python theme={null} thread = StringThread([]) thread = thread.system("Your name is Adaptive.") thread = thread.user("Hello there!") thread = thread.assistant("Hello, I'm Adaptive, and here to help you!") thread = thread.tool('{"result": "success"}') ``` ## Access turns and content ```python theme={null} # All turns as (role, str content) tuples all_turns = thread.get_turns() # All turns except the last assistant turn messages = thread.messages() # The last assistant turn's content, or None completion = thread.completion() # Content of the very last turn (any role) last = thread.last_content() ``` `get_turns()` returns every turn as a `(role, content)` tuple. For multimodal turns, images are represented as `<|image|>` in the string content. `messages()` returns all turns except the final one if it has the `assistant` role. This is useful when you need to split a thread into prompt and completion. ## Multimodal StringThread The difference with text-only is that `content` becomes a list of fragments instead of a plain string. There are two fragment types: | Type | Content key | Example | | --------------- | ----------- | ------------------------------------------------------- | | `TextFragment` | `text` | `{"type": "text", "text": "Describe this image."}` | | `ImageFragment` | `url` | `{"type": "image", "url": "data:image/png;base64,..."}` | Use `StringThread.from_fragments()` to create a multimodal thread. Don't forget the `await`! `from_fragments` is async because it loads and decodes images. ```python theme={null} from adaptive_harmony import StringThread, TextFragment, ImageFragment thread = await StringThread.from_fragments([ ("user", [ TextFragment(text="What's in this image?"), ImageFragment(url=f"data:image/png;base64,{image_data}"), ]), ]) ``` You can also pass fragments as plain dictionaries: ```python theme={null} thread = await StringThread.from_fragments([ ("user", [ {"type": "text", "text": "What's in this image?"}, {"type": "image", "url": f"data:image/png;base64,{image_data}"}, ]), ]) ``` A text-only `StringThread` is equivalent to a fragment thread with a single `TextFragment`: ```python theme={null} # These two are equivalent thread = StringThread([("user", "Hello world!")]) thread = await StringThread.from_fragments([ ("user", [{"type": "text", "text": "Hello world!"}]) ]) ``` Only `user` and `system` roles can contain images. The `assistant` role must be text-only. The fragment format in Harmony differs from the [chat completions API](/v0.14/core/inference). In Harmony, image fragments use `{"type": "image", "url": "..."}`, while the chat completions API uses `{"type": "image_url", "image_url": {"url": "..."}}`. ### Image encoding `ImageFragment` expects a `url` field with the full data URI. To base64-encode a local image, use the built-in helper: ```python theme={null} from adaptive_harmony.core.image_utils import image_to_base64 b64 = image_to_base64("photo.png") fragment = ImageFragment(url=f"data:image/png;base64,{b64}") ``` `image_to_base64` returns the raw base64 string (without the `data:...` prefix). It also allows you to resize images and convert to grayscale: | Parameter | Type | Description | | ----------------------- | ----------- | ------------------------------------------ | | `image_path` | str or Path | Path to the image file | | `format` | str | Output format (default: `"PNG"`) | | `longest_side_max_size` | int or None | Resize so the longest side fits this limit | | `black_and_white` | bool | Convert to grayscale (default: False) | ### Supported image formats The formats accepted depend on the context: * **In recipes** (`adaptive_harmony`): most image formats are supported (PNG, JPEG, GIF, WebP, BMP, TIFF, etc.). Images can be loaded from file paths, HTTP URLs, or `data:` URIs. * **Via the chat completions API** (SDK / OpenAI client): only **PNG, JPEG, GIF, and WebP** are accepted, and only `data:` URIs, HTTP URLs are rejected. We recommend using **PNG or JPEG** for maximum compatibility. ## Turn weighting During training, turn weights control how much each turn contributes to the loss. A weight of `0.0` means the model does not learn from that turn, while `1.0` means it contributes fully. This is how you tell the model which parts of a conversation to learn from: typically you want the model to learn from assistant responses, not from user prompts or system messages. By default, turns added with `.assistant()` get a weight of `1.0` and all other roles get `0.0`. When you [load a dataset](/v0.14/harmony/datasets) that contains completions, `with_weight_last_assistant_turn()` is applied automatically: only the final assistant turn is weighted. You can override this after loading using one of the methods below. ### Weighting methods Each method returns a new `StringThread` with updated weights: | Method | Behavior | | ------------------------------------------- | ------------------------------------------------------------------- | | `with_weight(w)` | Set weight `w` on all turns | | `with_weight_all_assistant_turns()` | Weight 1.0 on all assistant turns, 0.0 on others | | `with_weight_last_assistant_turn()` | Weight 1.0 on the last assistant turn only, 0.0 on others | | `with_weight_assistant_turns_from_index(i)` | Weight 1.0 on assistant turns starting from the i-th assistant turn | ```python theme={null} # Train on all assistant responses in a multi-turn conversation thread = thread.with_weight_all_assistant_turns() # Train only on the final assistant response thread = thread.with_weight_last_assistant_turn() # Train on assistant turns starting from the 2nd one (index 1) thread = thread.with_weight_assistant_turns_from_index(1) ``` ```python theme={null} # Override default weighting after loading a dataset dataset = await config.dataset.load(ctx) dataset = [thread.with_weight_all_assistant_turns() for thread in dataset] ``` ### Inspect weights ```python theme={null} # Returns list of weight for each turn turns_with_weights = thread.get_turn_weights() ``` # Structured generation Source: https://docs.adaptive-ml.com/v0.14/harmony/structured-output Generate text with Adaptive models that follows a desired JSON schema The `adaptive_harmony` library includes methods to help you generate structured outputs with LLMs that adhere to a specific JSON schema, as well as to render simplified, LLM-readable JSON schemas you can prompt your LLM with to explain what output structure to follow. You can achieve both by using annotated Pydantic Models such as the following: ```python theme={null} from typing import Literal from pydantic import BaseModel, Field class BinaryJudgeOutput(BaseModel): reasoning: str = Field( description="Reasoning to support the rationale behind the score." ) grade: Literal["PASS", "FAIL"] = Field( description="The grade for the sample." ) ``` The benefit of depending on Pydantic models is that the model definition becomes the single source of truth across the output structure instructions in a prompt, the parsing of a text response back into that model, and referring to known properties in the object that results from parsing and type validation. ### Instruct LLM to follow desired output structure Expanding on the example above, `BinaryJudgeOutput` is the response structure we would expect from an LLM judge that classifies completions as "PASS" or "FAIL" according to some user-defined eval criteria. A simple system prompt we could use for this judge would be something like: > You are an evaluator of human to AI interactions.
> > You will be given a full interaction between a human and an AI model, as well as an evaluation criterion.
> > Your task is to evaluate the AI's response against the criterion. If the response respects and complies with the criterion, you must grade it with a "PASS", otherwise you must grade it with a "FAIL".
> You must reason about the interaction and whether it respects the criterion in a short paragraph before you decide on the final grade.
> You must return your output as a valid JSON string that strictly adheres to the following schema, with no preamble or postamble:

> `{json_schema}` ### Render simplified JSON schema You could then create a simplified json schema from your model definition: ```python theme={null} from adaptive_harmony.core.structured_output import render_schema schema_str = render_schema(BinaryJudgeOutput) print(schema_str) ``` Output: ``` { "reasoning": str, "grade": Literal["PASS", "FAIL"] } reasoning: Reasoning to support the rationale behind the score. grade: The grade for the sample. ``` ## Generate with a response model For Pydantic-validated output without prompt engineering, pass the model class directly to `generate()`. The schema is enforced via constrained decoding and the result is parsed into your model. ```python theme={null} from adaptive_harmony import InferenceModel, StringThread async def main(): model: InferenceModel thread: StringThread result, parsed = await model.generate(thread, BinaryJudgeOutput) print(parsed.grade) # "PASS" or "FAIL" print(parsed.reasoning) ``` `generate(thread, response_model)` returns a `(StringThread, BaseModel)` tuple — the first item is the model's response thread, the second is the validated Pydantic instance. `generate_tokens(thread, response_model)` works the same way and returns a `(TokenizedThread, BaseModel)` tuple, which is what you want for RL training where you need raw tokens for log-probabilities. JSON Schemas emitted to the underlying engine include `additionalProperties: false` on every object schema, so output is compatible with strict-mode JSON validators (e.g., OpenAI's strict mode). ## Generate text and validate Pydantic model `generate_and_validate` is the older path. It instructs the LLM via prompt and validates the response after generation, with one automatic retry on parse failure. Prefer `generate(thread, MyModel)` above for new code; use `generate_and_validate` when you cannot use constrained decoding (e.g., when targeting a model that does not support it). ```python theme={null} from adaptive_harmony import HarmonyClient, InferenceModel, StringThread from adaptive_harmony.core.utils import stringify_thread from adaptive_harmony.core.structured_output import JsonParseError async def main(): # instantiate this before client: HarmonyClient # spawn an inference model before with client.model().spawn_inference() model: InferenceModel # original interaction being evaluated original_thread: StringThread # system prompt template from above system_prompt_template: str # convert original thread into a string representation stringified_original_thread = stringify_thread(original_thread) # build a thread to prompt the judge LLM with judging_thread = ( StringThread .system(system_prompt_template.format()) .user(f"INTERACTION TO EVALUATE:\n{stringified_original_thread}") ) # returns both the generated text and the validate pydantic model try: output_text, output_pydantic_model = await model.generate_and_validate( thread=judging_thread, pydantic_model=BinaryJudgeOutput ) except JsonParseError: print("Model failed to output valid structure") ``` By default, `.generate_and_validate()` retries generation once with correction instructions if the LLM fails to comply with the specified format. You can control how many retries are attempted by passing `max_parsing_retries` to the method. # Create a training recipe Source: https://docs.adaptive-ml.com/v0.14/harmony/training How to write your own recipe for model training Writing a training recipe requires three main components: 1. **A model to train** - the base model you want to fine-tune 2. **A training dataset** - data to train the model on 3. **Graders** - for RL methods, the functions that will provide a reward to the model ## Training recipe structure ### 1. Create an input configuration First, you must create an input configuration that will hold all the parameters you want to allow to be passed to your recipe. ```python theme={null} from typing import Annotated, List, Set from pydantic import Field from adaptive_harmony.runtime import InputConfig from adaptive_harmony.parameters import Model, Dataset, Grader, model_kinds class TrainingConfig(InputConfig): model_to_train: Annotated[ Model[model_kinds.Trainable], # Restrict to trainable models only Field( description="Base model to fine-tune", title="Model to Train", ), ] output_model_key: Annotated[ str, Field( description="New key for the resulting fine-tuned model", title="Output Model Key", ), ] dataset: Annotated[ Dataset, Field( description="Dataset to be used for training.", title="Training Dataset", ), ] graders: Annotated[ Set[Grader], Field( description="The graders, or reward functions, to train with. If several graders are provided, the student model will see an aggregate of their grades as an average. The student will learn to maximize that grade.", title="Graders", ), ] ``` ### 1. Load a model to train To train a model, you need to use the `spawn_train` method (see details [here](/v0.14/harmony/models)). This method creates a trainable instance of your model. We set the world\_size (the number of GPU's that the job is running on) as the `tp` degree of the model by grabbing it from the recipe context `ctx` (beware, you cannot TP across node boundaries, so this only works if your world size \< # GPUs available in each node). ```python theme={null} from adaptive_harmony.runtime import recipe_main, RecipeContext @recipe_main async def training_recipe(config: TrainingConfig, ctx: RecipeContext): # Spawn the model for training model_builder = await config.model_to_train.to_builder(ctx, tp=ctx.world_size) model = await model_builder.spawn_train("main", 4096) ``` ### 2. Load a dataset You can load datasets from Adaptive or external sources: ```python theme={null} # Load from Adaptive dataset = await config.dataset.load(ctx) ``` ### 3. Set up Graders For reinforcement learning training, you'll need reward functions. Below, we show the very common case where you have a reward strategy with several components and you want to have a final reward by aggregating all your grader score. You can use the `CombinedGrader` class to do this aggregation. ```python theme={null} from adaptive_harmony.graders.combined_grader import CombinedGrader graders = [await grader.load(ctx) for grader in config.graders] combined_grader = CombinedGrader("combined", graders) ``` You do not need to rely solely on Graders you import from Adaptive to train on; you can write your own [custom grader](/v0.14/harmony/graders) within your recipe, and pass it to `CombinedGrader` along with other graders you've created and registered in the platform. ### 4. Call your training class Everything is now setup to call and run your training class. The `GRPO` class will run the main training loop. ```python theme={null} from adaptive_harmony.common.grpo import GRPO from adaptive_harmony.metric_logger import get_prod_logger logger = get_prod_logger() # Run training await GRPO( dataset=train_dataset, model=policy_model, grader=grader, logger=logger ).run() ``` See [Log job metrics](/v0.14/harmony/logging) for more information about tracking training metrics. ### 5. Save your trained model After training, save your model using the `model.save()` method. ```python theme={null} # Save with a custom name await model.save(config.output_model_key) ``` ## Training classes Harmony provides five main training algorithms, each suited for different use cases: ### SFT (Supervised Fine-Tuning) SFT is the foundation training method that teaches a model to follow instructions using demonstration data. ```python theme={null} from adaptive_harmony.common.sft import SFT from adaptive_harmony.metric_logger import get_prod_logger logger = get_prod_logger() # Run SFT training await SFT( dataset=dataset, model=model, logger=logger, lr=1e-5, samples_per_batch=512, max_grad_norm=1.0, epochs=1 ).run() ``` **Key Parameters:** * **`dataset`**: List of `StringThread` objects containing training examples * **`model`**: Training model instance * **`logger`**: Metric logger for tracking training metrics. See [Log job metrics](/v0.14/harmony/logging) for available loggers. (default: `StdoutLogger()`) * **`stage_notifier`**: Progress notifier (default: `JobNotifier().stage_notifier("SFT Training")`) * **`callbacks`**: List of training callbacks (default: `[]`) * **`lr`**: Learning rate (default: `1e-5`) * **`lr_scheduler`**: Optional learning rate scheduler function * **`samples_per_batch`**: Number of samples per training batch (default: `512`) * **`max_grad_norm`**: Gradient clipping norm (default: `1.0`) * **`epochs`**: Number of training epochs (default: `1`) * **`weight_decay`**: Weight decay coefficient (default: `0`) * **`skip_nan_gradients`**: Skip batches with NaN gradients (default: `False`) * **`checkpoint_restore_data`**: Optional `dict` of saved checkpoint state. Pass `ctx.checkpoint_restore_data` — the runtime populates it on platform resumes and leaves it `None` otherwise. See [Resuming from checkpoints](#resuming-from-checkpoints). * **`checkpoint_frequency`**: How often to checkpoint (default: `0.2`) ### PPO (Proximal Policy Optimization) PPO is a reinforcement learning algorithm that uses a reward function to improve model behavior, and estimates token-level advantages during training by use of a separate value model. ```python theme={null} from adaptive_harmony.common.ppo import PPO from adaptive_harmony.metric_logger import get_prod_logger logger = get_prod_logger() # Run PPO training await PPO( dataset=dataset, model=model, value_model=value_model, grader=grader, logger=logger, lr_policy=0.75e-6, lr_value=1e-6, samples_per_batch=128, samples_per_mini_batch=128, max_grad_norm=1.0, clip_range=0.1, kl_beta=0.01, max_num_ppo_steps=100 ).run() ``` **Key Parameters:** * **`dataset`**: List of `StringThread` prompts * **`model`**: Policy model for training * **`value_model`**: Value model for advantage estimation (must be a scalar model) * **`grader`**: Grader that returns reward values * **`logger`**: Metric logger (default: `StdoutLogger()`) * **`stage_notifier`**: Progress notifier (default: `JobNotifier().stage_notifier("PPO Training")`) * **`callbacks`**: List of training callbacks (default: `[]`) * **`max_num_ppo_steps`**: Total number of PPO training steps. Pass exactly one of `max_num_ppo_steps` or `epochs`. * **`epochs`**: Number of dataset passes (float). Pass exactly one of `max_num_ppo_steps` or `epochs`. * **`value_only_fraction`**: Fraction of training for value-only updates (default: `0.25`) * **`lr_policy`**: Policy learning rate (default: `0.75e-6`) * **`lr_scheduler_policy`**: Optional policy learning rate scheduler * **`lr_scheduler_value`**: Optional value learning rate scheduler * **`lr_value`**: Value learning rate (default: `1e-6`) * **`samples_per_batch`**: Number of samples per PPO step (default: `128`) * **`samples_per_mini_batch`**: Mini-batch size for gradient updates (default: `128`) * **`mini_epochs_per_batch`**: Number of epochs per batch (default: `1`) * **`max_grad_norm`**: Gradient clipping norm (default: `1.0`) * **`clip_range`**: PPO clipping range (default: `0.1`) * **`kl_beta`**: KL divergence penalty coefficient (default: `0.1`) * **`gae_lambda`**: GAE lambda parameter for advantage estimation (default: `0.95`) * **`gae_gamma`**: GAE gamma discount factor (default: `1.0`) * **`weight_decay`**: Weight decay coefficient (default: `0`) * **`skip_nan_gradients`**: Skip batches with NaN gradients (default: `False`) * **`checkpoint_restore_data`**: Optional `dict` of saved checkpoint state. Pass `ctx.checkpoint_restore_data` — the runtime populates it on platform resumes and leaves it `None` otherwise. See [Resuming from checkpoints](#resuming-from-checkpoints). * **`checkpoint_frequency`**: How often to checkpoint (default: `0.2`) ### GRPO (Group Relative Policy Optimization) GRPO generates a group of several completions per prompt and uses relative ranking for training. The advantage for each completion is calculated as its individual reward minus the group's mean reward. It is important the `samples_per_batch` is larger than `samples_per_mini_batch`, so that there is sample diversity for each gradient update. ```python theme={null} from adaptive_harmony.common.grpo import GRPO from adaptive_harmony.metric_logger import get_prod_logger logger = get_prod_logger() # Run GRPO training await GRPO( dataset=dataset, model=model, grader=grader, logger=logger, lr=7.5e-7, samples_per_batch=128, samples_per_mini_batch=128, max_grad_norm=1.0, clip_range=0.1, kl_beta=0.01, max_num_grpo_steps=100, completions_per_sample=8 ).run() ``` **Key Parameters:** * **`dataset`**: List of `StringThread` prompts * **`model`**: Training model * **`grader`**: Grader that returns reward values * **`logger`**: Metric logger (default: `StdoutLogger()`) * **`stage_notifier`**: Progress notifier (default: `JobNotifier().stage_notifier("GRPO Training")`) * **`callbacks`**: List of training callbacks (default: `[]`) * **`max_num_grpo_steps`**: Total number of GRPO training steps. Pass exactly one of `max_num_grpo_steps` or `epochs`. * **`epochs`**: Number of dataset passes (float). Pass exactly one of `max_num_grpo_steps` or `epochs`. * **`completions_per_sample`**: Number of completions generated per prompt (default: `8`) * **`lr`**: Learning rate (default: `7.5e-7`) * **`lr_scheduler`**: Optional learning rate scheduler function * **`samples_per_batch`**: Number of samples per batch (default: `128`) * **`samples_per_mini_batch`**: Mini-batch size for gradient updates (default: `128`) * **`mini_epochs_per_batch`**: Number of epochs per batch (default: `1`) * **`max_grad_norm`**: Gradient clipping norm (default: `1.0`) * **`clip_range`**: Clipping range for policy updates (default: `0.1`) * **`kl_beta`**: KL divergence penalty coefficient (default: `0.01`) * **`weight_decay`**: Weight decay coefficient (default: `0.0`) * **`skip_nan_gradients`**: Skip batches with NaN gradients (default: `False`) * **`checkpoint_restore_data`**: Optional `dict` of saved checkpoint state. Pass `ctx.checkpoint_restore_data` — the runtime populates it on platform resumes and leaves it `None` otherwise. See [Resuming from checkpoints](#resuming-from-checkpoints). * **`checkpoint_frequency`**: How often to checkpoint (default: `0.2`) ### GSPO (Group Sequence Policy Optimization) GSPO is a variant of GRPO that computes advantages at the sequence level rather than per-token, making it more aligned with the goal of most training runs (where token-level advantages are not meaningful at the task level) and generally more effective. Like GRPO, it generates multiple completions per prompt for relative ranking, but optimizes the entire sequence as a unit. **GSPO is recommended over GRPO for most use cases.** ```python theme={null} from adaptive_harmony.common.gspo import GSPO from adaptive_harmony.metric_logger import get_prod_logger logger = get_prod_logger() # Run GSPO training await GSPO( dataset=dataset, model=model, grader=grader, logger=logger, lr=7.5e-7, samples_per_batch=128, samples_per_mini_batch=128, max_grad_norm=1.0, clip_range=0.01, kl_beta=0.01, max_num_gspo_steps=100, completions_per_sample=8 ).run() ``` **Key Parameters:** * **`dataset`**: List of `StringThread` prompts * **`model`**: Training model * **`grader`**: Grader that returns reward values * **`logger`**: Metric logger (default: `StdoutLogger()`) * **`stage_notifier`**: Progress notifier (default: `JobNotifier().stage_notifier("GSPO Training")`) * **`callbacks`**: List of training callbacks (default: `[]`) * **`max_num_gspo_steps`**: Total number of GSPO training steps. Pass exactly one of `max_num_gspo_steps` or `epochs`. * **`epochs`**: Number of dataset passes (float). Pass exactly one of `max_num_gspo_steps` or `epochs`. * **`completions_per_sample`**: Number of completions per prompt (default: `8`, typically higher than GRPO) * **`lr`**: Learning rate (default: `7.5e-7`) * **`lr_scheduler`**: Optional learning rate scheduler function * **`samples_per_batch`**: Number of samples per batch (default: `128`) * **`samples_per_mini_batch`**: Mini-batch size for gradient updates (default: `128`) * **`mini_epochs_per_batch`**: Number of epochs per batch (default: `1`) * **`max_grad_norm`**: Gradient clipping norm (default: `1.0`) * **`clip_range`**: Clipping range (default: `0.01`, lower than GRPO's `0.1`) * **`kl_beta`**: KL divergence penalty coefficient (default: `0.01`) * **`weight_decay`**: Weight decay coefficient (default: `0.0`) * **`skip_nan_gradients`**: Skip batches with NaN gradients (default: `False`) * **`checkpoint_restore_data`**: Optional `dict` of saved checkpoint state. Pass `ctx.checkpoint_restore_data` — the runtime populates it on platform resumes and leaves it `None` otherwise. See [Resuming from checkpoints](#resuming-from-checkpoints). * **`checkpoint_frequency`**: How often to checkpoint (default: `0.2`) ### DPO (Direct Preference Optimization) DPO trains models using preference data (preferred vs non-preferred responses) without explicit reward modeling. ```python theme={null} from adaptive_harmony.common.dpo import DPO from adaptive_harmony.metric_logger import get_prod_logger logger = get_prod_logger() # Run DPO training await DPO( dataset=dataset, model=model, logger=logger, lr=1e-6, samples_per_batch=32, max_grad_norm=1.0, kl_beta=0.1, epochs=1 ).run() ``` **Key Parameters:** * **`dataset`**: List of tuples containing `(preferred_response, dispreferred_response)` as `StringThread` pairs * **`model`**: Training model * **`logger`**: Metric logger (default: `StdoutLogger()`) * **`stage_notifier`**: Progress notifier (default: `JobNotifier().stage_notifier("DPO Training")`) * **`callbacks`**: List of training callbacks (default: `[]`) * **`lr`**: Learning rate (default: `1e-6`) * **`lr_scheduler`**: Optional learning rate scheduler function * **`samples_per_batch`**: Number of samples per batch (default: `32`) * **`max_grad_norm`**: Gradient clipping norm (default: `1.0`) * **`kl_beta`**: DPO beta parameter controlling KL divergence from reference model (default: `0.1`) * **`epochs`**: Number of training epochs (default: `1`) * **`skip_nan_gradients`**: Skip batches with NaN gradients (default: `False`) ## Checkpointing All training classes support automatic checkpointing to save your model periodically during training. This is useful for long-running jobs where you want to recover from failures or inspect intermediate model states. ### Using checkpoint\_frequency Set the `checkpoint_frequency` parameter to control how often checkpoints are saved. The frequency is measured as a fraction of training completion (0.0 to 1.0). ```python theme={null} from adaptive_harmony.common.gspo import GSPO await GSPO( dataset=train_dataset, model=policy_model, grader=safety_grader, logger=logger, checkpoint_frequency=0.2 # Save every 20% of training ).run() ``` **How it works:** * Checkpoints are saved at regular intervals based on training progress * Default frequency is `0.2` (every 20% of training) * Models are saved with names like `{model_name}-{progress}`, e.g., `policy-0.2`, `policy-0.4`, `policy-0.6` * The final trained model must still be saved explicitly using `model.save()` after training completes ### Resuming from checkpoints Pass `ctx.checkpoint_restore_data` to your trainer. The runtime populates this field on the recipe context automatically when the platform resumes a previously-checkpointed job, and leaves it as `None` for fresh runs — trainers fall through to a fresh start when it is `None`. ```python theme={null} await GSPO( dataset=train_dataset, model=policy_model, grader=safety_grader, logger=logger, checkpoint_restore_data=ctx.checkpoint_restore_data, checkpoint_frequency=0.2, ).run() ``` You don't need to add anything to your `InputConfig` for this to work — `RecipeContext` exposes `checkpoint_restore_data` directly. The platform sets it from the resume artifact attached to the job. **What gets restored**: * Model weights and optimizer state * Dataset position (which samples have been processed) * Logger step counter * Callback state (when they last executed) * Training progress percentage The checkpoint is validated against the recipe type and hyperparameters — a mismatch raises before training starts to prevent corrupt runs. ### Multi-stage recipes If your recipe runs more than one trainer in sequence (for example, SFT followed by GSPO), each trainer is treated as a stage. The runtime tracks the stage counter and resumes each stage from its own checkpoint — earlier stages skip, the active stage restarts from its latest checkpoint, later stages run fresh. No code changes are required: call your trainers in order, pass `ctx.checkpoint_restore_data` to each one, and the runtime does the rest. ### Best practices * **Set appropriate frequency**: For long jobs, use `checkpoint_frequency=0.1` or `0.2`. For shorter jobs, you can use higher values like `0.25` or `0.5`. * **Storage considerations**: Checkpoints can be large for big models. Make sure you have sufficient storage for multiple checkpoints. * **Cleanup**: Remove intermediate checkpoints after training completes if you only need the final model. # Introduction Source: https://docs.adaptive-ml.com/v0.14/introduction Adaptive Engine documentation Adaptive Engine is a platform for deploying specialized agents using reinforcement learning and synthetic data. Use Adaptive Engine to fine-tune, serve and evaluate LLMs for any task. ## Get started Fine-tune a model in 10 minutes ## Core concepts Organize models, data, and training runs Deploy and run inference Upload training and evaluation data Run training and evaluation jobs # Quickstart Source: https://docs.adaptive-ml.com/v0.14/quickstart Fine-tune a model in 10 minutes This guide walks you through training your first model on Adaptive Engine. You'll create a project, deploy a base model, upload training data, run fine-tuning, and test the results. Train a customer service agent to respond like a poet. Projects organize your models, data, and training runs.