Release notes

​

Breaking changes

• “Use Cases” renamed to “Projects” across SDK, GraphQL API, REST API, and URL paths.
  • client.use_cases → client.projects (client.use_cases deprecated but still works)
  • client.default_use_case / client.set_default_use_case() → client.default_project / client.set_default_project()
  • models.add_to_use_case() → models.add_to_project()
  • models.detach(use_case=...) → models.detach(project=...) — project is now required
  • models.deploy(use_case=...) → models.deploy(project=...)
  • jobs.run(use_case=...) / jobs.list(use_case=...) → jobs.run(project=...) / jobs.list(project=...)
  • interactions.list(use_case=...) → interactions.list(project=...)
  • graders.list/get/delete/lock(use_case=...) → graders.list/get/delete/lock(project=...)
  • recipes.list(use_case=...) → recipes.list(project=...)
  • feedback.get_key(feedback_key) → feedback.get_key(project, feedback_key) — project is now a required positional argument
  • feedback.create_metric() now requires project as a parameter
  • GraphQL: useCase / UseCaseData → project / ProjectData
  • Integration topic patterns: existing subscriptions using the old use_case: prefix (e.g. use_case:*:job:*:*) continue to work — they are automatically matched against the new project: topics. No action required.
• feedback.link() and feedback.unlink() removed.

​

User interface

• Multimodal image input in chat and interaction store: attach images alongside text for models with vision capabilities.
• Interactive recipe sessions visible in run history with progress reporting, artifact uploads, and cancellation.
• Multi-file recipe viewer: browse and edit individual files within a recipe.
• Analytics dashboard pre-populated with default plots for new projects.
• Hourly granularity in the usage dashboard.
• ISB column selector: choose which columns to display in the interaction table.
• Filter interactions by session ID; copy session ID from any interaction detail panel.
• Download a selection of interactions as a file for offline analysis.
• Delete interactions from the interaction store.
• Add multiple models to a project in a single action from the model registry.
• Click-to-copy model key in the project model registry.
• Parent training run link shown in model detail page.
• Run logs visible on the run preview sheet.

​

Metrics

• Three categories unified under one system: system metrics (auto-computed: TTFT, latency, token counts), grader metrics (scores produced by judges and custom graders), and user metrics (custom metrics, formerly “feedback”).
• Metrics are now project-scoped; the legacy link/unlink mechanism is removed.
• New project Metrics page replaces the org-level Feedbacks page, with metric detail view, create/edit/delete dialogs, category filters, and search.
• Metric pickers across the product group options by category (System / Grader / User).
• Metrics can be deleted; system and built-in grader metrics are protected.

​

Inference & model management

• Speculative decoding: train an accelerated model with a built-in recipe to reduce inference latency.
• Multimodal inference end-to-end: streaming for vision models, improved image encoder handling, completions API aligned with OpenAI image format.
• Flash Attention 3 C++ backend with split-KV aware graph capture.

​

Training & runs

• KV-caching for training (diff KV cache): reduces activation sizes for long context training.
• Recipe names and descriptions: set a human-readable name and description when uploading a recipe.
• New built-in recipe for aligning a draft model to a target (speculative decoding).
• DPO training supports dataset looping.

​

SDK

• extra_params exposed for external models in recipes and via models.add_external().
• New client.create_service_account() method.
• Named interactive sessions via name parameter in RecipeContext.
• chat.create() store parameter to control whether completions are saved.

​

Evaluation & data management

• AI Judge graders now return metadata alongside scores.
• Dataset download preserves the original uploaded filename.

​

Administration & infrastructure

• Notifications: subscribe to webhook, Slack, or email notifications for platform events (job completion, failure) with topic-based filtering and pattern matching.
• HIPAA audit logging: full audit trail for query events, file access events, and connection events, each with user identity, IP address, and timestamp.
• User management: full CRUD for users at the organization level; first and last activity timestamps surfaced in the UI.
• Service accounts: create dedicated bot users with API keys for CI/CD pipelines and automated workflows; API key usage is audited.
• Permissions and roles visible in the UI.
• Security: updated dependencies addressing CVEs.

​

Breaking changes

• SDK: jobs.run() parameter reorder - args moved from required positional to optional keyword.
• SDK: interactions filter format - advancedFilter → advancedFilters, label filter structure changed.
• SDK: ModelserviceStatus enum - removed DETACHED value.
• SDK: async methods removed - async attach() and async deploy() no longer available.
• SDK: models.detach() - use_case parameter now required (was optional).
• SDK: models.update() - removed attached parameter.
• Recipes: grader setup() and teardown() calls removed from production recipes.
• Sandbox enabled by default - opt-out with ENABLE_SANDBOX=0.

​

User interface

• New sidebar presentation with split use case overview page.
• New model registry UI: model details sidebar, table grouping, filtering by name/size/status.
• Chat comparison page for side-by-side interaction analysis.
• Raw/formatted selector and copy buttons in completion details.
• Artifact status reflected in runs and evaluations UI.
• Recipe upload by file with improved recipe fields display in run form.
• GPU count and GPU time displayed in run details; prefill on clone run.
• Improved dataset upload dialog and labels display in interaction store.
• Grader tooltip showing metric column header details.

​

Inference & model management

• Differentiable KV cache with CuTe-DSL Flash Attention kernels.
• LoRA models are now trainable directly.
• Organization-level model deletion.
• OpenAI-compatible external models by endpoint.
• Model/use case bindings with use case stored in model registry.
• Model statuses: published and stable.
• LoRA backbone reuse and deduplicated spawn logic.
• Configurable max prefill size via DEFAULT_MAX_TOKENS_IN_INF_BATCH.
• TP support for FP8 MoE.
• Suggested tensor parallelism logic based on model size.
• Multimodal prefill splitting across batches.
• Static memory usage limits via GPU_MEMORY_UTILIZATION.
• Session claim mechanism after disconnection.

​

Training & runs

• Logger parameter for PPO and GRPO classes.
• Multimodal dataset support.
• Recipe OOM errors reported to user.
• Optional defaults for recipe launch with improved upload SDK.
• recipe_key now optional (inferred from file/directory name).
• New entrypoint parameter for custom entry points in directories.

​

SDK

• New artifacts.download(artifact_id, destination_path) method.
• New models.add_to_use_case() method.
• models.list() accepts optional filter parameter.
• models.deploy() signature changed: new use_case, placement, make_default params.
• chat.create() new store parameter to control completion storage.
• JobArtifactStatus enum: PENDING, PROCESSING, READY, ERROR.

​

Evaluation & data management

• Text search API on completions and prompts.
• Advanced filter API for completions.
• Preference dataset processing improvements.
• Dataset with feedback selection in recipes.
• Dataset deletion runs as background task.

​

Administration & infrastructure

• HIPAA compliance: model and use case reporting for chat/completions.
• JWT authentication on internal API for recipes.
• TLS support in Redis and Mangrove.
• Sandbox improvements: HuggingFace import fixes, git support, syscall filtering.
• Job permissions support.
• Multiple replicas cancellation support.
• Configurable RAM limits on Sandkasten.
• Library upgrades for CVE fixes.

​

Breaking changes

• Model keys: model path is now used as the key in the model registry instead of other identifiers.

​

Inference & model management

• Support custom endpoint URL for OpenAI Completions API-compatible external models.
• Make API keys optional for external models.
• Support per-request tool override for chat completions.
• New connectivity check endpoint (ADAPTIVE_URL/health).
• New min_gen_len setter method for InferenceModel and TrainingModel.
• Support GPU inference partition resize in SDK.
• Improve FP8 MoE inference.

​

Training & runs

• First GSPO (Group Sequence Policy Optimization) implementation.
• Add support for context with assistant turn on multi-turn generation with env_grpo.
• Improved model saving and checkpointing.
• New skip_nan_gradients argument in model.optim_step().
• Multi-file recipe support with proper frontend handling.
• Recipe schema improvements and better parsing.

​

Evaluation & data management

• Labels and feedback annotations in chat UI and interaction store.
• Multimodal dataset support with image handling.
• Ability to upload GB-sized datasets.
• Enhanced dataset creation with immutable dataset files for reuse in recipes.
• Improved dataset artifact management.

​

Administration & infrastructure

• Add preflight checks to ensure environment compatibility.
• Better resource management and allocation tracking.

​

User interface

• Global search bar with cmd+K.

​

Inference & model management

• Allow external model spawning via HTTP.
• Support for detached models in chat.
• Rich magic for better REPL experience.
• Better timeout configuration for client SDK.

​

Training & runs

• Display active run in compute pool detail page.
• Enrich the parameters of the SFT recipe.
• Rejection sampling production recipe.
• GRPO KL divergence fix.
• Better handling of env_grpo sample loading.
• Improved callback system for training.

​

Evaluation & data management

• AI Judge workbench v3 with enhanced UI.
• Custom grader support in product.
• Dataset viewer page introduction.
• Improved dataset chunked upload in SDK.
• Better dataset source tracking.
• Visual improvements to interaction store browsing.
• External feedback endpoint in evaluation wizard.

​

Administration & infrastructure

• Dynamic world size support (experimental).
• GPU metrics and Redis connection management cleanup.

​

Breaking changes

• Model output artifact changes for better organization.

​

User interface

• New recipes-centric use case navigation.
• New split view to better navigate runs.

​

Inference & model management

• MCP (Model Context Protocol) with all turns support.
• System prompt support in chat settings.
• Temperature control in chat settings.
• Better external model handling and API integration.

​

Training & runs

• Loss clamping support.
• Callbacks and training recipe cleanup.
• Better skip-token-masking loss computation.
• Improved dataset shuffling with seeding.

​

Evaluation & data management

• Judge Playground v2 with enhanced UI.
• Parse XML stream from response in chat.
• User metadata support in interaction store.
• Dataset generation from interaction store filters.
• Metric aggregation controls in header.
• Better evaluation result reporting.
• Dataset artifacts with proper management.
• Improved interaction state persistence in localStorage.

​

Administration & infrastructure

• Better resource management logging.
• Add UI controls to reset and resize GPU inference partitions.

​

Breaking changes

• New message format migration for completions.

​

User interface

• Better form input styles following Epoch Design System.

​

Inference & model management

• Add model search in model registry.
• Model service configuration improvements.
• Better DMA (Direct Memory Access) handling.
• External model API key management improvements.
• OpenAI Response API support.
• Model conversion re-added with better handling.

​

Training & runs

• Better model initialization fixes.
• Improved training callback system.

​

Evaluation & data management

• Pre-built criteria with documentation links.
• Increase robustness of Amazon S3 support integration in custom recipes.
• Better recipe editor UI.
• Enhanced interaction store with tooltip for all turns.
• Improve MLFlow integration: let users view their use cases runs.
• Add utility to upload and update custom recipes.

​

Administration & infrastructure

• Error management improvements with new error pages.
• Contract usage reporting.

​

Training & runs

• Shuffling in GRPO with better batching.
• Improved built-in recipes with better parameters.
• Grader evaluation support in Harmony.

​

Evaluation & data management

• Tool providers CRUD operations.
• Ability to link tool providers with model services.
• Custom grader support with enhanced UI.
• New summarization recipe.
• Interaction store general UI refactor.
• UX improvement in the AI judge workbench.
• Evaluation error reporting in new evaluations page.

​

Administration & infrastructure

• Team permission selector in use case creation UI.
• Job partition improvements.
• Kill router on connection drop.
• Add API to create and delete users ahead of their SSO registration.

​

Use interface

• Introduce use case overview dropdown.
• Read-only permission UI.

​

Inference & model management

• Extending integration of Google API models.
• Expose max_ttft parameter at request level.

​

Evaluation & data management

• Refactoring judge & prompt playground.
• Preset metric visibility in the side-by-side view.
• Improvement to built-in AI judges.
• New evaluation wizard.
• New evaluation results table.
• Better evaluation exports.
• Adding support for grader evaluation in custom recipes.

​

Training & runs

• Custom recipes improvements.
• Adding job partition concept, allowing to run on subset of available GPUs.
• Adding Infiniband health check.
• Better training arguments and world size requirements removed.

​

Administration & infrastructure

• Add team removal method in SDK.
• Display available GPU partitions.

​

User interface

• New design system.
• Read-only permission UI.
• Improve Hugging Face model import UI.

​

Evaluation & data management

• Adding source metadata to identify origin of datasets.
• Extend evaluation to support more models evaluated in parallel.
• UI for external feedback endpoints (RLEF).
• Access individual records from feedback detail page.
• Support optional metadata saving in data generation jobs.

​

Training & runs

• Increase KV cache length in GRPO recipe.
• Create dedicated URLs for run detail pages.
• Expose more RL parameters in the training API.
• Support journaling & replay in reward servers (RLEF).
• Add APIs for RAG dataset generation.
• Multi-judge training SDK.

​

User interface

• Add use case search.
• New use case-centric navigation.

​

Inference & model management

• Add support for Anthropic and NVIDIA NIM external models.
• Add compute configuration (placement) to model endpoints.
• Improve tokenization speed.
• Custom inference kernels for A100, L40S, H100, H200.
• Add richer inference metadata: parameters, latencies.

​

Evaluation & data management

• Display interaction metadata in interaction detail page.
• Export raw interactions (JSONL) from the interaction store.
• Add ability to evaluate existing completions.

​

Training & runs

• Addition of GRPO.
• Display validation in training run details.
• Better OOM management.
• Improve granular timestamp reporting.
• Improve custom attention.
• Remove sync points in training.
• Improve job status UI.

​

Administration & infrastructure

• Grafana logs integration.

​

Inference & model management

• Add support for inference autoscaling on Kubernetes.

​

Evaluation & data management

• Filter feedback by label.
• Improve feedback display in interaction detail page.
• Add support for annotation (scalar, boolean, text comments) in the interaction store.

​

Training & runs

• Add Tensorboard integration.
• Improve built-in SFT recipe & add SFT-specific UI launcher.
• Add reward servers (RLEF) in SDK.

​

Administration & infrastructure

• Expose concept of compute pools.

​

Inference & model management

• Adding concept of deployment placement, to manage partitioning and distribution of resources.

​

Training & runs

• Extend Weight & Bias integration to multi-step jobs.
• New UI to directly train on uploaded dataset.

​

Inference & model management

• Integration with Azure OpenAI endpoints.

​

Evaluation & data management

• New UI to enter granular AI judge policies for evaluation and training.
• New dataset upload & browsing page.

​

Administration & infrastructure

• Extend permission management APIs and default team behavior.
• GPU memory management improvements.