Release notes

​

Breaking changes

• Model registry key: model path now used for key in the model registry instead of other identifiers.

​

Inference & model management

• Support passing custom endpoints for external models.
• Make API keys optional for external models.
• Add per-request tool override support for chat completions.
• Add connectivity check endpoint.
• Support for min_gen_len parameter.
• Support GPU inference partition resize from SDK.
• Improve FP8 MoE inference.

​

Training & runs

• First GSPO (Generalized Self-Play Optimization) implementation.
• Add support for context with assistant turn on multi-turn generation with env_grpo.
• Improved model saving and checkpointing.
• Support for skip_nan_gradients flag in model.optim_step.
• Multi-file recipe support with proper frontend handling.
• Recipe schema improvements and better parsing.

​

Evaluation & data management

• Labels and feedback annotations in chat UI and interaction store.
• Multimodal dataset support with image handling.
• Ability to upload GB-sized datasets.
• Enhanced dataset creation with immutable dataset files for reuse in recipes.
• Improved dataset artifact management.

​

Administration & infrastructure

• Add preflight checks to ensure environment compatibility.
• Better resource management and allocation tracking.

​

User interface

• Global search bar with cmd+K.

​

Inference & model management

• Allow external model spawning via HTTP.
• Support for detached models in chat.
• Rich magic for better REPL experience.
• Better timeout configuration for client SDK.

​

Training & runs

• Display active run in compute pool detail page.
• Enrich the parameters of the SFT recipe.
• Rejection sampling production recipe.
• GRPO KL divergence fix.
• Better handling of env_grpo sample loading.
• Improved callback system for training.

​

Evaluation & data management

• AI Judge workbench v3 with enhanced UI.
• Custom grader support in product.
• Dataset viewer page introduction.
• Improved dataset chunked upload in SDK.
• Better dataset source tracking.
• Visual improvements to interaction store browsing.
• External feedback endpoint in evaluation wizard.

​

Administration & infrastructure

• Dynamic world size support (experimental).
• GPU metrics and Redis connection management cleanup.

​

Breaking changes

• Model output artifact changes for better organization.

​

User interface

• New recipes-centric use case navigation.
• New split view to better navigate runs.

​

Inference & model management

• MCP (Model Context Protocol) with all turns support.
• System prompt support in chat settings.
• Temperature control in chat settings.
• Better external model handling and API integration.

​

Training & runs

• Loss clamping support.
• Callbacks and training recipe cleanup.
• Better skip-token-masking loss computation.
• Improved dataset shuffling with seeding.

​

Evaluation & data management

• Judge Playground v2 with enhanced UI.
• Parse XML stream from response in chat.
• User metadata support in interaction store.
• Dataset generation from interaction store filters.
• Metric aggregation controls in header.
• Better evaluation result reporting.
• Dataset artifacts with proper management.
• Improved interaction state persistence in localStorage.

​

Administration & infrastructure

• Better resource management logging.
• Add UI controls to reset and resize GPU inference partitions.

​

Breaking changes

• New message format migration for completions.

​

User interface

• Better form input styles following Epoch Design System.

​

Inference & model management

• Add model search in model registry.
• Model service configuration improvements.
• Better DMA (Direct Memory Access) handling.
• External model API key management improvements.
• OpenAI Response API support.
• Model conversion re-added with better handling.

​

Training & runs

• Better model initialization fixes.
• Improved training callback system.

​

Evaluation & data management

• Pre-built criteria with documentation links.
• Increase robustness of Amazon S3 support integration in custom recipes.
• Better recipe editor UI.
• Enhanced interaction store with tooltip for all turns.
• Improve MLFlow integration: let users view their use cases runs.
• Add utility to upload and update custom recipes.

​

Administration & infrastructure

• Error management improvements with new error pages.
• Contract usage reporting.

​

Training & runs

• Shuffling in GRPO with better batching.
• Improved built-in recipes with better parameters.
• Grader evaluation support in Harmony.

​

Evaluation & data management

• Tool providers CRUD operations.
• Ability to link tool providers with model services.
• Custom grader support with enhanced UI.
• New summarization recipe.
• Interaction store general UI refactor.
• UX improvement in the AI judge workbench.
• Evaluation error reporting in new evaluations page.

​

Administration & infrastructure

• Team permission selector in use case creation UI.
• Job partition improvements.
• Kill router on connection drop.
• Add API to create and delete users ahead of their SSO registration.

​

Use interface

• Introduce use case overview dropdown.
• Read-only permission UI.

​

Inference & model management

• Extending integration of Google API models.
• Expose max_ttft parameter at request level.

​

Evaluation & data management

• Refactoring judge & prompt playground.
• Preset metric visibility in the side-by-side view.
• Improvement to built-in AI judges.
• New evaluation wizard.
• New evaluation results table.
• Better evaluation exports.
• Adding support for grader evaluation in custom recipes.

​

Training & runs

• Custom recipes improvements.
• Adding job partition concept, allowing to run on subset of available GPUs.
• Adding Infiniband health check.
• Better training arguments and world size requirements removed.

​

Administration & infrastructure

• Add team removal method in SDK.
• Display available GPU partitions.

​

User interface

• New design system.
• Read-only permission UI.
• Improve Hugging Face model import UI.

​

Evaluation & data management

• Adding source metadata to identify origin of datasets.
• Extend evaluation to support more models evaluated in parallel.
• UI for external feedback endpoints (RLEF).
• Access individual records from feedback detail page.
• Support optional metadata saving in data generation jobs.

​

Training & runs

• Increase KV cache length in GRPO recipe.
• Create dedicated URLs for run detail pages.
• Expose more RL parameters in the training API.
• Support journaling & replay in reward servers (RLEF).
• Add APIs for RAG dataset generation.
• Multi-judge training SDK.

​

User interface

• Add use case search.
• New use case-centric navigation.

​

Inference & model management

• Add support for Anthropic and NVIDIA NIM external models.
• Add compute configuration (placement) to model endpoints.
• Improve tokenization speed.
• Custom inference kernels for A100, L40S, H100, H200.
• Add richer inference metadata: parameters, latencies.

​

Evaluation & data management

• Display interaction metadata in interaction detail page.
• Export raw interactions (JSONL) from the interaction store.
• Add ability to evaluate existing completions.

​

Training & runs

• Addition of GRPO.
• Display validation in training run details.
• Better OOM management.
• Improve granular timestamp reporting.
• Improve custom attention.
• Remove sync points in training.
• Improve job status UI.

​

Administration & infrastructure

• Grafana logs integration.

​

Inference & model management

• Add support for inference autoscaling on Kubernetes.

​

Evaluation & data management

• Filter feedback by label.
• Improve feedback display in interaction detail page.
• Add support for annotation (scalar, boolean, text comments) in the interaction store.

​

Training & runs

• Add Tensorboard integration.
• Improve built-in SFT recipe & add SFT-specific UI launcher.
• Add reward servers (RLEF) in SDK.

​

Administration & infrastructure

• Expose concept of compute pools.

​

Inference & model management

• Adding concept of deployment placement, to manage partitioning and distribution of resources.

​

Training & runs

• Extend Weight & Bias integration to multi-step jobs.
• New UI to directly train on uploaded dataset.

​

Inference & model management

• Integration with Azure OpenAI endpoints.

​

Evaluation & data management

• New UI to enter granular AI judge policies for evaluation and training.
• New dataset upload & browsing page.

​

Administration & infrastructure

• Extend permission management APIs and default team behavior.
• GPU memory management improvements.