Log run metrics

Overview

All training classes in adaptive_harmony support logging metrics to external tracking services. Metrics include training loss, validation scores, grader evaluations, generation statistics, and any custom metrics from callbacks.

Available loggers

Adaptive Harmony supports four logging backends:

WandbLogger (Weights & Biases)

Integration with Weights & Biases for experiment tracking and visualization.

from adaptive_harmony.metric_logger import WandbLogger

logger = WandbLogger(
    project_name="my-project",      # W&B project name
    run_name="safety-training-v1",  # Name for this specific run
    entity="my-team"                # W&B team/username (optional)
)

Installation:

pip install adaptive-harmony[wandb]

Features:

Real-time metric visualization
Table logging for sample completions and evaluations
Hyperparameter tracking
Direct link to W&B dashboard via logger.training_monitoring_link

Setup:

Set your W&B API key: export WANDB_API_KEY=your_key_here
Or login via wandb login

MLFlowLogger

Integration with MLflow for experiment tracking.

from adaptive_harmony.metric_logger import MLFlowLogger

logger = MLFlowLogger(
    project_name="my-project",           # MLflow experiment name
    run_name="safety-training-v1",       # Run name within experiment
    tracking_uri="http://mlflow:5000",   # MLflow tracking server URI
    monitoring_link="http://mlflow:5000/experiments/1",  # Optional dashboard link
    experiment_tags={"team": "research"},  # Optional experiment-level tags
    run_tags={"version": "v1.0"}          # Optional run-level tags
)

Installation:

pip install adaptive-harmony[mlflow]

Features:

Metric and parameter logging
Table logging (exported as JSON artifacts)
Integration with MLflow UI
Support for custom tags and metadata

Setup:

Deploy an MLflow tracking server (can be deployed alongside Adaptive, check the Adaptive Helm Chart)
Set tracking_uri to your server URL

TBMetricsLogger (TensorBoard)

Integration with TensorBoard for visualization.

from adaptive_harmony.metric_logger import TBMetricsLogger

logger = TBMetricsLogger(
    run_name="safety-training-v1",           # Run identifier
    logging_dir="/path/to/tensorboard/logs", # Directory for TensorBoard logs
    monitoring_link="http://tensorboard:6006"  # Optional TensorBoard server URL
)

Installation:

pip install adaptive-harmony[tensorboard]

Features:

Scalar metric tracking
Table logging (exported as HTML files with auto-generated index)
Text logging for string metrics

Setup:

Specify a directory for logs
View with: tensorboard --logdir=/path/to/tensorboard/logs

StdoutLogger

Simple console output logger. Prints metrics to stdout using rich formatting. No external dependencies required.

from adaptive_harmony.metric_logger import StdoutLogger

logger = StdoutLogger()

Features:

Pretty-printed output to console
No setup required
Useful for local development and debugging

Using globally configured logger

An external logging solution can be deployed globally for all runs in an Adaptive deployment to use. This is accomplished with environment variables defined in the Adaptive Helm Chart. The get_prod_logger() function automatically returns the appropriate logger based on your environment configuration, with an active run. Runs are automatically organized in a similar fashion as Adaptive runs, use-case first, then run name. This is the recommended approach for production recipes running on Adaptive.

from adaptive_harmony.metric_logger import get_prod_logger

logger = get_prod_logger()

How it works: The function checks for logger configuration in the following priority order:

Weights & Biases - If WANDB_API_KEY environment variable is set
MLflow - If MLFLOW_TRACKING_URI environment variable is set
TensorBoard - If TENSORBOARD_LOGGING_DIR environment variable is set
StdoutLogger - Fallback if none of the above are configured

Automatic configuration: When Adaptive runs your recipe, it automatically sets environment variables for:

HARMONY_JOB_ID - Unique identifier for this run
HARMONY_USE_CASE - Use case name
HARMONY_RECIPE_NAME - Recipe name
HARMONY_JOB_NAME - Job name
Plus any logger-specific variables you’ve configured in your deployment

Example:

from adaptive_harmony.common.gspo import GSPO
from adaptive_harmony.metric_logger import get_prod_logger

logger = get_prod_logger()  # Automatically selects configured logger

await GSPO(
    dataset=train_dataset,
    model=policy_model,
    grader=safety_grader,
    logger=logger,
    max_num_gspo_steps=100,
    samples_per_batch=128,
    completions_per_sample=8
).run()

What gets logged

Training classes automatically log: Scalar metrics:

loss/train - Training loss
loss/grad_norm - Gradient norm
rewards/* - Grader scores and statistics
kl_divergence - KL divergence from reference model (RL methods)
learning_rate - Current learning rate

From callbacks:

validation/rewards/* - Validation grader scores (GraderEvalCallback)
validation/loss - Validation loss (ValidationLossCallback)
generation/* - Sample generation metrics (GenerateSamplesCallback)
Any custom metrics from your callbacks

Tables:

Sample completions with prompts and responses
Grader evaluations with reasoning
Validation results

Hyperparameters:

All trainer parameters (learning rate, batch size, etc.)
Model configuration
Recipe config parameters

Training classes automatically link their logger’s monitoring dashboard to Adaptive’s progress reporting UI. When users view your run’s progress in Adaptive, they can click through to view detailed metrics in the external logging solution you have set up globally for your Adaptive deployment. See Progress reporting for more details.

Logging custom metrics

You can log custom metrics from your training code by calling the logger directly:

logger({
    "custom_metric/accuracy": 0.95,
    "custom_metric/f1_score": 0.87,
    "custom_metric/message": "Training progressing well"
})

Or from custom callbacks by returning a dictionary from the callback method (see Training Callbacks).

Best practices

Use get_prod_logger() for recipes - Let Adaptive handle logger selection and configuration automatically
Use specific loggers for local development - When developing locally, you might want to use a specific logger:
logger = WandbLogger("dev-project", "local-test")
Configure one logger - Only set environment variables for one logger when deploying Adaptive to avoid confusion

Check monitoring links - Access the dashboard URL via:

print(f"View training at: {logger.training_monitoring_link}")

Close loggers when done - Training classes handle this automatically, but if you’re using loggers manually:

try:
    # Training code
    pass
finally:
    logger.close()  # Ensure logs are flushed

Getting Started

Building Blocks

Training & Evaluation

Cookbook

Updates

Overview

Available loggers

WandbLogger (Weights & Biases)

MLFlowLogger

TBMetricsLogger (TensorBoard)

StdoutLogger

Using globally configured logger

What gets logged

Logging custom metrics

Best practices

​Overview

​Available loggers

​WandbLogger (Weights & Biases)

​MLFlowLogger

​TBMetricsLogger (TensorBoard)

​StdoutLogger

​Using globally configured logger

​What gets logged

​Logging custom metrics

​Best practices

Overview

Available loggers

WandbLogger (Weights & Biases)

MLFlowLogger

TBMetricsLogger (TensorBoard)

StdoutLogger

Using globally configured logger

What gets logged

Logging custom metrics

Best practices