Skip to main content

​
Overview

All training classes in adaptive_harmony support logging metrics to external tracking services. Metrics include training loss, validation scores, grader evaluations, generation statistics, and any custom metrics from callbacks.

​
Available Loggers

Adaptive Harmony supports four logging backends:

​
WandbLogger (Weights & Biases)

Integration with Weights & Biases for experiment tracking and visualization. 
from adaptive_harmony.metric_logger import WandbLogger

logger = WandbLogger(
    project_name="my-project",      # W&B project name
    run_name="safety-training-v1",  # Name for this specific run
    entity="my-team"                # W&B team/username (optional)
)
Installation: 
pip install adaptive-harmony[wandb]
Features:
  • Real-time metric visualization
  • Table logging for sample completions and evaluations
  • Hyperparameter tracking
  • Direct link to W&B dashboard via logger.training_monitoring_link
Setup:
  • Set your W&B API key: export WANDB_API_KEY=your_key_here
  • Or login via wandb login

​
MLFlowLogger

Integration with MLflow for experiment tracking. 
from adaptive_harmony.metric_logger import MLFlowLogger

logger = MLFlowLogger(
    project_name="my-project",           # MLflow experiment name
    run_name="safety-training-v1",       # Run name within experiment
    tracking_uri="http://mlflow:5000",   # MLflow tracking server URI
    monitoring_link="http://mlflow:5000/experiments/1",  # Optional dashboard link
    experiment_tags={"team": "research"},  # Optional experiment-level tags
    run_tags={"version": "v1.0"}          # Optional run-level tags
)
Installation: 
pip install adaptive-harmony[mlflow]
Features:
  • Metric and parameter logging
  • Table logging (exported as JSON artifacts)
  • Integration with MLflow UI
  • Support for custom tags and metadata
Setup:
  • Deploy an MLflow tracking server (can be deployed alongside Adaptive, check the Adaptive Helm Chart)
  • Set tracking_uri to your server URL

​
TBMetricsLogger (TensorBoard)

Integration with TensorBoard for visualization. 
from adaptive_harmony.metric_logger import TBMetricsLogger

logger = TBMetricsLogger(
    run_name="safety-training-v1",           # Run identifier
    logging_dir="/path/to/tensorboard/logs", # Directory for TensorBoard logs
    monitoring_link="http://tensorboard:6006"  # Optional TensorBoard server URL
)
Installation: 
pip install adaptive-harmony[tensorboard]
Features:
  • Scalar metric tracking
  • Table logging (exported as HTML files with auto-generated index)
  • Text logging for string metrics
Setup:
  • Specify a directory for logs
  • View with: tensorboard --logdir=/path/to/tensorboard/logs

​
StdoutLogger

Simple console output logger. Prints metrics to stdout using rich formatting. No external dependencies required. 
from adaptive_harmony.metric_logger import StdoutLogger

logger = StdoutLogger()
Features:
  • Pretty-printed output to console
  • No setup required
  • Useful for local development and debugging

​
Using globally configured logger

An external logging solution can be deployed globally for all runs in an Adaptive deployment to use. This is accomplished with environment variables defined in the Adaptive Helm Chart. The get_prod_logger() function automatically returns the appropriate logger based on your environment configuration, with an active run. Runs are automatically organized in a similar fashion as Adaptive runs, use-case first, then run name. This is the recommended approach for production recipes running on Adaptive. 
from adaptive_harmony.metric_logger import get_prod_logger

logger = get_prod_logger()
How it works: The function checks for logger configuration in the following priority order:
  1. Weights & Biases - If WANDB_API_KEY environment variable is set
  2. MLflow - If MLFLOW_TRACKING_URI environment variable is set
  3. TensorBoard - If TENSORBOARD_LOGGING_DIR environment variable is set
  4. StdoutLogger - Fallback if none of the above are configured
Automatic configuration: When Adaptive runs your recipe, it automatically sets environment variables for:
  • HARMONY_JOB_ID - Unique identifier for this run
  • HARMONY_USE_CASE - Use case name
  • HARMONY_RECIPE_NAME - Recipe name
  • HARMONY_JOB_NAME - Job name
  • Plus any logger-specific variables you’ve configured in your deployment
Example: 
from adaptive_harmony.common.gspo import GSPO
from adaptive_harmony.metric_logger import get_prod_logger

logger = get_prod_logger()  # Automatically selects configured logger

await GSPO(
    dataset=train_dataset,
    model=policy_model,
    grader=safety_grader,
    logger=logger,
    max_num_gspo_steps=100,
    samples_per_batch=128,
    completions_per_sample=8
).run()

​
What Gets Logged

Training classes automatically log: Scalar metrics:
  • loss/train - Training loss
  • loss/grad_norm - Gradient norm
  • rewards/* - Grader scores and statistics
  • kl_divergence - KL divergence from reference model (RL methods)
  • learning_rate - Current learning rate
From callbacks:
  • validation/rewards/* - Validation grader scores (GraderEvalCallback)
  • validation/loss - Validation loss (ValidationLossCallback)
  • generation/* - Sample generation metrics (GenerateSamplesCallback)
  • Any custom metrics from your callbacks
Tables:
  • Sample completions with prompts and responses
  • Grader evaluations with reasoning
  • Validation results
Hyperparameters:
  • All trainer parameters (learning rate, batch size, etc.)
  • Model configuration
  • Recipe config parameters
Training classes automatically link their logger’s monitoring dashboard to Adaptive’s progress reporting UI. When users view your run’s progress in Adaptive, they can click through to view detailed metrics in the external logging solution you have set up globally for your Adaptive deployment. See Progress reporting for more details.

​
Logging Custom Metrics

You can log custom metrics from your training code by calling the logger directly: 
logger({
    "custom_metric/accuracy": 0.95,
    "custom_metric/f1_score": 0.87,
    "custom_metric/message": "Training progressing well"
})
Or from custom callbacks by returning a dictionary from the callback method (see Training Callbacks).

​
Best Practices

  1. Use get_prod_logger() for recipes - Let Adaptive handle logger selection and configuration automatically
  2. Use specific loggers for local development - When developing locally, you might want to use a specific logger: 
    logger = WandbLogger("dev-project", "local-test")
  3. Configure one logger - Only set environment variables for one logger when deploying Adaptive to avoid confusion
  4. Check monitoring links - Access the dashboard URL via: 
    print(f"View training at: {logger.training_monitoring_link}")
  5. Close loggers when done - Training classes handle this automatically, but if you’re using loggers manually: 
    try:
    # Training code
    pass
finally:
    logger.close()  # Ensure logs are flushed