Adapting models to align with human or AI feedback through fine-tuning is at the core of Adaptive Engine. It enables users to bootstrap initial models to beat frontier performance on specific tasks using only synthetic data, and then continuously improve them with production feedback.

Adaptive Engine has built-in, robust recipes for supervised fine-tuning and reinforcement learning. These recipes can also be customized, allowing you to tweak and explore hyperparameters.

Adaptive Engine supports 2 training objectives:

  1. Adapt using existing feedback - fine-tunes a model to improve an outcome you have previously logged.
  2. Teach behaviour with natural language guidelines - provide simple textual guidelines to define what constitutes a good and bad completion for your use case; an AI judge will use them to align your model with the desired behaviour. Reference completions and existing feedback are not required, only prompts are used.

You can use the Adaptive SDK to launch either of the above:

Create an Adaptive client first

# Train on previously uploaded dataset
dataset_name = "support-dataset.jsonl"
dataset_key = [dataset.key for dataset in client.datasets.list() if dataset.name==dataset_name][0]

adapt_job = client.training.job.create(
    model_key="llama-3.1-8b-instruct",
    config = {
        "output_name": "llama-8b-support-acceptance",
        "training_config": {
            "training_objective": {
                "metric": {
                    "metric_key": "acceptance"
                    }
            },
        },
        "sample_config": {
            "datasource": {"dataset": {"dataset": dataset_key}}
        }
    }
)

If you want more control over training, you can customize the training method, sample selection and hyperparameters in your config. See the SDK Reference for the full training config specification.

.create will create and register a new model you can deploy for inference, or A/B Test against the base model or others for validation.