Model loading guide - Adaptive ML Documentation

Loading a model

To load a model in a custom recipe, you should use the harmony client and provide the model path to the model method:

from adaptive_harmony.runtime import recipe_main, RecipeContext

@recipe_main
async def main(ctx: RecipeContext):
    client = ctx.client
    model = await client.model("model_registry://llama-3.1-8b-instruct").tp(1).spawn_train("model", 4096) 

Passing models as input

If a model should be given as input, there are two ways to give it.

From the context

In custom recipes, you are always asked to input a model to train. This model is retrieved from the recipe context in the attribute model_to_train. The model can then by loaded using the client. Make sure to pass the model path attribute to the harmony client and not the model itself.

from adaptive_harmony.runtime import recipe_main, RecipeContext

@recipe_main
async def main(ctx: RecipeContext):
    client = ctx.client
    assert ctx.model_to_train, "Model must be set for training"
    print("Model To Train: {ctx.model_to_train}")

    model = await client.model(ctx.model_to_train.path).tp(ctx.world_size).spawn_train("model", 4096) 

From the config

In custom recipes, you can pass models in the recipe config. The model is then loaded by giving its path to the harmony client.

from typing import Annotated
from adaptive_harmony import InputConfig, AdaptiveModel

class MyConfig(InputConfig):
    judge_model: AdaptiveModel

@recipe_main
async def main(config: InputConfig, ctx: RecipeContext):
    client = ctx.client
    print("Judge Model: {config.judge_model}")

    model = await client.model(config.judge_model.path).tp(1).spawn_train("model", 4096) 

Load a model from Hugging Face

Use load_from_hf() to load models directly from Hugging Face if you have an internet connection:

from adaptive_harmony.core.dataset import load_from_hf

model = load_from_hf("mistralai/Mistral-7B-Instruct-v0.3")

⌘I