Load Graders

You can load Graders that have been previously created in the platform automatically using the Grader class (excluding Recipe Graders, which only exist as code within your custom recipe).

Include AdaptiveGrader in InputConfig

In order to do this, you must only make sure that one or more AdaptiveGraders can be passed to your recipe as input:

from pydantic import Field
from typing import Annotated, Set

from adaptive_harmony.runtime import InputConfig, AdaptiveGrader

class GraderRecipeConfig(InputConfig):
    graders: Annotated[
        Set[AdaptiveGrader],
        Field(
            description="The graders to load.",
            title="Graders",
        ),
    ]

The AdaptiveGrader object will contain all the configuration necessary to instantiate a Grader in code, with the exact same properties you’ve defined in the Adaptive platform.

Load with Grader.from_config

You can then automatically load and setup your graders with Grader.from_config():

from adaptive_harmony.graders import Grader
from adaptive_harmony.runtime import RecipeContext, recipe_main

@recipe_main
async def main(config: GraderRecipeConfig, ctx: RecipeContext):
    graders = []
    for grader_config in config.graders:
        grader = Grader.from_config(
            grader_config=grader_config,
            client=ctx.client,
            # the following params are optional
            # used for Binary Judge and prebuilt Graders
            tp=2,
            kv_cache_len=200_000
        )
        # setup Grader
        await grader.setup()
        graders.append(grader)

Now the Graders are ready to use for training, evaluation, or anything else you find use for!

Start

Core

Advanced

Deploy

Updates

Include AdaptiveGrader in InputConfig

Load with Grader.from_config

​Include AdaptiveGrader in InputConfig

​Load with Grader.from_config

Include AdaptiveGrader in InputConfig

Load with Grader.from_config