Recipe graders are custom grading code that are defined within recipes. They provide great flexibility to define any grading logic without having to deploy a reward server.
Custom recipe graders integrate directly with your training or evaluation recipes.In Training recipes, the graders defined in the recipe can be directly passed to the training classes, to be used as rewards.In Evaluation recipes, creating an evaluation sample associated with a grader_key (hardcoded in the grader code) will log the score and optional reasoning to the UI under a custom grader with the same key. If you haven’t pre-created a custom grader in the platform that matches the grader_key used in your code, it will be created automatically the first time you run an Evaluation recipe where the grader key is used.
