Interactions

An interaction is the combination of a prompt, a model’s completion, and optionally feedback and metadata (such as labels). When you make an inference request to Adaptive, the resulting interaction is automatically logged to the Adaptive Interaction Store. You can also log interactions ad-hoc for an existing model that is attached to a use case, which is useful for evaluation or model tuning. You can log interactions as follows:

Create an Adaptive client first

response = adaptive.interactions.create(
    model="model_key",
    completion="Model's completion",
    messages=[
        {"role": "system", "content": "System prompt"},
        {"role": "user", "content":"User input"}    
    ],
    feedbacks=[
        {"feedback_key": "metric_key_1", "value": 0, "details": "Feedback details 1"},
        {"feedback_key": "metric_key_2", "value": 1, "details": "Feedback details 2"}
    ]
    labels={"label_key": "label_value"}
)

Load large dataset asynchronously

Adaptive SDK

    from adaptive_sdk import Adaptive, AsyncAdaptive
    import json
    import asyncio
    from tqdm.asyncio import tqdm_asyncio
    from tqdm import tqdm

    async def upload_data_async(
        adaptive: AsyncAdaptive,
        data: list[dict], 
        labels: dict[str,str],
        max_concurrency: int = 30
    ):
        async def coroutine(semaphore, coroutine):
            async with semaphore:
                return await coroutine

        semaphore = asyncio.Semaphore(max_concurrency)
        coroutines = []
        for i in data:
            coroutines.append(
                coroutine(
                    semaphore,
                    adaptive.interactions.create(
                        messages=i["messages"],
                        completion=i["completion"],
                        labels=labels
                    ),
                )
            )

        print("\nLoading data...")
        results = await tqdm_asyncio.gather(*coroutines)

    if __name__=="__main__":
        url = "REPLACE_ADAPTIVE_URL"
        api_key = "REPLACE_ADAPTIVE_API_KEY"
        use_case_key = "REPLACE_EXISTING_USE_CASE"

        # `data` is assumed to have this format
        data = [
            {
                "messages": [
                    {"role": "system", "content": "You are a good bot."},
                    {"role": "user", "content": "Hey there!"}
                ],
                "completion": "Hey back at you!"
            }
        ]
        labels = {"dataset": "my_custom_dataset"}
        
        adaptive = AsyncAdaptive(base_url=url, api_key=api_key)
        adaptive.set_default_use_case(use_case_key)
        asyncio.run(upload_data_async(adaptive, data, labels))

All automatically or manually logged interactions can be explored in the Interactions page of the Adaptive UI. You can also retrieve interactions programmatically via SDK for ad-hoc analysis. See the SDK Reference for all interaction-related methods.

.

Platform

Inference and Feedback

Datasets

Evaluation

Fine-tuning

Custom Recipes

Integrations

Deployment