Skip to main content
Open on GitHub

Confident AI

Confident AI is a creator of the DeepEval.

DeepEval is a package for unit testing LLMs. Using DeepEval, everyone can build robust language models through faster iterations using both unit testing and integration testing. `DeepEval provides support for each step in the iteration from synthetic data creation to testing.

Installation and Setup​

You need to get the DeepEval API credentials.

You need to install the DeepEval Python package:

pip install deepeval

Callbacks​

See an example.

Modern Integration Example​

Install the required packages:

pip install deepeval langchain langchain-openai

Authenticate with your API key:

import os
import deepeval

# Load API key from environment variable for security
api_key = os.environ.get("DEEPEVAL_API_KEY")
deepeval.login(api_key)

Use the new callback handler:

from deepeval.integrations.langchain import CallbackHandler

handler = CallbackHandler(
    name="My Trace",
    tags=["production", "v1"],
    metadata={"experiment": "A/B"},
    thread_id="thread-123",
    user_id="user-456"
)

See the full example.