Quickstart - Maihem

What is Maihem?

Maihem creates AI agents that automatically test your LLM applications, by simulating conversations between your Target Agent and Maihem Agents.

The benefits of Maihem are:

Get access

Book a call to get access to the Maihem platform.

Generate API key

Sign into your Maihem account and generate an API key by going to your account settings.

Set API key as environment variable

Before using the maihem package, you need to set your maihem API key as an environment variable:

Environment variable: MAIHEM_API_KEY

.env file
MAIHEM_API_KEY=<your_maihem_api_key>

Terminal
MAIHEM_API_KEY=<your_maihem_api_key> python your_script.py

Install the Python SDK

Choose Dev Mode or Test Mode

Simulate a single conversation to quickly debug your Target Agent, by highly customizing how the Maihem Agent behaves.

Simulate many conversations in parallel to generate a test set, and evaluate the performance of your Target Agent across different metrics.

Create config file

Create a config.yaml file, where you will pass the parameters that will define the simulations.

In Dev Mode, you need to define at least one Target Agent and one Maihem Agent.
In Test Mode, you need to define at least one Target Agent and one Test.

Create Maihem script

Before you run the script, you need to set the following:

Select parameters
- Dev Mode or Test Mode
- target_agent_identifier
- (For Dev Mode) maihem_agent_identifier
- (For Test Mode) test_identifier
- Path to config file config_path
Implement chat function (wrap you target agent in the function provided in the script)

Copy the script below and save it as run_maihem.py:

Change the default values in the script or pass them

Run Maihem script

See results in Maihem webapp

Go to your account to visualize the conversations and evaluations.