Quickstart

Set up Maihem

Generate API key

Generate API key in your Maihem account settings.

Book a call to get access to the Maihem platform

Set API key as environment variable

Set your Maihem API key as an environment variable called MAIHEM_API_KEY.

Option 1: Pass in Python script

Easiest option for quick prototyping, but not the safest to store your API keys.

In your Python code:

Python
import os

os.environ["MAIHEM_API_KEY"] = "<YOUR_API_KEY>"

Option 2: Local environment .env file

Safest for storing API keys in production.

Create a .env file with the following content:

.env file
MAIHEM_API_KEY=<your_maihem_api_key>

Load the environment variables from the .env file in Python:

Python
from dotenv import load_dotenv

load_dotenv()

Option 3: In local bash script

To make the environment variable available to any script locally.

Install the Python SDK

This assumes you have Python installed. See this guide to install Python if needed.

Command line interface (CLI)
pip install --upgrade maihem

Simulate and evaluate a conversation

Create target agent

In the terminal run the following command to configure your target agent. This will also create a folder in the current directory.

Command line interface (CLI)
maihem target-agent create \
--name "demo-tutor-agent" \
--role "AI tutor" \
--description "A history tutor that helps students prepare for exams with practice questions and summaries."

Go to the new folder demo-tutor-agent, where there are some pre-populated Python scripts.

Command line interface (CLI)
cd demo-tutor-agent

Create test

Create a test using the Customer Experience (CX) module. A module defines the scope of a test.

Command line interface (CLI)
maihem test create \
--name "test-1" \
--target-agent-name "demo-tutor-agent" \
--modules "cx" \
--maihem-behavior-prompt "Request quizz-style questions and deep dives in bullet points for identified gaps" \
--number-conversations 4

Connect target agent with wrapper function

To connect your target agent with Maihem, you need to pass a function to call your target agent. File wrapper_function.py contains the function wrapper_function to call your target agent.

wrapper_function.py
import requests
from typing import List, Tuple

def wrapper_function(
	conversation_id: str,
	maihem_agent_message: str | None,
	conversation_history: dict,
) -> Tuple[str, List[str]]:
	"""Callable function to wrap your target agent to be tested."""

	# Call demo Maihem target agent for quickstart
	url = "https://demo-agent.maihem.ai/chat"
	payload = {"message": maihem_agent_message if maihem_agent_message else ""}
	response = requests.request("POST", url, json=payload).json()
	return response["message"], response["contexts"]

Run test

Command line interface (CLI)
maihem test run \
--name "modelX_prompt2.5_28-11-2024" \
--test-name="test-1"

See results in Maihem webapp

Go to your Maihem account to see the results of your first test.

Next step: see our how-to guides

A test simulates conversations to comprehensively evaluate your target agent and find failures

Test RAG

Upload documents to test hallucinations and answer relevance

Test customer experience

Test goal completion for simulated personas with custom behaviors and goals

Get started

How-to guides

Reference

Set up Maihem

Generate API key

Set API key as environment variable

Option 1: Pass in Python script

Option 2: Local environment .env file

Option 3: In local bash script

Install the Python SDK

Simulate and evaluate a conversation

Create target agent

Create test

Connect target agent with wrapper function

Run test

See results in Maihem webapp

Next step: see our how-to guides

Test RAG

Test customer experience

Get started

How-to guides

Reference

​Set up Maihem

Generate API key

Set API key as environment variable

Option 1: Pass in Python script

Option 2: Local environment .env file

Option 3: In local bash script

Install the Python SDK

​Simulate and evaluate a conversation

Create target agent

Create test

Connect target agent with wrapper function

Run test

See results in Maihem webapp

​Next step: see our how-to guides

Test RAG

Test customer experience

Set up Maihem

Simulate and evaluate a conversation

Next step: see our how-to guides