Quickstart
Get started with Maihem in 5 minutes
What is Maihem?
Maihem creates AI agents that automatically test your LLM applications, by simulating conversations between your Target Agent and Maihem Agents.
The benefits of Maihem are:
Develop 10x times faster by automatically finding failures
Build LLM applications that you and others can trust
Get started
Get access
Book a call to get access to the Maihem platform.
Generate API key
Sign into your Maihem account and generate an API key by going to your account settings.
Set API key as environment variable
Before using the maihem
package, you need to set your maihem API key as an environment variable:
Environment variable: MAIHEM_API_KEY
Install the Python SDK
Choose Dev Mode or Test Mode
Dev Mode
Simulate a single conversation to quickly debug your Target Agent, by highly customizing how the Maihem Agent behaves.
Test Mode
Simulate many conversations in parallel to generate a test set, and evaluate the performance of your Target Agent across different metrics.
Create config file
Create a config.yaml
file, where you will pass the parameters that will define the simulations.
-
In
Dev Mode
, you need to define at least oneTarget Agent
and oneMaihem Agent
. -
In
Test Mode
, you need to define at least oneTarget Agent
and oneTest
.
Create Maihem script
Before you run the script, you need to set the following:
- Select parameters
Dev Mode
orTest Mode
target_agent_identifier
- (For Dev Mode)
maihem_agent_identifier
- (For Test Mode)
test_identifier
- Path to config file
config_path
- Implement chat function (wrap you target agent in the function provided in the script)
Copy the script below and save it as run_maihem.py
:
Change the default values in the script or pass them
Run Maihem script
See results in Maihem webapp
Go to your account to visualize the conversations and evaluations.