Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

MADSci Experiment Manager

Manages experimental runs across a MADSci-powered lab, providing experiment design, tracking, and lifecycle management.

Features

Installation

See the main README for installation options. This package is available as:

Dependencies: MongoDB database (see example_lab)

Usage

Quick Start

Use the example_lab as a starting point:

# Start with working example
docker compose up  # From repo root
# Experiment Manager available at http://localhost:8002/docs

# Or run standalone
python src/madsci_experiment_manager/madsci/experiment_manager/experiment_server.py

Manager Setup

For custom deployments, see example_experiment.manager.yaml for configuration options.

Environment Variables

The Experiment Manager supports configuration via environment variables with the EXPERIMENT_ prefix:

Example:

export EXPERIMENT_SERVER_URL=http://localhost:8002
export EXPERIMENT_DB_URL=mongodb://localhost:27017
export EXPERIMENT_MANAGER_DEFINITION=my_experiment.manager.yaml

Configuration files are also supported: .env, experiments.env, settings.toml, experiments.settings.toml, etc.

Experiment Client

Use ExperimentClient to manage experiments programmatically:

from madsci.client.experiment_client import ExperimentClient
from madsci.common.types.experiment_types import (
    ExperimentDesign,
    ExperimentRegistration,
)

client = ExperimentClient("http://localhost:8002")

# Design an experiment
design = ExperimentDesign(
    experiment_name="Compound Screen Experiment",
    experiment_description="Screen compounds for activity",
    resource_conditions=[]  # Define required resources/conditions
)

# Register and start an experiment
experiment = client.start_experiment(
    experiment_design=design,
    run_name="Screen Run 1",
    run_description="Testing compound A at concentration 10"
)

# Get experiment details
experiment_details = client.get_experiment(experiment.experiment_id)

# Control experiment lifecycle
paused = client.pause_experiment(experiment.experiment_id)
continued = client.continue_experiment(experiment.experiment_id)
ended = client.end_experiment(experiment.experiment_id)

Core Concepts

Experiment Designs

Templates defining experimental parameters and structure:

ExperimentDesign Fields:

ExperimentRegistration Fields:

Experiment Runs

Individual executions of an experiment design:

Experiment States:

Experiment Application

The ExperimentApplication class provides scaffolding for custom experiment logic:

from madsci.experiment_application.experiment_application import ExperimentApplication

class MyExperiment(ExperimentApplication):
    def run_experiment(self, experiment_id: str) -> dict:
        # Custom experimental logic
        # Use other MADSci clients (workcell, data, etc.)
        return {"result": "success"}

app = MyExperiment(experiment_server_url="http://localhost:8002")
app.start()

API Endpoints

The Experiment Manager provides the following REST endpoints:

Experiment Management

Experiment Lifecycle Control

Service Management

Full API documentation is available at http://localhost:8002/docs when the service is running.

Integration with MADSci Ecosystem

The Experiment Manager coordinates with other MADSci components:

Example: See example_lab/ for complete integration examples with all managers working together.

Database Migration Tools

MADSci Experiment Manager includes automated MongoDB migration tools that handle schema changes and version tracking for the experiment management system.

Features

Usage

Standard Usage

# Run migration for experiments database (auto-detects schema file)
python -m madsci.common.mongodb_migration_tool --database madsci_experiments

# Migrate with explicit database URL
python -m madsci.common.mongodb_migration_tool --db-url mongodb://localhost:27017 --database madsci_experiments

# Use custom schema file
python -m madsci.common.mongodb_migration_tool --database madsci_experiments --schema-file /path/to/schema.json

# Create backup only
python -m madsci.common.mongodb_migration_tool --database madsci_experiments --backup-only

# Restore from backup
python -m madsci.common.mongodb_migration_tool --database madsci_experiments --restore-from /path/to/backup

# Check version compatibility without migrating
python -m madsci.common.mongodb_migration_tool --database madsci_experiments --check-version

Docker Usage

When running in Docker containers, use docker-compose to execute migration commands:

# Run migration for experiments database in Docker
docker-compose run --rm experiment-manager python -m madsci.common.mongodb_migration_tool --db-url 'mongodb://mongodb:27017' --database 'madsci_experiments' --schema-file '/app/madsci/experiment_manager/schema.json'

# Create backup only in Docker
docker-compose run --rm experiment-manager python -m madsci.common.mongodb_migration_tool --db-url 'mongodb://mongodb:27017' --database 'madsci_experiments' --schema-file '/app/madsci/experiment_manager/schema.json' --backup-only

# Check version compatibility in Docker
docker-compose run --rm experiment-manager python -m madsci.common.mongodb_migration_tool --db-url 'mongodb://mongodb:27017' --database 'madsci_experiments' --schema-file '/app/madsci/experiment_manager/schema.json' --check-version

Server Integration

The Experiment Manager server automatically checks for version compatibility on startup. If a mismatch is detected, the server will refuse to start and display migration instructions:

DATABASE INITIALIZATION REQUIRED! SERVER STARTUP ABORTED!
The database exists but needs version tracking setup.
To resolve this issue, run the migration tool and restart the server.

Schema File Location

The migration tool automatically searches for schema files in:

Backup Location

Backups are stored in .madsci/mongodb/backups/ with timestamped filenames:

Requirements

Manager Services
MADSci Event Manager
Manager Services
MADSci Resource Manager