Using Comet with Tune

Comet is a tool to manage and optimize the entire ML lifecycle, from experiment tracking, model optimization and dataset versioning to model production monitoring.

Example

To illustrate logging your trial results to Comet, we’ll define a simple training function that simulates a loss metric:

import numpy as np
from ray import tune


def train_function(config):
    for i in range(30):
        loss = config["mean"] + config["sd"] * np.random.randn()
        tune.report({"loss": loss})

Now, given that you provide your Comet API key and your project name like so:

api_key = "YOUR_COMET_API_KEY"
project_name = "YOUR_COMET_PROJECT_NAME"

You can add a Comet logger by specifying the callbacks argument in your RunConfig() accordingly:

from ray.air.integrations.comet import CometLoggerCallback

tuner = tune.Tuner(
    train_function,
    tune_config=tune.TuneConfig(
        metric="loss",
        mode="min",
    ),
    run_config=tune.RunConfig(
        callbacks=[
            CometLoggerCallback(
                api_key=api_key, project_name=project_name, tags=["comet_example"]
            )
        ],
    ),
    param_space={"mean": tune.grid_search([1, 2, 3]), "sd": tune.uniform(0.2, 0.8)},
)
results = tuner.fit()

print(results.get_best_result().config)

Tune Comet Logger

Ray Tune offers an integration with Comet through the CometLoggerCallback, which automatically logs metrics and parameters reported to Tune to the Comet UI.

For the full API, see ray.air.integrations.comet.CometLoggerCallback.