This example is adapted from Continual AI Avalanche quick start https://avalanche.continualai.org/

Incremental Learning with Ray AIRΒΆ

In this example, we show how to use Ray AIR to incrementally train a simple image classification PyTorch model on a stream of incoming tasks.

Each task is a random permutation of the MNIST Dataset, which is a common benchmark used for continual training. After training on all the tasks, the model is expected to be able to make predictions on data from any task.

In this example, we use just a naive finetuning strategy, where the model is trained on each task, without any special methods to prevent catastrophic forgetting. Model performance is expected to be poor.

More precisely, this example showcases domain incremental training, in which during prediction/testing time, the model is asked to predict on data from tasks trained on so far with the task ID not provided. This is opposed to task incremental training, where the task ID is provided during prediction/testing time.

For more information on the 3 different categories for incremental/continual learning, please see β€œThree scenarios for continual learning” by van de Ven and Tolias

This example will cover the following:

  1. Loading a PyTorch Dataset to Ray Datasets

  2. Create an Iterator[ray.data.Datasets] abstraction to represent a stream of data to train on for incremental training.

  3. Implement a custom Ray AIR preprocessor to preprocess the Dataset.

  4. Incrementally train a model using data parallel training.

  5. Use our trained model to perform batch prediction on test data.

  6. Incrementally deploying our trained model with Ray Serve and performing online prediction queries.

Step 1: Installations and Initializing RayΒΆ

To get started, let’s first install the necessary packages: Ray AIR, torch, and torchvision. Uncomment the below lines and run the cell to install the necessary packages.

# !pip install -q "ray[air]"
# !pip install -q torch
# !pip install -q torchvision
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Requirement already satisfied: ray[data,serve,tune] in /usr/local/lib/python3.7/dist-packages (2.0.0.dev0)
Requirement already satisfied: numpy>=1.16 in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (1.21.6)
Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (3.7.0)
Requirement already satisfied: grpcio!=1.44.0,>=1.28.1 in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (1.43.0)
Requirement already satisfied: msgpack<2.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (1.0.3)
Requirement already satisfied: protobuf>=3.15.3 in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (3.17.3)
Requirement already satisfied: aiosignal in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (1.2.0)
Requirement already satisfied: frozenlist in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (1.3.0)
Requirement already satisfied: virtualenv in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (20.14.1)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (3.13)
Requirement already satisfied: attrs in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (21.4.0)
Requirement already satisfied: click<=8.0.4,>=7.0 in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (7.1.2)
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (2.23.0)
Requirement already satisfied: jsonschema in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (4.3.3)
Requirement already satisfied: pyarrow<7.0.0,>=6.0.1 in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (6.0.1)
Requirement already satisfied: fsspec in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (2022.5.0)
Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (1.3.5)
Requirement already satisfied: tensorboardX>=1.9 in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (2.5)
Requirement already satisfied: tabulate in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (0.8.9)
Requirement already satisfied: aiorwlock in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (1.3.0)
Requirement already satisfied: starlette in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (0.19.1)
Requirement already satisfied: prometheus-client<0.14.0,>=0.7.1 in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (0.13.1)
Requirement already satisfied: py-spy>=0.2.0 in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (0.3.12)
Requirement already satisfied: smart-open in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (6.0.0)
Requirement already satisfied: gpustat>=1.0.0b1 in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (1.0.0b1)
Requirement already satisfied: colorful in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (0.5.4)
Requirement already satisfied: aiohttp>=3.7 in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (3.8.1)
Requirement already satisfied: fastapi in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (0.78.0)
Requirement already satisfied: aiohttp-cors in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (0.7.0)
Requirement already satisfied: uvicorn==0.16.0 in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (0.16.0)
Requirement already satisfied: opencensus in /usr/local/lib/python3.7/dist-packages (from ray[data,serve,tune]) (0.9.0)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from uvicorn==0.16.0->ray[data,serve,tune]) (4.2.0)
Requirement already satisfied: asgiref>=3.4.0 in /usr/local/lib/python3.7/dist-packages (from uvicorn==0.16.0->ray[data,serve,tune]) (3.5.2)
Requirement already satisfied: h11>=0.8 in /usr/local/lib/python3.7/dist-packages (from uvicorn==0.16.0->ray[data,serve,tune]) (0.13.0)
Requirement already satisfied: asynctest==0.13.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp>=3.7->ray[data,serve,tune]) (0.13.0)
Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp>=3.7->ray[data,serve,tune]) (2.0.12)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.7/dist-packages (from aiohttp>=3.7->ray[data,serve,tune]) (6.0.2)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.7/dist-packages (from aiohttp>=3.7->ray[data,serve,tune]) (4.0.2)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.7/dist-packages (from aiohttp>=3.7->ray[data,serve,tune]) (1.7.2)
Requirement already satisfied: psutil in /usr/local/lib/python3.7/dist-packages (from gpustat>=1.0.0b1->ray[data,serve,tune]) (5.4.8)
Requirement already satisfied: six>=1.7 in /usr/local/lib/python3.7/dist-packages (from gpustat>=1.0.0b1->ray[data,serve,tune]) (1.15.0)
Requirement already satisfied: blessed>=1.17.1 in /usr/local/lib/python3.7/dist-packages (from gpustat>=1.0.0b1->ray[data,serve,tune]) (1.19.1)
Requirement already satisfied: nvidia-ml-py3>=7.352.0 in /usr/local/lib/python3.7/dist-packages (from gpustat>=1.0.0b1->ray[data,serve,tune]) (7.352.0)
Requirement already satisfied: wcwidth>=0.1.4 in /usr/local/lib/python3.7/dist-packages (from blessed>=1.17.1->gpustat>=1.0.0b1->ray[data,serve,tune]) (0.2.5)
Requirement already satisfied: idna>=2.0 in /usr/local/lib/python3.7/dist-packages (from yarl<2.0,>=1.0->aiohttp>=3.7->ray[data,serve,tune]) (2.10)
Requirement already satisfied: pydantic!=1.7,!=1.7.1,!=1.7.2,!=1.7.3,!=1.8,!=1.8.1,<2.0.0,>=1.6.2 in /usr/local/lib/python3.7/dist-packages (from fastapi->ray[data,serve,tune]) (1.9.1)
Requirement already satisfied: anyio<5,>=3.4.0 in /usr/local/lib/python3.7/dist-packages (from starlette->ray[data,serve,tune]) (3.6.1)
Requirement already satisfied: sniffio>=1.1 in /usr/local/lib/python3.7/dist-packages (from anyio<5,>=3.4.0->starlette->ray[data,serve,tune]) (1.2.0)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.7/dist-packages (from jsonschema->ray[data,serve,tune]) (5.7.1)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.7/dist-packages (from jsonschema->ray[data,serve,tune]) (0.18.1)
Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from jsonschema->ray[data,serve,tune]) (4.11.3)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.7/dist-packages (from importlib-resources>=1.4.0->jsonschema->ray[data,serve,tune]) (3.8.0)
Requirement already satisfied: google-api-core<3.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from opencensus->ray[data,serve,tune]) (1.31.5)
Requirement already satisfied: opencensus-context>=0.1.2 in /usr/local/lib/python3.7/dist-packages (from opencensus->ray[data,serve,tune]) (0.1.2)
Requirement already satisfied: pytz in /usr/local/lib/python3.7/dist-packages (from google-api-core<3.0.0,>=1.0.0->opencensus->ray[data,serve,tune]) (2022.1)
Requirement already satisfied: google-auth<2.0dev,>=1.25.0 in /usr/local/lib/python3.7/dist-packages (from google-api-core<3.0.0,>=1.0.0->opencensus->ray[data,serve,tune]) (1.35.0)
Requirement already satisfied: googleapis-common-protos<2.0dev,>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from google-api-core<3.0.0,>=1.0.0->opencensus->ray[data,serve,tune]) (1.56.1)
Requirement already satisfied: setuptools>=40.3.0 in /usr/local/lib/python3.7/dist-packages (from google-api-core<3.0.0,>=1.0.0->opencensus->ray[data,serve,tune]) (57.4.0)
Requirement already satisfied: packaging>=14.3 in /usr/local/lib/python3.7/dist-packages (from google-api-core<3.0.0,>=1.0.0->opencensus->ray[data,serve,tune]) (21.3)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<2.0dev,>=1.25.0->google-api-core<3.0.0,>=1.0.0->opencensus->ray[data,serve,tune]) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth<2.0dev,>=1.25.0->google-api-core<3.0.0,>=1.0.0->opencensus->ray[data,serve,tune]) (4.8)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<2.0dev,>=1.25.0->google-api-core<3.0.0,>=1.0.0->opencensus->ray[data,serve,tune]) (4.2.4)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=14.3->google-api-core<3.0.0,>=1.0.0->opencensus->ray[data,serve,tune]) (3.0.9)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<2.0dev,>=1.25.0->google-api-core<3.0.0,>=1.0.0->opencensus->ray[data,serve,tune]) (0.4.8)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->ray[data,serve,tune]) (1.24.3)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->ray[data,serve,tune]) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->ray[data,serve,tune]) (2022.5.18.1)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas->ray[data,serve,tune]) (2.8.2)
Requirement already satisfied: distlib<1,>=0.3.1 in /usr/local/lib/python3.7/dist-packages (from virtualenv->ray[data,serve,tune]) (0.3.4)
Requirement already satisfied: platformdirs<3,>=2 in /usr/local/lib/python3.7/dist-packages (from virtualenv->ray[data,serve,tune]) (2.5.2)
Found existing installation: ray 2.0.0.dev0
Uninstalling ray-2.0.0.dev0:
  Successfully uninstalled ray-2.0.0.dev0
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting ray==3.0.0.dev0
  Downloading https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-3.0.0.dev0-cp37-cp37m-manylinux2014_x86_64.whl (54.9 MB)
     |β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 54.9 MB 74.4 MB/s 
?25hRequirement already satisfied: msgpack<2.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (1.0.3)
Requirement already satisfied: virtualenv in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (20.14.1)
Requirement already satisfied: protobuf>=3.15.3 in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (3.17.3)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (3.13)
Requirement already satisfied: click<=8.0.4,>=7.0 in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (7.1.2)
Requirement already satisfied: attrs in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (21.4.0)
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (2.23.0)
Requirement already satisfied: frozenlist in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (1.3.0)
Requirement already satisfied: aiosignal in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (1.2.0)
Requirement already satisfied: numpy>=1.16 in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (1.21.6)
Requirement already satisfied: jsonschema in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (4.3.3)
Requirement already satisfied: grpcio<=1.43.0,>=1.28.1 in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (1.43.0)
Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from ray==3.0.0.dev0) (3.7.0)
Requirement already satisfied: six>=1.5.2 in /usr/local/lib/python3.7/dist-packages (from grpcio<=1.43.0,>=1.28.1->ray==3.0.0.dev0) (1.15.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.7/dist-packages (from jsonschema->ray==3.0.0.dev0) (0.18.1)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from jsonschema->ray==3.0.0.dev0) (4.2.0)
Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from jsonschema->ray==3.0.0.dev0) (4.11.3)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.7/dist-packages (from jsonschema->ray==3.0.0.dev0) (5.7.1)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.7/dist-packages (from importlib-resources>=1.4.0->jsonschema->ray==3.0.0.dev0) (3.8.0)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->ray==3.0.0.dev0) (2.10)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->ray==3.0.0.dev0) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->ray==3.0.0.dev0) (1.24.3)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->ray==3.0.0.dev0) (2022.5.18.1)
Requirement already satisfied: platformdirs<3,>=2 in /usr/local/lib/python3.7/dist-packages (from virtualenv->ray==3.0.0.dev0) (2.5.2)
Requirement already satisfied: distlib<1,>=0.3.1 in /usr/local/lib/python3.7/dist-packages (from virtualenv->ray==3.0.0.dev0) (0.3.4)
Installing collected packages: ray
Successfully installed ray-3.0.0.dev0
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Requirement already satisfied: torch in /usr/local/lib/python3.7/dist-packages (1.11.0+cu113)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from torch) (4.2.0)
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Requirement already satisfied: torchvision in /usr/local/lib/python3.7/dist-packages (0.12.0+cu113)
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from torchvision) (2.23.0)
Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /usr/local/lib/python3.7/dist-packages (from torchvision) (7.1.2)
Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from torchvision) (1.21.6)
Requirement already satisfied: torch==1.11.0 in /usr/local/lib/python3.7/dist-packages (from torchvision) (1.11.0+cu113)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from torchvision) (4.2.0)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->torchvision) (2.10)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->torchvision) (2022.5.18.1)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->torchvision) (3.0.4)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->torchvision) (1.24.3)

Then, let’s initialize Ray! We can just import and call ray.init(). If you are running on a Ray cluster, then you can do ray.init("auto") to connect to the cluster instead of initiailzing a new local Ray instance.

import ray
ray.init()
# If runnning on a cluster, use the below line instead.
# ray.init("auto")
2022-05-25 22:25:31,150	INFO services.py:1483 -- View the Ray dashboard at http://127.0.0.1:8265
RayContext(dashboard_url='127.0.0.1:8265', python_version='3.7.13', ray_version='3.0.0.dev0', ray_commit='ac620aeec0c0f68c92328ace0b2a5835f5b14b26', address_info={'node_ip_address': '172.28.0.2', 'raylet_ip_address': '172.28.0.2', 'redis_address': None, 'object_store_address': '/tmp/ray/session_2022-05-25_22-25-28_641559_1518/sockets/plasma_store', 'raylet_socket_name': '/tmp/ray/session_2022-05-25_22-25-28_641559_1518/sockets/raylet', 'webui_url': '127.0.0.1:8265', 'session_dir': '/tmp/ray/session_2022-05-25_22-25-28_641559_1518', 'metrics_export_port': 61030, 'gcs_address': '172.28.0.2:62940', 'address': '172.28.0.2:62940', 'node_id': '97455d0de12f3393126427ed2b1ef0a009f0bd3fb97177cb86b42d92'})

Step 2: Define our PyTorch ModelΒΆ

Now that we have the necessary installations, let’s define our PyTorch model. For this example to classify MNIST images, we will use a simple multi-layer perceptron.

import torch.nn as nn

class SimpleMLP(nn.Module):
    def __init__(self, num_classes=10, input_size=28 * 28):
        super(SimpleMLP, self).__init__()

        self.features = nn.Sequential(
          nn.Linear(input_size, 512),
          nn.ReLU(inplace=True),
          nn.Dropout(),
        )
        self.classifier = nn.Linear(512, num_classes)
        self._input_size = input_size

    def forward(self, x):
        x = x.contiguous()
        x = x.view(-1, self._input_size)
        x = self.features(x)
        x = self.classifier(x)
        return x

Step 3: Create the Stream of tasksΒΆ

We can now create a stream of tasks (where each task contains a dataset to train on). For this example, we will create an artificial stream of tasks consisting of permuted variations of MNIST, which is a classic benchmark in continual learning research.

For real-world scenarios, this step is not necessary as fresh data will already be arriving as a stream of tasks. It does not need to be artificially created.

3a: Load MNIST Dataset to a Ray DatasetΒΆ

Let’s first define a simple function that will return the original MNIST Dataset as a distributed Ray Dataset. Ray Datasets are the standard way to load and exchange data in Ray libraries and applications, read more about them here!

The function in the below code snippet does the following:

  1. Downloads the MNIST Dataset from torchvision in-memory

  2. Loads the in-memory Torch Dataset into a Ray Dataset

  3. Converts the Ray Dataset into a Pandas format. Instead of the Ray Dataset iterating over tuples, it will have 2 columns: β€œimage” & β€œlabel”.

This will allow us to apply built-in preprocessors to the Ray Dataset and allow Ray Datasets to be used with Ray AIR Predictors.

For this example, since we are just working with MNIST dataset, which is small, we use the SimpleTorchDataSource which just loads the full MNIST dataset into memory.

For loading larger datasets in a parallel fashion, you should use Ray Dataset’s additional read APIs to load data from parquet, csv, image files, and more!

import pandas as pd

import torchvision
from torchvision.transforms import RandomCrop

import ray
from ray.data.datasource.torch_datasource import SimpleTorchDatasource


def get_mnist_dataset(train: bool = True) -> ray.data.Dataset:
    """Returns MNIST Dataset as a ray.data.Dataset.
    
    Args:
        train: Whether to return the train dataset or test dataset.
    """

    def mnist_dataset_factory():
        if train:
            # Only perform random cropping on the Train dataset.
            transform = RandomCrop(28, padding=4)
        else:
            transform = None
        return torchvision.datasets.MNIST("./data", download=True, train=train, transform=transform)

    def convert_batch_to_pandas(batch):
        images = [np.array(item[0]) for item in batch]
        labels = [item[1] for item in batch]

        df = pd.DataFrame({"image": images, "label": labels})

        return df

    mnist_dataset = ray.data.read_datasource(
        SimpleTorchDatasource(), dataset_factory=mnist_dataset_factory
    )
    mnist_dataset = mnist_dataset.map_batches(convert_batch_to_pandas)
    return mnist_dataset

3b: Create our Stream abstractionΒΆ

Now we can create our β€œstream” abstraction. This abstraction provides two methods (generate_train_stream and generate_test_stream) that each returns an Iterator over Ray Datasets. Each item in this iterator contains a unique permutation of MNIST, and is one task that we want to train on.

In this example, β€œthe stream of tasks” is contrived since all the data for all tasks exist already in an offline setting. For true online continual learning, you would want to implement a custom dataset iterator that reads from some stream datasource to produce new tasks. The only abstraction that’s needed is Iterator[ray.data.Dataset].

Note that the test dataset stream has the same permutations that are used for the training dataset stream. In general for continual learning, it is expected that the data distribution of the test/prediction data follows what the model was trained on. If you notice that the distribution of new prediction queries is changing compared to the distribution of the training data, then you should probably trigger training of a new task.

from typing import Iterator, List
import random
import numpy as np

from ray.data import ActorPoolStrategy


class PermutedMNISTStream:
    """Generates streams of permuted MNIST Datasets.
    
    Example:
        
        permuted_mnist = PermutedMNISTStream(n_tasks=3)
        train_stream = permuted_mnist.generate_train_stream()
        
        # Iterate through the train_stream
        for train_dataset in train_stream:
            ...
    
    Args:
        n_tasks: The number of tasks to generate.
    """

    def __init__(self, n_tasks: int = 3):
        self.n_tasks = n_tasks
        self.permutations = [
            np.random.permutation(28 * 28) for _ in range(self.n_tasks)
        ]

        self.train_mnist_dataset = get_mnist_dataset(train=True)
        self.test_mnist_dataset = get_mnist_dataset(train=False)

    def random_permute_dataset(
        self, dataset: ray.data.Dataset, permutation: np.ndarray
    ):
        """Randomly permutes the pixels for each image in the dataset."""

        class PixelsPermutation(object):
            def __call__(self, batch):
                batch["image"] = batch["image"].map(lambda image: image.reshape(-1)[permutation].reshape(28, 28))
                return batch

        return dataset.map_batches(PixelsPermutation, compute=ActorPoolStrategy(), batch_format="pandas")

    def generate_train_stream(self) -> Iterator[ray.data.Dataset]:
        for permutation in self.permutations:
            permuted_mnist_dataset = self.random_permute_dataset(
                self.train_mnist_dataset, permutation
            )
            yield permuted_mnist_dataset

    def generate_test_stream(self) -> Iterator[ray.data.Dataset]:
        for permutation in self.permutations:
            mnist_dataset = get_mnist_dataset(train=False)
            permuted_mnist_dataset = self.random_permute_dataset(
                self.test_mnist_dataset, permutation
            )
            yield permuted_mnist_dataset

    def generate_test_samples(self, num_samples: int = 10) -> List[np.ndarray]:
      """Generates num_samples permuted MNIST images."""
      random_permutation =  random.choice(self.permutations)
      return self.random_permute_dataset(self.test_mnist_dataset.random_shuffle().limit(num_samples), random_permutation).to_pandas()["image"].to_list()

Step 4: Define the logic for Training and Inference/PredictionΒΆ

Now that we can get an Iterator over Ray Datasets, we can incrementally train our model in a data parallel fashion via Ray Train, while incrementally deploying our model via Ray Serve. Let’s define some helper functions to allow us to do this!

If you are not familiar with data parallel training, it is a form of distributed training strategies, where we have multiple model replicas, and each replica trains on a different batch of data. After each batch, the gradients are synchronized across the replicas. This effecitively allows us to train on more data in a shorter amount of time.

4a: Define our training logic for each Data Parallel workerΒΆ

The first thing we need to do is to define the training loop that will be run on each training worker.

The training loop takes in a config Dict as an argument that we can use to pass in any configurations for training.

This is just standard PyTorch training, with the difference being that we can leverage Ray Train’s utility functions:

  • ray.train.torch.prepare_model(...): This will prepare the model for distributed training by wrapping it in PyTorch DistributedDataParallel and moving it to the correct accelerator device.

  • ray.train.get_dataset_shard(...): This will get the Ray Dataset shard for this particular Data Parallel worker.

  • ray.train.save_checkpoint(...): This will tell Ray Train to save the provided arguments as a checkpoint. Checkpoints will be written to disk under the ~/ray_results directory.

  • ray.train.load_checkpoint(): Returns a checkpoint to resume from. This is useful for either fault tolerance purposes, or for our purposes, to continue training the same model on a new incoming dataset.

from ray import train

from torch.optim import SGD
from torch.nn import CrossEntropyLoss

from torch.nn.modules.utils import consume_prefix_in_state_dict_if_present

def train_loop_per_worker(config: dict):
    num_epochs = config["num_epochs"]
    learning_rate = config["learning_rate"]
    momentum = config["momentum"]
    batch_size = config["batch_size"]

    model = SimpleMLP(num_classes=10)

    # Load model from checkpoint if there is a checkpoint to load from.
    checkpoint_to_load = train.load_checkpoint()
    if checkpoint_to_load:
        state_dict_to_resume_from = checkpoint_to_load["model"]
        model.load_state_dict(state_dict=state_dict_to_resume_from)

    model = train.torch.prepare_model(model)

    optimizer = SGD(model.parameters(), lr=learning_rate, momentum=momentum)
    criterion = CrossEntropyLoss()

    # Get the Ray Dataset shard for this data parallel worker, and convert it to a PyTorch Dataset.
    dataset_shard = train.get_dataset_shard("train").to_torch(
        label_column="label",
        batch_size=batch_size,
        unsqueeze_feature_tensors=False,
        unsqueeze_label_tensor=False,
    )

    for epoch_idx in range(num_epochs):
        for iteration, (train_mb_x, train_mb_y) in enumerate(dataset_shard):
            optimizer.zero_grad()
            train_mb_x = train_mb_x.to(train.torch.get_device())
            train_mb_y = train_mb_y.to(train.torch.get_device())

            # Forward
            logits = model(train_mb_x)
            # Loss
            loss = criterion(logits, train_mb_y)
            # Backward
            loss.backward()
            # Update
            optimizer.step()

            if train.world_rank() == 0 and iteration % 500 == 0:
                print(f"loss: {loss.item():>7f}, epoch: {epoch_idx}, iteration: {iteration}")

        # Checkpoint model after every epoch.
        state_dict = model.state_dict()
        consume_prefix_in_state_dict_if_present(state_dict, "module.")
        train.save_checkpoint(model=state_dict)

4b: Define our PreprocessorΒΆ

Next, we define our Preprocessor to preprocess our data before training and prediction. Our preprocessor will normalize the MNIST Images by the mean and standard deviation of the MNIST training dataset. This is a common operation to do on MNIST to improve training: https://discuss.pytorch.org/t/normalization-in-the-mnist-example/457

from ray.data.preprocessors import BatchMapper

from torchvision import transforms

def preprocess_images(df: pd.DataFrame) -> pd.DataFrame:
    """Preprocess images by scaling each channel in the image."""

    torchvision_transforms = transforms.Compose(
      [transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))]
    )

    df["image"] = df["image"].map(torchvision_transforms)
    return df

mnist_normalize_preprocessor = BatchMapper(fn=preprocess_images)

4c: Define logic for Batch/Offline Prediction.ΒΆ

After training on each task, we want to use our trained model to do batch (i.e. offline) inference on a test dataset.

To do this, we leverage the built-in ray.air.BatchPredictor. We define a batch_predict function that will take in a Checkpoint and a Test Dataset and outputs the accuracy our model achieves on the test dataset.

from ray.train.batch_predictor import BatchPredictor
from ray.train.torch import TorchPredictor

def batch_predict(checkpoint: ray.air.Checkpoint, test_dataset: ray.data.Dataset) -> float:
  """Perform batch prediction on the provided test dataset, and return accuracy results."""

  batch_predictor = BatchPredictor.from_checkpoint(checkpoint, predictor_cls=TorchPredictor, model=SimpleMLP(num_classes=10))
  model_output = batch_predictor.predict(
            data=test_dataset, feature_columns=["image"], unsqueeze=False
        )
  
  # Postprocess model outputs.
  # Convert logits outputted from model into actual class predictions.
  def convert_logits_to_classes(df):
     best_class = df["predictions"].map(lambda x: np.array(x).argmax())
     df["predictions"] = best_class
     return df
    
  prediction_results = model_output.map_batches(convert_logits_to_classes, batch_format="pandas")
  
  # Then, for each prediction output, see if it matches with the ground truth
  # label.
  zipped_dataset = test_dataset.zip(prediction_results)

  def calculate_prediction_scores(df):
      return pd.DataFrame({"correct": df["predictions"] == df["label"]})

  correct_dataset = zipped_dataset.map_batches(
      calculate_prediction_scores, batch_format="pandas"
  )

  return correct_dataset.sum(on="correct") / correct_dataset.count()

4d: Define logic for Deploying and Querying our modelΒΆ

In addition to batch inference, we also want to deploy our model so that we can submit live queries to it for online inference. We use Ray Serve’s ModelWrapperDeployment utility to deploy our trained model.

Once we deploy the model, we can send HTTP requests to our deployment.

from typing import List
import requests
from requests import Response
import numpy as np

from ray.serve.http_adapters import NdArray


def deploy_model(checkpoint: ray.air.Checkpoint) -> str:
  """Deploys the model from the provided Checkpoint and returns the URL for the endpoint of the model deployment."""
  def json_to_pandas(payload: NdArray) -> pd.DataFrame:
      """Accepts an NdArray JSON from an HTTP body and converts it to a Pandas dataframe."""
      # Have to explicitly convert to float since np.array reads as a double.
      arr = np.array(payload.array, dtype=np.float32)
      # We have to specify an image column since our preprocessor requires it.
      df = pd.DataFrame({"image": [arr]})
      return df

  deployment = ModelWrapperDeployment.options(name="mnist_model", route_prefix="/mnist_predict", version=f"v{task_idx}", num_replicas=2)
  deployment.deploy(batching_params=False, http_adapter=json_to_pandas, predictor_cls=TorchPredictor, checkpoint=latest_checkpoint, model=SimpleMLP(num_classes=10))
  return deployment.url

# Function that queries our deployed model
def query_deployment(test_samples: List[np.ndarray], endpoint_uri: str) -> List[Response]:
  """Given a set of test samples, queries the model deployment at the provided endpoint and returns the results."""
  results = []
  # Have to convert to Python List since Numpy arrays are not Json serializable.
  for sample in test_samples:
    results.append(requests.post(endpoint_uri, json={"array": sample.tolist()}))
  # TODO: Figure out how Serve deals with Pandas DataFrame returned by Predictors.
  return results

Step 5: Putting it all togetherΒΆ

Once we have defined our training logic and our preprocessor, we can put everything together!

For each dataset in our stream, we do the following:

  1. Train on the dataset in Data Parallel fashion. We create a TorchTrainer, specify the config for the training loop we defined above, the dataset to train on, and how much we want to scale. TorchTrainer also accepts a checkpoint arg to continue training from a previously saved checkpoint.

  2. Get the saved checkpoint from the training run.

  3. Test our trained model on a test set containing test data from all the tasks trained on so far.

  4. After training on each task, we deploy our model so we can query it for predictions.

In this example, the training and test data for each task is well-defined beforehand by the benchmark. For real-world scenarios, this probably will not be the case. It is very likely that the prediction requests after training on one task will become the training data for the next task.

from ray.train.torch import TorchTrainer
from ray.train.torch import TorchPredictor
from ray.air import Checkpoint
from ray import serve
from ray.serve.model_wrappers import ModelWrapperDeployment
from ray.serve.http_adapters import json_to_ndarray

# The number of tasks (i.e. datasets in our stream) that we want to use for this example.
n_tasks = 3

# Number of epochs to train each task for.
num_epochs = 4
# Batch size.
batch_size = 32
# Optimizer args.
learning_rate = 0.001
momentum = 0.9

# Number of data parallel workers to use for training.
num_workers = 1
# Whether to use GPU or not.
use_gpu = True

permuted_mnist = PermutedMNISTStream(n_tasks=n_tasks)
train_stream = permuted_mnist.generate_train_stream()
test_stream = permuted_mnist.generate_test_stream()

latest_checkpoint = None

accuracy_for_all_tasks = []
task_idx = 0
all_test_datasets_seen_so_far = []
for train_dataset, test_dataset in zip(train_stream, test_stream):
  print(f"Starting training for task: {task_idx}")
  task_idx += 1

  # *********Training*****************

  trainer = TorchTrainer(
          train_loop_per_worker=train_loop_per_worker,
          train_loop_config={
              "num_epochs": num_epochs,
              "learning_rate": learning_rate,
              "momentum": momentum,
              "batch_size": batch_size,
          },
          # Have to specify trainer_resources as 0 so that the example works on Colab. 
          scaling_config={"num_workers": num_workers, "use_gpu": use_gpu, "trainer_resources": {"CPU": 0}},
          datasets={"train": train_dataset},
          preprocessor=BatchMapper(fn=preprocess_images),
          resume_from_checkpoint=latest_checkpoint,
      )
  result = trainer.fit()
  latest_checkpoint = result.checkpoint

  # **************Batch Prediction**************************

  # We can do batch prediction on the test data for the tasks seen so far.
  # TODO: Fix type signature in Ray Datasets
  # TODO: Fix dataset.union when used with empty list.
  if len(all_test_datasets_seen_so_far) > 0:
    full_test_dataset = test_dataset.union(*all_test_datasets_seen_so_far)
  else:
    full_test_dataset = test_dataset

  all_test_datasets_seen_so_far.append(test_dataset)

  accuracy_for_this_task = batch_predict(latest_checkpoint, full_test_dataset)
  print(f"Accuracy for task {task_idx}: {accuracy_for_this_task}")
  accuracy_for_all_tasks.append(accuracy_for_this_task)

  # *************Model Deployment & Online Inference***************************
  
  # We can also deploy our model to do online inference with Ray Serve.
  # Start Ray Serve.
  serve.start()
  test_samples = permuted_mnist.generate_test_samples()
  endpoint_uri = deploy_model(latest_checkpoint)
  online_inference_results = query_deployment(test_samples, endpoint_uri)

  if ray.available_resources().get("CPU", 0) < num_workers+1:
    # If there are no more CPUs left, then shutdown the Serve replicas so we can continue training on the next task.
    serve.shutdown()

  
serve.shutdown()
(_prepare_read pid=1772) 2022-05-25 22:25:35,236	WARNING torch_datasource.py:56 -- `SimpleTorchDatasource` doesn't support parallel reads. The `parallelism` argument will be ignored.
Read->Map_Batches: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:05<00:00,  5.92s/it]
(_prepare_read pid=1772) 2022-05-25 22:25:53,593	WARNING torch_datasource.py:56 -- `SimpleTorchDatasource` doesn't support parallel reads. The `parallelism` argument will be ignored.
Read->Map_Batches: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:00<00:00,  2.51it/s]
Map Progress (2 actors 1 pending): 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:02<00:00,  2.72s/it]
(_prepare_read pid=1978) 2022-05-25 22:25:58,761	WARNING torch_datasource.py:56 -- `SimpleTorchDatasource` doesn't support parallel reads. The `parallelism` argument will be ignored.
Read->Map_Batches: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:00<00:00,  2.41it/s]
Map Progress (1 actors 1 pending): 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:01<00:00,  1.37s/it]
Starting training for task: 0

== Status ==
Current time: 2022-05-25 22:27:16 (running for 00:01:14.46)
Memory usage on this node: 4.7/12.7 GiB
Using FIFO scheduling algorithm.
Resources requested: 0/2 CPUs, 0/1 GPUs, 0.0/7.31 GiB heap, 0.0/3.66 GiB objects (0.0/1.0 accelerator_type:T4)
Result logdir: /root/ray_results/TorchTrainer_2022-05-25_22-26-01
Number of trials: 1/1 (1 TERMINATED)
Trial name status loc
TorchTrainer_a8585_00000TERMINATED172.28.0.2:2126


(_map_block_nosplit pid=2159) /usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py:133: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  ../torch/csrc/utils/tensor_numpy.cpp:178.)
(_map_block_nosplit pid=2159)   img = torch.from_numpy(pic.transpose((2, 0, 1))).contiguous()
(BaseWorkerMixin pid=2197) 2022-05-25 22:26:19,944	INFO torch.py:347 -- Setting up process group for: env:// [rank=0, world_size=1]
(BaseWorkerMixin pid=2197) 2022-05-25 22:26:20,033	INFO torch.py:98 -- Moving model to device: cuda:0
(BaseWorkerMixin pid=2197) loss: 2.315190, epoch: 0, iteration: 0
(BaseWorkerMixin pid=2197) loss: 1.464406, epoch: 0, iteration: 500
(BaseWorkerMixin pid=2197) loss: 1.279081, epoch: 0, iteration: 1000
(BaseWorkerMixin pid=2197) loss: 1.052461, epoch: 0, iteration: 1500
(BaseWorkerMixin pid=2197) loss: 0.816213, epoch: 1, iteration: 0
(BaseWorkerMixin pid=2197) loss: 1.019127, epoch: 1, iteration: 500
(BaseWorkerMixin pid=2197) loss: 0.525613, epoch: 1, iteration: 1000
(BaseWorkerMixin pid=2197) loss: 0.570595, epoch: 1, iteration: 1500
(BaseWorkerMixin pid=2197) loss: 0.572004, epoch: 2, iteration: 0
(BaseWorkerMixin pid=2197) loss: 0.543432, epoch: 2, iteration: 500
(BaseWorkerMixin pid=2197) loss: 0.350156, epoch: 2, iteration: 1000
(BaseWorkerMixin pid=2197) loss: 0.443743, epoch: 2, iteration: 1500
(BaseWorkerMixin pid=2197) loss: 0.438318, epoch: 3, iteration: 0
(BaseWorkerMixin pid=2197) loss: 0.342512, epoch: 3, iteration: 500
(BaseWorkerMixin pid=2197) loss: 0.302048, epoch: 3, iteration: 1000
(BaseWorkerMixin pid=2197) loss: 0.414025, epoch: 3, iteration: 1500
2022-05-25 22:27:16,013	ERROR checkpoint_manager.py:193 -- Result dict has no key: training_iteration. checkpoint_score_attr must be set to a key of the result dict. Valid keys are ['trial_id', 'experiment_id', 'date', 'timestamp', 'pid', 'hostname', 'node_ip', 'config', 'done']
Trial TorchTrainer_a8585_00000 completed. Last result: 
2022-05-25 22:27:16,138	INFO tune.py:753 -- Total run time: 74.68 seconds (74.45 seconds for the tuning loop).
Map Progress (1 actors 1 pending):   0%|          | 0/1 [00:01<?, ?it/s](BlockWorker pid=2267) /usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py:133: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  ../torch/csrc/utils/tensor_numpy.cpp:178.)
(BlockWorker pid=2267)   img = torch.from_numpy(pic.transpose((2, 0, 1))).contiguous()
Map Progress (2 actors 1 pending): 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:04<00:00,  4.18s/it]
Map_Batches: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:01<00:00,  1.63s/it]
Map_Batches: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:00<00:00, 13.60it/s]
Shuffle Map: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:00<00:00, 24.76it/s]
Shuffle Reduce: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:00<00:00, 49.17it/s]
Accuracy for task 1: 0.946
(ServeController pid=2382) INFO 2022-05-25 22:27:23,467 controller 2382 checkpoint_path.py:17 - Using RayInternalKVStore for controller checkpoint and recovery.
(ServeController pid=2382) INFO 2022-05-25 22:27:23,470 controller 2382 http_state.py:115 - Starting HTTP proxy with name 'SERVE_CONTROLLER_ACTOR:XnXlnS:SERVE_PROXY_ACTOR-node:172.28.0.2-0' on node 'node:172.28.0.2-0' listening on '127.0.0.1:8000'
Shuffle Map:   0%|          | 0/1 [00:00<?, ?it/s](HTTPProxyActor pid=2415) INFO:     Started server process [2415]
Shuffle Map: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:01<00:00,  1.40s/it]
Shuffle Reduce: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:00<00:00,  7.72it/s]
Map Progress (1 actors 1 pending): 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:01<00:00,  1.18s/it]
(ServeController pid=2382) INFO 2022-05-25 22:27:28,825 controller 2382 deployment_state.py:1219 - Adding 2 replicas to deployment 'mnist_model'.
(HTTPProxyActor pid=2415) INFO 2022-05-25 22:27:32,954 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 4.8ms
(HTTPProxyActor pid=2415) INFO 2022-05-25 22:27:32,977 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 21.4ms
(HTTPProxyActor pid=2415) INFO 2022-05-25 22:27:32,985 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 4.4ms
(mnist_model pid=2573) INFO 2022-05-25 22:27:32,976 mnist_model mnist_model#vDEhSp replica.py:483 - HANDLE __call__ OK 15.5ms
(mnist_model pid=2573) INFO 2022-05-25 22:27:32,992 mnist_model mnist_model#vDEhSp replica.py:483 - HANDLE __call__ OK 5.6ms
(mnist_model pid=2575) INFO 2022-05-25 22:27:32,952 mnist_model mnist_model#QdDxIB replica.py:483 - HANDLE __call__ OK 0.4ms
(mnist_model pid=2575) INFO 2022-05-25 22:27:32,982 mnist_model mnist_model#QdDxIB replica.py:483 - HANDLE __call__ OK 0.3ms
(HTTPProxyActor pid=2415) INFO 2022-05-25 22:27:32,997 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 11.8ms
(HTTPProxyActor pid=2415) INFO 2022-05-25 22:27:33,008 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 6.1ms
(HTTPProxyActor pid=2415) INFO 2022-05-25 22:27:33,017 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 7.1ms
(HTTPProxyActor pid=2415) INFO 2022-05-25 22:27:33,022 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 2.3ms
(HTTPProxyActor pid=2415) INFO 2022-05-25 22:27:33,031 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 7.3ms
(HTTPProxyActor pid=2415) INFO 2022-05-25 22:27:33,036 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 2.3ms
(HTTPProxyActor pid=2415) INFO 2022-05-25 22:27:33,044 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 6.4ms
(HTTPProxyActor pid=2415) INFO 2022-05-25 22:27:33,048 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 2.3ms
(HTTPProxyActor pid=2415) INFO 2022-05-25 22:27:33,057 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 6.9ms
(HTTPProxyActor pid=2415) INFO 2022-05-25 22:27:33,061 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 2.2ms
(HTTPProxyActor pid=2415) INFO 2022-05-25 22:27:33,070 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 6.4ms
(HTTPProxyActor pid=2415) INFO 2022-05-25 22:27:33,074 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 2.3ms
(HTTPProxyActor pid=2415) INFO 2022-05-25 22:27:33,082 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 6.4ms
(HTTPProxyActor pid=2415) INFO 2022-05-25 22:27:33,088 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 3.3ms
(mnist_model pid=2573) INFO 2022-05-25 22:27:33,016 mnist_model mnist_model#vDEhSp replica.py:483 - HANDLE __call__ OK 4.6ms
(mnist_model pid=2573) INFO 2022-05-25 22:27:33,029 mnist_model mnist_model#vDEhSp replica.py:483 - HANDLE __call__ OK 4.4ms
(mnist_model pid=2573) INFO 2022-05-25 22:27:33,043 mnist_model mnist_model#vDEhSp replica.py:483 - HANDLE __call__ OK 4.3ms
(mnist_model pid=2573) INFO 2022-05-25 22:27:33,056 mnist_model mnist_model#vDEhSp replica.py:483 - HANDLE __call__ OK 4.6ms
(mnist_model pid=2573) INFO 2022-05-25 22:27:33,068 mnist_model mnist_model#vDEhSp replica.py:483 - HANDLE __call__ OK 4.3ms
(mnist_model pid=2573) INFO 2022-05-25 22:27:33,081 mnist_model mnist_model#vDEhSp replica.py:483 - HANDLE __call__ OK 4.2ms
(mnist_model pid=2575) INFO 2022-05-25 22:27:33,007 mnist_model mnist_model#QdDxIB replica.py:483 - HANDLE __call__ OK 0.2ms
(mnist_model pid=2575) INFO 2022-05-25 22:27:33,021 mnist_model mnist_model#QdDxIB replica.py:483 - HANDLE __call__ OK 0.2ms
(mnist_model pid=2575) INFO 2022-05-25 22:27:33,035 mnist_model mnist_model#QdDxIB replica.py:483 - HANDLE __call__ OK 0.2ms
(mnist_model pid=2575) INFO 2022-05-25 22:27:33,047 mnist_model mnist_model#QdDxIB replica.py:483 - HANDLE __call__ OK 0.2ms
(mnist_model pid=2575) INFO 2022-05-25 22:27:33,060 mnist_model mnist_model#QdDxIB replica.py:483 - HANDLE __call__ OK 0.2ms
(mnist_model pid=2575) INFO 2022-05-25 22:27:33,073 mnist_model mnist_model#QdDxIB replica.py:483 - HANDLE __call__ OK 0.2ms
(mnist_model pid=2575) INFO 2022-05-25 22:27:33,086 mnist_model mnist_model#QdDxIB replica.py:483 - HANDLE __call__ OK 0.2ms
(HTTPProxyActor pid=2415) INFO 2022-05-25 22:27:33,122 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 25.8ms
(HTTPProxyActor pid=2415) INFO 2022-05-25 22:27:33,134 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 3.0ms
(HTTPProxyActor pid=2415) INFO 2022-05-25 22:27:33,142 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 6.6ms
(mnist_model pid=2573) INFO 2022-05-25 22:27:33,117 mnist_model mnist_model#vDEhSp replica.py:483 - HANDLE __call__ OK 14.4ms
(mnist_model pid=2573) INFO 2022-05-25 22:27:33,141 mnist_model mnist_model#vDEhSp replica.py:483 - HANDLE __call__ OK 4.4ms
(mnist_model pid=2575) INFO 2022-05-25 22:27:33,133 mnist_model mnist_model#QdDxIB replica.py:483 - HANDLE __call__ OK 0.4ms
(ServeController pid=2382) INFO 2022-05-25 22:27:33,225 controller 2382 deployment_state.py:1243 - Removing 2 replicas from deployment 'mnist_model'.
Map Progress (2 actors 1 pending): 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:02<00:00,  2.58s/it]
(_prepare_read pid=2726) 2022-05-25 22:27:40,353	WARNING torch_datasource.py:56 -- `SimpleTorchDatasource` doesn't support parallel reads. The `parallelism` argument will be ignored.
Read->Map_Batches: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:00<00:00,  2.20it/s]
Map Progress (1 actors 1 pending): 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:01<00:00,  1.41s/it]
Starting training for task: 1

== Status ==
Current time: 2022-05-25 22:28:52 (running for 00:01:09.00)
Memory usage on this node: 5.0/12.7 GiB
Using FIFO scheduling algorithm.
Resources requested: 0/2 CPUs, 0/1 GPUs, 0.0/7.31 GiB heap, 0.0/3.66 GiB objects (0.0/1.0 accelerator_type:T4)
Result logdir: /root/ray_results/TorchTrainer_2022-05-25_22-27-43
Number of trials: 1/1 (1 TERMINATED)
Trial name status loc
TorchTrainer_e4f66_00000TERMINATED172.28.0.2:2875


(_map_block_nosplit pid=2909) /usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py:133: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  ../torch/csrc/utils/tensor_numpy.cpp:178.)
(_map_block_nosplit pid=2909)   img = torch.from_numpy(pic.transpose((2, 0, 1))).contiguous()
(BaseWorkerMixin pid=2948) 2022-05-25 22:28:01,917	INFO torch.py:347 -- Setting up process group for: env:// [rank=0, world_size=1]
(BaseWorkerMixin pid=2948) 2022-05-25 22:28:02,063	INFO torch.py:98 -- Moving model to device: cuda:0
(BaseWorkerMixin pid=2948) loss: 3.347775, epoch: 0, iteration: 0
(BaseWorkerMixin pid=2948) loss: 1.343975, epoch: 0, iteration: 500
(BaseWorkerMixin pid=2948) loss: 0.768560, epoch: 0, iteration: 1000
(BaseWorkerMixin pid=2948) loss: 0.607410, epoch: 0, iteration: 1500
(BaseWorkerMixin pid=2948) loss: 0.578952, epoch: 1, iteration: 0
(BaseWorkerMixin pid=2948) loss: 0.473788, epoch: 1, iteration: 500
(BaseWorkerMixin pid=2948) loss: 0.609530, epoch: 1, iteration: 1000
(BaseWorkerMixin pid=2948) loss: 0.741895, epoch: 1, iteration: 1500
(BaseWorkerMixin pid=2948) loss: 0.417272, epoch: 2, iteration: 0
(BaseWorkerMixin pid=2948) loss: 0.510404, epoch: 2, iteration: 500
(BaseWorkerMixin pid=2948) loss: 0.422137, epoch: 2, iteration: 1000
(BaseWorkerMixin pid=2948) loss: 0.403623, epoch: 2, iteration: 1500
(BaseWorkerMixin pid=2948) loss: 0.384720, epoch: 3, iteration: 0
(BaseWorkerMixin pid=2948) loss: 0.414567, epoch: 3, iteration: 500
(BaseWorkerMixin pid=2948) loss: 0.274302, epoch: 3, iteration: 1000
(BaseWorkerMixin pid=2948) loss: 0.348169, epoch: 3, iteration: 1500
2022-05-25 22:28:52,221	ERROR checkpoint_manager.py:193 -- Result dict has no key: training_iteration. checkpoint_score_attr must be set to a key of the result dict. Valid keys are ['trial_id', 'experiment_id', 'date', 'timestamp', 'pid', 'hostname', 'node_ip', 'config', 'done']
Trial TorchTrainer_e4f66_00000 completed. Last result: 
2022-05-25 22:28:52,344	INFO tune.py:753 -- Total run time: 69.20 seconds (68.99 seconds for the tuning loop).
Map Progress (1 actors 1 pending):   0%|          | 0/2 [00:01<?, ?it/s](BlockWorker pid=3027) /usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py:133: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  ../torch/csrc/utils/tensor_numpy.cpp:178.)
(BlockWorker pid=3027)   img = torch.from_numpy(pic.transpose((2, 0, 1))).contiguous()
Map Progress (2 actors 1 pending): 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 2/2 [00:05<00:00,  2.64s/it]
Map_Batches: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 2/2 [00:01<00:00,  1.07it/s]
Map_Batches: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 2/2 [00:01<00:00,  1.55it/s]
Shuffle Map: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 2/2 [00:00<00:00,  3.78it/s]
Shuffle Reduce: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:00<00:00, 72.95it/s]
Accuracy for task 2: 0.9261
(ServeController pid=3209) INFO 2022-05-25 22:29:02,797 controller 3209 checkpoint_path.py:17 - Using RayInternalKVStore for controller checkpoint and recovery.
(ServeController pid=3209) INFO 2022-05-25 22:29:02,802 controller 3209 http_state.py:115 - Starting HTTP proxy with name 'SERVE_CONTROLLER_ACTOR:lsPTvu:SERVE_PROXY_ACTOR-node:172.28.0.2-0' on node 'node:172.28.0.2-0' listening on '127.0.0.1:8000'
Shuffle Map:   0%|          | 0/1 [00:00<?, ?it/s](HTTPProxyActor pid=3241) INFO:     Started server process [3241]
Shuffle Map: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:01<00:00,  1.54s/it]
Shuffle Reduce: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:00<00:00,  8.17it/s]
Map Progress (1 actors 1 pending): 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:01<00:00,  1.15s/it]
(ServeController pid=3209) INFO 2022-05-25 22:29:08,327 controller 3209 deployment_state.py:1219 - Adding 2 replicas to deployment 'mnist_model'.
(HTTPProxyActor pid=3241) INFO 2022-05-25 22:29:12,440 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 5.3ms
(mnist_model pid=3402) INFO 2022-05-25 22:29:12,438 mnist_model mnist_model#Egafuf replica.py:483 - HANDLE __call__ OK 0.3ms
(mnist_model pid=3401) INFO 2022-05-25 22:29:12,460 mnist_model mnist_model#uumYOV replica.py:483 - HANDLE __call__ OK 15.4ms
(HTTPProxyActor pid=3241) INFO 2022-05-25 22:29:12,466 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 24.3ms
(HTTPProxyActor pid=3241) INFO 2022-05-25 22:29:12,471 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 2.8ms
(HTTPProxyActor pid=3241) INFO 2022-05-25 22:29:12,481 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 7.6ms
(HTTPProxyActor pid=3241) INFO 2022-05-25 22:29:12,487 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 3.9ms
(HTTPProxyActor pid=3241) INFO 2022-05-25 22:29:12,496 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 6.9ms
(HTTPProxyActor pid=3241) INFO 2022-05-25 22:29:12,501 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 2.3ms
(HTTPProxyActor pid=3241) INFO 2022-05-25 22:29:12,509 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 6.8ms
(HTTPProxyActor pid=3241) INFO 2022-05-25 22:29:12,514 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 2.6ms
(HTTPProxyActor pid=3241) INFO 2022-05-25 22:29:12,523 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 6.8ms
(HTTPProxyActor pid=3241) INFO 2022-05-25 22:29:12,528 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 2.3ms
(HTTPProxyActor pid=3241) INFO 2022-05-25 22:29:12,537 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 7.2ms
(HTTPProxyActor pid=3241) INFO 2022-05-25 22:29:12,542 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 2.5ms
(HTTPProxyActor pid=3241) INFO 2022-05-25 22:29:12,550 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 6.7ms
(HTTPProxyActor pid=3241) INFO 2022-05-25 22:29:12,556 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 3.7ms
(HTTPProxyActor pid=3241) INFO 2022-05-25 22:29:12,564 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 7.0ms
(mnist_model pid=3401) INFO 2022-05-25 22:29:12,480 mnist_model mnist_model#uumYOV replica.py:483 - HANDLE __call__ OK 5.1ms
(mnist_model pid=3401) INFO 2022-05-25 22:29:12,495 mnist_model mnist_model#uumYOV replica.py:483 - HANDLE __call__ OK 4.5ms
(mnist_model pid=3401) INFO 2022-05-25 22:29:12,508 mnist_model mnist_model#uumYOV replica.py:483 - HANDLE __call__ OK 4.6ms
(mnist_model pid=3401) INFO 2022-05-25 22:29:12,522 mnist_model mnist_model#uumYOV replica.py:483 - HANDLE __call__ OK 4.6ms
(mnist_model pid=3401) INFO 2022-05-25 22:29:12,536 mnist_model mnist_model#uumYOV replica.py:483 - HANDLE __call__ OK 4.7ms
(mnist_model pid=3401) INFO 2022-05-25 22:29:12,549 mnist_model mnist_model#uumYOV replica.py:483 - HANDLE __call__ OK 4.4ms
(mnist_model pid=3401) INFO 2022-05-25 22:29:12,563 mnist_model mnist_model#uumYOV replica.py:483 - HANDLE __call__ OK 4.7ms
(mnist_model pid=3402) INFO 2022-05-25 22:29:12,470 mnist_model mnist_model#Egafuf replica.py:483 - HANDLE __call__ OK 0.3ms
(mnist_model pid=3402) INFO 2022-05-25 22:29:12,485 mnist_model mnist_model#Egafuf replica.py:483 - HANDLE __call__ OK 0.2ms
(mnist_model pid=3402) INFO 2022-05-25 22:29:12,500 mnist_model mnist_model#Egafuf replica.py:483 - HANDLE __call__ OK 0.2ms
(mnist_model pid=3402) INFO 2022-05-25 22:29:12,513 mnist_model mnist_model#Egafuf replica.py:483 - HANDLE __call__ OK 0.3ms
(mnist_model pid=3402) INFO 2022-05-25 22:29:12,527 mnist_model mnist_model#Egafuf replica.py:483 - HANDLE __call__ OK 0.2ms
(mnist_model pid=3402) INFO 2022-05-25 22:29:12,540 mnist_model mnist_model#Egafuf replica.py:483 - HANDLE __call__ OK 0.2ms
(mnist_model pid=3402) INFO 2022-05-25 22:29:12,554 mnist_model mnist_model#Egafuf replica.py:483 - HANDLE __call__ OK 0.2ms
(HTTPProxyActor pid=3241) INFO 2022-05-25 22:29:12,586 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 4.6ms
(HTTPProxyActor pid=3241) INFO 2022-05-25 22:29:12,596 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 9.3ms
(HTTPProxyActor pid=3241) INFO 2022-05-25 22:29:12,601 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 2.7ms
(HTTPProxyActor pid=3241) INFO 2022-05-25 22:29:12,610 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 7.0ms
(mnist_model pid=3401) INFO 2022-05-25 22:29:12,594 mnist_model mnist_model#uumYOV replica.py:483 - HANDLE __call__ OK 6.3ms
(mnist_model pid=3401) INFO 2022-05-25 22:29:12,609 mnist_model mnist_model#uumYOV replica.py:483 - HANDLE __call__ OK 4.6ms
(mnist_model pid=3402) INFO 2022-05-25 22:29:12,583 mnist_model mnist_model#Egafuf replica.py:483 - HANDLE __call__ OK 0.3ms
(mnist_model pid=3402) INFO 2022-05-25 22:29:12,600 mnist_model mnist_model#Egafuf replica.py:483 - HANDLE __call__ OK 0.3ms
(ServeController pid=3209) INFO 2022-05-25 22:29:12,699 controller 3209 deployment_state.py:1243 - Removing 2 replicas from deployment 'mnist_model'.
Map Progress (2 actors 1 pending): 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:02<00:00,  2.56s/it]
(_prepare_read pid=3556) 2022-05-25 22:29:19,825	WARNING torch_datasource.py:56 -- `SimpleTorchDatasource` doesn't support parallel reads. The `parallelism` argument will be ignored.
Read->Map_Batches: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:00<00:00,  2.44it/s]
Map Progress (1 actors 1 pending): 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:01<00:00,  1.41s/it]
Starting training for task: 2

== Status ==
Current time: 2022-05-25 22:30:31 (running for 00:01:09.12)
Memory usage on this node: 5.0/12.7 GiB
Using FIFO scheduling algorithm.
Resources requested: 0/2 CPUs, 0/1 GPUs, 0.0/7.31 GiB heap, 0.0/3.66 GiB objects (0.0/1.0 accelerator_type:T4)
Result logdir: /root/ray_results/TorchTrainer_2022-05-25_22-29-22
Number of trials: 1/1 (1 TERMINATED)
Trial name status loc
TorchTrainer_2040e_00000TERMINATED172.28.0.2:3703


(_map_block_nosplit pid=3738) /usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py:133: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  ../torch/csrc/utils/tensor_numpy.cpp:178.)
(_map_block_nosplit pid=3738)   img = torch.from_numpy(pic.transpose((2, 0, 1))).contiguous()
(BaseWorkerMixin pid=3778) 2022-05-25 22:29:41,392	INFO torch.py:347 -- Setting up process group for: env:// [rank=0, world_size=1]
(BaseWorkerMixin pid=3778) 2022-05-25 22:29:41,549	INFO torch.py:98 -- Moving model to device: cuda:0
(BaseWorkerMixin pid=3778) loss: 4.353125, epoch: 0, iteration: 0
(BaseWorkerMixin pid=3778) loss: 1.147782, epoch: 0, iteration: 500
(BaseWorkerMixin pid=3778) loss: 0.609233, epoch: 0, iteration: 1000
(BaseWorkerMixin pid=3778) loss: 0.606812, epoch: 0, iteration: 1500
(BaseWorkerMixin pid=3778) loss: 0.494777, epoch: 1, iteration: 0
(BaseWorkerMixin pid=3778) loss: 0.776362, epoch: 1, iteration: 500
(BaseWorkerMixin pid=3778) loss: 0.376833, epoch: 1, iteration: 1000
(BaseWorkerMixin pid=3778) loss: 0.478181, epoch: 1, iteration: 1500
(BaseWorkerMixin pid=3778) loss: 0.413856, epoch: 2, iteration: 0
(BaseWorkerMixin pid=3778) loss: 0.668218, epoch: 2, iteration: 500
(BaseWorkerMixin pid=3778) loss: 0.318078, epoch: 2, iteration: 1000
(BaseWorkerMixin pid=3778) loss: 0.427121, epoch: 2, iteration: 1500
(BaseWorkerMixin pid=3778) loss: 0.369263, epoch: 3, iteration: 0
(BaseWorkerMixin pid=3778) loss: 0.479945, epoch: 3, iteration: 500
(BaseWorkerMixin pid=3778) loss: 0.457482, epoch: 3, iteration: 1000
(BaseWorkerMixin pid=3778) loss: 0.318416, epoch: 3, iteration: 1500
2022-05-25 22:30:31,831	ERROR checkpoint_manager.py:193 -- Result dict has no key: training_iteration. checkpoint_score_attr must be set to a key of the result dict. Valid keys are ['trial_id', 'experiment_id', 'date', 'timestamp', 'pid', 'hostname', 'node_ip', 'config', 'done']
Trial TorchTrainer_2040e_00000 completed. Last result: 
2022-05-25 22:30:31,953	INFO tune.py:753 -- Total run time: 69.33 seconds (69.12 seconds for the tuning loop).
Map Progress (1 actors 1 pending):   0%|          | 0/3 [00:01<?, ?it/s](BlockWorker pid=3857) /usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py:133: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  ../torch/csrc/utils/tensor_numpy.cpp:178.)
(BlockWorker pid=3857)   img = torch.from_numpy(pic.transpose((2, 0, 1))).contiguous()
Map Progress (2 actors 1 pending):  33%|β–ˆβ–ˆβ–ˆβ–Ž      | 1/3 [00:04<00:08,  4.24s/it](BlockWorker pid=3886) /usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py:133: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  ../torch/csrc/utils/tensor_numpy.cpp:178.)
(BlockWorker pid=3886)   img = torch.from_numpy(pic.transpose((2, 0, 1))).contiguous()
Map Progress (2 actors 1 pending): 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 3/3 [00:06<00:00,  2.16s/it]
Map_Batches: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 3/3 [00:01<00:00,  1.53it/s]
Map_Batches: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 3/3 [00:00<00:00, 19.25it/s]
Shuffle Map: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 3/3 [00:00<00:00, 97.56it/s]
Shuffle Reduce: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:00<00:00, 64.24it/s]
Accuracy for task 3: 0.9001333333333333
(ServeController pid=4011) INFO 2022-05-25 22:30:43,081 controller 4011 checkpoint_path.py:17 - Using RayInternalKVStore for controller checkpoint and recovery.
(ServeController pid=4011) INFO 2022-05-25 22:30:43,084 controller 4011 http_state.py:115 - Starting HTTP proxy with name 'SERVE_CONTROLLER_ACTOR:viEsyL:SERVE_PROXY_ACTOR-node:172.28.0.2-0' on node 'node:172.28.0.2-0' listening on '127.0.0.1:8000'
Shuffle Map:   0%|          | 0/1 [00:00<?, ?it/s](HTTPProxyActor pid=4043) INFO:     Started server process [4043]
Shuffle Map: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:01<00:00,  1.61s/it]
Shuffle Reduce: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:00<00:00,  7.16it/s]
Map Progress (1 actors 1 pending): 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:01<00:00,  1.36s/it]
(ServeController pid=4011) INFO 2022-05-25 22:30:48,663 controller 4011 deployment_state.py:1219 - Adding 2 replicas to deployment 'mnist_model'.
(HTTPProxyActor pid=4043) INFO 2022-05-25 22:30:52,754 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 5.0ms
(HTTPProxyActor pid=4043) INFO 2022-05-25 22:30:52,771 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 15.8ms
(HTTPProxyActor pid=4043) INFO 2022-05-25 22:30:52,777 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 3.1ms
(HTTPProxyActor pid=4043) INFO 2022-05-25 22:30:52,788 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 9.0ms
(HTTPProxyActor pid=4043) INFO 2022-05-25 22:30:52,794 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 3.5ms
(HTTPProxyActor pid=4043) INFO 2022-05-25 22:30:52,803 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 7.0ms
(HTTPProxyActor pid=4043) INFO 2022-05-25 22:30:52,808 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 2.5ms
(HTTPProxyActor pid=4043) INFO 2022-05-25 22:30:52,817 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 7.2ms
(HTTPProxyActor pid=4043) INFO 2022-05-25 22:30:52,822 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 2.6ms
(mnist_model pid=4199) INFO 2022-05-25 22:30:52,770 mnist_model mnist_model#kzOVuE replica.py:483 - HANDLE __call__ OK 11.5ms
(mnist_model pid=4199) INFO 2022-05-25 22:30:52,787 mnist_model mnist_model#kzOVuE replica.py:483 - HANDLE __call__ OK 6.1ms
(mnist_model pid=4199) INFO 2022-05-25 22:30:52,802 mnist_model mnist_model#kzOVuE replica.py:483 - HANDLE __call__ OK 4.8ms
(mnist_model pid=4199) INFO 2022-05-25 22:30:52,815 mnist_model mnist_model#kzOVuE replica.py:483 - HANDLE __call__ OK 4.6ms
(mnist_model pid=4200) INFO 2022-05-25 22:30:52,752 mnist_model mnist_model#QFllkk replica.py:483 - HANDLE __call__ OK 0.3ms
(mnist_model pid=4200) INFO 2022-05-25 22:30:52,776 mnist_model mnist_model#QFllkk replica.py:483 - HANDLE __call__ OK 0.3ms
(mnist_model pid=4200) INFO 2022-05-25 22:30:52,793 mnist_model mnist_model#QFllkk replica.py:483 - HANDLE __call__ OK 0.2ms
(mnist_model pid=4200) INFO 2022-05-25 22:30:52,807 mnist_model mnist_model#QFllkk replica.py:483 - HANDLE __call__ OK 0.2ms
(mnist_model pid=4200) INFO 2022-05-25 22:30:52,821 mnist_model mnist_model#QFllkk replica.py:483 - HANDLE __call__ OK 0.2ms
(HTTPProxyActor pid=4043) INFO 2022-05-25 22:30:52,848 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 24.9ms
(HTTPProxyActor pid=4043) INFO 2022-05-25 22:30:52,853 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 2.6ms
(HTTPProxyActor pid=4043) INFO 2022-05-25 22:30:52,869 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 13.4ms
(mnist_model pid=4199) INFO 2022-05-25 22:30:52,847 mnist_model mnist_model#kzOVuE replica.py:483 - HANDLE __call__ OK 8.4ms
(mnist_model pid=4199) INFO 2022-05-25 22:30:52,867 mnist_model mnist_model#kzOVuE replica.py:483 - HANDLE __call__ OK 6.6ms
(mnist_model pid=4200) INFO 2022-05-25 22:30:52,852 mnist_model mnist_model#QFllkk replica.py:483 - HANDLE __call__ OK 0.2ms
(HTTPProxyActor pid=4043) INFO 2022-05-25 22:30:52,984 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 3.5ms
(HTTPProxyActor pid=4043) INFO 2022-05-25 22:30:52,995 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 9.0ms
(HTTPProxyActor pid=4043) INFO 2022-05-25 22:30:53,001 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 3.4ms
(HTTPProxyActor pid=4043) INFO 2022-05-25 22:30:53,011 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 8.1ms
(HTTPProxyActor pid=4043) INFO 2022-05-25 22:30:53,016 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 2.7ms
(HTTPProxyActor pid=4043) INFO 2022-05-25 22:30:53,025 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 7.4ms
(HTTPProxyActor pid=4043) INFO 2022-05-25 22:30:53,030 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 307 2.5ms
(HTTPProxyActor pid=4043) INFO 2022-05-25 22:30:53,045 http_proxy 172.28.0.2 http_proxy.py:320 - POST /mnist_predict 200 11.9ms
(mnist_model pid=4199) INFO 2022-05-25 22:30:52,993 mnist_model mnist_model#kzOVuE replica.py:483 - HANDLE __call__ OK 5.9ms
(mnist_model pid=4199) INFO 2022-05-25 22:30:53,010 mnist_model mnist_model#kzOVuE replica.py:483 - HANDLE __call__ OK 5.2ms
(mnist_model pid=4199) INFO 2022-05-25 22:30:53,024 mnist_model mnist_model#kzOVuE replica.py:483 - HANDLE __call__ OK 4.9ms
(mnist_model pid=4199) INFO 2022-05-25 22:30:53,043 mnist_model mnist_model#kzOVuE replica.py:483 - HANDLE __call__ OK 4.9ms
(mnist_model pid=4200) INFO 2022-05-25 22:30:52,982 mnist_model mnist_model#QFllkk replica.py:483 - HANDLE __call__ OK 0.3ms
(mnist_model pid=4200) INFO 2022-05-25 22:30:52,999 mnist_model mnist_model#QFllkk replica.py:483 - HANDLE __call__ OK 0.2ms
(mnist_model pid=4200) INFO 2022-05-25 22:30:53,015 mnist_model mnist_model#QFllkk replica.py:483 - HANDLE __call__ OK 0.2ms
(mnist_model pid=4200) INFO 2022-05-25 22:30:53,029 mnist_model mnist_model#QFllkk replica.py:483 - HANDLE __call__ OK 0.3ms
(ServeController pid=4011) INFO 2022-05-25 22:30:53,125 controller 4011 deployment_state.py:1243 - Removing 2 replicas from deployment 'mnist_model'.

Now that we have finished all of our training, let’s see the accuracy of our model after training on each task.

We should see the accuracy decrease over time. This is to be expected since we are using just a naive fine-tuning strategy so our model is prone to catastrophic forgetting.

As we increase the number of tasks, the model performance on all the tasks trained on so far should decrease.

accuracy_for_all_tasks
[0.946, 0.9261, 0.9001333333333333]

[Optional] Step 6: Compare against full training.ΒΆ

We have now incrementally trained our simple multi-layer perceptron. Let’s compare the incrementally trained model via fine tuning against a model that is trained on all the tasks up front.

Since we are using a naive fine-tuning strategy, we should expect that our incrementally trained model will perform worse than the the one that is fully trained! However, there’s various other strategies that have been developed and are actively being researched to improve accuracy for incremental training. And overall, incremental/continual learning allows you to train in many real world settings where the entire dataset is not available up front, but new data is arriving at a relatively high rate.

Let’s first combine all of our datasets for each task into a single, unified Dataset

train_stream = permuted_mnist.generate_train_stream()

# Collect all datasets in the stream into a single dataset.
all_training_datasets = []
for train_dataset in train_stream:
  all_training_datasets.append(train_dataset)
combined_training_dataset = all_training_datasets[0].union(*all_training_datasets[1:])


combined_training_dataset = combined_training_dataset.random_shuffle()
Map Progress (2 actors 1 pending): 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:02<00:00,  2.93s/it]
Map Progress (2 actors 1 pending): 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:03<00:00,  3.11s/it]
Map Progress:   0%|          | 0/1 [00:00<?, ?it/s]
(scheduler +8m58s) Tip: use `ray status` to view detailed cluster status. To disable these messages, set RAY_SCHEDULER_EVENTS=0.
(scheduler +8m58s) Warning: The following resource request cannot be scheduled right now: {'CPU': 1.0}. This is likely due to all cluster resources being claimed by actors. Consider creating fewer actors or adding more nodes to this Ray cluster.
Map Progress (2 actors 1 pending): 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:03<00:00,  3.06s/it]
Shuffle Map: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 3/3 [00:04<00:00,  1.64s/it]
Shuffle Reduce: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 3/3 [00:02<00:00,  1.07it/s]

Then, we train a new model on the unified Dataset using the same configurations as before.

# Now we do training with the same configurations as before
trainer = TorchTrainer(
            train_loop_per_worker=train_loop_per_worker,
            train_loop_config={
                "num_epochs": num_epochs,
                "learning_rate": learning_rate,
                "momentum": momentum,
                "batch_size": batch_size,
            },
            # Have to specify trainer_resources as 0 so that the example works on Colab. 
            scaling_config={"num_workers": num_workers, "use_gpu": use_gpu, "trainer_resources": {"CPU": 0}},
            datasets={"train": combined_training_dataset},
            preprocessor=BatchMapper(fn=preprocess_images),
        )
result = trainer.fit()
full_training_checkpoint = result.checkpoint
== Status ==
Current time: 2022-05-18 23:52:49 (running for 00:03:27.40)
Memory usage on this node: 7.0/12.7 GiB
Using FIFO scheduling algorithm.
Resources requested: 0/2 CPUs, 0/1 GPUs, 0.0/7.34 GiB heap, 0.0/3.67 GiB objects (0.0/1.0 accelerator_type:T4)
Result logdir: /root/ray_results/TorchTrainer_2022-05-18_23-49-22
Number of trials: 1/1 (1 TERMINATED)
Trial name status loc
TorchTrainer_24496_00000TERMINATED172.28.0.2:4630


(_map_block_nosplit pid=4666) /usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py:133: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  ../torch/csrc/utils/tensor_numpy.cpp:178.)
(_map_block_nosplit pid=4666)   img = torch.from_numpy(pic.transpose((2, 0, 1))).contiguous()
(BaseWorkerMixin pid=4709) 2022-05-18 23:50:06,950	INFO torch.py:347 -- Setting up process group for: env:// [rank=0, world_size=1]
(BaseWorkerMixin pid=4709) 2022-05-18 23:50:07,011	INFO torch.py:98 -- Moving model to device: cuda:0
(BaseWorkerMixin pid=4709) loss: 2.373475, epoch: 0, iteration: 0
(BaseWorkerMixin pid=4709) loss: 1.699985, epoch: 0, iteration: 500
(BaseWorkerMixin pid=4709) loss: 1.636039, epoch: 0, iteration: 1000
(BaseWorkerMixin pid=4709) loss: 1.334987, epoch: 0, iteration: 1500
(BaseWorkerMixin pid=4709) loss: 1.152312, epoch: 0, iteration: 2000
(BaseWorkerMixin pid=4709) loss: 0.998297, epoch: 0, iteration: 2500
(BaseWorkerMixin pid=4709) loss: 1.434949, epoch: 0, iteration: 3000
(BaseWorkerMixin pid=4709) loss: 0.971171, epoch: 0, iteration: 3500
(BaseWorkerMixin pid=4709) loss: 0.796480, epoch: 0, iteration: 4000
(BaseWorkerMixin pid=4709) loss: 0.802282, epoch: 0, iteration: 4500
(BaseWorkerMixin pid=4709) loss: 0.731363, epoch: 0, iteration: 5000
(BaseWorkerMixin pid=4709) loss: 0.847772, epoch: 0, iteration: 5500
(BaseWorkerMixin pid=4709) loss: 0.879676, epoch: 1, iteration: 0
(BaseWorkerMixin pid=4709) loss: 0.564319, epoch: 1, iteration: 500
(BaseWorkerMixin pid=4709) loss: 0.714444, epoch: 1, iteration: 1000
(BaseWorkerMixin pid=4709) loss: 0.565163, epoch: 1, iteration: 1500
(BaseWorkerMixin pid=4709) loss: 0.739525, epoch: 1, iteration: 2000
(BaseWorkerMixin pid=4709) loss: 0.510878, epoch: 1, iteration: 2500
(BaseWorkerMixin pid=4709) loss: 0.814798, epoch: 1, iteration: 3000
(BaseWorkerMixin pid=4709) loss: 0.473765, epoch: 1, iteration: 3500
(BaseWorkerMixin pid=4709) loss: 0.557866, epoch: 1, iteration: 4000
(BaseWorkerMixin pid=4709) loss: 0.674371, epoch: 1, iteration: 4500
(BaseWorkerMixin pid=4709) loss: 0.532800, epoch: 1, iteration: 5000
(BaseWorkerMixin pid=4709) loss: 0.832442, epoch: 1, iteration: 5500
(BaseWorkerMixin pid=4709) loss: 0.557547, epoch: 2, iteration: 0
(BaseWorkerMixin pid=4709) loss: 0.355255, epoch: 2, iteration: 500
(BaseWorkerMixin pid=4709) loss: 0.426749, epoch: 2, iteration: 1000
(BaseWorkerMixin pid=4709) loss: 0.484543, epoch: 2, iteration: 1500
(BaseWorkerMixin pid=4709) loss: 0.360856, epoch: 2, iteration: 2000
(BaseWorkerMixin pid=4709) loss: 0.444718, epoch: 2, iteration: 2500
(BaseWorkerMixin pid=4709) loss: 0.596777, epoch: 2, iteration: 3000
(BaseWorkerMixin pid=4709) loss: 0.289816, epoch: 2, iteration: 3500
(BaseWorkerMixin pid=4709) loss: 0.407941, epoch: 2, iteration: 4000
(BaseWorkerMixin pid=4709) loss: 0.438239, epoch: 2, iteration: 4500
(BaseWorkerMixin pid=4709) loss: 0.379983, epoch: 2, iteration: 5000
(BaseWorkerMixin pid=4709) loss: 0.527786, epoch: 2, iteration: 5500
(BaseWorkerMixin pid=4709) loss: 0.598584, epoch: 3, iteration: 0
(BaseWorkerMixin pid=4709) loss: 0.355202, epoch: 3, iteration: 500
(BaseWorkerMixin pid=4709) loss: 0.392683, epoch: 3, iteration: 1000
(BaseWorkerMixin pid=4709) loss: 0.415264, epoch: 3, iteration: 1500
(BaseWorkerMixin pid=4709) loss: 0.417230, epoch: 3, iteration: 2000
(BaseWorkerMixin pid=4709) loss: 0.289974, epoch: 3, iteration: 2500
(BaseWorkerMixin pid=4709) loss: 0.648514, epoch: 3, iteration: 3000
(BaseWorkerMixin pid=4709) loss: 0.369468, epoch: 3, iteration: 3500
(BaseWorkerMixin pid=4709) loss: 0.378548, epoch: 3, iteration: 4000
(BaseWorkerMixin pid=4709) loss: 0.392761, epoch: 3, iteration: 4500
(BaseWorkerMixin pid=4709) loss: 0.555575, epoch: 3, iteration: 5000
(BaseWorkerMixin pid=4709) loss: 0.394487, epoch: 3, iteration: 5500
2022-05-18 23:52:49,915	ERROR checkpoint_manager.py:193 -- Result dict has no key: training_iteration. checkpoint_score_attr must be set to a key of the result dict. Valid keys are ['trial_id', 'experiment_id', 'date', 'timestamp', 'pid', 'hostname', 'node_ip', 'config', 'done']
Trial TorchTrainer_24496_00000 completed. Last result: 
2022-05-18 23:52:50,042	INFO tune.py:753 -- Total run time: 207.53 seconds (207.39 seconds for the tuning loop).

Then, let’s test model that was trained on all the tasks up front.

# Then, we used the fully trained model and do batch prediction on the entire test set.

# `full_test_dataset` should already contain the combined test datasets.
fully_trained_accuracy = batch_predict(full_training_checkpoint, full_test_dataset)
Map Progress (1 actors 1 pending):   0%|          | 0/3 [00:01<?, ?it/s](BlockWorker pid=4840) /usr/local/lib/python3.7/dist-packages/torchvision/transforms/functional.py:133: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at  ../torch/csrc/utils/tensor_numpy.cpp:178.)
(BlockWorker pid=4840)   img = torch.from_numpy(pic.transpose((2, 0, 1))).contiguous()
Map Progress (2 actors 1 pending): 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 3/3 [00:06<00:00,  2.25s/it]
Map Progress: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 3/3 [00:01<00:00,  1.51it/s]
Map Progress: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 3/3 [00:01<00:00,  1.94it/s]
Shuffle Map: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 3/3 [00:00<00:00,  5.53it/s]
Shuffle Reduce: 100%|β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ| 1/1 [00:00<00:00, 65.42it/s]

Finally, let’s compare the accuracies between the incrementally trained model and the fully trained model. We should see that the fully trained model performs better.

print("Fully trained model accuracy: ", fully_trained_accuracy)
print("Incrementally trained model accuracy: ", accuracy_for_all_tasks[-1])
Fully trained model accuracy:  0.9468
Incrementally trained model accuracy:  0.9207666666666666

Next StepsΒΆ

Once you’ve completed this notebook, you should be set to play around with scalable incremental training using Ray, either by trying more fancy algorithms for incremental learning other than naive fine-tuning, or attempting to scale out to larger datasets!

If you run into any issues, or have any feature requests, please file an issue on the Ray Github.