Tabular data training and serving with Keras and Ray AIR

This notebook is adapted from a Keras tutorial. It uses Chicago Taxi dataset and a DNN Keras model to predict whether a trip may generate a big tip.

In this example, we showcase how to achieve the same tasks as the Keras Tutorial using Ray AIR, covering every step from data ingestion to pushing a model to serving.

  1. Read a CSV into Ray Dataset.

  2. Process the dataset by chaining Ray AIR preprocessors.

  3. Train the model using the TensorflowTrainer from AIR.

  4. Serve the model using Ray Serve and the above preprocessors.

Uncomment and run the following line in order to install all the necessary dependencies:

# ! pip install "tensorflow>=2.8.0" "ray[tune, data, serve]>=1.12.1"
# ! pip install fastapi

Set up Ray

We will use ray.init() to initialize a local cluster. By default, this cluster will be composed of only the machine you are running this notebook on. If you wish to attach to an existing Ray cluster, you can do so through ray.init(address="auto").

from pprint import pprint
import ray

ray.shutdown()
ray.init()
2022-09-15 12:47:35,721	INFO worker.py:1508 -- Started a local Ray instance. View the dashboard at 127.0.0.1:8265 

Ray

Python version: 3.9.13
Ray version: 3.0.0.dev0
Dashboard: http://127.0.0.1:8265

We can check the resources our cluster is composed of. If you are running this notebook on your local machine or Google Colab, you should see the number of CPU cores and GPUs available on the said machine.

pprint(ray.cluster_resources())
{'CPU': 24.0,
 'memory': 34702953678.0,
 'node:10.108.96.251': 1.0,
 'object_store_memory': 17351476838.0}

Getting the data

Let’s start with defining a helper function to get the data to work with. Some columns are dropped for simplicity.

import pandas as pd

INPUT = "input"
LABEL = "is_big_tip"

def get_data() -> pd.DataFrame:
    """Fetch the taxi fare data to work on."""
    _data = pd.read_csv(
        "https://raw.githubusercontent.com/tensorflow/tfx/master/"
        "tfx/examples/chicago_taxi_pipeline/data/simple/data.csv"
    )
    _data[LABEL] = _data["tips"] / _data["fare"] > 0.2
    # We drop some columns here for the sake of simplicity.
    return _data.drop(
        [
            "tips",
            "fare",
            "dropoff_latitude",
            "dropoff_longitude",
            "pickup_latitude",
            "pickup_longitude",
            "pickup_census_tract",
        ],
        axis=1,
    )
data = get_data()

Now let’s take a look at the data. Notice that some values are missing. This is exactly where preprocessing comes into the picture. We will come back to this in the preprocessing session below.

data.head(5)
pickup_community_area trip_start_month trip_start_hour trip_start_day trip_start_timestamp trip_miles dropoff_census_tract payment_type company trip_seconds dropoff_community_area is_big_tip
0 NaN 5 19 6 1400269500 0.0 NaN Credit Card Chicago Elite Cab Corp. (Chicago Carriag 0.0 NaN False
1 NaN 3 19 5 1362683700 0.0 NaN Unknown Chicago Elite Cab Corp. 300.0 NaN False
2 60.0 10 2 3 1380593700 12.6 NaN Cash Taxi Affiliation Services 1380.0 NaN False
3 10.0 10 1 2 1382319000 0.0 NaN Cash Taxi Affiliation Services 180.0 NaN False
4 14.0 5 7 5 1369897200 0.0 NaN Cash Dispatch Taxi Affiliation 1080.0 NaN False

We continue to split the data into training and test data. For the test data, we separate out the features to run serving on as well as labels to compare serving results with.

import numpy as np
from sklearn.model_selection import train_test_split
from typing import Tuple


def split_data(data: pd.DataFrame) -> Tuple[ray.data.Dataset, pd.DataFrame, np.array]:
    """Split the data in a stratified way.

    Returns:
        A tuple containing train dataset, test data and test label.
    """
    # There is a native offering in Ray Dataset for split as well.
    # However, supporting stratification is a TODO there. So use
    # scikit-learn equivalent here.
    train_data, test_data = train_test_split(
        data, stratify=data[[LABEL]], random_state=1113
    )
    _train_ds = ray.data.from_pandas(train_data)
    _test_label = test_data[LABEL].values
    _test_df = test_data.drop([LABEL], axis=1)
    return _train_ds, _test_df, _test_label

train_ds, test_df, test_label = split_data(data)
print(f"There are {train_ds.count()} samples for training and {test_df.shape[0]} samples for testing.")
There are 11251 samples for training and 3751 samples for testing.

Preprocessing

Let’s focus on preprocessing first. Usually, input data needs to go through some preprocessing before being fed into model. It is a good idea to package preprocessing logic into a modularized component so that the same logic can be applied to both training data as well as data for online serving or offline batch prediction.

In AIR, this component is a Preprocessor. It is constructed in a way that allows easy composition.

Now let’s construct a chained preprocessor composed of simple preprocessors, including

  1. Imputer for filling missing features;

  2. OneHotEncoder for encoding categorical features;

  3. BatchMapper where arbitrary user-defined function can be applied to batches of records; and so on. Take a look at Preprocessor. The output of the preprocessing step goes into model for training.

from ray.data.preprocessors import (
    BatchMapper,
    Chain,
    OneHotEncoder,
    SimpleImputer,
)

def get_preprocessor():
    """Construct a chain of preprocessors."""
    imputer1 = SimpleImputer(
        ["dropoff_census_tract"], strategy="most_frequent"
    )
    imputer2 = SimpleImputer(
        ["pickup_community_area", "dropoff_community_area"],
        strategy="most_frequent",
    )
    imputer3 = SimpleImputer(["payment_type"], strategy="most_frequent")
    imputer4 = SimpleImputer(
        ["company"], strategy="most_frequent")
    imputer5 = SimpleImputer(
        ["trip_start_timestamp", "trip_miles", "trip_seconds"], strategy="mean"
    )

    ohe = OneHotEncoder(
        columns=[
            "trip_start_hour",
            "trip_start_day",
            "trip_start_month",
            "dropoff_census_tract",
            "pickup_community_area",
            "dropoff_community_area",
            "payment_type",
            "company",
        ],
        max_categories={
            "dropoff_census_tract": 25,
            "pickup_community_area": 20,
            "dropoff_community_area": 20,
            "payment_type": 2,
            "company": 7,
        },
    )

    def batch_mapper_fn(df):
        df["trip_start_year"] = pd.to_datetime(df["trip_start_timestamp"], unit="s").dt.year
        df = df.drop(["trip_start_timestamp"], axis=1)
        return df

    def concat_for_tensor(dataframe):
        from ray.data.extensions import TensorArray
        result = {}
        feature_cols = [col for col in dataframe.columns if col != LABEL]
        result[INPUT] = TensorArray(dataframe[feature_cols].to_numpy(dtype=np.float32))
        if LABEL in dataframe.columns:
            result[LABEL] = dataframe[LABEL]
        return  pd.DataFrame(result)

    chained_pp = Chain(
        imputer1,
        imputer2,
        imputer3,
        imputer4,
        imputer5,
        ohe,
        BatchMapper(batch_mapper_fn, batch_format="pandas"),
        BatchMapper(concat_for_tensor, batch_format="pandas")
    )
    return chained_pp

Now let’s define some constants for clarity.

# Note that `INPUT_SIZE` here is corresponding to the output dimension
# of the previously defined processing steps.
# This is used to specify the input shape of Keras model.
INPUT_SIZE = 120
# The training batch size. Based on `NUM_WORKERS`, each worker
# will get its own share of this batch size. For example, if
# `NUM_WORKERS = 2`, each worker will work on 4 samples per batch.
BATCH_SIZE = 8
# Number of epoch. Adjust it based on how quickly you want the run to be.
EPOCH = 1
# Number of training workers.
# Adjust this accordingly based on the resources you have!
NUM_WORKERS = 2

Training

Let’s starting with defining a simple Keras model for the classification task.

import tensorflow as tf

def build_model():
    model = tf.keras.models.Sequential()
    model.add(tf.keras.Input(shape=(INPUT_SIZE,)))
    model.add(tf.keras.layers.Dense(50, activation="relu"))
    model.add(tf.keras.layers.Dense(1, activation="sigmoid"))
    return model
2022-09-15 12:47:46.916778: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2022-09-15 12:47:46.916796: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.

Now let’s define the training loop. This code will be run on each training worker in a distributed fashion. See more details here.

from ray.air import session, Checkpoint
from ray.train.tensorflow import TensorflowCheckpoint

def train_loop_per_worker():
    dataset_shard = session.get_dataset_shard("train")

    strategy = tf.distribute.experimental.MultiWorkerMirroredStrategy()
    with strategy.scope():
        model = build_model()
        model.compile(
            loss="binary_crossentropy",
            optimizer="adam",
            metrics=["accuracy"],
        )

    for epoch in range(EPOCH):            
        tf_dataset = dataset_shard.to_tf(feature_columns=INPUT, label_columns=LABEL, batch_size=BATCH_SIZE, drop_last=True)

        model.fit(tf_dataset, verbose=0)
        # This saves checkpoint in a way that can be used by Ray Serve coherently.
        session.report(
            {},
            checkpoint=TensorflowCheckpoint.from_model(model),
        )

Now let’s define a trainer that takes in the training loop, the training dataset as well the preprocessor that we just defined.

And run it!

Notice that you can tune how long you want the run to be by changing EPOCH.

from ray.train.tensorflow import TensorflowTrainer
from ray.air.config import ScalingConfig

trainer = TensorflowTrainer(
    train_loop_per_worker=train_loop_per_worker,
    scaling_config=ScalingConfig(num_workers=NUM_WORKERS),
    datasets={"train": train_ds},
    preprocessor=get_preprocessor(),
)
result = trainer.fit()

Tune Status

Current time:2022-09-15 12:48:04
Running for: 00:00:15.04
Memory: 13.3/62.7 GiB

System Info

Using FIFO scheduling algorithm.
Resources requested: 0/24 CPUs, 0/0 GPUs, 0.0/32.32 GiB heap, 0.0/16.16 GiB objects

Trial Status

Trial name status loc iter total time (s) _timestamp _time_this_iter_s _training_iteration
TensorflowTrainer_474c9_00000TERMINATED10.108.96.251:295304 1 13.0928 1663271284 9.65519 1
(pid=295304) 2022-09-15 12:47:50.848771: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
(pid=295304) 2022-09-15 12:47:50.848794: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(_map_block_nosplit pid=295335) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(_map_block_nosplit pid=295335)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(RayTrainWorker pid=295385) 2022-09-15 12:47:54.230543: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
(RayTrainWorker pid=295385) 2022-09-15 12:47:54.230561: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
(RayTrainWorker pid=295386) 2022-09-15 12:47:54.216251: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
(RayTrainWorker pid=295386) 2022-09-15 12:47:54.216269: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
(RayTrainWorker pid=295385) WARNING:tensorflow:From /tmp/ipykernel_294478/1947373158.py:7: _CollectiveAllReduceStrategyExperimental.__init__ (from tensorflow.python.distribute.collective_all_reduce_strategy) is deprecated and will be removed in a future version.
(RayTrainWorker pid=295385) Instructions for updating:
(RayTrainWorker pid=295385) use distribute.MultiWorkerMirroredStrategy instead
(RayTrainWorker pid=295385) 2022-09-15 12:47:55.157100: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
(RayTrainWorker pid=295385) 2022-09-15 12:47:55.157117: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
(RayTrainWorker pid=295385) 2022-09-15 12:47:55.157133: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (corvus): /proc/driver/nvidia/version does not exist
(RayTrainWorker pid=295385) 2022-09-15 12:47:55.157485: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
(RayTrainWorker pid=295385) To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
(RayTrainWorker pid=295385) 2022-09-15 12:47:55.171317: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:272] Initialize GrpcChannelCache for job worker -> {0 -> 10.108.96.251:55211, 1 -> 10.108.96.251:44543}
(RayTrainWorker pid=295385) 2022-09-15 12:47:55.171441: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:272] Initialize GrpcChannelCache for job worker -> {0 -> 10.108.96.251:55211, 1 -> 10.108.96.251:44543}
(RayTrainWorker pid=295385) 2022-09-15 12:47:55.172395: I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:438] Started server with target: grpc://10.108.96.251:55211
(RayTrainWorker pid=295386) WARNING:tensorflow:From /tmp/ipykernel_294478/1947373158.py:7: _CollectiveAllReduceStrategyExperimental.__init__ (from tensorflow.python.distribute.collective_all_reduce_strategy) is deprecated and will be removed in a future version.
(RayTrainWorker pid=295386) Instructions for updating:
(RayTrainWorker pid=295386) use distribute.MultiWorkerMirroredStrategy instead
(RayTrainWorker pid=295386) 2022-09-15 12:47:55.157159: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
(RayTrainWorker pid=295386) 2022-09-15 12:47:55.157173: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
(RayTrainWorker pid=295386) 2022-09-15 12:47:55.157185: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (corvus): /proc/driver/nvidia/version does not exist
(RayTrainWorker pid=295386) 2022-09-15 12:47:55.157520: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
(RayTrainWorker pid=295386) To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
(RayTrainWorker pid=295386) 2022-09-15 12:47:55.170593: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:272] Initialize GrpcChannelCache for job worker -> {0 -> 10.108.96.251:55211, 1 -> 10.108.96.251:44543}
(RayTrainWorker pid=295386) 2022-09-15 12:47:55.170771: I tensorflow/core/distributed_runtime/rpc/grpc_channel.cc:272] Initialize GrpcChannelCache for job worker -> {0 -> 10.108.96.251:55211, 1 -> 10.108.96.251:44543}
(RayTrainWorker pid=295386) 2022-09-15 12:47:55.171709: I tensorflow/core/distributed_runtime/rpc/grpc_server_lib.cc:438] Started server with target: grpc://10.108.96.251:44543
(RayTrainWorker pid=295385) 2022-09-15 12:47:55.310107: W tensorflow/core/framework/dataset.cc:768] Input of GeneratorDatasetOp::Dataset will not be optimized because the dataset does not implement the AsGraphDefInternal() method needed to apply optimizations.
(RayTrainWorker pid=295386) 2022-09-15 12:47:55.296963: W tensorflow/core/framework/dataset.cc:768] Input of GeneratorDatasetOp::Dataset will not be optimized because the dataset does not implement the AsGraphDefInternal() method needed to apply optimizations.

Trial Progress

Trial name _time_this_iter_s _timestamp _training_iterationdate done episodes_total experiment_id experiment_taghostname iterations_since_restorenode_ip pidshould_checkpoint time_since_restore time_this_iter_s time_total_s timestamp timesteps_since_restoretimesteps_total training_iterationtrial_id warmup_time
TensorflowTrainer_474c9_00000 9.65519 1663271284 12022-09-15_12-48-04True 396e9e2217344c59835ca753e7b794e6 0corvus 110.108.96.251295304True 13.0928 13.0928 13.0928 1663271284 0 1474c9_00000 0.00268364
2022-09-15 12:48:05,026	INFO tune.py:762 -- Total run time: 15.61 seconds (15.03 seconds for the tuning loop).

Moving on to Serve

We will use Ray Serve to serve the trained model. A core concept of Ray Serve is Deployment. It allows you to define and update your business logic or models that will handle incoming requests as well as how this is exposed over HTTP or in Python.

In the case of serving model, ray.serve.air_integrations.Predictor and ray.serve.air_integrations.PredictorDeployment wrap a ray.air.checkpoint.Checkpoint into a Ray Serve deployment that can readily serve HTTP requests. Note, Checkpoint captures both model and preprocessing steps in a way compatible with Ray Serve and ensures that ml workload can transition seamlessly between training and serving.

This removes the boilerplate code and minimizes the effort to bring your model to production!

Generally speaking the http request can either send in json or data. Upon receiving this payload, Ray Serve would need some “adapter” to convert the request payload into some shape or form that can be accepted as input to the preprocessing steps. In this case, we send in a json request and converts it to a pandas DataFrame through dataframe_adapter, defined below.

from fastapi import Request

async def dataframe_adapter(request: Request):
    """Serve HTTP Adapter that reads JSON and converts to pandas DataFrame."""
    content = await request.json()
    return pd.DataFrame.from_dict(content)

Now let’s wrap everything in a serve endpoint that exposes a URL to where requests can be sent to.

from ray import serve
from ray.air.checkpoint import Checkpoint
from ray.train.tensorflow import TensorflowPredictor
from ray.serve import PredictorDeployment


def serve_model(checkpoint: Checkpoint, model_definition, adapter, name="Model") -> str:
    """Expose a serve endpoint.

    Returns:
        serve URL.
    """
    serve.run(
        PredictorDeployment.options(name=name).bind(
            TensorflowPredictor,
            checkpoint,
            batching_params=dict(max_batch_size=2, batch_wait_timeout_s=5),
            model_definition=model_definition,
            http_adapter=adapter,
        )
    )
    return f"http://localhost:8000/"
import ray
# Generally speaking, training and serving are done in totally different ray clusters.
# To simulate that, let's shutdown the old ray cluster in preparation for serving.
ray.shutdown()

endpoint_uri = serve_model(result.checkpoint, build_model, dataframe_adapter)
2022-09-15 12:48:08,857	INFO worker.py:1508 -- Started a local Ray instance. View the dashboard at 127.0.0.1:8265 
(ServeController pid=296884) INFO 2022-09-15 12:48:10,001 controller 296884 http_state.py:129 - Starting HTTP proxy with name 'SERVE_CONTROLLER_ACTOR:SERVE_PROXY_ACTOR-e267525ede59531a70cb23315fa9c2541b01709d7d70bd94279272a9' on node 'e267525ede59531a70cb23315fa9c2541b01709d7d70bd94279272a9' listening on '127.0.0.1:8000'
(HTTPProxyActor pid=296922) INFO:     Started server process [296922]
(ServeController pid=296884) INFO 2022-09-15 12:48:10,617 controller 296884 deployment_state.py:1277 - Adding 1 replica to deployment 'Model'.
(ServeReplica:Model pid=296954) 2022-09-15 12:48:11.264200: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
(ServeReplica:Model pid=296954) 2022-09-15 12:48:11.264217: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
(ServeReplica:Model pid=296954) 2022-09-15 12:48:12.148951: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
(ServeReplica:Model pid=296954) 2022-09-15 12:48:12.148965: W tensorflow/stream_executor/cuda/cuda_driver.cc:269] failed call to cuInit: UNKNOWN ERROR (303)
(ServeReplica:Model pid=296954) 2022-09-15 12:48:12.148980: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (corvus): /proc/driver/nvidia/version does not exist
(ServeReplica:Model pid=296954) 2022-09-15 12:48:12.149111: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
(ServeReplica:Model pid=296954) To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

Let’s write a helper function to send requests to this endpoint and compare the results with labels.

import json
import requests
import pandas as pd
import numpy as np

NUM_SERVE_REQUESTS = 10

def send_requests(df: pd.DataFrame, label: np.array):
    for i in range(NUM_SERVE_REQUESTS):
        one_row = df.iloc[[i]].to_dict()
        serve_result = requests.post(endpoint_uri, data=json.dumps(one_row), headers={"Content-Type": "application/json"}).json()
        print(
            f"request{i} prediction: {serve_result[0]['predictions']} "
            f"- label: {str(label[i])}"
        )
send_requests(test_df, test_label)
request0 prediction: 3.7036345474916743e-06 - label: True
(HTTPProxyActor pid=296922) INFO 2022-09-15 12:48:21,875 http_proxy 10.108.96.251 http_proxy.py:315 - POST / 200 5067.7ms
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /tmp/ipykernel_294478/255618366.py:45: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954) INFO 2022-09-15 12:48:21,874 Model Model#iuFpgx replica.py:505 - HANDLE __call__ OK 5063.9ms
request1 prediction: 0.00018989769159816206 - label: False
(HTTPProxyActor pid=296922) INFO 2022-09-15 12:48:26,935 http_proxy 10.108.96.251 http_proxy.py:315 - POST / 200 5055.4ms
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /tmp/ipykernel_294478/255618366.py:45: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954) INFO 2022-09-15 12:48:26,934 Model Model#iuFpgx replica.py:505 - HANDLE __call__ OK 5053.3ms
request2 prediction: 8.654507333005768e-09 - label: False
(HTTPProxyActor pid=296922) INFO 2022-09-15 12:48:31,996 http_proxy 10.108.96.251 http_proxy.py:315 - POST / 200 5057.0ms
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /tmp/ipykernel_294478/255618366.py:45: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954) INFO 2022-09-15 12:48:31,995 Model Model#iuFpgx replica.py:505 - HANDLE __call__ OK 5054.5ms
request3 prediction: 1.922958574596123e-07 - label: False
(HTTPProxyActor pid=296922) INFO 2022-09-15 12:48:37,056 http_proxy 10.108.96.251 http_proxy.py:315 - POST / 200 5056.5ms
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /tmp/ipykernel_294478/255618366.py:45: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954) INFO 2022-09-15 12:48:37,055 Model Model#iuFpgx replica.py:505 - HANDLE __call__ OK 5054.5ms
request4 prediction: 2.8968736387469107e-07 - label: False
(HTTPProxyActor pid=296922) INFO 2022-09-15 12:48:42,114 http_proxy 10.108.96.251 http_proxy.py:315 - POST / 200 5054.4ms
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /tmp/ipykernel_294478/255618366.py:45: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954) INFO 2022-09-15 12:48:42,113 Model Model#iuFpgx replica.py:505 - HANDLE __call__ OK 5052.2ms
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /tmp/ipykernel_294478/255618366.py:45: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(HTTPProxyActor pid=296922) INFO 2022-09-15 12:48:47,187 http_proxy 10.108.96.251 http_proxy.py:315 - POST / 200 5068.5ms
(ServeReplica:Model pid=296954) INFO 2022-09-15 12:48:47,186 Model Model#iuFpgx replica.py:505 - HANDLE __call__ OK 5066.2ms
request5 prediction: 1.6935454150157625e-09 - label: False
request6 prediction: 5.132831937260107e-09 - label: False
(HTTPProxyActor pid=296922) INFO 2022-09-15 12:48:52,263 http_proxy 10.108.96.251 http_proxy.py:315 - POST / 200 5073.2ms
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /tmp/ipykernel_294478/255618366.py:45: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954) INFO 2022-09-15 12:48:52,262 Model Model#iuFpgx replica.py:505 - HANDLE __call__ OK 5070.9ms
request7 prediction: 0.0011921558761969209 - label: False
(HTTPProxyActor pid=296922) INFO 2022-09-15 12:48:57,336 http_proxy 10.108.96.251 http_proxy.py:315 - POST / 200 5068.3ms
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /tmp/ipykernel_294478/255618366.py:45: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954) INFO 2022-09-15 12:48:57,335 Model Model#iuFpgx replica.py:505 - HANDLE __call__ OK 5065.8ms
request8 prediction: 1.3840471879689176e-08 - label: False
(HTTPProxyActor pid=296922) INFO 2022-09-15 12:49:02,412 http_proxy 10.108.96.251 http_proxy.py:315 - POST / 200 5072.7ms
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /tmp/ipykernel_294478/255618366.py:45: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954) INFO 2022-09-15 12:49:02,411 Model Model#iuFpgx replica.py:505 - HANDLE __call__ OK 5070.3ms
request9 prediction: 0.00012493788381107152 - label: True
(HTTPProxyActor pid=296922) INFO 2022-09-15 12:49:07,485 http_proxy 10.108.96.251 http_proxy.py:315 - POST / 200 5069.0ms
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /home/pdmurray/Desktop/workspace/ray/python/ray/data/preprocessors/encoder.py:225: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954)   df[f"{column}_{column_value}"] = (df[column] == column_value).astype(
(ServeReplica:Model pid=296954) /tmp/ipykernel_294478/255618366.py:45: PerformanceWarning: DataFrame is highly fragmented.  This is usually the result of calling `frame.insert` many times, which has poor performance.  Consider joining all columns at once using pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe = frame.copy()`
(ServeReplica:Model pid=296954) INFO 2022-09-15 12:49:07,484 Model Model#iuFpgx replica.py:505 - HANDLE __call__ OK 5066.8ms