Training a model with distributed XGBoost
Training a model with distributed XGBoost#
In this example we will train a model in Ray AIR using distributed XGBoost.
Let’s start with installing our dependencies:
!pip install -qU "ray[tune]" xgboost_ray
Then we need some imports:
from typing import Tuple
import ray
from ray.train.batch_predictor import BatchPredictor
from ray.train.xgboost import XGBoostPredictor
from ray.train.xgboost import XGBoostTrainer
from ray.air.config import ScalingConfig
from ray.data.dataset import Dataset
from ray.air.result import Result
from ray.data.preprocessors import StandardScaler
/home/ubuntu/ray/venv/lib/python3.8/site-packages/xgboost/compat.py:31: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
from pandas import MultiIndex, Int64Index
FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
FutureWarning: pandas.Float64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
FutureWarning: pandas.UInt64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
Next we define a function to load our train, validation, and test datasets.
def prepare_data() -> Tuple[Dataset, Dataset, Dataset]:
dataset = ray.data.read_csv("s3://anonymous@air-example-data/breast_cancer.csv")
train_dataset, valid_dataset = dataset.train_test_split(test_size=0.3)
test_dataset = valid_dataset.drop_columns(["target"])
return train_dataset, valid_dataset, test_dataset
The following function will create a XGBoost trainer, train it, and return the result.
def train_xgboost(num_workers: int, use_gpu: bool = False) -> Result:
train_dataset, valid_dataset, _ = prepare_data()
# Scale some random columns
columns_to_scale = ["mean radius", "mean texture"]
preprocessor = StandardScaler(columns=columns_to_scale)
# XGBoost specific params
params = {
"tree_method": "approx",
"objective": "binary:logistic",
"eval_metric": ["logloss", "error"],
}
trainer = XGBoostTrainer(
scaling_config=ScalingConfig(num_workers=num_workers, use_gpu=use_gpu),
label_column="target",
params=params,
datasets={"train": train_dataset, "valid": valid_dataset},
preprocessor=preprocessor,
num_boost_round=100,
)
result = trainer.fit()
print(result.metrics)
return result
Once we have the result, we can do batch inference on the obtained model. Let’s define a utility function for this.
def predict_xgboost(result: Result):
_, _, test_dataset = prepare_data()
batch_predictor = BatchPredictor.from_checkpoint(
result.checkpoint, XGBoostPredictor
)
predicted_labels = (
batch_predictor.predict(test_dataset)
.map_batches(lambda df: (df > 0.5).astype(int), batch_format="pandas")
)
print(f"PREDICTED LABELS")
predicted_labels.show()
shap_values = batch_predictor.predict(test_dataset, pred_contribs=True)
print(f"SHAP VALUES")
shap_values.show()
Now we can run the training:
result = train_xgboost(num_workers=2, use_gpu=False)
2022-06-22 17:28:55,841 INFO services.py:1477 -- View the Ray dashboard at http://127.0.0.1:8270
2022-06-22 17:28:58,044 WARNING read_api.py:260 -- The number of blocks in this dataset (1) limits its parallelism to 1 concurrent tasks. This is much less than the number of available CPU slots in the cluster. Use `.repartition(n)` to increase the number of dataset blocks.
Map_Batches: 100%|██████████| 1/1 [00:00<00:00, 40.28it/s]
== Status ==
Current time: 2022-06-22 17:29:15 (running for 00:00:16.11)
Memory usage on this node: 11.5/31.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 0/8 CPUs, 0/0 GPUs, 0.0/12.35 GiB heap, 0.0/6.18 GiB objects
Result logdir: /home/ubuntu/ray_results/XGBoostTrainer_2022-06-22_17-28-58
Number of trials: 1/1 (1 TERMINATED)
Current time: 2022-06-22 17:29:15 (running for 00:00:16.11)
Memory usage on this node: 11.5/31.0 GiB
Using FIFO scheduling algorithm.
Resources requested: 0/8 CPUs, 0/0 GPUs, 0.0/12.35 GiB heap, 0.0/6.18 GiB objects
Result logdir: /home/ubuntu/ray_results/XGBoostTrainer_2022-06-22_17-28-58
Number of trials: 1/1 (1 TERMINATED)
| Trial name | status | loc | iter | total time (s) | train-logloss | train-error | valid-logloss |
|---|---|---|---|---|---|---|---|
| XGBoostTrainer_cc863_00000 | TERMINATED | 172.31.43.110:1493910 | 100 | 12.5164 | 0.005874 | 0 | 0.078188 |
(pid=1493910) /home/ubuntu/ray/venv/lib/python3.8/site-packages/xgboost/compat.py:31: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1493910) from pandas import MultiIndex, Int64Index
(pid=1493910) FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1493910) FutureWarning: pandas.Float64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1493910) FutureWarning: pandas.UInt64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(XGBoostTrainer pid=1493910) UserWarning: Dataset 'train' has 1 blocks, which is less than the `num_workers` 2. This dataset will be automatically repartitioned to 2 blocks.
(XGBoostTrainer pid=1493910) UserWarning: Dataset 'valid' has 1 blocks, which is less than the `num_workers` 2. This dataset will be automatically repartitioned to 2 blocks.
(XGBoostTrainer pid=1493910) 2022-06-22 17:29:04,073 INFO main.py:980 -- [RayXGBoost] Created 2 new actors (2 total actors). Waiting until actors are ready for training.
(pid=1494007) /home/ubuntu/ray/venv/lib/python3.8/site-packages/xgboost/compat.py:31: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494007) from pandas import MultiIndex, Int64Index
(pid=1494008) /home/ubuntu/ray/venv/lib/python3.8/site-packages/xgboost/compat.py:31: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494008) from pandas import MultiIndex, Int64Index
(pid=1494009) /home/ubuntu/ray/venv/lib/python3.8/site-packages/xgboost/compat.py:31: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494009) from pandas import MultiIndex, Int64Index
(pid=1494007) FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494007) FutureWarning: pandas.Float64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494007) FutureWarning: pandas.UInt64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494008) FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494008) FutureWarning: pandas.Float64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494008) FutureWarning: pandas.UInt64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(_RemoteRayXGBoostActor pid=1494008) 2022-06-22 17:29:07,324 WARNING __init__.py:190 -- DeprecationWarning: `ray.worker.get_resource_ids` is a private attribute and access will be removed in a future Ray version.
(pid=1494009) FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494009) FutureWarning: pandas.Float64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494009) FutureWarning: pandas.UInt64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(_RemoteRayXGBoostActor pid=1494009) 2022-06-22 17:29:07,421 WARNING __init__.py:190 -- DeprecationWarning: `ray.worker.get_resource_ids` is a private attribute and access will be removed in a future Ray version.
(XGBoostTrainer pid=1493910) 2022-06-22 17:29:07,874 INFO main.py:1025 -- [RayXGBoost] Starting XGBoost training.
(_RemoteRayXGBoostActor pid=1494008) [17:29:07] task [xgboost.ray]:139731353900128 got new rank 0
(_RemoteRayXGBoostActor pid=1494008) FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(_RemoteRayXGBoostActor pid=1494009) [17:29:07] task [xgboost.ray]:140076138558608 got new rank 1
(_RemoteRayXGBoostActor pid=1494009) FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(_QueueActor pid=1494006) /home/ubuntu/ray/venv/lib/python3.8/site-packages/xgboost/compat.py:31: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(_QueueActor pid=1494006) from pandas import MultiIndex, Int64Index
(_QueueActor pid=1494006) FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(_QueueActor pid=1494006) FutureWarning: pandas.Float64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(_QueueActor pid=1494006) FutureWarning: pandas.UInt64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
Result for XGBoostTrainer_cc863_00000:
date: 2022-06-22_17-29-09
done: false
experiment_id: dc3dac01a34043cfb5751907e2bc648e
hostname: ip-172-31-43-110
iterations_since_restore: 1
node_ip: 172.31.43.110
pid: 1493910
should_checkpoint: true
time_since_restore: 7.967940330505371
time_this_iter_s: 7.967940330505371
time_total_s: 7.967940330505371
timestamp: 1655918949
timesteps_since_restore: 0
train-error: 0.017588
train-logloss: 0.464648
training_iteration: 1
trial_id: cc863_00000
valid-error: 0.081871
valid-logloss: 0.496374
warmup_time: 0.004768848419189453
(XGBoostTrainer pid=1493910) 2022-06-22 17:29:14,546 INFO main.py:1516 -- [RayXGBoost] Finished XGBoost training on training data with total N=398 in 10.52 seconds (6.66 pure XGBoost training time).
Result for XGBoostTrainer_cc863_00000:
date: 2022-06-22_17-29-14
done: true
experiment_id: dc3dac01a34043cfb5751907e2bc648e
experiment_tag: '0'
hostname: ip-172-31-43-110
iterations_since_restore: 100
node_ip: 172.31.43.110
pid: 1493910
should_checkpoint: true
time_since_restore: 12.516392230987549
time_this_iter_s: 0.03008890151977539
time_total_s: 12.516392230987549
timestamp: 1655918954
timesteps_since_restore: 0
train-error: 0.0
train-logloss: 0.005874
training_iteration: 100
trial_id: cc863_00000
valid-error: 0.040936
valid-logloss: 0.078188
warmup_time: 0.004768848419189453
2022-06-22 17:29:15,362 INFO tune.py:734 -- Total run time: 16.94 seconds (16.08 seconds for the tuning loop).
{'train-logloss': 0.005874, 'train-error': 0.0, 'valid-logloss': 0.078188, 'valid-error': 0.040936, 'time_this_iter_s': 0.03008890151977539, 'should_checkpoint': True, 'done': True, 'timesteps_total': None, 'episodes_total': None, 'training_iteration': 100, 'trial_id': 'cc863_00000', 'experiment_id': 'dc3dac01a34043cfb5751907e2bc648e', 'date': '2022-06-22_17-29-14', 'timestamp': 1655918954, 'time_total_s': 12.516392230987549, 'pid': 1493910, 'hostname': 'ip-172-31-43-110', 'node_ip': '172.31.43.110', 'config': {}, 'time_since_restore': 12.516392230987549, 'timesteps_since_restore': 0, 'iterations_since_restore': 100, 'warmup_time': 0.004768848419189453, 'experiment_tag': '0'}
And perform inference on the obtained model:
predict_xgboost(result)
2022-06-22 17:29:16,463 WARNING read_api.py:260 -- The number of blocks in this dataset (1) limits its parallelism to 1 concurrent tasks. This is much less than the number of available CPU slots in the cluster. Use `.repartition(n)` to increase the number of dataset blocks.
Map_Batches: 100%|██████████| 1/1 [00:00<00:00, 46.14it/s]
Map_Batches: 0%| | 0/1 [00:00<?, ?it/s](pid=1494373) /home/ubuntu/ray/venv/lib/python3.8/site-packages/xgboost/compat.py:31: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494373) from pandas import MultiIndex, Int64Index
(pid=1494373) FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494373) FutureWarning: pandas.Float64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494373) FutureWarning: pandas.UInt64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
Map Progress (1 actors 1 pending): 100%|██████████| 1/1 [00:01<00:00, 1.90s/it]
(BlockWorker pid=1494373) FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
Map_Batches: 100%|██████████| 1/1 [00:00<00:00, 75.10it/s]
PREDICTED LABELS
{'predictions': 1}
{'predictions': 1}
{'predictions': 0}
{'predictions': 1}
{'predictions': 1}
{'predictions': 1}
{'predictions': 1}
{'predictions': 1}
{'predictions': 0}
{'predictions': 1}
{'predictions': 0}
{'predictions': 1}
{'predictions': 1}
{'predictions': 1}
{'predictions': 1}
{'predictions': 0}
{'predictions': 0}
{'predictions': 1}
{'predictions': 1}
{'predictions': 0}
Map_Batches: 0%| | 0/1 [00:00<?, ?it/s](pid=1494403) /home/ubuntu/ray/venv/lib/python3.8/site-packages/xgboost/compat.py:31: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494403) from pandas import MultiIndex, Int64Index
(pid=1494413) /home/ubuntu/ray/venv/lib/python3.8/site-packages/xgboost/compat.py:31: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494413) from pandas import MultiIndex, Int64Index
(pid=1494403) FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494403) FutureWarning: pandas.Float64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494403) FutureWarning: pandas.UInt64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494413) FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494413) FutureWarning: pandas.Float64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494413) FutureWarning: pandas.UInt64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
Map Progress (1 actors 1 pending): 100%|██████████| 1/1 [00:01<00:00, 1.88s/it]
SHAP VALUES
{'predictions_0': 0.009930070489645004, 'predictions_1': 0.546318531036377, 'predictions_2': -0.006533853709697723, 'predictions_3': 0.022934239357709885, 'predictions_4': 0.32817941904067993, 'predictions_5': 0.004407345782965422, 'predictions_6': 0.013881205581128597, 'predictions_7': 0.568859875202179, 'predictions_8': -0.27460771799087524, 'predictions_9': 0.013218197971582413, 'predictions_10': 0.009325551800429821, 'predictions_11': 0.04015672579407692, 'predictions_12': 0.11667086184024811, 'predictions_13': 0.9853533506393433, 'predictions_14': 0.05529181659221649, 'predictions_15': -0.005734208971261978, 'predictions_16': -0.0008497871458530426, 'predictions_17': 0.16138489544391632, 'predictions_18': -0.36162295937538147, 'predictions_19': 0.003658014815300703, 'predictions_20': 0.393682062625885, 'predictions_21': 0.6647266149520874, 'predictions_22': 1.7201099395751953, 'predictions_23': 0.35084351897239685, 'predictions_24': 0.4841834604740143, 'predictions_25': 0.013311417773365974, 'predictions_26': 0.8087116479873657, 'predictions_27': 0.5730299353599548, 'predictions_28': 0.1818174123764038, 'predictions_29': 0.1068669781088829, 'predictions_30': 0.5059375166893005}
{'predictions_0': 0.009930070489645004, 'predictions_1': 0.39265793561935425, 'predictions_2': -0.007271876558661461, 'predictions_3': 0.022934239357709885, 'predictions_4': -0.06737710535526276, 'predictions_5': 0.004407345782965422, 'predictions_6': 0.04250117391347885, 'predictions_7': 0.7946916222572327, 'predictions_8': 0.30412089824676514, 'predictions_9': 0.013218197971582413, 'predictions_10': 0.009325551800429821, 'predictions_11': 0.04129280149936676, 'predictions_12': -0.012254374101758003, 'predictions_13': 0.2327558547258377, 'predictions_14': 0.05529181659221649, 'predictions_15': 0.6405980587005615, 'predictions_16': -0.0014769809786230326, 'predictions_17': 0.16138489544391632, 'predictions_18': -0.3877210319042206, 'predictions_19': -0.0022315792739391327, 'predictions_20': 0.3209536671638489, 'predictions_21': -0.018735788762569427, 'predictions_22': 1.482913851737976, 'predictions_23': 0.3135913908481598, 'predictions_24': 0.5150958299636841, 'predictions_25': 0.013311417773365974, 'predictions_26': 1.0875523090362549, 'predictions_27': 0.7323897480964661, 'predictions_28': 0.12313760071992874, 'predictions_29': 0.1068669781088829, 'predictions_30': 0.5059375166893005}
{'predictions_0': -0.01911582238972187, 'predictions_1': -0.41554388403892517, 'predictions_2': -0.0034923271741718054, 'predictions_3': -0.06306137144565582, 'predictions_4': -0.3192429542541504, 'predictions_5': -0.009094981476664543, 'predictions_6': -0.08258295059204102, 'predictions_7': -0.8088644742965698, 'predictions_8': 0.2545676529407501, 'predictions_9': 0.015295587480068207, 'predictions_10': 0.0021318818908184767, 'predictions_11': -0.003732672892510891, 'predictions_12': -0.054907385259866714, 'predictions_13': -0.9062053561210632, 'predictions_14': -0.029618918895721436, 'predictions_15': 0.4303477704524994, 'predictions_16': -0.007885736413300037, 'predictions_17': 0.17541413009166718, 'predictions_18': -0.3190936744213104, 'predictions_19': 0.0024438181426376104, 'predictions_20': -0.6092430353164673, 'predictions_21': -0.3519248068332672, 'predictions_22': -2.2588469982147217, 'predictions_23': -0.569831371307373, 'predictions_24': -1.0422284603118896, 'predictions_25': -0.031086977571249008, 'predictions_26': -0.5648106932640076, 'predictions_27': -1.0225528478622437, 'predictions_28': -0.1411924958229065, 'predictions_29': -0.1724514663219452, 'predictions_30': 0.5059375166893005}
{'predictions_0': 0.009930070489645004, 'predictions_1': 0.5999099016189575, 'predictions_2': -0.007271876558661461, 'predictions_3': 0.022934239357709885, 'predictions_4': 0.31011122465133667, 'predictions_5': 0.004407345782965422, 'predictions_6': 0.013881205581128597, 'predictions_7': 0.5183905363082886, 'predictions_8': -0.27460771799087524, 'predictions_9': -0.015900276601314545, 'predictions_10': 0.009325551800429821, 'predictions_11': 0.04129280149936676, 'predictions_12': 0.11667086184024811, 'predictions_13': 1.0816324949264526, 'predictions_14': -0.01614229381084442, 'predictions_15': -0.23922261595726013, 'predictions_16': -0.0008497871458530426, 'predictions_17': 0.16138489544391632, 'predictions_18': -0.36162295937538147, 'predictions_19': -0.0022315792739391327, 'predictions_20': 0.3189953863620758, 'predictions_21': 0.6754519939422607, 'predictions_22': 1.513157606124878, 'predictions_23': 0.35084351897239685, 'predictions_24': 0.09065212309360504, 'predictions_25': 0.013311417773365974, 'predictions_26': 1.039388656616211, 'predictions_27': 0.5762963891029358, 'predictions_28': 0.1816803514957428, 'predictions_29': 0.1068669781088829, 'predictions_30': 0.5059375166893005}
{'predictions_0': 0.009930070489645004, 'predictions_1': 0.41290122270584106, 'predictions_2': -0.007271876558661461, 'predictions_3': 0.04113232344388962, 'predictions_4': 0.4761468768119812, 'predictions_5': 0.004407345782965422, 'predictions_6': 0.04250117391347885, 'predictions_7': 0.6889638304710388, 'predictions_8': 0.30412089824676514, 'predictions_9': -0.00703495554625988, 'predictions_10': 0.009325551800429821, 'predictions_11': 0.04129280149936676, 'predictions_12': -0.012254374101758003, 'predictions_13': 0.8955986499786377, 'predictions_14': 0.05529181659221649, 'predictions_15': 0.6405980587005615, 'predictions_16': -0.0010774275287985802, 'predictions_17': 0.16581103205680847, 'predictions_18': 0.4200459420681, 'predictions_19': 0.003658014815300703, 'predictions_20': 0.31559276580810547, 'predictions_21': 0.08991634845733643, 'predictions_22': 1.374340295791626, 'predictions_23': 0.4179628789424896, 'predictions_24': 0.6432731747627258, 'predictions_25': 0.013311417773365974, 'predictions_26': 1.0250954627990723, 'predictions_27': 0.6740144491195679, 'predictions_28': -0.12222569435834885, 'predictions_29': 0.1068669781088829, 'predictions_30': 0.5059375166893005}
{'predictions_0': 0.009930070489645004, 'predictions_1': 0.6223654747009277, 'predictions_2': -0.007271876558661461, 'predictions_3': 0.04113232344388962, 'predictions_4': 0.030091704800724983, 'predictions_5': 0.004407345782965422, 'predictions_6': 0.04250117391347885, 'predictions_7': 0.520163893699646, 'predictions_8': -0.13476626574993134, 'predictions_9': 0.013218197971582413, 'predictions_10': 0.004969821777194738, 'predictions_11': 0.04129280149936676, 'predictions_12': 0.11667086184024811, 'predictions_13': 1.0668600797653198, 'predictions_14': -0.46553897857666016, 'predictions_15': -0.06545835733413696, 'predictions_16': -0.0010774275287985802, 'predictions_17': 0.16138489544391632, 'predictions_18': 0.406240850687027, 'predictions_19': -0.004930071532726288, 'predictions_20': 0.303047239780426, 'predictions_21': 0.7582043409347534, 'predictions_22': 1.4096102714538574, 'predictions_23': 0.43191614747047424, 'predictions_24': 0.47695621848106384, 'predictions_25': 0.013311417773365974, 'predictions_26': 1.0482016801834106, 'predictions_27': 0.5110929608345032, 'predictions_28': -0.12222569435834885, 'predictions_29': 0.1068669781088829, 'predictions_30': 0.5059375166893005}
{'predictions_0': 0.009930070489645004, 'predictions_1': 0.5661754012107849, 'predictions_2': -0.0019909553229808807, 'predictions_3': 0.04113232344388962, 'predictions_4': 0.292957067489624, 'predictions_5': 0.004407345782965422, 'predictions_6': 0.013881205581128597, 'predictions_7': 0.6406195759773254, 'predictions_8': -0.27460771799087524, 'predictions_9': -0.007835762575268745, 'predictions_10': 0.009325551800429821, 'predictions_11': 0.04129280149936676, 'predictions_12': -0.012254374101758003, 'predictions_13': 0.21169273555278778, 'predictions_14': -0.02233714796602726, 'predictions_15': -0.9655348658561707, 'predictions_16': -0.0010828523663803935, 'predictions_17': 0.16581103205680847, 'predictions_18': 0.4200459420681, 'predictions_19': -0.0022315792739391327, 'predictions_20': 0.3136344850063324, 'predictions_21': 0.7264875173568726, 'predictions_22': 1.5534507036209106, 'predictions_23': 0.41967567801475525, 'predictions_24': 0.41908225417137146, 'predictions_25': 0.013311417773365974, 'predictions_26': 1.0032241344451904, 'predictions_27': 0.6091798543930054, 'predictions_28': 0.18174558877944946, 'predictions_29': 0.08268986642360687, 'predictions_30': 0.5059375166893005}
{'predictions_0': 0.009930070489645004, 'predictions_1': 0.40430790185928345, 'predictions_2': -0.007271876558661461, 'predictions_3': 0.04113232344388962, 'predictions_4': -0.33117765188217163, 'predictions_5': 0.004407345782965422, 'predictions_6': 0.013881205581128597, 'predictions_7': 0.737379252910614, 'predictions_8': -0.27460771799087524, 'predictions_9': 0.007771771401166916, 'predictions_10': 0.009325551800429821, 'predictions_11': 0.04129280149936676, 'predictions_12': -0.012254374101758003, 'predictions_13': 0.18173672258853912, 'predictions_14': -0.02233714796602726, 'predictions_15': 0.6811268329620361, 'predictions_16': -0.0014769809786230326, 'predictions_17': 0.16581103205680847, 'predictions_18': 0.406240850687027, 'predictions_19': -0.0022315792739391327, 'predictions_20': 0.31559276580810547, 'predictions_21': -0.031238339841365814, 'predictions_22': 1.7573171854019165, 'predictions_23': 0.3838556110858917, 'predictions_24': 0.14492939412593842, 'predictions_25': 0.013311417773365974, 'predictions_26': 1.1148661375045776, 'predictions_27': 0.7173758149147034, 'predictions_28': 0.18174558877944946, 'predictions_29': 0.1068669781088829, 'predictions_30': 0.5059375166893005}
{'predictions_0': -0.11485510319471359, 'predictions_1': 0.8875605463981628, 'predictions_2': -0.05685592442750931, 'predictions_3': -0.06306137144565582, 'predictions_4': 0.013515152037143707, 'predictions_5': 0.0024520084261894226, 'predictions_6': 0.04250117391347885, 'predictions_7': 0.4688224792480469, 'predictions_8': -0.14307911694049835, 'predictions_9': -0.06901921331882477, 'predictions_10': 0.003290211781859398, 'predictions_11': 0.003781725186854601, 'predictions_12': 0.08078579604625702, 'predictions_13': 0.8216619491577148, 'predictions_14': 0.0463210791349411, 'predictions_15': -0.13255546987056732, 'predictions_16': -0.0010776874842122197, 'predictions_17': 0.21504947543144226, 'predictions_18': -0.27619078755378723, 'predictions_19': -0.0025743553414940834, 'predictions_20': -0.9275118112564087, 'predictions_21': 1.0307989120483398, 'predictions_22': -2.7721312046051025, 'predictions_23': -0.4765593707561493, 'predictions_24': 0.5464398264884949, 'predictions_25': 0.004862718749791384, 'predictions_26': -0.619586706161499, 'predictions_27': -0.2188473790884018, 'predictions_28': 0.10512445122003555, 'predictions_29': 0.1068669781088829, 'predictions_30': 0.5059375166893005}
{'predictions_0': 0.0011090459302067757, 'predictions_1': -0.5595121383666992, 'predictions_2': -0.002613282995298505, 'predictions_3': 0.04113232344388962, 'predictions_4': 0.5254822969436646, 'predictions_5': 0.004407345782965422, 'predictions_6': 0.013881205581128597, 'predictions_7': 0.5551156997680664, 'predictions_8': -0.27460771799087524, 'predictions_9': 0.013218197971582413, 'predictions_10': 0.009325551800429821, 'predictions_11': 0.03278094530105591, 'predictions_12': -0.012305313721299171, 'predictions_13': -1.6061651706695557, 'predictions_14': -0.01887715980410576, 'predictions_15': 0.6405980587005615, 'predictions_16': -0.0014769809786230326, 'predictions_17': -0.1776299774646759, 'predictions_18': 0.4914793074131012, 'predictions_19': 0.014222224242985249, 'predictions_20': 0.311679869890213, 'predictions_21': -0.08892179280519485, 'predictions_22': 1.5504939556121826, 'predictions_23': 0.5073927044868469, 'predictions_24': 0.6315706968307495, 'predictions_25': 0.005095706321299076, 'predictions_26': 0.7821602821350098, 'predictions_27': 0.7409825325012207, 'predictions_28': 0.1818174123764038, 'predictions_29': 0.07541733235120773, 'predictions_30': 0.5059375166893005}
{'predictions_0': -0.01911582238972187, 'predictions_1': -0.41529545187950134, 'predictions_2': -0.0034923271741718054, 'predictions_3': -0.06306137144565582, 'predictions_4': -0.3192429542541504, 'predictions_5': 0.0166567862033844, 'predictions_6': -0.08258295059204102, 'predictions_7': -0.8088644742965698, 'predictions_8': 0.2545676529407501, 'predictions_9': 0.015658268705010414, 'predictions_10': 0.0021318818908184767, 'predictions_11': -0.003732672892510891, 'predictions_12': -0.054124802350997925, 'predictions_13': -1.0786054134368896, 'predictions_14': -0.03029802441596985, 'predictions_15': 0.46246394515037537, 'predictions_16': -0.007885736413300037, 'predictions_17': -0.168026864528656, 'predictions_18': -0.3190936744213104, 'predictions_19': 0.0024438181426376104, 'predictions_20': -0.6661943197250366, 'predictions_21': -0.2179996222257614, 'predictions_22': -2.689664125442505, 'predictions_23': -0.569831371307373, 'predictions_24': -0.6587631106376648, 'predictions_25': -0.031086977571249008, 'predictions_26': -0.554345428943634, 'predictions_27': -0.9785504937171936, 'predictions_28': -0.14026358723640442, 'predictions_29': 0.08221252262592316, 'predictions_30': 0.5059375166893005}
{'predictions_0': 0.009930070489645004, 'predictions_1': 0.46925264596939087, 'predictions_2': -0.007271876558661461, 'predictions_3': 0.04113232344388962, 'predictions_4': 0.49770334362983704, 'predictions_5': 0.004407345782965422, 'predictions_6': 0.04250117391347885, 'predictions_7': 0.7922263741493225, 'predictions_8': 0.30412089824676514, 'predictions_9': -0.020953189581632614, 'predictions_10': 0.009325551800429821, 'predictions_11': 0.04129280149936676, 'predictions_12': 0.10843253135681152, 'predictions_13': 0.21460701525211334, 'predictions_14': -0.01614229381084442, 'predictions_15': 0.6405980587005615, 'predictions_16': -0.0014769809786230326, 'predictions_17': 0.16581103205680847, 'predictions_18': 0.4200459420681, 'predictions_19': -0.004930071532726288, 'predictions_20': 0.31877532601356506, 'predictions_21': -0.01986948773264885, 'predictions_22': 1.556177020072937, 'predictions_23': 0.445791631937027, 'predictions_24': 0.5183241367340088, 'predictions_25': 0.013311417773365974, 'predictions_26': 1.139808177947998, 'predictions_27': 0.5462087988853455, 'predictions_28': -0.12236276268959045, 'predictions_29': 0.1068669781088829, 'predictions_30': 0.5059375166893005}
{'predictions_0': 0.009930070489645004, 'predictions_1': 0.4225478768348694, 'predictions_2': -0.007271876558661461, 'predictions_3': 0.022934239357709885, 'predictions_4': 0.38781455159187317, 'predictions_5': 0.004407345782965422, 'predictions_6': 0.013881205581128597, 'predictions_7': 0.6254116892814636, 'predictions_8': -0.28174325823783875, 'predictions_9': 0.008572578430175781, 'predictions_10': 0.009325551800429821, 'predictions_11': 0.022453438490629196, 'predictions_12': 0.09116627275943756, 'predictions_13': 1.01081383228302, 'predictions_14': 0.051560889929533005, 'predictions_15': -0.7526140809059143, 'predictions_16': -0.0010774275287985802, 'predictions_17': 0.13414263725280762, 'predictions_18': -0.36162295937538147, 'predictions_19': -0.004930071532726288, 'predictions_20': 0.21481025218963623, 'predictions_21': -1.7817566394805908, 'predictions_22': 1.5544458627700806, 'predictions_23': 0.37933894991874695, 'predictions_24': -0.5005015134811401, 'predictions_25': 0.013311417773365974, 'predictions_26': 1.2116931676864624, 'predictions_27': 0.44261839985847473, 'predictions_28': -0.1333807408809662, 'predictions_29': 0.12356309592723846, 'predictions_30': 0.5059375166893005}
{'predictions_0': 0.009930070489645004, 'predictions_1': 0.383541464805603, 'predictions_2': -0.007271876558661461, 'predictions_3': 0.022934239357709885, 'predictions_4': -0.2877747118473053, 'predictions_5': 0.004407345782965422, 'predictions_6': 0.04250117391347885, 'predictions_7': 0.7181150317192078, 'predictions_8': -0.13476626574993134, 'predictions_9': 0.008572578430175781, 'predictions_10': 0.009325551800429821, 'predictions_11': 0.04129280149936676, 'predictions_12': 0.11661991477012634, 'predictions_13': 1.1911046504974365, 'predictions_14': 0.05529181659221649, 'predictions_15': -0.9263618588447571, 'predictions_16': -0.0010774275287985802, 'predictions_17': 0.16138489544391632, 'predictions_18': -0.3877210319042206, 'predictions_19': -0.0022315792739391327, 'predictions_20': 0.33373206853866577, 'predictions_21': -0.10588698834180832, 'predictions_22': 1.7216718196868896, 'predictions_23': 0.33134832978248596, 'predictions_24': 0.07947567105293274, 'predictions_25': 0.013311417773365974, 'predictions_26': 1.1455278396606445, 'predictions_27': 0.6355651617050171, 'predictions_28': -0.12222569435834885, 'predictions_29': 0.08268986642360687, 'predictions_30': 0.5059375166893005}
{'predictions_0': 0.0006543132476508617, 'predictions_1': -0.9144008159637451, 'predictions_2': -0.002613282995298505, 'predictions_3': 0.04113232344388962, 'predictions_4': 0.3871055543422699, 'predictions_5': 0.004407345782965422, 'predictions_6': 0.013881205581128597, 'predictions_7': 0.6403470635414124, 'predictions_8': -0.27460771799087524, 'predictions_9': 0.05645249783992767, 'predictions_10': 0.009325551800429821, 'predictions_11': 0.02168312668800354, 'predictions_12': 0.11661991477012634, 'predictions_13': 1.00325608253479, 'predictions_14': 0.0834161564707756, 'predictions_15': 0.6231592893600464, 'predictions_16': -0.0010774275287985802, 'predictions_17': 0.16138489544391632, 'predictions_18': 0.37865450978279114, 'predictions_19': 0.014222224242985249, 'predictions_20': 0.38761574029922485, 'predictions_21': -0.02124016545712948, 'predictions_22': 1.7526684999465942, 'predictions_23': 0.5112042427062988, 'predictions_24': 0.5034342408180237, 'predictions_25': 0.005095706321299076, 'predictions_26': 0.9831169843673706, 'predictions_27': 0.6794330477714539, 'predictions_28': 0.1802661120891571, 'predictions_29': 0.1068669781088829, 'predictions_30': 0.5059375166893005}
{'predictions_0': -0.0415695421397686, 'predictions_1': -1.239405870437622, 'predictions_2': 0.017459018155932426, 'predictions_3': 0.04113232344388962, 'predictions_4': 0.4762759208679199, 'predictions_5': 0.004407345782965422, 'predictions_6': 0.04250117391347885, 'predictions_7': 0.9339500665664673, 'predictions_8': 0.2958080470561981, 'predictions_9': 0.07594440132379532, 'predictions_10': 0.003290211781859398, 'predictions_11': 0.01872287504374981, 'predictions_12': -0.047330088913440704, 'predictions_13': 0.056555308401584625, 'predictions_14': 0.05581967905163765, 'predictions_15': 0.6933314800262451, 'predictions_16': -0.0023022573441267014, 'predictions_17': 0.13856875896453857, 'predictions_18': 0.3069765865802765, 'predictions_19': 0.06924822926521301, 'predictions_20': -0.34636712074279785, 'predictions_21': -0.8682994246482849, 'predictions_22': -0.9693347215652466, 'predictions_23': -0.4783455729484558, 'predictions_24': 0.8846019506454468, 'predictions_25': -0.053959351032972336, 'predictions_26': -0.5963033437728882, 'predictions_27': -0.5447815656661987, 'predictions_28': -0.14065201580524445, 'predictions_29': 0.1068669781088829, 'predictions_30': 0.5059375166893005}
{'predictions_0': -0.024941062554717064, 'predictions_1': -1.5298322439193726, 'predictions_2': 0.01525309681892395, 'predictions_3': -0.11309991031885147, 'predictions_4': 0.5172238945960999, 'predictions_5': 0.0024520084261894226, 'predictions_6': 0.04250117391347885, 'predictions_7': 0.5354383587837219, 'predictions_8': 0.2944106459617615, 'predictions_9': -0.0466727539896965, 'predictions_10': 0.003290211781859398, 'predictions_11': 0.0188205074518919, 'predictions_12': -0.051007892936468124, 'predictions_13': -1.3502459526062012, 'predictions_14': -0.033535201102495193, 'predictions_15': 0.10678261518478394, 'predictions_16': -0.0014769809786230326, 'predictions_17': 0.13856875896453857, 'predictions_18': 0.3855004608631134, 'predictions_19': -0.008676297031342983, 'predictions_20': -0.3423697352409363, 'predictions_21': -0.8320394158363342, 'predictions_22': -0.677459716796875, 'predictions_23': -0.5385064482688904, 'predictions_24': 0.49956056475639343, 'predictions_25': 0.005095706321299076, 'predictions_26': 1.6938838958740234, 'predictions_27': 0.842454195022583, 'predictions_28': -0.14346282184123993, 'predictions_29': 0.07541733235120773, 'predictions_30': 0.5059375166893005}
{'predictions_0': 0.009930070489645004, 'predictions_1': -0.27544116973876953, 'predictions_2': -0.002613282995298505, 'predictions_3': 0.04113232344388962, 'predictions_4': -0.055784229189157486, 'predictions_5': 0.004407345782965422, 'predictions_6': 0.04250117391347885, 'predictions_7': 0.6722134947776794, 'predictions_8': 0.30412089824676514, 'predictions_9': 0.007771771401166916, 'predictions_10': 0.009325551800429821, 'predictions_11': 0.022453438490629196, 'predictions_12': 0.10843253135681152, 'predictions_13': 0.8989681005477905, 'predictions_14': -0.02233714796602726, 'predictions_15': 0.5417506694793701, 'predictions_16': -0.0010774275287985802, 'predictions_17': 0.16581103205680847, 'predictions_18': 0.406240850687027, 'predictions_19': -0.004930071532726288, 'predictions_20': 0.31123194098472595, 'predictions_21': -0.0816090777516365, 'predictions_22': 2.1133084297180176, 'predictions_23': 0.4506610333919525, 'predictions_24': -0.7193129658699036, 'predictions_25': 0.013311417773365974, 'predictions_26': 1.1290695667266846, 'predictions_27': 0.6277202367782593, 'predictions_28': -0.12229753285646439, 'predictions_29': 0.1477402150630951, 'predictions_30': 0.5059375166893005}
{'predictions_0': 0.0006543132476508617, 'predictions_1': -0.8427131175994873, 'predictions_2': -0.002613282995298505, 'predictions_3': 0.04113232344388962, 'predictions_4': -0.2789610028266907, 'predictions_5': 0.004407345782965422, 'predictions_6': 0.04250117391347885, 'predictions_7': 0.7143340706825256, 'predictions_8': 0.30412089824676514, 'predictions_9': 0.0335814394056797, 'predictions_10': 0.009325551800429821, 'predictions_11': -0.11305594444274902, 'predictions_12': -0.018516669049859047, 'predictions_13': 0.10892590880393982, 'predictions_14': -0.08540848642587662, 'predictions_15': -0.09607099741697311, 'predictions_16': -0.0014769809786230326, 'predictions_17': 0.13856875896453857, 'predictions_18': 0.44763222336769104, 'predictions_19': 0.014222224242985249, 'predictions_20': 0.33993175625801086, 'predictions_21': -0.6289445757865906, 'predictions_22': 1.987050175666809, 'predictions_23': 0.5226719379425049, 'predictions_24': -0.6285008788108826, 'predictions_25': 0.003129873890429735, 'predictions_26': 1.1616733074188232, 'predictions_27': 0.8699957132339478, 'predictions_28': -0.12550924718379974, 'predictions_29': 0.1477402150630951, 'predictions_30': 0.5059375166893005}
{'predictions_0': -0.01911582238972187, 'predictions_1': -0.41529545187950134, 'predictions_2': -0.0034923271741718054, 'predictions_3': -0.11309991031885147, 'predictions_4': -0.30164235830307007, 'predictions_5': -0.009094981476664543, 'predictions_6': -0.08258295059204102, 'predictions_7': -0.8088644742965698, 'predictions_8': 0.2545676529407501, 'predictions_9': 0.015658268705010414, 'predictions_10': -0.08612797409296036, 'predictions_11': -0.003732672892510891, 'predictions_12': -0.24946996569633484, 'predictions_13': -0.9515364766120911, 'predictions_14': -0.030668942257761955, 'predictions_15': 0.4303477704524994, 'predictions_16': -0.007885736413300037, 'predictions_17': -0.168026864528656, 'predictions_18': 0.25735917687416077, 'predictions_19': 0.0024438181426376104, 'predictions_20': -0.5780957937240601, 'predictions_21': -0.3737794756889343, 'predictions_22': -2.412389039993286, 'predictions_23': -0.6488395929336548, 'predictions_24': -0.6861838698387146, 'predictions_25': -0.031086977571249008, 'predictions_26': -0.5648106932640076, 'predictions_27': -0.9229058623313904, 'predictions_28': -0.14026358723640442, 'predictions_29': -0.1724514663219452, 'predictions_30': 0.5059375166893005}
(BlockWorker pid=1494413) FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494469) /home/ubuntu/ray/venv/lib/python3.8/site-packages/xgboost/compat.py:31: FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494469) from pandas import MultiIndex, Int64Index
(pid=1494469) FutureWarning: pandas.Int64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494469) FutureWarning: pandas.Float64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.
(pid=1494469) FutureWarning: pandas.UInt64Index is deprecated and will be removed from pandas in a future version. Use pandas.Index with the appropriate dtype instead.