Using Keras & TensorFlow with Tune
Contents
Using Keras & TensorFlow with Tune#
Example#
import argparse
import os
from filelock import FileLock
from tensorflow.keras.datasets import mnist
import ray
from ray import air, tune
from ray.tune.schedulers import AsyncHyperBandScheduler
from ray.tune.integration.keras import TuneReportCallback
def train_mnist(config):
# https://github.com/tensorflow/tensorflow/issues/32159
import tensorflow as tf
batch_size = 128
num_classes = 10
epochs = 12
with FileLock(os.path.expanduser("~/.data.lock")):
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
model = tf.keras.models.Sequential(
[
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(config["hidden"], activation="relu"),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(num_classes, activation="softmax"),
]
)
model.compile(
loss="sparse_categorical_crossentropy",
optimizer=tf.keras.optimizers.SGD(lr=config["lr"], momentum=config["momentum"]),
metrics=["accuracy"],
)
model.fit(
x_train,
y_train,
batch_size=batch_size,
epochs=epochs,
verbose=0,
validation_data=(x_test, y_test),
callbacks=[TuneReportCallback({"mean_accuracy": "accuracy"})],
)
def tune_mnist(num_training_iterations):
sched = AsyncHyperBandScheduler(
time_attr="training_iteration", max_t=400, grace_period=20
)
tuner = tune.Tuner(
tune.with_resources(train_mnist, resources={"cpu": 2, "gpu": 0}),
tune_config=tune.TuneConfig(
metric="mean_accuracy",
mode="max",
scheduler=sched,
num_samples=10,
),
run_config=air.RunConfig(
name="exp",
stop={"mean_accuracy": 0.99, "training_iteration": num_training_iterations},
),
param_space={
"threads": 2,
"lr": tune.uniform(0.001, 0.1),
"momentum": tune.uniform(0.1, 0.9),
"hidden": tune.randint(32, 512),
},
)
results = tuner.fit()
print("Best hyperparameters found were: ", results.get_best_result().config)
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument(
"--smoke-test", action="store_true", help="Finish quickly for testing"
)
args, _ = parser.parse_known_args()
if args.smoke_test:
ray.init(num_cpus=4)
tune_mnist(num_training_iterations=5 if args.smoke_test else 300)
2022-07-22 16:16:58,114 INFO services.py:1483 -- View the Ray dashboard at http://127.0.0.1:8269
2022-07-22 16:17:00,822 WARNING function_trainable.py:619 -- Function checkpointing is disabled. This may result in unexpected behavior when using checkpointing features or certain schedulers. To enable, set the train function arguments to be `func(config, checkpoint_dir=None)`.
== Status ==
Current time: 2022-07-22 16:18:36 (running for 00:01:35.04)
Memory usage on this node: 9.0/16.0 GiB
Using AsyncHyperBand: num_stopped=0 Bracket: Iter 320.000: None | Iter 80.000: None | Iter 20.000: None
Resources requested: 0/16 CPUs, 0/0 GPUs, 0.0/5.47 GiB heap, 0.0/2.0 GiB objects
Current best trial: 55a9b_00002 with mean_accuracy=0.9904166460037231 and parameters={'threads': 2, 'lr': 0.09518133271957563, 'momentum': 0.8254987643140009, 'hidden': 258}
Result logdir: /Users/kai/ray_results/exp
Number of trials: 10/10 (10 TERMINATED)
Current time: 2022-07-22 16:18:36 (running for 00:01:35.04)
Memory usage on this node: 9.0/16.0 GiB
Using AsyncHyperBand: num_stopped=0 Bracket: Iter 320.000: None | Iter 80.000: None | Iter 20.000: None
Resources requested: 0/16 CPUs, 0/0 GPUs, 0.0/5.47 GiB heap, 0.0/2.0 GiB objects
Current best trial: 55a9b_00002 with mean_accuracy=0.9904166460037231 and parameters={'threads': 2, 'lr': 0.09518133271957563, 'momentum': 0.8254987643140009, 'hidden': 258}
Result logdir: /Users/kai/ray_results/exp
Number of trials: 10/10 (10 TERMINATED)
| Trial name | status | loc | hidden | lr | momentum | acc | iter | total time (s) |
|---|---|---|---|---|---|---|---|---|
| train_mnist_55a9b_00000 | TERMINATED | 127.0.0.1:51968 | 276 | 0.0406397 | 0.817788 | 0.98455 | 12 | 78.3252 |
| train_mnist_55a9b_00001 | TERMINATED | 127.0.0.1:51977 | 380 | 0.0873557 | 0.524634 | 0.983717 | 12 | 74.9888 |
| train_mnist_55a9b_00002 | TERMINATED | 127.0.0.1:51984 | 258 | 0.0951813 | 0.825499 | 0.990417 | 11 | 64.1272 |
| train_mnist_55a9b_00003 | TERMINATED | 127.0.0.1:51991 | 255 | 0.0971683 | 0.23161 | 0.977633 | 12 | 60.8475 |
| train_mnist_55a9b_00004 | TERMINATED | 127.0.0.1:52000 | 303 | 0.00440117 | 0.325439 | 0.90775 | 12 | 55.5722 |
| train_mnist_55a9b_00005 | TERMINATED | 127.0.0.1:52007 | 92 | 0.0651919 | 0.710183 | 0.974867 | 12 | 44.8092 |
| train_mnist_55a9b_00006 | TERMINATED | 127.0.0.1:52016 | 211 | 0.0731116 | 0.127751 | 0.97025 | 12 | 42.1217 |
| train_mnist_55a9b_00007 | TERMINATED | 127.0.0.1:52021 | 181 | 0.0362389 | 0.790345 | 0.979967 | 12 | 41.7632 |
| train_mnist_55a9b_00008 | TERMINATED | 127.0.0.1:52007 | 142 | 0.0323741 | 0.660418 | 0.969367 | 12 | 14.1527 |
| train_mnist_55a9b_00009 | TERMINATED | 127.0.0.1:51984 | 97 | 0.0244971 | 0.175045 | 0.9407 | 12 | 12.6405 |
2022-07-22 16:17:01,834 INFO plugin_schema_manager.py:52 -- Loading the default runtime env schemas: ['/Users/kai/coding/ray/python/ray/_private/runtime_env/../../runtime_env/schemas/working_dir_schema.json', '/Users/kai/coding/ray/python/ray/_private/runtime_env/../../runtime_env/schemas/pip_schema.json'].
(train_mnist pid=51968) 2022-07-22 16:17:08.627419: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
(train_mnist pid=51968) To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
(train_mnist pid=51968) /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:356: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
(train_mnist pid=51968) "The `lr` argument is deprecated, use `learning_rate` instead.")
(train_mnist pid=51968) 2022-07-22 16:17:08.947939: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
(train_mnist pid=51977) 2022-07-22 16:17:14.473677: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
(train_mnist pid=51977) To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
(train_mnist pid=51977) /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:356: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
(train_mnist pid=51977) "The `lr` argument is deprecated, use `learning_rate` instead.")
(train_mnist pid=51977) 2022-07-22 16:17:14.635104: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
(train_mnist pid=51984) 2022-07-22 16:17:20.406624: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
(train_mnist pid=51984) To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
(train_mnist pid=51984) /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:356: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
(train_mnist pid=51984) "The `lr` argument is deprecated, use `learning_rate` instead.")
(train_mnist pid=51984) 2022-07-22 16:17:20.681960: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
(train_mnist pid=51991) 2022-07-22 16:17:26.109460: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
(train_mnist pid=51991) To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
(train_mnist pid=51991) /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:356: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
(train_mnist pid=51991) "The `lr` argument is deprecated, use `learning_rate` instead.")
(train_mnist pid=51991) 2022-07-22 16:17:26.303375: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
(train_mnist pid=52000) 2022-07-22 16:17:31.899252: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
(train_mnist pid=52000) To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
(train_mnist pid=52000) /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:356: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
(train_mnist pid=52000) "The `lr` argument is deprecated, use `learning_rate` instead.")
(train_mnist pid=52000) 2022-07-22 16:17:32.300424: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
(train_mnist pid=52007) 2022-07-22 16:17:37.937471: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
(train_mnist pid=52007) To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
(train_mnist pid=52007) /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:356: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
(train_mnist pid=52007) "The `lr` argument is deprecated, use `learning_rate` instead.")
(train_mnist pid=52007) 2022-07-22 16:17:38.263888: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
(train_mnist pid=52016) 2022-07-22 16:17:43.657379: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
(train_mnist pid=52016) To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
(train_mnist pid=52016) /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:356: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
(train_mnist pid=52016) "The `lr` argument is deprecated, use `learning_rate` instead.")
(train_mnist pid=52016) 2022-07-22 16:17:43.828809: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
Result for train_mnist_55a9b_00000:
date: 2022-07-22_16-17-10
done: false
experiment_id: 3659349c38c746cfb71b4db5eb9302a0
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 1
mean_accuracy: 0.8903833627700806
node_ip: 127.0.0.1
pid: 51968
time_since_restore: 2.439258098602295
time_this_iter_s: 2.439258098602295
time_total_s: 2.439258098602295
timestamp: 1658503030
timesteps_since_restore: 0
training_iteration: 1
trial_id: 55a9b_00000
warmup_time: 0.003445863723754883
Result for train_mnist_55a9b_00004:
date: 2022-07-22_16-17-33
done: false
experiment_id: 6eb62b7cb38f442a867a9094f0664701
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 1
mean_accuracy: 0.6376166939735413
node_ip: 127.0.0.1
pid: 52000
time_since_restore: 2.4364511966705322
time_this_iter_s: 2.4364511966705322
time_total_s: 2.4364511966705322
timestamp: 1658503053
timesteps_since_restore: 0
training_iteration: 1
trial_id: 55a9b_00004
warmup_time: 0.0030939579010009766
Result for train_mnist_55a9b_00006:
date: 2022-07-22_16-17-45
done: false
experiment_id: 9594405e38084311a891b48addd13f75
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 1
mean_accuracy: 0.8557000160217285
node_ip: 127.0.0.1
pid: 52016
time_since_restore: 1.8570480346679688
time_this_iter_s: 1.8570480346679688
time_total_s: 1.8570480346679688
timestamp: 1658503065
timesteps_since_restore: 0
training_iteration: 1
trial_id: 55a9b_00006
warmup_time: 0.003566741943359375
Result for train_mnist_55a9b_00001:
date: 2022-07-22_16-17-15
done: false
experiment_id: fcdeb049f9614755a9b7c9420ca2ae5e
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 1
mean_accuracy: 0.8887666463851929
node_ip: 127.0.0.1
pid: 51977
time_since_restore: 1.9353628158569336
time_this_iter_s: 1.9353628158569336
time_total_s: 1.9353628158569336
timestamp: 1658503035
timesteps_since_restore: 0
training_iteration: 1
trial_id: 55a9b_00001
warmup_time: 0.0029449462890625
Result for train_mnist_55a9b_00005:
date: 2022-07-22_16-17-39
done: false
experiment_id: 8dbd22e6caed4fe39351dffa3ef14eac
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 1
mean_accuracy: 0.8789666891098022
node_ip: 127.0.0.1
pid: 52007
time_since_restore: 2.3337321281433105
time_this_iter_s: 2.3337321281433105
time_total_s: 2.3337321281433105
timestamp: 1658503059
timesteps_since_restore: 0
training_iteration: 1
trial_id: 55a9b_00005
warmup_time: 0.005449056625366211
Result for train_mnist_55a9b_00002:
date: 2022-07-22_16-17-21
done: false
experiment_id: c4f803baf65f4d4e9fd6abc85b2fd00c
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 1
mean_accuracy: 0.9112833142280579
node_ip: 127.0.0.1
pid: 51984
time_since_restore: 2.3220012187957764
time_this_iter_s: 2.3220012187957764
time_total_s: 2.3220012187957764
timestamp: 1658503041
timesteps_since_restore: 0
training_iteration: 1
trial_id: 55a9b_00002
warmup_time: 0.0028328895568847656
Result for train_mnist_55a9b_00003:
date: 2022-07-22_16-17-27
done: false
experiment_id: 469478f02b4a43f5b44c40e59989ad39
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 1
mean_accuracy: 0.8743166923522949
node_ip: 127.0.0.1
pid: 51991
time_since_restore: 2.0278611183166504
time_this_iter_s: 2.0278611183166504
time_total_s: 2.0278611183166504
timestamp: 1658503047
timesteps_since_restore: 0
training_iteration: 1
trial_id: 55a9b_00003
warmup_time: 0.0033779144287109375
(train_mnist pid=52021) 2022-07-22 16:17:51.567914: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
(train_mnist pid=52021) To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
(train_mnist pid=52021) /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:356: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
(train_mnist pid=52021) "The `lr` argument is deprecated, use `learning_rate` instead.")
(train_mnist pid=52021) 2022-07-22 16:17:52.977183: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
Result for train_mnist_55a9b_00005:
date: 2022-07-22_16-17-54
done: false
experiment_id: 8dbd22e6caed4fe39351dffa3ef14eac
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 3
mean_accuracy: 0.9490833282470703
node_ip: 127.0.0.1
pid: 52007
time_since_restore: 17.22033405303955
time_this_iter_s: 2.672102928161621
time_total_s: 17.22033405303955
timestamp: 1658503074
timesteps_since_restore: 0
training_iteration: 3
trial_id: 55a9b_00005
warmup_time: 0.005449056625366211
Result for train_mnist_55a9b_00006:
date: 2022-07-22_16-17-54
done: false
experiment_id: 9594405e38084311a891b48addd13f75
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 3
mean_accuracy: 0.9327999949455261
node_ip: 127.0.0.1
pid: 52016
time_since_restore: 11.758372068405151
time_this_iter_s: 3.0426323413848877
time_total_s: 11.758372068405151
timestamp: 1658503074
timesteps_since_restore: 0
training_iteration: 3
trial_id: 55a9b_00006
warmup_time: 0.003566741943359375
Result for train_mnist_55a9b_00003:
date: 2022-07-22_16-17-55
done: false
experiment_id: 469478f02b4a43f5b44c40e59989ad39
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 3
mean_accuracy: 0.9454166889190674
node_ip: 127.0.0.1
pid: 51991
time_since_restore: 29.733185052871704
time_this_iter_s: 3.0363340377807617
time_total_s: 29.733185052871704
timestamp: 1658503075
timesteps_since_restore: 0
training_iteration: 3
trial_id: 55a9b_00003
warmup_time: 0.0033779144287109375
Result for train_mnist_55a9b_00000:
date: 2022-07-22_16-17-55
done: false
experiment_id: 3659349c38c746cfb71b4db5eb9302a0
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 3
mean_accuracy: 0.958216667175293
node_ip: 127.0.0.1
pid: 51968
time_since_restore: 47.272178173065186
time_this_iter_s: 3.2986061573028564
time_total_s: 47.272178173065186
timestamp: 1658503075
timesteps_since_restore: 0
training_iteration: 3
trial_id: 55a9b_00000
warmup_time: 0.003445863723754883
Result for train_mnist_55a9b_00004:
date: 2022-07-22_16-17-55
done: false
experiment_id: 6eb62b7cb38f442a867a9094f0664701
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 3
mean_accuracy: 0.8524500131607056
node_ip: 127.0.0.1
pid: 52000
time_since_restore: 24.11396098136902
time_this_iter_s: 3.2331089973449707
time_total_s: 24.11396098136902
timestamp: 1658503075
timesteps_since_restore: 0
training_iteration: 3
trial_id: 55a9b_00004
warmup_time: 0.0030939579010009766
Result for train_mnist_55a9b_00002:
date: 2022-07-22_16-17-55
done: false
experiment_id: c4f803baf65f4d4e9fd6abc85b2fd00c
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 3
mean_accuracy: 0.9695500135421753
node_ip: 127.0.0.1
pid: 51984
time_since_restore: 35.78592824935913
time_this_iter_s: 3.021165132522583
time_total_s: 35.78592824935913
timestamp: 1658503075
timesteps_since_restore: 0
training_iteration: 3
trial_id: 55a9b_00002
warmup_time: 0.0028328895568847656
Result for train_mnist_55a9b_00001:
date: 2022-07-22_16-17-56
done: false
experiment_id: fcdeb049f9614755a9b7c9420ca2ae5e
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 3
mean_accuracy: 0.9560333490371704
node_ip: 127.0.0.1
pid: 51977
time_since_restore: 42.38909387588501
time_this_iter_s: 3.753290891647339
time_total_s: 42.38909387588501
timestamp: 1658503076
timesteps_since_restore: 0
training_iteration: 3
trial_id: 55a9b_00001
warmup_time: 0.0029449462890625
Result for train_mnist_55a9b_00005:
date: 2022-07-22_16-18-00
done: false
experiment_id: 8dbd22e6caed4fe39351dffa3ef14eac
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 5
mean_accuracy: 0.9611166715621948
node_ip: 127.0.0.1
pid: 52007
time_since_restore: 23.303561210632324
time_this_iter_s: 2.933852195739746
time_total_s: 23.303561210632324
timestamp: 1658503080
timesteps_since_restore: 0
training_iteration: 5
trial_id: 55a9b_00005
warmup_time: 0.005449056625366211
Result for train_mnist_55a9b_00007:
date: 2022-07-22_16-18-01
done: false
experiment_id: d9469b1fc58b41db88da5446dc2a3b23
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 1
mean_accuracy: 0.8797500133514404
node_ip: 127.0.0.1
pid: 52021
time_since_restore: 12.469872951507568
time_this_iter_s: 12.469872951507568
time_total_s: 12.469872951507568
timestamp: 1658503081
timesteps_since_restore: 0
training_iteration: 1
trial_id: 55a9b_00007
warmup_time: 0.0028028488159179688
Result for train_mnist_55a9b_00006:
date: 2022-07-22_16-18-01
done: false
experiment_id: 9594405e38084311a891b48addd13f75
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 5
mean_accuracy: 0.9499499797821045
node_ip: 127.0.0.1
pid: 52016
time_since_restore: 18.780059814453125
time_this_iter_s: 3.3080599308013916
time_total_s: 18.780059814453125
timestamp: 1658503081
timesteps_since_restore: 0
training_iteration: 5
trial_id: 55a9b_00006
warmup_time: 0.003566741943359375
Result for train_mnist_55a9b_00003:
date: 2022-07-22_16-18-02
done: false
experiment_id: 469478f02b4a43f5b44c40e59989ad39
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 5
mean_accuracy: 0.9601166844367981
node_ip: 127.0.0.1
pid: 51991
time_since_restore: 36.93912100791931
time_this_iter_s: 3.4057939052581787
time_total_s: 36.93912100791931
timestamp: 1658503082
timesteps_since_restore: 0
training_iteration: 5
trial_id: 55a9b_00003
warmup_time: 0.0033779144287109375
Result for train_mnist_55a9b_00000:
date: 2022-07-22_16-18-02
done: false
experiment_id: 3659349c38c746cfb71b4db5eb9302a0
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 5
mean_accuracy: 0.970466673374176
node_ip: 127.0.0.1
pid: 51968
time_since_restore: 54.49850010871887
time_this_iter_s: 3.4417831897735596
time_total_s: 54.49850010871887
timestamp: 1658503082
timesteps_since_restore: 0
training_iteration: 5
trial_id: 55a9b_00000
warmup_time: 0.003445863723754883
Result for train_mnist_55a9b_00004:
date: 2022-07-22_16-18-02
done: false
experiment_id: 6eb62b7cb38f442a867a9094f0664701
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 5
mean_accuracy: 0.8777499794960022
node_ip: 127.0.0.1
pid: 52000
time_since_restore: 31.513713121414185
time_this_iter_s: 3.506195068359375
time_total_s: 31.513713121414185
timestamp: 1658503082
timesteps_since_restore: 0
training_iteration: 5
trial_id: 55a9b_00004
warmup_time: 0.0030939579010009766
Result for train_mnist_55a9b_00002:
date: 2022-07-22_16-18-02
done: false
experiment_id: c4f803baf65f4d4e9fd6abc85b2fd00c
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 5
mean_accuracy: 0.979283332824707
node_ip: 127.0.0.1
pid: 51984
time_since_restore: 43.266417026519775
time_this_iter_s: 3.3383469581604004
time_total_s: 43.266417026519775
timestamp: 1658503082
timesteps_since_restore: 0
training_iteration: 5
trial_id: 55a9b_00002
warmup_time: 0.0028328895568847656
Result for train_mnist_55a9b_00001:
date: 2022-07-22_16-18-04
done: false
experiment_id: fcdeb049f9614755a9b7c9420ca2ae5e
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 5
mean_accuracy: 0.9692999720573425
node_ip: 127.0.0.1
pid: 51977
time_since_restore: 50.620792865753174
time_this_iter_s: 4.001068115234375
time_total_s: 50.620792865753174
timestamp: 1658503084
timesteps_since_restore: 0
training_iteration: 5
trial_id: 55a9b_00001
warmup_time: 0.0029449462890625
Result for train_mnist_55a9b_00005:
date: 2022-07-22_16-18-06
done: false
experiment_id: 8dbd22e6caed4fe39351dffa3ef14eac
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 7
mean_accuracy: 0.96711665391922
node_ip: 127.0.0.1
pid: 52007
time_since_restore: 29.40476107597351
time_this_iter_s: 2.976076126098633
time_total_s: 29.40476107597351
timestamp: 1658503086
timesteps_since_restore: 0
training_iteration: 7
trial_id: 55a9b_00005
warmup_time: 0.005449056625366211
Result for train_mnist_55a9b_00007:
date: 2022-07-22_16-18-07
done: false
experiment_id: d9469b1fc58b41db88da5446dc2a3b23
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 3
mean_accuracy: 0.951033353805542
node_ip: 127.0.0.1
pid: 52021
time_since_restore: 18.96213722229004
time_this_iter_s: 3.252371311187744
time_total_s: 18.96213722229004
timestamp: 1658503087
timesteps_since_restore: 0
training_iteration: 3
trial_id: 55a9b_00007
warmup_time: 0.0028028488159179688
Result for train_mnist_55a9b_00006:
date: 2022-07-22_16-18-08
done: false
experiment_id: 9594405e38084311a891b48addd13f75
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 7
mean_accuracy: 0.9584500193595886
node_ip: 127.0.0.1
pid: 52016
time_since_restore: 25.336583852767944
time_this_iter_s: 3.311979055404663
time_total_s: 25.336583852767944
timestamp: 1658503088
timesteps_since_restore: 0
training_iteration: 7
trial_id: 55a9b_00006
warmup_time: 0.003566741943359375
Result for train_mnist_55a9b_00003:
date: 2022-07-22_16-18-09
done: false
experiment_id: 469478f02b4a43f5b44c40e59989ad39
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 7
mean_accuracy: 0.9675499796867371
node_ip: 127.0.0.1
pid: 51991
time_since_restore: 43.7107310295105
time_this_iter_s: 3.3927559852600098
time_total_s: 43.7107310295105
timestamp: 1658503089
timesteps_since_restore: 0
training_iteration: 7
trial_id: 55a9b_00003
warmup_time: 0.0033779144287109375
Result for train_mnist_55a9b_00000:
date: 2022-07-22_16-18-09
done: false
experiment_id: 3659349c38c746cfb71b4db5eb9302a0
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 7
mean_accuracy: 0.9763000011444092
node_ip: 127.0.0.1
pid: 51968
time_since_restore: 61.30248522758484
time_this_iter_s: 3.4063682556152344
time_total_s: 61.30248522758484
timestamp: 1658503089
timesteps_since_restore: 0
training_iteration: 7
trial_id: 55a9b_00000
warmup_time: 0.003445863723754883
Result for train_mnist_55a9b_00002:
date: 2022-07-22_16-18-09
done: false
experiment_id: c4f803baf65f4d4e9fd6abc85b2fd00c
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 7
mean_accuracy: 0.9840666651725769
node_ip: 127.0.0.1
pid: 51984
time_since_restore: 50.212465047836304
time_this_iter_s: 3.43766188621521
time_total_s: 50.212465047836304
timestamp: 1658503089
timesteps_since_restore: 0
training_iteration: 7
trial_id: 55a9b_00002
warmup_time: 0.0028328895568847656
Result for train_mnist_55a9b_00004:
date: 2022-07-22_16-18-09
done: false
experiment_id: 6eb62b7cb38f442a867a9094f0664701
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 7
mean_accuracy: 0.8899999856948853
node_ip: 127.0.0.1
pid: 52000
time_since_restore: 38.63890194892883
time_this_iter_s: 3.5783908367156982
time_total_s: 38.63890194892883
timestamp: 1658503089
timesteps_since_restore: 0
training_iteration: 7
trial_id: 55a9b_00004
warmup_time: 0.0030939579010009766
Result for train_mnist_55a9b_00005:
date: 2022-07-22_16-18-12
done: false
experiment_id: 8dbd22e6caed4fe39351dffa3ef14eac
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 9
mean_accuracy: 0.9712333083152771
node_ip: 127.0.0.1
pid: 52007
time_since_restore: 35.3185760974884
time_this_iter_s: 3.0241990089416504
time_total_s: 35.3185760974884
timestamp: 1658503092
timesteps_since_restore: 0
training_iteration: 9
trial_id: 55a9b_00005
warmup_time: 0.005449056625366211
Result for train_mnist_55a9b_00001:
date: 2022-07-22_16-18-12
done: false
experiment_id: fcdeb049f9614755a9b7c9420ca2ae5e
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 7
mean_accuracy: 0.9755333065986633
node_ip: 127.0.0.1
pid: 51977
time_since_restore: 58.57745599746704
time_this_iter_s: 3.936232089996338
time_total_s: 58.57745599746704
timestamp: 1658503092
timesteps_since_restore: 0
training_iteration: 7
trial_id: 55a9b_00001
warmup_time: 0.0029449462890625
Result for train_mnist_55a9b_00007:
date: 2022-07-22_16-18-14
done: false
experiment_id: d9469b1fc58b41db88da5446dc2a3b23
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 5
mean_accuracy: 0.9648333191871643
node_ip: 127.0.0.1
pid: 52021
time_since_restore: 25.25843620300293
time_this_iter_s: 3.094501256942749
time_total_s: 25.25843620300293
timestamp: 1658503094
timesteps_since_restore: 0
training_iteration: 5
trial_id: 55a9b_00007
warmup_time: 0.0028028488159179688
Result for train_mnist_55a9b_00006:
date: 2022-07-22_16-18-15
done: false
experiment_id: 9594405e38084311a891b48addd13f75
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 9
mean_accuracy: 0.9646666646003723
node_ip: 127.0.0.1
pid: 52016
time_since_restore: 32.048911809921265
time_this_iter_s: 3.315690755844116
time_total_s: 32.048911809921265
timestamp: 1658503095
timesteps_since_restore: 0
training_iteration: 9
trial_id: 55a9b_00006
warmup_time: 0.003566741943359375
Result for train_mnist_55a9b_00003:
date: 2022-07-22_16-18-15
done: false
experiment_id: 469478f02b4a43f5b44c40e59989ad39
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 9
mean_accuracy: 0.9729499816894531
node_ip: 127.0.0.1
pid: 51991
time_since_restore: 50.50909209251404
time_this_iter_s: 3.4110782146453857
time_total_s: 50.50909209251404
timestamp: 1658503095
timesteps_since_restore: 0
training_iteration: 9
trial_id: 55a9b_00003
warmup_time: 0.0033779144287109375
Result for train_mnist_55a9b_00000:
date: 2022-07-22_16-18-16
done: false
experiment_id: 3659349c38c746cfb71b4db5eb9302a0
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 9
mean_accuracy: 0.9807666540145874
node_ip: 127.0.0.1
pid: 51968
time_since_restore: 68.26757216453552
time_this_iter_s: 3.4475879669189453
time_total_s: 68.26757216453552
timestamp: 1658503096
timesteps_since_restore: 0
training_iteration: 9
trial_id: 55a9b_00000
warmup_time: 0.003445863723754883
Result for train_mnist_55a9b_00002:
date: 2022-07-22_16-18-16
done: false
experiment_id: c4f803baf65f4d4e9fd6abc85b2fd00c
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 9
mean_accuracy: 0.9872999787330627
node_ip: 127.0.0.1
pid: 51984
time_since_restore: 57.01431703567505
time_this_iter_s: 3.3804008960723877
time_total_s: 57.01431703567505
timestamp: 1658503096
timesteps_since_restore: 0
training_iteration: 9
trial_id: 55a9b_00002
warmup_time: 0.0028328895568847656
Result for train_mnist_55a9b_00004:
date: 2022-07-22_16-18-16
done: false
experiment_id: 6eb62b7cb38f442a867a9094f0664701
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 9
mean_accuracy: 0.8989166617393494
node_ip: 127.0.0.1
pid: 52000
time_since_restore: 45.67929005622864
time_this_iter_s: 3.4561610221862793
time_total_s: 45.67929005622864
timestamp: 1658503096
timesteps_since_restore: 0
training_iteration: 9
trial_id: 55a9b_00004
warmup_time: 0.0030939579010009766
Result for train_mnist_55a9b_00005:
date: 2022-07-22_16-18-18
done: false
experiment_id: 8dbd22e6caed4fe39351dffa3ef14eac
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 11
mean_accuracy: 0.9744333624839783
node_ip: 127.0.0.1
pid: 52007
time_since_restore: 41.49077916145325
time_this_iter_s: 3.172250270843506
time_total_s: 41.49077916145325
timestamp: 1658503098
timesteps_since_restore: 0
training_iteration: 11
trial_id: 55a9b_00005
warmup_time: 0.005449056625366211
Result for train_mnist_55a9b_00001:
date: 2022-07-22_16-18-20
done: false
experiment_id: fcdeb049f9614755a9b7c9420ca2ae5e
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 9
mean_accuracy: 0.9806166887283325
node_ip: 127.0.0.1
pid: 51977
time_since_restore: 66.64132380485535
time_this_iter_s: 4.0674309730529785
time_total_s: 66.64132380485535
timestamp: 1658503100
timesteps_since_restore: 0
training_iteration: 9
trial_id: 55a9b_00001
warmup_time: 0.0029449462890625
Result for train_mnist_55a9b_00007:
date: 2022-07-22_16-18-20
done: false
experiment_id: d9469b1fc58b41db88da5446dc2a3b23
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 7
mean_accuracy: 0.970716655254364
node_ip: 127.0.0.1
pid: 52021
time_since_restore: 31.897236108779907
time_this_iter_s: 3.3691420555114746
time_total_s: 31.897236108779907
timestamp: 1658503100
timesteps_since_restore: 0
training_iteration: 7
trial_id: 55a9b_00007
warmup_time: 0.0028028488159179688
Result for train_mnist_55a9b_00005:
date: 2022-07-22_16-18-21
done: true
experiment_id: 8dbd22e6caed4fe39351dffa3ef14eac
experiment_tag: 5_hidden=92,lr=0.0652,momentum=0.7102
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 12
mean_accuracy: 0.9748666882514954
node_ip: 127.0.0.1
pid: 52007
time_since_restore: 44.80922222137451
time_this_iter_s: 3.3184430599212646
time_total_s: 44.80922222137451
timestamp: 1658503101
timesteps_since_restore: 0
training_iteration: 12
trial_id: 55a9b_00005
warmup_time: 0.005449056625366211
Result for train_mnist_55a9b_00006:
date: 2022-07-22_16-18-22
done: false
experiment_id: 9594405e38084311a891b48addd13f75
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 11
mean_accuracy: 0.9679166674613953
node_ip: 127.0.0.1
pid: 52016
time_since_restore: 39.08963179588318
time_this_iter_s: 3.4860758781433105
time_total_s: 39.08963179588318
timestamp: 1658503102
timesteps_since_restore: 0
training_iteration: 11
trial_id: 55a9b_00006
warmup_time: 0.003566741943359375
Result for train_mnist_55a9b_00003:
date: 2022-07-22_16-18-23
done: false
experiment_id: 469478f02b4a43f5b44c40e59989ad39
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 11
mean_accuracy: 0.9771833419799805
node_ip: 127.0.0.1
pid: 51991
time_since_restore: 57.6213219165802
time_this_iter_s: 3.4615819454193115
time_total_s: 57.6213219165802
timestamp: 1658503103
timesteps_since_restore: 0
training_iteration: 11
trial_id: 55a9b_00003
warmup_time: 0.0033779144287109375
Result for train_mnist_55a9b_00000:
date: 2022-07-22_16-18-23
done: false
experiment_id: 3659349c38c746cfb71b4db5eb9302a0
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 11
mean_accuracy: 0.98416668176651
node_ip: 127.0.0.1
pid: 51968
time_since_restore: 75.32713007926941
time_this_iter_s: 3.443808078765869
time_total_s: 75.32713007926941
timestamp: 1658503103
timesteps_since_restore: 0
training_iteration: 11
trial_id: 55a9b_00000
warmup_time: 0.003445863723754883
Result for train_mnist_55a9b_00002:
date: 2022-07-22_16-18-23
done: true
experiment_id: c4f803baf65f4d4e9fd6abc85b2fd00c
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 11
mean_accuracy: 0.9904166460037231
node_ip: 127.0.0.1
pid: 51984
time_since_restore: 64.12720203399658
time_this_iter_s: 3.508151054382324
time_total_s: 64.12720203399658
timestamp: 1658503103
timesteps_since_restore: 0
training_iteration: 11
trial_id: 55a9b_00002
warmup_time: 0.0028328895568847656
Result for train_mnist_55a9b_00004:
date: 2022-07-22_16-18-23
done: false
experiment_id: 6eb62b7cb38f442a867a9094f0664701
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 11
mean_accuracy: 0.9052166938781738
node_ip: 127.0.0.1
pid: 52000
time_since_restore: 52.687995195388794
time_this_iter_s: 3.420351982116699
time_total_s: 52.687995195388794
timestamp: 1658503103
timesteps_since_restore: 0
training_iteration: 11
trial_id: 55a9b_00004
warmup_time: 0.0030939579010009766
Result for train_mnist_55a9b_00006:
date: 2022-07-22_16-18-25
done: true
experiment_id: 9594405e38084311a891b48addd13f75
experiment_tag: 6_hidden=211,lr=0.0731,momentum=0.1278
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 12
mean_accuracy: 0.9702500104904175
node_ip: 127.0.0.1
pid: 52016
time_since_restore: 42.1216938495636
time_this_iter_s: 3.03206205368042
time_total_s: 42.1216938495636
timestamp: 1658503105
timesteps_since_restore: 0
training_iteration: 12
trial_id: 55a9b_00006
warmup_time: 0.003566741943359375
Result for train_mnist_55a9b_00003:
date: 2022-07-22_16-18-26
done: true
experiment_id: 469478f02b4a43f5b44c40e59989ad39
experiment_tag: 3_hidden=255,lr=0.0972,momentum=0.2316
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 12
mean_accuracy: 0.9776333570480347
node_ip: 127.0.0.1
pid: 51991
time_since_restore: 60.8474760055542
time_this_iter_s: 3.226154088973999
time_total_s: 60.8474760055542
timestamp: 1658503106
timesteps_since_restore: 0
training_iteration: 12
trial_id: 55a9b_00003
warmup_time: 0.0033779144287109375
Result for train_mnist_55a9b_00000:
date: 2022-07-22_16-18-26
done: true
experiment_id: 3659349c38c746cfb71b4db5eb9302a0
experiment_tag: 0_hidden=276,lr=0.0406,momentum=0.8178
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 12
mean_accuracy: 0.9845499992370605
node_ip: 127.0.0.1
pid: 51968
time_since_restore: 78.32520508766174
time_this_iter_s: 2.998075008392334
time_total_s: 78.32520508766174
timestamp: 1658503106
timesteps_since_restore: 0
training_iteration: 12
trial_id: 55a9b_00000
warmup_time: 0.003445863723754883
Result for train_mnist_55a9b_00007:
date: 2022-07-22_16-18-26
done: false
experiment_id: d9469b1fc58b41db88da5446dc2a3b23
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 9
mean_accuracy: 0.9751333594322205
node_ip: 127.0.0.1
pid: 52021
time_since_restore: 37.76195311546326
time_this_iter_s: 2.7159180641174316
time_total_s: 37.76195311546326
timestamp: 1658503106
timesteps_since_restore: 0
training_iteration: 9
trial_id: 55a9b_00007
warmup_time: 0.0028028488159179688
Result for train_mnist_55a9b_00004:
date: 2022-07-22_16-18-26
done: true
experiment_id: 6eb62b7cb38f442a867a9094f0664701
experiment_tag: 4_hidden=303,lr=0.0044,momentum=0.3254
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 12
mean_accuracy: 0.9077500104904175
node_ip: 127.0.0.1
pid: 52000
time_since_restore: 55.57219409942627
time_this_iter_s: 2.8841989040374756
time_total_s: 55.57219409942627
timestamp: 1658503106
timesteps_since_restore: 0
training_iteration: 12
trial_id: 55a9b_00004
warmup_time: 0.0030939579010009766
Result for train_mnist_55a9b_00001:
date: 2022-07-22_16-18-27
done: false
experiment_id: fcdeb049f9614755a9b7c9420ca2ae5e
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 11
mean_accuracy: 0.9830166697502136
node_ip: 127.0.0.1
pid: 51977
time_since_restore: 73.19760584831238
time_this_iter_s: 2.7281620502471924
time_total_s: 73.19760584831238
timestamp: 1658503107
timesteps_since_restore: 0
training_iteration: 11
trial_id: 55a9b_00001
warmup_time: 0.0029449462890625
Result for train_mnist_55a9b_00008:
date: 2022-07-22_16-18-28
done: false
experiment_id: 8dbd22e6caed4fe39351dffa3ef14eac
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 1
mean_accuracy: 0.8477166891098022
node_ip: 127.0.0.1
pid: 52007
time_since_restore: 6.2436230182647705
time_this_iter_s: 6.2436230182647705
time_total_s: 6.2436230182647705
timestamp: 1658503108
timesteps_since_restore: 0
training_iteration: 1
trial_id: 55a9b_00008
warmup_time: 0.005449056625366211
Result for train_mnist_55a9b_00001:
date: 2022-07-22_16-18-28
done: true
experiment_id: fcdeb049f9614755a9b7c9420ca2ae5e
experiment_tag: 1_hidden=380,lr=0.0874,momentum=0.5246
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 12
mean_accuracy: 0.9837166666984558
node_ip: 127.0.0.1
pid: 51977
time_since_restore: 74.98881888389587
time_this_iter_s: 1.791213035583496
time_total_s: 74.98881888389587
timestamp: 1658503108
timesteps_since_restore: 0
training_iteration: 12
trial_id: 55a9b_00001
warmup_time: 0.0029449462890625
Result for train_mnist_55a9b_00009:
date: 2022-07-22_16-18-29
done: false
experiment_id: c4f803baf65f4d4e9fd6abc85b2fd00c
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 1
mean_accuracy: 0.7675999999046326
node_ip: 127.0.0.1
pid: 51984
time_since_restore: 5.303471088409424
time_this_iter_s: 5.303471088409424
time_total_s: 5.303471088409424
timestamp: 1658503109
timesteps_since_restore: 0
training_iteration: 1
trial_id: 55a9b_00009
warmup_time: 0.0028328895568847656
Result for train_mnist_55a9b_00007:
date: 2022-07-22_16-18-30
done: true
experiment_id: d9469b1fc58b41db88da5446dc2a3b23
experiment_tag: 7_hidden=181,lr=0.0362,momentum=0.7903
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 12
mean_accuracy: 0.9799666404724121
node_ip: 127.0.0.1
pid: 52021
time_since_restore: 41.763158082962036
time_this_iter_s: 1.0622038841247559
time_total_s: 41.763158082962036
timestamp: 1658503110
timesteps_since_restore: 0
training_iteration: 12
trial_id: 55a9b_00007
warmup_time: 0.0028028488159179688
Result for train_mnist_55a9b_00008:
date: 2022-07-22_16-18-33
done: false
experiment_id: 8dbd22e6caed4fe39351dffa3ef14eac
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 8
mean_accuracy: 0.9599000215530396
node_ip: 127.0.0.1
pid: 52007
time_since_restore: 11.612935304641724
time_this_iter_s: 0.6818761825561523
time_total_s: 11.612935304641724
timestamp: 1658503113
timesteps_since_restore: 0
training_iteration: 8
trial_id: 55a9b_00008
warmup_time: 0.005449056625366211
Result for train_mnist_55a9b_00009:
date: 2022-07-22_16-18-34
done: false
experiment_id: c4f803baf65f4d4e9fd6abc85b2fd00c
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 9
mean_accuracy: 0.9319833517074585
node_ip: 127.0.0.1
pid: 51984
time_since_restore: 10.803268194198608
time_this_iter_s: 0.606992244720459
time_total_s: 10.803268194198608
timestamp: 1658503114
timesteps_since_restore: 0
training_iteration: 9
trial_id: 55a9b_00009
warmup_time: 0.0028328895568847656
Result for train_mnist_55a9b_00008:
date: 2022-07-22_16-18-36
done: true
experiment_id: 8dbd22e6caed4fe39351dffa3ef14eac
experiment_tag: 8_hidden=142,lr=0.0324,momentum=0.6604
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 12
mean_accuracy: 0.9693666696548462
node_ip: 127.0.0.1
pid: 52007
time_since_restore: 14.152745008468628
time_this_iter_s: 0.5980076789855957
time_total_s: 14.152745008468628
timestamp: 1658503116
timesteps_since_restore: 0
training_iteration: 12
trial_id: 55a9b_00008
warmup_time: 0.005449056625366211
Result for train_mnist_55a9b_00009:
date: 2022-07-22_16-18-36
done: true
experiment_id: c4f803baf65f4d4e9fd6abc85b2fd00c
experiment_tag: 9_hidden=97,lr=0.0245,momentum=0.1750
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 12
mean_accuracy: 0.9406999945640564
node_ip: 127.0.0.1
pid: 51984
time_since_restore: 12.640528202056885
time_this_iter_s: 0.5808131694793701
time_total_s: 12.640528202056885
timestamp: 1658503116
timesteps_since_restore: 0
training_iteration: 12
trial_id: 55a9b_00009
warmup_time: 0.0028328895568847656
2022-07-22 16:18:36,803 INFO tune.py:738 -- Total run time: 95.98 seconds (95.03 seconds for the tuning loop).
Best hyperparameters found were: {'threads': 2, 'lr': 0.09518133271957563, 'momentum': 0.8254987643140009, 'hidden': 258}
More Keras and TensorFlow Examples#
Memory NN Example: Example of training a Memory NN on bAbI with Keras using PBT.
TensorFlow MNIST Example: Converts the Advanced TF2.0 MNIST example to use Tune with the Trainable. This uses
tf.function. Original code from tensorflow: https://www.tensorflow.org/tutorials/quickstart/advancedKeras Cifar10 Example: A contributed example of tuning a Keras model on CIFAR10 with the PopulationBasedTraining scheduler.