Using Keras & TensorFlow with Tune¶
Example¶
import argparse
import os
from filelock import FileLock
from tensorflow.keras.datasets import mnist
import ray
from ray import air, tune
from ray.tune.schedulers import AsyncHyperBandScheduler
from ray.tune.integration.keras import TuneReportCallback
def train_mnist(config):
# https://github.com/tensorflow/tensorflow/issues/32159
import tensorflow as tf
batch_size = 128
num_classes = 10
epochs = 12
with FileLock(os.path.expanduser("~/.data.lock")):
(x_train, y_train), (x_test, y_test) = mnist.load_data()
x_train, x_test = x_train / 255.0, x_test / 255.0
model = tf.keras.models.Sequential(
[
tf.keras.layers.Flatten(input_shape=(28, 28)),
tf.keras.layers.Dense(config["hidden"], activation="relu"),
tf.keras.layers.Dropout(0.2),
tf.keras.layers.Dense(num_classes, activation="softmax"),
]
)
model.compile(
loss="sparse_categorical_crossentropy",
optimizer=tf.keras.optimizers.SGD(lr=config["lr"], momentum=config["momentum"]),
metrics=["accuracy"],
)
model.fit(
x_train,
y_train,
batch_size=batch_size,
epochs=epochs,
verbose=0,
validation_data=(x_test, y_test),
callbacks=[TuneReportCallback({"mean_accuracy": "accuracy"})],
)
def tune_mnist(num_training_iterations):
sched = AsyncHyperBandScheduler(
time_attr="training_iteration", max_t=400, grace_period=20
)
tuner = tune.Tuner(
tune.with_resources(
train_mnist,
resources={"cpu": 2, "gpu": 0}
),
tune_config=tune.TuneConfig(
metric="mean_accuracy",
mode="max",
scheduler=sched,
num_samples=10,
),
run_config=air.RunConfig(
name="exp",
stop={"mean_accuracy": 0.99, "training_iteration": num_training_iterations},
),
param_space={
"threads": 2,
"lr": tune.uniform(0.001, 0.1),
"momentum": tune.uniform(0.1, 0.9),
"hidden": tune.randint(32, 512),
},
)
results = tuner.fit()
print("Best hyperparameters found were: ", results.get_best_result().config)
if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument(
"--smoke-test", action="store_true", help="Finish quickly for testing"
)
parser.add_argument(
"--server-address",
type=str,
default=None,
required=False,
help="The address of server to connect to if using " "Ray Client.",
)
args, _ = parser.parse_known_args()
if args.smoke_test:
ray.init(num_cpus=4)
elif args.server_address:
ray.init(f"ray://{args.server_address}")
tune_mnist(num_training_iterations=5 if args.smoke_test else 300)
2022-07-22 16:16:58,114 INFO services.py:1483 -- View the Ray dashboard at http://127.0.0.1:8269
2022-07-22 16:17:00,822 WARNING function_trainable.py:619 -- Function checkpointing is disabled. This may result in unexpected behavior when using checkpointing features or certain schedulers. To enable, set the train function arguments to be `func(config, checkpoint_dir=None)`.
== Status ==
Current time: 2022-07-22 16:18:36 (running for 00:01:35.04)
Memory usage on this node: 9.0/16.0 GiB
Using AsyncHyperBand: num_stopped=0 Bracket: Iter 320.000: None | Iter 80.000: None | Iter 20.000: None
Resources requested: 0/16 CPUs, 0/0 GPUs, 0.0/5.47 GiB heap, 0.0/2.0 GiB objects
Current best trial: 55a9b_00002 with mean_accuracy=0.9904166460037231 and parameters={'threads': 2, 'lr': 0.09518133271957563, 'momentum': 0.8254987643140009, 'hidden': 258}
Result logdir: /Users/kai/ray_results/exp
Number of trials: 10/10 (10 TERMINATED)
Current time: 2022-07-22 16:18:36 (running for 00:01:35.04)
Memory usage on this node: 9.0/16.0 GiB
Using AsyncHyperBand: num_stopped=0 Bracket: Iter 320.000: None | Iter 80.000: None | Iter 20.000: None
Resources requested: 0/16 CPUs, 0/0 GPUs, 0.0/5.47 GiB heap, 0.0/2.0 GiB objects
Current best trial: 55a9b_00002 with mean_accuracy=0.9904166460037231 and parameters={'threads': 2, 'lr': 0.09518133271957563, 'momentum': 0.8254987643140009, 'hidden': 258}
Result logdir: /Users/kai/ray_results/exp
Number of trials: 10/10 (10 TERMINATED)
| Trial name | status | loc | hidden | lr | momentum | acc | iter | total time (s) |
|---|---|---|---|---|---|---|---|---|
| train_mnist_55a9b_00000 | TERMINATED | 127.0.0.1:51968 | 276 | 0.0406397 | 0.817788 | 0.98455 | 12 | 78.3252 |
| train_mnist_55a9b_00001 | TERMINATED | 127.0.0.1:51977 | 380 | 0.0873557 | 0.524634 | 0.983717 | 12 | 74.9888 |
| train_mnist_55a9b_00002 | TERMINATED | 127.0.0.1:51984 | 258 | 0.0951813 | 0.825499 | 0.990417 | 11 | 64.1272 |
| train_mnist_55a9b_00003 | TERMINATED | 127.0.0.1:51991 | 255 | 0.0971683 | 0.23161 | 0.977633 | 12 | 60.8475 |
| train_mnist_55a9b_00004 | TERMINATED | 127.0.0.1:52000 | 303 | 0.00440117 | 0.325439 | 0.90775 | 12 | 55.5722 |
| train_mnist_55a9b_00005 | TERMINATED | 127.0.0.1:52007 | 92 | 0.0651919 | 0.710183 | 0.974867 | 12 | 44.8092 |
| train_mnist_55a9b_00006 | TERMINATED | 127.0.0.1:52016 | 211 | 0.0731116 | 0.127751 | 0.97025 | 12 | 42.1217 |
| train_mnist_55a9b_00007 | TERMINATED | 127.0.0.1:52021 | 181 | 0.0362389 | 0.790345 | 0.979967 | 12 | 41.7632 |
| train_mnist_55a9b_00008 | TERMINATED | 127.0.0.1:52007 | 142 | 0.0323741 | 0.660418 | 0.969367 | 12 | 14.1527 |
| train_mnist_55a9b_00009 | TERMINATED | 127.0.0.1:51984 | 97 | 0.0244971 | 0.175045 | 0.9407 | 12 | 12.6405 |
2022-07-22 16:17:01,834 INFO plugin_schema_manager.py:52 -- Loading the default runtime env schemas: ['/Users/kai/coding/ray/python/ray/_private/runtime_env/../../runtime_env/schemas/working_dir_schema.json', '/Users/kai/coding/ray/python/ray/_private/runtime_env/../../runtime_env/schemas/pip_schema.json'].
(train_mnist pid=51968) 2022-07-22 16:17:08.627419: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
(train_mnist pid=51968) To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
(train_mnist pid=51968) /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:356: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
(train_mnist pid=51968) "The `lr` argument is deprecated, use `learning_rate` instead.")
(train_mnist pid=51968) 2022-07-22 16:17:08.947939: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
(train_mnist pid=51977) 2022-07-22 16:17:14.473677: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
(train_mnist pid=51977) To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
(train_mnist pid=51977) /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:356: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
(train_mnist pid=51977) "The `lr` argument is deprecated, use `learning_rate` instead.")
(train_mnist pid=51977) 2022-07-22 16:17:14.635104: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
(train_mnist pid=51984) 2022-07-22 16:17:20.406624: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
(train_mnist pid=51984) To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
(train_mnist pid=51984) /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:356: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
(train_mnist pid=51984) "The `lr` argument is deprecated, use `learning_rate` instead.")
(train_mnist pid=51984) 2022-07-22 16:17:20.681960: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
(train_mnist pid=51991) 2022-07-22 16:17:26.109460: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
(train_mnist pid=51991) To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
(train_mnist pid=51991) /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:356: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
(train_mnist pid=51991) "The `lr` argument is deprecated, use `learning_rate` instead.")
(train_mnist pid=51991) 2022-07-22 16:17:26.303375: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
(train_mnist pid=52000) 2022-07-22 16:17:31.899252: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
(train_mnist pid=52000) To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
(train_mnist pid=52000) /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:356: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
(train_mnist pid=52000) "The `lr` argument is deprecated, use `learning_rate` instead.")
(train_mnist pid=52000) 2022-07-22 16:17:32.300424: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
(train_mnist pid=52007) 2022-07-22 16:17:37.937471: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
(train_mnist pid=52007) To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
(train_mnist pid=52007) /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:356: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
(train_mnist pid=52007) "The `lr` argument is deprecated, use `learning_rate` instead.")
(train_mnist pid=52007) 2022-07-22 16:17:38.263888: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
(train_mnist pid=52016) 2022-07-22 16:17:43.657379: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
(train_mnist pid=52016) To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
(train_mnist pid=52016) /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:356: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
(train_mnist pid=52016) "The `lr` argument is deprecated, use `learning_rate` instead.")
(train_mnist pid=52016) 2022-07-22 16:17:43.828809: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
Result for train_mnist_55a9b_00000:
date: 2022-07-22_16-17-10
done: false
experiment_id: 3659349c38c746cfb71b4db5eb9302a0
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 1
mean_accuracy: 0.8903833627700806
node_ip: 127.0.0.1
pid: 51968
time_since_restore: 2.439258098602295
time_this_iter_s: 2.439258098602295
time_total_s: 2.439258098602295
timestamp: 1658503030
timesteps_since_restore: 0
training_iteration: 1
trial_id: 55a9b_00000
warmup_time: 0.003445863723754883
Result for train_mnist_55a9b_00004:
date: 2022-07-22_16-17-33
done: false
experiment_id: 6eb62b7cb38f442a867a9094f0664701
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 1
mean_accuracy: 0.6376166939735413
node_ip: 127.0.0.1
pid: 52000
time_since_restore: 2.4364511966705322
time_this_iter_s: 2.4364511966705322
time_total_s: 2.4364511966705322
timestamp: 1658503053
timesteps_since_restore: 0
training_iteration: 1
trial_id: 55a9b_00004
warmup_time: 0.0030939579010009766
Result for train_mnist_55a9b_00006:
date: 2022-07-22_16-17-45
done: false
experiment_id: 9594405e38084311a891b48addd13f75
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 1
mean_accuracy: 0.8557000160217285
node_ip: 127.0.0.1
pid: 52016
time_since_restore: 1.8570480346679688
time_this_iter_s: 1.8570480346679688
time_total_s: 1.8570480346679688
timestamp: 1658503065
timesteps_since_restore: 0
training_iteration: 1
trial_id: 55a9b_00006
warmup_time: 0.003566741943359375
Result for train_mnist_55a9b_00001:
date: 2022-07-22_16-17-15
done: false
experiment_id: fcdeb049f9614755a9b7c9420ca2ae5e
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 1
mean_accuracy: 0.8887666463851929
node_ip: 127.0.0.1
pid: 51977
time_since_restore: 1.9353628158569336
time_this_iter_s: 1.9353628158569336
time_total_s: 1.9353628158569336
timestamp: 1658503035
timesteps_since_restore: 0
training_iteration: 1
trial_id: 55a9b_00001
warmup_time: 0.0029449462890625
Result for train_mnist_55a9b_00005:
date: 2022-07-22_16-17-39
done: false
experiment_id: 8dbd22e6caed4fe39351dffa3ef14eac
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 1
mean_accuracy: 0.8789666891098022
node_ip: 127.0.0.1
pid: 52007
time_since_restore: 2.3337321281433105
time_this_iter_s: 2.3337321281433105
time_total_s: 2.3337321281433105
timestamp: 1658503059
timesteps_since_restore: 0
training_iteration: 1
trial_id: 55a9b_00005
warmup_time: 0.005449056625366211
Result for train_mnist_55a9b_00002:
date: 2022-07-22_16-17-21
done: false
experiment_id: c4f803baf65f4d4e9fd6abc85b2fd00c
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 1
mean_accuracy: 0.9112833142280579
node_ip: 127.0.0.1
pid: 51984
time_since_restore: 2.3220012187957764
time_this_iter_s: 2.3220012187957764
time_total_s: 2.3220012187957764
timestamp: 1658503041
timesteps_since_restore: 0
training_iteration: 1
trial_id: 55a9b_00002
warmup_time: 0.0028328895568847656
Result for train_mnist_55a9b_00003:
date: 2022-07-22_16-17-27
done: false
experiment_id: 469478f02b4a43f5b44c40e59989ad39
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 1
mean_accuracy: 0.8743166923522949
node_ip: 127.0.0.1
pid: 51991
time_since_restore: 2.0278611183166504
time_this_iter_s: 2.0278611183166504
time_total_s: 2.0278611183166504
timestamp: 1658503047
timesteps_since_restore: 0
training_iteration: 1
trial_id: 55a9b_00003
warmup_time: 0.0033779144287109375
(train_mnist pid=52021) 2022-07-22 16:17:51.567914: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA
(train_mnist pid=52021) To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
(train_mnist pid=52021) /Users/kai/.pyenv/versions/3.7.7/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:356: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
(train_mnist pid=52021) "The `lr` argument is deprecated, use `learning_rate` instead.")
(train_mnist pid=52021) 2022-07-22 16:17:52.977183: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
Result for train_mnist_55a9b_00005:
date: 2022-07-22_16-17-54
done: false
experiment_id: 8dbd22e6caed4fe39351dffa3ef14eac
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 3
mean_accuracy: 0.9490833282470703
node_ip: 127.0.0.1
pid: 52007
time_since_restore: 17.22033405303955
time_this_iter_s: 2.672102928161621
time_total_s: 17.22033405303955
timestamp: 1658503074
timesteps_since_restore: 0
training_iteration: 3
trial_id: 55a9b_00005
warmup_time: 0.005449056625366211
Result for train_mnist_55a9b_00006:
date: 2022-07-22_16-17-54
done: false
experiment_id: 9594405e38084311a891b48addd13f75
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 3
mean_accuracy: 0.9327999949455261
node_ip: 127.0.0.1
pid: 52016
time_since_restore: 11.758372068405151
time_this_iter_s: 3.0426323413848877
time_total_s: 11.758372068405151
timestamp: 1658503074
timesteps_since_restore: 0
training_iteration: 3
trial_id: 55a9b_00006
warmup_time: 0.003566741943359375
Result for train_mnist_55a9b_00003:
date: 2022-07-22_16-17-55
done: false
experiment_id: 469478f02b4a43f5b44c40e59989ad39
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 3
mean_accuracy: 0.9454166889190674
node_ip: 127.0.0.1
pid: 51991
time_since_restore: 29.733185052871704
time_this_iter_s: 3.0363340377807617
time_total_s: 29.733185052871704
timestamp: 1658503075
timesteps_since_restore: 0
training_iteration: 3
trial_id: 55a9b_00003
warmup_time: 0.0033779144287109375
Result for train_mnist_55a9b_00000:
date: 2022-07-22_16-17-55
done: false
experiment_id: 3659349c38c746cfb71b4db5eb9302a0
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 3
mean_accuracy: 0.958216667175293
node_ip: 127.0.0.1
pid: 51968
time_since_restore: 47.272178173065186
time_this_iter_s: 3.2986061573028564
time_total_s: 47.272178173065186
timestamp: 1658503075
timesteps_since_restore: 0
training_iteration: 3
trial_id: 55a9b_00000
warmup_time: 0.003445863723754883
Result for train_mnist_55a9b_00004:
date: 2022-07-22_16-17-55
done: false
experiment_id: 6eb62b7cb38f442a867a9094f0664701
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 3
mean_accuracy: 0.8524500131607056
node_ip: 127.0.0.1
pid: 52000
time_since_restore: 24.11396098136902
time_this_iter_s: 3.2331089973449707
time_total_s: 24.11396098136902
timestamp: 1658503075
timesteps_since_restore: 0
training_iteration: 3
trial_id: 55a9b_00004
warmup_time: 0.0030939579010009766
Result for train_mnist_55a9b_00002:
date: 2022-07-22_16-17-55
done: false
experiment_id: c4f803baf65f4d4e9fd6abc85b2fd00c
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 3
mean_accuracy: 0.9695500135421753
node_ip: 127.0.0.1
pid: 51984
time_since_restore: 35.78592824935913
time_this_iter_s: 3.021165132522583
time_total_s: 35.78592824935913
timestamp: 1658503075
timesteps_since_restore: 0
training_iteration: 3
trial_id: 55a9b_00002
warmup_time: 0.0028328895568847656
Result for train_mnist_55a9b_00001:
date: 2022-07-22_16-17-56
done: false
experiment_id: fcdeb049f9614755a9b7c9420ca2ae5e
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 3
mean_accuracy: 0.9560333490371704
node_ip: 127.0.0.1
pid: 51977
time_since_restore: 42.38909387588501
time_this_iter_s: 3.753290891647339
time_total_s: 42.38909387588501
timestamp: 1658503076
timesteps_since_restore: 0
training_iteration: 3
trial_id: 55a9b_00001
warmup_time: 0.0029449462890625
Result for train_mnist_55a9b_00005:
date: 2022-07-22_16-18-00
done: false
experiment_id: 8dbd22e6caed4fe39351dffa3ef14eac
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 5
mean_accuracy: 0.9611166715621948
node_ip: 127.0.0.1
pid: 52007
time_since_restore: 23.303561210632324
time_this_iter_s: 2.933852195739746
time_total_s: 23.303561210632324
timestamp: 1658503080
timesteps_since_restore: 0
training_iteration: 5
trial_id: 55a9b_00005
warmup_time: 0.005449056625366211
Result for train_mnist_55a9b_00007:
date: 2022-07-22_16-18-01
done: false
experiment_id: d9469b1fc58b41db88da5446dc2a3b23
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 1
mean_accuracy: 0.8797500133514404
node_ip: 127.0.0.1
pid: 52021
time_since_restore: 12.469872951507568
time_this_iter_s: 12.469872951507568
time_total_s: 12.469872951507568
timestamp: 1658503081
timesteps_since_restore: 0
training_iteration: 1
trial_id: 55a9b_00007
warmup_time: 0.0028028488159179688
Result for train_mnist_55a9b_00006:
date: 2022-07-22_16-18-01
done: false
experiment_id: 9594405e38084311a891b48addd13f75
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 5
mean_accuracy: 0.9499499797821045
node_ip: 127.0.0.1
pid: 52016
time_since_restore: 18.780059814453125
time_this_iter_s: 3.3080599308013916
time_total_s: 18.780059814453125
timestamp: 1658503081
timesteps_since_restore: 0
training_iteration: 5
trial_id: 55a9b_00006
warmup_time: 0.003566741943359375
Result for train_mnist_55a9b_00003:
date: 2022-07-22_16-18-02
done: false
experiment_id: 469478f02b4a43f5b44c40e59989ad39
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 5
mean_accuracy: 0.9601166844367981
node_ip: 127.0.0.1
pid: 51991
time_since_restore: 36.93912100791931
time_this_iter_s: 3.4057939052581787
time_total_s: 36.93912100791931
timestamp: 1658503082
timesteps_since_restore: 0
training_iteration: 5
trial_id: 55a9b_00003
warmup_time: 0.0033779144287109375
Result for train_mnist_55a9b_00000:
date: 2022-07-22_16-18-02
done: false
experiment_id: 3659349c38c746cfb71b4db5eb9302a0
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 5
mean_accuracy: 0.970466673374176
node_ip: 127.0.0.1
pid: 51968
time_since_restore: 54.49850010871887
time_this_iter_s: 3.4417831897735596
time_total_s: 54.49850010871887
timestamp: 1658503082
timesteps_since_restore: 0
training_iteration: 5
trial_id: 55a9b_00000
warmup_time: 0.003445863723754883
Result for train_mnist_55a9b_00004:
date: 2022-07-22_16-18-02
done: false
experiment_id: 6eb62b7cb38f442a867a9094f0664701
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 5
mean_accuracy: 0.8777499794960022
node_ip: 127.0.0.1
pid: 52000
time_since_restore: 31.513713121414185
time_this_iter_s: 3.506195068359375
time_total_s: 31.513713121414185
timestamp: 1658503082
timesteps_since_restore: 0
training_iteration: 5
trial_id: 55a9b_00004
warmup_time: 0.0030939579010009766
Result for train_mnist_55a9b_00002:
date: 2022-07-22_16-18-02
done: false
experiment_id: c4f803baf65f4d4e9fd6abc85b2fd00c
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 5
mean_accuracy: 0.979283332824707
node_ip: 127.0.0.1
pid: 51984
time_since_restore: 43.266417026519775
time_this_iter_s: 3.3383469581604004
time_total_s: 43.266417026519775
timestamp: 1658503082
timesteps_since_restore: 0
training_iteration: 5
trial_id: 55a9b_00002
warmup_time: 0.0028328895568847656
Result for train_mnist_55a9b_00001:
date: 2022-07-22_16-18-04
done: false
experiment_id: fcdeb049f9614755a9b7c9420ca2ae5e
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 5
mean_accuracy: 0.9692999720573425
node_ip: 127.0.0.1
pid: 51977
time_since_restore: 50.620792865753174
time_this_iter_s: 4.001068115234375
time_total_s: 50.620792865753174
timestamp: 1658503084
timesteps_since_restore: 0
training_iteration: 5
trial_id: 55a9b_00001
warmup_time: 0.0029449462890625
Result for train_mnist_55a9b_00005:
date: 2022-07-22_16-18-06
done: false
experiment_id: 8dbd22e6caed4fe39351dffa3ef14eac
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 7
mean_accuracy: 0.96711665391922
node_ip: 127.0.0.1
pid: 52007
time_since_restore: 29.40476107597351
time_this_iter_s: 2.976076126098633
time_total_s: 29.40476107597351
timestamp: 1658503086
timesteps_since_restore: 0
training_iteration: 7
trial_id: 55a9b_00005
warmup_time: 0.005449056625366211
Result for train_mnist_55a9b_00007:
date: 2022-07-22_16-18-07
done: false
experiment_id: d9469b1fc58b41db88da5446dc2a3b23
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 3
mean_accuracy: 0.951033353805542
node_ip: 127.0.0.1
pid: 52021
time_since_restore: 18.96213722229004
time_this_iter_s: 3.252371311187744
time_total_s: 18.96213722229004
timestamp: 1658503087
timesteps_since_restore: 0
training_iteration: 3
trial_id: 55a9b_00007
warmup_time: 0.0028028488159179688
Result for train_mnist_55a9b_00006:
date: 2022-07-22_16-18-08
done: false
experiment_id: 9594405e38084311a891b48addd13f75
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 7
mean_accuracy: 0.9584500193595886
node_ip: 127.0.0.1
pid: 52016
time_since_restore: 25.336583852767944
time_this_iter_s: 3.311979055404663
time_total_s: 25.336583852767944
timestamp: 1658503088
timesteps_since_restore: 0
training_iteration: 7
trial_id: 55a9b_00006
warmup_time: 0.003566741943359375
Result for train_mnist_55a9b_00003:
date: 2022-07-22_16-18-09
done: false
experiment_id: 469478f02b4a43f5b44c40e59989ad39
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 7
mean_accuracy: 0.9675499796867371
node_ip: 127.0.0.1
pid: 51991
time_since_restore: 43.7107310295105
time_this_iter_s: 3.3927559852600098
time_total_s: 43.7107310295105
timestamp: 1658503089
timesteps_since_restore: 0
training_iteration: 7
trial_id: 55a9b_00003
warmup_time: 0.0033779144287109375
Result for train_mnist_55a9b_00000:
date: 2022-07-22_16-18-09
done: false
experiment_id: 3659349c38c746cfb71b4db5eb9302a0
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 7
mean_accuracy: 0.9763000011444092
node_ip: 127.0.0.1
pid: 51968
time_since_restore: 61.30248522758484
time_this_iter_s: 3.4063682556152344
time_total_s: 61.30248522758484
timestamp: 1658503089
timesteps_since_restore: 0
training_iteration: 7
trial_id: 55a9b_00000
warmup_time: 0.003445863723754883
Result for train_mnist_55a9b_00002:
date: 2022-07-22_16-18-09
done: false
experiment_id: c4f803baf65f4d4e9fd6abc85b2fd00c
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 7
mean_accuracy: 0.9840666651725769
node_ip: 127.0.0.1
pid: 51984
time_since_restore: 50.212465047836304
time_this_iter_s: 3.43766188621521
time_total_s: 50.212465047836304
timestamp: 1658503089
timesteps_since_restore: 0
training_iteration: 7
trial_id: 55a9b_00002
warmup_time: 0.0028328895568847656
Result for train_mnist_55a9b_00004:
date: 2022-07-22_16-18-09
done: false
experiment_id: 6eb62b7cb38f442a867a9094f0664701
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 7
mean_accuracy: 0.8899999856948853
node_ip: 127.0.0.1
pid: 52000
time_since_restore: 38.63890194892883
time_this_iter_s: 3.5783908367156982
time_total_s: 38.63890194892883
timestamp: 1658503089
timesteps_since_restore: 0
training_iteration: 7
trial_id: 55a9b_00004
warmup_time: 0.0030939579010009766
Result for train_mnist_55a9b_00005:
date: 2022-07-22_16-18-12
done: false
experiment_id: 8dbd22e6caed4fe39351dffa3ef14eac
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 9
mean_accuracy: 0.9712333083152771
node_ip: 127.0.0.1
pid: 52007
time_since_restore: 35.3185760974884
time_this_iter_s: 3.0241990089416504
time_total_s: 35.3185760974884
timestamp: 1658503092
timesteps_since_restore: 0
training_iteration: 9
trial_id: 55a9b_00005
warmup_time: 0.005449056625366211
Result for train_mnist_55a9b_00001:
date: 2022-07-22_16-18-12
done: false
experiment_id: fcdeb049f9614755a9b7c9420ca2ae5e
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 7
mean_accuracy: 0.9755333065986633
node_ip: 127.0.0.1
pid: 51977
time_since_restore: 58.57745599746704
time_this_iter_s: 3.936232089996338
time_total_s: 58.57745599746704
timestamp: 1658503092
timesteps_since_restore: 0
training_iteration: 7
trial_id: 55a9b_00001
warmup_time: 0.0029449462890625
Result for train_mnist_55a9b_00007:
date: 2022-07-22_16-18-14
done: false
experiment_id: d9469b1fc58b41db88da5446dc2a3b23
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 5
mean_accuracy: 0.9648333191871643
node_ip: 127.0.0.1
pid: 52021
time_since_restore: 25.25843620300293
time_this_iter_s: 3.094501256942749
time_total_s: 25.25843620300293
timestamp: 1658503094
timesteps_since_restore: 0
training_iteration: 5
trial_id: 55a9b_00007
warmup_time: 0.0028028488159179688
Result for train_mnist_55a9b_00006:
date: 2022-07-22_16-18-15
done: false
experiment_id: 9594405e38084311a891b48addd13f75
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 9
mean_accuracy: 0.9646666646003723
node_ip: 127.0.0.1
pid: 52016
time_since_restore: 32.048911809921265
time_this_iter_s: 3.315690755844116
time_total_s: 32.048911809921265
timestamp: 1658503095
timesteps_since_restore: 0
training_iteration: 9
trial_id: 55a9b_00006
warmup_time: 0.003566741943359375
Result for train_mnist_55a9b_00003:
date: 2022-07-22_16-18-15
done: false
experiment_id: 469478f02b4a43f5b44c40e59989ad39
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 9
mean_accuracy: 0.9729499816894531
node_ip: 127.0.0.1
pid: 51991
time_since_restore: 50.50909209251404
time_this_iter_s: 3.4110782146453857
time_total_s: 50.50909209251404
timestamp: 1658503095
timesteps_since_restore: 0
training_iteration: 9
trial_id: 55a9b_00003
warmup_time: 0.0033779144287109375
Result for train_mnist_55a9b_00000:
date: 2022-07-22_16-18-16
done: false
experiment_id: 3659349c38c746cfb71b4db5eb9302a0
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 9
mean_accuracy: 0.9807666540145874
node_ip: 127.0.0.1
pid: 51968
time_since_restore: 68.26757216453552
time_this_iter_s: 3.4475879669189453
time_total_s: 68.26757216453552
timestamp: 1658503096
timesteps_since_restore: 0
training_iteration: 9
trial_id: 55a9b_00000
warmup_time: 0.003445863723754883
Result for train_mnist_55a9b_00002:
date: 2022-07-22_16-18-16
done: false
experiment_id: c4f803baf65f4d4e9fd6abc85b2fd00c
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 9
mean_accuracy: 0.9872999787330627
node_ip: 127.0.0.1
pid: 51984
time_since_restore: 57.01431703567505
time_this_iter_s: 3.3804008960723877
time_total_s: 57.01431703567505
timestamp: 1658503096
timesteps_since_restore: 0
training_iteration: 9
trial_id: 55a9b_00002
warmup_time: 0.0028328895568847656
Result for train_mnist_55a9b_00004:
date: 2022-07-22_16-18-16
done: false
experiment_id: 6eb62b7cb38f442a867a9094f0664701
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 9
mean_accuracy: 0.8989166617393494
node_ip: 127.0.0.1
pid: 52000
time_since_restore: 45.67929005622864
time_this_iter_s: 3.4561610221862793
time_total_s: 45.67929005622864
timestamp: 1658503096
timesteps_since_restore: 0
training_iteration: 9
trial_id: 55a9b_00004
warmup_time: 0.0030939579010009766
Result for train_mnist_55a9b_00005:
date: 2022-07-22_16-18-18
done: false
experiment_id: 8dbd22e6caed4fe39351dffa3ef14eac
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 11
mean_accuracy: 0.9744333624839783
node_ip: 127.0.0.1
pid: 52007
time_since_restore: 41.49077916145325
time_this_iter_s: 3.172250270843506
time_total_s: 41.49077916145325
timestamp: 1658503098
timesteps_since_restore: 0
training_iteration: 11
trial_id: 55a9b_00005
warmup_time: 0.005449056625366211
Result for train_mnist_55a9b_00001:
date: 2022-07-22_16-18-20
done: false
experiment_id: fcdeb049f9614755a9b7c9420ca2ae5e
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 9
mean_accuracy: 0.9806166887283325
node_ip: 127.0.0.1
pid: 51977
time_since_restore: 66.64132380485535
time_this_iter_s: 4.0674309730529785
time_total_s: 66.64132380485535
timestamp: 1658503100
timesteps_since_restore: 0
training_iteration: 9
trial_id: 55a9b_00001
warmup_time: 0.0029449462890625
Result for train_mnist_55a9b_00007:
date: 2022-07-22_16-18-20
done: false
experiment_id: d9469b1fc58b41db88da5446dc2a3b23
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 7
mean_accuracy: 0.970716655254364
node_ip: 127.0.0.1
pid: 52021
time_since_restore: 31.897236108779907
time_this_iter_s: 3.3691420555114746
time_total_s: 31.897236108779907
timestamp: 1658503100
timesteps_since_restore: 0
training_iteration: 7
trial_id: 55a9b_00007
warmup_time: 0.0028028488159179688
Result for train_mnist_55a9b_00005:
date: 2022-07-22_16-18-21
done: true
experiment_id: 8dbd22e6caed4fe39351dffa3ef14eac
experiment_tag: 5_hidden=92,lr=0.0652,momentum=0.7102
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 12
mean_accuracy: 0.9748666882514954
node_ip: 127.0.0.1
pid: 52007
time_since_restore: 44.80922222137451
time_this_iter_s: 3.3184430599212646
time_total_s: 44.80922222137451
timestamp: 1658503101
timesteps_since_restore: 0
training_iteration: 12
trial_id: 55a9b_00005
warmup_time: 0.005449056625366211
Result for train_mnist_55a9b_00006:
date: 2022-07-22_16-18-22
done: false
experiment_id: 9594405e38084311a891b48addd13f75
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 11
mean_accuracy: 0.9679166674613953
node_ip: 127.0.0.1
pid: 52016
time_since_restore: 39.08963179588318
time_this_iter_s: 3.4860758781433105
time_total_s: 39.08963179588318
timestamp: 1658503102
timesteps_since_restore: 0
training_iteration: 11
trial_id: 55a9b_00006
warmup_time: 0.003566741943359375
Result for train_mnist_55a9b_00003:
date: 2022-07-22_16-18-23
done: false
experiment_id: 469478f02b4a43f5b44c40e59989ad39
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 11
mean_accuracy: 0.9771833419799805
node_ip: 127.0.0.1
pid: 51991
time_since_restore: 57.6213219165802
time_this_iter_s: 3.4615819454193115
time_total_s: 57.6213219165802
timestamp: 1658503103
timesteps_since_restore: 0
training_iteration: 11
trial_id: 55a9b_00003
warmup_time: 0.0033779144287109375
Result for train_mnist_55a9b_00000:
date: 2022-07-22_16-18-23
done: false
experiment_id: 3659349c38c746cfb71b4db5eb9302a0
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 11
mean_accuracy: 0.98416668176651
node_ip: 127.0.0.1
pid: 51968
time_since_restore: 75.32713007926941
time_this_iter_s: 3.443808078765869
time_total_s: 75.32713007926941
timestamp: 1658503103
timesteps_since_restore: 0
training_iteration: 11
trial_id: 55a9b_00000
warmup_time: 0.003445863723754883
Result for train_mnist_55a9b_00002:
date: 2022-07-22_16-18-23
done: true
experiment_id: c4f803baf65f4d4e9fd6abc85b2fd00c
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 11
mean_accuracy: 0.9904166460037231
node_ip: 127.0.0.1
pid: 51984
time_since_restore: 64.12720203399658
time_this_iter_s: 3.508151054382324
time_total_s: 64.12720203399658
timestamp: 1658503103
timesteps_since_restore: 0
training_iteration: 11
trial_id: 55a9b_00002
warmup_time: 0.0028328895568847656
Result for train_mnist_55a9b_00004:
date: 2022-07-22_16-18-23
done: false
experiment_id: 6eb62b7cb38f442a867a9094f0664701
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 11
mean_accuracy: 0.9052166938781738
node_ip: 127.0.0.1
pid: 52000
time_since_restore: 52.687995195388794
time_this_iter_s: 3.420351982116699
time_total_s: 52.687995195388794
timestamp: 1658503103
timesteps_since_restore: 0
training_iteration: 11
trial_id: 55a9b_00004
warmup_time: 0.0030939579010009766
Result for train_mnist_55a9b_00006:
date: 2022-07-22_16-18-25
done: true
experiment_id: 9594405e38084311a891b48addd13f75
experiment_tag: 6_hidden=211,lr=0.0731,momentum=0.1278
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 12
mean_accuracy: 0.9702500104904175
node_ip: 127.0.0.1
pid: 52016
time_since_restore: 42.1216938495636
time_this_iter_s: 3.03206205368042
time_total_s: 42.1216938495636
timestamp: 1658503105
timesteps_since_restore: 0
training_iteration: 12
trial_id: 55a9b_00006
warmup_time: 0.003566741943359375
Result for train_mnist_55a9b_00003:
date: 2022-07-22_16-18-26
done: true
experiment_id: 469478f02b4a43f5b44c40e59989ad39
experiment_tag: 3_hidden=255,lr=0.0972,momentum=0.2316
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 12
mean_accuracy: 0.9776333570480347
node_ip: 127.0.0.1
pid: 51991
time_since_restore: 60.8474760055542
time_this_iter_s: 3.226154088973999
time_total_s: 60.8474760055542
timestamp: 1658503106
timesteps_since_restore: 0
training_iteration: 12
trial_id: 55a9b_00003
warmup_time: 0.0033779144287109375
Result for train_mnist_55a9b_00000:
date: 2022-07-22_16-18-26
done: true
experiment_id: 3659349c38c746cfb71b4db5eb9302a0
experiment_tag: 0_hidden=276,lr=0.0406,momentum=0.8178
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 12
mean_accuracy: 0.9845499992370605
node_ip: 127.0.0.1
pid: 51968
time_since_restore: 78.32520508766174
time_this_iter_s: 2.998075008392334
time_total_s: 78.32520508766174
timestamp: 1658503106
timesteps_since_restore: 0
training_iteration: 12
trial_id: 55a9b_00000
warmup_time: 0.003445863723754883
Result for train_mnist_55a9b_00007:
date: 2022-07-22_16-18-26
done: false
experiment_id: d9469b1fc58b41db88da5446dc2a3b23
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 9
mean_accuracy: 0.9751333594322205
node_ip: 127.0.0.1
pid: 52021
time_since_restore: 37.76195311546326
time_this_iter_s: 2.7159180641174316
time_total_s: 37.76195311546326
timestamp: 1658503106
timesteps_since_restore: 0
training_iteration: 9
trial_id: 55a9b_00007
warmup_time: 0.0028028488159179688
Result for train_mnist_55a9b_00004:
date: 2022-07-22_16-18-26
done: true
experiment_id: 6eb62b7cb38f442a867a9094f0664701
experiment_tag: 4_hidden=303,lr=0.0044,momentum=0.3254
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 12
mean_accuracy: 0.9077500104904175
node_ip: 127.0.0.1
pid: 52000
time_since_restore: 55.57219409942627
time_this_iter_s: 2.8841989040374756
time_total_s: 55.57219409942627
timestamp: 1658503106
timesteps_since_restore: 0
training_iteration: 12
trial_id: 55a9b_00004
warmup_time: 0.0030939579010009766
Result for train_mnist_55a9b_00001:
date: 2022-07-22_16-18-27
done: false
experiment_id: fcdeb049f9614755a9b7c9420ca2ae5e
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 11
mean_accuracy: 0.9830166697502136
node_ip: 127.0.0.1
pid: 51977
time_since_restore: 73.19760584831238
time_this_iter_s: 2.7281620502471924
time_total_s: 73.19760584831238
timestamp: 1658503107
timesteps_since_restore: 0
training_iteration: 11
trial_id: 55a9b_00001
warmup_time: 0.0029449462890625
Result for train_mnist_55a9b_00008:
date: 2022-07-22_16-18-28
done: false
experiment_id: 8dbd22e6caed4fe39351dffa3ef14eac
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 1
mean_accuracy: 0.8477166891098022
node_ip: 127.0.0.1
pid: 52007
time_since_restore: 6.2436230182647705
time_this_iter_s: 6.2436230182647705
time_total_s: 6.2436230182647705
timestamp: 1658503108
timesteps_since_restore: 0
training_iteration: 1
trial_id: 55a9b_00008
warmup_time: 0.005449056625366211
Result for train_mnist_55a9b_00001:
date: 2022-07-22_16-18-28
done: true
experiment_id: fcdeb049f9614755a9b7c9420ca2ae5e
experiment_tag: 1_hidden=380,lr=0.0874,momentum=0.5246
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 12
mean_accuracy: 0.9837166666984558
node_ip: 127.0.0.1
pid: 51977
time_since_restore: 74.98881888389587
time_this_iter_s: 1.791213035583496
time_total_s: 74.98881888389587
timestamp: 1658503108
timesteps_since_restore: 0
training_iteration: 12
trial_id: 55a9b_00001
warmup_time: 0.0029449462890625
Result for train_mnist_55a9b_00009:
date: 2022-07-22_16-18-29
done: false
experiment_id: c4f803baf65f4d4e9fd6abc85b2fd00c
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 1
mean_accuracy: 0.7675999999046326
node_ip: 127.0.0.1
pid: 51984
time_since_restore: 5.303471088409424
time_this_iter_s: 5.303471088409424
time_total_s: 5.303471088409424
timestamp: 1658503109
timesteps_since_restore: 0
training_iteration: 1
trial_id: 55a9b_00009
warmup_time: 0.0028328895568847656
Result for train_mnist_55a9b_00007:
date: 2022-07-22_16-18-30
done: true
experiment_id: d9469b1fc58b41db88da5446dc2a3b23
experiment_tag: 7_hidden=181,lr=0.0362,momentum=0.7903
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 12
mean_accuracy: 0.9799666404724121
node_ip: 127.0.0.1
pid: 52021
time_since_restore: 41.763158082962036
time_this_iter_s: 1.0622038841247559
time_total_s: 41.763158082962036
timestamp: 1658503110
timesteps_since_restore: 0
training_iteration: 12
trial_id: 55a9b_00007
warmup_time: 0.0028028488159179688
Result for train_mnist_55a9b_00008:
date: 2022-07-22_16-18-33
done: false
experiment_id: 8dbd22e6caed4fe39351dffa3ef14eac
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 8
mean_accuracy: 0.9599000215530396
node_ip: 127.0.0.1
pid: 52007
time_since_restore: 11.612935304641724
time_this_iter_s: 0.6818761825561523
time_total_s: 11.612935304641724
timestamp: 1658503113
timesteps_since_restore: 0
training_iteration: 8
trial_id: 55a9b_00008
warmup_time: 0.005449056625366211
Result for train_mnist_55a9b_00009:
date: 2022-07-22_16-18-34
done: false
experiment_id: c4f803baf65f4d4e9fd6abc85b2fd00c
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 9
mean_accuracy: 0.9319833517074585
node_ip: 127.0.0.1
pid: 51984
time_since_restore: 10.803268194198608
time_this_iter_s: 0.606992244720459
time_total_s: 10.803268194198608
timestamp: 1658503114
timesteps_since_restore: 0
training_iteration: 9
trial_id: 55a9b_00009
warmup_time: 0.0028328895568847656
Result for train_mnist_55a9b_00008:
date: 2022-07-22_16-18-36
done: true
experiment_id: 8dbd22e6caed4fe39351dffa3ef14eac
experiment_tag: 8_hidden=142,lr=0.0324,momentum=0.6604
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 12
mean_accuracy: 0.9693666696548462
node_ip: 127.0.0.1
pid: 52007
time_since_restore: 14.152745008468628
time_this_iter_s: 0.5980076789855957
time_total_s: 14.152745008468628
timestamp: 1658503116
timesteps_since_restore: 0
training_iteration: 12
trial_id: 55a9b_00008
warmup_time: 0.005449056625366211
Result for train_mnist_55a9b_00009:
date: 2022-07-22_16-18-36
done: true
experiment_id: c4f803baf65f4d4e9fd6abc85b2fd00c
experiment_tag: 9_hidden=97,lr=0.0245,momentum=0.1750
hostname: Kais-MacBook-Pro.local
iterations_since_restore: 12
mean_accuracy: 0.9406999945640564
node_ip: 127.0.0.1
pid: 51984
time_since_restore: 12.640528202056885
time_this_iter_s: 0.5808131694793701
time_total_s: 12.640528202056885
timestamp: 1658503116
timesteps_since_restore: 0
training_iteration: 12
trial_id: 55a9b_00009
warmup_time: 0.0028328895568847656
2022-07-22 16:18:36,803 INFO tune.py:738 -- Total run time: 95.98 seconds (95.03 seconds for the tuning loop).
Best hyperparameters found were: {'threads': 2, 'lr': 0.09518133271957563, 'momentum': 0.8254987643140009, 'hidden': 258}
More Keras and TensorFlow Examples¶
Memory NN Example: Example of training a Memory NN on bAbI with Keras using PBT.
TensorFlow MNIST Example: Converts the Advanced TF2.0 MNIST example to use Tune with the Trainable. This uses
tf.function. Original code from tensorflow: https://www.tensorflow.org/tutorials/quickstart/advancedKeras Cifar10 Example: A contributed example of tuning a Keras model on CIFAR10 with the PopulationBasedTraining scheduler.