Table Of Contents
In [42]:
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
tf.__version__Out [42]:
'2.14.0'
Prep Simple Data
Here, the dependent variable will JUST BE +10 of the independent variable:
In [43]:
# model.fit(tf.expand_dims(X, axis=-1), y, epochs=5)
# Create features
indVar = np.array([-7.0, -4.0, -1.0, 2.0, 5.0, 8.0, 11.0, 14.0])
# Create labels
depVar = np.array([3.0, 6.0, 9.0, 12.0, 15.0, 18.0, 21.0, 24.0])
# Visualize it
plt.scatter(indVar, depVar);
Train A Model
In [44]:
# Set random seed
tf.random.set_seed(42)
# Create a model using the Sequential API
layer1 = tf.keras.layers.Dense(1)
model = tf.keras.Sequential()
model.add(layer1)
# Compile the model
model.compile(loss=tf.keras.losses.mae, # mae is short for mean absolute error
optimizer=tf.keras.optimizers.SGD(), # SGD is short for stochastic gradient descent
metrics=["mae"])
# Fit the model
model.fit(tf.expand_dims(indVar, axis=-1), depVar, epochs=5)Epoch 1/5 1/1 [==============================] - 1s 1s/step - loss: 16.8788 - mae: 16.8788 Epoch 2/5 1/1 [==============================] - 0s 15ms/step - loss: 16.5976 - mae: 16.5976 Epoch 3/5 1/1 [==============================] - 0s 14ms/step - loss: 16.3163 - mae: 16.3163 Epoch 4/5 1/1 [==============================] - 0s 14ms/step - loss: 16.0351 - mae: 16.0351 Epoch 5/5 1/1 [==============================] - 0s 14ms/step - loss: 15.7538 - mae: 15.7538
Out [44]:
<keras.src.callbacks.History at 0x7f29c91a90>
Predict With The Model
In [45]:
# Make a prediction with the model
model.predict([17.0])1/1 [==============================] - 0s 178ms/step
Out [45]:
array([[-8.86956]], dtype=float32)
hmmm.... the model guess was 12, but we know the answer should be 27....it's a little off.
Improve the Model: More Epochs
Many ways can be used to improve the performance of a model:
- During creation:
- use a different number of layers
- use different types of layers
- change the activation function of each layer
- During model compilation:
- change the optimizer function (i.e adam)
- change the learning rate of the optimizer function (higher leads to bigger "Steps", lower leads to smaller "steps")
- During model fitting:
- add more epochs
- add more data
Increase the training epoch count
In [46]:
# Create & compile a model (same as above)
modelTwo = tf.keras.Sequential()
modelTwo.add(layer1)
modelTwo.compile(loss=tf.keras.losses.mae,
optimizer=tf.keras.optimizers.SGD(),
metrics=["mae"])
modelTwo.fit(tf.expand_dims(indVar, axis=-1), depVar, epochs=100) # train for 100 epochs not 10Epoch 1/100 1/1 [==============================] - 1s 952ms/step - loss: 15.4726 - mae: 15.4726 Epoch 2/100 1/1 [==============================] - 0s 15ms/step - loss: 15.1913 - mae: 15.1913 Epoch 3/100 1/1 [==============================] - 0s 14ms/step - loss: 14.9138 - mae: 14.9138 Epoch 4/100 1/1 [==============================] - 0s 15ms/step - loss: 14.7813 - mae: 14.7813 Epoch 5/100 1/1 [==============================] - 0s 15ms/step - loss: 14.6488 - mae: 14.6488 Epoch 6/100 1/1 [==============================] - 0s 15ms/step - loss: 14.5163 - mae: 14.5163 Epoch 7/100 1/1 [==============================] - 0s 15ms/step - loss: 14.3838 - mae: 14.3838 Epoch 8/100 1/1 [==============================] - 0s 15ms/step - loss: 14.2513 - mae: 14.2513 Epoch 9/100 1/1 [==============================] - 0s 15ms/step - loss: 14.1188 - mae: 14.1188 Epoch 10/100 1/1 [==============================] - 0s 16ms/step - loss: 13.9863 - mae: 13.9863 Epoch 11/100 1/1 [==============================] - 0s 15ms/step - loss: 13.8538 - mae: 13.8538 Epoch 12/100 1/1 [==============================] - 0s 15ms/step - loss: 13.7213 - mae: 13.7213 Epoch 13/100 1/1 [==============================] - 0s 15ms/step - loss: 13.5888 - mae: 13.5888 Epoch 14/100 1/1 [==============================] - 0s 15ms/step - loss: 13.4563 - mae: 13.4563 Epoch 15/100 1/1 [==============================] - 0s 15ms/step - loss: 13.3238 - mae: 13.3238 Epoch 16/100 1/1 [==============================] - 0s 15ms/step - loss: 13.1913 - mae: 13.1913 Epoch 17/100 1/1 [==============================] - 0s 15ms/step - loss: 13.0588 - mae: 13.0588 Epoch 18/100 1/1 [==============================] - 0s 15ms/step - loss: 12.9263 - mae: 12.9263 Epoch 19/100 1/1 [==============================] - 0s 14ms/step - loss: 12.7938 - mae: 12.7938 Epoch 20/100 1/1 [==============================] - 0s 14ms/step - loss: 12.6613 - mae: 12.6613 Epoch 21/100 1/1 [==============================] - 0s 14ms/step - loss: 12.5288 - mae: 12.5288 Epoch 22/100 1/1 [==============================] - 0s 15ms/step - loss: 12.3963 - mae: 12.3963 Epoch 23/100 1/1 [==============================] - 0s 16ms/step - loss: 12.2638 - mae: 12.2638 Epoch 24/100 1/1 [==============================] - 0s 15ms/step - loss: 12.1313 - mae: 12.1313 Epoch 25/100 1/1 [==============================] - 0s 15ms/step - loss: 11.9988 - mae: 11.9988 Epoch 26/100 1/1 [==============================] - 0s 15ms/step - loss: 11.8663 - mae: 11.8663 Epoch 27/100 1/1 [==============================] - 0s 15ms/step - loss: 11.7338 - mae: 11.7338 Epoch 28/100 1/1 [==============================] - 0s 15ms/step - loss: 11.6013 - mae: 11.6013 Epoch 29/100 1/1 [==============================] - 0s 15ms/step - loss: 11.4688 - mae: 11.4688 Epoch 30/100 1/1 [==============================] - 0s 14ms/step - loss: 11.3363 - mae: 11.3363 Epoch 31/100 1/1 [==============================] - 0s 16ms/step - loss: 11.2038 - mae: 11.2038 Epoch 32/100 1/1 [==============================] - 0s 26ms/step - loss: 11.0713 - mae: 11.0713 Epoch 33/100 1/1 [==============================] - 0s 20ms/step - loss: 10.9388 - mae: 10.9388 Epoch 34/100 1/1 [==============================] - 0s 15ms/step - loss: 10.8063 - mae: 10.8063 Epoch 35/100 1/1 [==============================] - 0s 15ms/step - loss: 10.6738 - mae: 10.6738 Epoch 36/100 1/1 [==============================] - 0s 17ms/step - loss: 10.5413 - mae: 10.5413 Epoch 37/100 1/1 [==============================] - 0s 21ms/step - loss: 10.4088 - mae: 10.4088 Epoch 38/100 1/1 [==============================] - 0s 17ms/step - loss: 10.2763 - mae: 10.2763 Epoch 39/100 1/1 [==============================] - 0s 19ms/step - loss: 10.1438 - mae: 10.1438 Epoch 40/100 1/1 [==============================] - 0s 19ms/step - loss: 10.0113 - mae: 10.0113 Epoch 41/100 1/1 [==============================] - 0s 17ms/step - loss: 9.8788 - mae: 9.8788 Epoch 42/100 1/1 [==============================] - 0s 18ms/step - loss: 9.7463 - mae: 9.7463 Epoch 43/100 1/1 [==============================] - 0s 17ms/step - loss: 9.6138 - mae: 9.6138 Epoch 44/100 1/1 [==============================] - 0s 18ms/step - loss: 9.4813 - mae: 9.4813 Epoch 45/100 1/1 [==============================] - 0s 18ms/step - loss: 9.3488 - mae: 9.3488 Epoch 46/100 1/1 [==============================] - 0s 17ms/step - loss: 9.2163 - mae: 9.2163 Epoch 47/100 1/1 [==============================] - 0s 17ms/step - loss: 9.0838 - mae: 9.0838 Epoch 48/100 1/1 [==============================] - 0s 17ms/step - loss: 8.9513 - mae: 8.9513 Epoch 49/100 1/1 [==============================] - 0s 17ms/step - loss: 8.8188 - mae: 8.8188 Epoch 50/100 1/1 [==============================] - 0s 16ms/step - loss: 8.6863 - mae: 8.6863 Epoch 51/100 1/1 [==============================] - 0s 17ms/step - loss: 8.5538 - mae: 8.5538 Epoch 52/100 1/1 [==============================] - 0s 17ms/step - loss: 8.4213 - mae: 8.4213 Epoch 53/100 1/1 [==============================] - 0s 18ms/step - loss: 8.2888 - mae: 8.2888 Epoch 54/100 1/1 [==============================] - 0s 17ms/step - loss: 8.1563 - mae: 8.1563 Epoch 55/100 1/1 [==============================] - 0s 18ms/step - loss: 8.0238 - mae: 8.0238 Epoch 56/100 1/1 [==============================] - 0s 19ms/step - loss: 7.8913 - mae: 7.8913 Epoch 57/100 1/1 [==============================] - 0s 17ms/step - loss: 7.7588 - mae: 7.7588 Epoch 58/100 1/1 [==============================] - 0s 28ms/step - loss: 7.6263 - mae: 7.6263 Epoch 59/100 1/1 [==============================] - 0s 24ms/step - loss: 7.4938 - mae: 7.4938 Epoch 60/100 1/1 [==============================] - 0s 22ms/step - loss: 7.3613 - mae: 7.3613 Epoch 61/100 1/1 [==============================] - 0s 15ms/step - loss: 7.2288 - mae: 7.2288 Epoch 62/100 1/1 [==============================] - 0s 14ms/step - loss: 7.0963 - mae: 7.0963 Epoch 63/100 1/1 [==============================] - 0s 15ms/step - loss: 7.0106 - mae: 7.0106 Epoch 64/100 1/1 [==============================] - 0s 14ms/step - loss: 7.0050 - mae: 7.0050 Epoch 65/100 1/1 [==============================] - 0s 14ms/step - loss: 6.9994 - mae: 6.9994 Epoch 66/100 1/1 [==============================] - 0s 14ms/step - loss: 6.9938 - mae: 6.9938 Epoch 67/100 1/1 [==============================] - 0s 15ms/step - loss: 6.9881 - mae: 6.9881 Epoch 68/100 1/1 [==============================] - 0s 14ms/step - loss: 6.9825 - mae: 6.9825 Epoch 69/100 1/1 [==============================] - 0s 14ms/step - loss: 6.9769 - mae: 6.9769 Epoch 70/100 1/1 [==============================] - 0s 14ms/step - loss: 6.9713 - mae: 6.9713 Epoch 71/100 1/1 [==============================] - 0s 15ms/step - loss: 6.9656 - mae: 6.9656 Epoch 72/100 1/1 [==============================] - 0s 15ms/step - loss: 6.9600 - mae: 6.9600 Epoch 73/100 1/1 [==============================] - 0s 14ms/step - loss: 6.9544 - mae: 6.9544 Epoch 74/100 1/1 [==============================] - 0s 14ms/step - loss: 6.9488 - mae: 6.9488 Epoch 75/100 1/1 [==============================] - 0s 15ms/step - loss: 6.9431 - mae: 6.9431 Epoch 76/100 1/1 [==============================] - 0s 15ms/step - loss: 6.9375 - mae: 6.9375 Epoch 77/100 1/1 [==============================] - 0s 19ms/step - loss: 6.9319 - mae: 6.9319 Epoch 78/100 1/1 [==============================] - 0s 15ms/step - loss: 6.9263 - mae: 6.9263 Epoch 79/100 1/1 [==============================] - 0s 15ms/step - loss: 6.9206 - mae: 6.9206 Epoch 80/100 1/1 [==============================] - 0s 16ms/step - loss: 6.9150 - mae: 6.9150 Epoch 81/100 1/1 [==============================] - 0s 15ms/step - loss: 6.9094 - mae: 6.9094 Epoch 82/100 1/1 [==============================] - 0s 15ms/step - loss: 6.9038 - mae: 6.9038 Epoch 83/100 1/1 [==============================] - 0s 15ms/step - loss: 6.8981 - mae: 6.8981 Epoch 84/100 1/1 [==============================] - 0s 16ms/step - loss: 6.8925 - mae: 6.8925 Epoch 85/100 1/1 [==============================] - 0s 14ms/step - loss: 6.8869 - mae: 6.8869 Epoch 86/100 1/1 [==============================] - 0s 17ms/step - loss: 6.8813 - mae: 6.8813 Epoch 87/100 1/1 [==============================] - 0s 14ms/step - loss: 6.8756 - mae: 6.8756 Epoch 88/100 1/1 [==============================] - 0s 15ms/step - loss: 6.8700 - mae: 6.8700 Epoch 89/100 1/1 [==============================] - 0s 16ms/step - loss: 6.8644 - mae: 6.8644 Epoch 90/100 1/1 [==============================] - 0s 14ms/step - loss: 6.8588 - mae: 6.8588 Epoch 91/100 1/1 [==============================] - 0s 16ms/step - loss: 6.8531 - mae: 6.8531 Epoch 92/100 1/1 [==============================] - 0s 14ms/step - loss: 6.8475 - mae: 6.8475 Epoch 93/100 1/1 [==============================] - 0s 15ms/step - loss: 6.8419 - mae: 6.8419 Epoch 94/100 1/1 [==============================] - 0s 15ms/step - loss: 6.8363 - mae: 6.8363 Epoch 95/100 1/1 [==============================] - 0s 15ms/step - loss: 6.8306 - mae: 6.8306 Epoch 96/100 1/1 [==============================] - 0s 14ms/step - loss: 6.8250 - mae: 6.8250 Epoch 97/100 1/1 [==============================] - 0s 14ms/step - loss: 6.8194 - mae: 6.8194 Epoch 98/100 1/1 [==============================] - 0s 15ms/step - loss: 6.8138 - mae: 6.8138 Epoch 99/100 1/1 [==============================] - 0s 14ms/step - loss: 6.8081 - mae: 6.8081 Epoch 100/100 1/1 [==============================] - 0s 14ms/step - loss: 6.8025 - mae: 6.8025
Out [46]:
<keras.src.callbacks.History at 0x7f29c920d0>
Predict Again
In [47]:
# Try and predict what y would be if X was 17.0
modelTwo.predict([17.0]) # the right answer is 27.0 (y = X + 10)1/1 [==============================] - 0s 168ms/step
Out [47]:
array([[29.515434]], dtype=float32)
the model guessed 29, and the expected response is 27: CLOSER!!
Improve the model: adding a layer
In [48]:
# CREATE & COMPILE & FITthe model
modelThree = tf.keras.Sequential()
m3L1 = tf.keras.layers.Dense(100, activation="relu")
m3L2 = tf.keras.layers.Dense(1)
modelThree.add(m3L1)
modelThree.add(m3L2)
modelThree.compile(loss=tf.keras.losses.mae,
optimizer=tf.keras.optimizers.SGD(),
metrics=["mae"])
modelThree.fit(tf.expand_dims(indVar, axis=-1), depVar, epochs=100)Epoch 1/100 1/1 [==============================] - 1s 1s/step - loss: 12.8534 - mae: 12.8534 Epoch 2/100 1/1 [==============================] - 0s 17ms/step - loss: 12.2273 - mae: 12.2273 Epoch 3/100 1/1 [==============================] - 0s 15ms/step - loss: 11.6047 - mae: 11.6047 Epoch 4/100 1/1 [==============================] - 0s 16ms/step - loss: 10.9744 - mae: 10.9744 Epoch 5/100 1/1 [==============================] - 0s 15ms/step - loss: 10.3196 - mae: 10.3196 Epoch 6/100 1/1 [==============================] - 0s 15ms/step - loss: 9.6382 - mae: 9.6382 Epoch 7/100 1/1 [==============================] - 0s 15ms/step - loss: 8.9336 - mae: 8.9336 Epoch 8/100 1/1 [==============================] - 0s 14ms/step - loss: 8.1899 - mae: 8.1899 Epoch 9/100 1/1 [==============================] - 0s 15ms/step - loss: 7.4065 - mae: 7.4065 Epoch 10/100 1/1 [==============================] - 0s 15ms/step - loss: 6.5808 - mae: 6.5808 Epoch 11/100 1/1 [==============================] - 0s 15ms/step - loss: 5.6946 - mae: 5.6946 Epoch 12/100 1/1 [==============================] - 0s 15ms/step - loss: 4.7415 - mae: 4.7415 Epoch 13/100 1/1 [==============================] - 0s 15ms/step - loss: 4.1203 - mae: 4.1203 Epoch 14/100 1/1 [==============================] - 0s 15ms/step - loss: 3.9730 - mae: 3.9730 Epoch 15/100 1/1 [==============================] - 0s 15ms/step - loss: 3.9780 - mae: 3.9780 Epoch 16/100 1/1 [==============================] - 0s 15ms/step - loss: 3.9079 - mae: 3.9079 Epoch 17/100 1/1 [==============================] - 0s 15ms/step - loss: 3.9745 - mae: 3.9745 Epoch 18/100 1/1 [==============================] - 0s 15ms/step - loss: 3.9150 - mae: 3.9150 Epoch 19/100 1/1 [==============================] - 0s 15ms/step - loss: 3.9486 - mae: 3.9486 Epoch 20/100 1/1 [==============================] - 0s 15ms/step - loss: 3.9223 - mae: 3.9223 Epoch 21/100 1/1 [==============================] - 0s 15ms/step - loss: 3.9227 - mae: 3.9227 Epoch 22/100 1/1 [==============================] - 0s 15ms/step - loss: 3.9297 - mae: 3.9297 Epoch 23/100 1/1 [==============================] - 0s 15ms/step - loss: 3.9032 - mae: 3.9032 Epoch 24/100 1/1 [==============================] - 0s 15ms/step - loss: 3.9448 - mae: 3.9448 Epoch 25/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8772 - mae: 3.8772 Epoch 26/100 1/1 [==============================] - 0s 15ms/step - loss: 3.9524 - mae: 3.9524 Epoch 27/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8832 - mae: 3.8832 Epoch 28/100 1/1 [==============================] - 0s 15ms/step - loss: 3.9267 - mae: 3.9267 Epoch 29/100 1/1 [==============================] - 0s 16ms/step - loss: 3.8907 - mae: 3.8907 Epoch 30/100 1/1 [==============================] - 0s 15ms/step - loss: 3.9005 - mae: 3.9005 Epoch 31/100 1/1 [==============================] - 0s 15ms/step - loss: 3.9023 - mae: 3.9023 Epoch 32/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8811 - mae: 3.8811 Epoch 33/100 1/1 [==============================] - 0s 15ms/step - loss: 3.9136 - mae: 3.9136 Epoch 34/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8545 - mae: 3.8545 Epoch 35/100 1/1 [==============================] - 0s 15ms/step - loss: 3.9213 - mae: 3.9213 Epoch 36/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8525 - mae: 3.8525 Epoch 37/100 1/1 [==============================] - 0s 16ms/step - loss: 3.9035 - mae: 3.9035 Epoch 38/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8602 - mae: 3.8602 Epoch 39/100 1/1 [==============================] - 0s 14ms/step - loss: 3.8790 - mae: 3.8790 Epoch 40/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8756 - mae: 3.8756 Epoch 41/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8573 - mae: 3.8573 Epoch 42/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8834 - mae: 3.8834 Epoch 43/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8305 - mae: 3.8305 Epoch 44/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8914 - mae: 3.8914 Epoch 45/100 1/1 [==============================] - 0s 14ms/step - loss: 3.8230 - mae: 3.8230 Epoch 46/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8791 - mae: 3.8791 Epoch 47/100 1/1 [==============================] - 0s 14ms/step - loss: 3.8309 - mae: 3.8309 Epoch 48/100 1/1 [==============================] - 0s 14ms/step - loss: 3.8591 - mae: 3.8591 Epoch 49/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8464 - mae: 3.8464 Epoch 50/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8323 - mae: 3.8323 Epoch 51/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8545 - mae: 3.8545 Epoch 52/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8052 - mae: 3.8052 Epoch 53/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8626 - mae: 3.8626 Epoch 54/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7946 - mae: 3.7946 Epoch 55/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8534 - mae: 3.8534 Epoch 56/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8065 - mae: 3.8065 Epoch 57/100 1/1 [==============================] - 0s 17ms/step - loss: 3.8333 - mae: 3.8333 Epoch 58/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8183 - mae: 3.8183 Epoch 59/100 1/1 [==============================] - 0s 16ms/step - loss: 3.8060 - mae: 3.8060 Epoch 60/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8266 - mae: 3.8266 Epoch 61/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7786 - mae: 3.7786 Epoch 62/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8350 - mae: 3.8350 Epoch 63/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7672 - mae: 3.7672 Epoch 64/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8278 - mae: 3.8278 Epoch 65/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7830 - mae: 3.7830 Epoch 66/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8059 - mae: 3.8059 Epoch 67/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7913 - mae: 3.7913 Epoch 68/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7784 - mae: 3.7784 Epoch 69/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7998 - mae: 3.7998 Epoch 70/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7507 - mae: 3.7507 Epoch 71/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8084 - mae: 3.8084 Epoch 72/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7408 - mae: 3.7408 Epoch 73/100 1/1 [==============================] - 0s 15ms/step - loss: 3.8039 - mae: 3.8039 Epoch 74/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7568 - mae: 3.7568 Epoch 75/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7773 - mae: 3.7773 Epoch 76/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7654 - mae: 3.7654 Epoch 77/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7495 - mae: 3.7495 Epoch 78/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7740 - mae: 3.7740 Epoch 79/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7214 - mae: 3.7214 Epoch 80/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7828 - mae: 3.7828 Epoch 81/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7180 - mae: 3.7180 Epoch 82/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7754 - mae: 3.7754 Epoch 83/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7317 - mae: 3.7317 Epoch 84/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7473 - mae: 3.7473 Epoch 85/100 1/1 [==============================] - 0s 14ms/step - loss: 3.7404 - mae: 3.7404 Epoch 86/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7192 - mae: 3.7192 Epoch 87/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7493 - mae: 3.7493 Epoch 88/100 1/1 [==============================] - 0s 15ms/step - loss: 3.6910 - mae: 3.6910 Epoch 89/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7582 - mae: 3.7582 Epoch 90/100 1/1 [==============================] - 0s 15ms/step - loss: 3.6976 - mae: 3.6976 Epoch 91/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7447 - mae: 3.7447 Epoch 92/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7073 - mae: 3.7073 Epoch 93/100 1/1 [==============================] - 0s 14ms/step - loss: 3.7163 - mae: 3.7163 Epoch 94/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7162 - mae: 3.7162 Epoch 95/100 1/1 [==============================] - 0s 15ms/step - loss: 3.6878 - mae: 3.6878 Epoch 96/100 1/1 [==============================] - 0s 14ms/step - loss: 3.7253 - mae: 3.7253 Epoch 97/100 1/1 [==============================] - 0s 15ms/step - loss: 3.6593 - mae: 3.6593 Epoch 98/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7376 - mae: 3.7376 Epoch 99/100 1/1 [==============================] - 0s 15ms/step - loss: 3.6749 - mae: 3.6749 Epoch 100/100 1/1 [==============================] - 0s 15ms/step - loss: 3.7126 - mae: 3.7126
Out [48]:
<keras.src.callbacks.History at 0x7f291fd190>
In [49]:
# Try and predict what y would be if X was 17.0
modelThree.predict([17.0]) # the right answer is 27.0 (y = X + 10)1/1 [==============================] - 0s 223ms/step
Out [49]:
array([[32.356853]], dtype=float32)
hmm... the output is 32. The goal is 27. This is WORSE than the previous model with less layers.
This seems to be a crux of machine-learning: experimentation.
Not sure how much it relates to this "learn-by-building" approach or not.
Improve the model: activation, optimization, and learning-rate
In [50]:
# activation=None
# optimizers.Adam()
# CREATE & COMPILE & FITthe model
mFour = tf.keras.Sequential()
m4L1 = tf.keras.layers.Dense(50, activation=None)
m4L2 = tf.keras.layers.Dense(1)
mFour.add(m4L1)
mFour.add(m4L2)
mFour.compile(loss=tf.keras.losses.mae,
optimizer=tf.keras.optimizers.Adam(learning_rate=0.01),
metrics=["mae"])
mFour.fit(tf.expand_dims(indVar, axis=-1), depVar, epochs=100)Epoch 1/100 1/1 [==============================] - 2s 2s/step - loss: 13.9719 - mae: 13.9719 Epoch 2/100 1/1 [==============================] - 0s 16ms/step - loss: 13.3078 - mae: 13.3078 Epoch 3/100 1/1 [==============================] - 0s 17ms/step - loss: 12.6463 - mae: 12.6463 Epoch 4/100 1/1 [==============================] - 0s 15ms/step - loss: 11.9838 - mae: 11.9838 Epoch 5/100 1/1 [==============================] - 0s 15ms/step - loss: 11.3169 - mae: 11.3169 Epoch 6/100 1/1 [==============================] - 0s 15ms/step - loss: 10.6430 - mae: 10.6430 Epoch 7/100 1/1 [==============================] - 0s 15ms/step - loss: 9.9593 - mae: 9.9593 Epoch 8/100 1/1 [==============================] - 0s 15ms/step - loss: 9.2629 - mae: 9.2629 Epoch 9/100 1/1 [==============================] - 0s 15ms/step - loss: 8.5505 - mae: 8.5505 Epoch 10/100 1/1 [==============================] - 0s 15ms/step - loss: 7.8185 - mae: 7.8185 Epoch 11/100 1/1 [==============================] - 0s 15ms/step - loss: 7.0638 - mae: 7.0638 Epoch 12/100 1/1 [==============================] - 0s 17ms/step - loss: 6.7142 - mae: 6.7142 Epoch 13/100 1/1 [==============================] - 0s 18ms/step - loss: 6.9971 - mae: 6.9971 Epoch 14/100 1/1 [==============================] - 0s 16ms/step - loss: 7.2948 - mae: 7.2948 Epoch 15/100 1/1 [==============================] - 0s 15ms/step - loss: 7.6387 - mae: 7.6387 Epoch 16/100 1/1 [==============================] - 0s 15ms/step - loss: 7.7572 - mae: 7.7572 Epoch 17/100 1/1 [==============================] - 0s 15ms/step - loss: 7.6730 - mae: 7.6730 Epoch 18/100 1/1 [==============================] - 0s 15ms/step - loss: 7.4355 - mae: 7.4355 Epoch 19/100 1/1 [==============================] - 0s 15ms/step - loss: 7.0819 - mae: 7.0819 Epoch 20/100 1/1 [==============================] - 0s 19ms/step - loss: 6.7707 - mae: 6.7707 Epoch 21/100 1/1 [==============================] - 0s 15ms/step - loss: 6.4845 - mae: 6.4845 Epoch 22/100 1/1 [==============================] - 0s 15ms/step - loss: 6.1899 - mae: 6.1899 Epoch 23/100 1/1 [==============================] - 0s 18ms/step - loss: 6.0660 - mae: 6.0660 Epoch 24/100 1/1 [==============================] - 0s 15ms/step - loss: 6.0265 - mae: 6.0265 Epoch 25/100 1/1 [==============================] - 0s 15ms/step - loss: 6.2063 - mae: 6.2063 Epoch 26/100 1/1 [==============================] - 0s 15ms/step - loss: 6.3025 - mae: 6.3025 Epoch 27/100 1/1 [==============================] - 0s 15ms/step - loss: 6.2996 - mae: 6.2996 Epoch 28/100 1/1 [==============================] - 0s 15ms/step - loss: 6.2099 - mae: 6.2099 Epoch 29/100 1/1 [==============================] - 0s 15ms/step - loss: 6.0424 - mae: 6.0424 Epoch 30/100 1/1 [==============================] - 0s 15ms/step - loss: 5.8039 - mae: 5.8039 Epoch 31/100 1/1 [==============================] - 0s 15ms/step - loss: 5.5068 - mae: 5.5068 Epoch 32/100 1/1 [==============================] - 0s 15ms/step - loss: 5.4012 - mae: 5.4012 Epoch 33/100 1/1 [==============================] - 0s 15ms/step - loss: 5.2927 - mae: 5.2927 Epoch 34/100 1/1 [==============================] - 0s 16ms/step - loss: 5.2614 - mae: 5.2614 Epoch 35/100 1/1 [==============================] - 0s 18ms/step - loss: 5.2559 - mae: 5.2559 Epoch 36/100 1/1 [==============================] - 0s 15ms/step - loss: 5.2080 - mae: 5.2080 Epoch 37/100 1/1 [==============================] - 0s 15ms/step - loss: 5.1211 - mae: 5.1211 Epoch 38/100 1/1 [==============================] - 0s 15ms/step - loss: 4.9985 - mae: 4.9985 Epoch 39/100 1/1 [==============================] - 0s 15ms/step - loss: 4.8433 - mae: 4.8433 Epoch 40/100 1/1 [==============================] - 0s 15ms/step - loss: 4.6586 - mae: 4.6586 Epoch 41/100 1/1 [==============================] - 0s 15ms/step - loss: 4.5234 - mae: 4.5234 Epoch 42/100 1/1 [==============================] - 0s 15ms/step - loss: 4.4325 - mae: 4.4325 Epoch 43/100 1/1 [==============================] - 0s 15ms/step - loss: 4.3340 - mae: 4.3340 Epoch 44/100 1/1 [==============================] - 0s 15ms/step - loss: 4.2826 - mae: 4.2826 Epoch 45/100 1/1 [==============================] - 0s 15ms/step - loss: 4.1589 - mae: 4.1589 Epoch 46/100 1/1 [==============================] - 0s 15ms/step - loss: 3.9608 - mae: 3.9608 Epoch 47/100 1/1 [==============================] - 0s 16ms/step - loss: 3.8120 - mae: 3.8120 Epoch 48/100 1/1 [==============================] - 0s 15ms/step - loss: 3.6564 - mae: 3.6564 Epoch 49/100 1/1 [==============================] - 0s 15ms/step - loss: 3.5064 - mae: 3.5064 Epoch 50/100 1/1 [==============================] - 0s 15ms/step - loss: 3.3872 - mae: 3.3872 Epoch 51/100 1/1 [==============================] - 0s 15ms/step - loss: 3.2206 - mae: 3.2206 Epoch 52/100 1/1 [==============================] - 0s 15ms/step - loss: 3.0274 - mae: 3.0274 Epoch 53/100 1/1 [==============================] - 0s 16ms/step - loss: 2.8648 - mae: 2.8648 Epoch 54/100 1/1 [==============================] - 0s 15ms/step - loss: 2.6918 - mae: 2.6918 Epoch 55/100 1/1 [==============================] - 0s 15ms/step - loss: 2.5084 - mae: 2.5084 Epoch 56/100 1/1 [==============================] - 0s 15ms/step - loss: 2.3143 - mae: 2.3143 Epoch 57/100 1/1 [==============================] - 0s 15ms/step - loss: 2.1095 - mae: 2.1095 Epoch 58/100 1/1 [==============================] - 0s 15ms/step - loss: 1.8937 - mae: 1.8937 Epoch 59/100 1/1 [==============================] - 0s 17ms/step - loss: 1.6668 - mae: 1.6668 Epoch 60/100 1/1 [==============================] - 0s 15ms/step - loss: 1.4285 - mae: 1.4285 Epoch 61/100 1/1 [==============================] - 0s 17ms/step - loss: 1.2146 - mae: 1.2146 Epoch 62/100 1/1 [==============================] - 0s 15ms/step - loss: 0.9407 - mae: 0.9407 Epoch 63/100 1/1 [==============================] - 0s 15ms/step - loss: 0.7351 - mae: 0.7351 Epoch 64/100 1/1 [==============================] - 0s 15ms/step - loss: 0.4361 - mae: 0.4361 Epoch 65/100 1/1 [==============================] - 0s 16ms/step - loss: 0.3288 - mae: 0.3288 Epoch 66/100 1/1 [==============================] - 0s 15ms/step - loss: 0.2512 - mae: 0.2512 Epoch 67/100 1/1 [==============================] - 0s 15ms/step - loss: 0.4235 - mae: 0.4235 Epoch 68/100 1/1 [==============================] - 0s 15ms/step - loss: 0.5682 - mae: 0.5682 Epoch 69/100 1/1 [==============================] - 0s 15ms/step - loss: 0.6563 - mae: 0.6563 Epoch 70/100 1/1 [==============================] - 0s 15ms/step - loss: 0.7068 - mae: 0.7068 Epoch 71/100 1/1 [==============================] - 0s 15ms/step - loss: 0.7562 - mae: 0.7562 Epoch 72/100 1/1 [==============================] - 0s 15ms/step - loss: 0.7907 - mae: 0.7907 Epoch 73/100 1/1 [==============================] - 0s 15ms/step - loss: 0.6607 - mae: 0.6607 Epoch 74/100 1/1 [==============================] - 0s 15ms/step - loss: 0.6651 - mae: 0.6651 Epoch 75/100 1/1 [==============================] - 0s 15ms/step - loss: 0.5813 - mae: 0.5813 Epoch 76/100 1/1 [==============================] - 0s 15ms/step - loss: 0.3394 - mae: 0.3394 Epoch 77/100 1/1 [==============================] - 0s 15ms/step - loss: 0.3367 - mae: 0.3367 Epoch 78/100 1/1 [==============================] - 0s 15ms/step - loss: 0.1364 - mae: 0.1364 Epoch 79/100 1/1 [==============================] - 0s 15ms/step - loss: 0.3209 - mae: 0.3209 Epoch 80/100 1/1 [==============================] - 0s 15ms/step - loss: 0.4119 - mae: 0.4119 Epoch 81/100 1/1 [==============================] - 0s 15ms/step - loss: 0.3162 - mae: 0.3162 Epoch 82/100 1/1 [==============================] - 0s 15ms/step - loss: 0.4758 - mae: 0.4758 Epoch 83/100 1/1 [==============================] - 0s 16ms/step - loss: 0.4860 - mae: 0.4860 Epoch 84/100 1/1 [==============================] - 0s 14ms/step - loss: 0.3739 - mae: 0.3739 Epoch 85/100 1/1 [==============================] - 0s 14ms/step - loss: 0.4546 - mae: 0.4546 Epoch 86/100 1/1 [==============================] - 0s 15ms/step - loss: 0.3964 - mae: 0.3964 Epoch 87/100 1/1 [==============================] - 0s 15ms/step - loss: 0.2002 - mae: 0.2002 Epoch 88/100 1/1 [==============================] - 0s 15ms/step - loss: 0.2906 - mae: 0.2906 Epoch 89/100 1/1 [==============================] - 0s 15ms/step - loss: 0.2834 - mae: 0.2834 Epoch 90/100 1/1 [==============================] - 0s 15ms/step - loss: 0.0937 - mae: 0.0937 Epoch 91/100 1/1 [==============================] - 0s 15ms/step - loss: 0.1835 - mae: 0.1835 Epoch 92/100 1/1 [==============================] - 0s 15ms/step - loss: 0.1459 - mae: 0.1459 Epoch 93/100 1/1 [==============================] - 0s 16ms/step - loss: 0.1355 - mae: 0.1355 Epoch 94/100 1/1 [==============================] - 0s 15ms/step - loss: 0.1572 - mae: 0.1572 Epoch 95/100 1/1 [==============================] - 0s 15ms/step - loss: 0.1068 - mae: 0.1068 Epoch 96/100 1/1 [==============================] - 0s 15ms/step - loss: 0.2074 - mae: 0.2074 Epoch 97/100 1/1 [==============================] - 0s 14ms/step - loss: 0.2497 - mae: 0.2497 Epoch 98/100 1/1 [==============================] - 0s 15ms/step - loss: 0.1256 - mae: 0.1256 Epoch 99/100 1/1 [==============================] - 0s 15ms/step - loss: 0.4174 - mae: 0.4174 Epoch 100/100 1/1 [==============================] - 0s 15ms/step - loss: 0.4758 - mae: 0.4758
Out [50]:
<keras.src.callbacks.History at 0x7f290b9310>
In [51]:
# Try and predict what y would be if X was 17.0
mFour.predict([17.0]) # the right answer is 27.0 (y = X + 10)1/1 [==============================] - 0s 188ms/step
Out [51]:
array([[27.622854]], dtype=float32)
It guessed 27.62, VERY CLOSE to the goal of 27!