Modeling With Transformations Notebook
Use column transformers, one-hot encoding, normalization, and model comparison in a practical machine learning workflow.
Summary
The make column transformer (docs) function can take a list of transformer functions along with a list of columns to apply the transformer to. This creates a transformer instance. The transformer instance, then, gets fitted to the data with the fit method. The transformer instnace, then, gets used with the data with the transform method.
Notebook Highlights
Transform
The make column transformer (docs) function can take a list of transformer functions along with a list of columns to apply the transformer to. This creates a transformer instance. The transformer instance, then, gets...
Build A Model
This will be based on the insurance model 2 model that can be found in the modeling and wrangling notebook. This model version, though, will use normalized data: one hot encoded column values and scaled column values.
Table Of Contents
- Modeling With Transformations
- Import Some Data
- Split into Training & Testing Data
- Transform
- Normalize
- Compare normalized vs non-normalized
- Build A Model
- Review The Model
- Experiment With The Model
- Double The Epochs
- Review The Model
- Add A Layer, Change Layer Values
- Review The Model
- Change the Learning Rate less epochs
- Review The Model
- Change the Learning Rate Again
- Review The Model
Modeling With Transformations
import tensorflow as tf
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from sklearn.compose import make_column_transformer
from sklearn.preprocessing import MinMaxScaler, OneHotEncoder
from sklearn.model_selection import train_test_splitImport Some Data
dataUrl = "https://raw.githubusercontent.com/stedy/Machine-Learning-with-R-datasets/master/insurance.csv"
dataFromWeb = pd.read_csv(dataUrl)
dataFromWeb.head()labelField = 'charges'
featureData = dataFromWeb.drop(labelField, axis=1)
labelData = dataFromWeb[labelField]Split into Training & Testing Data
testDataPercentage = .2 # how much of our data should we use for "testing"
randomVal = 42
feature_training_data, feature_testing_data, label_training_data, label_testing_data = train_test_split(featureData,
labelData,
test_size=testDataPercentage,
random_state=randomVal) # set random state for reproducible splitsTransform
The make_column_transformer (docs) function can take a list of transformer functions along with a list of columns to apply the transformer to. This creates a transformer instance.
The transformer instance, then, gets fitted to the data with the fit method.
The transformer instnace, then, gets used with the data with the transform method.
Here will be applied two transformers:
dataTransformer = make_column_transformer(
# get all values between 0 and 1
(MinMaxScaler(), ["age", "bmi", "children"]),
(OneHotEncoder(handle_unknown="ignore"), ["sex", "smoker", "region"])
)
dataTransformer.fit(feature_training_data)Normalize
normailized_feature_training_data = dataTransformer.transform(feature_training_data)
normailized_feature_testing_data = dataTransformer.transform(feature_testing_data)Compare normalized vs non-normalized
normailized_feature_training_data[0]feature_training_data.loc[0]normailized_feature_training_data.shapefeature_training_data.shapeBuild A Model
This will be based on the insurance_model_2 model that can be found in the modeling-and-wrangling notebook.
This model version, though, will use normalized data: one-hot-encoded column values and scaled column values.
tf.random.set_seed(42)
m = tf.keras.Sequential()
epochs = 100
# different & more layers
l1 = tf.keras.layers.Dense(100)
l2 = tf.keras.layers.Dense(10)
l3 = tf.keras.layers.Dense(1)
m.add(l1)
m.add(l2)
m.add(l3)
# Compile the model
m.compile(loss=tf.keras.losses.mae,
optimizer=tf.keras.optimizers.Adam(),
metrics=['mae'])
# Fit the model and save the history (we can plot this)
m_history = m.fit(normailized_feature_training_data, label_training_data, epochs=epochs, verbose=0)Review The Model
m.summary()m.evaluate(normailized_feature_testing_data,label_testing_data)print(f'Training Label Median: {label_training_data.median()}')
print(f'Training Label Mean: {label_training_data.mean()}')
print(f'm MAE: {m.get_metrics_result()["mae"].numpy()}')Compare this model mae to the insurance_model_2 (im2) model in modeling-and-wrangling:
im2had an MAE of~4700- the new model mae looks to be
~3400
Normalizing this model's data, with one-hot-encoding and scaling, made this model perform better!
Experiment With The Model
Double The Epochs
m2 = tf.keras.Sequential()
m2epochs = 200
m2.add(l1)
m2.add(l2)
m2.add(l3)
# Compile the model
m2.compile(loss=tf.keras.losses.mae,
optimizer=tf.keras.optimizers.Adam(),
metrics=['mae'])
# Fit the model and save the history (we can plot this)
m2_history = m2.fit(normailized_feature_training_data, label_training_data, epochs=m2epochs, verbose=0)Review The Model
m2.summary()m2.evaluate(normailized_feature_testing_data,label_testing_data)print(f'Training Label Median: {label_training_data.median()}')
print(f'Training Label Mean: {label_training_data.mean()}')
print(f'm2 MAE: {m2.get_metrics_result()["mae"].numpy()}')
print(f'SHAPE: {normailized_feature_training_data.shape}')Increasing the Epochs ?slightly? made a positive impact on reducing the mae!
Add A Layer, Change Layer Values
m3 = tf.keras.Sequential()
l4 = tf.keras.layers.Dense(100)
m3.add(l1)
m3.add(l4)
m3.add(l2)
m3.add(l3)
# Compile the model
m3.compile(loss=tf.keras.losses.mae,
optimizer=tf.keras.optimizers.Adam(),
metrics=['mae'])
# Fit the model and save the history (we can plot this)
# , verbose=0
m3_history = m3.fit(normailized_feature_training_data, label_training_data, epochs=m2epochs)Review The Model
m3.summary()m3.evaluate(normailized_feature_testing_data,label_testing_data)print(f'Training Label Median: {label_training_data.median()}')
print(f'Training Label Mean: {label_training_data.mean()}')
print(f'm3 MAE: {m3.get_metrics_result()["mae"].numpy()} vs m2 MAE: {m2.get_metrics_result()["mae"].numpy()}')Adding a layer made the mae roughly the same
Change the Learning Rate less epochs
m4 = tf.keras.Sequential()
# l4 = tf.keras.layers.Dense(100)
m4.add(l1)
# m4.add(l4)
m4.add(l2)
m4.add(l3)
# Compile the model
m4.compile(loss=tf.keras.losses.mae,
optimizer=tf.keras.optimizers.Adam(learning_rate=.008),
metrics=['mae'])
# Fit the model and save the history (we can plot this)
# , verbose=0
m4_history = m4.fit(normailized_feature_training_data, label_training_data, epochs=epochs)Review The Model
m4.summary()m4.evaluate(normailized_feature_testing_data,label_testing_data)print(f'Training Label Median: {label_training_data.median()}')
print(f'Training Label Mean: {label_training_data.mean()}')
print(f'm3 MAE: {m4.get_metrics_result()["mae"].numpy()} vs m2 MAE: {m2.get_metrics_result()["mae"].numpy()}')setting the learning rate to .008, compared to the default .001, made the outcome slightly worse here :/
Change the Learning Rate Again
m5 = tf.keras.Sequential()
# l4 = tf.keras.layers.Dense(100)
m5.add(l1)
# m5.add(l4)
m5.add(l2)
m5.add(l3)
# Compile the model
m5.compile(loss=tf.keras.losses.mae,
optimizer=tf.keras.optimizers.Adam(learning_rate=.01),
metrics=['mae'])
# Fit the model and save the history (we can plot this)
# , verbose=0
m5_history = m5.fit(normailized_feature_training_data, label_training_data, epochs=epochs)Review The Model
m5.summary()m5.evaluate(normailized_feature_testing_data,label_testing_data)print(f'Training Label Median: {label_training_data.median()}')
print(f'Training Label Mean: {label_training_data.mean()}')
print(f'm5 MAE: {m5.get_metrics_result()["mae"].numpy()} vs m2 MAE: {m2.get_metrics_result()["mae"].numpy()}')setting the learning rate to .01, compared to the default .001, made the outcome slightly worse here