In the previous post we covered the fundamentals needed to understand the code. If you have not yet read it I recommend giving it a read. In this blog post, I will cover how to use the config file to create a dynamic tensorflow model. This post assumes that you know the basics of creating custom models in tensorflow using the keras model class. If not I recommend reading this blog from the tensorflow website. This blog covers only creation of model, you can use the model the same way you use a regular Functional API or Sequential model. We will be using the Functional API from tensorflow. Also, attached is the Github link to the notebook.
Once the above pre-requisites are satisfied lets start with our tutorial. If you recollect in the previous post we had a main attribute “layer_type” and based on the value of this attribute sub-attributes were decided. Let’s use this in our code.
A Tip: Keep an eye on the key names defined here in the code and config dictionary defined in the previous post.
We start by passing our array of python dictionaries to the class in the __init__() function. Once this is done we have to use this information on our code. Let’s start simple and use the first dictionary from our list which defines our Input layer parameters and put this in our Input layer arguments.
def __init__(self, arch_parameters):
"""
Initialize the model and all the necessary variables
"""
super(CustomDynamicModel, self).__init__()
self.model = None
# Create Architecture
ip = tf.keras.layers.Input(shape=(arch_parameters[0]["input_width"],
arch_parameters[0]["input_height"],
arch_parameters[0]["input_channels"]),
name=arch_parameters[0]["layer_namescope"])
Next let’s iterate through the list from the second layer to the second-last layer. In terms of python indexing this means start a for loop with range(1,len(config_array)-1). Now inside our for loop we have to create conditional blocks for each “layer_type” that we are going to use in our model. Here I am using Conv2D, Maxpool2D, ReLu and BatchNormalization. As you can see below for each layer type I have a conditional block and inside each block I define what should be done.
An important note here would be that the parameter names/key names here in the code should match with the key names/attribute names in the config file defined in the previous blog post.
for i, layer in enumerate(arch_parameters[1:-1]):
if layer["layer_type"] == "conv":
if i == 0:
# Just so that we use ip for the first layer instead of x as a calling argument
x = tf.keras.layers.Conv2D(filters=layer["layer_num_filters"],
kernel_size=layer["layer_kernel_size"],
activation=layer["layer_activation"],
strides=layer["layer_strides"],
padding=layer["layer_padding"],
name=layer["layer_namescope"])(ip)
else:
x = tf.keras.layers.Conv2D(filters=layer["layer_num_filters"],
kernel_size=layer["layer_kernel_size"],
activation=layer["layer_activation"],
strides=layer["layer_strides"],
padding=layer["layer_padding"],
name=layer["layer_namescope"])(x)
elif layer["layer_type"] == "maxpool":
x = tf.keras.layers.MaxPool2D(padding=layer["layer_padding"],
pool_size=layer["layer_pool_size"],
name=layer["layer_namescope"])(x)
elif layer["layer_type"] == "relu" :
x = tf.keras.layers.ReLU(name=layer["layer_namescope"])(x)
elif layer["layer_type"] == "BatchNorm":
x = tf.keras.layers.BatchNormalization(name=layer["layer_namescope"])(x)
Once this is done let’s add our final layer in our model. Even here you can add conditional blocks if your architecture can have two different types of layers as your final block. Here I have shown how you can create conditional blocks for Dense and Conv2D layers it is similar to what I have done inside for loop.
if arch_parameters[-1]["layer_type"] == "conv":
out = tf.keras.layers.Conv2D(filters=arch_parameters[-1]["layer_num_filters"],
kernel_size=arch_parameters[-1]["layer_kernel_size"],
activation=arch_parameters[-1]["layer_activation"],
strides=arch_parameters[-1]["layer_strides"],
padding=arch_parameters[-1]["layer_padding"],
name=arch_parameters[-1]["layer_namescope"])(x)
elif arch_parameters[-1]["layer_type"] == "dense":
x = tf.keras.layers.Flatten(name="model_flatten")(x)
out = tf.keras.layers.Dense(units=arch_parameters[-1]["layer_units"],
activation=arch_parameters[-1]["layer_activation"],
name=arch_parameters[-1]["layer_namescope"])(x)
We are almost through just one last step. We have the input variable and the output variable, let’s use that to create our model.
self.model = tf.keras.Model(inputs=ip, outputs=out, name="CustomDynamicModel")
We are done but for ease of understanding let’s bring this all together.
class CustomDynamicModel(tf.keras.Model):
"""
CustomDynamicModel Architecture
"""
def __init__(self, arch_parameters):
"""
Initialize the model and all the necessary variables
"""
super(CustomDynamicModel, self).__init__()
self.model = None
# Create Architecture
ip = tf.keras.layers.Input(shape=(arch_parameters[0]["input_width"],
arch_parameters[0]["input_height"],
arch_parameters[0]["input_channels"]),
name=arch_parameters[0]["layer_namescope"])
for i, layer in enumerate(arch_parameters[1:-1]):
if layer["layer_type"] == "conv":
if i == 0:
# Just so that we use ip for the first layer instead of x as a calling argument
x = tf.keras.layers.Conv2D(filters=layer["layer_num_filters"],
kernel_size=layer["layer_kernel_size"],
activation=layer["layer_activation"],
strides=layer["layer_strides"],
padding=layer["layer_padding"],
name=layer["layer_namescope"])(ip)
else:
x = tf.keras.layers.Conv2D(filters=layer["layer_num_filters"],
kernel_size=layer["layer_kernel_size"],
activation=layer["layer_activation"],
strides=layer["layer_strides"],
padding=layer["layer_padding"],
name=layer["layer_namescope"])(x)
elif layer["layer_type"] == "maxpool":
x = tf.keras.layers.MaxPool2D(padding=layer["layer_padding"],
pool_size=layer["layer_pool_size"],
name=layer["layer_namescope"])(x)
elif layer["layer_type"] == "relu" :
x = tf.keras.layers.ReLU(name=layer["layer_namescope"])(x)
elif layer["layer_type"] == "BatchNorm":
x = tf.keras.layers.BatchNormalization(name=layer["layer_namescope"])(x)
if arch_parameters[-1]["layer_type"] == "conv":
out = tf.keras.layers.Conv2D(filters=arch_parameters[-1]["layer_num_filters"],
kernel_size=arch_parameters[-1]["layer_kernel_size"],
activation=arch_parameters[-1]["layer_activation"],
strides=arch_parameters[-1]["layer_strides"],
padding=arch_parameters[-1]["layer_padding"],
name=arch_parameters[-1]["layer_namescope"])(x)
elif arch_parameters[-1]["layer_type"] == "dense":
x = tf.keras.layers.Flatten(name="model_flatten")(x)
out = tf.keras.layers.Dense(units=arch_parameters[-1]["layer_units"],
activation=arch_parameters[-1]["layer_activation"],
name=arch_parameters[-1]["layer_namescope"])(x)
self.model = tf.keras.Model(inputs=ip, outputs=out, name="CustomDynamicModel")
def call(self, inputs, training=False):
"""
Call the model
"""
return self.model(inputs)
def summary(self):
return self.model.summary()
Voila !! We are done. Now wasn’t that easy. Do you want to see if it works? Alright alright alright.. (A Matthew McConaughey reference) 😛
# Create model
model = CustomDynamicModel(arch_parameters)
# Get summary
model.summary()
Model: "CustomDynamicModel"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
model_ip (InputLayer) [(None, 28, 28, 1)] 0
_________________________________________________________________
model_block_1_conv3_1 (Conv2 (None, 26, 26, 32) 320
_________________________________________________________________
model_block_1_conv3_2 (Conv2 (None, 26, 26, 64) 18496
_________________________________________________________________
model_block_1_max_pool_1 (Ma (None, 13, 13, 64) 0
_________________________________________________________________
model_block_1_BatchNorm_1 (B (None, 13, 13, 64) 256
_________________________________________________________________
model_block_1_conv1_1 (Conv2 (None, 11, 11, 128) 73856
_________________________________________________________________
model_block_1_relu_1 (ReLU) (None, 11, 11, 128) 0
_________________________________________________________________
model_flatten (Flatten) (None, 15488) 0
_________________________________________________________________
model_dense_final (Dense) (None, 10) 154890
=================================================================
Total params: 247,818
Trainable params: 247,690
Non-trainable params: 128
_________________________________________________________________
So, now that you know how to create a dynamic model go ahead use this technique, it is flexible for all kinds of models, you just need to know where and how to tweak it. If you have any difficulties in understanding reach out to me or post it in the comments I would be glad to help you out.
It might seem like a lot of work for a small model and to be honest it is, but for large models this few lines of code turns out to be really helpful. You can also create your own custom layers or custom models and use it here. It is also helpful if in case you need to design a UI which can take input from the user for the model.
If you found this helpful, please like, comment and share it with your friends or people who you think will find this interesting. Comment below how you handle such situations. Let me know if you have any doubts.
In the Github notebook, I have used Jupyter notebook so the the config file is basically another code block, however, one can easily create a config file and then import it just like in a normal python code.
Happy coding !! See you in the next blog post 🙂
Just AI Stuff 
Leave a comment