Find out how to Construct TensorFlow Fashions with the Keras Practical API | by Derrick Mwiti | Nov, 2022

Examples and code for constructing customized fashions with Keras Practical API

The Keras Practical API gives a strategy to construct versatile and complicated neural networks in TensorFlow. The Practical API is used to design networks that aren’t linear. On this article, you’ll uncover that the Keras Practical API is used to create networks that:

• Are non-linear.
• Share layers.
• Have a number of inputs and outputs.

We used the Sequential API within the CNN tutorial to construct a picture classification mannequin with Keras and TensorFlow. The Sequential API includes stacking layers. One layer is adopted by one other layer till the ultimate dense layer. This makes designing networks with the Sequential API straightforward and easy.

parameters = {"form":28, "activation": "relu", "courses": 10, "items":12, "optimizer":"adam", "epochs":1,"kernel_size":3,"pool_size":2, "dropout":0.5}
# Setup the layers
mannequin = keras.Sequential(
[
layers.Conv2D(32, kernel_size=(parameters["kernel_size"], parameters["kernel_size"]), input_shape =(parameters["shape"], parameters["shape"], 1),activation=parameters["activation"]),
layers.MaxPooling2D(pool_size=(parameters["pool_size"], parameters["pool_size"])),
layers.Conv2D(64, kernel_size=(parameters["kernel_size"], parameters["kernel_size"]), activation=parameters["activation"]),
layers.MaxPooling2D(pool_size=(parameters["pool_size"], parameters["pool_size"])),
layers.Flatten(),
layers.Dropout(parameters["dropout"]),
layers.Dense(parameters["classes"], activation="softmax"),
]
)

The Sequential API limits you to at least one enter and one output. Nevertheless, it’s possible you’ll wish to design neural networks with a number of inputs and outputs in sure situations. For instance, given a picture of an individual, you’ll be able to design a community to foretell a number of attributes resembling gender, age, and hair colour. It is a community with one enter however a number of outputs. To attain this, the Sequential API is required. Plotting the community reveals that the layers are organized in a linear method.

keras.utils.plot_model(mannequin, "mannequin.png",show_shapes=True)

Designing Practical fashions is a little bit completely different from creating Sequential fashions. Let’s have a look at these variations.

The primary distinction is the requirement to create an enter layer. With the Sequential API, you don’t should outline the enter layer. Defining the enter form within the first layer is enough.

The inputs layer accommodates the form and sort of information to be handed to the community.

inputs = keras.Enter(form=(parameters["shape"], parameters["shape"], 1))
inputs.form
# TensorShape([None, 28, 28, 1])
inputs.dtype
# tf.float32

The enter layer is outlined with out the batch dimension if the info is one-dimensional.

inputs = keras.Enter(form=(784,))

The subsequent distinction is how layers are linked utilizing the Practical API. To create the connection, we create one other layer and go the inputs layer to it. That is greatest understood by contemplating every of the layers as a operate. Because the layers are capabilities, they are often known as with parameters. For instance, let’s go the inputs to a Conv2D layer.

conv2D = layers.Conv2D(32)
x = conv2D(inputs)
x
# <KerasTensor: form=(None, 26, 26, 32) dtype=float32 (created by layer 'conv2d_7')>

Within the above instance, we create a Conv2D layer, name it as a operate and go the inputs. The ensuing output’s form is completely different from the preliminary inputs form on account of being handed to the convolution layer.

The above instance reveals methods to outline and join the networks verbosely. Nevertheless, the syntax could be simplified. The simplified model appears to be like like this:

conv2D = Conv2d(...) (inputs)

conv2D() is much like conv2D.__call__(self,....). Python objects implement the __call__() methodology. Keras layers additionally implement this methodology. The strategy returns the output given an enter tensor.

inputs = keras.Enter(form=(parameters["shape"], parameters["shape"], 1))
conv2D = layers.Conv2D(32, kernel_size=(parameters["kernel_size"], parameters["kernel_size"]), input_shape =(parameters["shape"], parameters["shape"], 1),activation=parameters["activation"])(inputs)
conv2D
# <KerasTensor: form=(None, 26, 26, 32) dtype=float32 (created by layer 'conv2d_8')>

Let’s add a number of extra layers to the community to display methods to create a Keras mannequin when layers are outlined utilizing the Practical API.

parameters = {"form":28, "activation": "relu", "courses": 10, "items":12, "optimizer":"adam", "epochs":1,"kernel_size":3,"pool_size":2, "dropout":0.5}
inputs = keras.Enter(form=(parameters["shape"], parameters["shape"], 1))
conv2D = layers.Conv2D(32, kernel_size=(parameters["kernel_size"], parameters["kernel_size"]), input_shape =(parameters["shape"], parameters["shape"], 1),activation=parameters["activation"])(inputs)
maxPooling2D = layers.MaxPooling2D(pool_size=(parameters["pool_size"], parameters["pool_size"]))(conv2D)
conv2D_2 =layers.Conv2D(64, kernel_size=(parameters["kernel_size"], parameters["kernel_size"]), activation=parameters["activation"])(maxPooling2D)
maxPooling2D_2 = layers.MaxPooling2D(pool_size=(parameters["pool_size"], parameters["pool_size"]))(conv2D_2)
flatten =   layers.Flatten()(maxPooling2D_2)
dropout = layers.Dropout(parameters["dropout"])(flatten)
ouputs = layers.Dense(parameters["classes"], activation="softmax")(dropout)

A Keras mannequin is created utilizing the keras.Mannequin operate whereas passing the inputs and outputs.

mannequin = keras.Mannequin(inputs=inputs, outputs=outputs, identify="mnist_model")

We are able to plot the mannequin to substantiate that it’s much like the one we outlined utilizing the Sequential API.

keras.utils.plot_model(mannequin, "mannequin.png",show_shapes=True)

Coaching and evaluating fashions are the identical within the Practical API and the Sequential API. keras.Mannequin avails the match and consider strategies.

(x_train, y_train), (x_test, y_test) = keras.datasets.mnist.load_data()
x_train = x_train.astype("float32") / 255
x_test = x_test.astype("float32") / 255
mannequin.compile(
loss=keras.losses.SparseCategoricalCrossentropy(from_logits=True),
optimizer=keras.optimizers.RMSprop(),
metrics=["accuracy"],
)
historical past = mannequin.match(x_train, y_train, batch_size=64, epochs=2, validation_split=0.2)
test_scores = mannequin.consider(x_test, y_test, verbose=2)
print("Check loss:", test_scores[0])
print("Check accuracy:", test_scores[1])

Mannequin saving and serialization work the identical within the Practical API and the Sequential API. For example, we will save all the mannequin utilizing mannequin.save().

mannequin.save("saved_model")
del mannequin
mannequin = keras.fashions.load_model("saved_model")
mannequin.abstract()

A Practical mannequin with linear layers could be transformed right into a Sequential mannequin by creating an occasion of Sequential and including the layers.

seq_model = keras.fashions.Sequential()
for layer in mannequin.layers:
seq_model.add(layer)
seq_model.abstract()

Equally, we will convert Sequential networks to Practical fashions.

inputs = keras.Enter(batch_shape=seq_model.layers[0].input_shape)
x = inputs
for layer in seq_model.layers:
x = layer(x) 
outputs = x
func_model = keras.Mannequin(inputs=inputs, outputs=outputs, identify="func_mnist_model")
func_model.abstract()

Let’s have a look at methods to outline normal neural networks utilizing the Practical Keras API.

We begin by defining a neural community with a number of hidden layers and plot the mannequin.

inputs = keras.Enter(form=(parameters["shape"], parameters["shape"], 1))
dense1 = layers.Dense(128)(inputs)
dropout = layers.Dropout(parameters["dropout"])(dense1)
dense2 = layers.Dense(128)(dropout)
dropout1 = layers.Dropout(parameters["dropout"])(dense2)
outputs = layers.Dense(parameters["classes"], activation="softmax")(dropout1)
mannequin = keras.Mannequin(inputs=inputs, outputs=outputs, identify="mnist_model")
keras.utils.plot_model(mannequin, "mannequin.png",show_shapes=True)

Subsequent, we have a look at methods to outline Convolutional Neural Networks utilizing the Practical API. The community has convolution, pooling, flatten, and dense layers.

inputs = keras.Enter(form=(parameters["shape"], parameters["shape"], 1))
conv2D = layers.Conv2D(32, kernel_size=(parameters["kernel_size"], parameters["kernel_size"]), input_shape =(parameters["shape"], parameters["shape"], 1),activation=parameters["activation"])(inputs)
maxPooling2D = layers.MaxPooling2D(pool_size=(parameters["pool_size"], parameters["pool_size"]))(conv2D)
conv2D_2 =layers.Conv2D(64, kernel_size=(parameters["kernel_size"], parameters["kernel_size"]), activation=parameters["activation"])(maxPooling2D)
maxPooling2D_2 = layers.MaxPooling2D(pool_size=(parameters["pool_size"], parameters["pool_size"]))(conv2D_2)
flatten =   layers.Flatten()(maxPooling2D_2)
dropout = layers.Dropout(parameters["dropout"])(flatten)
outputs = layers.Dense(parameters["classes"], activation="softmax")(dropout)
mannequin = keras.Mannequin(inputs=inputs, outputs=outputs, identify="mnist_model")
keras.utils.plot_model(mannequin, "mannequin.png",show_shapes=True)

Let’s have a look at the definition of a bidirectional LSTM utilizing the Practical API. The community accommodates an Embedding layer.

inputs = keras.Enter(784,)
embedding = layers.Embedding(512, 64, input_length=1024)(inputs)
bidirectional1 = layers.Bidirectional(layers.LSTM(64, return_sequences=True))(embedding)
bidirectional2 = layers.Bidirectional(layers.LSTM(64,))(bidirectional1)
dense1 = layers.Dense(32, activation='relu')(bidirectional2)
outputs = layers.Dense(1, activation='sigmoid')(dense1)
mannequin = keras.Mannequin(inputs=inputs, outputs=outputs, identify="lstm_model")
keras.utils.plot_model(mannequin, "mannequin.png",show_shapes=True)

Defining layers with the Practical API permits the creation of networks that share sure layers. Shared layers are used a number of instances in a community.

This instance defines a CNN with one enter layer shared by two convolution blocks. We then be part of the outputs from these blocks utilizing the concatenate layer. After that, we go the end result to a DropOut layer and at last to totally linked layer.

inputs = keras.Enter(form=(parameters["shape"], parameters["shape"], 1))
conv2D = layers.Conv2D(32, kernel_size=(parameters["kernel_size"], parameters["kernel_size"]), input_shape =(parameters["shape"], parameters["shape"], 1),activation=parameters["activation"])(inputs)
maxPooling2D = layers.MaxPooling2D(pool_size=(parameters["pool_size"], parameters["pool_size"]))(conv2D)
flatten1 =   layers.Flatten()(maxPooling2D)
conv2D_2 = layers.Conv2D(64, kernel_size=(parameters["kernel_size"], parameters["kernel_size"]), activation=parameters["activation"])(inputs)
maxPooling2D_2 = layers.MaxPooling2D(pool_size=(parameters["pool_size"], parameters["pool_size"]))(conv2D_2)
flatten2 =   layers.Flatten()(maxPooling2D_2)
# merge layers
merged_layers = layers.concatenate([flatten1, flatten2])
dropout = layers.Dropout(parameters["dropout"])(merged_layers)
outputs = layers.Dense(parameters["classes"], activation="softmax")(dropout)
mannequin = keras.Mannequin(inputs=inputs, outputs=outputs, identify="mnist_model")
keras.utils.plot_model(mannequin, "mannequin.png",show_shapes=True)

Plotting the community reveals the connection between the completely different layers.

On this instance, we create an embedding layer shared by two bidirectional LSTMs. A shared characteristic extraction layer permits sharing the identical characteristic extractor a number of instances within the community. For instance, sharing this data between two inputs could make it doable to coach a community with much less information.

inputs = keras.Enter(784,)
embedding = layers.Embedding(512, 64, input_length=1024)(inputs)
bidirectional1 = layers.Bidirectional(layers.LSTM(64, return_sequences=True))(embedding)
bidirectional2 = layers.Bidirectional(layers.LSTM(64, return_sequences=True))(embedding)
# merge layers
merged_layers = layers.concatenate([bidirectional1, bidirectional2])
dense1 = layers.Dense(32, activation='relu')(merged_layers)
outputs = layers.Dense(1, activation='sigmoid')(dense1)
mannequin = keras.Mannequin(inputs=inputs, outputs=outputs, identify="lstm_model")
keras.utils.plot_model(mannequin, "mannequin.png",show_shapes=True)

Subsequent, let’s focus on the a number of inputs and outputs state of affairs.

Networks with a number of inputs and outputs can be outlined utilizing the Practical API. This isn’t doable with the Sequential API.

On this instance, we outline a community that takes two inputs of various lengths. We go the inputs to dense layers and sum them utilizing the add layer.

input1 = keras.Enter(form=(16,))
x1 =layers.Dense(8, activation='relu')(input1)
input2 = layers.Enter(form=(32,))
x2 = layers.Dense(8, activation='relu')(input2)
# equal to `added = tf.keras.layers.add([x1, x2])`
added = layers.Add()([x1, x2])
out = layers.Dense(4)(added)
mannequin = keras.Mannequin(inputs=[input1, input2], outputs=out)
keras.utils.plot_model(mannequin, "mannequin.png",show_shapes=True)

The Practical API permits the definition of fashions with a number of outputs. The instance beneath defines a convolutional neural community with two output layers. For example, given a picture of an individual, this community can predict gender and hair colour.

image_input = keras.Enter(form=(parameters["shape"], parameters["shape"], 3), identify="photos") 
x = layers.Conv2D(filters=32,kernel_size=(3,3),activation='relu')(image_input)
x = layers.MaxPooling2D(pool_size=(2,2))(x)
x = layers.Conv2D(filters=32,kernel_size=(3,3), activation='relu')(x)
x = layers.Dropout(0.25)(x)
x = layers.Conv2D(filters=64,kernel_size=(3,3), activation='relu')(x)
x = layers.MaxPooling2D(pool_size=(2,2))(x)
x = layers.Dropout(0.25)(x)
x = layers.Flatten()(x)
x = layers.Dense(128, activation='relu')(x)
x = layers.Dropout(0.25)(x)
gender_prediction = layers.Dense(3, activation='softmax')(x)
age_prediction = layers.Dense(3, activation='softmax')(x)
mannequin = keras.Mannequin(
inputs=image_input,
outputs=[gender_prediction, age_prediction],
)
keras.utils.plot_model(mannequin, "mannequin.png",show_shapes=True)

The Practical API additionally permits the definition of a number of fashions utilizing the identical layers. That is doable as a result of making a mannequin utilizing the Practical API requires solely the inputs and outputs. For instance, that is relevant within the encode decoder community structure.

encoder_input = keras.Enter(form=(28, 28, 1), identify="img")
x = layers.Conv2D(16, 3, activation="relu")(encoder_input)
x = layers.Conv2D(32, 3, activation="relu")(x)
x = layers.MaxPooling2D(3)(x)
x = layers.Conv2D(32, 3, activation="relu")(x)
x = layers.Conv2D(16, 3, activation="relu")(x)
encoder_output = layers.GlobalMaxPooling2D()(x)
encoder = keras.Mannequin(encoder_input, encoder_output, identify="encoder")
encoder.abstract()
x = layers.Reshape((4, 4, 1))(encoder_output)
x = layers.Conv2DTranspose(16, 3, activation="relu")(x)
x = layers.Conv2DTranspose(32, 3, activation="relu")(x)
x = layers.UpSampling2D(3)(x)
x = layers.Conv2DTranspose(16, 3, activation="relu")(x)
decoder_output = layers.Conv2DTranspose(1, 3, activation="relu")(x)
autoencoder = keras.Mannequin(encoder_input, decoder_output, identify="autoencoder")
autoencoder.abstract()

Keras Practical API is useful when designing non-linear networks. In case you suppose that you could be have to convert a community to a non-linear construction, then you must use the Practical API. A number of the strengths of the Practical API embody:

• It’s much less verbose in comparison with subclassing the Mannequin class.
• The requirement to create an Enter ensures that all Practical networks will run as a result of passing the incorrect form results in an instantaneous error.
• A Practical mannequin is simpler to plot and examine.
• Simple to serialize and save Practical fashions as a result of they’re information buildings.

Nevertheless, one downside of utilizing the Practical API is that it doesn’t help dynamic architectures resembling recursive networks or Tree RNNs.

Maintain greatest practices in thoughts whereas working with the Keras Practical API:

• At all times print a abstract of the community to affirm that the shapes of the varied layers are as anticipated.
• Plot the community to make sure the layers are linked as you anticipate them.
• Title the layers to make it straightforward to establish them within the community plot and abstract. For instance Conv2D(...,identify="first_conv_layer").
• Use variable names which can be associated to the layers, for instance, conv1 and conv2 for convolution layers. This may make clear the kind of layer when inspecting plots and community abstract.
• As an alternative of making a customized coaching loop, use the keras.Mannequin to create fashions as a result of it makes it simpler to coach fashions by way of the match methodology and consider them with the evalaute methodology.