Tensorflow weight clustering API – An optimization toolkit for heavy-weight fashions

Tensorflow mannequin optimization is a framework utilized to optimize fashions developed in order that they are often pushed for deployment simply. The TensorFlow mannequin optimization framework mainly goals to condense big and memory-heavy Tensorflow fashions into lighter variations of fashions in order that the fashions developed might be simply built-in into edge units. The burden clustering API is one such use case of the Tensorflow mannequin optimization library which goals to scale back the massive community sizes. On this article allow us to perceive the right way to use the load clustering API  to optimize Tensorflow mannequin efficiency.

Desk of Contents

1. Introduction to TensorFlow mannequin optimization 
2. Advantages of Tensorflow mannequin optimization
3. Condensing deep studying fashions with weight clustering API 
4. Abstract

Introduction to TensorFlow mannequin optimization 

Tensorflow mannequin optimization is likely one of the libraries of Tensorflow which is used to optimize the fashions developed. The optimization library primarily goals to seamlessly combine heavy TensorFlow fashions on edge units with sure reminiscence constraints. Fashions optimized utilizing the TensorFlow optimization library allow us to combine the fashions developed with units with sure {hardware} constraints and in some circumstances the place the units use sure accelerators. 

Advantages of Tensorflow mannequin optimization

Any mannequin developed when optimized is liable to yield higher outcomes and efficiency. So the Tensorflow mannequin optimization is one such framework formulated by Tensorflow that goals to optimize the mannequin to its greatest kind in order that it may be used seamlessly on varied units. So allow us to look into a few of the advantages of Tensorflow mannequin optimization.

• Discount in reminiscence occupancy advantages storing and integrating the fashions on person units simply. Discount in reminiscence additionally facilitates simple downloading of the mannequin.
• The optimized fashions are additionally simple to combine on units with reminiscence constraints together with sooner integration because the fashions will likely be lighter.
• Appreciable discount in latency might be achieved by mannequin optimization and the fashions may even have the flexibility to carry out the required duties rapidly.
• Phenomenal efficiency might be achieved on varied accelerators like TPUs as they’re fabricated to course of sooner for optimized fashions.

Now allow us to look right into a case research on the right way to use the load clustering API for optimizing the Tensorflow fashions.

Condensing deep studying fashions with weight clustering API 

For understanding how the load clustering API optimizes the fashions first allow us to develop a easy Tensorflow mannequin. So right here allow us to use the style MNIST dataset to develop a easy deep studying mannequin.

import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
from tensorflow.keras.layers import Flatten,Dense,Dropout,Conv2D,MaxPooling2D
from tensorflow.keras.fashions import Sequential
from tensorflow.keras.utils import to_categorical
 
%matplotlib inline
from tensorflow.keras.datasets import fashion_mnist
(X_train,Y_train),(X_test,Y_test)=fashion_mnist.load_data()
model1=Sequential()
model1.add(Conv2D(32,kernel_size=2,input_shape=(28,28,1),activation='relu'))
model1.add(MaxPooling2D(pool_size=(2,2)))
model1.add(Conv2D(16,kernel_size=2,activation='relu'))
model1.add(MaxPooling2D(pool_size=(2,2)))
model1.add(Flatten())
model1.add(Dense(125,activation='relu'))
model1.add(Dense(10,activation='softmax'))
model1.compile(loss="sparse_categorical_crossentropy",optimizer="adam",metrics=['accuracy'])

So now we’ve got developed a mannequin with sure layers and suitably compiled with the required parameters. Now allow us to match the mannequin on the cut up information.

model1_fit_res=model1.match(X_train,Y_train,epochs=10,validation_data=(X_test,Y_test))

Now because the mannequin is fitted allow us to attempt to consider sure parameters from the mannequin.

print('Mannequin coaching loss : {} and coaching accuracy is : {}'.format(model1.consider(X_train,Y_train)[0],model1.consider(X_train,Y_train)[1]))

print('Mannequin testing loss : {} and testing accuracy is : {}'.format(model1.consider(X_test,Y_test)[0],model1.consider(X_test,Y_test)[1]))

Till now we’ve got constructed a easy deep studying mannequin and have fitted it in opposition to the info and evaluated sure parameters from it. Now allow us to look into the right way to use the load clustering API of the Tensorflow mannequin optimization library and allow us to attempt to consider for any change in efficiency after utilizing the load clustering API.

Utilizing the load clustering API

Allow us to first set up the library into the working atmosphere and allow us to attempt to import the library into the working atmosphere.

! pip set up -q tensorflow-model-optimization
import tensorflow_model_optimization as tfmot

Now that we’ve got efficiently imported the TensorFlow mannequin optimization library into the working atmosphere allow us to look into the right way to use the load clustering API accordingly.

cluster_weights = tfmot.clustering.keras.cluster_weights
cent_init = tfmot.clustering.keras.CentroidInitialization

Right here the cluster weights inbuilt perform of the clustering library is used for creating cluster weights and centroid initialization is taken up utilizing the Centroid Initialization perform of clustering. Now allow us to present some random parameters for the variety of clusters and declare the centroids accordingly. Utilizing the clustered parameters a clustered mannequin will likely be created as proven under.

clustering_params = {
 'number_of_clusters': 32,
 'cluster_centroids_init': cent_init.LINEAR
}

### Clustering the unique mannequin
cluster_model = cluster_weights(model1, **clustering_params)

Now allow us to compile the mannequin accordingly and match it in opposition to the cut up information and consider sure parameters from the clustered mannequin fitted.

### Optimizing the clustered mannequin with adam optimizer and random studying price
from tensorflow.keras import optimizers
optim = tf.keras.optimizers.Adam(learning_rate=1e-5)
cluster_model.compile(loss="sparse_categorical_crossentropy",optimizer=optim,metrics=['accuracy'])
cluster_mod_res=cluster_model.match(X_train,Y_train,epochs=10,validation_data=(X_test,Y_test))

print('Mannequin coaching loss : {} and coaching accuracy is : {}'.format(cluster_model.consider(X_train,Y_train)[0],cluster_model.consider(X_train,Y_train)[1]))

print('Mannequin testing loss : {} and testing accuracy is : {}'.format(cluster_model.consider(X_test,Y_test)[0],cluster_model.consider(X_test,Y_test)[1]))

Right here we are able to see that the mannequin efficiency after clustering has seen an almost 0.5% improve within the take a look at accuracy. An essential level to notice is that the mannequin efficiency will not be decreased after utilizing the clustering API. So now allow us to perceive what really is the advantages of utilizing the load clustering API of the Tensorflow mannequin optimization library.

Allow us to save each the unique deep studying mannequin and the load clustered mannequin and allow us to attempt to acquire their reminiscence occupancies within the working drive in type of bytes.

import tempfile
_, keras_file = tempfile.mkstemp('.h5')
print('Saving mannequin to: ', keras_file)
tf.keras.fashions.save_model(model1, keras_file, include_optimizer=False)
final_clus_model = tfmot.clustering.keras.strip_clustering(cluster_model)

_, clustered_keras_file = tempfile.mkstemp('.h5')
print('Saving clustered mannequin to: ', clustered_keras_file)
tf.keras.fashions.save_model(final_clus_model, clustered_keras_file,
                          include_optimizer=False)

Now allow us to create a user-defined perform that considers the mannequin developed as a parameter and is answerable for yielding the reminiscence of the fashions in bytes.

def get_gzipped_model_size(file):
 # It returns the scale of the gzipped mannequin in bytes.
 import os
 import zipfile
 
 _, zipped_file = tempfile.mkstemp('.zip')
 with zipfile.ZipFile(zipped_file, 'w', compression=zipfile.ZIP_DEFLATED) as f:
   f.write(file)
 
 return os.path.getsize(zipped_file)
print("Dimension of gzipped clustered Keras mannequin: %.2f bytes" % (get_gzipped_model_size(clustered_keras_file)))
print("Dimension of gzipped authentic Keras mannequin: %.2f bytes" % (get_gzipped_model_size(keras_file)))

So right here we are able to see that the Weight clustering API has condensed the unique Keras mannequin when it comes to reminiscence occupancy but it surely has not compromised the mannequin efficiency. So that is how the load clustering API will likely be answerable for condensing a big TensorFlow mannequin to lighter fashions in order that it will possibly simply be built-in into edge units as edge units typically function on minimal reminiscence sources.

Abstract

Typically, TensorFlow fashions developed for real-time functions are heavy fashions and so they typically devour extra reminiscence. So these big fashions developed can’t be built-in as it’s on edge units or on the person finish. That is the place the Tensorflow mannequin optimization framework assists by optimizing the fashions in varied methods by making them reminiscence pleasant and making it simple to combine on edge units. The fashions parsed by means of this framework reproduce the unique deep studying mannequin as it’s.