Classify Yoga Postures Using Transfer Learning

A beginner’s guide to image classification using transfer learning

Photo by Jose Vazquez on Unsplash

As a part of this article, we will see how to classify yoga posture images using transfer learning. We first start with loading the basic libraries required for the project.

import numpy as np # linear algebra
import pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)
#for visualizations
import seaborn as sns
import matplotlib.pyplot as plt
#for model creation
import tensorflow as tf
from tensorflow.keras import layers
from tensorflow.keras import Model
from tensorflow.keras.preprocessing.image import ImageDataGenerator, img_to_array,load_img
#for going through directories
import os

For the task we are using an image dataset available on Kaggle, through the below code snippet we check the number of classes in the dataset and their names.

# This is the main path to our data set.
root_path = '../input/yoga-posture-dataset'
# The folder names are our Classes
class_names = sorted(os.listdir(root_path))
class_names.remove('Poses.json')
n_classes = len(class_names)
print(f"Total Number of Classes : {n_classes}")
print(f"Classes : \n{class_names}")

Output:

Image by Author

We have a total of 47 classes, let’s take a look at the different images.

train_data_path = '../input/yoga-posture-dataset'
def plot_images(path):
    count = 1
    fig = plt.figure(figsize=(10, 10))
    for i in os.listdir(path):
        if count < 26:
            f_path = os.path.join(path,i)
            fig.add_subplot(5,5,count)
            plt.imshow(load_img(f"{os.path.join(f_path,os.listdir(f_path)[0])}"))
            plt.axis('off')
            plt.title(f'{i}')
            count+=1
    plt.show()
plot_images(train_data_path)

Output:

Image by Author

Explanation of code:

To view the different images from various classes, we have defined a function plot_images. Within it, we use the os library from python to view the subdirectories or look up all files within a directory.

os.path_join: would concatenate the two path strings passed to it.

os.listdir: would list all subdirectories in a given path.

As we are dealing with images we have also used TensorFlow's methods load_img and img_to_array as below:

f_path = "../input/yoga-posture-dataset/Adho Mukha Svanasana"
sample_image  = load_img(f"{os.path.join(f_path, os.listdir(f_path)[0])}")
# Convert the image into its numpy array representation
sample_array = img_to_array(sample_image)
print(f"Each image has shape: {sample_array.shape}")

We have RGB images of different sizes, we would need to resize the images before using them. We would do the same using ImageDataGenerator.

#We have added image augmentation to the data
train_datagen = ImageDataGenerator(rescale=1.0/255.,validation_split=0.2
                                  ,width_shift_range=0.2,
                                     height_shift_range=0.2,
                                     shear_range=.2,
                                     zoom_range=.2,
                                     horizontal_flip=True,
                                  fill_mode='nearest')
train_generator = train_datagen.flow_from_directory(directory=train_data_path,
batch_size=32, 
class_mode='categorical',
target_size=(150, 150),
subset = 'training')
validation_generator = train_datagen.flow_from_directory(directory=train_data_path,
batch_size=32, 
class_mode='categorical',
target_size=(150, 150),
subset='validation')

Code Explanation:

We create an object of ImageDataGenerator to generate batches of the images and augment the data using horizontal_flip, zoom, shear range, and so on. Tensorflow has an amazing article on the same, in case you would want to have a deeper look. We also resized the images to be of shape (150,150,3).

We then, load existing pre-trained weights for our transfer learning model which is an Inception V3 using wget and initialize the model using the pre-trained weights. The below code snippet works on the same.

!wget --no-check-certificate \
    https://storage.googleapis.com/mledu-datasets/inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5 \
    -O /tmp/inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5
from tensorflow.keras.applications.inception_v3 import InceptionV3
# Create an instance of the inception model from the local pre-trained weights
local_weights_file = '/tmp/inception_v3_weights_tf_dim_ordering_tf_kernels_notop.h5'
pre_trained = InceptionV3(input_shape=(150,150,3),
                         include_top=False,
                         weights=None)
pre_trained.load_weights(local_weights_file)

We would only add a classification head at the top of the InceptionV3 model and not train the entire InceptionV3 from scratch, the reason being we do not have the data nor the computational resources required for the training of such a complex model. Using the below code statement we turn off the training of the Inception V3 layers.

for layers in pre_trained.layers:
    layers.trainable = False

Next, we have to add a classification head customized for our task at hand, meaning it would classify the data into the 47 classes which we have in our data.

last_desired_layer = pre_trained.get_layer('mixed7')
print('last layer output shape: ', last_desired_layer.output_shape)
last_output = last_desired_layer.output
print('last layer output: ', last_output)
x = tf.keras.layers.Flatten()(last_output)
x = tf.keras.layers.Dense(1024,activation='relu')(x)
x = tf.keras.layers.Dropout(0.2)(x)
x = tf.keras.layers.Dense(47,activation='softmax')(x)        
model = Model(inputs=pre_trained.input, outputs=x)

We then compile and train our model as below.

model.compile(optimizer = 'adam', 
                loss = tf.keras.metrics.categorical_crossentropy,
                metrics = ['accuracy'])
callbacks = MyCallback()
history = model.fit(train_generator,
                    validation_data = validation_generator,
                    epochs = 10,
                    verbose = 2,
                    callbacks=callbacks)

The graph shows the training and validation accuracies.

Image by Author

Concluding Thoughts:

We were able to achieve an accuracy of 70% in just 10 epochs on our validation data, this was possible because we used a pre-trained InceptionV3 network.

We know that building an image classification model from scratch requires huge amounts of data, and transfer learning is a lifesaver in scenarios like our task of building a classifier with a mere 2000 images and 47 image classes.

In case you have any doubts regarding the code explained or you find any mistakes in the writeup/code, please feel free to comment.