A dataset of Sattriya dance: Classical dance of Assam

Sattriya is an Indian classical dance of Assam composed of complex hand gestures, body moments, facial expressions and background music. At a first glance for common people meaning of mudras incorporated in Sattriya dance become difficult to understand. Authors have generated a dataset of Sattriya double handed gestures performed by total fifty artists from different dance academies which can be used to explore ways to build an automated system using machine learning and deep learning algorithms. There are eight double handed gestures or mudras made available through the referred dataset. Mendeley platform is used to store the dataset and samples were collected by visiting some protruding Sattriya dance schools located in Guwahati, Assam. In this article, various tables are used to illustrate variations of the mudras in terms of appearance, title, and compared to other Indian classical dances. To segment and analyze the mudras watershed segmentation is used. In this article researchers have enumerated a few very specific characteristics of Sattriya mudra that may be helpful to academics working in various disciplines of study.


Subject
Artificial Intelligence Specific subject area The dataset of double handed mudras of Sattriya dance can be especially employed in the development of an automatic recognition system that makes use of deep neural networks and machine learning methods.Data format Raw Type of data Image Data collection Digital samples of Sattriya dance mudras were captured with video of at least 20-30 s duration from the source location using DSLR Canon Camera (EOS 1200 D).Videos were recorded with colored background to reduce injected noise during photo capturing and films.This is an easy and feasible method to segregate the raw videos into frames later.At least three to five videos of each mudra were captured from where the best was selected as video sample.

Data source location
Video samples were collected from some renown Sattriya dance academy situated at Guwahati, Assam, India.All these places were visited by the researchers and captured a number of videos for each of the mudras from different angles.Most significant sources of Sattriya dance school contributed to our work are as follows: • Chandamukha Sattriya Dance Academy, Bamunimaidan, Guwahati-21,Assam,India. • Gandharba Kala Kendra,Hengerabari,Guwahati-781036,Assam,India • Mrs. Parismita Bora, Geetanagar, Guwahati-781021,Assam,India.

Value of the Data
It is possible to develop an automatic system from this dataset that may be used to spot and digitally identify a performer's flaws in real-time Sattriya dance, drama (also known as 'Bhaona') and pictures.
In the referred dataset an image set of double handed mudras of Sattriya dance along with their names, three different views of each mudra are uploaded.This will help the researchers to perceive every possible narration of the mudras from three different angles Right, Left and Front.By employing CNN to forecast mudra, this will aid artificial neural networks in their learning process.
The double-handed Sattriya dance sample set aids in comparison research and may assist develop a larger Indian classical dance mudra dataset.

Data Description
The videos of the hand mudras were captured from front side and the camera was rotating 180 from left to right and again from right to left.Table 1 below shows double handed mudras along with their names made available in Mendeley platform with title "Sattriya-08 Double Handed Mudra's Dataset".
In this work we have used several transformations to increase the sample size such that our dataset becomes voluminous and ready for deep learning model.Brightness adjustment, zooming in and out, shifting and rotations were performed as transformation on the sample set.The augmentation performed on rotation parameter is rotation _range = 7.It means during augmentation an image can be randomly rotated between any degree ranged from + 7 to −7.Horizontal and vertical flip was performed on the samples.Vertical flips implies that object's Sea creature Monster and it also describes seacreature .

Samput
Deep thought Anger left side becomes right and vice versa.Similarly horizontal flip means flipping along the horizontal axis.Shear_range = 0.1 was applied which performed shearing of the images within a range of + 0.1 to −0.1 or −10 % to + 10 %.The brightness parameter was augmented as brightness_range = [0.8,1.0], it means that brightness of the samples can be adjusted randomly within the said range.Except for a small number, every mudra displayed in this publication was pre-processed and had back drop removed.Several samples are identified as having body part in the background since they were halted near to the performer's torso and thus making background removal challenging.The eight double handed Sattriya dance mudras supplied as part of our dataset are displayed in Table 1 above.We have included three views of each mudra front, right and left.Each mudra's name, image, meaning and application are shown in Table 1 .

Similarity with other Indian classical dances
A couple of the mudras used in Sattriya dance are unique to it, while others resemble with mudras used in other Indian classical dances (most notably Bharat Natyam).Sattriya dance is profoundly influenced and developed in 'Sattras'(which are residencies for followers of Sankardev's Ekasarana dharma, the religion he founded) [ 1 , 2 ].Hence mudras in Sattriya dance depicts characters from Hindu myths basically Lord Bishnu.Most of the mudras used here resembles with mudras of Odissi, Bharat Natyam and Kathakali.Some of them are comparable in appearance but with different identification.On the other hand some are very specific to only Sattriya dance.Table 4 shows three such mudras which are similar in hand gesture but different in title, ie."Makara2" in Sattriya but "Matsya" in Bharat Natyam.The sources of these samples are available in [3][4][5] .A few mudras were artistically created by the renowned Assamese polymaths Madhabdev and Srimanta Sankardev.These unique mudras convey precise meaning within themes of Sattriya dance, contributes to identity and helps in conveying heritage of Assam.Table 2 below shows three such mudras "Bardhaman", "Gajadanta" and "Makar1".
The two mudras "Dol" and "Samput" in Table 3 below have the same title in Sattriya and Bharatnatyam, although their expression differs on average.Table 4 on the other hand displays three mudras with similar expressions but distinct names; the table itself lists the names of the mudras in Sattriya and Bharat Natyam.

Experimental Design, Materials and Methods
The digital samples of Sattriya dance mudras were captured with video of at least 20-30 s duration from the source location using DSLR Canon Camera (EOS 1200 D).Videos were recorded with coloured background to reduce injected noise during photo capturing and films.This is an easy and feasible method to segregate the raw videos into frames later.At least three to five videos of each mudra were captured from where the best was selected as sample video.

Data acquisition
The sample collection phase was started in the month of July 2022 and ended in November 2022.In this research work plenty of field works were carried out at different Sattriya dance academy situated in Guwahati, Assam, India.The primary sources of sample were "Chandamukha Sattriya Academy", "Gandharba Kala Kendra", dance instructor Mrs. Parismita Bora and Mrs. Dreamly Gogoi (Guru of Gandharba Kala Kendra).Utmost care have been taken to film the dancer from her left side to her right side at the moment of recording.Now it was easy to segregate the image frames from the raw video into three most prominent segments called Left, Right and Front.The refereed dataset uploaded in Mendeley platform consists of all these three types of images for each recommended mudra.
After collection the samples were pre-processed.More than one video for each mudra for each dancer were taken from more than one dance academy and merged under the same folder.Thus we ensure variability in the dataset.During pre-processing authors have performed Watershed segmentation technique on the sample set.

Watershed segmentation algorithm
Depending on constraint of the application, availability of resources and image description, watershed segmentation techniques was applied on the source dataset.The watershed technique simulates floods on a picture by placing seed points (markers) in strategic locations to generate basins based on the gradient of the image [7] .It is an effective method for picture segmentation, particularly when the forms of the objects are complicated and erratic.This algorithm did not perform well for all the samples.Although most samples were successfully segmented without backdrop, several flaws are found in mudras like Dol and Makar1.
In order to determine quality of the segmentation results, it is crucial to evaluate the performance of watershed technique.The effectiveness of watershed segmentation may be accessed using varieties of methods and measures.Visual inspection is one of them and we have adopted this technique for performance analysis.The segmented images were visually reviewed and compared to the ground truth and expectations.In this method researchers looked for over segmentation, under segmentation and other artefacts.From Table 5 above it is understandable that watershed segmentation captures all the objects in the frame when background is not removed.This is the reason why Dol and Makar1 are challenging and could not be segmented up to the mark with watershed segmentation.Both of these mudras have to pause very close to the performer's body and background removal becomes quite difficult.Since background removal is unfeasible, segmentation is thus insufficient for them.
We used deep neural networks like ResNet-50 and Convolution Neural Network (CNN) which performed brilliantly with a trustworthy solution.Direct application of the CNN and ResNet 50 networks to the data set produced good accuracy and loss results.The resultant accuracy and loss curves along with the performance tables for both the models are shown below.
Skin color-based segmentation is one of the segmentation methods that may be used on the double-handed mudra dataset.It works well to recognize finger color from background.Background subtraction is another approach which can recognize fingers by subtracting background which are static by nature.It may be useful for video frames.Another segmentation method that incorporates geometrical characteristics including aspect ratio, finger position [8] .

Convolution neural network
A deep learning architecture called Convolutional Neural Network (CNN or ConvNet) is primarily made for processing and analyzing visual input, such as pictures and videos.It has performed remarkably well in a variety of computer vision tasks, such as picture segmentation, object identification, and image classification.Each CNN model consists of Input layer, Convolutional layer, Activation function, Pooling layer, Dense layer, Normalization layer, Flatten layer and Output layer [9] .The result obtained by training the double handed Sattriya dance dataset is satisfactory and are shown in Table 6 .On the other hand Table 7 shows evaluation of CNN model in terms of accuracy, precision and recall.Figs. 3 and 4 show the accuracy and loss curves of the trained CNN model.The proposed CNN model architecture is shown in Table 10 with respect to hidden layers, flattening layers, filter size etc.

ResNet-50 architecture
ResNet-50 is a convolutional neural network architecture with 50 layers that is a member of the Residual Networks (ResNets) family of deep neural networks [10] .It offers a unique method for expanding a CNN's convolutional layer structure without encountering the vanishing gradient problem.It is possible due to the skip connections seen in ResNet 50 model that allows the network to bypass certain layers.In this research ResNet 50 model is used to train the mudra dataset and achieved quite good accuracy.As shown in Table 8 , we were able to get 94.05 % validation accuracy at 39th epoch.Fig. 1 shows accuracy curve while Fig. 2 shows the loss curve of ResNet-50 model.The performance tables have been reduced in size by the removal of many     11 below.

Data analysis
In this work it is clear that CNN shows better training accuracy but less validation accuracy compared to ResNet-50.In general, more training accuracy but comparatively less validation accuracy occurs due to a number of reasons in machine learning and deep learning algorithms.The most significant problems include overfitting, data mismatch, short datasets, and bias in the data.
Over fitting occurs when a model learns to match training data too closely and broad patterns are replaced by noise and outliers.As a result, it does well with training data but badly with validation data that hasn't been seen.CNNs are known to be prone to overfitting because of their complexity and huge number of parameters.The likely hood of overfitting increases with network depth and parameter count because the network may acquire generalizable characteristics via memorization of the training set rather than by observation.On the other hand ResNet-50 architecture by the use of residual or skip connections helps in mitigating overfitting in a number of ways like feature reuse, training of deeper networks etc.In case of data mismatch the datasets used for training and validation may not always be representative of one another.These datasets may differ significantly, in which case the model may perform well on the training data but poorly on the validation data.
If the dataset is relatively small the model may struggle to generalize from the few examples in the training data.In such circumstances, the model may learn the training data too thoroughly and have trouble generalizing to new data.
Another cause is bias in data.if the training data is biased or unbalanced the model may learn to predict the majority class well but perform badly on the minority class.This might result in a lower validation accuracy and a greater training accuracy.
Here while training on CNN model, limited data volume and model overfitting were the main contributors to higher training accuracy and lower validation accuracy.Even though we employed data augmentation prior to training, more data with variation is still required to increase the validation accuracy.On the other hand ResNet-50 shows quite a satisfactory training accuracy as well as improved validation accuracy too at 39 epoch.

Performance comparison with other Indian classical hand mudra dataset
The whole referred dataset was segregated into front, left, and right side perspectives.This was done in order to have a more varied perspective on the samples and to view them from different angles.A com prehensive technique for detecting Sattriya dance mudras might be made using this dataset.Sample variances must be used while training the CNN model in order to produce a reliable, error-free recognizer.Other datasets and learning algorithms with good accuracy are also available for traditional Indian dance mudra.An extensive dataset containing many Indian traditional dance mudra was trained using a few learning methods, according to the article [11] .Researchers of that experiment trained an extremely complicated dataset on a CNN-based model, yet their identification rate was just 93.33 %.Other classifiers like VGG, SVM and Deep ANN were able to get only 87.10, 75 % and 84.7 % accuracy respectively on that same

Limitations
This data article basically shows double handed mudras of Sattriya dance which are not easily available on common platform.The researchers of this article collected these mudras of their own, so it is unique and purely original.Our samples are analogous to hand gesture and that is the reason why we selected performers with background knowledge on Sattriya dance.In this case a perfect dancer and a sophisticated digital camera are main possessions while collecting the samples.While capturing photographs from various perspectives and environments, the image's look varies, which might affect the model's performance while recognising the mudras.This is the main limitation on this dataset.Moreover the sample size of double handed mudras used here for segmentation and classification are relatively small, it may affect the performance of a generalized model.
There is scope to improve the dataset in future.Increased number of performers, collection of long time and short time videos, including more classes with noisy samples are some of the possible future improvement of the sample set.The above mentioned extensions of the present data will definitely increase efficiency of classification models.

Declaration of generative AI and AI-assisted technologies in the writing process
During the preparation of this work the author(s) used Paraphrasing Tool -QuillBot AI in order to improve English and readability.
After using this tool/service, the author(s) reviewed and edited the content as needed and take(s) full responsibility for the content of the publication.

Table 1
Image of double handed mudras along with name, meaning and usage.
( continued on next page )

Table 2
Three double-handed distinctive mudras from Sattriya.

Table 4
Mudras with a comparable sentiment but a little title change.

Table 5
Segmented image using Watershed algorithm (with and without background).

Table 6
Evaluation of CNN model performance.

Table 7
Evaluation of CNN model in terms of accuracy, precision and recall.

Table 9
Evaluation of ResNet 50 model in terms of accuracy, precision and recall.

Table 10
Summary table of CNN ModelType: sequential.

Table 11
Summary performance table of both the models.They discovered that the recommended model, which could more accurately classify Kathakali dance than other forms, gave the least accuracy to the Bharatanatyam dance style.In this case the CNN model was able to out performs the ResNet-34 model.They employed less samples than required for a deep neural network, that is why this occurred.Accuracy may be raised by increasing the number of training epochs and volume of sample sets.Their model was able to get 78.88 % average accuracy.On the other hand using CNN model we were able to achieve 96.8 % and 89.32 % training and validation accuracy, 95.11 % training and 94.05 % validation accuracy using ResNet-50 model.