Human Action Recognition in Military Obstacle Crossing Using HOG and Region-Based Descriptors
DOI:
https://doi.org/10.62411/jcta.12195Keywords:
Histogram of oriented gradient, HOGReG, Human action recognition, Neural Network Classifier, Obstacle crossing competition, Region descriptorAbstract
Human action recognition involves recognizing and classifying actions performed by humans. It has many applications, including sports, healthcare, and surveillance. Challenges such as a limited number of classes of activities and variations within inter and intra-class groups lead to high misclassification rates in some of the intelligent systems developed. Existing studies focused mainly on using public datasets with little focus on real-life action datasets, with limited research on HAR for military obstacle-crossing activities. This paper focuses on recognizing human actions in an obstacle-crossing competition video sequence where multiple participants are performing different obstacle-crossing activities. This study proposes a feature descriptor approach that combines a Histogram of Oriented Gradient and Region Descriptors (HOGReG) for human action recognition in a military obstacle crossing competition. The dataset was captured during military trainees’ obstacle-crossing exercises at a military training institution to achieve this objective. Images were segmented into background and foreground using a Grabcut-based segmentation algorithm, and thereafter, features were extracted and used for classification. The features were extracted using a Histogram of Oriented Gradient (HOG) and region descriptors from segmented images. The extracted features are presented to a neural network classifier for classification and evaluation. The experimental results recorded 63.8%, 82.6%, and 86.4% recognition accuracies using the region descriptors HOG and HOGReG, respectively. The region descriptor gave a training time of 5.6048 seconds, while HOG and HOGReG reported 32.233 and 31.975 seconds, respectively. The outcome shows how effectively the suggested model performed.References
I. Jegham, A. Ben Khalifa, I. Alouani, and M. A. Mahjoub, “Vision-based human action recognition: An overview and real world challenges,” Forensic Sci. Int. Digit. Investig., vol. 32, p. 200901, Mar. 2020, doi: 10.1016/j.fsidi.2019.200901.
A. Kumar, “What is image processing? meaning, techniques, segmentation & important facts to know,” Simplilearn, 2024. https://www.simplilearn.com/image-processing-article#:~:text=One of the most common,distance between the eyes%2C etc.
Y. Kong and Y. Fu, “Human Action Recognition and Prediction: A Survey,” Int. J. Comput. Vis., vol. 130, no. 5, pp. 1366–1401, May 2022, doi: 10.1007/s11263-022-01594-9.
K. Host and M. Ivašić-Kos, “An overview of Human Action Recognition in sports based on Computer Vision,” Heliyon, vol. 8, no. 6, p. e09633, Jun. 2022, doi: 10.1016/j.heliyon.2022.e09633.
S. Zhang, Z. Wei, J. Nie, L. Huang, S. Wang, and Z. Li, “A Review on Human Activity Recognition Using Vision-Based Method,” J. Healthc. Eng., vol. 2017, pp. 1–31, 2017, doi: 10.1155/2017/3090343.
N. Gupta, S. K. Gupta, R. K. Pathak, V. Jain, P. Rashidi, and J. S. Suri, “Human activity recognition in artificial intelligence framework: a narrative review,” Artif. Intell. Rev., vol. 55, no. 6, pp. 4755–4808, Aug. 2022, doi: 10.1007/s10462-021-10116-x.
Y. Li and L. Wang, “Human Activity Recognition Based on Residual Network and BiLSTM,” Sensors, vol. 22, no. 2, p. 635, Jan. 2022, doi: 10.3390/s22020635.
K. Soomro and A. R. Zamir, “Action Recognition in Realistic Sports Videos,” in Computer Vision in Sports, T. B. Moeslund, G. Thomas, and A. Hilton, Eds. Cham: Springer International Publishing, 2014, pp. 181–208. doi: 10.1007/978-3-319-09396-3_9.
B. Russell, A. McDaid, W. Toscano, and P. Hume, “Moving the Lab into the Mountains: A Pilot Study of Human Activity Recog-nition in Unstructured Environments,” Sensors, vol. 21, no. 2, p. 654, Jan. 2021, doi: 10.3390/s21020654.
S. Patil, S. Shelke, S. Joldapke, V. Jumle, and S. Chikhale, “Review on Human Activity Recognition for Military Restricted Areas,” Int. J. Res. Appl. Sci. Eng. Technol., vol. 10, no. 12, pp. 603–606, Dec. 2022, doi: 10.22214/ijraset.2022.47926.
M. A. Khan, M. Sharif, T. Akram, M. Raza, T. Saba, and A. Rehman, “Hand-crafted and deep convolutional neural network features fusion and selection strategy: An application to intelligent human action recognition,” Appl. Soft Comput., vol. 87, p. 105986, 2020, doi: 10.1016/j.asoc.2019.105986.
B. Gahtan, S. Funk, E. Kodesh, I. Ketko, T. Kuflik, and A. M. Bronstein, “WearableMil: An End-to-End Framework for Military Activity Recognition and Performance Monitoring,” ArXiv. Oct. 07, 2024. [Online]. Available: http://arxiv.org/abs/2410.05452
C. I. Patel, D. Labana, S. Pandya, K. Modi, H. Ghayvat, and M. Awais, “Histogram of Oriented Gradient-Based Fusion of Features for Human Action Recognition in Action Video Sequences,” Sensors, vol. 20, no. 24, p. 7299, Dec. 2020, doi: 10.3390/s20247299.
E. Sansano, R. Montoliu, and Ó. Belmonte Fernández, “A study of deep neural networks for human activity recognition,” Comput. Intell., vol. 36, no. 3, pp. 1113–1139, Aug. 2020, doi: 10.1111/coin.12318.
H. E. Azzag, I. E. Zeroual, and A. Ladjailia, “Real-Time Human Action Recognition Using Deep Learning,” Int. J. Appl. Evol. Comput., vol. 13, no. 2, pp. 1–10, Dec. 2022, doi: 10.4018/IJAEC.315633.
B. Ren, M. Liu, R. Ding, and H. Liu, “A Survey on 3D Skeleton-Based Action Recognition Using Learning Method,” Cyborg Bionic Syst., vol. 5, p. 100, Feb. 2020, [Online]. Available: http://arxiv.org/abs/2002.05907
M. M. Islam, S. Nooruddin, F. Karray, and G. Muhammad, “Human activity recognition using tools of convolutional neural net-works: A state of the art review, data sets, challenges, and future prospects,” Comput. Biol. Med., vol. 149, p. 106060, Oct. 2022, doi: 10.1016/j.compbiomed.2022.106060.
P. M. D. Alex, A. Ravikumar, J. Selvaraj, and A. Sahayadhas, “Research on Human Activity Identification Based on Image Processing and Artificial Intelligence,” Int. J. Eng. Technol., vol. 7, no. 3.27, p. 174, Aug. 2018, doi: 10.14419/ijet.v7i3.27.17754.
K. P. S. Kumar and R. Bhavani, “Human activity recognition in egocentric video using HOG, GiST and color features,” Multimed. Tools Appl., vol. 79, no. 5–6, pp. 3543–3559, Feb. 2020, doi: 10.1007/s11042-018-6034-1.
C. Schuldt, I. Laptev, and B. Caputo, “Recognizing human actions: a local SVM approach,” in Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., 2004, vol. 3, pp. 32-36 Vol.3. doi: 10.1109/ICPR.2004.1334462.
P. S. Tan, K. M. Lim, and C. P. Lee, “Human Action Recognition with Sparse Autoencoder and Histogram of Oriented Gradients,” in 2020 IEEE 2nd International Conference on Artificial Intelligence in Engineering and Technology (IICAIET), Sep. 2020, pp. 1–5. doi: 10.1109/IICAIET49801.2020.9257863.
D. K. Singh, “Human Action Recognition in Video,” in Advanced Informatics for Computing Research, D. Singh, P.-A. Hsiung, K. B. G. Hawari, P. Lingras, and P. K. Singh, Eds. Singapore: Springer Singapore, 2019, pp. 54–66. doi: 10.1007/978-981-13-3140-4_6.
O. Elharrouss, N. Almaadeed, S. Al-Maadeed, A. Bouridane, and A. Beghdadi, “A combined multiple action recognition and summarization for surveillance video sequences,” Appl. Intell., vol. 51, no. 2, pp. 690–712, Feb. 2021, doi: 10.1007/s10489-020-01823-z.
S. P. Sahoo, R. Silambarasi, and S. Ari, “Fusion of histogram based features for Human Action Recognition,” in 2019 5th International Conference on Advanced Computing & Communication Systems (ICACCS), Mar. 2019, pp. 1012–1016. doi: 10.1109/ICACCS.2019.8728473.
S. Gundu and H. Syed, “Vision-Based HAR in UAV Videos Using Histograms and Deep Learning Techniques,” Sensors, vol. 23, no. 5, p. 2569, Feb. 2023, doi: 10.3390/s23052569.
A. R. Javed, M. U. Sarwar, S. Khan, C. Iwendi, M. Mittal, and N. Kumar, “Analyzing the Effectiveness and Contribution of Each Axis of Tri-Axial Accelerometer Sensor for Accurate Activity Recognition,” Sensors, vol. 20, no. 8, p. 2216, Apr. 2020, doi: 10.3390/s20082216.
M. Ha, “Top-Heavy CapsNets Based on Spatiotemporal Non-Local for Action Recognition,” J. Comput. Theor. Appl., vol. 2, no. 1, pp. 39–50, May 2024, doi: 10.62411/jcta.10551.
N. Dalal and B. Triggs, “Histograms of Oriented Gradients for Human Detection,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), 2005, vol. 1, pp. 886–893. doi: 10.1109/CVPR.2005.177.
M. Tyagi, “Histogram of Oriented Gradients: An Overview,” builtin, 2024. https://builtin.com/articles/histogram-of-oriented-gradients
M. E. Irhebhude, “Object Detection, Recognition and Re-identification in Video Footage,” Loughborough University, 2015. [Online]. Available: https://repository.lboro.ac.uk/articles/thesis/Object_detection_recognition_and_re-identification_in_video_footage/9406961?file=17024141
M. E. Irhebhude, A. Nawahda, and E. A. Edirisinghe, “View invariant vehicle type recognition and counting system using multiple features,” Int. J. Comput. Vis. Signal Process., vol. 6, no. 1, pp. 20–32, 2016.
A. M. Ayalew, A. O. Salau, B. T. Abeje, and B. Enyew, “Detection and classification of COVID-19 disease from X-ray images using convolutional neural networks and histogram of oriented gradients,” Biomed. Signal Process. Control, vol. 74, p. 103530, Apr. 2022, doi: 10.1016/j.bspc.2022.103530.
B. Bhattarai, R. Subedi, R. R. Gaire, E. Vazquez, and D. Stoyanov, “Histogram of Oriented Gradients meet deep learning: A novel multi-task deep network for 2D surgical image semantic segmentation,” Med. Image Anal., vol. 85, p. 102747, Apr. 2023, doi: 10.1016/j.media.2023.102747.
B. S. Morse, “Lecture 9: Shape description (regions),” Brigham Young University (1998–2000). 2000. [Online]. Available: https://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/MORSE/region-props-and-moments.pdf
M. Sonka, V. Hlavac, and R. Boyle, Image Processing, Analysis, and Machine Vision, 4th ed. Boston, MA: Cengage Learning, 2014. doi: 10.1007/978-1-4899-3216-7.
R. C. Gonzalez and W. Richard E, Digital image processing, 3rd ed. United States of America: Pearson Education Ltd., 2008.
P. Sharma and R. S. Anand, “Depth data and fusion of feature descriptors for static gesture recognition,” IET Image Process., vol. 14, no. 5, pp. 909–920, Apr. 2020, doi: 10.1049/iet-ipr.2019.0230.
L. Kandlbauer, K. Khodier, D. Ninevski, and R. Sarc, “Sensor-based Particle Size Determination of Shredded Mixed Commercial Waste based on two-dimensional Images,” Waste Manag., vol. 120, pp. 784–794, Feb. 2021, doi: 10.1016/j.wasman.2020.11.003.
R. Kruse, S. Mostaghim, C. Borgelt, C. Braune, and M. Steinbrecher, “Multi-layer Perceptrons,” in Computational Intelligence, Cham: Springer, Cham, 2022, pp. 53–124. doi: 10.1007/978-3-030-42227-1_5.
M. Banoula, “An Overview on Multilayer Perceptron (MLP),” Simplilearn, 2023. https://www.simplilearn.com/tutorials/deep-learning-tutorial/multilayer-perceptron
Z. Wang, Y. Lv, R. Wu, and Y. Zhang, “Review of GrabCut in Image Processing,” Mathematics, vol. 11, no. 8, p. 1965, Apr. 2023, doi: 10.3390/math11081965.
Mathworks, “regionprops,” Mathworks, 2024. https://www.mathworks.com/help/images/ref/regionprops.html
X. Cai and G. Han, “Background Subtraction Based on Modified Pulse Coupled Neural Network in Compressive Domain,” IEEE Access, vol. 8, pp. 114422–114432, 2020, doi: 10.1109/ACCESS.2020.3003724.
Mathworks, “horzcat; Concatenate arrays horizontally,” Mathworks, 2024. https://www.mathworks.com/help/matlab/ref/double.horzcat.html
M. E. Irhebhude, A. O. Kolawole, and F. Abdullahi, “Northern Nigeria Human Age Estimation From Facial Images Using Rotation Invariant Local Binary Pattern Features with Principal Component Analysis,” Egypt. Comput. Sci. J., vol. 45, no. 1, 2021.
M. E. Irhebhude, A. O. Kolawole, and H. K. Goma, “A Gender Recognition System Using Facial Images with High Dimensional Data,” Malaysian J. Appl. Sci., vol. 6, no. 1, pp. 27–45, Apr. 2021, doi: 10.37231/myjas.2021.6.1.275.
R. Raj and A. Kos, “An improved human activity recognition technique based on convolutional neural network,” Sci. Rep., vol. 13, no. 1, p. 22581, Dec. 2023, doi: 10.1038/s41598-023-49739-1.
D. Bhattacharya, D. Sharma, W. Kim, M. F. Ijaz, and P. K. Singh, “Ensem-HAR: An Ensemble Deep Learning Model for Smartphone Sensor-Based Human Activity Recognition for Measurement of Elderly Health Monitoring,” Biosensors, vol. 12, no. 6, p. 393, Jun. 2022, doi: 10.3390/bios12060393.
Ž. Ð. Vujovic, “Classification Model Evaluation Metrics,” Int. J. Adv. Comput. Sci. Appl., vol. 12, no. 6, pp. 599–606, 2021, doi: 10.14569/IJACSA.2021.0120670.
A. Tharwat, “Classification assessment methods,” Appl. Comput. Informatics, vol. 17, no. 1, pp. 168–192, Jan. 2021, doi: 10.1016/j.aci.2018.08.003.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Adeola O. Kolawole, Martins E. Irhebhude, Philip O. Odion
This work is licensed under a Creative Commons Attribution 4.0 International License.
