MUET logo

Improving Human Action Recognition Through Cross-View Camera-Aware Augmentation

Authors

  • Zahid Hussain Khaskheli

    PhD Scholar at the Institute of Information and Communication Technologies (IICT), Mehran University of Engineering and Technology, Jamshoro 76060, Pakistan
    Author

  • Qasim Ali

    Department of Software Engineering, Mehran University of Engineering and Technology, Jamshoro 76060, Pakistan
    Author

  • M Shehram Shah

    Department of Software Engineering, Mehran University of Engineering and Technology, Jamshoro 76060, Pakistan
    Author

  • Din Muhammad

    Department of Software Engineering, Mehran University of Engineering and Technology, Jamshoro 76060, Pakistan
    Author

  • Farast Ali

    Department of Computer Science, Bahria University, Karachi 75260, Pakistan
    Author

  • Hina Ali

    Department of Software Engineering, Mehran University of Engineering and Technology, Jamshoro 76060, Pakistan
    Author

  • Maria Khalid

    Islamabad Model College for Girls, Islamabad 44000, Pakistan
    Author

  • Zoha Waheed

    Computational Science and Technology, National University of Sciences and Technology (NUST), Islamabad 4000, Pakistan
    Author

  • Umer Gaffar

    Department of Statistical and Actuarial Sciences, University of the Punjab, Lahore 54590, Pakistan
    Author

DOI:

https://doi.org/10.22581/muet1982.0905

Keywords:

Human Action Recognition, 3D Neural Networks, Deep Learning, Data Augmentation, Cross-View Action Recognition, Spatiotemporal Feature Learning

Abstract

Human Action Recognition (HAR) is vital in surveillance, sports analysis, and Human–Computer Interaction (HCI). Publicly available benchmarks mostly suffer from limited sample sizes, class imbalance, occlusions, and viewpoint variability, which may affect model generalization. Most existing HAR augmentation pipelines rely on generic or heuristic transformations that do not explicitly model camera-induced variations. To address these challenges, we propose an Augmented Data Generator Framework (ADGF) for HAR that integrates multiple parameter-controlled augmentation strategies designed to approximate realistic camera-induced variability while preserving temporal consistency. Our method integrates four transformation strategies. Each augmentation technique is parameterized across validated variation ranges, realistically simulating camera viewpoint changes, motion diversity, and partial occlusions, while preserving frame-to-frame temporal coherence, compared with uniformly applied conventional augmentations. ADGF distributes controlled transformations to maintain a balance and preserve action semantics. We evaluate performance with standard metrics, and perform an ablation study to measure each augmentation's contribution. With the combined augmented set, a top-1 accuracy on HMDB51 rises from 87.5% (baseline) to 97.7%, and final training loss falls from 0.21 to 0.12.  Required epochs drop from 38 to 25 (≈34% fewer epochs). Cross-dataset testing on UCF101 demonstrates improved robustness, with accuracy increasing from 85.1% to 96.5%. The ablation study shows that each augmentation yields a positive gain with viewpoint variation, that occlusion simulation produces the largest single-technique improvements, and that the full combination produces the strongest transferability. These results indicate that a balanced structured augmentation strategy substantially improves both in-dataset performance and cross-dataset robustness. Implementation details, augmentation configurations, and experimental settings will be made publicly available upon publication to support reproducibility.

Downloads

References

[1] M. Karim, S. Khalid, A. Aleryani, J. Khan, I. Ullah, and Z. Ali, “Human Action Recognition Systems: A Review of the Trends and State-of-the-Art,” IEEE Access, vol. 12, pp. 36372–36390, 2024, doi: 10.1109/ACCESS.2024.3373199.

[2] G. Diraco, G. Rescio, A. Caroppo, A. Manni, and A. Leone, “Human Action Recognition in Smart Living Services and Applications: Context Awareness, Data Availability, Personalization, and Privacy,” Sensors, vol. 23, no. 13, p. 6040, June 2023, doi: 10.3390/s23136040.

[3] A. Sanchez-Comas, K. Synnes, and J. Hallberg, “Hardware for Recognition of Human Activities: A Review of Smart Home and AAL Related Technologies,” Sensors, vol. 20, no. 15, p. 4227, July 2020, doi: 10.3390/s20154227.

[4] L. E. Haddad, M. Hanoune, and A. Ettaoufik, “Computer Vision with Deep Learning for Human Activity Recognition: Features Representation,” in Engineering Applications of Artificial Intelligence, A. Chakir, J. F. Andry, A. Ullah, R. Bansal, and M. Ghazouani, Eds., in Synthesis Lectures on Engineering, Science, and Technology. , Cham: Springer Nature Switzerland, 2024, pp. 41–66. doi: 10.1007/978-3-031-50300-9_3.

[5] Y. Zhang, B. Li, H. Fang, and Q. Meng, “Current Advances on Deep Learning-based Human Action Recognition from Videos: a Survey,” in 2021 20th IEEE International Conference on Machine Learning and Applications (ICMLA), Pasadena, CA, USA: IEEE, Dec. 2021, pp. 304–311. doi: 10.1109/ICMLA52953.2021.00054.

[6] I. Laptev, “On Space-Time Interest Points,” Int. J. Comput. Vis., vol. 64, no. 2–3, pp. 107–123, Sept. 2005, doi: 10.1007/s11263-005-1838-7.

[7] C. Shorten and T. M. Khoshgoftaar, “A survey on Image Data Augmentation for Deep Learning,” J. Big Data, vol. 6, no. 1, p. 60, July 2019, doi: 10.1186/s40537-019-0197-0.

[8] H. Wang and C. Schmid, “Action Recognition with Improved Trajectories,” in 2013 IEEE International Conference on Computer Vision (ICCV), Sydney, Australia: IEEE, Dec. 2013, pp. 3551–3558, doi: 10.1109/ICCV.2013.441.

[9] M. Ahsan Habib et al., “Exploring Progress in Text-to-Image Synthesis: An In-Depth Survey on the Evolution of Generative Adversarial Networks,” IEEE Access, vol. 12, pp. 178401–178440, 2024, doi: 10.1109/ACCESS.2024.3435541.

[10] N. E. Khalifa, M. Loey, and S. Mirjalili, “A comprehensive survey of recent trends in deep learning for digital images augmentation,” Artif. Intell. Rev., vol. 55, no. 3, pp. 2351–2377, Mar. 2022, doi: 10.1007/s10462-021-10066-4.

[11] T. Kumar, R. Brennan, A. Mileo, and M. Bendechache, “Image Data Augmentation Approaches: A Comprehensive Survey and Future Directions,” IEEE Access, vol. 12, pp. 187536–187571, 2024, doi: 10.1109/ACCESS.2024.3470122.

[12] J. Wang, Y. Chen, S. Hao, X. Peng, and L. Hu, “Deep Learning for Sensor-based Activity Recognition: A Survey,” Pattern Recognit. Lett., vol. 119, pp. 3–11, Mar. 2019, doi: 10.1016/j.patrec.2018.02.010.

[13] L. Bibbò and M. M. B. R. Vellasco, “Human Activity Recognition (HAR) in Healthcare,” Appl. Sci., vol. 13, no. 24, p. 13009, Dec. 2023, doi: 10.3390/app132413009.

[14] R. Ullah et al., “Vision-Based Activity Recognition for Unobtrusive Monitoring of the Elderly in Care Settings,” Technologies, vol. 13, no. 5, p. 184, May 2025, doi: 10.3390/technologies13050184.

[15] O. D. Lara and M. A. Labrador, “A Survey on Human Activity Recognition using Wearable Sensors,” IEEE Commun. Surv. Tutor., vol. 15, no. 3, pp. 1192–1209, 2013, doi: 10.1109/SURV.2012.110112.00192.

[16] A. Herath, M. Harandi, and F. Porikli, “Going Deeper into Action Recognition: A Survey,” Image Vis. Comput., vol. 60, pp. 4–21, Apr. 2017, doi: 10.1016/j.imavis.2017.01.010.

[17] P. Kumar, S. Chauhan, and L. K. Awasthi, “Human Activity Recognition (HAR) Using Deep Learning: Review, Methodologies, Progress and Future Research Directions,” Arch. Comput. Methods Eng., vol. 31, no. 1, pp. 179–219, Jan. 2024, doi: 10.1007/s11831-023-09986-x.

[18] C. I. Patel, D. Labana, S. Pandya, K. Modi, H. Ghayvat, and M. Awais, “Histogram of Oriented Gradient-Based Fusion of Features for Human Action Recognition in Action Video Sequences,” Sensors, vol. 20, no. 24, p. 7299, Dec. 2020, doi: 10.3390/s20247299.

[19] S. P. Sahoo, S. Ari, K. Mahapatra, and S. P. Mohanty, “HAR-Depth: A Novel Framework for Human Action Recognition Using Sequential Learning and Depth Estimated History Images,” IEEE Trans. Emerg. Top. Comput. Intell., vol. 5, no. 5, pp. 813–825, Oct. 2021, doi: 10.1109/TETCI.2020.3014367.

[20] I. D. Mienye and T. G. Swart, “A Comprehensive Review of Deep Learning: Architectures, Recent Advances, and Applications,” Information, vol. 15, no. 12, p. 755, Nov. 2024, doi: 10.3390/info15120755.

[21] A. Deng, T. Yang, and C. Chen, “A Large-scale Study of Spatiotemporal Representation Learning with a New Benchmark on Action Recognition,” in 2023 IEEE/CVF International Conference on Computer Vision (ICCV), Paris, France: IEEE, Oct. 2023, pp. 20462–20474. doi: 10.1109/ICCV51070.2023.01876.

[22] T. Singh and D. K. Vishwakarma, “Video benchmarks of human action datasets: a review,” Artif. Intell. Rev., vol. 52, no. 2, pp. 1107–1154, Aug. 2019, doi: 10.1007/s10462-018-9651-1.

[23] L. Yuan, Z. He, Q. Wang, and L. Xu, “Advancing Human Motion Recognition with SkeletonCLIP++: Weighted Video Feature Integration and Enhanced Contrastive Sample Discrimination,” Sensors, vol. 24, no. 4, p. 1189, Feb. 2024, doi: 10.3390/s24041189.

[24] E. Goceri, “Medical image data augmentation: techniques, comparisons and interpretations,” Artif. Intell. Rev., vol. 56, no. 11, pp. 12561–12605, Nov. 2023, doi: 10.1007/s10462-023-10453-z.

[25] N. Akhtar, M. Saddique, K. Asghar, U. I. Bajwa, M. Hussain, and Z. Habib, “Digital Video Tampering Detection and Localization: Review, Representations, Challenges and Algorithm,” Mathematics, vol. 10, no. 2, p. 168, Jan. 2022, doi: 10.3390/math10020168.

[26] T. Salimans et al., “Improved Techniques for Training GANs,” in Adv. Neural Inf. Process. Syst. (NeurIPS), 2016, pp. 2234–2242.

[27] C. Xu, B. Chai, J. He, X. Zhang, and E. W. M. Ma, “Deep Learning-Based Human Action Recognition: A Survey,” Pattern Recognit., vol. 139, p. 109524, July 2023, doi: 10.1016/j.patcog.2023.109524.

[28] M. A. Hassan, M. U. G. Khan, R. Iqbal, O. Riaz, A. K. Bashir, and U. Tariq, “Predicting humans future motion trajectories in video streams using generative adversarial network,” Multimed. Tools Appl., vol. 83, no. 5, pp. 15289–15311, Sept. 2021, doi: 10.1007/s11042-021-11457-z.

[29] M.-Y. Liu, X. Huang, J. Yu, T.-C. Wang, and A. Mallya, “Generative Adversarial Networks for Image and Video Synthesis: Algorithms and Applications,” Proc. IEEE, vol. 109, no. 5, pp. 839–862, May 2021, doi: 10.1109/JPROC.2021.3049196.

[30] L. Perez and J. Wang, “The Effectiveness of Data Augmentation in Image Classification using Deep Learning,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), Honolulu, HI, USA: IEEE, July 2017, pp. 1–8, doi: 10.1109/CVPRW.2017.19.

[31] A. Schäfer, G. Reis, and D. Stricker, “The Gesture Authoring Space: Authoring Customised Hand Gestures for Grasping Virtual Objects in Immersive Virtual Environments,” in Mensch und Computer 2022, Darmstadt Germany: ACM, Sept. 2022, pp. 85–95. doi: 10.1145/3543758.3543766.

[32] M. Ehatisham-Ul-Haq et al., “Robust Human Activity Recognition Using Multimodal Feature-Level Fusion,” IEEE Access, vol. 7, pp. 60736–60751, 2019, doi: 10.1109/ACCESS.2019.2913393.

[33] B. X. B. Yu, Y. Liu, X. Zhang, S. Zhong, and K. C. C. Chan, “MMNet: A Model-Based Multimodal Network for Human Action Recognition in RGB-D Videos,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 45, no. 3, pp. 3522–3538, Mar. 2023, doi: 10.1109/TPAMI.2022.3177813.

[34] I. M. De La Jara, C. Rodriguez-Opazo, E. Marrese-Taylor, and F. Bravo-Marquez, “An empirical study of the effect of video encoders on Temporal Video Grounding,” in 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), Paris, France: IEEE, Oct. 2023, pp. 2842–2847. doi: 10.1109/ICCVW60793.2023.00306.

[35] M. Almushyti and F. W. B. Li, “Recognising Human-Object Interactions Using Attention-based LSTMs,” Comput. Graph. Vis. Comput. CGVC, p. 5 pages, 2019, doi: 10.2312/CGVC.20191269.

[36] Y. Huang, H. Chen, K. Huang, M. Chen, J. Wang, and B. Liu, “Optimization of Space-Time image velocimetry based on deep residual learning,” Measurement, vol. 232, p. 114688, June 2024, doi: 10.1016/j.measurement.2024.114688.

[37] D. Tran, H. Wang, L. Torresani, J. Ray, Y. LeCun, and M. Paluri, “A Closer Look at Spatiotemporal Convolutions for Action Recognition,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake

Published

2026-07-01

Issue

Vol. 45 No. 3 (2026): July Issue

Section

Articles

License

Copyright (c) 2026 Mehran University Research Journal of Engineering and Technology

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

How to Cite

Improving Human Action Recognition Through Cross-View Camera-Aware Augmentation. (2026). Mehran University Research Journal of Engineering and Technology, 45(3), 205-218. https://doi.org/10.22581/muet1982.0905

Most read articles by the same author(s)

Similar Articles

1-10 of 76

You may also start an advanced similarity search for this article.