Video Class-Incremental Learning for Action Recognition

Authors

  • E. Mohanapriya * Department of Computer Science and Engineering, Sri Sivasubramaniya Nadar College of Engineering, Kalavakkam, Tamil Nadu, India.
  • T.T. Mirnalinee Department of Computer Science and Engineering, Sri Sivasubramaniya Nadar College of Engineering, Kalavakkam, Tamil Nadu, India.

https://doi.org/10.22105/scfa.v2i2.63

Abstract

In the domain of video-based action recognition, overcoming catastrophic forgetting while continuously learning new classes remains a major challenge. We propose a Video Class-Incremental Learning (VCIL) framework that addresses this issue by employing a teacher-student knowledge distillation strategy. Our approach leverages both response-based distillation, which aligns the student model’s predictions with the teacher’s softened outputs, and feature-based distillation, which ensures the student retains internal feature representations learned by the teacher. With the UCF101 action recognition dataset and a 3D ResNet backbone, our approach extracts spatiotemporal features to recognize actions in multiple incremental steps. Our model is tested with various settings (10×5, 5×10, 2×25) and has high accuracy for retaining knowledge of past classes and learning new courses. The results show that our approach is efficient in preventing forgetting and maintaining high performance on new tasks.

Keywords:

Incremental learning, Action recognition, Knowledge distillation, Deep learning

References

  1. [1] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A. A., … ., & Hadsell, R. (2017). Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences, 114(13), 3521–3526. https://doi.org/10.1073/pnas.1611835114
  2. [2] Feng, T., Wang, M., & Yuan, H. (2022). Overcoming catastrophic forgetting in incremental object detection via elastic response distillation. 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR) (pp. 9427–9436). IEEE. https://doi.org/10.1109/CVPR52688.2022.00921
  3. [3] Zenke, F., Poole, B., & Ganguli, S. (2017). Continual learning through synaptic intelligence. Proceedings of the 34th international conference on machine learning (pp. 3987–3995). PMLR. https://proceedings.mlr.press/v70/zenke17a.html
  4. [4] Li, Z., & Hoiem, D. (2018). Learning without Forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12), 2935–2947. https://doi.org/10.1109/TPAMI.2017.2773081
  5. [5] Hinton, G., Vinyals, O., & Dean, J. (2015). Distilling the knowledge in a neural network. https://doi.org/10.48550/arXiv.1503.02531
  6. [6] Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., … ., & Bengio, Y. (2020). Generative adversarial networks. Communications of the acm, 63(11), 139–144. https://doi.org/10.1145/3422622
  7. [7] Rebuffi, S. A., Kolesnikov, A., Sperl, G., & Lampert, C. H. (2017). iCaRL: Incremental classifier and representation learning. Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2001–2010). IEEE. https://doi.org/10.1109/CVPR.2017.587
  8. [8] Kingma, D. P. (2017). Variational inference & deep learning [Thesis]. chrome-extension://efaidnbmnnnibpcajpcglclefindmkaj/https://pure.uva.nl/ws/files/17891313/Thesis.pdf
  9. [9] Liu, C., Wang, Y., Li, D., & Wang, X. (2024). Domain-incremental learning without forgetting based on random vector functional link networks. Pattern recognition, 151, 110430. https://doi.org/10.1016/j.patcog.2024.110430
  10. [10] Chi, Z., Gu, L., Liu, H., Wang, Y., Yu, Y., & Tang, J. (2022). Metafscil: A meta-learning approach for few-shot class incremental learning. 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR) (pp. 14166–14175). IEEE. https://doi.org/10.1109/CVPR52688.2022.01377
  11. [11] Park, J., Kang, M., & Han, B. (2021). Class-incremental learning for action recognition in videos. 2021 IEEE/CVF international conference on computer vision (ICCV) (pp. 13698–13707). IEEE. https://doi.org/10.1109/ICCV48922.2021.01344
  12. [12] Maraghi, V. O., & Faez, K. (2022). Class-incremental learning on video-based action recognition by distillation of various knowledge. Computational intelligence and neuroscience, 2022(1), 4879942. https://doi.org/10.1155/2022/4879942
  13. [13] Pei, Y., Qing, Z., Zhang, S., Wang, X., Zhang, Y., Zhao, D., & Qian, X. (2023). Space-time prompting for video class-incremental learning. 2023 IEEE/CVF international conference on computer vision (ICCV) (pp. 11932–11942).IEEE. https://doi.org/10.1109/ICCV51070.2023.01096
  14. [14] Luo, Z., Liu, Y., Schiele, B., & Sun, Q. (2023). Class-incremental exemplar compression for class-incremental learning. 2023 IEEE/CVF conference on computer vision and pattern recognition (CVPR) (pp. 11371–11380). IEEE. https://doi.org/10.1109/CVPR52729.2023.01094

Downloads

Published

2025-06-17

Issue

Vol. 2 No. 2 (2025): Soft Computing Fusion with Applications

Section

Articles

How to Cite

Video Class-Incremental Learning for Action Recognition. (2025). Soft Computing Fusion With Applications , 2(2), 120-126. https://doi.org/10.22105/scfa.v2i2.63

Similar Articles

11-19 of 19

You may also start an advanced similarity search for this article.