Abstract

We introduce a novel technique for motion segmentation in video frames that combines Neighborhood Preserving Embedding (NPE) with motion detection strategies. Our method starts by extracting visual features using SIFT descriptors and then computes nearest neighbors and pairwise distances to construct the NPE matrix. Simultaneously, we apply optical flow and thresholding to identify areas of motion. By merging the spatial relationships captured through NPE with temporal motion cues, our approach effectively distinguishes moving objects from static backgrounds. When tested on real-world video sequences, the method achieved an F1-score of up to 0.88. Outperforming conventional techniques like Graph Cuts and standard Optical Flow by 3 to 5%. These promising results suggest that our approach is well-suited for real-time surveillance applications and opens new avenues for research in efficient motion segmentation

References

• Horn, B. K. P., Schunck, B. G.: Determining Optical Flow, Artificial Intelligence, Vol. 17, No. 1–3, pp. 185–203, 1981. https://doi.org/10.1016/0004-3702(81)90024-2.
• [Brox, 04] Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High Accuracy Optical Flow Estimation Based on a Theory for Warping, in Pajdla, T., Matas, J. (eds), Computer Vision – ECCV 2004, Lecture Notes in Computer Science, Vol. 3024, Springer, Berlin, Heidelberg, 2004. https://doi.org/10.1007/978-3-540-24673-2_3
• [Stauffer, 99] Stauffer, C., Grimson, W. E. L.: Adaptive Background Mixture Models for Real-Time Tracking, Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 1999), Fort Collins, USA, Vol. 2, pp. 246–252, 1999. doi: 10.1109/CVPR.1999.784637
• [Gao, 23] Gao, M., Zheng, F., Yu, J. J. Q., et al.: Deep Learning for Video Object Segmentation: A Review, Artificial Intelligence Review, Vol. 56, pp. 457–531, 2023. https://doi.org/10.1007/s10462-022-10176-7
• [Byeon, 18] Byeon, M., Yoo, H., Kim, K., Oh, S., Choi, J. Y.: Unified Optimization Framework for Localization and Tracking of Multiple Targets with Multiple Cameras, Computer Vision and Image Understanding, Vol. 166, pp. 51–65, 2018. https://doi.org/10.1016/j.cviu.2017.10.009.
• [Lu, 11] Lu, K., Zhao, J.: Neighborhood Preserving Regression for Image Retrieval, Neurocomputing, Vol. 74, No. 9, pp. 1467–1473, 2011.https://doi.org/10.1016/j.neucom.2010.11.025.
• [He, 05] He, X., Cai, D., Yan, S.: Neighborhood Preserving Embedding, Proceedings of the IEEE International Conference on Computer Vision (ICCV 2005), Vol. 2, pp. 1208–1213, 2005. https://doi.org/10.1109/ICCV.2005.167
• [Zhou, 19] Zhou, Q., Liu, Y., Gao, Y., Wang, X., Hu, R.: Motion Segmentation Using Sparse Representations and NPE-Based Similarity Measures, Neurocomputing, Vol. 350, pp. 28–38, 2019. 10.1016/j.neucom.2019.02.078.
• [Wang, 19] Wang, X., Zhang, S., Yu, J., Liu, W.: Video Motion Segmentation Using Enhanced NPE-Based Features, Multimedia Tools and Applications, Vol. 78, No. 13, pp. 18543–18558, 2019. 10.1007/s11042-019-7289-8.
• [Wang, 21] Wang, Q., Zhou, X., Yang, M., Gao, Y.: Motion Segmentation Using NPE-Based Adaptive Distance Metric Learning, Signal Processing, Vol. 184, Article 108055, 2021. Doi 10.1016/j.sigpro.2021.108055.
• [Ma, 21] Ma, J., Liu, S., Jiang, H.: A New Method for Motion Segmentation Using NPE and Sparse Dictionary Learning, Multimedia Tools and Applications, Vol. 80, No. 13, pp. 19945–19960, 2021. Doi 10.1007/s11042-021-10810-9.
• [Hu, 20] Hu, J., Liu, H., Zhang, Z.: Motion Segmentation Using NPE and Sparse Representation, Journal of Intelligent & Fuzzy Systems, Vol. 38, No. 6, pp. 7047–7056, 2020. Doi 10.3233/JIFS-190864.
• [Liu, 24] Liu, X., Zhao, Y., Wen, S., Chen, B., Ge, S. S.: Motion Segmentation with Event Camera: N-Patches Optical Flow Estimation and Pairwise Markov Random Fields, Expert Systems with Applications, Vol. 254, Article 124342, 2024.
• [Li, 22] Li, X., Zheng, W., Zong, Y.: Motion Cues Guided Feature Aggregation and Enhancement for Video Object Segmentation, Neurocomputing, Vol. 493, pp. 176–190, 2022.
• [Sun, 23] Sun, J., Mao, Y., Dai, Y., Zhong, Y., Wang, J.: MUNet: Motion Uncertainty-Aware Semi-Supervised Video Object Segmentation, Pattern Recognition, Vol. 138, Article 109399, 2023