Conditional Matting For Post-Segmentation Refinement Segment Anything Model

Authors

  • Al Birr Karim Susanto Dian Nuswantoro University
  • Moch Arief Soeleman Dian Nuswantoro University
  • Fikri Budiman Dian Nuswantoro University

DOI:

https://doi.org/10.33633/jais.v8i3.9024

Abstract

Segment Anything Model (SAM) is a model capable of performing object segmentation in images without requiring any additional training. Although the segmentation produced by SAM lacks high precision, this model holds interesting potential for more accurate segmentation tasks. In this study, we propose a Post-Processing method called Conditional Matting 4 (CM4) to enhance high-precision object segmentation, including prominent, occluded, and complex boundary objects in the segmentation results from SAM. The proposed CM4 Post-Processing method incorporates the use of morphological operations, DistilBERT, InSPyReNet, Grounding DINO, and ViTMatte. We combine these methods to improve the object segmentation produced by SAM. Evaluation is conducted using metrics such as IoU, SAD, MAD, Grad, and Conn. The results of this study show that the proposed CM4 Post-Processing method successfully improves object segmentation with a SAD evaluation score of 20.42 (a 27% improvement from the previous study) and an MSE evaluation score of 21.64 (a 45% improvement from the previous study) compared to the previous research on the AIM-500 dataset. The significant improvement in evaluation scores demonstrates the enhanced capability of CM4 in achieving high precision and overcoming the limitations of the initial segmentation produced by SAM. The contribution of this research lies in the development of an effective CM4 Post-Processing method for enhancing object segmentation in images with high precision. This method holds potential for various computer vision applications that require accurate and detailed object segmentation.

References

A. Kirillov et al., “Segment Anything.” arXiv, Apr. 05, 2023. doi: 10.48550/arXiv.2304.02643.

J. Yao, X. Wang, L. Ye, and W. Liu, “Matte Anything: Interactive Natural Image Matting with Segment Anything Models.” arXiv, Jun. 06, 2023. doi: 10.48550/arXiv.2306.04121.

J. Yao, X. Wang, S. Yang, and B. Wang, “ViTMatte: Boosting Image Matting with Pretrained Plain Vision Transformers.” arXiv, May 24, 2023. doi: 10.48550/arXiv.2305.15272.

S. Liu et al., “Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection.” arXiv, Mar. 20, 2023. doi: 10.48550/arXiv.2303.05499.

H. K. Cheng, J. Chung, Y.-W. Tai, and C.-K. Tang, “CascadePSP: Toward Class-Agnostic and Very High-Resolution Segmentation via Global and Local Refinement.” arXiv, May 05, 2020. doi: 10.48550/arXiv.2005.02551.

T. Shen et al., “High Quality Segmentation for Ultra High-resolution Images.” arXiv, Dec. 26, 2021. doi: 10.48550/arXiv.2111.14482.

P. Krähenbühl and V. Koltun, “Efficient Inference in Fully Connected CRFs with Gaussian Edge Potentials.” arXiv, Oct. 20, 2012. doi: 10.48550/arXiv.1210.5644.

P. A. Dias and H. Medeiros, “Semantic Segmentation Refinement by Monte Carlo Region Growing of High Confidence Detections.” arXiv, Feb. 21, 2018. doi: 10.48550/arXiv.1802.07789.

L. Ke et al., “Segment Anything in High Quality.” arXiv, Jun. 02, 2023. doi: 10.48550/arXiv.2306.01567.

Y. Zhang et al., “Recognize Anything: A Strong Image Tagging Model.” arXiv, Jun. 09, 2023. doi: 10.48550/arXiv.2306.03514.

V. Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter.” arXiv, Feb. 29, 2020. doi: 10.48550/arXiv.1910.01108.

T. Kim, K. Kim, J. Lee, D. Cha, J. Lee, and D. Kim, “Revisiting Image Pyramid Structure for High Resolution Salient Object Detection.” arXiv, Nov. 16, 2022. doi: 10.48550/arXiv.2209.09475.

X. Qin, H. Dai, X. Hu, and D.-P. Fan, “Highly Accurate Dichotomous Image Segmentation”.

J. Li, J. Zhang, and D. Tao, “Deep Automatic Natural Image Matting.” arXiv, Jul. 15, 2021. Accessed: May 20, 2023. [Online]. Available: http://arxiv.org/abs/2107.07235

Downloads

Published

2023-11-30

Issue

Vol. 8 No. 3 (2023): Journal of Applied Intelligent System

Section

Articles

License

 
Authors who publish with this journal agree to the following terms:

  1. Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of the work's authorship and initial publication in this journal.
  2. Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
  3. Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).