This work aims to investigate and improve adversarial patch attacks for semantic segmentation, a task increasingly deployed in security-critical applications. Existing attacks often overlook pixel-level uncertainty and spatial variation, resulting in inefficient optimization and limited effectiveness. The purpose of this study is to design an uncertainty-aware attack framework that better identifies and exploits structurally vulnerable regions in segmentation models.
We propose a two-stage uncertainty-aware adversarial patch attack framework. The first stage computes pixel-wise entropy to identify locally uncertain regions. The second stage applies a confidence-based inter-pixel weighting strategy that prioritizes vulnerable pixels by comparing their confidence to a global statistical threshold. These components are unified into a dynamic loss reweighting mechanism. Experiments are conducted on Cityscapes and BDD100 K using ICNet, DDRNet, and SegFormer.
Experimental results show that the proposed method outperforms existing patch-based attacks such as SSAP. By effectively targeting uncertain and structurally vulnerable regions, our method achieves stronger degradation of segmentation performance, with mIoU reduced to as low as 8%. The results demonstrate both high attack effectiveness and strong cross-dataset and cross-model generalization.
This work is the first to incorporate pixel-level uncertainty into adversarial patch optimization for semantic segmentation. Unlike prior patch-based attacks that treat all pixels uniformly, our method explicitly models local entropy and confidence-driven spatial variation, enabling more targeted and effective perturbation. The proposed dynamic loss reweighting framework provides a novel perspective on exploiting structural vulnerabilities in dense prediction tasks. This approach offers both theoretical and practical value for understanding segmentation robustness and designing stronger uncertainty-guided attacks.
