A summary of attacks and defenses for ViTs and SAM
| Attack/Defense | Method | Year | Category | Subcategory | Target models | Datasets |
|---|---|---|---|---|---|---|
| Attacks and defenses for ViT (Section 2.1) | ||||||
| Adversarial Attack | Patch-Fool (Fu et al., 2022) | 2022 | White-box | Patch Attack | DeiT, ResNet | ImageNet |
| SlowFormer (Navaneet et al., 2024) | 2024 | White-box | Patch Attack | ATS, AdaViT | ImageNet | |
| PE-Attack (Gao et al., 2024g) | 2024 | White-box | Position Embedding Attack | ViT, DeiT, BEiT | ImageNet, GLUE, wmt13/16, Food-101, CIFAR100, etc. | |
| Attention-Fool (Lovisotto et al., 2022) | 2022 | White-box | Attention Attack | ViT, DeiT, DETR | ImageNet | |
| AAS (Jain and Dutta, 2024) | 2024 | White-box | Attention Attack | ViT-B | ImageNet, CIFAR10/100 | |
| SE-TR (Naseer et al., 2021) | 2022 | Black-box | Transfer-based Attack | DeiT, T2T, TnT, DINO, DETR | ImageNet | |
| ATA (Wang et al., 2022a) | 2022 | Black-box | Transfer-based Attack | ViT, DeiT, ConViT | ImageNet | |
| PNA-PatchOut (Wei et al., 2022) | 2022 | Black-box | Transfer-based Attack | ViT, DeiT, TNT, LeViT, PiT, CaiT, ConViT, Visformer | ImageNet | |
| LPM (Wei and Zhao, 2023) | 2023 | Black-box | Transfer-based Attack | ViT, PiT, DeiT, Visformer, LeViT, ConViT | ImageNet | |
| MIG (Ma et al., 2023) | 2023 | Black-box | Transfer-based Attack | ViT, TNT, Swin | ImageNet | |
| TGR (Zhang et al., 2023c) | 2023 | Black-box | Transfer-based Attack | DeiT, TNT, LeViT, ConViT | ImageNet | |
| VDC (Zhang et al., 2024k) | 2024 | Black-box | Transfer-based Attack | CaiT, TNT, LeViT, ConViT | ImageNet | |
| FDAP (Gao et al., 2024a) | 2024 | Black-box | Transfer-based Attack | ViT, DeiT, CaiT, ConViT, TNT | ImageNet | |
| SASD-WS (Wu et al., 2024d) | 2024 | Black-box | Transfer-based Attack | ViT, ResNet, DenseNet, VGG | ImageNet | |
| CRFA (Li et al., 2024aa) | 2024 | Black-box | Transfer-based Attack | ViT, DeiT, CaiT, TNT, Visformer, LeViT, ConvNeXt, RepLKNet | ImageNet | |
| FPRRen et al. 2025 | 2025 | Black-box | Transfer-based Attack | ViT, CaiT, PiT, Visformer, Swin, DeiT, CoaT, ResNet, VGG, DenseNet | ImageNet | |
| PAR (Shi et al., 2022) | 2022 | Black-box | Query-based Attack | ViT | ImageNet | |
| Adversarial Defense | AGAT (Wu et al., 2022a) | 2022 | Adversarial Training | Efficient training | ViT, CaiT, LeViT | ImageNet |
| ARD-PRM (Mo et al., 2022) | 2022 | Adversarial Training | Efficient training | ViT, DeiT, ConViT, Swin | ImageNet, CIFAR10 | |
| Patch-Vestiges (Li, 2022) | 2022 | Adversarial Detection | Patch-based Detection | ViT, ResNet | CIFAR10 | |
| ViTGuard (Sun et al., 2024a) | 2024 | Adversarial Detection | Attention-based Detection | ViT | ImageNet, CIFAR10/100 | |
| ARMRO (Liu et al., 2023d) | 2023 | Adversarial Detection | Attention-based Detection | ViT, DeiT | ImageNet, CIFAR10 | |
| Smoothed-Attention (Gu et al., 2022) | 2022 | Robust Architecture | Robust Attention | DeiT, ResNet | ImageNet | |
| Adversarial Defense | TAP (Guo et al., 2023a) | 2023 | Robust Architecture | Robust Attention | RVT, FAN | ImageNet, Cityscapes, COCO |
| RSPC (Guo et al., 2023b) | 2023 | Robust Architecture | Robust Attention | RVT, FAN | ImageNet, CIFAR10/100 | |
| FViT (Hu et al., 2024b) | 2024 | Robust Architecture | Robust Attention | ViT, DeiT, Swin | ImageNet, Cityscapes, COCO | |
| SATANikzad et al. 2025 | 2025 | Robust Architecture | Robust Attention | ViT, DeiT | ImageNet | |
| ADBMLi et al. 2024q | 2024 | Adversarial Purification | Diffusion-based Purification | WideResNet, ViT | CIFAR-10, ImageNet, SVHN | |
| CGDMP (Bai et al., 2024b) | 2024 | Adversarial Purification | Diffusion-based Purification | ResNet, XciT | CIFAR 10/100, GTSRB, ImageNet | |
| ADBM (Li et al., 2024q) | 2024 | Adversarial Purification | Diffusion-based Purification | WideResNet, ViT | CIFAR-10, ImageNet, SVHN | |
| OSCP (Lei et al., 2024) | 2024 | Adversarial Purification | Diffusion-based Purification | ViT, Swin, WideResNet | ImageNet, CelebA-HQ | |
| Backdoor Attack | BadViT (Yuan et al., 2023b) | 2023 | Data Poisoning | Patch-level Attack | DeiT, LeViT | ImageNet |
| TrojViT (Zheng et al., 2023) | 2023 | Data Poisoning | Patch-level Attack | DeiT, ViT, Swin | ImageNet, CIFAR10 | |
| SWARM (Yang et al., 2024d) | 2024 | Data Poisoning | Token-level Attack | ViT | VTAB-1k | |
| DBIA (Lv et al., 2023) | 2023 | Data Poisoning | Data-free Attack | ViT, DeiT, Swin | ImageNet, CIFAR10/100, GTSRB, GGFace | |
| MTBA (Li et al., 2024x) | 2024 | Data Poisoning | Multi-trigger Attack | ViT | ImageNet, CIFAR10 | |
| Backdoor defense | PatchDrop (Doan et al., 2023) | 2023 | Robust Inference | Patch Processing | ViT, DeiT, ResNet | ImageNet, CIFAR10 |
| Image Blocking (Subramanya et al., 2024) | 2023 | Robust Inference | Image Blocking | ViT, CaiT | ImageNet | |
| Attacks and defenses for SAM (Section 2.2) | ||||||
| Adversarial Attack | S-RA (Shen et al., 2024b) | 2024 | White-box | Prompt-agnostic Attack | SAM | SA-1B |
| Croce and Hein (2024) | 2024 | White-box | Prompt-agnostic Attack | SAM, SEEM | SA-1B | |
| Attack-SAM (Zhang et al., 2023b) | 2023 | Black-box | Transfer-based Attack | SAM | SA-1B | |
| PATA++ (Zheng and Zhang, 2023) | 2023 | Black-box | Transfer-based Attack | SAM | SA-1B | |
| UAD (Lu et al., 2024b) | 2024 | Black-box | Transfer-based Attack | SAM, FastSAM | SA-1B | |
| T-RA (Shen et al., 2024b) | 2024 | Black-box | Transfer-based Attack | SAM | SA-1B | |
| UMI-GRAT (Xia et al., 2024) | 2024 | Black-box | Transfer-based Attack | Medical SAM, Shadow-SAM, Camouflaged-SAM | CT-Scans, ISTD, COD10K, CAMO, CHAME | |
| Adversarial Attack | Han et al. (2023b) | 2023 | Black-box | Universal Attack | SAM | SA-1B |
| DarkSAM (Zhou et al., 2024i) | 2024 | Black-box | Universal Attack | SAM, HQ-SAM, PerSAM | ADE20K, Cityscapes, COCO, SA-1B | |
| Adversarial Defense | ASAM (Li et al., 2024a) | 2024 | Adversarial Tuning | Diffusion Model-based Tuning | SAM | Ade20k, VOC2012, COCO, DOORS, LVIS, etc. |
| Robust SAMLong et al. 2025 | 2024 | Adversarial Tuning | Parameter-Efficient Fine-Tuning | SAM, MedSAM, SAM-Adapter | SA-1B, VOC, COCO, DAVIS | |
| Backdoor& Poisoning Attack | BadSAM (Guan et al., 2024) | 2024 | Data Poisoning | Visual trigger | SAM | CAMO |
| UnSeg (Sun et al., 2024b) | 2024 | Data Poisoning | Unlearnable Examples | HQ-SAM, DINO, Rsprompter, UNet++, Mask2Former, DeepLabV3 | Cityscapes, VOC, COCO, Lung, Kvasir-seg, WHU, etc. | |
| Attack/Defense | Method | Year | Category | Subcategory | Target models | Datasets |
|---|---|---|---|---|---|---|
| Adversarial Attack | Patch-Fool ( | 2022 | White-box | Patch Attack | DeiT, ResNet | ImageNet |
| SlowFormer ( | 2024 | White-box | Patch Attack | ATS, AdaViT | ImageNet | |
| PE-Attack ( | 2024 | White-box | Position Embedding Attack | ViT, DeiT, BEiT | ImageNet, GLUE, wmt13/16, Food-101, CIFAR100, etc. | |
| Attention-Fool ( | 2022 | White-box | Attention Attack | ViT, DeiT, | ImageNet | |
| 2024 | White-box | Attention Attack | ViT-B | ImageNet, CIFAR10/100 | ||
| SE-TR ( | 2022 | Black-box | Transfer-based Attack | DeiT, T2T, TnT, DINO, | ImageNet | |
| 2022 | Black-box | Transfer-based Attack | ViT, DeiT, ConViT | ImageNet | ||
| PNA-PatchOut ( | 2022 | Black-box | Transfer-based Attack | ViT, DeiT, TNT, LeViT, PiT, CaiT, ConViT, Visformer | ImageNet | |
| 2023 | Black-box | Transfer-based Attack | ViT, PiT, DeiT, Visformer, LeViT, ConViT | ImageNet | ||
| 2023 | Black-box | Transfer-based Attack | ViT, TNT, Swin | ImageNet | ||
| 2023 | Black-box | Transfer-based Attack | DeiT, TNT, LeViT, ConViT | ImageNet | ||
| 2024 | Black-box | Transfer-based Attack | CaiT, TNT, LeViT, ConViT | ImageNet | ||
| 2024 | Black-box | Transfer-based Attack | ViT, DeiT, CaiT, ConViT, | ImageNet | ||
| SASD-WS ( | 2024 | Black-box | Transfer-based Attack | ViT, ResNet, DenseNet, | ImageNet | |
| 2024 | Black-box | Transfer-based Attack | ViT, DeiT, CaiT, TNT, Visformer, LeViT, ConvNeXt, RepLKNet | ImageNet | ||
| 2025 | Black-box | Transfer-based Attack | ViT, CaiT, PiT, Visformer, Swin, DeiT, CoaT, ResNet, VGG, DenseNet | ImageNet | ||
| 2022 | Black-box | Query-based Attack | ViT | ImageNet | ||
| Adversarial Defense | 2022 | Adversarial Training | Efficient training | ViT, CaiT, LeViT | ImageNet | |
| ARD-PRM ( | 2022 | Adversarial Training | Efficient training | ViT, DeiT, ConViT, Swin | ImageNet, CIFAR10 | |
| Patch-Vestiges ( | 2022 | Adversarial Detection | Patch-based Detection | ViT, ResNet | CIFAR10 | |
| ViTGuard ( | 2024 | Adversarial Detection | Attention-based Detection | ViT | ImageNet, CIFAR10/100 | |
| 2023 | Adversarial Detection | Attention-based Detection | ViT, DeiT | ImageNet, CIFAR10 | ||
| Smoothed-Attention ( | 2022 | Robust Architecture | Robust Attention | DeiT, ResNet | ImageNet | |
| Adversarial Defense | 2023 | Robust Architecture | Robust Attention | RVT, | ImageNet, Cityscapes, | |
| 2023 | Robust Architecture | Robust Attention | RVT, | ImageNet, CIFAR10/100 | ||
| FViT ( | 2024 | Robust Architecture | Robust Attention | ViT, DeiT, Swin | ImageNet, Cityscapes, | |
| 2025 | Robust Architecture | Robust Attention | ViT, DeiT | ImageNet | ||
| 2024 | Adversarial Purification | Diffusion-based Purification | WideResNet, ViT | CIFAR-10, ImageNet, | ||
| 2024 | Adversarial Purification | Diffusion-based Purification | ResNet, XciT | |||
| 2024 | Adversarial Purification | Diffusion-based Purification | WideResNet, ViT | CIFAR-10, ImageNet, | ||
| 2024 | Adversarial Purification | Diffusion-based Purification | ViT, Swin, WideResNet | ImageNet, CelebA-HQ | ||
| Backdoor Attack | BadViT ( | 2023 | Data Poisoning | Patch-level Attack | DeiT, LeViT | ImageNet |
| TrojViT ( | 2023 | Data Poisoning | Patch-level Attack | DeiT, ViT, Swin | ImageNet, CIFAR10 | |
| 2024 | Data Poisoning | Token-level Attack | ViT | VTAB-1k | ||
| 2023 | Data Poisoning | Data-free Attack | ViT, DeiT, Swin | ImageNet, CIFAR10/100, GTSRB, GGFace | ||
| 2024 | Data Poisoning | Multi-trigger Attack | ViT | ImageNet, CIFAR10 | ||
| Backdoor defense | PatchDrop ( | 2023 | Robust Inference | Patch Processing | ViT, DeiT, ResNet | ImageNet, CIFAR10 |
| Image Blocking ( | 2023 | Robust Inference | Image Blocking | ViT, CaiT | ImageNet | |
| Adversarial Attack | S-RA ( | 2024 | White-box | Prompt-agnostic Attack | SA-1B | |
| 2024 | White-box | Prompt-agnostic Attack | SAM, | SA-1B | ||
| Attack-SAM ( | 2023 | Black-box | Transfer-based Attack | SA-1B | ||
| PATA++ ( | 2023 | Black-box | Transfer-based Attack | SA-1B | ||
| 2024 | Black-box | Transfer-based Attack | SAM, FastSAM | SA-1B | ||
| T-RA ( | 2024 | Black-box | Transfer-based Attack | SA-1B | ||
| UMI-GRAT ( | 2024 | Black-box | Transfer-based Attack | Medical SAM, Shadow-SAM, Camouflaged-SAM | CT-Scans, ISTD, COD10K, CAMO, | |
| Adversarial Attack | 2023 | Black-box | Universal Attack | SA-1B | ||
| DarkSAM ( | 2024 | Black-box | Universal Attack | SAM, HQ-SAM, PerSAM | ADE20K, Cityscapes, COCO, SA-1B | |
| Adversarial Defense | 2024 | Adversarial Tuning | Diffusion Model-based Tuning | Ade20k, VOC2012, COCO, DOORS, LVIS, etc. | ||
| Robust | 2024 | Adversarial Tuning | Parameter-Efficient Fine-Tuning | SAM, MedSAM, SAM-Adapter | SA-1B, VOC, COCO, | |
| Backdoor& Poisoning Attack | BadSAM ( | 2024 | Data Poisoning | Visual trigger | ||
| UnSeg ( | 2024 | Data Poisoning | Unlearnable Examples | HQ-SAM, DINO, Rsprompter, UNet++, Mask2Former, DeepLabV3 | Cityscapes, VOC, COCO, Lung, Kvasir-seg, WHU, etc. | |
Sharing content requires targeting cookies to be enabled. Please update your cookie preferences to use this feature.