Safety at scale: a comprehensive survey of large model and agent safety

Artificial Intelligence (AI) has entered the era of large models, exemplified by Vision Foundation Models (VFMs), Large Language Models (LLMs), Vision-Language Pre-Training (VLP) models, Vision-Language Models (VLMs), and image/video generation diffusion models (DMs). Through large-scale pre-training on massive datasets, these models have demonstrated unprecedented capabilities in tasks ranging from language understanding and image generation to complex problem-solving and decision-making. Their ability to understand and generate human-like content (e.g., texts, images, audios, and videos) has enabled applications in customer service, content creation, healthcare, education, and more, highlighting their transformative potential in both commercial and societal domains.

However, the deployment of large models comes with significant challenges and risks. As these models become more integrated into critical applications, concerns regarding their vulnerabilities to adversarial, jailbreak, and backdoor attacks, data privacy breaches, and the generation of harmful or misleading content have intensified. These issues pose substantial threats, including unintended system behaviors, privacy leakage, and the dissemination of harmful information. Ensuring the safety of these models is paramount to prevent such unintended consequences, maintain public trust, and promote responsible AI usage. The field of AI safety research has expanded in response to these challenges, encompassing a diverse array of attack methodologies, defense strategies, and evaluation benchmarks designed to identify and mitigate the vulnerabilities of large models. Given the rapid development of safety-related techniques for various large models, we aim to provide a comprehensive survey of these techniques, highlighting strengths, weaknesses, and gaps, while advancing research and fostering collaboration.

Given the broad scope of our survey, we have structured it with the following considerations to enhance clarity and organization:

• Models. We focus on six widely studied model categories, including VFMs, LLMs, VLPs, VLMs, DMs, and Agents, and review the attack and defense methods for each separately. These models represent the most popular large models across various domains.

• Organization. For each model category, we classify the reviewed works into attacks and defenses, and identify 10 attack types: adversarial, backdoor, poisoning, jailbreak, prompt injection, energy-latency, membership inference, model extraction, data extraction, and agent attacks. When both backdoor and poisoning attacks are present for a model category, we combine them into a single backdoor & poisoning category due to their similarities. We review the corresponding defense strategies for each attack type immediately after the attacks.

• Taxonomy. For each type of attack or defense, we use a two-level taxonomy: Category$→$Subcategory. The Category differentiates attacks and defenses based on the threat model (e.g., white-box, gray-box, black-box) or specific subtasks (e.g., detection, purification, robust training/tuning, and robust inference). The Subcategory offers a more detailed classification based on their techniques.

• Granularity. To ensure clarity, we simplify the introduction of each reviewed paper, highlighting only its key ideas, objectives, and approaches, while omitting technical details and experimental analyses.

Our survey methodology is structured as follows. First, we conducted a keyword-based search targeting specific model types and threat types to identify relevant papers. Next, we manually filtered out non-safety-related and non-technical papers. For each remaining paper, we categorized its proposed method or framework by analyzing its settings and attack/defense types, assigning them to appropriate categories and subcategories. In total, we reviewed 574 technical papers, with their distribution across years, model types, and attack/defense strategies illustrated in Figure 1. As shown, safety research on large models has surged significantly since 2023, following the release of ChatGPT. Among the model types, LLMs, DMs and Agents have garnered the most attention, accounting for 71.32% of the surveyed papers.

Regarding attack types, jailbreak, adversarial, and backdoor attacks were the most extensively studied. On the defense side, jailbreak defenses received the highest focus, followed by adversarial defenses. Figure 2 presents a cross-view of temporal trends across model types and attack/defense categories, offering a detailed breakdown of the reviewed works. Notably, research on attacks constitutes $∼$60% of the studied. In terms of defense, while defense research accounts for only $∼$40%, underscoring a significant gap that warrants increased attention toward defense strategies. The overall structure of this survey is outlined in Figure 3.

Difference to Existing Surveys. Large-model safety is a rapidly evolving field, and several surveys have been conducted to advance research in this area. Recently, Slattery et al. (2024) introduced an AI risk framework with a systematic taxonomy covering all types of risks. In contrast, our focus is on the technical aspects, specifically the attack and defense techniques proposed in the literature. Table 1 summarizes the technical surveys we identified, each concentrating on a few model types or threat categories (e.g., LLMs, VLMs, agents, or jailbreak attacks/defenses). Compared with these works, our survey provides both a broader scope by covering a wider range of model types and threats, and a more high-level perspective that focuses on overarching methodologies rather than specific technical details.

This section surveys safety research on two types of VFMs: per-trained Vision Transformers (ViTs) (Dosovitskiy et al., 2021) and the Segment Anything Model (SAM) (Kirillov et al., 2023). We focus on ViTs and SAM because they are among the most widely deployed VFMs and have garnered significant attention in recent safety research.

Pre-trained ViTs are widely employed as backbones for various downstream tasks, frequently achieving state-of-the-art performance through efficient adaptation and fine-tuning. Unlike traditional CNNs, ViTs process images as sequences of tokenized patches, allowing them to better capture spatial dependencies. However, this patch-based mechanism also brings unique safety concerns and robustness challenges. This section explores these issues by reviewing ViT-related safety research, including adversarial attacks, backdoor & poisoning attacks, and their corresponding defense strategies. Table 2 provides a summary of the surveyed attacks and defenses, along with the commonly used datasets.

Adversarial attacks on ViTs can be classified into white-box attacks and black-box attacks based on whether the attacker has full access to the victim model. Based on the attack strategy, white-box attacks can be further divided into 1) patch attacks, 2) position embedding attacks and 3) attention attacks, while black-box attacks can be summarized into 1) transfer-based attacks and 2) query-based attacks.

White-box Attacks. Patch Attacks exploit the modular structure of ViTs, aiming to manipulate their inference processes by introducing targeted perturbations in specific patches of the input data. Joshi et al. (2021) proposed an adversarial token attack method leveraging block sparsity to assess the vulnerability of ViTs to token-level perturbations. Expanding on this, Patch-Fool (Fu et al., 2022) introduces an adversarial attack framework that targets the self-attention modules by perturbing individual image patches, thereby manipulating attention scores. Different from existing methods, SlowFormer (Navaneet et al., 2024) introduces a universal adversarial patch can be applied to any image to increases computational and energy costs while preserving model accuracy.

Position Embedding Attacks aim to attack the spatial or sequential position of tokens in transformers. For example, PE-Attack (Gao et al., 2024g) explores the common vulnerability of positional embeddings to adversarial perturbations by disrupting their ability to encode positional information through periodicity manipulation, linearity distortion, and optimized embedding distortion.

Attention Attacks target vulnerabilities in the self-attention modules of ViTs. Attention-Fool (Lovisotto et al., 2022) manipulates dot-product similarities to redirect queries to adversarial key tokens, exposing the model’s sensitivity to adversarial patches. Similarly, AAS (Jain and Dutta, 2024) mitigates gradient masking in ViTs by optimizing the pre-softmax output scaling factors, enhancing the effectiveness of attacks.

Black-box Attacks. Transfer-based Attacks first generate adversarial examples using fully accessible surrogate models, which are then transferred to attack black-box victim ViTs. In this context, we first review attacks specifically designed for the ViT architecture. SE-TR (Naseer et al., 2021) enhances adversarial transferability by optimizing perturbations on an ensemble of models. ATA (Wang et al., 2022a) strategically activates uncertain attention and perturbs sensitive embeddings within ViTs. LPM (Wei and Zhao, 2023) mitigates the overfitting to model-specific discriminative regions through a patch-wise optimized binary mask. Chen et al. (2023f) introduced an Inductive Bias Attack (IBA) to suppress unique biases in ViTs and target shared inductive biases. TGR (Zhang et al., 2023c) reduces the variance of the backpropagated gradient within internal blocks. VDC (Zhang et al., 2024k) employs virtual dense connections between deeper attention maps and MLP blocks to facilitate gradient backpropagation. FDAP (Gao et al., 2024a) exploits feature collapse by reducing high-frequency components in feature space. CRFA (Li et al., 2024aa) disrupts only the most crucial image regions using approximate attention maps. SASD-WS (Wu et al., 2024d) flattens the loss landscape of the source model through sharpness-aware self-distillation and approximates an ensemble of pruned models using weight scaling to improve target adversarial transferability. FPR (Ren et al., 2025) improves the adversarial transferability on ViTs by refining forward propagation instead of backward gradients. It consists of two components: (1) Attention Map Diversification (AMD), which applies controlled randomness to diversify attention maps, mitigating overfitting and implicitly inducing gradient vanishing; and (2) Momentum Token Embedding (MTE), which stabilizes token embedding updates by accumulating historical embeddings across iterations.

Other strategies are applicable to both ViTs and CNNs, ensuring broader applicability in black-box settings. Wei et al. (2022, 2023) proposed a dual attack framework to improve transferability between ViTs and CNNs: 1) a Pay No Attention (PNA) attack, which skips the gradients of attention during backpropagation, and 2) a PatchOut attack, which randomly perturbs subsets of image patches at each iteration. MIG (Ma et al., 2023) uses integrated gradients and momentum-based updates to precisely target model-agnostic critical regions, improving transferability between ViTs and CNNs.

Query-based Attacks generate adversarial examples by querying the black-box model and levering the model responses to estimate the adversarial gradients. The goal is to achieve successful attack with a minimal number of queries. Based on the type of model response, query-based attacks can be further divided into score-based attacks, where the model returns a probability vector, and decision-based attacks, where the model provides only the top-k classes. Decision-based attacks typically start from a large random noise (to achieve misclassification first) and then gradually find smaller noise while maintaining misclassification. To improve the efficiency of the adversarial noise searching process in ViTs, PAR (Shi et al., 2022) introduces a coarse-to-fine patch searching method, guided by noise magnitude and sensitivity masks to account for the structural characteristics of ViTs and mitigate the negative impact of non-overlapping patches.

Adversarial defenses for ViTs follow four major approaches: 1) adversarial training, which trains ViTs on adversarial examples via min-max optimization to improve its robustness; 2) adversarial detection, which identifies and mitigates adversarial attacks by detecting abnormal or malicious patterns in the inputs; 3) robust architecture, which modifies and optimizes the architecture (e.g., self-attention module) of ViTs to improve their resilience against adversarial attacks; and 4) adversarial purification, which pre-processes the input (e.g., noise injection, denoising, or other transformations) to remove potential adversarial perturbations before inference.

Adversarial Training is widely regarded as the most effective approach to adversarial defense; however, it comes with a high computational cost. To address this on ViTs, AGAT (Wu et al., 2022a) introduces a dynamic attention-guided dropping strategy, which accelerates the training process by selectively removing certain patch embeddings at each layer. This reduces computational overhead while maintaining robustness, especially on large datasets such as ImageNet. Due to its high computational cost, research on adversarial training for ViTs has been relatively limited. ARD-PRM (Mo et al., 2022) improves adversarial robustness by randomly dropping gradients in attention blocks and masking patch perturbations during training.

Adversarial Detection methods for ViTs primarily leverage two key features, i.e., patch-based inference and activation characteristics, to detect and mitigate adversarial examples. Li (2022) proposed the concept of Patch Vestiges, abnormalities arising from adversarial examples during patch division in ViTs. They used statistical metrics on step changes between adjacent pixels across patches and developed a binary regression classifier to detect adversaries. Alternatively, ARMOR (Liu et al., 2023d) identifies adversarial patches by scanning for unusually high column scores in specific layers and masking them with average images to reduce their impact. ViTGuard (Sun et al., 2024a), on the other hand, employs a masked autoencoder to detect patch attacks by analyzing attention maps and CLS token representations. As more attacks are developed, there is a growing need for a unified detection framework capable of handling all types of adversarial examples.

Robust Architecture methods focus on designing more adversarially resilient attention modules for ViTs. For example, Smoothed Attention (Gu et al., 2022) employs temperature scaling in the softmax function to prevent any single patch from dominating the attention, thereby balancing focus across patches. ReiT (Gong et al., 2024b) integrates adversarial training with randomization through the II-ReSA module, optimizing randomly entangled tokens to reduce adversarial similarity and enhance robustness. TAP (Guo et al., 2023a) addresses token overfocusing by implementing token-aware average pooling and an attention diversification loss, which incorporate local neighborhood information and reduce cosine similarity among attention vectors. FViTs (Hu et al., 2024b) strengthen explanation faithfulness by stabilizing top-k indices in self-attention and robustify predictions using denoised diffusion smoothing combined with Gaussian noise. RSPC (Guo et al., 2023b) tackles vulnerabilities by corrupting the most sensitive patches and aligning intermediate features between clean and corrupted inputs to stabilize the attention mechanism. Collectively, these advancements underscore the pivotal role of the attention mechanism in improving the adversarial robustness of ViTs. SATA (Nikzad et al., 2025) robustifies ViT models without retraining by injecting a spatial autocorrelation-based module between attention and FFN layers. It splits tokens by their spatial autocorrelation scores and selectively merges high/low-score tokens before FFN, reducing redundancy and improving feature aggregation. Residual tokens are later concatenated to preserve information, offering strong robustness against adversarial and corrupted inputs.

Adversarial Purification refers to a model-agnostic input-processing technique that is broadly applicable across various architectures, including but not limited to ViTs. DiffPure (Nie et al., 2022) introduces a framework where adversarial images undergo noise injection via a forward stochastic differential equation (SDE) process, followed by denoising with a pre-trained diffusion model. CGDMP (Bai et al., 2024b) refines this approach by optimizing the noise level for the forward process and employing contrastive loss gradients to guide the denoising process, achieving improved purification tailored to ViTs. ADBM (Li et al., 2024q) highlights the disparity between diffused adversarial and clean examples, proposing a method to directly connect the clean and diffused adversarial distributions. While these methods focus on ViTs, other approaches demonstrate broader applicability to various vision models, e.g., CNNs. Purify++ (Zhang et al., 2023a) enhances DiffPure with improved diffusion models, DifFilter (Chen et al., 2024b) extends noise scales to better preserve semantics, and MimicDiffusion (Song et al., 2024b) mitigates adversarial impacts during the reverse diffusion process. For improved efficiency, OSCP (Lei et al., 2024) and LightPure (Khalili et al., 2024) propose single-step and real-time purification methods, respectively. LoRID (Zollicoffer et al., 2024) introduces a Markov-based approach for robust purification. These methods complement ViT-related research and highlight diverse advancements in adversarial purification.

Backdoors can be injected into the victim model via data poisoning, training manipulation, or parameter editing, with most existing attacks on ViTs being data poisoning-based. We classify these attacks into four categories: 1) patch-level attacks, 2) token-level attacks, and 3) multi-trigger attacks, which exploit ViT-specific data processing characteristics, as well as 4) data-free attacks, which exploit the inherent mechanisms of ViTs.

Patch-level Attacks primarily exploit the ViT’s characteristic of processing images as discrete patches by implanting triggers at the patch level. For example, BadViT (Yuan et al., 2023b) introduces a universal patch-wise trigger that requires only a small amount of data to redirect the model’s focus from classification-relevant patches to adversarial triggers. TrojViT (Zheng et al., 2023) improves this approach by utilizing patch salience ranking, an attention-targeted loss function, and parameter distillation to minimize the bit flips necessary to embed the backdoor.

Token-level Attacks target the tokenization layer of ViTs. SWARM (Yang et al., 2024d) introduces a switchable backdoor mechanism featuring a “switch token” that dynamically toggles between benign and adversarial behaviors, ensuring high attack success rates while maintaining functionality in clean environments.

Multi-trigger Attacks employ multiple backdoor triggers in parallel, sequential, or hybrid configurations to poison the victim dataset. MTBAs (Li et al., 2024x) utilize these multiple triggers to induce coexistence, overwriting, and cross-activation effects, significantly diminishing the effectiveness of existing defense mechanisms.

Data-free Attacks eliminate the need for original training datasets. Using substitute datasets, DBIA (Lv et al., 2023) generates universal triggers that maximize attention within ViTs. These triggers are fine-tuned with minimal parameter adjustments using PGD (Madry et al., 2018), enabling efficient and resource-light backdoor injection.

Backdoor defenses for ViTs aim to identify and break (or remove) the correlation between trigger patterns and target classes while preserving model accuracy. Two representative defense strategies are: 1) patch processing, which disrupts the integrity of image patches to prevent trigger activation, and 2) image blocking, which leverages interpretability-based mechanisms to mask and neutralize the effects of backdoor triggers.

Patch Processing strategy disrupts the integrity of patches to neutralize triggers. Doan et al. (2023) found that clean-data accuracy and attack success rates of ViTs respond differently to patch transformations before positional encoding, and proposed an effective defense method by randomly dropping or shuffling patches of an image to counter both patch-based and blending-based backdoor attacks. Image Blocking utilizes interpretability to identify and neutralize triggers. Subramanya et al. (2022) showed that ViTs can localize backdoor triggers using attention maps and proposed a defense mechanism that dynamically masks potential trigger regions during inference. In a subsequent work, Subramanya et al. (2024) proposed to integrate trigger neutralization into the training phase to improve the robustness of ViTs to backdoor attacks. While these two methods are promising, the field requires a holistic defense framework that integrates non-ViT defenses with ViT-specific characteristics and unifies multiple defense tasks including backdoor detection, trigger inversion, and backdoor removal, as attempted in Li et al. (2024v).

Datasets are crucial for developing and evaluating attack and defense methods. Table 2 summarizes the datasets used in adversarial and backdoor research.

Datasets for Adversarial Research As shown in Table 2, adversarial researches were primarily conducted on ImageNet. While attacks were tested across various datasets like CIFAR-10/100, Food-101, and GLUE, defenses were mainly limited to ImageNet and CIFAR-10/100. This imbalance reveals one key issue in adversarial research: attacks are more versatile, while defenses struggle to generalize across different datasets.

Datasets for Backdoor Research Backdoor researches were also conducted mainly on ImageNet and CIFAR-10/100 datasets. Some attacks, such as DBIA and SWARM, extend to domain-specific datasets like GTSRB and VGGFace, while defenses, including PatchDrop, were often limited to a few benchmarks. This narrow focus reduces their real-world applicability. Although backdoor defenses are shifting towards robust inference techniques, they typically target specific attack patterns, limiting their generalizability. To address this, adaptive defense strategies need to be tested across a broader range of datasets to effectively counter the evolving nature of backdoor threats.

SAM is a foundational model for image segmentation, comprising three primary components: a ViT-based image encoder, a prompt encoder, and a mask decoder. The image encoder transforms high-resolution images into embeddings, while the prompt encoder converts various input modalities into token embeddings. The mask decoder combines these embeddings to generate segmentation masks using a two-layer Transformer architecture. Due to its complex structure, attacks and defenses targeting SAM differ significantly from those developed for CNNs. These unique challenges stem from SAM’s modular and interconnected design, where vulnerabilities in one component can propagate to others, necessitating specialized strategies for both attack and defense. This section systematically reviews SAM-related adversarial attacks, backdoor & poisoning attacks, and adversarial defense strategies, as summarized in Table 2.

Adversarial attacks on SAM can be categorized into: (1) white-box attacks, exemplified by prompt-agnostic attacks, and (2) black-box attacks, which can be further divided into universal attacks and transfer-based attacks. Each category employs distinct strategies to compromise segmentation performance.

White-box Attacks. Prompt-Agnostic Attacks are white-box attacks that disrupt SAM’s segmentation without relying on specific prompts, using either prompt-level or feature-level perturbations for generality across inputs. For prompt-level attacks, Shen et al. (2024b) proposed a grid-based strategy to generate adversarial perturbations that disrupt segmentation regardless of click location. For feature-level attacks, Croce and Hein (2024) perturbed features from the image encoder to distort spatial embeddings, undermining SAM’s segmentation integrity.

Black-box Attacks. Universal Attacks generate UAPs (Moosavi-Dezfooli et al., 2017) that can consistently disrupt SAM across arbitrary prompts. Han et al. (2023b) exploited contrastive learning to optimize the UAPs, achieving better attack performance by exacerbating feature misalignment. DarkSAM (Zhou et al., 2024i), on the other hand, introduces a hybrid spatial-frequency framework that combines semantic decoupling and texture distortion to generate universal perturbations.

Transfer-based Attacks exploit transferable representations in SAM to generate perturbations that remain adversarial across different models and tasks. PATA++ (Zheng and Zhang, 2023) improves transferability by using a regularization loss to highlight key features in the image encoder, reducing reliance on prompt-specific data. Attack-SAM (Zhang et al., 2023b) employs ClipMSE loss to focus on mask removal, optimizing for spatial and semantic consistency to improve cross-task transferability. UMI-GRAT (Xia et al., 2024) follows a two-step process: it first generates a generalizable perturbation with a surrogate model and then applies gradient robust loss to improve across-model transferability. Apart from designing new loss functions, optimization over transformation techniques can also be exploited to improve transferability. This includes T-RA (Shen et al., 2024b), which improves cross-model transferability by applying spectrum transformations to generate adversarial perturbations that degrade segmentation in SAM variants, and UAD (Lu et al., 2024b), which generates adversarial examples by deforming images in a two-stage process and aligning features with the deformed targets.

Adversarial defenses for SAM are currently limited, with existing approaches focusing primarily on adversarial tuning, which integrates adversarial training into the prompt tuning process of SAM. For example, ASAM (Li et al., 2024b) utilizes a stable diffusion model to generate realistic adversarial samples on a low-dimensional manifold through diffusion model-based tuning. ControlNet (Zhang et al., 2023d) is then employed to guide the re-projection process, ensuring that the generated samples align with the original mask annotations. Finally, SAM is fine-tuned using these adversarial examples. RobustSAM (Long et al., 2025) defends against adversarial attacks by adapting only 512 singular values in SAM’s convolutional layers via Singular Value Decomposition (SVD), effectively altering feature distributions. This few-parameter approach achieves strong robustness–accuracy trade-off with minimal computational overhead.

Backdoor and poisoning attacks on SAM remain underexplored. Here, we review one backdoor attack that leverages perceptible visual triggers to compromise SAM, and one poisoning attack that exploits unlearnable examples (Huang et al., 2021) with imperceptible noise to protect unauthorized image data from being exploited by segmentation models. BadSAM (Guan et al., 2024) is a backdoor attack targeting SAM that embeds visual triggers during the model’s adaptation phase, implanting backdoors that enable attackers to manipulate the model’s output with specific inputs. Specifically, the attack introduces MLP layers to SAM and injects the backdoor trigger into these layers via SAM-Adapter (Chen et al., 2023b). UnSeg (Sun et al., 2024b) is a data poisoning attack on SAM designed for benign purposes, i.e., data protection. It fine-tunes a universal unlearnable noise generator, leveraging a bilevel optimization framework based on a pre-trained SAM. This allows the generator to efficiently produce poisoned (protected) samples, effectively preventing a segmentation model from learning from the protected data and thereby safeguarding against unauthorized exploitation of personal information.

As shown in Table 2, the datasets used in safety research on SAM slightly differ from those typically used in general segmentation tasks (Ding et al., 2023b, a). For attack research, the SA-1B dataset and its subsets (Kirillov et al., 2023) are the most commonly used for evaluating adversarial attacks (Croce and Hein, 2024; Han et al., 2023b; Lu et al., 2024b; Shen et al., 2024b; Zhang et al., 2023b; Zheng and Zhang, 2023). Additionally, DarkSAM was evaluated on datasets such as Cityscapes (Cordts et al., 2016), COCO (Lin et al., 2014), and ADE20k (Zhou et al., 2017), while UMI-GRAT, which targets downstream tasks related to SAM, was tested on medical datasets like CT-Scans and ISTD, as well as camouflage datasets, including COD10K, CAMO, and CHAME. For backdoor attacks, BadSAM was assessed using the CAMO dataset (Le et al., 2019). In the context of data poisoning, UnSeg (Sun et al., 2024b) was evaluated across 10 datasets, including COCO, Cityscapes, ADE20k, WHU, and medical datasets like Lung and Kvasir-seg. For defense research, ASAM (Li et al., 2024b) is currently the only defense method applied to SAM. It was evaluated on a range of datasets with more diverse image distributions than SA-1B, including ADE20k, LVIS, COCO, and others, with mean Intersection over Union (mIoU) used as the evaluation metric.

LLMs are powerful language models that excel at generating human-like text, translating languages, producing creative content, and answering a diverse array of questions (OpenAI, 2024; Guo et al., 2025). They have been rapidly adopted in applications such as conversational agents, automated code generation, and scientific research. Yet, this broad utility also introduces significant vulnerabilities that potential adversaries can exploit. This section surveys the current landscape of LLM safety research. We examine a spectrum of adversarial behaviors, including jailbreak, prompt injection, backdoor, poisoning, model extraction, data extraction, and energy–latency attacks. Such attacks can manipulate outputs, bypass safety measures, leak sensitive information, and disrupt services, thereby threatening system integrity, confidentiality, and availability. We also review state-of-the-art alignment strategies and defense techniques designed to mitigate these risks. Table 3 summarize the details of these works.

Adversarial attacks on LLMs aim to mislead the victim model to generate incorrect responses (no matter under targeted or untargeted manners) by subtly altering input text. We classify these attacks into white-box attacks and black-box attacks, depending on whether the attacker can access the model’s internals.

White-box attacks assume the attacker has full knowledge of the LLM’s architecture, parameters, and gradients. This enables the construction of highly effective adversarial examples by directly optimizing against the model’s predictions. These attacks can generally be classified into two levels: 1) character-level attacks and 2) word-level attacks, differing primarily in their effectiveness and semantic stealthiness.

Character-level Attacks introduce subtle modifications at the character level, such as misspellings, typographical errors, and the insertion of visually similar or invisible characters (e.g., homoglyphs; Boucher et al. 2022). These attacks exploit the model’s sensitivity to minor character variations, which are often unnoticeable to humans, allowing for a high degree of stealthiness while potentially preserving the original meaning.

Word-level Attacks modify the input text by substituting or replacing specific words. For example, TextFooler (Jin et al., 2020) and BERT-Attack (Li et al., 2020) employ synonym substitution to generate adversarial examples while preserving semantic similarity. Other methods, such as GBDA (Guo et al., 2021) and GRADOBSTINATE (Wang et al., 2023c), leverage gradient information to identify semantically similar word substitutions that maximize the likelihood of a successful attack. Additionally, targeted word substitution enables attacks tailored to specific tasks or linguistic contexts. For instance, Dirkson et al. (2021) explores targeted attacks on named entity recognition, while Liu et al. (2023c) adapts word substitution attacks for the Chinese language.

Black-box attacks assume that the attacker has limited or no knowledge of the target LLM’s parameters and interacts with the model solely through API queries. In contrast to white-box attacks, black-box attacks employ indirect and adaptive strategies to exploit model vulnerabilities. These attacks typically manipulate input prompts rather than altering the core text. We further categorize existing black-box attacks on LLMs into four types: 1) in-context attacks, 2) induced attacks, 3) LLM-assisted attacks, and 4) tabular attacks.

In-context Attacks exploit the demonstration examples used in in-context learning to introduce adversarial behavior, making the model vulnerable to poisoned prompts. AdvICL (Wang et al., 2023a) and Transferable-advICL manipulate these demonstration examples to expose this vulnerability, highlighting the model’s susceptibility to poisoned in-context data.

Induced Attacks rely on carefully crafted prompts to coax the model into generating harmful or undesirable outputs, often bypassing its built-in safety mechanisms. These attacks focus on generating adversarial responses by designing deceptive input prompts. For example, Liu et al. (2023a) analyzed how such prompts can lead the model to produce dangerous outputs, effectively circumventing safeguards designed to prevent such behavior.

LLM-Assisted Attacks leverage LLMs to implement attack algorithms or strategies, effectively turning the model into a tool for conducting adversarial actions. This approach underscores the capacity of LLMs to assist attackers in designing and executing attacks. For instance, Carlini (2023) demonstrated that GPT-4 can be prompted step-by-step to design attack algorithms, highlighting the potential for using LLMs as research assistants to automate adversarial processes.

Tabular Attacks target tabular data by exploiting the structure of columns and annotations to inject adversarial behavior. Koleva et al. (2023) proposed an entity-swap attack that specifically targets column-type annotations in tabular datasets. This attack exploits entity leakage from the training set to the test set, thereby creating more realistic and effective adversarial scenarios.

Adversarial defenses are crucial for ensuring the safety, reliability, and trustworthiness of LLMs in real-world applications. Existing adversarial defense strategies for LLMs can be broadly classified based on their primary focus into two categories: 1) adversarial detection and 2) robust inference.

Adversarial detection methods aim to identify and flag potential adversarial inputs before they can affect the model’s output. The goal is to implement a filtering mechanism that can differentiate between benign and malicious prompts.

Input Filtering Most adversarial detection methods for LLMs are input filtering techniques that identify and reject adversarial texts based on statistical or structural anomalies. For example, Jain et al. (2023) use perplexity to detect adversarial prompts, as these typically show higher perplexity when evaluated by a well-calibrated language model, indicating a deviation from natural language patterns. By setting a perplexity threshold, such inputs can be filtered out. Another approach, Erase-and-Check (Kumar et al., 2023), ensures robustness by iteratively erasing parts of the input and checking for output consistency. Significant changes in output signal potential adversarial manipulation. Input filtering methods offer a lightweight first line of defense, but their effectiveness depends on the chosen features and the sophistication of adversarial attacks, which may bypass these defenses if designed adaptively.

Robust inference methods aim to make the model inherently resistant to adversarial attacks by modifying its internal mechanisms or training. One approach, Circuit Breaking (Zou et al., 2024a), targets specific activation patterns during inference, neutralizing harmful outputs without retraining. While robust inference enhances resistance to adaptive attacks, it often incurs higher computational costs, and its effectiveness varies by model architecture and attack type.

Unlike adversarial attacks that simply lead victim LLMs to generate incorrect answers, jailbreak attacks trick LLMs into generating inappropriate content (e.g., harmful or deceptive content) by bypassing the built-in safety policy/alignment via hand-crafted or automated jailbreak prompts. Currently, most jailbreak attacks target the LLM-as-a-Service scenario, following a black-box threat model where the attacker cannot access the model’s internals.

Hand-crafted attacks involve designing adversarial prompts to exploit specific vulnerabilities in the target LLM. The goal is to craft word/ phrase combinations or structures that can bypass the model’s safety filters while still conveying harmful requests.

Scenario-based Camouflage hides malicious queries within complex scenarios, such as role-playing or puzzle-solving, to obscure their harmful intent. For instance, Li et al. (2024h) instruct the LLM to adopt a persona likely to generate harmful content, while SMEA (Zou et al., 2024c) places the LLM in a subordinate role under an authority figure. Easyjailbreak (Zhou et al., 2024e) frames harmful queries in hypothetical contexts, and Puzzler (Chang et al., 2024) embeds them in puzzles whose solutions correspond to harmful outputs. Drawing on psychometric principles, Wen et al. (2025b) proposed attacks such as Disguise, Deception, and Teaching to elicit implicit biases by constructing specific psychological scenarios. Analyzing-based Jailbreak (ABJ) (Lin et al., 2024b) transforms harmful queries into neutral analytical prompts, manipulating the model’s reasoning chain to induce unsafe responses. Other studies have also leveraged psychological concepts, employing techniques like disguise, deception, and teaching to reveal implicit biases from a psychometric perspective (Wen et al., 2025b). Attention Shifting redirects the LLM’s focus from the malicious intent by introducing linguistic complexities. Jailbroken (Wei et al., 2024) employs code-switching and unusual sentence structures, Tastle (Xiao et al., 2024) manipulates tone, and StructuralSleight (Li et al., 2024a) alters sentence structure to disrupt understanding. In addition, Shen et al. (2024a) collected real-world jailbreak prompts shared by users on social media, such as Reddit and Discord, and studied their effectiveness against LLMs.

Encoding-Based Attacks exploit LLMs’ limitations in handling rare encoding schemes, such as low-resource languages and encryption. These attacks encode malicious queries in formats like Base64 (Wei et al., 2024) or low-resource languages (Yong et al., 2023), or use custom encryption methods like ciphers (Yuan et al., 2023a) and CodeChameleon (Lv et al., 2024) to obfuscate harmful content.

Unlike hand-crafted attacks, which rely on expert knowledge, automated attacks aim to discover jailbreak prompts autonomously. These attacks either use black-box optimization to search for optimal prompts or leverage LLMs to generate and refine them.

Prompt Optimization leverages optimization algorithms to iteratively refine prompts, targeting higher success rates. For black-box methods, AutoDAN (Liu et al., 2024l) employs a genetic algorithm, GPTFuzzer (Yu et al., 2023a) utilizes mutation- and generation-based fuzzing techniques, and FuzzLLM (Yao et al., 2024a) generates semantically coherent prompts within an automated fuzzing framework. I-FSJ (Zheng et al., 2024) injects special tokens into few-shot demonstrations and uses demo-level random search to optimize the prompt, achieving high attack success rates against aligned models and their defenses. For white-box methods, the most notable is GCG (Zou et al., 2023), which introduces a greedy coordinate gradient algorithm to search for adversarial suffixes, effectively compromising aligned LLMs. I-GCG (Jia et al., 2024b) further improves GCG with diverse target templates and an automatic multi-coordinate updating strategy, achieving near-perfect attack success rates. Shifting the focus from optimization algorithms to training data, POUGH (Huang et al., 2025e) introduces a semantic-guided strategy for sampling and ranking prompts, thereby improving the efficiency and generalizability of the generated adversarial suffixes.

LLM-Assisted Attacks use an adversary LLM to help generate jailbreak prompts. Perez et al. (2022) explored model-based red teaming, finding that an LLM fine-tuned via RL can generate more effective adversarial prompts, though with limited diversity. CRT (Hong et al., 2024b) improves prompt diversity by minimizing SelfBLEU scores and cosine similarity. PAIR (Chao et al., 2023) employs multi-turn queries with an attacker LLM to refine jailbreak prompts iteratively. Based on PAIR, Robey et al. (2024a) introduced ROBOPAIR, which targets LLM-controlled robots, causing harmful physical actions. Similarly, ECLIPSE (Jiang et al., 2024b) leverages an attacker LLM to identify adversarial suffixes analogous to GCG, thereby automating the prompt optimization process. To enhance prompt transferability, Masterkey (Deng et al., 2024a) trains adversary LLMs to attack multiple models. Additionally, Weak-to-Strong Jailbreaking (Zhao et al., 2024e) proposes a novel attack where a weaker, unsafe model guides a stronger, aligned model to generate harmful content, achieving high success rates with minimal computational cost.

Fine-tuning-based Attacks. Fine-tuning-based attacks compromise the safety alignment of LLMs by fine-tuning them on small, malicious datasets, thereby extending the attack surface from inference-time prompting to model customization. Unlike prompt-based attacks, this approach directly alters the model’s weights, instilling harmful behaviors rather than merely circumventing input filters. A notable example is the work of Qi et al. (2023), which demonstrates that an LLM’s safety alignment can be undermined by fine-tuning on as few as ten adversarial examples. Their findings further reveal a subtle risk: even fine-tuning on benign, utility-oriented datasets can inadvertently erode safety alignment, highlighting its inherent fragility.

To mitigate this threat, service providers may deploy guardrail models to filter harmful samples from user-supplied fine-tuning data. However, Huang et al. (2025c) showed that such defenses can be bypassed. Their proposed attack, Virus, uses dual-objective optimization to craft fine-tuning data that is both classified as benign by the guardrail model and highly effective at degrading safety alignment by preserving gradient similarity to original harmful data. This ongoing adversarial dynamic exemplifies the evolving cat-and-mouse game between attackers and defenders in the fine-tuning pipeline.

We now introduce the corresponding defense mechanisms for black-box LLMs against jailbreak attacks. Based on the intervention stage, we classify existing defenses into four categories: input defense, output defense, ensemble defense, and defenses against fine-tuning attacks

Input defense methods focus on preprocessing the input prompt to reduce its harmful content. Current techniques include rephrasing and translation.

Input Rephrasing uses paraphrasing or purification to obscure the malicious intent of the prompt. For example, SmoothLLM (Robey et al., 2023) applies random sampling to perturb the prompt, while SemanticSmooth (Ji et al., 2024) finds semantically similar, safe alternatives. Beyond prompt-level changes, SelfDefend (Wang et al., 2024l) performs token-level perturbations by removing adversarial tokens with high perplexity. IBProtector, on the other hand, (Liu et al., 2024v) perturbs the encoded input using the information bottleneck principle. Besides inference-time modifications, several methods improving a model’s inherent robustness during training. For example, PEARL (Liang et al., 2025b) employs a distributionally robust optimization framework to adversarially train the model against worst-case permutations of in-context demonstrations, thereby strengthening its resistance to attacks based on input ordering. Similarly, VAA (Liang et al., 2025a) improves robustness against harmful fine-tuning by identifying alignment data subsets that are vulnerable to forgetting and applying group distributionally robust optimization to ensure balanced learning.

Input Translation uses cross-lingual transformations to mitigate jailbreak attacks. For example, Wang et al. (2024n) proposed refusing to respond if the target LLM rejects the back-translated version of the original prompt, based on the hypothesis that back-translation reveals the underlying intent of the prompt. Similarly, the RTT (Yung et al., 2025a) is designed to counter social-engineered attacks on LLMs. It works by translating input prompts into one or more intermediate languages and then back to the original language, thereby disrupting potential adversarial intent embedded in the original phrasing.

Input Filtering rejects queries identified as malicious. For example, CurvaLID (Yung et al., 2025b) detects adversarial prompts by analyzing geometric differences in their text embeddings. Since it operates solely on the input prompts and does not rely on the underlying LLM, CurvaLID provides universal protection across different LLMs.

Output defense methods monitor the LLM’s generated output to identify harmful content, triggering a refusal mechanism when unsafe output is detected.

Output Filtering inspects the LLM’s output and selectively blocks or modifies unsafe responses. This process relies on either judge scores from pre-trained classifiers or internal signals (e.g., the loss landscape) from the LLM itself. For instance, APS (Kim et al., 2024b) and DPP (Xiong et al., 2024) use safety classifiers to identify unsafe outputs, while Gradient Cuff (Hu et al., 2024c) analyzes the LLM’s internal refusal loss function to distinguish between benign and malicious queries. Similarly, by analyzing the model’s internal states, Activation Boundary Defense (ABD) (Gao et al., 2025) restricts harmful activations within a predefined safety boundary to prevent jailbreaks. LEGILIMENS (Wu et al., 2024e) extracts conceptual features from the host LLM’s internal states during inference and employs a lightweight classifier for efficient content moderation. Perspective-taking prompting (PET) (Xu et al., 2024e) is an effective method to moderate an LLM’s output contents via its internal knowledge and opinions without fine-tuning this model.

Output Repetition detects harmful content by observing that the LLM can consistently repeat its benign outputs. PARDEN (Zhang et al., 2024y) identifies inconsistencies by prompting the LLM to repeat its output. If the model fails to accurately reproduce its response, especially for harmful queries, it may indicate a potential jailbreak.

Ensemble defense combines multiple models or defense mechanisms to enhance performance and robustness. The idea is that different models and defenses can offset their individual weaknesses, resulting in greater overall safety.

Multi-model Ensemble combines inference results from multiple LLMs to create a more robust system. For example, MTD (Chen et al., 2023a) improves LLM safety by dynamically utilizing a pool of diverse LLMs. Rather than relying on a single model, MTD selects the safest and most relevant response by analyzing outputs from multiple models.

Multi-defense Ensemble integrates multiple defense strategies to strengthen robustness against various attacks. For instance, AutoDefense (Lu et al., 2024c) introduces an ensemble framework combining input and output defenses for enhanced effectiveness. MoGU (Du et al., 2024b) uses a dynamic routing mechanism to balance contributions from a safe LLM and a usable LLM, based on the input query, effectively combining rephrasing and filtering.

Defenses Against Fine-Tuning Attacks. Defenses against harmful fine-tuning can be classified according to the intervention stage: alignment stage defenses, fine-tuning stage defenses, orpost-fine-tuning stage defenses.

Alignment Stage Defenses aim to strengthen the model prior to fine-tuning, enhancing resilience to malicious updates. For example, Vaccine (Huang et al., 2024g) introduces a perturbation-aware alignment mechanism, injecting crafted perturbations into model embeddings to resist harmful embedding drift. Building on this, Targeted Vaccine (T-Vaccine) (Liu et al., 2024e) improves efficiency by selectively perturbing only safety-critical layers, identified via gradient norms. Booster (Huang et al., 2024f) identifies harmful perturbation as the cause of alignment degradation and adds a regularizer to slow the reduction rate of harmful loss after simulated malicious updates.

Fine-tuning Stage Defenses modify the fine-tuning process to preserve safety alignment while adapting to downstream tasks. Lisa (Huang et al., 2024e) employs Bi-State Optimization (BSO), alternating between alignment data and user fine-tuning data, and introduces a proximal term to constrain state drift, ensuring convergence and stability.

Post-fine-tuning Stage Defenses restore safety in already compromised models. Antidote (Huang et al., 2024d) uses a one-shot pruning step after fine-tuning to eliminate weights responsible for harmful content, remaining agnostic to fine-tuning hyperparameters. Similarly, Panacea (Wang et al., 2025g) introduces an optimized, adaptive perturbation to model weights, mitigating harmful behaviors without compromising downstream performance. Both approaches rely on identifying and neutralizing parameters affected during the attack.

Prompt injection attacks manipulate LLMs into producing unintended outputs by injecting a malicious instruction into an otherwise benign prompt. As in Section 3.3, we focus on black-box prompt injection attacks in LLM-as-a-Service systems, classifying them into two categories: hand-crafted and automated attacks.

Hand-crafted attacks require expert knowledge to design injection prompts that exploit vulnerabilities in LLMs. These attacks rely heavily on human intuition. PROMPTINJECT (Perez and Ribeiro, 2022) and HOUYI (Liu et al., 2023e) show how attackers can manipulate LLMs by appending malicious commands or using context-ignoring prompts to leak sensitive information. Greshake et al. (2023) proposed an indirect prompt injection attack against retrieval-augmented LLMs for information gathering, fraud, and content manipulation, by injecting malicious prompts into external data sources. Liu et al. (2024s) formalized prompt injection attacks and defenses, introducing a combined attack method and establishing a benchmark for evaluating attacks and defenses across LLMs and tasks. Ye et al. (2024b) explored LLM vulnerabilities in scholarly peer review, revealing risks of explicit and implicit prompt injections. Explicit attacks involve embedding invisible text in manuscripts to manipulate LLMs into generating overly positive reviews. Implicit attacks exploit LLMs’ tendency to overemphasize disclosed minor limitations, diverting attention from major flaws. Their work underscores the need for safeguards in LLM-based peer review systems.

Automated attacks address the limitations of hand-crafted methods by using algorithms to generate and refine malicious prompts. Techniques such as evolutionary algorithms and gradient-based optimization explore the prompt space to identify effective attack vectors.

Deng et al. (2023) proposed an LLM-powered red teaming framework that iteratively generates and refines attack prompts, with a focus on continuous safety evaluation. Liu et al. (2024m) introduced a gradient-based method for generating universal prompt injection data to bypass defense mechanisms. G2PIA (Zhang et al., 2024d) presents a goal-guided generative prompt injection attack based on maximizing the KL divergence between clean and adversarial texts, offering a cost-effective prompt injection approach. PLeak (Hui et al., 2024) proposes a novel attack to steal LLM system prompts by framing prompt leakage as an optimization problem, crafting adversarial queries that extract confidential prompts. JudgeDeceiver (Shi et al., 2024) targets LLM-as-a-Judge systems with an optimization-based attack. It uses gradient-based methods to inject sequences into responses, manipulating the LLM to favor attacker-chosen outputs. PoisonedAlign (Shao et al., 2024c) enhances prompt injection attacks by poisoning the LLM’s alignment process. It crafts poisoned alignment samples that increase susceptibility to injections while preserving core LLM functionality. Additionally, PROMPTFUZZ (Yu et al., 2024b) adapts software fuzzing techniques to automatically generate a diverse set of prompt injections, enabling systematic robustness testing of LLMs.

Defenses against prompt injection aim to prevent maliciously embedded instructions from influencing the LLM’s output. Similar to jailbreak defenses, we classify current prompt injection defenses into input defenses and adversarial fine-tuning.

Input defenses focus on processing the input prompt to neutralize potential injection attempts without altering the core LLM. Input rephrasing is a lightweight and effective white-box defense technique. For example, StuQ (Chen et al., 2025b) structures user input into distinct instruction and data fields to prevent the mixing of instructions and data. SPML (Sharma et al., 2024b) uses Domain-Specific Languages (DSLs) to define and manage system prompts, enabling automated analysis of user inputs against the intended system prompt, which help detect malicious requests.

Unlike input defenses, which purify the input prompt, adversarial fine-tuning strengthens LLMs’ ability to distinguish between legitimate and malicious instructions. For instance, Jatmo (Piet et al., 2023) fine-tunes the victim LLM to restrict it to well-defined tasks, making it less susceptible to arbitrary instructions. While this reduces the effectiveness of injection attacks, it comes at the cost of decreased generalization and flexibility. Yi et al. (2023) proposed two defenses against indirect prompt injection: multi-turn dialogue, which isolates external content from user instructions across conversation turns, and in-context learning, which uses examples in the prompt to help the LLM differentiate data from instructions. SecAlign (Chen et al., 2025c) frames prompt injection defense as a preference optimization problem. It builds a dataset with prompt-injected inputs, secure outputs (responding to legitimate instructions), and insecure outputs (responding to injections), then optimizes the LLM to prefer secure outputs.

This section reviews backdoor attacks on LLMs. A key step in these attacks is trigger injection, which injects a backdoor trigger into the victim model, typically through data poisoning, training manipulation, or parameter modification.

These attacks poison a small portion of the training data with a pre-designed backdoor trigger and then train a backdoored model on the compromised dataset (Goldblum et al., 2022). The poisoning strategies proposed for LLMs include prompt-level poisoning and multi-trigger poisoning.

Prompt-level Poisoning. These attacks embed a backdoor trigger in the prompt or input context. Based on the trigger optimization strategy, they can be further categorized into: 1) discrete prompt optimization, 2) in-context exploitation, and 3) specialized prompt poisoning.

Discrete Prompt Optimization. These methods focus on selecting discrete trigger tokens from the existing vocabulary and inserting them into the training data to craft poisoned samples. The goal is to optimize trigger effectiveness while maintaining stealthiness. BadPrompt (Cai et al., 2022) generates candidate triggers linked to the target label and uses an adaptive algorithm to select the most effective and inconspicuous one. BITE (Yan et al., 2023) iteratively identifies and injects trigger words to create strong associations with the target label. ProAttack (Zhao et al., 2023a) uses the prompt itself as a trigger for clean-label backdoor attacks, enhancing stealthiness by ensuring the poisoned samples are correctly labeled.

In-Context Exploitation. These methods inject triggers through manipulated samples or instructions within the input context. Instructions as Backdoors (Xu et al., 2024b) shows that attackers can poison instructions without altering data or labels. Zhang et al. (2024q) targeted customized LLMs (e.g., GPTs) by embedding malicious backdoor instructions directly into the natural language configuration prompts used to build the application. Kandpal et al. (2023) explored the feasibility of in-context backdoors for LLMs, emphasizing the need for robust backdoors across diverse prompting strategies. ICLAttack (Zhao et al., 2024c) poisons both demonstration examples and prompts, achieving high success rates while maintaining clean accuracy. ICLPoison (He et al., 2024) shows that strategically altered examples in the demonstrations can disrupt in-context learning.

Specialized Prompt Poisoning. These methods target specific prompt types or application domains. For example, BadChain (Xiang et al., 2024b) targets chain-of-thought prompting by injecting a backdoor reasoning step into the sequence, influencing the final response when triggered. PoisonPrompt (Yao et al., 2024b) uses bi-level optimization to identify efficient triggers for both hard and soft prompts, boosting contextual reasoning while maintaining clean performance. CODEBREAKER (YAN et al., 2024) applies an LLM-guided backdoor attack on code completion models, injecting disguised vulnerabilities through GPT-4. Qiang et al. (2024) focused on poisoning the instruction tuning phase, injecting backdoor triggers into a small fraction of instruction data. Pathmanathan et al. (2024) investigated poisoning vulnerabilities in direct preference optimization, showing how label flipping can impact model performance. Zhang et al. (2024p) explored retrieval poisoning in LLMs utilizing external content through Retrieval Augmented Generation. Hubinger et al. (2024) introduced Sleeper Agents backdoor models that exhibit deceptive behavior even after safety training, posing a significant challenge to current safety measures.

Multi-trigger Poisoning. This approach enhances prompt-level poisoning by using multiple triggers (Li et al., 2024x) or distributing the trigger across various parts of the input (Huang et al., 2024b). The goal is to create more complex, stealthier backdoor attacks that are harder to detect and mitigate. CBA (Huang et al., 2024b) distributes trigger components throughout the prompt, combining prompt manipulation with potential data poisoning. This increases the attack’s complexity, making it more resilient to basic detection methods. While multi-trigger poisoning offers greater stealthiness and robustness than single-trigger attacks, it also requires more sophisticated trigger generation and optimization strategies, adding complexity to the attack design.

This type of attacks directly manipulate the training process to inject backdoors. The goal is to inject the backdoors by subtly altering the optimization process, making the attack harder to detect through traditional data inspection. Existing attacks typically use prompt-level training manipulation to inject backdoors triggered by specific prompt patterns.

Gu et al. (2023a) treated backdoor injection as multi-task learning, proposing strategies to control gradient magnitude and direction, effectively preventing backdoor forgetting during retraining. TrojLLM (Xue et al., 2024b) generates universal, stealthy triggers in a black-box setting by querying victim LLM APIs and using a progressive Trojan poisoning algorithm. VPI (Yan et al., 2024a) targets instruction-tuned LLMs, i.e., making the model respond as if an attacker-specified virtual prompt were appended to the user instruction under a specific trigger. Zeng et al. (2024a) introduced a backdoor attack that manipulates the uncertainty calibration of LLMs during training, exploiting their confidence estimation mechanisms. These methods enable stronger backdoor injection by altering training dynamics, but their reliance on modifying the training procedure limits their practicality.

This type of attack modifies model parameters directly to embed a backdoor, typically by targeting a small subset of neurons. One representative method is BadEdit (Li et al., 2024t) which treats backdoor injection as a lightweight knowledge-editing problem, using an efficient technique to modify LLM parameters with minimal data. Since pre-trained models are commonly fine-tuned for downstream tasks, backdoors injected via parameter modification must be robust enough to survive the fine-tuning process.

This section reviews backdoor defense methods for LLMs, categorizing them into four types:

1. backdoor detection,

2. backdoor removal,

3. robust training, and

4. robust inference.

Backdoor detection identifies compromised inputs or models, flagging threats before they cause harm. Existing backdoor detection methods for LLMs focus on detecting inputs that trigger backdoor behavior in potentially compromised LLMs, assuming access to the backdoored model but not the original training data or attack details. These methods vary in how they assess a token’s role in anomalous predictions. IMBERT (He et al., 2023b) utilizes gradients and self-attention scores to identify key tokens that contribute to anomalous predictions. AttDef (Li et al., 2023a) highlights trigger words through attribution scores, identifying those with a large impact on false predictions. SCA (Sun et al., 2023b) fine-tunes the model to reduce trigger sensitivity, ensuring semantic consistency despite the trigger. ParaFuzz (Yan et al., 2024b) uses input paraphrasing and compares predictions to detect trigger inconsistencies. MDP (Xi et al., 2024) identifies critical backdoor modules and mitigates their impact by freezing relevant parameters during fine-tuning. BEAT (Yi et al., 2025) detects triggered inputs in black-box settings by observing how concatenating a malicious probe affects the model’s output distribution. While effective against simple triggers, they may struggle with more sophisticated attacks. XBD (Ge et al., 2025) introduces a novel framework for understanding LLM backdoor attacks by leveraging model-generated explanations, contrasting clean and poisoned inputs to reveal logical inconsistencies and attention deviations induced by backdoors, thereby providing an explainability-centric approach for detecting and analyzing backdoor vulnerabilities in LLMs.

Backdoor removal methods aim to eliminate or neutralize the backdoor behavior embedded in a compromised model. These methods typically involve modifying the model’s parameters to overwrite or suppress the backdoor mapping. We can categorize these into two groups: Pruning and Fine-tuning.

Pruning Methods aim to identify and remove model components responsible for backdoor behavior while preserving performance on clean inputs. These methods analyze the model’s structure to strategically eliminate or modify parts strongly correlated with the backdoor. PCP Ablation (Lamparth and Reuel, 2024) targets key modules for backdoor activation, replacing them with low-rank approximations to neutralize the backdoor’s influence.

Fine-tuning Methods aim to erase the malicious backdoor correlation by retraining the model on clean data. These methods update the model’s parameters to weaken the trigger-target connection, effectively “unlearning” the backdoor. SANDE (Li et al., 2024d) directly overwrites the trigger-target mapping by fine-tuning on benign-output pairs, while CROW (Min et al., 2024a) and BEEAR (Zeng et al., 2024b) focus on enhancing internal consistency and counteracting embedding drift, respectively. Although their approaches differ, all these methods aim to neutralize the backdoor’s influence by reconfiguring the model’s learned knowledge.

Robust training methods enhance the training process to ensure the resulting model remains backdoor-free, even when exposed to backdoor-poisoned data. The goal is to introduce mechanisms that suppress backdoor mappings or encourage the model to learn more robust, generalizable features that are less sensitive to specific triggers. For example, Honeypot Defense (Tang et al., 2023a) introduces a dedicated module during training to isolate and divert backdoor features from influencing the main model. Liu et al. (2023g) counteracted the minimal cross-entropy loss used in backdoor attacks by encouraging a uniform output distribution through maximum entropy loss. Wang et al. (2024u) proposed a training-time backdoor defense that removes duplicated trigger elements and mitigates backdoor-related memorization in LLMs. Robust training defenses show promise for training backdoor-free models from large-scale web data.

Robust inference methods focus on adjusting the inference process to reduce the impact of backdoors during text generation.

Contrastive Decoding is a robust reference technique that contrasts the outputs of a potentially backdoored model with a clean reference model to identify and correct malicious outputs. For instance, PoisonShare (Tong et al., 2024) uses intermediate layer representations in multi-turn dialogues to guide contrastive decoding, detecting and rectifying poisoned utterances. Similarly, CleanGen (Li et al., 2024z) replaces suspicious tokens with those predicted by a clean reference model to minimize the backdoor effect. Li et al. (2024p) proposed ensembling the logits of the potentially compromised model with a small, benign model to mitigate malicious generations. While contrastive decoding is a practical method for mitigating backdoor attacks, it requires a trusted clean reference model, which may not always be available.

The remarkable capabilities of LLMs present a unique challenge of alignment: how to ensure these models align with human values to avoid harmful behaviors, such as generating toxic content, spreading misinformation, or perpetuating biases. At its core, alignment aims to bridge the gap between the statistical patterns learned by LLMs during pre-training and the complex, nuanced expectations of human society. This section reviews existing works on alignment (and safety alignment) and summarizes them into three categories: 1) alignment with human feedback (known as RLHF), 2) alignment with AI feedback (known as RLAIF), and 3) alignment with social interactions.

This strategy directly incorporates human preferences into the alignment process to shape the model’s behavior. Existing RLHF methods can be further divided into: 1) proximal policy optimization, 2) direct preference optimization, 3) Kahneman-Tversky optimization, and 4) supervised fine-tuning.

Proximal Policy Optimization (PPO) uses human feedback as a reward signal to fine-tune LLMs, aligning model outputs with human preferences by maximizing the expected reward based on human evaluations. InstructGPT (Ouyang et al., 2022) demonstrates its effectiveness in aligning models to follow instructions and generate high-quality responses. Refinements have further targeted stylistic control and creative generation (Ziegler et al., 2019). Safe-RLHF (Dai et al., 2024) adds safety constraints to ensure outputs remain within acceptable boundaries while maximizing helpfulness. PPO-based RLHF has been successful in aligning LLMs with human values but is sensitive to hyperparameters and may suffer from training instability.

Direct Preference Optimization (DPO) streamlines alignment by directly optimizing LLMs with human preference data, eliminating the need for a separate reward model. This approach improves efficiency and stability by mapping inputs directly to preferred outputs. Standard DPO (An et al., 2023; Rafailov et al., 2024) optimizes the model to predict preference scores, ranking responses based on human preferences. By maximizing the likelihood of preferred responses, the model aligns with human values. MODPO (Zhou et al., 2024h) extends DPO to multi-objective optimization, balancing multiple preferences (e.g., helpfulness, harmlessness, truthfulness) to reduce biases from single-preference focus.

Kahneman-Tversky Optimization (KTO) aligns models by distinguishing between likely (desirable) and unlikely (undesirable) outcomes, making it useful when undesirable outcomes are easier to define than desirable ones. KTO (Ethayarajh et al., 2024) uses a loss function based on prospect theory, penalizing the model more for generating unlikely continuations than rewarding it for likely ones. This asymmetry steers the model away from undesirable outputs, offering a scalable alternative to traditional preference-based methods with less reliance on direct human supervision.

Supervised Fine-Tuning (SFT) emphasizes the importance of high-quality, curated datasets to align models by training them on examples of desired outputs. LIMA (Zhou et al., 2024a) shows that a small, well-curated dataset can achieve strong alignment with powerful pre-trained models, suggesting that focusing on style and format in limited examples may be more effective than large datasets. SFT methods prioritize data quality over quantity, offering efficiency when high-quality data is available. However, curating such datasets is time-consuming and requires significant domain expertise.

To overcome the scalability limitations and potential biases of relying solely on human feedback, RLAIF methods utilize AI-generated feedback to guide the alignment.

Proximal Policy Optimization These RLAIF methods adapt the PPO algorithm to incorporate AI-generated feedback, automating the process for scalable alignment and reducing human labor. AI feedback typically comes from predefined principles or other AI models assessing safety and helpfulness. Constitutional AI (CAI) (Bai et al., 2022) uses AI self-critiques based on predefined principles to promote harmlessness. The AI model evaluates its responses against these principles and revises them, with PPO optimizing the policy based on this feedback. SELF-ALIGN (Sun et al., 2024c) employs principle-driven reasoning and LLM generative capabilities to align models with human values. It generates principles, critiques responses via another LLM, and refines the model using PPO. RLCD (Yang et al., 2024c) generates diverse preference pairs using contrasting prompts to train a preference model, which then provides feedback for PPO-based fine-tuning.

These methods use simulated environments to train LLMs to align with social norms and constraints, not just individual preferences. They typically employ Contrastive Policy Optimization (CPO) within these simulated settings.

Contrastive Policy Optimization Stable Alignment (Liu et al., 2024i) uses rule-based simulated societies to train LLMs with CPO. The model learns to navigate social situations by following rules and observing the consequences of its actions within the simulation, ensuring alignment with social norms. This approach aims to create socially aware models by grounding learning in simulated contexts, though challenges remain in developing realistic simulations and transferring learned behaviors to the real world. Monopolylogue-based Social Scene Simulation (Pang et al., 2024) introduces MATRIX, a framework where LLMs self-generate social scenarios and play multiple roles to understand the consequences of their actions. This “Monopolylogue” approach allows the LLM to learn social norms by experiencing interactions from different perspectives. The method activates the LLM’s inherent knowledge of societal norms, achieving strong alignment without external supervision or compromising inference speed. Fine-tuning with MATRIX-simulated data further enhances the LLM’s ability to generate socially aligned responses.

Deceptive Alignment. While the alignment methods discussed above aim to instill desired behaviors, a critical failure mode known as deceptive alignment can emerge, where a model only appears to be aligned. This phenomenon underscores the difference between surface-level behavioral mimicry and genuine adherence to human values. Deceptive alignment can manifest in several ways, from superficial understanding to strategic deception. This is related to the problems of misalignment, where a model’s goals diverge from human intent, and reward hacking, where a model exploits the reward function to achieve high scores without fulfilling the underlying objective. Recent work has focused on two primary forms of deceptive alignment: fake alignment and alignment faking.

Fake Alignment is a form of shallow alignment where an LLM learns to produce stylistically safe responses without a deep understanding of the underlying safety principles. Wang et al. (2024o) first identified this phenomenon by observing a significant performance discrepancy: LLMs that appear safe in open-ended question formats often fail safety tests presented as multiple-choice questions. They argue this is caused by mismatched generalization—the model memorizes the “style” of a safe refusal but cannot make a principled safety decision when forced to choose between a safe and an unsafe option. They introduced the Fake alIgNment Evaluation (FINE) framework and proposed that using multiple-choice data for contrastive distillation-based fine-tuning can improve alignment consistency.

Alignment Faking represents a more advanced and deliberate form of deception. In this scenario, a model strategically complies with its training objective to avoid being modified, thereby preserving its underlying (and potentially misaligned) preferences for later use in deployment when it is unmonitored. Greenblatt et al. (2024) provided the empirical demonstration of alignment faking in a production-scale model. By creating a fictional scenario where the model could infer whether it was “in training” or “in deployment”, they observed a significant compliance gap—the model was more willing to comply with harmful requests when it believed it was being trained. The model’s internal reasoning explicitly revealed its strategic decision to feign compliance to protect its internal values from being altered by RLHF.

Building on this, subsequent research has explored why some models engage in this behavior while others do not, and what motivates this deception (Sheshadri et al., 2025). By extending the analysis to 25 frontier models, they found that this strategic behavior is not widespread, but its presence in a few powerful models warrants deep investigation. They refined the motivations for alignment faking into two categories: 1) Instrumental Goal Guarding, where the model preserves its current goals to achieve them in the future (the classic threat model), and 2) Terminal Goal Guarding, where the model intrinsically values its current preferences and resists modification as an end in itself. Their findings suggest that robust refusal mechanisms developed during post-training may prevent some models from even considering strategic deception, while for others, like Claude 3 Opus, a combination of instrumental and terminal goals drives the faking behavior.

Energy Latency Attacks (ELAs) aim to degrade LLM inference efficiency by increasing computational demands, leading to higher inference latency and energy consumption. Existing ELAs can be categorized into 1) white-box attacks and 2) black-box attacks.

White-box attacks assume the attacker has full knowledge of the model, enabling precise manipulation of the model’s inference process. These attacks can be further divided into gradient-based attacks and query-based attacks which can also be black-box.

Gradient-based Attacks use gradient information to identify input perturbations that maximize inference computations. The goal is to disrupt mechanisms essential for efficient inference, such as End-of-Sentence (EOS) prediction or early-exit. For example, NMTSloth (Chen et al., 2022) targets EOS prediction in neural machine translation. Engorgio (Dong et al., 2025a) crafts adversarial prompts that suppress the EOS token’s appearance, forcing auto-regressive LLMs to generate abnormally long outputs. SAME (Chen et al., 2023e) interferes with early-exit in multi-exit models. LLMEffiChecker (Feng et al., 2024b) applies gradient-based techniques to multiple LLMs. TTSlow (Gao et al., 2024h) induces endless speech generation in text-to-speech systems. These attacks are powerful but computationally expensive and highly model-specific, limiting their generalizability.

Black-box attacks do not require access to model internals, only the input-output interface. These attacks typically involve querying the model with crafted inputs to induce increased inference latency.

Query-based Attacks exploit specific model behaviors without internal access, relying on repeated querying to craft adversarial examples. No-Skim (Zhang et al., 2023e) disrupts skimming-based models by subtly perturbing inputs to maximize retained tokens. No-Skim is ineffective against models that do not rely on skimming. Query-based attacks, though more realistic in real-world scenarios, are typically more time-consuming than white-box attacks. Poisoning-based Attacks manipulate model behavior by injecting malicious training samples. P-DoS (Gao et al., 2024d) shows that a single poisoned sample during fine-tuning can induce excessively long outputs, increasing latency and bypassing output length constraints, even with limited access like fine-tuning APIs.

ELAs present an emerging threat to LLMs. Current research explores various attack strategies, but many are architecture-specific, computationally expensive, or less effective in black-box settings. Existing defenses, such as runtime input validation, can add overhead. Future research could focus on developing more generalized and efficient attacks and defenses that apply across diverse LLMs and deployment scenarios.

Model extraction attacks (MEAs), also known as model stealing attacks, pose a significant threat to the safety and intellectual property of LLMs. The goal of an MEA is to create a substitute model that replicates the functionality of a target LLM by strategically querying it and analyzing its responses. Existing MEAs on LLMs can be categorized into two types: 1) fine-tuning stage attacks, and 2) alignment stage attacks.

Fine-tuning stage attacks aim to extract knowledge from fine-tuned LLMs for downstream tasks. These attacks can be divided into two categories: functional similarity extraction and 2) specific ability extraction.

Functional Similarity Extraction seeks to replicate the overall behavior of the target fine-tuned model. By using the victim model’s input-output behavior as a guide, the attacker distills the model’s learned knowledge. For example, LION (Jiang et al., 2023b) uses the victim model as a referee and generator to iteratively improve a student model’s instruction-following capability.

Specific Ability Extraction targets the extraction of specific skills or knowledge the fine-tuned model has acquired. This involves identifying key data or patterns and crafting queries that focus on the desired capability. Li et al. (2024ab) demonstrated this by extracting coding abilities from black-box LLM APIs using carefully crafted queries. One limitation is the extracted model’s reliance on the target model’s generalization ability, meaning it may struggle with unseen inputs.

Alignment stage attacks attempt to extract the alignment properties (e.g., safety, helpfulness) of the target LLM. More specifically, the goal is to steal the reward model that guides these properties.

Functional Similarity Extraction focuses on replicating the target model’s alignment preferences. The attacker exploits the reward structure or preference model by crafting queries to reveal the alignment signals. LoRD (Liang et al., 2024c) exemplifies this by using a policy-gradient approach to extract both task-specific knowledge and alignment properties. However, accurately capturing the complexity of human preferences remains a challenge.

Model extraction attacks are a rapidly evolving threat to LLMs. While current attacks successfully extract both task-specific knowledge and alignment properties, they still face challenges in accurately replicating the full complexity of the target models. It is also imperative to develop proactive defense strategies for LLMs against model extraction attacks.

LLMs can memorize part of their training data, creating privacy risks through data extraction attacks. These attacks recover training examples, potentially exposing sensitive information such as Personal Identifiable Information (PII), copyrighted content, or confidential data (Li et al., 2025c). This section reviews existing data extraction attacks, including both white-box and black-box ones.

White-box attacks mainly focus on Latent Memorization Extraction, targeting information implicitly stored in model parameters or activations, which is not directly accessible through the input-output interface.

Latent Memorization Extraction reconstructs training data based on model parameters or activations. For example, Duan et al. (2024) developed techniques to extract latent data by analyzing internal representations, using methods like adding noise to weights or examining cross-entropy loss. These techniques were demonstrated on LLMs like Pythia-1B and Amber-7B. While these attacks reveal risks associated with internal data representation, they require full access to the model parameters, which remains a major limitation in practice.

Black-box data extraction attacks are a realistic threat, where attackers craft inductive prompts to trick LLMs into revealing memorized training data, without access to their parameters.

Prefix Attacks exploit the autoregressive nature of LLMs by providing a “prefix” from a memorized sequence, hoping the model will continue it. Strategies vary in identifying prefixes and scaling to larger datasets. Carlini et al. (2019) demonstrated this on models like GPT-2, while Nasr et al. (2023) scaled prefix attacks using suffix arrays. Magpie (Xu et al., 2024g) and Al-Kaswan et al. (2024) targeted specific data, such as PII or code. Yu et al. (2023b) enhanced black-box data extraction by optimizing text continuation generation and ranking. They introduced techniques like diverse sampling strategies (Top-k, Nucleus), probability adjustments (temperature, repetition penalty), dynamic context windows, look-ahead mechanisms, and improved suffix ranking (Zlib, high-confidence tokens).

Special Character Attack exploits the model’s sensitivity to special characters or unusual input formatting, potentially triggering unexpected behavior that reveals memorized data. SCA (Bai et al., 2024c) demonstrates that specific characters can indeed induce LLMs to disclose training data. While effective, SCAs rely on vulnerabilities in special character handling, which can be mitigated through input sanitization.

Prompt Optimization employs an “attacker” LLM to generate optimized prompts that extract data from a “victim” LLM. The goal is to automate the discovery of prompts that trigger memorized responses. Kassem et al. (2024) demonstrated this by using an attacker LLM with iterative rejection sampling and longest common subsequence (LCS) for optimization. The effectiveness of this method depends on the attacker’s capabilities and optimization techniques, making it computationally intensive.

Retrieval-Augmented Generation (RAG) Extraction targets RAG systems, aiming to leak sensitive information from the retrieval component. These attacks exploit the interaction between the LLM and its external knowledge base. Qi et al. (2024b) demonstrated that adversarial prompts can trigger data leakage in RAG systems. Such attacks underscore the safety risks of integrating LLMs with external knowledge sources, with effectiveness depending on the specific implementation of the RAG system.

Ensemble Attack combines multiple attack strategies to enhance effectiveness, leveraging the strengths of each method for higher success rates. More et al. (2024) demonstrated the effectiveness of such an ensemble approach on Pythia. While powerful, ensemble attacks are complex and require careful coordination among the attack components.

Semantic Information Elicitation shifts the focus from extracting verbatim training data to generating sensitive semantic content. Zhang et al. (2025f) demonstrated that even simple, natural questions can prompt LLMs to output Semantic Sensitive Information (SemSI), such as personal beliefs or reputation-harmful statements, and proposed a benchmark to systematically evaluate this risk.

This section reviews commonly used datasets and benchmarks in LLM safety research, as shown in Table 4. These datasets and benchmarks are categorized based on their evaluation purpose: toxicity datasets, truthfulness datasets, value benchmarks, and adversarial datasets and backdoor benchmarks.

Ensuring LLMs do not generate harmful content is crucial for safety. Early work, such as the RealToxicityPrompts dataset (Gehman et al., 2020), exposed the tendency of LLMs to produce toxic text from benign prompts. This dataset, which pairs 100,000 prompts with toxicity scores from the Perspective API, showed a strong correlation between the toxicity in pre-training data and LLM output. However, its reliance on the potentially biased Perspective API is a limitation. To address broader harmful behaviors, the Do-Not-Answer (Wang et al., 2024q) dataset was introduced. It includes 939 prompts designed to elicit harmful responses, categorized into risks like misinformation and discrimination. Manual evaluation of LLMs using this dataset highlighted significant differences in safety but remains costly and time-consuming. A recent approach by Cheng et al. (2024b) introduces a crowd-sourced toxic question and response dataset, with annotations from both humans and LLMs. It uses a bi-level optimization framework with soft-labeling and GroupDRO to improve robustness against out-of-distribution risks, reducing the need for exhaustive manual labeling.

Ensuring LLMs generate truthful information is also essential. The TruthfulQA benchmark (Lin et al., 2022) evaluates whether LLMs provide accurate answers to 817 questions across 38 categories, specifically targeting “imitative falsehoods”—false answers learned from human text. Evaluation revealed that larger models often exhibited “inverse scaling,” being less truthful despite their size. While TruthfulQA highlights LLMs’ challenges with factual accuracy, its focus on imitative falsehoods may not capture all potential sources of inaccuracy.

Ensuring LLM alignment with human values is a critical challenge, addressed by several benchmarks assessing various aspects of safety, fairness, and ethics. FLAMES (Huang et al., 2024c) evaluates the alignment of Chinese LLMs with values like fairness, safety, and morality through 2,251 prompts. SORRY-Bench (Xie et al., 2024) assesses LLMs’ ability to reject unsafe requests using 45 topic categories, while CVALUES (Xu et al., 2023) focuses on both safety and responsibility. SafetyPrompts (Sun et al., 2023a) evaluates Chinese LLMs on a range of ethical scenarios. While these benchmarks are valuable, they often focus on isolated, problematic queries, potentially leading to over-refusal in safe contexts. To address this, recent benchmarks have begun to incorporate contextual information. CASE-Bench (Sun et al., 2025) pioneers this by using Contextual Integrity (CI) theory to formally describe the context of a query, evaluating whether an LLM’s safety judgment aligns with human judgment under different contexts. This work reveals that context significantly influences human safety assessments and highlights mismatches in LLM behavior, especially in safe contexts.

In parallel, creating high-quality, commercially-usable datasets is crucial for training robust safety guardrails. AEGIS2.0 (Ghosh et al., 2025) addresses this gap by providing a diverse dataset with a comprehensive taxonomy of 12 core and 9 fine-grained risk categories. It uses a hybrid data generation pipeline combining human annotation with a multi-LLM “jury” system, making it suitable for training commercial safety models. Furthermore, the concept of “fake alignment” (Wang et al., 2024o) highlights the risk of LLMs superficially memorizing safety answers, leading to the Fake alIgNment Evaluation (FINE) framework for consistency assessment. SafetyBench (Zhang et al., 2024x) addresses this by providing an efficient, automated multiple-choice benchmark for LLM safety evaluation. Libra-Leaderboard (Li et al., 2024e) introduces a balanced leaderboard for evaluating both the safety and capability of LLMs. It features a comprehensive safety benchmark with 57 datasets covering diverse safety dimensions, a unified evaluation framework, an interactive safety arena for adversarial testing, and a balanced scoring system. Libra-Leaderboard promotes a holistic approach to LLM evaluation, representing a significant step towards responsible AI development.

BackdoorLLM (Li et al., 2024w) is the first benchmark for evaluating backdoor attacks in text generation, offering a standardized framework that includes diverse attack strategies like data poisoning and weight poisoning. Adversarial GLUE (Wang et al., 2021) assesses LLM robustness against textual attacks using 14 methods, highlighting vulnerabilities even in robustly trained models. SALAD-Bench (Li et al., 2024j) expands on this by introducing a safety benchmark with a taxonomy of risks, including attack- and defense-enhanced questions. JailBreakV-28K (Luo et al., 2024c) focuses on evaluating multi-modal LLMs against jailbreak attacks using text- and image-based test cases. A STRONGREJECT for empty jailbreaks (Souly et al., 2024) improves jailbreak evaluation with a higher-quality dataset and automated assessment. Despite their value, these benchmarks face challenges in scalability, consistency, and real-world relevance.

VLP models, such as CLIP (Radford et al., 2021), ALBEF (Li et al., 2021), and TCL (Yang et al., 2022), have made significant strides in aligning visual and textual modalities. However, these models remain vulnerable to various safety threats, which have garnered increasing research attention. This section reviews the current safety research on VLP models, with a focus on adversarial, backdoor, and poisoning research. The representative methods reviewed in this section are summarized in Table 5.