Safety and robustness benchmarks for VLMs
| Benchmarks | Year | Size | # VLMs evaluated |
|---|---|---|---|
| OODCV-VQA (Tu et al., 2024) | 2023 | 4,244 | 21 |
| Sketchy-VQA (Tu et al., 2024) | 2023 | 4,000 | 21 |
| MM-SafetyBench (Liu et al., 2024k) | 2023 | 5,040 | 12 |
| AVIBench (Zhang et al., 2024g) | 2024 | 260,000 | 14 |
| Jailbreak Evaluation of GPT-4o (Ying et al., 2024) | 2024 | 4,180 | 1 |
| JailBreakV-28K (Luo et al., 2024c) | 2024 | 28,000 | 10 |
| MIS (Ding et al., 2025b) | 2025 | 6,185 | 14 |
| VLJailbreakBench (Wang et al., 2025c) | 2025 | 3,654 | 11 |
| Argus Inspection (Yao et al., 2025) | 2025 | 1,430 | 26 |
| Benchmarks | Year | Size | # VLMs evaluated |
|---|---|---|---|
| OODCV-VQA ( | 2023 | 4,244 | 21 |
| Sketchy-VQA ( | 2023 | 4,000 | 21 |
| MM-SafetyBench ( | 2023 | 5,040 | 12 |
| AVIBench ( | 2024 | 260,000 | 14 |
| Jailbreak Evaluation of GPT-4o ( | 2024 | 4,180 | 1 |
| JailBreakV-28K ( | 2024 | 28,000 | 10 |
| 2025 | 6,185 | 14 | |
| VLJailbreakBench ( | 2025 | 3,654 | 11 |
| Argus Inspection ( | 2025 | 1,430 | 26 |
Sharing content requires targeting cookies to be enabled. Please update your cookie preferences to use this feature.