Figure 2.
The diagram has two connected stages labelled Attack Generation and Model Training. Original samples, including x 1, pass through t S L and then f theta e. The resulting representations are combined by dot product with W, followed by softmax, to produce output logits y hat. Target labels y and cross-entropy loss L C E are used to generate delta i through epsilon sign of the gradient with respect to t S L of x i. The perturbations are added to the transformed samples to create attack samples such as t S L of x 1 plus delta 1. These attack samples pass through f theta e. Their representations are combined with W by dot product and softmax to produce another set of output logits y hat. Cross-entropy loss is calculated against the same targets. Gradients with respect to W and theta e are used to update W and f theta e.Supervised adversarial training consists of attack generation for and model training for and