Table 1
Transformer-based PLMs.
Encoder-only models (Bidirectional)
BERT [36] RoBERTa [107] ELECTRA [27]
Decoder-only models (Unidirectional)
PaLM [24] GPT-1, 2 and 3 [17, 133, 134] Transformer XL [31]
Encoder-Decoder models (Sequence to sequence)
BART [99] T5 [135]
Encoder-only models (Bidirectional)
BERT [
36
] RoBERTa [
107
] ELECTRA [
27
]
Decoder-only models (Unidirectional)
PaLM [
24
] GPT-1, 2 and 3 [
17
,
133
,
134
] Transformer XL [
31
]
Encoder-Decoder models (Sequence to sequence)
BART [
99
] T5 [
135
]
[ViewLarge]
Close Modal
Close Modal
This Feature Is Available To Subscribers Only
Sign In
or
Create an Account
Close Modal
Close Modal