A summary of stance detection studies
| Source | Natural language | Data type and name | Techniques | Compared with | Result |
|---|---|---|---|---|---|
| ML approaches | |||||
| Tun and Hninn Myint (2019) 2019 | English | Twitter (SemEval2016 Task A and Task B) | Naive Bayes classifier and decision tree classifier | SVM-ngrams F1 score 68.98 in Task A Majority Class F1 score 29.72 in Task B | F1 score 68.6 in Task A F1 score 59.2 in Task B |
| Aldayel and Magdy (2019) 2019 | Twitter (SemEval2016 Task 6) | SVM model with a linear kernel | linear SVM model F1 score of 68.98% (Mohammad, Kiritchenko, Sobhani, Zhu, & Cherry, 2016) | F-measure of 72.49% | |
| Al-Ghadir et al. (2021) 2021 | WKNN, CKNN and SVM | – | WKNN was the most powerful classifier. -score of 76.45% | ||
| Darwish et al. (2020) 2020 | English and Turkish | 2 type DS 1-label dataset “Kavanaugh” (English), “Trump” (English) and “Erdogan” (Turkish) 2-Unlabeled “Collected tweets on six polarizing topics in the USA” | Unsupervised approach UMAP, mean shift | supervised approach SVM fasttext (Joulin, Grave, Bojanowski, & Mikolov, 2016) Precision 86.0% | precision 99.1% cluster purity |
| Ahmed et al. (2023) | English | “COVID-19 All Vaccines Tweets” | Extra tree classifier (ETC) | TF-IDF, BoW, Word2Vec | ETC outperform BoW with 92% accuracy |
| DL approaches | |||||
| Sobhani et al. (2017) 2017 | English | collected tweets related to the 2016 US election. Selected four presidential aspirants: “Donald Trump,” “Hillary Clinton” “Ted Cruz” and “Bernie Sanders” as targets | deep RNNs (Seq2Seq) | SVM (Pedregosa et al., 2011) F-macro of 52.05 | F-macro of 54.81 |
| Padnekar et al. (2020) 2020 | English | “Fake News Challenge (FNC)” | BiLSTM | – | Accuracy 94% |
| Santosh et al. (2019) 2020 | Siamese adaptation of LSTM networks | Baseline (MLP-6) FNC score 0.819 | FNC score of 0.85 | ||
| Mohtarami et al. (2018) 2021 | CNN + LSTM (sMemNN) | Baseline CNN LSTM macro-F1 of 40.33 | Macro-F1 of 56.75 | ||
| Transformer approaches | |||||
| Schiller et al. (2021) 2021 | English | combined datasets from different domains (ibmcs- semeval2019t7- semeval2016t6- fnc1- snipes- scd- perspectrum- iac1- arc- argmin) | BERT | – | Transfer learning and multi-dataset learning can improve the performance |
| Alhindi et al. (2021) 2021 | Arabic | Arabic Stance Detection dataset (AraStance) of 4,063 claim it covers false and true claims from multiple domains (e.g. politics, sports, health) and several Arab countries | BERT | – | Accuracy of 85% and a Macro F1 score of 78% |
| Lin et al. (2020) 2020 | English | “FAKE NEWS CHALLENGE” STAGE 1(FNC-1) | BERT | CNN + LSTM LSTM + CNN Macro-F1 of 40.33 (Mohtarami et al., 2018) | Macro-F1 of 75.96 |
| Kayalvizhi et al. (2021) 2020 | Italian | Italian tweets about the Sardines movement | BERT | encoder-decoder model F1 score of 0.4473 | F1-average score of 0.47 |
| Ghosh et al. (2019) 2020 | English | SemEval 2016 and MPCHI | BERT | CNN (Pkudblab at SemEval-2016 Task 6, n.d. ) F1 score of 0.690 (semval dataset) | F1 score of 0.75 |
| Müller et al. (2020) 2020 | English | Maternal Vaccine Stance (MVS) | CT-BERT | BERT-LARGE (Devlin et al., 2019) Mean F1 -score of 0.802 | Mean F1 score of 0.833 |
| Source | Natural language | Data type and name | Techniques | Compared with | Result |
|---|---|---|---|---|---|
| ML approaches | |||||
| English | Twitter (SemEval2016 Task A and Task B) | Naive Bayes classifier and decision tree classifier | SVM-ngrams | F1 score 68.6 in Task A | |
| Twitter (SemEval2016 Task 6) | SVM model with a linear kernel | linear SVM model F1 score of 68.98% ( | F-measure of 72.49% | ||
| WKNN, CKNN and SVM | – | WKNN was the most powerful classifier. | |||
| English and Turkish | 2 type DS | Unsupervised approach | supervised approach | precision 99.1% cluster purity | |
| English | “COVID-19 All Vaccines Tweets” | Extra tree classifier (ETC) | TF-IDF, BoW, Word2Vec | ETC outperform BoW with 92% accuracy | |
| DL approaches | |||||
| English | collected tweets related to the 2016 US election. Selected four presidential aspirants: “Donald Trump,” “Hillary Clinton” “Ted Cruz” and “Bernie Sanders” as targets | deep RNNs (Seq2Seq) | SVM ( | F-macro of 54.81 | |
| English | “Fake News Challenge (FNC)” | BiLSTM | – | Accuracy 94% | |
| Siamese adaptation of LSTM networks | Baseline (MLP-6) | FNC score of 0.85 | |||
| CNN + LSTM (sMemNN) | Baseline CNN | Macro-F1 of 56.75 | |||
| Transformer approaches | |||||
| English | combined datasets from different domains (ibmcs- semeval2019t7- semeval2016t6- fnc1- snipes- scd- perspectrum- iac1- arc- argmin) | BERT | – | Transfer learning and multi-dataset learning can improve the performance | |
| Arabic | Arabic Stance Detection dataset (AraStance) of 4,063 claim it covers false and true claims from multiple domains (e.g. politics, sports, health) and several Arab countries | BERT | – | Accuracy of 85% and a Macro F1 score of 78% | |
| English | “FAKE NEWS CHALLENGE” STAGE 1(FNC-1) | BERT | CNN + LSTM | Macro-F1 of 75.96 | |
| Italian | Italian tweets about the Sardines movement | BERT | encoder-decoder model | F1-average score of 0.47 | |
| English | SemEval 2016 and MPCHI | BERT | CNN ( | F1 score of 0.75 | |
| English | Maternal Vaccine Stance (MVS) | CT-BERT | BERT-LARGE ( | Mean F1 score of 0.833 | |
Source(s): Authors’ own work
Sharing content requires targeting cookies to be enabled. Please update your cookie preferences to use this feature.