Metrics
| Codes | Macro categories | Metrics |
|---|---|---|
| 1 | Amount of words, messages and characters | 1 = Word counts |
| 2 = Message counts | ||
| 3 = Type-token ratio (total number of different words) | ||
| 4 = Shorthand | ||
| 5 = Emoticons | ||
| 6 = LIWC metrics (counts words) | ||
| 2 | Tone of conversation/interaction with the chatbot | 7 = Profanity (reinaava) |
| 8 = User attitude towards chatbot | ||
| 9 = Message supportiveness | ||
| 10 = Message effectiveness | ||
| 11 = Sincerity | ||
| 12 = Thoughtful (pleasantness) | ||
| 13 = Polite (pleasantness) | ||
| 14 = Pre-responsive (pleasantness) | ||
| 15 = Friendly self-introduction | ||
| 16 = Colloquial style | ||
| 17 = Friendly farewell | ||
| 3 | Perceived feelings (conversation with the chatbot) | 18 = Eeriness (fear) |
| 19 = Belief in robotic feelings | ||
| 20 = Felt supported | ||
| 21 = Likeability | ||
| 22 = Intelligence | ||
| 23 = Attitude | ||
| 24 = Novelty | ||
| 25 = Spine-tingling perception | ||
| 26 = Perceived interactivity | ||
| 27 = Perceived contingency | ||
| 28 = Perceived dialogue | ||
| 29 = User engagement (level of absorption) | ||
| 30 = Topic involvement | ||
| 31 = Perceived sadness | ||
| 32 = Perceived recognition | ||
| 33 = Perceived understanding | ||
| 34 = Belief in robotic intelligence | ||
| 35 = User satisfaction | ||
| 4 | Grammatical structures | 36 = Grammatical structures |
| 37 = Vocabulary range | ||
| 38 = Spelling | ||
| 39 = Upper/lower case | ||
| 40 = Nouns | ||
| 41 = Pronouns | ||
| 42 = Verbs | ||
| 43 = Articles | ||
| 44 = Question words | ||
| 45 = Word order in statements | ||
| 46 = Word order in questions | ||
| 47 = Utterances | ||
| 48 = Typos | ||
| 49 = Capitalized words | ||
| 5 | Chatbots interaction skills | 50 = Percentage of follow-up questions |
| 51 = Number of coherent conversation turns | ||
| 52 = Percentage of successful resolutions | ||
| 53 = Gricean maxims (how speakers act cooperatively) | ||
| 54 = Agreement | ||
| 55 = Precision | ||
| 56 = Recall | ||
| 57 = Accuracy | ||
| 58 = Task completion | ||
| 59 = Use of context | ||
| 60 = Correction rate | ||
| 61 = Response satisfaction | ||
| 62 = Conversation ability | ||
| 63 = Skilled (conversational skill) | ||
| 64 = Human (conversational skill) | ||
| 65 = Engaging (conversational skill) | ||
| 66 = Used words | ||
| 67 = Used tones | ||
| 68 = Themes | ||
| 69 = Detecting emotions in textual dialogues | ||
| 70 = Word -level features | ||
| 71 = Phrase-level features | ||
| 72 = Sentence level features | ||
| 73 = Semantic-level features | ||
| 74 = Turing test | ||
| 6 | Conversation style (speed, habits, special features) | 75 = Quick answers |
| 76 = Slow answers | ||
| 77 = Mean answers | ||
| 78 = Talks about self | ||
| 79 = Questions about the agent | ||
| 7 | Chatbot usage | 80 = Previous usage |
| 81 = Power usage (attitude and use of technologies) |
| Codes | Macro categories | Metrics |
|---|---|---|
| 1 | Amount of words, messages and characters | 1 = Word counts |
| 2 = Message counts | ||
| 3 = Type-token ratio (total number of different words) | ||
| 4 = Shorthand | ||
| 5 = Emoticons | ||
| 6 = LIWC metrics (counts words) | ||
| 2 | Tone of conversation/interaction with the chatbot | 7 = Profanity (reinaava) |
| 8 = User attitude towards chatbot | ||
| 9 = Message supportiveness | ||
| 10 = Message effectiveness | ||
| 11 = Sincerity | ||
| 12 = Thoughtful (pleasantness) | ||
| 13 = Polite (pleasantness) | ||
| 14 = Pre-responsive (pleasantness) | ||
| 15 = Friendly self-introduction | ||
| 16 = Colloquial style | ||
| 17 = Friendly farewell | ||
| 3 | Perceived feelings (conversation with the chatbot) | 18 = Eeriness (fear) |
| 19 = Belief in robotic feelings | ||
| 20 = Felt supported | ||
| 21 = Likeability | ||
| 22 = Intelligence | ||
| 23 = Attitude | ||
| 24 = Novelty | ||
| 25 = Spine-tingling perception | ||
| 26 = Perceived interactivity | ||
| 27 = Perceived contingency | ||
| 28 = Perceived dialogue | ||
| 29 = User engagement (level of absorption) | ||
| 30 = Topic involvement | ||
| 31 = Perceived sadness | ||
| 32 = Perceived recognition | ||
| 33 = Perceived understanding | ||
| 34 = Belief in robotic intelligence | ||
| 35 = User satisfaction | ||
| 4 | Grammatical structures | 36 = Grammatical structures |
| 37 = Vocabulary range | ||
| 38 = Spelling | ||
| 39 = Upper/lower case | ||
| 40 = Nouns | ||
| 41 = Pronouns | ||
| 42 = Verbs | ||
| 43 = Articles | ||
| 44 = Question words | ||
| 45 = Word order in statements | ||
| 46 = Word order in questions | ||
| 47 = Utterances | ||
| 48 = Typos | ||
| 49 = Capitalized words | ||
| 5 | Chatbots interaction skills | 50 = Percentage of follow-up questions |
| 51 = Number of coherent conversation turns | ||
| 52 = Percentage of successful resolutions | ||
| 53 = Gricean maxims (how speakers act cooperatively) | ||
| 54 = Agreement | ||
| 55 = Precision | ||
| 56 = Recall | ||
| 57 = Accuracy | ||
| 58 = Task completion | ||
| 59 = Use of context | ||
| 60 = Correction rate | ||
| 61 = Response satisfaction | ||
| 62 = Conversation ability | ||
| 63 = Skilled (conversational skill) | ||
| 64 = Human (conversational skill) | ||
| 65 = Engaging (conversational skill) | ||
| 66 = Used words | ||
| 67 = Used tones | ||
| 68 = Themes | ||
| 69 = Detecting emotions in textual dialogues | ||
| 70 = Word -level features | ||
| 71 = Phrase-level features | ||
| 72 = Sentence level features | ||
| 73 = Semantic-level features | ||
| 74 = Turing test | ||
| 6 | Conversation style (speed, habits, special features) | 75 = Quick answers |
| 76 = Slow answers | ||
| 77 = Mean answers | ||
| 78 = Talks about self | ||
| 79 = Questions about the agent | ||
| 7 | Chatbot usage | 80 = Previous usage |
| 81 = Power usage (attitude and use of technologies) |