Example of the successful and failed dialogue generated by STORL and rule-based dialogue policy, respectively. Red colored texts indicate the mismatched user dialogue acts (DAs). Blue and green colored texts indicate the correct and wrong answers, respectively. The whole conversations are not shown to save space.
| Goal: | {‘train’: {‘info’:{‘arrive by’: ‘11:45’, ‘day’: ‘Wednesday’, ‘departure’: ‘Cambridge’, ‘destination’: ‘Peterborough’}, ‘request’: {‘duration’: ‘?’, ‘price’: ‘?’}}, ‘restaurant’: {‘info’: {‘food’: ‘Indian’, ‘price range’: ‘expensive’}, ‘request’: {‘postcode’: ‘?’}}} |
| STORL Policy | |
| User: | Please book that train for me now. I want to get there by 11:45. The train should leave on Wednesday. I’m departing from Cambridge. |
| User DA: | [[‘Inform’, ‘Train’, ‘none’, ‘none’], [‘Inform’, ‘Train’, ‘Arrive’, ‘11:45’], [‘Inform’, ‘Train’, ‘Day’, ‘Wednesday’], [‘Inform’, ‘Train’, ‘Departure’, ‘Cambridge’]] |
| BERT Predicted DA: | [[‘Inform’, ‘Train’, ‘Arrive’, ‘11:45’], [‘Inform’, ‘Train’, ‘Day’, ‘Wednesday’], [‘Inform’, ‘Train’, ‘Departure’, ‘Cambridge’]] |
| System: | Where would you like to go to? |
| User: | I am leaving for Peterborough. |
| User DA: | [[‘Inform’, ‘Train’, ‘Destination’, ‘Peterborough’]] |
| BERT Predicted DA: | [[‘Inform’, ‘Train’, ‘Departure’, ‘Peterborough’]] |
| System: | Would you like me to book the 06:09 train? |
| User: | Can you tell me the price? Can you give me the travel time please? |
| User DA: | [[‘Request’, ‘Train’, ‘Ticket’, ‘?’], [‘Request’, ‘Train’, ‘Time’, ‘?’]] |
| BERT Predicted DA: | [[‘Request’, ‘Train’, ‘Time’, ‘?’], [‘Request’, ‘Train’, ‘Ticket’, ‘?’]] |
| System: | The price is 16.50 pounds per ticket. The trip is 50 minutes. |
| Rule-Based Policy | |
| User: | Please book that train for me now. I want to get there by 11:45. The train should leave on Wednesday. I’m departing from Cambridge. |
| User DA: | [[‘Inform’, ‘Train’, ‘none’, ‘none’], [‘Inform’, ‘Train’, ‘Arrive’, ‘11:45’], [‘Inform’, ‘Train’, ‘Day’, ‘Wednesday’], [‘Inform’, ‘Train’, ‘Departure’, ‘Cambridge’]] |
| BERT Predicted DA: | [[‘Inform’, ‘Train’, ‘Arrive’, ‘11:45’], [‘Inform’, ‘Train’, ‘Day’, ‘Wednesday’], [‘Inform’, ‘Train’, ‘Departure’, ‘Cambridge’]] |
| System: | Where would you like to go to? |
| User: | I am leaving for Peterborough. |
| User DA: | [[‘Inform’, ‘Train’, ‘Destination’, ‘Peterborough’]] |
| BERT Predicted DA: | [[‘Inform’, ‘Train’, ‘Departure’, ‘Peterborough’]] |
| System: | Where is your destination? |
| User: | Can you also help me find a train going to Peterborough? |
| User DA: | [[‘Inform’, ‘Train’, ‘Destination’, ‘Peterborough’]] |
| BERT Predicted DA: | [[‘Inform’, ‘Train’, ‘Destination’, ‘Peterborough’]] |
| System: | I am sorry there are no trains to arrive at 11:45. There are no trains on Wednesday. There is no train leaving Peterborough. |
| User: | Please book that train for me now. I want to get there by 11:45. The train should leave on Wednesday. I’m departing from Cambridge. |
| User DA: | [[‘Inform’, ‘Train’, ‘none’, ‘none’], [‘Inform’, ‘Train’, ‘Arrive’, ‘11:45’], [‘Inform’, ‘Train’, ‘Day’, ‘Wednesday’], [‘Inform’, ‘Train’, ‘Departure’, ‘Cambridge’]] |
| BERT Predicted DA: | [[‘Inform’, ‘Train’, ‘Arrive’, ‘11:45’], [‘Inform’, ‘Train’, ‘Day’, ‘Wednesday’], [‘Inform’, ‘Train’, ‘Departure’, ‘Cambridge’]] |
| System: | Where would you like to go to? |
| User: | I am leaving for Peterborough. |
| User DA: | [[‘Inform’, ‘Train’, ‘Destination’, ‘Peterborough’]] |
| BERT Predicted DA: | [[‘Inform’, ‘Train’, ‘Departure’, ‘Peterborough’]] |
| System: | Would you like me to book the 06:09 train? |
| User: | Can you tell me the price? Can you give me the travel time please? |
| User DA: | [[‘Request’, ‘Train’, ‘Ticket’, ‘?’], [‘Request’, ‘Train’, ‘Time’, ‘?’]] |
| BERT Predicted DA: | [[‘Request’, ‘Train’, ‘Time’, ‘?’], [‘Request’, ‘Train’, ‘Ticket’, ‘?’]] |
| System: | The price is 16.50 pounds per ticket. The trip is 50 minutes. |
| User: | Please book that train for me now. I want to get there by 11:45. The train should leave on Wednesday. I’m departing from Cambridge. |
| User DA: | [[‘Inform’, ‘Train’, ‘none’, ‘none’], [‘Inform’, ‘Train’, ‘Arrive’, ‘11:45’], [‘Inform’, ‘Train’, ‘Day’, ‘Wednesday’], [‘Inform’, ‘Train’, ‘Departure’, ‘Cambridge’]] |
| BERT Predicted DA: | [[‘Inform’, ‘Train’, ‘Arrive’, ‘11:45’], [‘Inform’, ‘Train’, ‘Day’, ‘Wednesday’], [‘Inform’, ‘Train’, ‘Departure’, ‘Cambridge’]] |
| System: | Where would you like to go to? |
| User: | I am leaving for Peterborough. |
| User DA: | [[‘Inform’, ‘Train’, ‘Destination’, ‘Peterborough’]] |
| BERT Predicted DA: | [[‘Inform’, ‘Train’, ‘Departure’, ‘Peterborough’]] |
| System: | Where is your destination? |
| User: | Can you also help me find a train going to Peterborough? |
| User DA: | [[‘Inform’, ‘Train’, ‘Destination’, ‘Peterborough’]] |
| BERT Predicted DA: | [[‘Inform’, ‘Train’, ‘Destination’, ‘Peterborough’]] |
| System: | I am sorry there are no trains to arrive at 11:45. There are no trains on Wednesday. There is no train leaving Peterborough. |
Sharing content requires targeting cookies to be enabled. Please update your cookie preferences to use this feature.