Researchers in China say they have developed sarcasm detection AI that achieves state-of-the-art performance using a data set created by Twitter. AI uses multimodal learning that combines text and images, as both are often needed to understand if a person is sarcastic.
The researchers’ AI focuses on differences between text and image and combines these results to make predictions. It also compares hashtags with tweet text to rate the sentiment a user is trying to convey.
“In particular, the input tokens give high attention values to the areas of the image that contradict them, since incongruence is a key factor in sarcasm,” the article says. “Since the incongruity may only occur within the text (e.g. sarcastic text associated with an unrelated image), the within-modality incongruity must be considered.”
On a data set from Twitter, the model achieved a 2.74% improvement in the F1 sarcasm detection score compared to HFM, a multimodal detection model introduced last year. The new model also achieved an accuracy of 86% compared to 83% for HFM.
The paper was jointly published by the Chinese Academy of Sciences and the Institute of Information Technology in Beijing, China. The paper was presented this week at the Conference on Virtual Empirical Methods in Natural Language Processing (EMNLP).
AI is the latest example of multimodal sarcasm detection since AI researchers began examining sarcasm in multimodal content on Instagram, Tumblr, and Twitter in 2016.
Researchers at the University of Michigan and the University of Singapore used language models and computer vision to detect sarcasm in television shows. This model was described in an article entitled “Towards Multimodal Sarcasm Detection (An Obviously Perfect Paper)”. This work was highlighted last year in the Association for Computational Linguistics (ACL).
Best Practices for a Successful AI Center of Excellence: A Guide for CoEs and Business Units Access here