Victoria Sherratt
ABSTRACT
This paper proposes a multi-channel convolutional neural network (MC-CNN) for classifying memes and non-memes. Our architecture is trained and validated on a challenging dataset that includes non-meme formats with textual attributes, which are also circulated online but rarely accounted for in meme classification tasks. Alongside a transfer learning base, two additional channels capture low-level and fundamental features of memes that make them unique from other images with text. We contribute an approach which outperforms previous meme classifiers specifically in live data evaluation, and one that is better able to generalise ‘in the wild’. Our research aims to improve accurate collation of meme content to support continued research in meme content analysis, and meme-related sub-tasks such as harmful content detection.
Supplemental Material
- Tariq Habib Afridi, Aftab Alam, Muhammad Numan Khan, Jawad Khan, and Young-Koo Lee. 2020. A Multimodal Memes Classification: A Survey and Open Research Issues. (Sept. 2020). https://doi.org/10.48550/arXiv.2009.08395 arXiv:https://arxiv.org/abs/2009.08395Google Scholar
- Library of Congress American Folklore Centre. [n. d.]. Meme Generator: collected datasets. Available at: https://www.loc.gov/item/2018655320/ (2022-05-10).Google Scholar
- Kate Barnes, Tiernon Riesenmy, Minh Duc Trinh, Eli Lleshi, Nóra Balogh, and Roland Molontay. 2021. Dank or not? Analyzing and predicting the popularity of memes on Reddit. Applied Network Science 6, 1 (March 2021). https://doi.org/10.1007/s41109-021-00358-7Google ScholarCross Ref
- David M. Beskow, Sumeet Kumar, and Kathleen M. Carley. 2020. The evolution of political memes: Detecting and characterizing internet memes with multi-modal deep learning. Information Processing & Management 57, 2 (March 2020), 102170. https://doi.org/10.1016/j.ipm.2019.102170 Number: 2.Google ScholarDigital Library
- Tanmoy Chakraborty and Sarah Masud. 2022. Nipping in the bud: detection, diffusion and mitigation of hate speech on social media. ACM SIGWEB NewsletterWinter (2022), 1–9.Google ScholarDigital Library
- Jacob Cohen. 1960. A coefficient of agreement for nominal scales. Educational and psychological measurement 20, 1 (1960), 37–46.Google Scholar
- Dimitar Dimitrov, Bishr Bin Ali, Shaden Shaar, Firoj Alam, Fabrizio Silvestri, Hamed Firooz, Preslav Nakov, and Giovanni Da San Martino. 2021. Detecting Propaganda Techniques in Memes. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Association for Computational Linguistics, Online, 6603–6617. https://doi.org/10.18653/v1/2021.acl-long.516Google Scholar
- Yuhao Du, Muhammad Aamir Masood, and Kenneth Joseph. 2020. Understanding Visual Memes: An Empirical Analysis of Text Superimposed on Memes Shared on Twitter. Proceedings of the International AAAI Conference on Web and Social Media 14, 1 (May 2020), 153–164. https://doi.org/10.1609/icwsm.v14i1.7287Google ScholarCross Ref
- Abhimanyu Dubey, Esteban Moro, Manuel Cebrian, and Iyad Rahwan. 2018. MemeSequencer: Sparse Matching for Embedding Image Macros. In Proceedings of the 2018 World Wide Web Conference on World Wide Web - WWW ’18. ACM Press, Lyon, France, 1225–1235. https://doi.org/10.1145/3178876.3186021Google ScholarDigital Library
- Marta Dynel. 2016. “I has seen image macros!” Advice animals memes as visual-verbal jokes. International Journal of Communication 10 (2016), 29.Google Scholar
- Micah Hodosh, Peter Young, and Julia Hockenmaier. 2013. Flickr8k Dataset. Journal of Artificial Intelligence Research 47 (2013), 853–899.Google ScholarDigital Library
- Zaeem Hussain, Mingda Zhang, Xiaozhong Zhang, Keren Ye, Christopher Thomas, Zuha Agha, Nathan Ong, and Adriana Kovashka. 2017. Automatic understanding of image and video advertisements. In Proceedings of the IEEE conference on computer vision and pattern recognition (Honolulu, HI, USA). IEEE, 1705–1715. https://doi.org/10.1109/CVPR.2017.123Google ScholarCross Ref
- JaidedOCR. 2022. EasyOCR. Available at: https://www.jaided.ai/easyocr/ (2022-05-10).Google Scholar
- Douwe Kiela, Hamed Firooz, Aravind Mohan, Vedanuj Goswami, Amanpreet Singh, Pratik Ringshia, and Davide Testuggine. 2020. The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes. In Advances in Neural Information Processing Systems NIPS’20 (Vancouver, BC, Canada), Vol. 33. Curran Associates, Inc., Red Hook, NY, USA, Article 220, 2611–2624 pages.Google Scholar
- Hannah Kirk, Yennie Jun, Paulius Rauba, Gal Wachtel, Ruining Li, Xingjian Bai, Noah Broestl, Martin Doff-Sotta, Aleksandar Shtedritski, and Yuki M Asano. 2021. Memes in the Wild: Assessing the Generalizability of the Hateful Memes Challenge Dataset. In Proceedings of the 5th Workshop on Online Abuse and Harms (WOAH 2021). Association for Computational Linguistics, Online, 26–35. https://doi.org/10.18653/v1/2021.woah-1.4Google ScholarCross Ref
- Michele Knobel and Colin Lankshear. 2007. Online memes, affinities, and cultural production. A new literacies sampler 29 (2007), 199–227. Publisher: New York.Google Scholar
- Christos Koutlis, Manos Schinas, and Symeon Papadopoulos. 2023. MemeTector: Enforcing deep focus for meme detection. International Journal of Multimedia Information Retrieval (Jan. 2023). https://doi.org/10.5281/zenodo.7554267Google ScholarCross Ref
- Jure Leskovec, Lars Backstrom, and Jon Kleinberg. 2009. Meme-tracking and the dynamics of the news cycle. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. 497–506.Google ScholarDigital Library
- Martina Miliani, Giulia Giorgi, Ilir Rama, Guido Anselmi, and Gianluca E. Lebani. 2020. DANKMEMES @ EVALITA 2020: The Memeing of Life: Memes, Multimodality and Politics. In EVALITA. http://ceur-ws.org/Vol-2765/paper174.pdfGoogle Scholar
- Ankit Kumar Mishra and Sunil Saumya. 2021. IIIT_DWD@EACL2021: Identifying Troll Meme in Tamil using a hybrid deep learning approach. In Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages. Association for Computational Linguistics, Kyiv, 243–248. https://aclanthology.org/2021.dravidianlangtech-1.33Google Scholar
- Lawankorn Mookdarsanit and Pakpoom Mookdarsanit. 2021. Combating the hate speech in Thai textual memes. Indonesian Journal of Electrical Engineering and Computer Science 21, 3 (March 2021), 1493–1502. https://doi.org/10.11591/ijeecs.v21.i3.pp1493-1502 Number: 3.Google ScholarCross Ref
- Fausto Morales. 2019. Keras-OCR. Available at: https://keras-ocr.readthedocs.io/ (2022-02-05).Google Scholar
- Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 1532–1543.Google ScholarCross Ref
- Jesus Perez-Martin, Benjamin Bustos, and Magdalena Saldana. 2020. Semantic Search of Memes on Twitter. (Feb. 2020). https://doi.org/10.48550/arXiv.2002.01462 arXiv:https://arxiv.org/abs/2002.01462v4Google Scholar
- Shraman Pramanick, Dimitar Dimitrov, Rituparna Mukherjee, Shivam Sharma, Md. Shad Akhtar, Preslav Nakov, and Tanmoy Chakraborty. 2021. Detecting Harmful Memes and Their Targets. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. Association for Computational Linguistics, Online, 2783–2796. https://doi.org/10.18653/v1/2021.findings-acl.246Google ScholarCross Ref
- Joshua Roesslein. 2020. Tweepy: Twitter for Python!URL: https://github.com/tweepy/tweepy (2020).Google Scholar
- Giorgio Roffo and Alessandro Vinciarelli. 2016. Personality in computational advertising: A benchmark. In 4th Workshop on Emotions and Personality in Peronalized Systems. 18.Google Scholar
- Richard Rogers and Giulia Giorgi. 2023. What is a meme, technically speaking?Information, Communication & Society 0, 0 (2023), 1–19. https://doi.org/10.1080/1369118X.2023.2174790 arXiv:https://doi.org/10.1080/1369118X.2023.2174790Google Scholar
- Chhavi Sharma, Deepesh Bhageria, William Scott, Srinivas PYKL, Amitava Das, Tanmoy Chakraborty, Viswanath Pulabaigari, and Björn Gambäck. 2020. SemEval-2020 Task 8: Memotion Analysis- the Visuo-Lingual Metaphor!. In Proceedings of the Fourteenth Workshop on Semantic Evaluation. International Committee for Computational Linguistics, Barcelona (online), 759–773. https://doi.org/10.18653/v1/2020.semeval-1.99Google ScholarCross Ref
- Chhavi Sharma and Viswanath Pulabaigari. 2020. A Curious Case of Meme Detection: An Investigative Study. In Proceedings of the 16th International Conference on Web Information Systems and Technologies. SCITEPRESS - Science and Technology Publications, Budapest, Hungary, 327–338. https://doi.org/10.5220/0010110203270338Google ScholarCross Ref
- Chhavi Sharma, Viswanath Pulabaigari, and Amitava Das. 2020. Meme vs. Non-meme Classification using Visuo-linguistic Association:. In Proceedings of the 16th International Conference on Web Information Systems and Technologies. SCITEPRESS - Science and Technology Publications, Budapest, Hungary, 353–360. https://doi.org/10.5220/0010176303530360Google ScholarCross Ref
- Shivam Sharma, Firoj Alam, Md. Shad Akhtar, Dimitar Dimitrov, Giovanni Da San Martino, Hamed Firooz, Alon Halevy, Fabrizio Silvestri, Preslav Nakov, and Tanmoy Chakraborty. 2022. Detecting and Understanding Harmful Memes: A Survey. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, IJCAI-22. International Joint Conferences on Artificial Intelligence Organization, 5597–5606. https://doi.org/10.24963/ijcai.2022/781 Survey Track.Google ScholarCross Ref
- Limor Shifman. 2012. An anatomy of a YouTube meme. New media & society 14, 2 (2012), 187–203. Publisher: Sage Publications Sage UK: London, England.Google Scholar
- Limor Shifman. 2013. Memes in a digital world: Reconciling with a conceptual troublemaker. Journal of computer-mediated communication 18, 3 (2013), 362–377. Publisher: Oxford University Press Oxford, UK.Google ScholarCross Ref
- Karen Simonyan and Andrew Zisserman. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. CoRR abs/1409.1556 (2014). http://arxiv.org/abs/1409.1556Google Scholar
- Ray Smith. 2007. An overview of the Tesseract OCR engine. In Ninth international conference on document analysis and recognition (ICDAR 2007), Vol. 2. IEEE, 629–633.Google ScholarDigital Library
- Shardul Suryawanshi and Bharathi Raja Chakravarthi. 2021. Findings of the Shared Task on Troll Meme Classification in Tamil. In Proceedings of the First Workshop on Speech and Language Technologies for Dravidian Languages. Association for Computational Linguistics, Kyiv, 126–132. https://aclanthology.org/2021.dravidianlangtech-1.16Google Scholar
- Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics 2 (2014), 67–78.Google ScholarCross Ref
Index Terms
- Multi-channel Convolutional Neural Network for Precise Meme Classification
Recommendations
Hashtag-Guided Low-Resource Tweet Classification
WWW '23: Proceedings of the ACM Web Conference 2023Social media classification tasks (e.g., tweet sentiment analysis, tweet stance detection) are challenging because social media posts are typically short, informal, and ambiguous. Thus, training on tweets is challenging and demands large-scale human-...
Organized Behavior Classification of Tweet Sets using Supervised Learning Methods
WIMS '18: Proceedings of the 8th International Conference on Web Intelligence, Mining and SemanticsThere is an increasing incidence in negative propaganda and fake news, which has recently gained lots of attention during the 2016 elections in United States, France, and United Kingdom. Bots and hired users collaborate to make messages seen and persist ...
Are Rumors Always False?: Understanding Rumors Across Domains, Queries, and Ratings
Advanced Data Mining and ApplicationsAbstractRumors are increasingly becoming a critical issue on the Web threatening democracy, economics, and society on a global scale. With the advance of social media networks, people are sharing content in an unprecedented scale. This makes social ...
Comments