Analyzing the semantic roles in Bangladeshi memes: A multimodal approach to contextual role labeling and explanation generation

Abstract

Interpreting the semantic roles of entities in memes requires navigating complex interactions between visual images, embedded texts, and implicit cultural knowledge. This is more challenging for languages with limited resources such as Bengali. Most existing works on this topic have only analyzed general concepts such as hate speech detection and sentiment analysis. They have not considered the detailed narrative roles that are required to comprehend the meaning of the memes. Our goal is to fill this gap and create an entity-based framework for semantic role labeling and humor explanation. To accomplish this, we have developed a labeled dataset of 2,658 Bengali memes, each labeled with roles such as Hero, Villain, and Victim, along with local context notes that were used to resolve cultural misunderstandings during training. We propose and evaluate two complementary frameworks: a Triple-Stream Fusion model utilizing cross-attention, and a Vision-Language Model (VLM) pipeline leveraging Qwen2-VL for holistic reasoning. Experimental results demonstrate that the VLM approach significantly outperforms the fusion baseline, achieving an accuracy of 0.8068 and a Macro-F1 score of 0.7953. Furthermore, our fine-tuned humor explanation generator achieved a significantly improved LLM-asa- Judge quality score of 7.33/10 (compared to a 6.03 baseline), confirming its ability to produce culturally grounded interpretations. Our study is a strong benchmark on the task of examining satire and social commentary in Bengali memes and provides a simple approach to jointly label roles and generate explanations by combining images, text, and context.