Abstract
MeViTSA: Multimodal Ensemble Approach for Sentiment Analysis of Visually Integrated Text Data
Pattern Recognition. ICPR 2026 International Workshops, pp.433-447
Lecture Notes in Computer Science
ICPR 2026, 28th (Lyon, France, 17/08/2026–22/08/2026)
2027
Metrics
1 Record Views
Abstract
Existing Multimodal Sentiment Analysis (MSA) systems often have difficulty in understanding images with embedded text, like memes, ads and infographics. Existing methods considered any embedded text on an image to be some visual noise. This study presents MeViTSA, a dual-stream multimodal ensemble framework created to distinctly separate and analyze visual and linguistic meanings. The system employs a CLIP encoder for processing the input image and a T5 based classifier for handling the textual data extracted using Google Cloud Vision OCR from the input image, and combined through a fixed late fusion approach. A notable advancement is the addition of a text-extraction fallback feature, that is if OCR fails to find any text, a BLIP powered component creates a caption acting as a substitute textual cue guaranteeing that it improves architectural robustness and maintains a continuous semantic stream for text-deficient images. To rigorously evaluate this approach, we present DocImSentv1, a manually curated and balanced dataset of 2,054 images refined from the DocImSent benchmark. Extensive experiments demonstrate that MeViTSA establishes a new state-of-the-art, achieving 96.59% accuracy on DocImSentv1 and outperforming existing baselines on the widely used MVSA-Single and MVSA-Multiple datasets. Our dataset, DocImSentv1, will be available after the acceptance of this paper. The corresponding code repository can be accessed here.
Details
- Title
- MeViTSA: Multimodal Ensemble Approach for Sentiment Analysis of Visually Integrated Text Data
- Creators
- Soham Bhattacharya - Ramakrishna Mission Vivekananda Educational and Research InstituteAli Reza Alaei - Southern Cross UniversitySukalpa Chanda - Østfold University of Applied SciencesUmapada Pal - Indian Statistical Institute
- Contributors
- Ida-Maria Sintorn (Editor of compilation) - Uppsala UniversityUmapada Pal (Editor of compilation) - Indian Statistical InstituteShijian Lu (Editor of compilation) - Nanyang Technological University
- Publication Details
- Pattern Recognition. ICPR 2026 International Workshops, pp.433-447
- Conference
- ICPR 2026, 28th (Lyon, France, 17/08/2026–22/08/2026)
- Series
- Lecture Notes in Computer Science
- Publisher
- Springer Nature Switzerland; Cham
- Identifiers
- 9783032393784; 9783032393791; 991013405403102368
- Copyright
- © 2027 The Author(s), under exclusive license to Springer Nature Switzerland AG.
- Academic Unit
- Information Technology; School of Business and Tourism; Faculty of Science and Engineering; Engineering
- Language
- English
- Resource Type
- Abstract