Vision-language models (VLMs) face systematic limitations in multilingual applications. Multiple studies from the past twelve months document weaknesses in security, semantic understanding, and cultural adaptation. Research shows that English-centric design restricts performance in other languages.

English-centricity dominates model design

The TowerVision study, published on October 22, 2025 in revised form on November 6, 2025, demonstrates that most VLMs follow an English-centric design process. The investigation analyzed training data composition, encoder selection, and text backbones.

In response, researchers developed TowerVision, a family of open multilingual VLMs for image-text and video-text tasks, built on the multilingual text model Tower+. The system was evaluated in nine languages: English, European Portuguese, French, Dutch, German, Spanish, Italian, Korean, and Chinese.

Security risks in multilingual contexts

The MLingualFC study from June 5, 2026 demonstrates that VLMs have significant vulnerability to jailbreak attacks in multilingual environments. Structural visual prompts such as flowcharts can effectively overcome security measures.

The Multi-Everything study from November 6, 2024 revealed further challenges in hate speech detection: VLMs must handle multimodal, multilingual content and different cultural perceptions – a task in which current systems show weaknesses.

Semantic and syntactic limitations

The Seeing Syntax study from December 11, 2024 documents significant limitations in the text encoders of VLMs. The systems particularly fail at compositionality and semantic understanding of complex linguistic structures.

Additional problems arise with vision-language translation (VLT). A study from June 13, 2025 shows that VLT requires precise detection of multilingual text in images and its translation into the target language with visual context support. Despite strong multilingual capabilities of newer large vision-language models (LVLMs), there is a lack of systematic evaluation of their VLT performance.

TowerVision shows improvements through cultural context

TowerVision achieved competitive results on several multimodal multilingual benchmarks. The model outperforms existing approaches on culturally grounded tasks and multimodal translation – through integration of visual and cultural context during fine-tuning.

The results:

  • Improved performance on ALM-Bench and Multi30K (image tasks) and ViMUL-Bench (video tasks)
  • TowerVision outperformed existing models trained on substantially larger datasets
  • Multilingual vision-language training data improved cross-lingual generalization from high-resource to underrepresented languages and vice versa
  • Instruction-tuned LLMs did not always prove to be the optimal initialization point

Survey identifies area of tension

A survey paper from September 26, 2025 examined 31 multilingual VLMs and 21 benchmarks across encoder-only and generative architectures. The authors identified a key tension between language neutrality and practical multilinguality.

The research landscape shows that vision-language models exhibit fundamental fragility in multilingual contexts. Integration of multilingual training data and cultural contexts during training and fine-tuning proves critical for more robust performance – while significant challenges in security, syntactic understanding, and consistent multilinguality remain.