GraphRAG-Based NLP at Risk: Graphemic Dot-Level Adversarial Attack on Arabic Sentiment and LLM Retrieval-Augmented Models

While research on adversarial attacks has advanced significantly, most studies on Natural Language Processing (NLP) have predominantly focused on English, leaving the vulnerabilities of models trained on other languages largely unexplored. These attacks pose a direct challenge to the reliability of AI systems used to interpret Arabic-language content on social media platforms, where sentiment analysis and content moderation tools are routinely deployed. This study introduces a novel graphemic dot-level adversarial attack specifically designed to target large language models (LLMs) trained on Arabic text. Unlike traditional adversarial attacks, our method manipulates dots within Arabic characters, leveraging common spelling errors made by non-native Arabic speakers to create imperceptible, deceptive, and highly effective adversarial examples. These modifications, though minimal, significantly degrade the performance of widely used Arabic text Machine Learning (ML) classifiers such as sentiment analysis models, and the responses of LLMs such as GPT-4o-mini used in LLM-based Retrieval-Augmented Generation (RAG) systems. Our experiments reveal that even advanced LLM-driven retrieval models, which rely on graph knowledge to enhance response accuracy, remain highly susceptible to our fine-grained perturbations. Our results, using Telegram data sets, demonstrate that offensive AI is effective in NLP models for low-resource languages such as Arabic and emphasize the need for defense mechanisms to mitigate the impact of such adversarial manipulations. (The code and data are publicly available in the authors’ GitHub repository)This material is based upon research supported by the U.S. Office of Naval Research under award number N000142212549.)