RobustMAD: Robustness of Small Multimodal Models for Anomaly Detection

Industry 4.0 has driven the need for multimodal visual inspection assistants that enable interactive queries based on natural language and computer vision. However, large multimodal language models (MLLMs) present significant challenges for on-site deployment: their high computational cost and privacy risks when relying on cloud inference make them impractical. As an alternative, multimodal small language models (MSLMs) emerge, lighter and suitable for local industrial environments, but their adoption is hindered by the lack of robustness analysis and benchmarks that reflect real factory conditions. RobustMAD is the first benchmark specifically designed to evaluate the robustness of these models in industrial anomaly detection, considering open-ended queries about objects, anomalies, unanswerable problems, and visual degradations. Results show that, surprisingly, some MSLMs outperform even larger models like GPT-5 Nano, yet they still fall short of safety-critical requirements. Three recurring failure modes are identified: (i) fragile multimodal grounding under fine-grained distinctions or degraded visual conditions, (ii) insufficiently comprehensive responses, and (iii) weak logical grounding on unanswerable queries, leading to hallucinations. These deficiencies represent real operational risks. From a technical and business perspective, it is essential to address these failures through careful architecture and training design. In this context, companies like Q2BSTUDIO offer custom software development services that integrate artificial intelligence, cybersecurity, cloud AWS/Azure, and BI/Power BI to build robust solutions tailored to industrial needs. For instance, an inspection assistant could be combined with AI agents that analyze images in real time, while cybersecurity ensures sensitive data protection in cloud environments. Incorporating BI dashboards allows monitoring model performance and detecting error patterns. The key is to iterate on the failures identified by RobustMAD: improve multimodal grounding through data augmentation with realistic visual degradations; design mechanisms to verify response completeness; and implement rejection logic for unanswerable questions. Additionally, using AWS or Azure cloud infrastructure facilitates horizontal scaling and continuous model updates. For the future, Q2BSTUDIO proposes combining MSLMs with rule-based quality control systems and reinforcement learning, creating assistants that not only detect anomalies but also explain their decisions in an understandable way. In short, RobustMAD not only exposes current weaknesses but also paves the way toward reliable, secure, production-ready industrial multimodal assistants.