In this integrity breach aimed at an AI system, scholars managed to deceive AIOps tools:

AIOps pertains to the application of LLM-based agents to collect and evaluate application telemetry, encompassing system logs, performance metrics, traces, and alerts, to identify issues and subsequently propose or execute corrective measures. Entities such as Cisco have implemented AIOps within a conversational interface that administrators can utilize to inquire about system performance. Certain AIOps tools have the capability to react to these inquiries by automatically enacting solutions or recommending scripts to tackle the problems.

Nevertheless, these agents can be misled by fraudulent analytics data into making detrimental remedial choices, including reverting an installed package to a susceptible version.

The document: “When AIOps Become “AI Oops”: Subverting LLM-driven IT Operations via Telemetry Manipulation“:

Summary: AI for IT Operations (AIOps) is revolutionizing how organizations oversee complex software ecosystems by automating anomaly detection, incident diagnosis, and remediation. Contemporary AIOps solutions increasingly depend on autonomous LLM-based agents to decipher telemetry data and execute corrective measures with minimal human intervention, promising quicker response times and operational cost reductions.

In this research, we conduct the initial security evaluation of AIOps solutions, revealing that once again, AI-driven automation incurs significant security risks. We illustrate how adversaries can manipulate system telemetry to mislead AIOps agents into taking actions that undermine the integrity of the infrastructure they oversee. We introduce methods to reliably inject telemetry data through error-inducing requests that affect agent actions using a technique of adversarial reward-hacking; plausible yet erroneous interpretations of system errors that guide the agent’s decision-making. Our attack method, AIOpsDoom, is fully automated—merging reconnaissance, fuzzing, and LLM-driven adversarial input creation—and functions without any preceding knowledge of the target system.

To mitigate this threat, we suggest AIOpsShield, a defense strategy that sanitizes telemetry data by leveraging its structured nature and the limited role of user-generated content. Our tests demonstrate that AIOpsShield effectively prevents telemetry-based attacks without compromising the standard performance of agents.

Ultimately, this study unveils AIOps as a burgeoning attack vector for system infringement and emphasizes the urgent necessity for security-conscious AIOps design.