If you’ve ever attended a computer security course, you’ve likely familiarized yourself with the three pillars of computer security—confidentiality, integrity, and availability—termed the CIA triad. When discussing the security of a system, that is what we reference. All three are essential, but their importance varies in different scenarios. In a realm filled with artificial intelligence (AI) systems and intelligent agents, the concept of integrity will be crucial.
What does data integrity entail? It’s about ensuring that data cannot be altered—that’s the security perspective—but it extends beyond that. It includes the precision, completeness, and quality of data—all across both time and space. It aims to prevent accidental data loss; the “undo” function serves as a basic integrity tool. Furthermore, it ensures that data remains accurate at the moment of collection—that it originates from a reliable source, that significant information isn’t omitted, and that it doesn’t alter as it transitions between different formats. The capacity to reboot your computer also serves as an additional integrity safeguard.
The CIA triad has adapted in response to the Internet’s evolution. The initial phase of the Web—Web 1.0 from the 1990s to the early 2000s—emphasized availability. During this time, organizations and individuals hurried to digitize their content, resulting in an unmatched archive of human knowledge. Businesses globally established their online presence, leading to extensive digitization initiatives where volume was prioritized over quality. The focus on making information accessible overshadowed other considerations.
As web technologies advanced, attention shifted toward safeguarding the vast volumes of data flowing through online platforms. This marks Web 2.0: the current phase of the Internet. Interactive functions and user-generated content transformed the Web from a static medium into a collaborative platform. The surge of personal data and the rise of interactive e-commerce, social media, and other online environments necessitated both data protection and user privacy. Confidentiality took precedence.
We are on the brink of a new Web era: Web 3.0. This is a decentralized, distributed, intelligent Web. Peer-to-peer social networking systems aim to disrupt the control that technology monopolies have on our interactions. Tim Berners-Lee’s open W3C protocol, Solid, signifies a significant transformation in our perceptions of data ownership and governance. A future dominated by AI agents requires personal data and computations that are verifiable and trustworthy. In this landscape, data integrity will be at the forefront.
For instance, the 5G communications revolution isn’t merely about quicker access to videos; it’s about Internet-connected devices communicating with other Internet-connected devices autonomously. Absent data integrity, real-time vehicle-to-vehicle communication regarding traffic movements and conditions would be nonexistent. There wouldn’t be synchronized drone swarms, smart energy grids, or dependable mesh networks. Furthermore, there would be no secure means to empower AI agents.
Specifically, AI systems necessitate strong integrity controls due to their data processing methods. This entails technical measures to guarantee that data is precise, that its meaning is upheld during processing, that it yields trustworthy outcomes, and that humans can reliably modify it when inaccuracies arise. Just as a scientific device needs calibration to accurately gauge reality, AI systems require integrity controls that maintain the link between their data and the ground truth.
This extends beyond merely preventing data manipulation. It involves creating systems that sustain verifiable trust relationships between their inputs, processing, and outputs, enabling humans to comprehend and validate the AI’s actions. AI systems demand clean, consistent, and verifiable control mechanisms to learn and make sound decisions. Without this foundation of verifiable truth, AI systems risk devolving into a series of obscure boxes.
Recent events demonstrate numerous unsettling instances of integrity failures that naturally erode public confidence in AI systems. Machine-learning (ML) models trained without adequate consideration on large datasets have yielded predictably biased outcomes in hiring processes. Autonomous vehicles relying on incorrect data have made erroneous—and sometimes deadly—decisions. Medical diagnostic systems have provided flawed recommendations without the ability to justify their conclusions. A failure to implement integrity measures undermines AI systems and adversely affects those who rely on them.
These instances also emphasize how failures in AI integrity can emerge at various levels within system operations. At the training stage, data may be subtly tainted or biased even before any model development is initiated. At the model stage, foundational mathematics and training methodologies can introduce new integrity challenges even when working with clean data. During execution, environmental changes and runtime adjustments can compromise previously valid models. Moreover, at the output stage, the difficulty of verifying AI-generated content and tracking it through system processes introduces new integrity issues. Each level exacerbates the challenges that precede it, ultimately leading to human consequences, such as reinforced biases and reduced agency.
Consider it akin to securing a home. You don’t merely lock a door; you also ensure strong concrete foundations, durable framing, a solid roof, secure double-pane windows, and perhaps motion-sensor surveillance. In the same vein, we require comprehensive digital security at every layer to guarantee that the entire system can be trusted.
This multi-layered approach to security understanding becomes progressively vital as AI systems increase in complexity and autonomy, especially with large language models (LLMs) and deep learning frameworks making impactful decisions. Verifying the integrity of each layer is paramount when developing and deploying digital systems that influence human lives and societal outcomes.
At the foundational layer, information is stored in computer hardware. This constitutes the most fundamental encoding of our data, model weights, and computational guidelines. The next layer encompasses the file system structure: how those binary sequences are arranged into structured files and directories that a computer can access and process efficiently. In AI systems, this includes the methods of storing and organizing training data, model checkpoints, and hyperparameter configurations.
Why is this layered viewpoint significant? Vulnerabilities and integrity issues can surface at any level, so comprehending these layers aids security specialists and AI researchers in conducting thorough threat assessments. This permits the implementation of defense-in-depth strategies—from cryptographic verification of training data to robust model architectures to interpretable outputs. This multi-layered security framework becomes especially vital as AI systems assume more autonomous decision-making roles in critical fields such as healthcare, finance, and public safety. We must guarantee integrity and reliability at every layer of the stack.
The dangers of deploying AI without adequate integrity controls are severe and frequently underestimated. When AI systems function without sufficient security measures to address corrupted or manipulated data, they can generate subtly flawed outputs that appear valid on the surface. These failures may cascade through interconnected systems, magnifying errors and biases. In the absence of appropriate integrity
controls, an AI framework may be trained on contaminated data, make choices based on erroneous perceptions, or have outputs modified without notice. The consequences of this may vary from diminished performance to disastrous breakdowns.
We identify four domains where integrity is crucial in this Web 3.0 environment. The first is detailed access, which permits users and organizations to retain exact control over who can interact with and alter which information and for what objectives. The second is verification—much more intricate than the basic “Who are you?” verification systems of today—which guarantees that data access is correctly authenticated and approved at every stage. The third is clear data ownership, which allows data proprietors to be aware of when and how their data is utilized and establishes an auditable record of data origination. Lastly, the fourth is access uniformity: shared interfaces and protocols that facilitate consistent data access while preserving security.
Fortunately, we’re not beginning from zero. There are open W3C protocols that tackle some of these issues: decentralized identifiers for verifiable digital identity, the verifiable credentials data model for articulating digital credentials, ActivityPub for decentralized social networking (the technology behind Mastodon), Solid for distributed data storage and access, and WebAuthn for robust authentication standards. By offering standardized methods to verify data origin and maintain data integrity throughout its lifecycle, Web 3.0 cultivates the trustworthy environment essential for AI systems to function dependably. This architectural advancement for integrity control empowers users and helps guarantee that data remains reliable from creation and acquisition to processing and storage.
Integrity is vital for trust, both on technological and personal fronts. Looking ahead, integrity controls will significantly influence AI growth by transitioning from optional features to fundamental architectural necessities, much like how SSL certificates progressed from a banking luxury to a standard expectation for any web service.
Web 3.0 protocols can integrate integrity controls into their core, creating a more dependable framework for AI systems. Currently, we take availability as a given; anything below 100% uptime for essential websites is unacceptable. In the future, we will require the same guarantees for integrity. Achieving success will demand adherence to practical guidelines for maintaining data integrity throughout the AI lifecycle—from data collection through model training and eventually to deployment, usage, and evolution. These guidelines will cover not only technical measures but also governance structures and human supervision, akin to how privacy policies transformed from legal jargon into detailed frameworks for data stewardship. Shared standards and protocols, developed through industry collaboration and regulatory measures, will ensure consistent integrity controls across various AI systems and applications.
Just as the HTTPS protocol laid the groundwork for trusted e-commerce, it’s imperative to establish new integrity-centered standards to facilitate the trusted AI services of the future.
This article was crafted with Davi Ottenheimer and first appeared in Communications of the ACM.