“`html

Imagine the Web as a digital realm governed by its own social contract. In 2014, Tim Berners-Lee advocated for a “Magna Carta for the Web” to restore the equilibrium of influence between individuals and institutions. This aligns with the original charter’s mission: ensuring that those inhabiting a territory have a substantial interest in its governance.

Web 3.0—the distributed, decentralized Web of the future—is finally set to transform the Internet’s landscape by reallocating ownership to data creators. This will alter many aspects of what’s often referred to as the “CIA triad” of digital security: confidentiality, integrity, and availability. Among these three attributes, data integrity will take on critical significance.

When we possess agency in digital environments, we instinctively uphold their integrity—protecting them from deterioration and intentionally molding them. However, in domains dominated by remote platforms, where we’re merely temporary guests, that connection weakens. A rift appears between those profiting from data and those experiencing the repercussions of compromised integrity. Similar to homeowners who are committed to maintaining their property, users in the Web 3.0 framework will become caretakers of their personal digital domains.

This will be essential in a realm where AI entities not only respond to our inquiries but also take action on our behalf. These entities may conduct financial operations, orchestrate intricate workflows, and autonomously manage crucial infrastructure, making choices that resonate across entire sectors. As digital agents grow more self-sufficient and interconnected, the pressing question shifts from whether we will trust AI to what the foundation of that trust consists of. In this new era we’re stepping into, the cornerstone is not intelligence or efficiency—it’s integrity.

What Is Data Integrity?

In information systems, integrity is the assurance that data will not be altered without permission, and that all transformations can be validated throughout the data’s lifecycle. While availability guarantees that systems are operational and confidentiality prevents unauthorized access, integrity concentrates on whether data is precise, unchanged, and coherent across systems and through time.

This is a novel concept. The undo feature, which averts accidental data loss, exemplifies an integrity attribute. Additionally, the reboot process restores a computer to a known good state. Checksums exemplify integrity features, as do validations of network transmission. Without integrity, security measures can backfire. Encrypting flawed data merely entraps mistakes. Systems rated highly for availability yet propagate misinformation merely become amplifiers of risk.

All IT systems necessitate some form of data integrity, but the demand for it is especially pronounced in two domains today. First: Internet of Things devices interact directly with the physical environment, so corrupted input or output can lead to real-world damage. Second: AI systems are only as effective as the integrity of the data they are trained on and the soundness of their decision-making processes. If that base is precarious, the outcomes will be, too.

Integrity is evident in four primary aspects. The first, input integrity, pertains to the quality and authenticity of data entering a system. When this falters, the fallout can be severe. In 2021, Facebook’s global outage was induced by a single erroneous command—a mistake overlooked by automated systems. Safeguarding input integrity demands robust authentication of data sources, cryptographically signing sensor information, and diverse input channels for cross-validation.

The second concern is processing integrity, which guarantees that systems accurately transform inputs into outputs. In 2003, the U.S.-Canada blackout impacted 55 million individuals when a control-room process failed to refresh correctly, resulting in losses exceeding US $6 billion. Protecting processing integrity necessitates formally verifying algorithms, cryptographically securing models, and monitoring systems for unusual behavior.

Storage integrity concerns the accuracy of information as it is stored and conveyed. In 2023, the Federal Aviation Administration was compelled to suspend all U.S. departing flights due to a corrupted database file. Mitigating this risk requires cryptographic techniques that make any alteration computationally infeasible without detection, distributed storage frameworks to avert single points of failure, and stringent backup protocols.

Lastly, contextual integrity pertains to the suitable flow of information according to the norms of its broader context. It is insufficient for data to be accurate; it must also be utilized in ways that honor expectations and boundaries. For instance, if a smart speaker eavesdrops on casual family discussions and utilizes the data to build advertising profiles, that action would breach the anticipated limits of data collection. Maintaining contextual integrity necessitates clear data governance policies, principles that restrict data usage to its intended purposes, and mechanisms for enforcing information-flow constraints.

As AI systems increasingly undertake significant decisions with diminished human oversight, all these dimensions of integrity become vital.

The Need for Integrity in Web 3.0

As the digital arena transitions from Web 1.0 to Web 2.0 and now evolves toward Web 3.0, we’ve observed each era emphasize different aspects within the CIA triad of confidentiality, integrity, and availability.

Returning to our home analogy: When mere shelter is the foremost concern, availability takes precedence—the house must exist and function. Once that foundation is solid, confidentiality becomes necessary—locks on doors are required to keep others out. Only after establishing these fundamentals do you start to contemplate integrity, to guarantee that what resides inside the house remains reliable, unchanged, and consistent over time.

Web 1.0 of the 1990s focused on making information accessible. Organizations digitized their content, disseminating it for anyone to retrieve. In Web 2.0, the current iteration of the Web, e-commerce, social media, and cloud computing platforms prioritize confidentiality, as personal data has evolved into the currency of the Internet.

Yet, integrity seems to have been largely overlooked along the journey. In our present Web structure, where control is centralized and distanced from individual users, the emphasis on integrity has waned. The colossal social media platforms have created
“““html

contexts where individuals do not feel accountable for the accuracy or quality of information that spreads.

Web 3.0 is set to transform this situation by reinstating ownership to the data proprietors. This is not hypothetical; it is already taking shape. For instance, ActivityPub, the framework behind decentralized social networks such as Mastodon, merges content distribution with built-in attribution. Tim Berners-Lee’s Solid protocol reconfigures the Web around personal data containers featuring detailed access controls.

These innovations emphasize integrity through cryptographic validation that confirms authorship, decentralized frameworks that remove weak central authorities, machine-readable semantics that clarify meaning—structured data formats that enable computers to comprehend participants and actions, such as “Alice conducted surgery on Bob”—and transparent governance where rules are accessible to everyone. As AI systems become increasingly independent and communicate directly through standardized protocols, these integrity measures will be vital for sustaining trust.

Why Data Integrity Is Essential in AI

Integrity is vital for AI systems across four areas. The first is decision-making quality. With AI playing a larger role in health care, justice, and finance, the integrity of both data and the operating models fundamentally influences human well-being. The second area is accountability. Understanding failure causes necessitates dependable logging, audit trails, and system documentation.

The third aspect concerns the security among various components. Numerous authentication systems depend on the integrity of identity data and cryptographic keys. If these factors are compromised, malicious actors may impersonate reliable systems, potentially leading to systemic failures as AI agents interact and make decisions using corrupted credentials.

Last but not least, integrity is crucial in our societal definitions of safety. Governments globally are enacting regulations for AI that emphasize data precision, transparent algorithms, and verifiable assertions about system operations. Integrity serves as the foundation for fulfilling these legal responsibilities.

The significance of integrity only amplifies as AI systems are entrusted with more critical functions and function with minimal human intervention. While humans can sometimes catch integrity lapses, autonomous systems may not only overlook warning signals—they might exponentially escalate the severity of breaches. Without guarantees of integrity, organizations will hesitate to rely on AI systems for vital tasks, and we won’t unlock the full potential of AI.

Strategies for Developing AI Systems With Integrity

Picture an AI system as a home we’re collaboratively constructing. The integrity of this residence relies not on a single security element but on the thoughtful amalgamation of several aspects: strong foundations, well-built walls, clear pathways between rooms, and mutual agreements regarding how areas will be utilized.

We commence by setting the cornerstone: cryptographic validation. Digital signatures guarantee that data lineage can be traced, much like a title deed substantiates ownership. Decentralized identifiers function as digital passports, enabling components to validate identity independently. When the front entrance of our AI residence identifies visitors through their own keys instead of a susceptible central doorman, we foster resilience in the trust architecture.

Formal verification techniques empower us to mathematically confirm the structural integrity of essential components, ensuring systems can endure pressures exerted on them—especially in high-stakes areas where lives may rely on an AI’s conclusion.

Just as a well-designed home features distinct areas, trustworthy AI systems are created through meticulous compartmentalization. We do not depend on a single barrier but rather layer them to restrict how issues in one section might influence others. Just as a kitchen fire is contained by fire doors and separate smoke alarms, training data is isolated from the AI’s deductions and outputs to minimize the consequences of any individual failure or breach.

In every inch of this AI residence, we incorporate transparency into the architecture: The equivalent of expansive windows that let light flood into every area is clear pathways from input to output. We implement monitoring systems that continually scan for vulnerabilities, alerting us before minor concerns turn into catastrophic breakdowns.

However, a home isn’t merely a physical framework; it’s also the agreements we establish about how to coexist within it. Our governance structures represent these collective understandings. Before inviting new inhabitants, we provide them with certification criteria. Just as landlords perform credit assessments, we conduct integrity evaluations to vet newcomers. Moreover, we aspire to be good neighbors, aligning our community standards with broader societal norms. Perhaps most importantly, we acknowledge that our AI residence will shelter diverse individuals with distinct requirements. Our governance frameworks must mirror this diversity, bringing various stakeholders into the discussion. A genuinely trustworthy system cannot be crafted solely for its creators but must cater to anyone authorized to ultimately call it home.

That’s how we’ll build AI systems worthy of trust: not by naively relying on their flawlessness but because we have deliberately designed them with integrity measures at every tier.

A Linguistic Challenge

Unlike other attributes of security, such as “available” or “private,” we lack a common adjective form for “integrity.” This complicates discussions around it. Interestingly, there is an English term: “integrous.” The Oxford English Dictionary noted its usage in the mid-1600s but now calls it obsolete.

We contend that the term warrants revival. We need a way to describe a system characterized by integrity. We must be capable of discussing integrous systems design.

The Path Forward

Ensuring integrity in AI poses significant challenges. As models scale up and grow in complexity, retaining integrity without compromising performance becomes problematic. Integrity controls often necessitate computational resources that can hinder system efficiency—particularly challenging for real-time applications. Another issue is that emerging technologies such as quantum computing jeopardize existing cryptographic safeguards. Furthermore, the distributed nature of contemporary AI—which depends on extensive ecosystems of libraries, frameworks, and services—creates a vast attack surface.

Beyond technology, integrity is deeply influenced by social dynamics. Companies frequently prioritize speed to market over comprehensive integrity controls. Development teams may lack the specialized expertise to implement these controls and find it especially challenging to integrate them into legacy systems. While some governments have started enacting regulations for certain AI aspects, there is an urgent need for global harmony on governance.

“““html

for AI reliability.

Confronting these issues necessitates ongoing investigation into validating and maintaining reliability, in addition to recovering from infringements. Key focus areas encompass fault-tolerant algorithms for distributed learning, certifiable computation on encoded data, methodologies that preserve reliability in the face of hostile attacks, and standardized criteria for accreditation. Furthermore, we require interfaces that effectively convey reliability status to human supervisors.

As AI systems evolve and become more integrated into society, the importance of integrity has reached unprecedented levels. We are entering a phase where interactions between machines and autonomous agents will function with minimal human intervention and make decisions with significant consequences.

The encouraging aspect is that the resources for developing systems with integrity are already available. What is essential is a transformation in perspective: moving from viewing integrity as an afterthought to recognizing it as the fundamental organizing principle of AI security.

The forthcoming technological era will be characterized not by AI capabilities, but by our ability to trust it to understand or, more crucially, to accomplish what is right. Integrity—in all its aspects—will determine the outcome.

Sidebar: Instances of Integrity Failures

Ariane 5 Rocket (1996)
Processing reliability failure
A 64-bit velocity calculation was transformed into a 16-bit output, causing an error known as overflow. This compromised data prompted disastrous course corrections, resulting in the US $370 million rocket’s self-destruction.

NASA Mars Climate Orbiter (1999)
Processing reliability failure
Lockheed Martin’s software determined thrust in pound-seconds, while NASA’s navigation software anticipated newton-seconds. This discrepancy led to the $328 million spacecraft disintegrating in the Mars atmosphere.

Microsoft’s Tay Chatbot (2016)
Processing reliability failure
Unveiled on Twitter, Microsoft‘s AI chatbot was susceptible to a “repeat after me” command, meaning it would replicate any offensive material presented to it.

Boeing 737 MAX (2018)
Input reliability failure
Incorrect sensor data caused an automated flight-control system to continually push the airplane’s nose downward, resulting in a catastrophic crash.

SolarWinds Supply-Chain Attack (2020)
Storage reliability failure
Russian hackers compromised the procedure that SolarWinds employed to package its software, introducing malicious code that affected 18,000 customers, including nine federal agencies. The breach remained unnoticed for 14 months.

ChatGPT Data Leak (2023)
Storage reliability failure
A flaw in OpenAI’s ChatGPT combined various users’ chat histories. Users unexpectedly found other individuals’ conversations in their interfaces without any means to verify that the discussions weren’t theirs.

Midjourney Bias (2023)
Contextual reliability failure
Users noticed that the AI image generator frequently produced biased images of individuals, such as depicting white males as CEOs regardless of the prompt. The AI tool failed to accurately represent the context requested by users.

Prompt Injection Attacks (2023–)
Input reliability failure
Attackers incorporated concealed prompts within emails, documents, and websites that hijacked AI assistants, making them perceive malicious commands as acceptable instructions.

CrowdStrike  Outage (2024)
Processing reliability failure
A defective software update from CrowdStrike resulted in 8.5 million Windows computers globally crashing—grounding flights, shutting down hospitals, and causing disruptions in banks. The update, which included a software logic error, had not undergone thorough testing protocols.

Voice-Clone Scams (2024)
Input and processing reliability failure
Fraudsters utilized AI-driven voice-cloning technology to imitate the voices of victims’ family members, deceiving individuals into transferring money. These scams succeeded because neither the phone systems nor the victims recognized the AI-generated voice as fraudulent.

This article was co-written with Davi Ottenheimer and originally appeared in IEEE Spectrum.

“`