SOURCE CODE: A Policy Agenda for Fostering Trust and Fairness in AI
AI systems are rapidly becoming part of the machinery of public life, but they sit on shaky foundations. We have seen AI being deployed for cancer screening, assisting people with disabilities, and even to address complex environmental challenges. Yet as these systems are deployed in increasingly consequential settings, the promise of AI to expand opportunity and increase effectiveness and boost productivity has been accompanied by harms that are no longer hypothetical. These harms fall into several recurring categories: systems can misallocate resources, misrepresent groups, fail to function reliably, or be deployed for illegitimate purposes, even when the technology works as intended.
For example, AI has affected who gets allocated critical resources. A widely used healthcare algorithm underestimated the needs of Black patients, limiting access to care. In finance, algorithmic decision-making has produced discriminatory outcomes in lending and underwriting.
Another observed harm is that failures in AI rollout can affect how people and communities are represented. AI systems have been shown to reinforce harmful stereotypes or render certain groups invisible altogether, particularly when they are trained on incomplete or biased data. A well known example is how facial recognition technologies are shown to perform significantly worse on darker-skinned individuals, raising concerns about misidentification and disproportionate surveillance.
Still, other harms are failures of basic system functionality. Gunshot detection systems have generated large numbers of false alerts, manipulating how evidence is used in criminal proceedings. The Michigan Integrated Data Automated System (MiDAS), which was used to find instances of fraud in state unemployment benefits, was incorrect in 85% of its fraud determinations.
Another category of concern arises not from failures in system design or performance, but from how AI systems are built and used in practice. These harms arise not from system failure but from how situations in which systems operate as intended yet still produce harmful outcomes. For example, algorithmic management tools in the workplace could intensify worker surveillance, destabilize scheduling, or reduce worker autonomy, even when operating accurately.
These harms explain why AI is facing a crisis of public trust. A June 2025 Pew study found that half of U.S. adults feel “more concerned than excited about the growing use of AI”, with only a small minority expressing optimism. AI cannot deliver broad public benefits, such as improved public services, if the people affected by it do not trust the systems shaping their lives. The public will not trust abstract statements of fairness, transparency, responsibility, or legitimacy. It will be rebuilt only if those commitments are translated into procedural institutional mechanisms: procurement rules, public engagement processes, sector-specific safeguards, and holistic remedies. Such commitments will mean that AI can be used legitimately in the public interest.
In many cases, the concern is whether the technology should even be used. This question raises deeper considerations around the concentration of power, human dignity, and the conditions under which innovation actually benefits the public. While AI can be used to support societal benefits, such as helping overworked healthcare practitioners, it can also be used in ways that harm human dignity, such as through surveillance or by restricting fair access to benefits or create new vulnerabilities like cybersecurity and data privacy risks. Building fairness and trust in AI requires more than improving system performance. Policymakers and the public must ask whether particular uses are legitimate, who benefits, who bears the risks, and what limits they should set and enforce.
Because these questions cannot be answered by abstract principles alone, the Federation of American Scientists worked with experts and practitioners across civil society and academia to bring together a policy agenda with ten actionable and high-impact solutions. We did this through our SOURCE CODE: AI Trust and Fairness Policy Sprint. When problems are urgent, institutions are uncertain, and traditional policymaking moves too slowly, our policy sprints create space to bring together experts across disciplines, from academics to technologists, advocates, and practitioners, and empower them to move quickly from diagnosis to action. Instead of debating what trust and fairness mean in the abstract, this sprint focused on what they look like in practice, and how it can be operationalized through specific policy levers.
This paper proceeds in three parts. First, we examine how fairness and trust are understood across different contexts, and why that creates friction in how we map the space. Second, we explore the challenges of implementing policy designed to install fairness guardrails. We highlight how gaps in policy, capacity, and real-world conditions can undermine even well-intentioned systems. Finally, we present a set of policy strategies across these key levers, offering actionable pathways for building fairer and more trustworthy AI systems.
What do we mean by ‘trust and fairness’ in AI?
Fairness and trust are often invoked as key components of AI governance and policy, but they can seem nebulous depending on the context and the community at hand. In general, we consider public trust to be the extent to which people see systems and institutions as reliable, accountable, and responsive to harms. AI fairness broadly concerns whether AI systems distribute benefits and burdens in ways that can be justified within a particular social, legal, and institutional context. Together, fairness and trust point to a broader question of legitimacy: whether an AI system should be used at all, for what purpose, and under what auspices.
This section will reflect on how different stakeholders view fairness and public trust to examine each perspective before turning to existing legal instruments, policy gaps, and what is needed for effective policy implementation in this field.
On fairness
Fairness in AI is not a single concept but is defined differently across technical, legal, and social domains. This plurality reflects how AI systems are “sociotechnical systems”: their effects depend not only on data and algorithms but also on the institutions, incentives, rules, and human decisions that shape their deployment.
In examining how society in general views fairness, literature shows that individuals often understand fairness not as a clearly defined principle but in contrast to experiences of unfairness– that is, the absence of harm. This means that individuals, communities, and different cultures perceive fairness differently based on their own lived experiences. In terms of AI fairness, society views fairness as specifically related to terms such as “equity, consistency, non-discrimination, impartiality, justice, honesty, and reasonableness.”
When evaluating fairness in AI systems, technical literature often distinguishes between two broad concepts. The first, individual fairness, asks whether similar individuals are treated comparably for a given task. This approach requires defining which characteristics are relevant to the task and what it means for two people to be meaningfully similar. The second, group fairness, examines whether outcomes are distributed unequally across groups. For example, in hiring, one group fairness approach might ask whether candidates from different demographic groups are selected at similar rates, while other approaches might focus on whether error rates or predictive accuracy differ across groups.
There are differences of opinion about what actually constitutes equal or similar outcomes in these definitions and how to predict them. On the one hand, historical data may be treated as a valid basis for predicting future outcomes. On the other hand, historical data itself could be shaped by historical and structural inequities, causing systems trained on it to reproduce existing patterns of discrimination. These competing considerations around historical data can create situations where an AI system is meeting one definition of fairness, but the actual outcome is creating unequal harms to a group or individual. The ProPublica investigation of the COMPAS risk assessment tool, an algorithm used to support criminal justice officials’ decisions on bail, sentencing, and early release, found that Black defendants were twice as likely to be labeled as high risk as white defendants. In theory, COMPAS satisfied one fairness criterion, in this case predictive parity, which means that risk scores are equally accurate across groups, but it also propagated the systemic inequalities of the U.S. criminal justice system.
Fairness cannot be understood as a fixed technical standard, but rather as a contested concept shaped by social, institutional, and legal contexts. Determining which definition of fairness should govern a particular AI system is a nuanced decision, one that requires a deep understanding of the sectoral context, stakeholders, and other elements within the scope of the AI system’s deployment. This becomes even more challenging when relying on existing legal frameworks that may only partially address the complexities of AI-driven decision-making.
On public trust
What does it mean for technology itself to be worthy of trust and, in turn, of adoption? Public trust in AI is not generated by technical performance alone. It is built when people can see that an AI system serves a legitimate purpose, works reliably in its deployment context, and remains subject to meaningful human oversight, public accountability, and remedies when things go wrong. We also explicitly see fairness as a component of public trust, and that the public will not trust AI if they view that its allocations of opportunities and burdens are unfair. Ultimately, public trust is dependent on the public seeing the use of AI as justified and legitimate. If it falters, then AI use will be seen as negative to society. For example, the prospect of AI-driven worker substitution is a major source of public concern, raising questions about whether the use of AI to replace human labor is legitimate. If such a substitution occurs at scale, it could further erode public trust in AI systems and their deployment.
Public sector adoption of AI is where governance approaches are first tested in practice, shaping both regulatory norms and broader public expectations. When government agencies deploy AI systems, they are effectively signaling what responsible use looks like. As a result, failures in public sector systems can have outsized consequences. When government use of AI leads to unfair outcomes, unreliable decisions, or a lack of accountability, it can erode trust not only in AI but in the government itself. For example, the use of automated prior authorization systems in Medicare Advantage has been associated with higher denial rates and barriers to medically necessary post-acute care, something that could directly affect public attitudes towards AI adoption in government services.
Across the federal government, different administrations have relied on the discourse that public trust in AI systems is essential to ensuring that the technology is disseminated across the public sphere. For example, Executive Order 13859 under the first Trump administration explicitly called for the use of AI in a manner that “fosters public trust and confidence,” a sentiment that carried over into Biden-era executive actions and remains in the current Trump administration OMB guidance, which defines how the federal government uses and acquires AI. These efforts have generally focused on identifying broad, high-risk uses of AI in the federal government and then pairing them with risk-mitigation requirements, leaving federal agencies to define implementation details and build the internal capacity to identify and enforce protections against AI uses that could erode public trust.
How can existing law be used as a tool for trust and fairness?
Existing anti-discrimination law provides an important, but incomplete, set of tools for addressing AI-related harms. In employment, Title VII of the Civil Rights Act of 1964 prohibits discrimination based on race, color, religion, sex, and national origin. Later case law and statutory amendments, including the Civil Rights Act of 1991, developed a framework that distinguishes between disparate treatment, where a person is intentionally treated differently because of a protected characteristic, and disparate impact, where a facially neutral practice causes unjustified adverse effects for a protected group. This framework is highly relevant to AI systems, which may produce unequal outcomes even when they do not explicitly use protected characteristics.
Existing statutory tools are also directly applicable to AI-related consumer harms that occur in specific sectors. The Equal Credit Opportunity Act (ECOA), for instance, remains a powerful tool for addressing bias and discrimination in financial services, especially in cases where discrimination arises from algorithmic decision-making outputs playing a determinative role in financial outcomes such as loan decisions. Recent enforcement actions clearly demonstrate how these existing laws can be applied in practice to algorithmic harms to build the much-needed public trust in AI systems. In 2025, the Massachusetts Attorney General’s Office fined a financial services company that used AI for student loan underwriting after determining that its algorithmic outputs were discriminatory. As part of the legal remedy, the company was required to inventory its models and retrain them to comply with anti-discrimination, consumer protection, and fair lending laws. Similarly, the Federal Trade Commission has already used its authority to address harmful AI deployments. Its settlement with Rite Aid over the use of facial recognition technology included an unfairness claim, underscoring that discriminatory or harmful AI practices can fall squarely within existing prohibitions on “unfair” conduct. These examples illustrate that while regulators are able to act, such interventions are reactive, occurring after harms have already materialized, and may not provide detailed guidance for how systems should be designed or governed in advance.
Yet, without statutory reform, the contemporary reliance on existing law and legal frameworks is out of necessity rather than choice. This is in part because legal frameworks typically do not define fairness in the abstract. Instead, the law operationalizes it through established doctrines, standards, and enforcement mechanisms. Many emerging AI concerns can be partially mapped onto well-established legal principles with effective recourse and remedies for consumers. This continuity becomes especially apparent in consumer-facing contexts.
For example, trust and fairness in consumer-facing technologies such as AI closely align with longstanding notions of consumer product safety. As such, public concerns about whether AI systems are reliable, transparent, and non-harmful mirror traditional expectations that the physical products consumers purchase should not pose undue risks or potential harms. In addition to notions of what constitutes product safety, the American legal system has long articulated what constitutes “unfair” conduct through statutes such as the Federal Trade Commission Act and the Dodd-Frank Act’s prohibition on unfair, deceptive, or abusive acts or practices (UDAAP).
In the absence of clear enforcement and interpretive guidance, however, legal gaps can translate into diffuse or ambiguous accountability, undermining public confidence in both the technologies themselves and the institutions responsible for overseeing them. One such example of this dynamic leading to widespread unease and declining public trust in technology is the bipartisan frustration over digital payment apps. Apps that use AI tools for automated content moderation or fraud detection can unknowingly “debank” and terminate the accounts of otherwise welcome clients.
Policymakers therefore face a difficult bind: without more deliberate efforts to operationalize fairness and accountability, public trust will remain elusive. Legal standards alone are not enough; they must be translated into systems that people experience as fair, reliable, and responsive to harm.
In the absence of new laws that directly address the challenges posed by artificial intelligence, practitioners must rely on existing legal frameworks that offer only partial mechanisms for accountability and redress when harm arises. This creates particular challenges in areas where concepts such as “trust” and “fairness” in AI systems are not explicitly codified, and where legal precedent is still limited or emerging. Existing legal frameworks may provide an important starting point for building public trust, but they do not fully capture the range of concerns raised by AI systems or the conditions needed to sustain public trust over time.
The challenge of implementation: why good intentions fail to produce trusted outcomes
In a rapidly evolving policy climate, governments across jurisdictions have implemented or proposed to create fairer, more trustworthy algorithmic systems and to protect people from harms. When designing policy interventions, it is important to take into account how prepared institutions are in taking on these responsibilities, and what resources, such as talent, processes, and even data availability, exist to ensure that a new policy has a fighting chance to succeed.
Take, for example, federal agencies’ implementation of OMB Memorandum M-24-10, a government-wide guidance document on how the federal government uses and acquires AI. Agencies are supposed to publicly publish a compliance plan describing the processes they will undertake to follow said guidance. In our analysis of these compliance plans, we found that agencies vary in technical expertise, staffing capacity, and institutional resources, which leads to inconsistent compliance and fragmented oversight practices. In this case, governance frameworks alone are insufficient; effective implementation depends on sustained investment in technical talent and administrative capacity.
At the state level, California’s implementation of Assembly Bill 302 illustrates a similar challenge of translating AI governance policies into accountability mechanisms. Although the law required the state to inventory high-risk automated decision systems used by agencies, California’s first public use-case inventory incorrectly stated that no such systems were in use, despite numerous publicly documented examples. This failure stemmed from weak implementation practices, including an informal reporting process that relied largely on agency self-reporting through email surveys.
New York’s Local Law 144, which codified bias auditing requirements for automated decision-making systems, has also faced its own constraints. For example, an audit of the law found that implementation challenges were compounded by limited mechanisms for leveraging expertise through interdepartmental collaboration. In addition, limitations in data quality and availability significantly constrain the ability to evaluate explicit bias or disparate impact. In New York, this challenge was particularly evident during implementation, as agencies struggled with both limited test data and the widespread absence of key demographic information needed to assess bias. In many cases, employers had not collected demographic data on applicants at all, and where such data did exist, it was often incomplete, leaving more applicants without demographic information than with it. This made it difficult to evaluate potential bias in automated decision-making systems used in employment decisions.
These barriers to implementation are a key motivation for the SOURCE CODE: AI Trust and Fairness Policy Sprint. In each of our memos, we take into account the resources, the stakeholders, and the capacity of each institution in bringing a policy idea to fruition. In developing our policy agenda, we have worked with experts across civil society and academia to identify solutions that are responsive to public concerns while remaining attentive to the realities of policy implementation. Across the policy memos, we outline actionable proposals that address several core policy levers that can help move AI governance from principle to practice.
Policy Levers to Advance AI Fairness and Build Public Trust
In undertaking this sprint, our focus is on how policy proposals can be implemented in light of current institutional and political realities. We recognize that broader debates over the values and governance of AI, including recent federal actions, have created uncertainty around the durability and implementation of comprehensive AI governance frameworks. Rather than concentrating on a single theory of AI governance, our sprint examines ten targeted ideas that can influence outcomes across different jurisdictions to provide trustworthy and fair outcomes for AI use. Our ideas can be categorized across four policy levers: government use of AI, public engagement, sector-specific interventions, and remedies.
Guardrails in Government Use of AI
Government use of AI represents one of the most immediate and consequential opportunities to shape how these systems function in practice. When public agencies adopt AI, they are not merely deploying tools but also setting precedents that can influence how systems are designed and governed more broadly. Public procurement, in this context, emerges as a critical but underexamined lever. The terms governments set when acquiring AI systems shape what vendors disclose, how systems are assessed, and what safeguards are built in from the outset. Several of the memos look at how AI is acquired across many sectors of public service, such as education, law enforcement, and healthcare, specific risk mitigation tools, and what final procurement agreements between governments and vendors should consider. Here are the policy ideas that specifically look at guardrails for government use of AI:
- How State Governments Should Purchase AI to Ensure Fair, Transparent, and Accountable Use by Jae Yeon Kim and Aniket Kesari
- How State Leaders Can Put People First in AI Decision-Making by Nicole Ozer and Brady Hirsch
- Prioritize Student Safety in K-12 Education By Establishing AI Procurement Guardrails by J.B. Branch
- The Federal Government Should Pilot a Decision Subject Representative Program for AI Systems Inspired by the FDA by Anna Lenhart
- How to Safely Bring AI into Law Enforcement AI-Generated Police Reports by Jon M. Peha
Public Engagement
Public engagement is treated as an afterthought in policymaking, in part because it is difficult to execute well. Policymakers and affected communities operate in different technical and cultural languages, making sustained dialogue challenging. Yet, meaningful public engagement in AI deployment can be a critical step to ensuring that AI use is both appropriate and trustworthy. Systems developed and deployed without concrete input from affected communities risk entrenching harm and undermining trust. Therefore, we have looked at practical ways that public engagement can be institutionalized at both the federal, state and local levels, as well as deployed to bring in underrepresented communities, such as rural populations, into the policy-making context. Here are policy ideas that move through what public engagement should like in practice:
- The Federal Government Should Pilot a Decision Subject Representative Program for AI Systems Inspired by the FDA by Anna Lenhart
- Empowering Communities through Community Benefit Agreements in AI-Fueled Data Center Development by Liza Paudel
- FairCare Verification Offers a Human-Centered Path for AI in Medicaid by Y. Tony Yang
- How State Leaders Can Put People First in AI Decision-Making by Nicole Ozer and Brady Hirsch
- Making Rural Communities Visible in Artificial Intelligence Through Rural Proofing in Kansas and Beyond by Ziwei Qi, Tatiana Lin, and Ayokunle Olagoke
Sector-Specific Interventions
AI does not operate in a vacuum, and neither should its governance. Each sector presents distinct risks, regulatory landscapes, and implementation challenges that must be accounted for in policy design. In K-12 education, procurement processes must consider privacy measures for underage individuals, surveillance, and the outsourcing of pedagogical judgment. Systems used to draft police reports risk introducing unverified or fabricated information into official records, underscoring the need for defined standards on use, human oversight, and disclosure. In labor markets, AI systems are reshaping wages, working conditions, job protections, and income stability. Healthcare has long been a contested space for automated decision-making systems because of its direct impact on access to care and quality of life. Existing sectoral protections will need to address algorithmic management, including requirements for transparency, notification, and avenues for contesting automated decisions. Here are policy proposals , spanning healthcare, education, labor and law enforcement:
- FairCare Verification Offers a Human-Centered Path for AI in Medicaid by Y. Tony Yang
- Move Algorithmic-Driven Pay and Scheduling Systems From Surveillance Pay to Fair Wages by Wilneida Negrón
- How to Safely Bring AI into Law Enforcement Through AI-Generated Police Reports by Jon M. Peha
- Making Rural Communities Visible in Artificial Intelligence Through Rural Proofing in Kansas and Beyond by Ziwei Qi, Tatiana Lin, and Ayokunle Olagoke
- Prioritize Student Safety in K-12 Education By Establishing AI Procurement Guardrails by J.B. Branch
Redress and Remedies
Although much of AI governance focuses on preventing harm, no system of safeguards will be perfect. This raises a critical question: what does redress look like when harms do occur? Existing approaches to recourse often fall short when applied in the real world, and incentives do not always align with accountability, and agencies or vendors may face legal, financial, or operational constraints that limit the availability or effectiveness of recourse mechanisms. These gaps point to the need to think more expansively about redress; we need to encode the individual right to contest decisions and insert it within a broader system of accountability. What institutional structures are needed to support meaningful recourse? And what forms of remedy, whether procedural, financial, or community-based, are appropriate? Here are two ideas that make harms structurally correctable:
- Settlement Wins Against Big Tech Should Underwrite Digital Resilience Funds by Gaurav Laroia and Charlotte Slaiman
- Empowering Communities through Community Benefit Agreements in AI-Fueled Data Center Development by Liza Paudel
From Ideas to Action
Our SOURCE CODE: AI Trust and Fairness Policy Sprint aims to advance the detailed policy solutions needed to foster public trust and implement fairness in the adoption of AI across diverse domains, from healthcare and government benefits to rural access, education, and worker protections.
We hope readers will engage deeply with these proposals, help bring them into practice, and build on them, developing new ideas that push this work even further. These ten proposals are not comprehensive, nor do they capture the full landscape of challenges that AI governance must address, from market concentration and labor displacement to infrastructure impacts and frontier-model risks. Rather, they are intended as an actionable starting point, an effort to illustrate what a detailed, implementable policy can look like.
The next step is to test these ideas in practice, learn from their successes and shortcomings, and translate those lessons into stronger governance frameworks. We hope this work serves as a foundation for a growing coalition of policymakers, practitioners, and communities committed to building AI systems that are fair, trustworthy, and accountable.
These ideas aim to advance the detailed policy solutions needed to foster public trust and implement fairness in the adoption of AI across diverse domains, from healthcare and government benefits to rural access, education, and worker protections.
The evidence is clear: algorithmic pay-setting is established in app-based work, and payroll/timekeeping failures show how software can produce systemic wage harm at scale
While a few states have taken steps to implement decision-making mechanisms for certain AI systems, too many leaders are simply accepting narratives about AI’s purported public benefit at face value – jumping to the “how” of AI implementation before thoroughly vetting potential systems and deciding whether they are appropriate to use at all.
When properly structured — with specific numeric targets, secured financial obligations, independent monitoring, and meaningful enforcement — CBAs transform data center deals into durable community partnerships.