AI systems are rapidly becoming part of the machinery of public life, but they sit on shaky foundations. We have seen AI being deployed for cancer screening, assisting people with disabilities, and even to address complex environmental challenges. Yet as these systems are deployed in increasingly consequential settings, the promise of AI to expand opportunity and increase effectiveness and boost productivity has been accompanied by harms that are no longer hypothetical. These harms fall into several recurring categories: systems can misallocate resources, misrepresent groups, fail to function reliably, or be deployed for illegitimate purposes, even when the technology works as intended. 

For example, AI has affected who gets allocated critical resources. A widely used healthcare algorithm underestimated the needs of Black patients, limiting access to care. In finance, algorithmic decision-making has produced discriminatory outcomes in lending and underwriting.   

Another observed harm is that failures in AI rollout can affect how people and communities are represented. AI systems have been shown to reinforce harmful stereotypes or render certain groups invisible altogether, particularly when they are trained on incomplete or biased data. A well known example is how facial recognition technologies are shown to perform significantly worse on darker-skinned individuals, raising concerns about misidentification and disproportionate surveillance. 

Still, other harms are failures of basic system functionality. Gunshot detection systems have generated large numbers of false alerts, manipulating how evidence is used in criminal proceedings. The Michigan Integrated Data Automated System (MiDAS), which was used to find instances of fraud in state unemployment benefits, was incorrect in 85% of its fraud determinations. 

Another category of concern arises not from failures in system design or performance, but from how AI systems are built and used in practice. These harms arise not from system failure but from how situations in which systems operate as intended yet still produce harmful outcomes. For example, algorithmic management tools in the workplace could intensify worker surveillance, destabilize scheduling, or reduce worker autonomy, even when operating accurately. 

These harms explain why AI is facing a crisis of public trust. A June 2025 Pew study found that half of U.S. adults feel “more concerned than excited about the growing use of AI”, with only a small minority expressing optimism. AI cannot deliver broad public benefits, such as improved public services, if the people affected by it do not trust the systems shaping their lives. The public will not trust abstract statements of fairness, transparency, responsibility, or legitimacy. It will be rebuilt only if those commitments are translated into procedural institutional mechanisms: procurement rules, public engagement processes, sector-specific safeguards, and holistic remedies. Such commitments will mean that AI can be used legitimately in the public interest. 

In many cases, the concern is whether the technology should even be used. This question raises deeper considerations around the concentration of power, human dignity, and the conditions under which innovation actually benefits the public. While AI can be used to support societal benefits, such as helping overworked healthcare practitioners, it can also be used in ways that harm human dignity, such as through surveillance or by restricting fair access to benefits or create new vulnerabilities like cybersecurity and data privacy risks. Building fairness and trust in AI requires more than improving system performance. Policymakers and the public must ask whether particular uses are legitimate, who benefits, who bears the risks, and what limits they should set and enforce.

Because these questions cannot be answered by abstract principles alone, the Federation of American Scientists worked with experts and practitioners across civil society and academia to bring together a policy agenda with ten actionable and high-impact solutions. We did this through our SOURCE CODE: AI Trust and Fairness Policy Sprint. When problems are urgent, institutions are uncertain, and traditional policymaking moves too slowly, our policy sprints create space to bring together experts across disciplines, from academics to technologists, advocates, and practitioners, and empower them to move quickly from diagnosis to action. Instead of debating what trust and fairness mean in the abstract, this sprint focused on what they look like in practice, and how it can be operationalized through specific policy levers.

This paper proceeds in three parts. First, we examine how fairness and trust are understood across different contexts, and why that creates friction in how we map the space. Second, we explore the challenges of implementing policy designed to install fairness guardrails. We highlight how gaps in policy, capacity, and real-world conditions can undermine even well-intentioned systems. Finally, we present a set of policy strategies across these key levers, offering actionable pathways for building fairer and more trustworthy AI systems.

What do we mean by ‘trust and fairness’ in AI?

Fairness and trust are often invoked as key components of AI governance and policy, but they can seem nebulous depending on the context and the community at hand. In general, we consider public trust to be the extent to which people see systems and institutions as reliable, accountable, and responsive to harms. AI fairness broadly concerns whether AI systems distribute benefits and burdens in ways that can be justified within a particular social, legal, and institutional context. Together, fairness and trust point to a broader question of legitimacy: whether an AI system should be used at all, for what purpose, and under what auspices. 

This section will reflect on how different stakeholders view fairness and public trust to examine each perspective before turning to existing legal instruments, policy gaps, and what is needed for effective policy implementation in this field. 

On fairness

Fairness in AI is not a single concept but is defined differently across technical, legal, and social domains. This plurality reflects how AI systems are “sociotechnical systems”: their effects depend not only on data and algorithms but also on the institutions, incentives, rules, and human decisions that shape their deployment. 

In examining how society in general views fairness, literature shows that individuals often understand fairness not as a clearly defined principle but in contrast to experiences of unfairness– that is, the absence of harm. This means that individuals, communities, and different cultures perceive fairness differently based on their own lived experiences. In terms of AI fairness, society views fairness as specifically related to terms such as “equity, consistency, non-discrimination, impartiality, justice, honesty, and reasonableness.” 

When evaluating fairness in AI systems, technical literature often distinguishes between two broad concepts. The first, individual fairness, asks whether similar individuals are treated comparably for a given task. This approach requires defining which characteristics are relevant to the task and what it means for two people to be meaningfully similar. The second, group fairness, examines whether outcomes are distributed unequally across groups. For example, in hiring, one group fairness approach might ask whether candidates from different demographic groups are selected at similar rates, while other approaches might focus on whether error rates or predictive accuracy differ across groups.

There are differences of opinion about what actually constitutes equal or similar outcomes in these definitions and how to predict them. On the one hand, historical data may be treated as a valid basis for predicting future outcomes. On the other hand, historical data itself could be shaped by historical and structural inequities, causing systems trained on it to reproduce existing patterns of discrimination. These competing considerations around historical data can create situations where an AI system is meeting one definition of fairness, but the actual outcome is creating unequal harms to a group or individual. The ProPublica investigation of the COMPAS risk assessment tool, an algorithm used to support criminal justice officials’ decisions on bail, sentencing, and early release, found that Black defendants were twice as likely to be labeled as high risk as white defendants. In theory, COMPAS satisfied one fairness criterion, in this case predictive parity, which means that risk scores are equally accurate across groups, but it also propagated the systemic inequalities of the U.S. criminal justice system. 

Fairness cannot be understood as a fixed technical standard, but rather as a contested concept shaped by social, institutional, and legal contexts. Determining which definition of fairness should govern a particular AI system is a nuanced decision, one that requires a deep understanding of the sectoral context, stakeholders, and other elements within the scope of the AI system’s deployment. This becomes even more challenging when relying on existing legal frameworks that may only partially address the complexities of AI-driven decision-making.

On public trust

What does it mean for technology itself to be worthy of trust and, in turn, of adoption? Public trust in AI is not generated by technical performance alone. It is built when people can see that an AI system serves a legitimate purpose, works reliably in its deployment context, and remains subject to meaningful human oversight, public accountability, and remedies when things go wrong. We also explicitly see fairness as a component of public trust, and that the public will not trust AI if they view that its allocations of opportunities and burdens are unfair. Ultimately, public trust is dependent on the public seeing the use of AI as justified and legitimate. If it falters, then AI use will be seen as negative to society. For example, the prospect of AI-driven worker substitution is a major source of public concern, raising questions about whether the use of AI to replace human labor is legitimate. If such a substitution occurs at scale, it could further erode public trust in AI systems and their deployment.

Public sector adoption of AI is where governance approaches are first tested in practice, shaping both regulatory norms and broader public expectations. When government agencies deploy AI systems, they are effectively signaling what responsible use looks like. As a result, failures in public sector systems can have outsized consequences. When government use of AI leads to unfair outcomes, unreliable decisions, or a lack of accountability, it can erode trust not only in AI but in the government itself. For example, the use of automated prior authorization systems in Medicare Advantage has been associated with higher denial rates and barriers to medically necessary post-acute care, something that could directly affect public attitudes towards AI adoption in government services. 

Across the federal government, different administrations have relied on the discourse that public trust in AI systems is essential to ensuring that the technology is disseminated across the public sphere. For example, Executive Order 13859 under the first Trump administration explicitly called for the use of AI in a manner that “fosters public trust and confidence,” a sentiment that carried over into Biden-era executive actions and remains in the current Trump administration OMB guidance, which defines how the federal government uses and acquires AI. These efforts have generally focused on identifying broad, high-risk uses of AI in the federal government and then pairing them with risk-mitigation requirements, leaving federal agencies to define implementation details and build the internal capacity to identify and enforce protections against AI uses that could erode public trust. 

How can existing law be used as a tool for trust and fairness?

Existing anti-discrimination law provides an important, but incomplete, set of tools for addressing AI-related harms. In employment, Title VII of the Civil Rights Act of 1964 prohibits discrimination based on race, color, religion, sex, and national origin. Later case law and statutory amendments, including the Civil Rights Act of 1991, developed a framework that distinguishes between disparate treatment, where a person is intentionally treated differently because of a protected characteristic, and disparate impact, where a facially neutral practice causes unjustified adverse effects for a protected group. This framework is highly relevant to AI systems, which may produce unequal outcomes even when they do not explicitly use protected characteristics.

Existing statutory tools are also directly applicable to AI-related consumer harms that occur in specific sectors. The Equal Credit Opportunity Act (ECOA), for instance, remains a powerful tool for addressing bias and discrimination in financial services, especially in cases where discrimination arises from algorithmic decision-making outputs playing a determinative role in financial outcomes such as loan decisions. Recent enforcement actions clearly demonstrate how these existing laws can be applied in practice to algorithmic harms to build the much-needed public trust in AI systems. In 2025, the Massachusetts Attorney General’s Office fined a financial services company that used AI for student loan underwriting after determining that its algorithmic outputs were discriminatory. As part of the legal remedy, the company was required to inventory its models and retrain them to comply with anti-discrimination, consumer protection, and fair lending laws. Similarly, the Federal Trade Commission has already used its authority to address harmful AI deployments. Its settlement with Rite Aid over the use of facial recognition technology included an unfairness claim, underscoring that discriminatory or harmful AI practices can fall squarely within existing prohibitions on “unfair” conduct. These examples illustrate that while regulators are able to act, such interventions are reactive, occurring after harms have already materialized, and may not provide detailed guidance for how systems should be designed or governed in advance.

Yet, without statutory reform, the contemporary reliance on existing law and legal frameworks is out of necessity rather than choice. This is in part because legal frameworks typically do not define fairness in the abstract. Instead, the law operationalizes it through established doctrines, standards, and enforcement mechanisms. Many emerging AI concerns can be partially mapped onto well-established legal principles with effective recourse and remedies for consumers. This continuity becomes especially apparent in consumer-facing contexts.

For example, trust and fairness in consumer-facing technologies such as AI closely align with longstanding notions of consumer product safety. As such, public concerns about whether AI systems are reliable, transparent, and non-harmful mirror traditional expectations that the physical products consumers purchase should not pose undue risks or potential harms. In addition to notions of what constitutes product safety, the American legal system has long articulated what constitutes “unfair” conduct through statutes such as the Federal Trade Commission Act and the Dodd-Frank Act’s prohibition on unfair, deceptive, or abusive acts or practices (UDAAP). 

In the absence of clear enforcement and interpretive guidance, however, legal gaps can translate into diffuse or ambiguous accountability, undermining public confidence in both the technologies themselves and the institutions responsible for overseeing them. One such example of this dynamic leading to widespread unease and declining public trust in technology is the bipartisan frustration over digital payment apps. Apps that use AI tools for automated content moderation or fraud detection can unknowingly “debank” and terminate the accounts of otherwise welcome clients. 

Policymakers therefore face a difficult bind: without more deliberate efforts to operationalize fairness and accountability, public trust will remain elusive. Legal standards alone are not enough; they must be translated into systems that people experience as fair, reliable, and responsive to harm. 

In the absence of new laws that directly address the challenges posed by artificial intelligence, practitioners must rely on existing legal frameworks that offer only partial mechanisms for accountability and redress when harm arises. This creates particular challenges in areas where concepts such as “trust” and “fairness” in AI systems are not explicitly codified, and where legal precedent is still limited or emerging. Existing legal frameworks may provide an important starting point for building public trust, but they do not fully capture the range of concerns raised by AI systems or the conditions needed to sustain public trust over time. 

The challenge of implementation: why good intentions fail to produce trusted outcomes 

In a rapidly evolving policy climate, governments across jurisdictions have implemented or proposed to create fairer, more trustworthy algorithmic systems and to protect people from harms. When designing policy interventions, it is important to take into account how prepared institutions are in taking on these responsibilities, and what resources, such as talent, processes, and even data availability, exist to ensure that a new policy has a fighting chance to succeed. 

Take, for example, federal agencies’ implementation of OMB Memorandum M-24-10, a government-wide guidance document on how the federal government uses and acquires AI. Agencies are supposed to publicly publish a compliance plan describing the processes they will undertake to follow said guidance. In our analysis of these compliance plans, we found that agencies vary in technical expertise, staffing capacity, and institutional resources, which leads to inconsistent compliance and fragmented oversight practices. In this case, governance frameworks alone are insufficient; effective implementation depends on sustained investment in technical talent and administrative capacity. 

At the state level, California’s implementation of Assembly Bill 302 illustrates a similar challenge of translating AI governance policies into accountability mechanisms. Although the law required the state to inventory high-risk automated decision systems used by agencies, California’s first public use-case inventory incorrectly stated that no such systems were in use, despite numerous publicly documented examples. This failure stemmed from weak implementation practices, including an informal reporting process that relied largely on agency self-reporting through email surveys.

New York’s Local Law 144, which codified bias auditing requirements for automated decision-making systems, has also faced its own constraints. For example, an audit of the law found that implementation challenges were compounded by limited mechanisms for leveraging expertise through interdepartmental collaboration. In addition, limitations in data quality and availability significantly constrain the ability to evaluate explicit bias or disparate impact. In New York, this challenge was particularly evident during implementation, as agencies struggled with both limited test data and the widespread absence of key demographic information needed to assess bias. In many cases, employers had not collected demographic data on applicants at all, and where such data did exist, it was often incomplete, leaving more applicants without demographic information than with it. This made it difficult to evaluate potential bias in automated decision-making systems used in employment decisions.

These barriers to implementation are a key motivation for the SOURCE CODE: AI Trust and Fairness Policy Sprint. In each of our memos, we take into account the resources, the stakeholders, and the capacity of each institution in bringing a policy idea to fruition. In developing our policy agenda, we have worked with experts across civil society and academia to identify solutions that are responsive to public concerns while remaining attentive to the realities of policy implementation. Across the policy memos, we outline actionable proposals that address several core policy levers that can help move AI governance from principle to practice.

Policy Levers to Advance AI Fairness and Build Public Trust

In undertaking this sprint, our focus is on how policy proposals can be implemented in light of current institutional and political realities. We recognize that broader debates over the values and governance of AI, including recent federal actions, have created uncertainty around the durability and implementation of comprehensive AI governance frameworks. Rather than concentrating on a single theory of AI governance, our sprint examines ten targeted ideas that can influence outcomes across different jurisdictions to provide trustworthy and fair outcomes for AI use. Our ideas can be categorized across four policy levers: government use of AI, public engagement, sector-specific interventions, and remedies. 

Guardrails in Government Use of AI

Government use of AI represents one of the most immediate and consequential opportunities to shape how these systems function in practice. When public agencies adopt AI, they are not merely deploying tools but also setting precedents that can influence how systems are designed and governed more broadly. Public procurement, in this context, emerges as a critical but underexamined lever. The terms governments set when acquiring AI systems shape what vendors disclose, how systems are assessed, and what safeguards are built in from the outset. Several of the memos look at how AI is acquired across many sectors of public service, such as education, law enforcement, and healthcare, specific risk mitigation tools, and what final procurement agreements between governments and vendors should consider.  Here are the policy ideas that specifically look at guardrails for government use of AI:

• How State Governments Should Purchase AI to Ensure Fair, Transparent, and Accountable Use by Jae Yeon Kim and Aniket Kesari 
• How State Leaders Can Put People First in AI Decision-Making by Nicole Ozer and Brady Hirsch
• Prioritize Student Safety in K-12 Education By Establishing AI Procurement Guardrails by J.B. Branch
• The Federal Government Should Pilot a Decision Subject Representative Program for AI Systems Inspired by the FDA by Anna Lenhart
• How to Safely Bring AI into Law Enforcement AI-Generated Police Reports by Jon M. Peha

Public Engagement 

Public engagement is treated as an afterthought in policymaking, in part because it is difficult to execute well. Policymakers and affected communities operate in different technical and cultural languages, making sustained dialogue challenging. Yet, meaningful public engagement in AI deployment can be a critical step to ensuring that AI use is both appropriate and trustworthy. Systems developed and deployed without concrete input from affected communities risk entrenching harm and undermining trust. Therefore, we have looked at practical ways that public engagement can be institutionalized at both the federal, state and local levels, as well as deployed to bring in underrepresented communities, such as rural populations, into the policy-making context.  Here are policy  ideas that move through what public engagement should like in practice:

• The Federal Government Should Pilot a Decision Subject Representative Program for AI Systems Inspired by the FDA by Anna Lenhart
• Empowering Communities through Community Benefit Agreements in AI-Fueled Data Center Development by Liza Paudel
• FairCare Verification Offers a Human-Centered Path for AI in Medicaid by Y. Tony Yang
• How State Leaders Can Put People First in AI Decision-Making by Nicole Ozer and Brady Hirsch
• Making Rural Communities Visible in Artificial Intelligence Through Rural Proofing in Kansas and Beyond by Ziwei Qi, Tatiana Lin, and Ayokunle Olagoke

Sector-Specific Interventions

AI does not operate in a vacuum, and neither should its governance. Each sector presents distinct risks, regulatory landscapes, and implementation challenges that must be accounted for in policy design. In K-12 education, procurement processes must consider privacy measures for underage individuals, surveillance, and the outsourcing of pedagogical judgment. Systems used to draft police reports risk introducing unverified or fabricated information into official records, underscoring the need for defined standards on use, human oversight, and disclosure. In labor markets, AI systems are reshaping wages, working conditions, job protections, and income stability. Healthcare has long been a contested space for automated decision-making systems because of its direct impact on access to care and quality of life. Existing sectoral protections will need to address algorithmic management, including requirements for transparency, notification, and avenues for contesting automated decisions. Here are policy proposals , spanning  healthcare, education, labor and law enforcement:

• FairCare Verification Offers a Human-Centered Path for AI in Medicaid by Y. Tony Yang
• Move Algorithmic-Driven Pay and Scheduling Systems From Surveillance Pay to Fair Wages by Wilneida Negrón
• How to Safely Bring AI into Law Enforcement Through AI-Generated Police Reports by Jon M. Peha
• Making Rural Communities Visible in Artificial Intelligence Through Rural Proofing in Kansas and Beyond by Ziwei Qi, Tatiana Lin, and Ayokunle Olagoke
• Prioritize Student Safety in K-12 Education By Establishing AI Procurement Guardrails by J.B. Branch

Redress and Remedies

Although much of AI governance focuses on preventing harm, no system of safeguards will be perfect. This raises a critical question: what does redress look like when harms do occur?  Existing approaches to recourse often fall short when applied in the real world, and incentives do not always align with accountability, and agencies or vendors may face legal, financial, or operational constraints that limit the availability or effectiveness of recourse mechanisms. These gaps point to the need to think more expansively about redress; we need to encode the individual right to contest decisions and insert it within a broader system of accountability. What institutional structures are needed to support meaningful recourse? And what forms of remedy, whether procedural, financial, or community-based, are appropriate? Here are two ideas that make harms structurally correctable:

• Settlement Wins Against Big Tech Should Underwrite Digital Resilience Funds by Gaurav Laroia and Charlotte Slaiman
• Empowering Communities through Community Benefit Agreements in AI-Fueled Data Center Development by Liza Paudel

From Ideas to Action

Our SOURCE CODE: AI Trust and Fairness Policy Sprint aims to advance the detailed policy solutions needed to foster public trust and implement fairness in the adoption of AI across diverse domains, from healthcare and government benefits to rural access, education, and worker protections.

We hope readers will engage deeply with these proposals, help bring them into practice, and build on them, developing new ideas that push this work even further. These ten proposals are not comprehensive, nor do they capture the full landscape of challenges that AI governance must address, from market concentration and labor displacement to infrastructure impacts and frontier-model risks. Rather, they are intended as an actionable starting point, an effort to illustrate what a detailed, implementable policy can look like. 

The next step is to test these ideas in practice, learn from their successes and shortcomings, and translate those lessons into stronger governance frameworks. We hope this work serves as a foundation for a growing coalition of policymakers, practitioners, and communities committed to building AI systems that are fair, trustworthy, and accountable.

Employers increasingly rely on scheduling, timekeeping, and payroll software to determine hours, eligibility, and pay. When monitoring data and optimization rules feed these systems, or what this memo refers to as “algorithmic wage-setting”—it rarely appears as a standalone tool. It shows up as configured rules and thresholds, time edits, automatic deductions, and eligibility flags that can quietly change compensable time and earnings. A 2025 Equitable Growth brief describes this dynamic as “surveillance pay”—the use of granular monitoring data integrated into pay systems to set compensation and calculate wages in ways that can disconnect time from pay and make outcomes harder to predict, audit, and challenge for discrimination.

States are already moving to regulate surveillance/algorithmic wage-setting, but proposals focus on prohibition and basic notice rights. This memo complements those efforts by centering the enforcement reality: payroll and timekeeping are the system of record and the regulatory choke point. It pairs guardrails on non-job-related data use with an enforcement operating model, audit-ready decision trails, integration/egress mapping, standardized audits and complaints triage, and minimum operational standards, so agencies can prove violations, correct errors quickly, and prevent repeat harm using preexisting wage-and-hour, civil rights, consumer protection, and procurement authority.

Challenge and Opportunity

Core labor protections like minimum wage, overtime, predictable scheduling, and anti-discrimination regulations, increasingly run through proprietary workplace systems that employers and vendors configure, but workers and regulators often cannot see or challenge. As these tools spread across white- and blue-collar industries—including healthcare, retail, logistics, food service, manufacturing, construction, and public services, they can normalize hidden wage loss, income volatility, and unequal treatment, especially when employers use surveillance-derived metrics to change pay tiers, incentives, benefits eligibility, or hours without clear notice or a workable way to challenge errors.

Why payroll and timekeeping are the focus.

In most workplaces, pay and schedules do not come from a single “algorithmic wage tool.” Instead, they come from connected systems that track hours, assign shifts, and apply workplace rules, that then feed into HR and payroll systems, which serve as the official record for compensation.

This memo focuses on payroll and timekeeping/scheduling because that’s where data turns into earnings: wages, hours paid, premiums, bonuses, and benefits eligibility. It is also where states can most realistically require auditable records, set clear limits on what data can influence pay decisions, and enforce worker rights.

Worker data typically flows through a simple data chain:

1. Capture: timekeeping, scheduling, attendance, and productivity/monitoring tools record events (clock-ins, breaks, shift changes, performance flags). 
2. Integrate: HR and payroll systems (and their vendors/subcontractors) pull those inputs together and link them to pay rules. 
3. Decide: configured rules, thresholds, or models trigger pay-affecting actions—time edits, automatic deductions, eligibility flags for premiums/bonuses, schedule adjustments, and pay calculations. 
4. Pay out: the results appear in payroll as wages, hours paid, premiums, bonuses, and take-home pay.

Because payroll and timekeeping are the official record, regulators cannot rely on the paycheck alone. Instead, regulators need to see and audit the system’s decision trail which includes the data sources that were used, the rule or thresholds that were applied, what changed (e.g., edits, deductions, eligibility flags), and who made (or approved) any changes.

Added risk pathway: third-party intermediaries.

Worker data does not always stay inside a single employer system. In instances, third parties such as verification services, analytics intermediaries, and sometimes data brokers/resellers collect and commercialize worker-related data and feed it back into workplace tools in the form of aggregated scores, flags, or “risk/reliability” signals that can affect scheduling, wages, and/or compensation.

Without clear limits and disclosure on this type of third-party data sourcing and onward sharing for pay and time keeping-affecting decisions (including brokered data and broker-derived scores), non-job-related data can also shape pay and scheduling indirectly while obscuring provenance (who supplied it), purpose (why it was used), and accountability (who is responsible) .

From a regulatory standpoint, the risks typically concentrate in four areas:

1. Implementation/configuration failures, rollouts, integrations, default settings, or rule changes that trigger underpayment or missing premiums. 
2. Improper inputs/uses in pay or time keeping decisions, such that non-job-related personal data (including surveillance-derived metrics and brokered inferences) used to set or modify wages, hours, eligibility, or incentives. 
3. Secondary use and onward sharing (data governance risk), in that worker pay/HR data repurposed, shared, or sold beyond payroll/service delivery, potentially re-entering decision systems as scores, flags, or eligibility signals. 
4. Black-box accountability gaps, in which systems that prevent workers, unions, and regulators from seeing which inputs and rules produced pay outcomes.

Understanding these is key from a regulatory standpoint because the question then becomes not only what the paycheck says, but what rules and input influenced any changes in wage or compensation calculations and whether those inputs are legitimate and traceable. 

The recommendations that follow do three things: (1) cut off high-risk data inputs, (2) require audit-ready decision trails, and (3) give workers enforceable rights to notice, explanation, and correction.

Why states should act now.

We already have evidence that algorithmic pay is common in some sectors in the labor markets, and that payroll “modernization” rollouts can cause widespread pay errors when software becomes the system of record. Even if “surveillance wages” is not yet widespread beyond the gig economy, which is the point: states can act upstream, before these tools harden into default infrastructure. At the same time and in parallel, states are also introducing surveillance-pricing prohibition signaling growing legislative appetite to regulate data-driven personalization and discrimination before it becomes default infrastructure.

Below are examples of the ways this trend is taking shape:

• Algorithmic pay in app-based work. The best-known example of algorithmic wage-setting is in the context of ride-hail and delivery platforms, where algorithms determine pay and can be difficult for workers to predict or contest, a classic “black box” accountability problem that vendors now export into traditional workplaces. Bloomberg Law notes this “baseline model” is now being exported to other industries through vendors marketing automated pay tools.

• Payroll systems failing at scale: Workday rollouts. After Seattle rolled out Workday, a third-party algorithmic payment system, workers filed a wage theft class action alleging underpayments and payroll problems. This is an example of how implementation/configuration failures at scale can take place when software becomes the system of recording. In another example, Oregon state workers also reached a $15 million class action settlement over wage errors tied to the implementation of Workday Payroll. Both cases show why “software errors” can function as de facto wage theft when employers run pay through complex proprietary systems that workers cannot debug and employers struggle to correct. 

• Timekeeping rules driving wage loss: time edits, rounding, and auto meal deductions. In a federal class action against Yale New Haven Health, workers alleged the employer rounded and edited time records and automatically deducted meal breaks even when breaks were missed, interrupted, or not fully taken, showing how pay harm can come from timekeeping configuration choices (and the lack of a transparent, contestable audit trail), not an “even when no one labels it “algorithmic wage-setting”.

• Timekeeping systems failing at scale: the Kronos/UKG outage. The 2021 ransomware attack on Kronos/UKG’s timekeeping platform left employers without their normal system of record for weeks, triggering wage-and-hour claims that employees went unpaid or underpaid when employers reconstructed hours and overtime. For example, Cargill reached a $2.4M settlement tied to unpaid wages/overtime allegations stemming from the outage, and other employers (e.g., Frito-Lay; Honda) have faced similar outage-related wage claims and settlements, illustrating how dependence on a single timekeeping/payroll platform can create systemic pay risk when the software fails.

These examples show how payroll and timekeeping systems are often the choke point because they encode pay rules, execute pay-affecting actions (like time edits and eligibility flags), and generate, or withhold, the audit trail regulators need to verify compliance. 

Harms this proposal targets (and what we know about scope)

This memo targets a specific set of harms that arise when employers route compensation decisions through timekeeping, scheduling, and payroll systems (often with third-party inputs). 

These harms fall into five buckets:

1. Hidden wage loss and underpayment.
   Examples include time edits and reclassifications, automatic deductions (e.g., meal breaks), missing premiums/differentials, or misapplied overtime triggers that reduce pay without a clear explanation or easy correction path.
   What we know: wage-and-hour complaints and litigation regularly surface these mechanisms, especially when payroll/timekeeping becomes the system of record.
   
2. Income volatility and scheduling instability.
   Automated scheduling and rule-based eligibility can drive unpredictable hours, unstable earnings, and difficulty budgeting, even more so, when rules change inside proprietary systems.
   What we know: volatility is well-documented in app and gig-based labor markets and is a growing concern as similar logic moves into traditional workplaces.
   
3. Discrimination and disparate impact at scale.
   Surveillance-derived metrics, proxy variables, and eligibility flags can embed unequal treatment in pay, hours allocation, or access to premiums/bonuses, especially when workers cannot see or contest the underlying rule or data input.
   What we know: civil rights risk is structural when decisioning relies on opaque metrics and limited contestability; disability advocates flag heightened vulnerability due to higher fixed costs and budgeting constraints.
   
4. Accountability failures (“black box” enforcement gaps).
   When the system’s decision trail is unavailable, employers can’t explain pay outcomes, workers can’t self-advocate, and agencies can’t prove violations, turning basic labor protections into an after-the-fact guessing game.
   What we know: this is a recurring barrier in investigations and disputes involving payroll/timekeeping platforms and integrated tools.
   
5. Data governance harms (secondary use and third-party re-entry).
   Worker pay/HR data may be repurposed, shared onward, or reintroduced via third-party scores/flags (e.g., verification, analytics intermediaries, brokers), shaping pay and scheduling indirectly while obscuring provenance and accountability.
   What we know: third-party ecosystems exist and can influence eligibility/access decisions; the risk increases when data egress and sourcing aren’t disclosed.
   

Given these harms, this memo seeks to reduce wage loss, volatility, and discrimination by (1) limiting high-risk inputs and secondary use, (2) requiring audit-ready decision trails and integration/egress visibility, and (3) giving workers practical rights to notice, explanation, and correction.

Plan of Action

Recommendation 1. Establish a clear guardrail on compensation data use.

Adopt legislation to create the bright-line ban, scope, and remedies, then reinforce it through existing wage/civil rights/UDAP enforcement and procurement requirements for public employers and contractors.

States should adopt a bright-line rule that bars employers and vendors from using non-job-related personal data, including brokered data and broker-derived scores or classifications—to set or change wages, hours, bonuses, differentials, benefits, or pay eligibility. “Non-job-related personal data” means any data or inference not reasonably necessary and proportionate to determine hours worked, pay owed, or job-related compensation factors, which are limited to seniority, job classification, documented skills/credentials, objective shift attributes (e.g., nights/weekends/hazard pay), location-based cost adjustments, and transparent performance metrics tied to job duties (not biometrics, health inferences, parenthood status, home address, or off-duty behavior). This targets the core risk: opaque, individualized wage manipulation.

To prevent loopholes and misclassification incentives, the guardrail should:

• Cover workers broadly. Apply to employees and to workers treated as independent contractors when an algorithm or platform determines compensation, since contractors are often most vulnerable to misclassification and therefore irregular pay, and exposed to variable, algorithm-set pay.
• Define prohibited practices clearly. Treat “surveillance wage-setting” as the use of surveillance-derived data or inferences to determine compensation at an individualized level.
• Create real accountability. Pair agency enforcement (e.g., labor and state Attorney Generals) with meaningful penalties and a private right of action. For this, states can look to bills such as Colorado’s HB26-1210, New York’s S8872 (and Assembly companion A9641), or Maryland’s HB0148 as models.
• Center disability and civil rights protection. Require accessibility, nondiscrimination testing, and meaningful appeal/human review for any permitted automated compensation practices.
• Preserve legitimate pay practices. Allow transparent, non-personalized wage premiums (e.g., seniority steps, COLA, hazard pay, shift differentials) and job-related market adjustments that do not rely on individualized surveillance.
• Limit secondary use and commercialization of compensation data. Prohibit employers and vendors from selling, licensing, or otherwise disclosing worker compensation data (and pay-derived eligibility flags) to third parties for purposes unrelated to payroll and scheduling/service delivery, and prohibit use of compensation data to train generalized AI models unless the data is truly deidentified and the use is strictly necessary for the contracted service.

Recommendation 2. Make enforcement practical: require audit-ready records for algorithmic pay and scheduling systems.

Use rulemaking/guidance and enforcement to require decision-trail records and standardized audits, reinforced through procurement requirements for public employers and contractors, and use targeted legislation only if agencies lack clear authority to compel retention/production or to cover vendors directly.

This recommendation targets two recurring failure modes: (1) rollout/configuration errors (especially during integrations) and (2) black-box systems that prevent regulators from showing what the software did and why. Guardrails only work if agencies can access the decision trail behind pay outcomes. 

Agencies already use payroll records/paystubs, time and attendance data, schedules, job classifications and rate tables, and worker complaints. But those records often show only the outcome, not the mechanism; they rarely reveal which rules, inputs, or system changes produced a pay result. To enforce wage and civil rights protections when software mediates pay and scheduling, agencies must also require retention and production of:

• Automated decision records (audit trails): logs of pay-affecting actions (time edits, auto-deductions, pay recalculations, eligibility changes for premiums/bonuses), including timestamps and the rule or data source that triggered each change.
• Rule/configuration history: the pay rules in effect over time, plus change history and approvals.
• Integration and egress map: (a) upstream systems feeding decisioning (timekeeping, scheduling, attendance, productivity, location/ratings); (b) any third-party sources supplying inputs (verification databases and any data brokers/resellers), including which fields/scores they provide and which pay/eligibility rules use them; and (c) whether worker pay/HR data is shared onward or sold, and for what purposes, including any analytics, benchmarking, or model-training uses unrelated to payroll/service delivery.

These missing records are not “nice to have;” they are the minimum evidence needed to audit pay outcomes when software is the system of record. To close this enforcement gap, states should do two things at once: (1) require retention and production of decision-trail records, and (2) standardize how agencies request, analyze, and enforce them. 

Actions states can take now include:

1. Modernize payroll recordkeeping. Require employers (and covered vendors where appropriate) to retain and produce audit trails, rule/configuration history, and integration/egress maps as standard payroll records.
2. Standardize an audit protocol (Labor and State Attorney Generals). Use a shared checklist and data request template to compare system outputs to hours worked/pay owed and identify repeat patterns (missing premiums, unexplained deductions, volatility, disparate impact). A small interagency working group should maintain templates, secure intake, and a vendor/system map.
   • Rapid supply-chain mapping: for each investigation, map (1) payroll/HRIS, timekeeping, scheduling, and monitoring systems; (2) each vendor/subcontractor processing worker data; (3) third-party sources supplying scores/flags; (4) which fields feed which pay/eligibility rules; and (5) any onward sharing/sale of worker data.
   • Audit templates should include both case-level review (individual decision trails) and pattern tests (aggregate metrics that reveal systematic underpayment, volatility, or disparities after rollouts or rule changes).
3. Use procurement as leverage. For public employers and contractors, require auditability, data retention, worker notice, and cooperation with investigations as contract conditions. Contracts should also prohibit undisclosed sale/sharing of workforce and pay data and prohibit using worker pay/HR data for analytics, benchmarking, or model training unrelated to the contracted service, with audit rights and penalties for noncompliance.
4. Set minimum standards for pay-affecting vendor practices (rule-setting and procurement). States do not need to regulate every feature of payroll and scheduling software to reduce harm. A practical approach is to set a small set of baselines, enforcement-ready standards through State Attorney General labor enforcement guidance, settlement terms, and public procurement that target the most common ways software drives wage loss and blocks accountability.

To make this action (#4) more concrete, states can start with a brief list of “minimum operational standards” that directly targets the most common ways payroll and timekeeping systems reduce pay and block accountability.

Four minimum operational standards can pursue:

• No silent time edits or auto-deductions. Prohibit break deductions, time reclassifications (e.g., “idle time”), or post-hoc hour changes without clear worker notice. Require an easy, timely process to review and correct records. (Example: an automatic meal-break deduction should not reduce paid time when a worker did not take a break, took a shorter break, or had the break interrupted, unless the worker can easily confirm or correct the record.)  States can also require logs that show what changed, when, and why. 
• Document and notify pay rules and eligibility changes. Require documentation and worker notice when systems change eligibility for premiums, bonuses, differentials, or overtime triggers and when pay rules/configurations change. States can also seek to prohibit earnings-impacting changes without a traceable record and a responsible human owner. 
• Exportable audit logs (decision trails) as payroll records. Require tamper-resistant, exportable logs that capture pay-affecting actions, the rule that triggered them, and the data source used, and treat these logs as payroll records subject to retention and production requirements. 
• Burden-shifting when records are missing. If required records are missing, place the risk on the employer and not the worker—using a rebuttable presumption in favor of the worker’s reasonable account of hours/pay, plus escalating penalties for recordkeeping failures (civil fines, enhanced damages for systemic violations, monitoring/injunctive relief, and procurement consequences). 

When to act. Agencies should open an investigation when complaints jump right after a new system rollout, when time edits or auto-deductions show up unusually often, when workers can’t get a plain-English explanation or timely correction, or when it looks like third-party/non-work data is affecting pay, hours, or eligibility. To do this consistently, agencies should use a simple, standardized intake and escalation process that logs the employer, the vendor/system (when known), and the issue type and flags patterns that should be reviewed by a designated triage team.

Recommendation 3. Guarantee worker-facing transparency and contestability: a right to know, a right to an explanation, and a right to correct.

Use agency guidance/rules and procurement to require notice, explanations, and fast corrections where agencies already have authority; use legislation to create new worker rights (access, deadlines, anti-retaliation) where needed; and use enforcement to hold employers and vendors accountable when notices or records are missing, false, or misleading.

Enforcement alone often leaves workers waiting months for relief. States should therefore require worker-facing transparency for any automated system that sets pay or materially shapes earnings through time classification, scheduling, differentials, bonuses, or pay eligibility so workers can spot problems early, document patterns, and seek timely correction. Aggregated reporting can help identify systemic issues, but it does not replace a worker’s right to see and contest the records that determine their individual pay.

Privacy and data-broker rules (e.g., CCPA/CPRA-style disclosure and Delete Act-style broker mechanisms) provide useful templates for disclosure and access rights in the worker-pay context.

A worker rights package focused on this issue would include: 

• Right to notice. Provide clear, plain-language notice when automated systems affect pay, time classification, hours allocation, or eligibility for premiums/bonuses. Notices should name the system(s), the categories of data used, the outcomes affected, and whether the employer/vendor relies on third-party sources (including brokers/resellers, verification databases, or analytics intermediaries).
• Right to an explanation. When pay, eligibility, or schedules change due to an automated rule, provide a readable explanation showing what changed; the rule/threshold applied; the input data sources; the effective date; the responsible human owner; and the identity of any third-party source that supplied an input, score, or eligibility flag.
• Right to inspect core records. Provide access to pay-related artifacts needed for self-advocacy, including time entries and edits (including auto-deductions); premium/differential eligibility status; incentive/bonus formulas that apply; and a history of pay-affecting adjustments within the pay period.
• Aggregated reporting for pattern detection (privacy-protective). Require aggregated reporting (by job role/site/shift/pay period, with privacy thresholds) so workers and unions can detect systemic patterns (frequent time edits, missing premiums, pay volatility, disparities). Examples include: % of shifts with auto-deductions; time edits per one hundred workers per pay period; premium eligibility changes over time by job class/site; weekly hours volatility pre/post rollout; and disparity checks such as premium loss rates (no tiny groups).
• Right to timely correction and human review. Require fast, accessible processes to challenge errors or automated determinations affecting pay, with clear deadlines for correction/back pay and escalation paths when errors cause hardship.
• Collective bargaining compatibility. Structure these rights so unions can incorporate them into CBAs and side letters, including access to aggregated reports (time edits/auto-deductions, eligibility changes, error rates, volatility, disparate impact indicators) and bargaining triggers when vendors, integrations, or pay rules change.

Worker-facing transparency also strengthens enforcement: it creates documentation, reduces information asymmetry, and helps agencies identify employers and vendors that warrant priority investigation.

Conclusion

Fair and trustworthy workplace technology starts with something workers understand: a paycheck they can trust and a schedule they can plan around. The evidence is clear: algorithmic pay-setting is established in app-based work, and payroll/timekeeping failures show how software can produce systemic wage harm at scale. States can act now using existing labor, civil rights, consumer protection, and procurement authority—strengthened by a prohibition on surveillance wage-setting, enforcement-ready decision trails, and worker rights to notice, explanation, and correction, so “efficiency” doesn’t come at the expense of fairness, dignity, accessibility, or basic economic security.

How State Leaders Can Put People First in AI Decision-Making is a framework to ask and answer the right foundational questions about artificial intelligence (AI) from the beginning. The public wants the government to take action to ensure the power of AI technology is used for good. In the current political climate, the work of state leaders is critical. The recommendations in this memo are focused on helping state leaders across the country ground decision-making about AI use in fairness, accountability, evidence-based inquiry, and inclusive governance so that AI can work for people.

Many state agencies have already deployed or are considering using AI in consequential decisions related to healthcare, housing, education, policing, finance, and other highly sensitive areas. While a few states have taken steps to implement decision-making mechanisms for certain AI systems, too many leaders are simply accepting narratives about AI’s purported public benefit at face value – jumping to the “how” of AI implementation before thoroughly vetting potential systems and deciding whether they are appropriate to use at all.

State officials may be eager, and even feel pressure, to tap into the potential benefits of AI in the hopes of better serving their constituents. But the personal, political, and operational risks of AI use should not be underestimated. People across the political spectrum are deeply concerned about the impact of AI on their lives and these concerns are well-founded. There have already been numerous examples where the failure to center people in AI decision-making and use has resulted in government systems that range from inefficient and wasteful to disruptive and downright dangerous, causing significant harm to, and backlash from, community members.

For AI’s potential benefits to be realized, state leaders need to implement consistent, inclusive people-first AI decision-making structures. Crucially, this process should ask the foundational question of whether to use AI in the first place. This policy memo provides timely guidance on:

• Why policymakers should adopt a people-centered AI decision-making process; 
• What consistent process to follow as the foundation of any AI decision-making policy; 
• How to operationalize this process through a flexible set of options designed to meet specific needs, structures, and opportunities in different states.

Rather than offering a one-size-fits-all approach, this memo provides a suite of mechanisms for engaging thoughtful AI decision-making with examples of how different state governments have tackled emerging AI issues. We give recommendations for how state leaders can implement the AI decision-making process for whichever path they choose, including methods to promote accountability so that the decision-making process is followed and can truly work to put people first.

Challenge and Opportunity

The use of AI by state agencies is growing. By 2024, 59% of state and local government employees reported that their agency had already made an AI application available for use and a majority of public sector employees reported using AI applications either several times a week or daily.

Generative AI (GenAI) systems and agentic AI systems are now joining machine learning and automated decision-making systems (ADS) that have been in use for many years – with the lines between the types of systems blurring as AI products become increasingly integrated.

AI is also being applied in many high-stakes situations where mistakes or bias can have life-altering ramifications. AI systems now make decisions that can affect the lives of tens of millions of low-income people in the United States, from determination of SNAP benefits, to Medicaid enrollment, to Social Security disability payments. Sixty percent of people in the United States live in a jurisdiction that employs some sort of pretrial risk assessment tool that uses AI. According to one AI surveillance vendor, thousands of police departments in the United States are using face surveillance.

While many policymakers may be enticed by the promise of AI, people across the country and political spectrum have deep concerns. As of 2025, only 17% of the general public believes AI will positively impact the United States. Americans broadly oppose AI being used in high-stakes decision-making, like health insurance, loan applications, and job screening. A 2025 poll of U.S. voters found that 82% said they do not trust technology leaders to tackle regulation independently. A supermajority – 69% – of the U.S. public does not think the government is doing enough to regulate AI.

How does the public feel about AI?

More than 50% of people in the U.S, and 65% of low-income people, fear being left behind by AI. Only 4 in 10 people ages 18-34 in the U.S. say that they “trust” AI and only 23% of people in the U.S. over age 55 trust AI systems. As AI advances, public anxiety grows. Polling reveals that 77% of people in the U.S. want companies to “take AI creation slowly to get it right the first time.”

Public concerns with AI are well-founded. Former high-profile staffers at several AI companies have warned that companies are moving too fast and minimizing AI’s deficiencies, with new AI systems “generating more errors, not fewer.” While the technology industry is pushing the pedal on AI, the public would like to hit the brakes and for leaders to “do something before it goes too far.”

In the rush to adopt AI, some government officials have been making mistakes. The most impacted communities, including low income and communities of color, often end up excluded from public deliberation about government use of technology. There are already numerous examples of how these same communities bear the brunt when there is a lack of people-centered AI decision-making:

• The state of Michigan abandoned a $47-million automated unemployment fraud detection system after it was found to have a false positive rate of 93%. Michigan had to pay $20 million to settle a civil rights class action lawsuit and spend an additional $78 million to install a new system.
• Automated systems caused delays in obtaining critical unemployment benefits for people in several states, including in California where a fraud detection algorithm system improperly flagged 600,000 people as having potentially fraudulent characteristics.
• Issues of discrimination related to AI algorithm use has been documented across numerous jurisdictions, including a family screening system used in Pennsylvania that gave more racially disparate recommendations than child welfare workers.
• AI errors are also creating dangerous interactions between police and community members. Police have wrongfully arrested people, many Black community members, because of AI-powered face surveillance. Maryland police also swarmed out of eight squad cars and surrounded a 16-year old and held him at gunpoint after his school’s AI security system mistakenly identified his bag of potato chips as a gun.

There are high costs for improper AI use – for the people whose lives are impacted, in the state dollars that are invested, and in how these actions can further undermine trust in government.

At their best, AI systems can help improve government functions. They have the potential to be used to triage community feedback, provide translation services that make government more accessible, facilitate emergency preparedness, or aid scientific research, among other uses. For example, Maryland’s Department of Labor is partnering with academic researchers to help test how AI can train staff and assist caseworkers with compliance regulations and other complex paperwork.

People want government leaders to take action to ensure AI technology is used for the public good. As the current administration has undermined safeguards at the federal level and issued executive orders attempting to stifle state action on AI, the continuing work of state leaders to safeguard rights and center people in AI decision-making has become even more critical.

A few states have already taken some steps to implement process mechanisms for AI decision-making and potential use. These include: Connecticut’s Act Concerning Artificial Intelligence, Automated Decision-Making and Personal Data Privacy and AI Responsible Use Framework; California’s State Guidelines for Evaluating Impacts of Generative AI on Vulnerable and Marginalized Communities; Maryland’s Responsible AI Policy;  New York State’s 2024 LOADinG Act; and Texas’ Responsible Artificial Intelligence Governance Act.

While these steps are an important start, more needs to be done given what is at stake with AI use and its potential impact on people’s rights, livelihoods, and personal safety. For the potential benefits of AI to actually be realized for community members, strong state leadership in this moment is needed to pierce through the hype. This memo lays out a plan of action for state leaders to implement consistent, inclusive people-first AI decision-making structures that do not skip over the foundational questions of why and whether to use AI in the first place.

Plan of Action

State leaders should establish a people-centered decision-making process that consistently and thoughtfully considers why and whether to use AI before jumping to use policies or other safeguards. This process should be followed whenever a state is considering the acquisition or use of an artificial intelligence system, whether through formal procurement, partnerships, in-kind donations, or other means. This decision-making process should be utilized when considering any AI system that has the potential to impact people’s rights, opportunity, well-being, safety, and security.

In the following section we provide:

• The four key steps a people-first AI decision-making process should follow.
• Examples of AI uses that should be prohibited.
• Recommendations for how to operationalize the decision-making process via executive or legislative action, depending on the needs and structure of a given state government.
• Methods to ensure that the people- centered decision-making process is followed and enforceable.

The Four Key Steps for People-First AI Decision-Making

Step 1. Articulate a specific and inclusive “why” for AI use that centers the interests and voices of diverse community members to identify problems and needs.

State leaders should ensure that the first step in decision-making about any existing or potential use of an AI system is for an agency to articulate a specific and inclusive “why” that centers the interests and voices of a wide range of community members. Particular attention should be paid to historically marginalized communities. This community engagement should happen pre-procurement or use of any AI system.

Key considerations for centering diverse community members include, but are not limited to:

Inclusivity and representation: Use multiple strategies to support participation from diverse stakeholders, including funding and support for state agency outreach. Develop potential partnerships with trusted local organizations such as community groups, faith-based organizations, schools, and neighborhood associations who can help spread the word, organize meetings, and share information and surveys with diverse community members.

Accessibility: Make it possible for diverse community members to be actively engaged through a combination of in-person and remote engagement mechanisms. Also provide asynchronous paper and online surveys distributed in multiple languages in easy-to-understand formats. Information about any proposed AI systems should describe how a system would work and what it would do in ways that the general public can understand. Schedule any in-person meetings in places and times when diverse community members will be able to attend and provide necessary support for participation, like childcare and transportation. Remote meetings should also be scheduled at a time in the day when working people and people with families can attend.

Power sharing: Centering diverse voices means meaningful collaboration, not token consultation. Community members should have genuine influence on determining what are the most important issues facing them and how they should be addressed. You should listen to community members about any non-AI solutions that they would prefer and why.

Transparency and Accountability: Be clear about the engagement process and ensure it allows for serial feedback. Make sure materials are publicly published and easily accessible on a government website in a timely manner to allow public engagement with the process. Articulate how community input will be incorporated and have a mechanism to report back to the community on how their input influenced the ultimate decision.

California took important steps to promote effective community consultation when it issued the State Guidelines for Evaluating Impacts of Generative AI on Vulnerable and Marginalized Communities. Authored by the state Government Operations Agency, Office of Data and Innovation, and California Department of Technology, the guidelines recognize the need for a systematic approach that leads with meaningful engagement with diverse communities and how critical it is to specifically consider potential impacts on vulnerable and marginalized communities. Appendix B of California’s guidelines provides some additional helpful guidance on key principles, structures, activities, and focus questions for community consultation.

Step 2. Conduct an AI Impact Assessment that evaluates public benefits and risks, including how the AI system would use people’s information, its impact on rights, and risks of discrimination and bias.

Technology vendors often tout the benefits and downplay costs and risks. It is crucial that amidst the hype state leaders create the structures and processes to support evidence-based decisions about a potential system’s public benefit and risks and avoid AI “snake oil” that wastes state resources and does more harm than good.

State leaders should ensure that there is an AI Impact Assessment (AIIA) to evaluate and explain how the proposed AI system will work, the evidence for its effectiveness and potential public benefit, and its potential for harm (for implementation advice, see below section, “Mechanisms to Operationalize People-Centered AI Decision-Making”). The process should include a public comment period for engagement with the AIIA so people can bring up additional information and concerns. Leaders should also ensure that any company they potentially contract with provides them with the necessary information to conduct an AIIA. Don’t let vendor claims, including claims about potential trade secrets, prevent meaningful review of its products and services.

An AI Impact Assessment (AIIA) should include:

• The intended use cases of the AI system, including its potential public benefits.
• The people and groups most likely to be impacted by the AI use.
• Information about:
  • How the AI system was built and trained, including the information used.
  • The kind of information that the AI system would collect, use, and output.
  • How the public would be informed when they are interacting with the AI system.
• Evidence demonstrating the effectiveness of the proposed AI system at directly addressing and solving the stated purpose and achieving the desired public benefit.
• Report with specificity on the potential public harms of the AI system related to:
  • Civil liberties and civil rights, including privacy and surveillance, free speech, due process, human autonomy, and bias and discrimination.
  • Equitable access to education, health, housing, employment, and other public services and benefits.
  • Energy consumption, carbon emissions, water usage, electronic waste, and other similar environmental impacts.
• Financial impacts, including initial purchase, personnel and other ongoing costs, and any current or potential sources of funding.
• What alternative interventions have been considered to address the stated purpose for the AI system.
• How the AI system would be monitored and regularly evaluated as to public benefit and harms, including the feasibility of meaningful human oversight. 
• The public comment period and how people can engage with the content of the AIIA.

Step 3. Use a decision-making standard that is based on diverse community considerations and an evidence-based inquiry that the public benefit justifies the proposed use and substantially outweighs the potential harms.

Decisions about why and how to deploy AI should be driven by the real needs and interests of impacted communities. Using the AI Impact Assessment and the input and preferences of potentially impacted communities, the agency or department should apply a public benefit standard, assessing whether such a purpose for the AI has been demonstrated and whether the evidence-based benefits of the particular use of AI substantially outweigh the potential harms.

This decision-making standard should give strong weight to the opinions of those who will be impacted by the technology, especially historically marginalized communities. Steps to accomplish this include: 

• Reviewing comments, survey responses, and meeting notes from the community engagement.
• Summarizing community support and concerns in a report.
• Explaining how community input shaped the decision.

Decisions should clearly articulate what quantitative and qualitative evidence was relied on for the decision. These considerations should be memorialized in a publicly accessible document.

Step 4. Conduct timely, ongoing evaluation of AI systems to determine whether they should continue to be used.

If a state entity moves forward with use of a particular AI system, state leaders should require timely review that centers impacted communities in the qualitative and quantitative evaluation of whether the system is achieving the intended public benefit. This review should also identify any harms arising from the AI use. If public benefits of the particular use of AI do not continue to substantially outweigh the harms, the AI use should end.

The review and evaluation processes should ensure:

• Ongoing feedback loops with people using the AI system with channels for community members to reach out about the benefits and problems related to the AI system.
• Proactive efforts to gather evidence of AI efficacy in meeting its original stated public benefit goals.
• Proactive efforts to assess whether the AI system has impacted people’s rights or otherwise affected equal opportunities to government services.
• Initial evaluation should be planned for no later two years after initial use and then regularly thereafter.

Recommendation 1. Some uses of AI are simply too dangerous. Get ahead by taking them off the table.

Putting people first in AI also means proactively prohibiting uses of AI systems and applications that are simply incompatible with democratic, civil, and human rights. Numerous evaluations from government leaders, academics, technologists, civil rights organizations, and groups representing vulnerable and marginalized communities have found that the threats stemming from the below applications of AI significantly outweigh the benefits. Your AI decision-making process should preclude the following:

• Social scoring systems
• Emotional Recognition systems
• Facial surveillance or other biometric surveillance
• AI use in predictive policing or family policing
• AI control over a person’s life and liberty, including in criminal justice and immigration

Many prudent city and state government officials have already preemptively taken some dangerous AI uses off the table. Maryland’s AI policy prohibits AI that violates fundamental rights, such as social scoring and emotional recognition. Montana’s AI law bans using AI for cognitive behavioral manipulation and sets hard limits on dragnet mass surveillance. And many cities have prohibited government use of face surveillance.

Recommendation 2. Mechanisms to Operationalize People-Centered AI Decision-Making

How to best implement the AI decision-making framework depends on the particular needs, opportunities, and structure of each state government. States that have taken steps to create a consistent process for AI evaluation and adoption have done so through different legal and legislative mechanisms. Which option to pursue – executive action, legislation, agency guidelines, or a combination of the three – is a decision that should be made by those most familiar with the contours of their particular state.

Executive Action – A Governor can issue an executive order requiring all executive agencies to follow a people-centered AI decision-making process. This executive order can identify an agency, or a subset of existing agencies, to develop the process itself and coordinate among different department leaders and staff to provide expertise and oversight that ensures compliance. If relying on an existing agency or state department, state leaders may find that an agency or department already focused on technology, information services, operations, or administrative service might be most well-suited to this role. Or an executive order can create a new entity to provide support.

• California’s Executive Order on AI provided for the Government Operations Agency, the California Department of General Services, the California Department of Technology, and the California Cybersecurity Integration Center to develop guidelines for government procurement and use of AI systems, including guidelines that agencies should follow for evaluating equity concerns of generative AI and its impact on marginalized communities.
• Maryland’s Executive Order created an AI subcabinet charged with facilitating statewide coordination on the responsible, ethical, and productive use of AI. Its work included recommending approaches and state policies.

Legislation – State lawmakers can enact legislation to require state entities to create and follow an AI decision-making process, either through direct statutory language or by tasking a state agency to develop policy and implementation guidelines.

• The New York state LOADinG Act required legal authorization for the use of automated decision-making systems by state agencies and required the publication of an impact assessment for any authorized AI use.
• Connecticut’s Public Act No. 23-16 directed the state’s Office of Policy and Management to develop Connecticut’s AI Responsible Use Framework. It also instructed the Department of Administrative Services to inventory all AI systems.
• Maryland enacted the Artificial Intelligence Governance Act of 2024 and required each executive agency to appoint an AI lead and for the State Department of Information Technology (DoIT) to work with the new AI subcabinet. The DoIT then issued the State of Maryland’s Responsible AI Policy and implementation guidance.

Recommendation 3. Provide Support Structures for State Agencies

State leaders should ensure that there are structures to support state agencies to operationalize the people-centered decision-making process, including conducting diverse community outreach, evidence-based AI Impact Assessment, and quantitative and qualitative evaluation.

This support can come from a variety of sources. State leaders should provide funding for existing staff or agencies to serve as point people, creating a diverse AI board, partnering with academic institutions to provide expertise, or a combination of these strategies.

• In California, the Governor designated multiple state agencies to work on AI implementation. The state government also partnered with academic institutions including UC Berkeley and Stanford to convene an AI Summit bringing together stakeholders from labor, business, nonprofit, academia, and technology companies. This summit convened a series of workshops with grassroots organizations and diverse community members to gather input on AI use and potential policies for assessing impact on vulnerable communities. In December 2025, the Governor also launched the California Innovation Council, an initiative to tap “California’s best and brightest to advance responsible AI.”
• In Maryland, the Governor established an AI subcabinet by executive order and state leaders enacted legislation for the state Department of Information Technology (DoIT) to work with the new AI subcabinet. The state issued a Responsible AI Policy that includes roles and responsibilities for the DoIT, the AI subcabinet, and each executive agency. The DoIT implementation guidance also helps state agencies understand, operationalize, and follow the AI Policy. Each agency has a designated AI Lead and the DoIT has an AI team that holds weekly office hours and answers questions that agencies may have about using AI and following the state’s AI policy.

Recommendation 4. Ensure the Process is Followed Through with Transparency, Accountability, and Oversight

It is also essential for state leaders to make sure the decision-making process does not just work on paper, but truly translates into people-centered transparency, accountability, and oversight of AI systems.

Any legislation, executive order, or agency guidelines should provide for public and private enforcement mechanisms so people can take action if rules are not followed. State leaders should also require a public inventory, updated at least annually, of all AI systems so the public knows what is in use. As discussed earlier, all assessment materials need to be publicly published in a timely manner during the process.

After the decision-making process is completed, state leaders should ensure that any agency that moves forward with an AI system is required to establish a robust use policy that will help protect people from abuse, misuse, and mistakes, with ongoing evaluation of the benefits and harms of the AI system. Developing a robust use policy is outside the scope of this memo, but please see the FAQ section for some resources.

Conclusion

State leaders can make AI work for people.

The future of government use of AI is still being written, and state governments have a powerful role to play. What we do now will help determine whether the power of AI will work for or against people’s rights and dignity.

If AI is to serve rights, justice, and democracy, leaders at the state level must act to implement a people-first process that centers diverse community members and asks and answers foundational questions about “why” and “whether” to use AI before skipping to the “how” of AI implementation. The recommendations in this memo help state leaders meet this moment and ground decision-making about AI use in fairness, accountability, evidence-based inquiry, and inclusive governance.

The views and opinions expressed herein are solely those of the author and do not necessarily reflect the views, positions, or policies of any organization, employer, board, institution, client, or other entity with which the author is affiliated.

By now, you’ve probably heard that most Americans do not trust AI. This distrust is especially concerning given how deeply these systems are already shaping access to healthcare, education, housing, jobs, and public benefits. Too often, these decisions happen without transparency, oversight, or meaningful avenues for recourse. At the same time, confidence in both technology companies and government institutions to manage AI responsibly remains low.

The stakes are clear, and the policy choices we make today will make or break society’s view of AI. We are currently at a critical opportunity to shape how AI is governed before harmful practices and inequities become further entrenched. To meet this moment, the Federation of American Scientists, with the support of the Kapor Foundation, launched a policy sprint, which is an intensive, time-bound effort designed to tackle complex challenges quickly and collaboratively. Policy sprints bring together experts from across disciplines, from academics, technologists, advocates, and practitioners, to develop practical, actionable solutions.

For our SOURCE CODE  AI Trust and Fairness Sprint, we’ve developed  10 memos with  leading experts that are detailed, implementable policy solutions. We have delved into why fairness is so hard to define and implement, and what is needed to promote public trust in our essay that frames this new policy agenda. These memos are not exhaustive; we know the landscape of challenges and potential solutions is far broader. Instead, we offer them as a starting point: ideas that we hope will not only serve as smart and actionable tools for policymakers, but also inspire the community to build out and advance new, detailed approaches.

• Jae Yeon Kim and Aniket Kesari show how government procurement can be a powerful but underused lever to require transparency, fairness, and accountability in AI systems.
  
• Nicole Ozer and Brady Hirsch emphasize that in order for AI to work for the people, state leaders must adopt people-centered AI decision making frameworks. hey explain how to ask and answer foundational questions about why and whether to use AI before skipping to the how of implementation.
  
• JB Branch highlights similar risks in K–12 education, proposing procurement guardrails and impact assessments to protect students’ rights and safety.
  
• Liza Paudel advances Community Benefit Agreements as a tool for ensuring that communities, especially those disproportionately impacted, secure enforceable protections as AI-driven data center development accelerates.
  
• Wilneida Negrón examines the rise of “surveillance pay,” calling for stronger regulatory safeguards to protect workers from opaque, data-driven compensation systems.
  
• Anna Lenhart proposes embedding “Decision Subject Representatives” into AI governance, giving impacted individuals a formal role in shaping high-stakes systems.
  
• Y. Tony Yang focuses on Medicaid and safety-net healthcare, proposing community-centered oversight, nurse-led audits, and patient protections to ensure AI improves rather than undermines care.
  
• Ziwei Qi, Tatiana Lin, and Ayokunle Olagoke highlight the risks of “algorithmic invisibility” in rural communities and call for “rural AI proofing” to ensure these populations are not overlooked in AI system design, governance, and implementation.
  
• Jon M. Peha explores the risks of AI-generated police reports and proposes coordinated federal and local strategies that foster innovation and prevent unsafe deployments.
  
• Gaurav Laroia and Charlotte Slaiman argue that enforcers must look beyond standard monetary penalties to truly protect communities from algorithmic harms. They show how historic tech settlements can be transformed into an active defense, building societal resilience and preparing the public for the next wave of technological change.

To structure our policy agenda, we have considered how these ideas have employed targeted mechanisms, which we call policy levers, to ensure legitimate use, fair outcomes, and public trust in AI. These levers include government use of AI, public engagement, sectoral considerations, and redress in legal remedies; many of our policy ideas use several of them.

This work was shaped by a multidisciplinary advisory working group of 17 experts from academia, civil rights organizations, think tanks, and beyond. Their insights helped refine the project’s focus, identify the most pressing policy opportunities, and strengthen each memo through expert review. Advisory members provided input in an individual capacity; their participation does not imply endorsement of the findings or recommendations. We are grateful for their support and expertise throughout this process.

• Quinn Anex-Ries, Center for Democracy & Technology
• Alex Ault, Lawyers’ Committee for Civil Rights Under Law
• Annette Bernhardt, UC Berkeley Labor Center
• Miranda Bogen, Center for Democracy & Technology
• Leah Frazier, Lawyers’ Committee for Civil Rights Under Law
• Sanmi Koyejo, Stanford University
• Sarah Myers West, AI NOW
• Tejas N. Narechania, UC Berkeley School of Law
• Cathy O’Neil
• Asad Ramzanali, Vanderbilt Policy Accelerator 
• Cody Venzke, ACLU 
• Jonathan Walter, Leadership Conference on Civil and Human Rights

The ideas generated through this sprint are practical, actionable, and grounded in existing authority. They demonstrate that policymakers already have many of the tools needed to govern AI effectively; they simply need to be deployed with urgency and intention. There is a narrowing window to shape the integration of AI systems into society. With thoughtful policy action, it is still possible to build systems that are fair, transparent, and accountable, and to earn the public trust that will ultimately determine AI’s future. We hope policymakers are ready to act.

Persons with disabilities (PWDs) are considered the largest minority in the nation and in the world. There are existing policies and procedures from agencies, directorates, or funding programs that provide support for Accessibility and Accommodations (A&A) in federally funded research efforts. Unfortunately, these policies and procedures all have different requirements, processes, deadlines, and restrictions. This lack of standardization can make it difficult to acquire the necessary support for PWDs by placing the onus on them or their Principal Investigators (PIs) to navigate complex and unique application processes for the same types of support. 

This memo proposes the development of a standardized, streamlined, rolling, post-award support mechanism to provide access and accommodations for PWDs as they conduct research and disseminate their work through conferences and convenings. The best case scenario is one wherein a PI or their institution can simply submit the identifying information for the award that has been made and then make a direct request for the support needed for a given PWD to work on the project. In a multi-year award such a request should be possible at any time within the award period. 

This could be implemented by a single, streamlined policy adopted by all agencies with the process handled internally. Or, by a new process across agencies under Office of Science and Technology Policy (OSTP) or Office of Management and Budget (OMB) that handles requests for accessibility and accommodations at federally funded research sites and at federally funded convenings. An alternative to a single streamlined policy across these agencies might be a new section in the uniform guidance for federal funding agencies, also known as 2 CFR 200.

This memo focuses on Federal Open Science funding programs to illustrate the challenges in getting A&A funding requests supported.  The authors have taken an informal look at agencies outside of science and technology funding.  We found similar challenges across federal grantmaking in the Arts and Humanities, Social Services, and Foreign Relations and Aid entities. Similar issues likely exist in private philanthropy as well.

Challenge and Opportunity

Deaf/hard-of-hearing (DHH), Blind/low-vision (BLV), and other differently abled academicians, senior personnel, students, and post-doctoral fellows engaged in federally funded research face challenges in acquiring accommodations for accessibility. These include, but are not limited to: 

• Human-provided ASL-English interpreting and interview transcription services for the DHH and non-DHH participants.  While there are some applications of artificial intelligence (AI) that show promise on the transcription side, there’s a long way to go on ASL interpretation in an AI provided model versus the use of human interpreters. 
• Visual and Pro-tactile interpreting/descriptive services for the BLV participants 
• Adaptive lab equipment and computing peripherals 
• Accessibility support or remediation for physical sites

Having these services available is crucial for promoting an inclusive research environment on a larger scale. 

Moving to a common, post-award process:

• Allows the PI and the reviewers more time and space to focus on the core research efforts being described in the initial proposal
• Removes any chance of the proposal funding being taken out of consideration due to higher costs in comparison to similar proposals in the pool
• Creates a standard, replicable pathway for seeking accommodations once the overall proposal has been funded. This is especially true if the support comes from a single process across all federal funding programs rather than within each agency.
• Allows for flexibility in accommodations. Needs vary from person-to-person and case-to-case. For example, in the case of workplace accommodations for DHH team members, one full-time researcher may request full-time ASL interpretation on-site, while another might prefer to work primarily through digital text channels; only requiring ASL interpretation for staff meetings and other group activities.
• Potentially reduces federal government financial and human resources currently expended in supporting such requests by eliminating duplication of effort across agencies or, at minimum streamlining processes within agencies.

Such a process might follow these steps below. The example below is from the National Science Foundation (NSF), but the same, or similar process could be done within any agency:

1. PI receives notification of grant award from NSF. PI identifies need for A & A services at start, or at any time during the grant period
2. PI (or SRS staff) submits request for A&A funding support to NSF. Request includes NSF program name and award number, the specifics of the requested A & A support, a budget justification and three vendor quotes (if needed)
3. Use of funds is authorized, and funding is released to PI’s institution and acquisition would follow their standard purchasing or contracting procedures
4. PI submits receipts/ paid vendor invoice to funding body
5. PI cites and documents use of funds in annual report, or equivalent, to NSF

Current Policies and Practices

Pre-Award Funding

Principal Investigators (PIs) who request A&A  support for themselves or for other members of the research team are sometimes required to apply for it in their initial grant proposals. This approach has several flaws. 

First and foremost, this funding process reduces the direct application of research dollars for these PIs and their teams compared to other researchers in the same program. Simply put, if two applicants are applying for a $100,000 grant, and one needs to fund $10,000 worth of accommodations, services, and equipment out of the award, they have $10,000 less to pursue the proposed research activities.  This essentially creates a “10% A & A tax” on the overall research funding request.

Lived Experience Example

In a real world example, the author and his colleague, the late Dr. Mel Chua, were awarded a $60,000, one year grant to do a qualitative research case study as part of the Ford Foundation Critical Digital Infrastructure Research cohort.  As Dr. Chua was Deaf, the PIs pointed out to Ford that $10,000 worth of support services would be needed to cover costs for 

• American Sign Language (ASL) interpreters during the qualitative interviews and advisory committee meetings
• Transcription of the interviews
• ASL Interpreting for conference dissemination and collection of comments at formal and informal meetings during those conferences

We communicated the fact that spending general research award money on those services would reduce the research work the funds were awarded to support.  The Ford Foundation understood and provided an additional $10,000 as post-award funding to cover those services. Ford did not inform the PIs as to whether that support came from another directed set of funds for A&A support or from discretionary dollars within the foundation.

Second, it can be limiting for the funded project to work with or hire PWDs as co-PIs, students, or if they weren’t already part of the original grant proposal. For example, suppose a research project is initially awarded funding for four years without A&A support and then a promising team member who is a PWD appears on the scene in year three who would require it. In this case, PIs then must: 

• Reallocate research dollars meant for other uses within the grant to support A&A;
• Find other funding to support those needs within their institution;
• Navigate the varied post-award support landscape, sometimes going so far as to write an entirely new full proposal with a significant review timeline, to try to get support. If this happens off cycle, the funding might not even arrive until the last few months of the fourth year. 
• Or, not hire the person in question because they can’t provide the needed A&A.

Post-Award Funding

Some agencies have programs for post-award supplemental funding that address the challenges described above. While these are well-intentioned, many are complicated and often have different timelines, requirements, etc. In some cases, a single supplemental funding source may be addressing all aspects of diversity, equity and inclusion as well as A&A.  The needs and costs in the first three categories are significantly different than in the last. Some post-award pools come from the same agency’s annual allocation program-wide. If those funds have been primarily expended on the initial awards for the solicitation, there may be little, or no money left to support post-award funding for needed accommodations. The table below briefly illustrates the range of variability across a subset of representative supplemental funding programs. There are links in the top row of the table to access the complete program information. Beyond the programs in this table, more extensive lists of NSF and NIH offerings are provided by those agencies. One example is the NSF Dear Colleague Letter Persons with Disabilities – STEM Engagement and Access.

Ideally these policies and procedures, and others like them, would be replaced by a common, post-award process. PIs or their institutions would simply submit the identifying information on the grant that had been awarded and the needs for Accommodations and Accessibility to support team members with disabilities at any time during the grant period.

Plan of Action

The OSTP, possibly in a National Science and Technology Council interworking group process,, should conduct an internal review of the A&A policies and procedures  for grant programs from federal scientific research aligned agencies. This could be led by OSTP directly or under their auspices and led by either NSF or the National Institute of Health (NIH).  Participants would be relevant personnel from DOE, DOD, NASA, USDA, EPA, NOAA, NIST and HHS, at minimum. The goal should be to create a draft of a single, streamlined policy and process, post-award, for all federal grant programs or a new section in the uniform guidance for federal funding agencies.

There should be an analysis of the percentages, size and amounts of awards currently being made to support A&A in research funding grant programs. It’s not clear how the various funding ranges and caps listed in the table above were determined or if they meet the needs. One goal of this analysis would be to determine how well current needs within and across agencies are being met and what future needs might be. 

A second goal would be to look at the level of duplication of effort and scope of manpower savings that might be attained by moving to a single, streamlined policy. This might be a coordinated process between OMB and OSTP or a separate one done by OMB. No matter how it is coordinated, an understanding of these issues should inform whatever new policies or new additions to 2 CFR 200 would emerge. 

A third goal of this evaluation could be to consider if the support for A&A post-award funding might best be served by a single entity across all federal grants, consolidating the personnel expertise and policy and process recommendations in one place. It would be a significant change, and could require an act of Congress to achieve, but from the point of view of the authors it might be the most efficient way to serve grantees who are PWDs. 

Once the initial reviews as described above, or a similar process is completed, the next step should be a convening of stakeholders outside of the federal government with the purpose of providing input to the streamlined draft policy. These stakeholder entities could include, but should not be limited to, the National Association for the Deaf, The American Foundation for the Blind, The American Association of People with Disabilities and the American Diabetes Association. One of the goals of that convening should be a discussion, and decision, as to whether a period of public comment should be put in place as well, before the new policy is adopted. 

Conclusion

The above plan of action should be pursued so that more PWDS will be able to participate, or have their participation improved, in federally funded research. A policy like the one described above lays the groundwork and provides a more level playing field for Open Science to become more accessible and accommodating.It also opens the door for streamlined processes, reduced duplication of effort and greater efficiency within the engine of Federal Science support.

Acknowledgments 

The roots of this effort began when the author and Dr. Mel Chua and Stephen Jacobs received funding for their research as part of the first Critical Digital Infrastructure research cohort and were able to negotiate for accessibility support services outside their award. Those who provided input on the position paper this was based on are: 

• Dr. Mel Chua, Independent Researcher
• Dr. Liz Hare, Quantitative Geneticist, Dog Genetics LLC 
• Dr. Christopher Kurz, Professor and Director of Mathematics and Science Language and Learning Lab, National Technical Institute for the Deaf
• Luticha Andre-Doucette, Catalyst Consulting

This action-ready policy memo is part of Day One 2025 — our effort to bring forward bold policy ideas, grounded in science and evidence, that can tackle the country’s biggest challenges and bring us closer to the prosperous, equitable and safe future that we all hope for whoever takes office in 2025 and beyond.

PLEASE NOTE (February 2025): Since publication several government websites have been taken offline. We apologize for any broken links to once accessible public data.

Congress should approve a new allowable use of Title I spending that specifically enables and encourages school districts to use funds for activities that support and drive equity-focused innovation. The persistent equity gap between wealthy and poor students in our country, and the continuing challenges caused by the pandemic, demand new, more effective strategies to help the students who are most underserved by our public education system.

Efforts focused on the distribution of all education funding, and Title I in particular, have focused on ensuring that funds flow to students and districts with the highest need. Given the persistence of achievement and opportunity gaps across race, class, and socioeconomic status, there is still work to be done on this front. Further, rapidly developing technologies such as artificial intelligence and immersive technologies are opening up new possibilities for students and teachers. However, these solutions are not enough. Realizing the full potential of funding streams and emerging technologies to transform student outcomes requires new solutions designed alongside the communities they are intended to serve. 

To finally close the equity gap, districts must invest in developing, evaluating, and implementing new solutions to meet the needs of students and families today and in a rapidly changing future. Using Title I funding to create a continuous, improvement-oriented research and development (R&D) infrastructure supporting innovations at scale will generate the systemic changes needed to reach the students in highest need of new, creative, and more effective solutions to support their learning. 

Challenge and Opportunity

Billions of dollars of federal funding have been distributed to school districts since the authorization of Title I federal funding under the Elementary and Secondary Education Act (ESEA), introduced in 1965 (later reauthorized under the Every Student Succeeds Act [ESSA]). In 2023 alone, Congress approved $18.4 billion in Title I funding. This funding is designed to provide targeted resources to school districts to ensure that students from low-income families can meet rigorous academic standards and have access to post-secondary opportunities. ESEA was authorized during the height of the Civil Rights Movement with the intent of addressing the two primary goals of (1) ensuring traditionally disadvantaged students were better served in an effort to create more equitable public education, and (2) addressing the funding disparities created by differences in local property taxes, the predominant source of education funding in most districts. These dual purposes were ultimately aimed at ensuring that a student’s zip code did not define their destiny.

The passing of ESEA was a watershed moment. Prior to its authorization, education policy was left mostly up to states and localities. In authorizing ESEA, the federal government launched ongoing involvement in public education and initiated a focus on principles of equity in education.

Further, research shows that school spending matters: Increased funding has been found to be associated with higher levels of student achievement. However, despite the increased spending for students from low-income families via Title I, the literature on outcomes of Title 1 funding is mixed. The limited impact of Title I funds on outcomes may be a result of municipalities using Title I funding to supplant or fill gaps in their overall funding and programs, instead of being used as an additive funding stream meant to equalize funding between poorer and richer districts. Additionally, while a taxonomy of options is provided to bring rigor and research to how districts use Title funding, the narrow set of options has not yielded the intended outcomes at scale. For instance, studies have repeatedly shown that school turnaround efforts have proven particularly stubborn and not shown the hoped-for outcomes.

The equity gap that ESEA was created to address has not been erased. There is still a persistent achievement gap between high- and low-income students in the nation. The emergence of COVID in 2020 uprooted the public education system, and its impact on student learning, as measured by test scores, is profound. Students lost ground across all focus areas and grades. Now, in the post-pandemic era, students have continued to lose ground. The “COVID Generation” of students are behind where they should be, and many are disengaged or questioning the value of their public education. Chronic absenteeism is increasing across all grades, races, and incomes. These challenges create an imperative for schools and districts to deepen their understanding of the interests and needs of students and families. The quick technological advancements in the education market are changing what is possible and available to students, while also raising important questions around ethics, student agency, and equitable access to technology. It is a moment of immense potential in public education. 

Title I funds are a key mechanism to addressing the array of challenges in education ranging from equity to fast-paced advancements in technology transforming the field. In its current form, Title I allocation occurs via four distribution criteria. The majority of funding is allocated via basic grants that are determined entirely on individual student income eligibility. The other three criteria allocate funding based on the concentration of student financial need within a district. Those looking to rethink allocation often argue for considering impact per dollar allocated, beyond solely need as a qualifying indicator for funding, essentially taking into account cost of living and services in an area to understand how far additional funding will stretch in order to more accurately equalize funding. It is essential that Title I is redesigned beyond redoing the distribution formula. The money allocated must be spent differently—more creatively, innovatively, and wisely—in order to ensure that the needs of the most vulnerable students are finally met.

Plan of Action

Title I needs a new allowable spending category approved that specifically enables and encourages districts to use funds for activities that drive equity-focused innovation. Making room for innovation grounded in equity is particularly important in this present moment. Equity has always been important, but there are now tools to better understand and implement systems to address it. As school districts continue to recover from the pandemic-related disruptions, explore new edtech learning options, and prepare for an increasingly diverse population of students for the future, they must be encouraged to drive the creation of better solutions for students via adding a spending category that indicates the value the federal government sees in innovating for equity. Some of the spending options highlighted below are feasible under the current Title I language. By encouraging these options tethered specifically to innovation, district leadership will feel more flexibility to spend on programs that can foster equity-driven innovation and create space for the new solutions that are needed to improve outcomes for students.

Innovation, in this context, is any systemic change that brings new services, tools, or ways of working into school districts that improve the learning opportunities and experience for students. Equity-focused innovation refers to innovation efforts that are specifically focused on improving equity within school systems. It is a solution-finding process to meet the needs of students and families. Innovation can be new, technology-driven tools for students, teachers, or others who support student learning. But innovation is not limited to technology. Allowing Title I funding to be used for activities that support and foster equity-driven innovation could also include:

• Improving data systems and usage: Ensure that school districts have agile data systems equipped to identify student weaknesses and determine the effectiveness of solutions. As more solutions come to market and are developed internally, both AI and otherwise, school systems will be able to better serve students qualifying for Title I funding if they can meaningfully assess what is and is not working and use that information to guide strategy and decision-making.
• Leadership development: Support the research and development, futurist, and equitable design skills of systems to enable leaders to guide innovation from within districts alongside students and families.
• Testing new solutions: Title I funding currently can be spent primarily on evidence-based programs; enabling the use of funding for innovative pilots that have community support would provide space to discover more effective solutions. 
• Incentivizing systemic district innovation: School districts could use funding to support the creation of innovation offices within their administration structure that are tasked with developing an innovation agenda rooted in district and student needs and spearheading solutions. 
• Building networks for change: District leaders charged with creating and sustaining new learning models, school models, and programs often do so in isolation. Allowing districts to fund the creation of new programs and support existing organizations that bring together school system innovators and researchers to capture and share best practices, promising new solutions, and lessons learned from testing can lead to better adoption and scale of promising new models. There are already networks that exist, for instance, the Regional Education Laboratory Program. Funding could be used to support these existing networks or to develop new networks specifically tailored to meet the needs of leaders driving these innovations.

Expanding Title I funding to make room for innovative ideas and solutions within school systems has the potential to unlock new, more effective solutions that will help close equity gaps, but spending available education funds on unproven ideas can be risky. It is essential that the Department of Education issues carefully constructed guardrails to allow ample space for new solutions to emerge and scale, while also protecting students and ensuring their educational needs are still met. These guardrails and design principles would ensure that funds are spent in impactful ways that support innovation and building an evidence base. Examples of guardrails for a school system spending Title I funding on innovation could include:

• Innovation agenda: There should be a clearly articulated, publicly available innovation agenda that lays out how needs are being identified using quantitative and qualitative data and research, the methods of how innovations are being developed and selected, the goals of the innovation and how the work will grow (or not) based on clearly defined metrics of success.
• Clear research & development process: New ideas, tools, and ways of working must come into the district with a clear R&D process that begins with student and community needs and then regularly interrogates what is and is not working, tries to understand the why behind what is working, and expands promising practices.
• Pilot size limits: Unproven and innovative ideas should begin as pilots in order to ensure they are tested, evaluated, and proven before being used more broadly.
• Timeline requirements for results: New innovation funded via Title I funding should have a limited timeline during which the system needs to show improvement and evidence of impact.
• Clear outcomes that the innovation is aiming for: Innovation is not about something new for the sake of something new. Innovation funding via Title I funding must be linked to specific outcomes that will help achieve the overarching programmatic goal of increasing educational equity in our country. 

While creating an authorized funding category for equity-focused innovation through Title I would have the most widespread impact, other ways to drive equitable innovation should also be pursued in the short term, such as through the new Comprehensive Center (CC), set to open in fall 2024, that will focus on equitable funding. It should prioritize developing the skills in district leaders to enable and drive equity-driven innovation. 

Conclusion

Investment in innovation through Title I funding can feel high risk compared to the more comfortable route of spending only on proven solutions. However, many ways of traditional spending are not currently working at scale. Investing in innovation creates the space to find solutions that actually work for students—especially those that are farthest from opportunity and whom Title I funding is intended to support. Despite the perceived risk, investing in innovation is not a high-risk path when coupled with a clear sense of the community need, guardrails to promote responsible R&D and piloting processes, predetermined outcome goals, and the data systems to support transparency on progress. Large-scale, federal investment in creating space for innovation through Title I funding in—an already well-known mode of district funding not currently realizing its desired impact—will create solutions within public education that give students the opportunities they need and deserve.

This action-ready policy memo is part of Day One 2025 — our effort to bring forward bold policy ideas, grounded in science and evidence, that can tackle the country’s biggest challenges and bring us closer to the prosperous, equitable and safe future that we all hope for whoever takes office in 2025 and beyond.

PLEASE NOTE (February 2025): Since publication several government websites have been taken offline. We apologize for any broken links to once accessible public data.

This memo was developed in partnership with the Alliance for Learning Innovation, a coalition dedicated to advocating for building a better research and development infrastructure in education for the benefit of all students. Read more education R&D memos developed in partnership with ALI here.

Have you ever noticed a lack of tree cover in certain areas of a city? Have you ever visited a city and been advised to avoid certain districts or communities? Perhaps you even recall these visual shifts occurring immediately after crossing a particular road or highway? 

If so, what you experienced was likely by design:

In the early 20^th century, Black communities across the U.S. were subjected to economic constraint and social isolation through housing policies that mandated segregation. Black communities were systematically excluded from the housing benefits offered by President Franklin D. Roosevelt’s New Deal and Homeowners’ Loan Corporation (HOLC). The HOLC served as the basis of the National Housing Act of 1934, which ratified the Federal Housing Authority (FHA). 

Housing policy discrimination was further exacerbated by the FHA refusing to insure mortgages near and within Black neighborhoods. The HOLC provided lenders with maps that circled areas with sizeable black populations with red markers—a practice now referred to as redlining. While the systematic practice of redlining ended in 1968 under The Fair Housing Act of 1968, redlining continues to economically impair over 11 million Americans—and less than half are Black.

You are probably thinking (1) how is this possible? (2) How could a defunct 20^th-century policy designed to discriminate against Black communities still impact over 11 million—mostly non-Black—Americans today? The answer is the same for both questions: place-based discrimination.

Policies such as redlining are designed to worsen the material conditions of a target group by preventing investment in the places where they live. Over time, this results in physical locations that are systemically denied access to features such as loans, enterprise, and ecosystem services  simply due to their location or place. Place-based discrimination is the principal mechanism of redlining effects, and consequently, costs taxpayers millions of dollars per year.

What is the problem?

Starting in the 1990s, during the Clinton Administration, billions of dollars in tax credits were devoted towards community development and economic growth through the use of special tax credits that attract private investments (Table 1). One of the principal agents from this funding to address place-based discrimination was the creation of Community Development Entities (CDEs). According to the New Markets Tax Credit Coalition, CDEs are private entities that have “demonstrated” an interest in serving or providing capital to low-income- communities (LICs) and individuals (LIIs). Once certified, CDEs are eligible to apply for a special tax credit, New Markets Tax Credit (NMTC), through the Community Development Financial Institution (CDFI) Fund. 

However, this program, and others like it, have had a negligible impact on addressing the systemic implications of redlining . A recent Urban Institute report found that inequity in capital flow and investment trends within cities (i.e., Chicago) is driven by residential lending patterns. Highlighting the inequalities that exist between investment among neighborhoods with different racial and income demographics, the analysts surmise that redressing economic downturn involves expanding investments into divested neighborhoods. To date, more than $71 billion have been awarded to CDE’s, and yet, historically-redlined areas remain economically desolate. If these programs are intended to economically revitalize historically-redlined areas, then these programs are not doing what they are supposed to do.

One example of this is the city of Philadelphia:

Philadelphia, a city in the top ten for redlined populations, possesses tens of thousands of vacant buildings and lots that are overlaid by redlining and riddled with brownfield sites. According to the Philadelphia Office of the Controller, historically redlined communities of Philadelphia continue to experience disproportionate amounts of poverty, poor health outcomes, limited educational attainment, unemployment, and violent crime compared to non-redlined areas in the city.

By analyzing HOLC assessment grades (1937) and New Market Tax Credit (NMTC) Program Eligibility (i.e., PolicyMap, projects from 2015-2019) for Philadelphia, PA, I found that of the 30+ Qualified Low-Income Community Investments (QLICIs) in historically-redlined areas, totaling over $400 million in tax credits, none are categorized as Community Development Entities (CDEs).

1937 vs. 2019

HOLC assessment grades (1937) vs. New Market Tax Credit (NMTC) Program Eligibility

1937 and 2019 overlaid

Of the 30+ Qualified Low-Income Community Investments (QLICIs) in historically-redlined areas, totaling over $400 million in tax credits, none are categorized as Community Development Entities (CDEs).

Meanwhile, the Philadelphia City Council just passed a budget that allocates a record $788 million to the Philadelphia Police Department (PPD). Recent studies show that fatal encounters with police are more likely to occur within historically-redlined areas. It appears the nicest buildings in redlined areas may very well be police stations.

Yet, public investment has been more concerned with maintaining systems of oppression than reversing them. Why continue to invest in systems that do not create wealth? No matter your perception of American policing, the following is clear: policing does not create wealth for distressed communities.  

Currently, there are 200+ cities and thousands of communities that are, like Philadelphia, enduring the systemic implications of redlining. 

What would happen if public investments were allocated towards restorative policy actions within historically-redlined areas?

A federal program that amalgamates the best elements of community-driven inventiveness into a vehicle for innovative and sustainable economic development. That is, a program that promotes economic revitalization of historically-redlined communities through multipurpose, community-owned enterprises called Innovative Neighborhood Markets (INMs).

What is the policy action?

One thing that urban policy initiatives have made clear, is that distressed communities are prime real-estate targets for private developers . A new federal effort could ensure that investment opportunities are also accessible to community members seeking to launch place-based businesses and enterprises. Businesses and enterprises of this sort will not only reduce urban blight in historically-redlined communities, but also serve as avenues for the direct state, local, and private investment needed to address historical inequities. 

The Biden-Harris Administration can combat redlining through a placed-based community investment program, coined Putting Redlines in the Green: Economic Revitalization Through Innovative Neighborhood Markets (PRITG), that affords historically-redlined communities the ability to establish their own profitable enterprise before outside parties (i.e., private developers).

These Innovative Neighborhood Markets (INMs) would be resource hubs that provide affordable grocery items (i.e., fresh produce, meats, dairy, etc.); an outlet for residents of the community to market goods and services (i.e., small businesses); and create cross-sector initiatives that build community enterprise and increase greenspace (i.e., Farm to Neighborhood Model [F2NM], parks, gardens, and tree cover). Most importantly, INM’s are community owned. Through community governance, the community elects and authorizes the types of place-based businesses and enterprises that are present within their INM.

Do you remember the Philadelphia example from earlier? 

Under PTRIG, a number of those underutilized structures or vacant spaces are transformed into a vested, profitable, and sustainable community resource. The majority of the financial capital remains within the community, and economic gains are partially earmarked for community revitalization (i.e., soil remediation for brownfield sites, community restoration, and construction of greenspace).

All Taxpayers Benefit

By legally and financially empowering communities with ownership, PRITG will incentivize investment and development that can actually reduce taxpayer liability. For example, the INM can generate the funding to invest in more attractive (and expensive) treecover and landscaping that will reduce the impact of heat islands and imperviousness related to redlining, thereby reducing taxpayer liability by more than $308 million dollars per year. Implementation of PTRIG will decrease taxpayer burden through profit-driven and self-supporting community services. 

“Fair and Equal” Access 

Another beneficial aspect of this policy involves increasing community access to financial provisions without third-party obstacles (i.e., CDEs and CDFIs). Black and Hispanic home loan applicants are charged higher interest rates than White home loan applicants, resulting in  Black and Hispanic borrowers paying $765 million in additional interest per year. Discriminatory practices only succeed in worsening community divestment and increasing the resident displacement which disproportionately impact minority residents. Through the economic-agency provided by PRITG, historically-redlined communities would have heightened protection against lending discrimination, gentrification, and displacement. 

Moreover, PTRIG would reinforce the Consumer Financial Protection Bureau (CFPB) and the Office of the Comptroller of the Currency (OCC)’s Combating Redlining Initiative in ensuring that formerly redlined neighborhoods receive “fair and equal access” to the lending opportunities that are—and always have been—available  to non-redlined, and majority-White, neighborhoods. 

While INMs possess aspects of grocery stores, community banks, business improvement districts (BIDs), and farmers markets, they would differ in one particular area: community wealth.

What is Community Wealth?

As someone who grew up in Champaign, Illinois (Douglas Park), and whose family currently lives in a historically-redlined community (Lansing, MI), it brings me peace to reimagine my community with an INM.

Until my early 20’s, I spent most of my life largely unaware of the importance of community wealth on individual empowerment and its impact on the maintenance of cultural identity. For me, reimagining my community with an INM is not just about correcting the past, it is about enriching the uniqueness of what makes our home, Home.

In general, a community wealth building process needs to address the lack of an asset in a way that builds community sustainability. That is the materialization of a communal epicenter(s) that produces a sense of ownership and pride.

So how would INMs build community wealth? Simple. The community, as a whole, would be defined as the ownership group. Each community member would be legally referenced as a shareholder of this newly acquired, financially-appreciating, community-owned enterprise. 

Community Ownership Key to Community Wealth

According to Evan Absher, Chief Executive Officer at Folks Capital, there are currently two broad ways of understanding community ownership. 

The first type involves community ownership in the form of trusts or fiduciary arrangements between a community entity and an independent financial establishment. This structure creates a community entity that holds the financial wealth and is subject to some form of community governance. This structure includes entities such as Community Investment Trusts, Community Land Trusts, and Mixed-Income Neighborhood Trust. These structures ensure permanent and lasting control of the land and fidelity to charitable purposes. However, these entities often do not increase actual ownership or produce meaningful wealth at the individual or family levels. Further, they are often nonprofits and can struggle with attracting capital and sustainability.  

The second type of community ownership is specifically targeted at individuals and families. These are models that focus on financial agency and ownership of land and property by people within communities. This concept includes models such as employee-ownership, Co-operatives, ROC-USA’s model, and Folks Capital’s Neighborhood Equity Model. These models have an advantage in wealth building and agency for the families involved. The benefit of this second concept of  community ownership is that community members have the autonomy to (1) choose to sell their ownership share back to the community fund; (2) receive pro rata (dividend) payments; and/or (3) if the community chooses, sell the enterprise to “would-be gentrifiers.” 

Regardless, the community receives more empowerment than was ever offered by previous economic revitalization models (i.e., Opportunity Zones) [See Table 1]. However these models sometimes lack the permanence or control of the other models. If not structured thoughtfully, this lack of control poses a risk of further gentrification.

Regardless of the approach, all models should seek first to center communities and people in the governance and benefits of the model. Institutionalizing models is not the objective. Closing the wealth gap and ending disparities in economic, health, and education outcomes are the ultimate goal. 

However, an important question is raised by this policy: who counts as community—especially when talking about the ownership of an individual building?

Are multiple communities expected to be consolidated into one community for the sake of ease? Would that be fair to those communities?

The challenge is making ownership meaningful. Understandably, a resident may possess more pride if their stake in an INM is $1000 opposed to 20 cents. 

Thus, communities that are smaller in size may be most benefited by the establishment of an INM. This is not to say that large historically-redlined areas do not stand to gain from INM establishment. Quite the contrary. INMs are designed to not only enfranchise the local communities , but also revitalize the place through restorative, economic, and environmental justice. 

Nevertheless, if PTRIG is to provide communities with tools that guarantee full community empowerment, then factors of community ownership should be considered. 

Now, one final question remains, and it can only be answered by those within historically-redlined communities: “Who is your community?”

Efforts to monitor and combat online harassment have fallen short due to a lack of cooperation and information-sharing across stakeholders, disproportionately hurting women, people of color, and LGBTQ+ individuals. We propose that the White House Task Force to Address Online Harassment and Abuse convene government actors, civil society organizations, and industry representatives to create an Anti-Online Harassment (AOH) Hub to improve and standardize responses to online harassment and to provide evidence-based recommendations to the Task Force. This Hub will include a data-collection mechanism for research and analysis while also connecting survivors with social media companies, law enforcement, legal support, and other necessary resources. This approach will open pathways for survivors to better access the support and recourse they need and also create standardized record-keeping mechanisms that can provide evidence for and enable long-term policy change. 

Challenge and Opportunity 

The online world is rife with hate and harassment, disproportionately hurting women, people of color, and LGBTQ+ individuals. A research study by Pew indicated that 47% of women were harassed online for their gender compared to 18% of men, while 54% of Black or Hispanic internet users faced race-based harassment online compared to 17% of White users. Seven in 10 LGBTQ+ adults have experienced online harassment, and 51% faced even more severe forms of abuse. Meanwhile, existing measures to combat online harassment continue to fall short, leaving victims with limited means for recourse or protection. 

Numerous factors contribute to these shortcomings. Social media companies are opaque, and when survivors turn to platforms for assistance, they are often met with automated responses and few means to appeal or even contact a human representative who could provide more personalized assistance. Many survivors of harassment face threats that escalate from online to real life, leading them to seek help from law enforcement. While most states have laws against cyberbullying, law enforcement agencies are often ill-trained and ill-equipped to navigate the complex web of laws involved and the available processes through which they could provide assistance. And while there are nongovernmental organizations and companies that develop tools and provide services for survivors of online harassment, the onus continues to lie primarily on the survivor to reach out and navigate what is often both an overwhelming and a traumatic landscape of needs. Although resources exist, finding the correct organizations and reaching out can be difficult and time-consuming. Most often, the burden remains on the victims to manage and monitor their own online presence and safety.

On a larger, systemic scale, the lack of available data to quantitatively analyze the scope and extent of online harassment hinders the ability of researchers and interested stakeholders to develop effective, long-term solutions and to hold social media companies accountable. Lack of large-scale, cross-sector and cross-platform data further hinders efforts to map out the exact scale of the issue, as well as provide evidence-based arguments for changes in policy. As the landscape of online abuse is ever changing and evolving, up-to-date information about the lexicons and phrases that are used in attacks also change.

Forming the AOH Hub will improve the collection and monitoring of online harassment while preserving victims’ privacy; this data can also be used to develop future interventions and regulations. In addition, the Hub will streamline the process of receiving aid for those targeted by online harassment.

Plan of Action

Aim of proposal

The White House Task Force to Address Online Harassment and Abuse should form an Anti-Online Harassment Hub to monitor and combat online harassment. This Hub will center around a database that collects and indexes incidents of online harassment and abuse from technology companies’ self-reporting, through connections civil society groups have with survivors of harassment, and from reporting conducted by the general public and by targets of online abuse. Civil society actors that have conducted past work in providing resources and monitoring harassment incidents, ranging from academics to researchers to nonprofits, will run the AOH Hub in consortium as a steering committee. There are two aims for the creation of this hub. 

First, the AOH Hub can promote collaboration within and across sectors, forging bonds among government, the technology sector, civil society, and the general public. This collaboration enables the centralization of connections and resources and brings together diverse resources and expertise to address a multifaceted problem. 

Second, the Hub will include a data collection mechanism that can be used to create a record for policy and other structural reform. At present, the lack of data limits the ability of external actors to evaluate whether social media companies have worked adequately to combat harmful behavior on their platforms. An external data collection mechanism enables further accountability and can build the record for Congress and the Federal Trade Commission to take action where social media companies fall short. The allocated federal funding will be used to (1) facilitate the initial convening of experts across government departments and nonprofit organizations; (2) provide support for the engineering structure required to launch the Hub and database; (3) support the steering committee of civil society actors that will maintain this service; and (4) create training units for law enforcement officials on supporting survivors of online harassment. 

Recommendation 1. Create a committee for governmental departments.

Survivors of online harassment struggle to find recourse, failed by legal technicalities in patchworks of laws across states and untrained law enforcement. The root of the problem is an outdated understanding of the implications and scale of online harassment and a lack of coordination across branches of government on who should handle online harassment and how to properly address such occurrences. A crucial first step is to examine and address these existing gaps. The Task Force should form a long-term committee of members across governmental departments whose work pertains to online harassment. This would include one person from each of the following organizations, nominated by senior staff:

• Department of Homeland Security
• Department of Justice
• Federal Bureau of Investigation
• Department of Health and Human Services
• Office on Violence Against Women
• Federal Trade Commission

This committee will be responsible for outlining fallibilities in the existing system and detailing the kind of information needed to fill those gaps. Then, the committee will outline a framework clearly establishing the recourse options available to harassment victims and the kinds of data collection required to prove a case of harassment. The framework should be completed within the first 6 months after the committee has been convened. After that, the committee will convene twice a year to determine how well the framework is working and, in the long term, implement reforms and updates to current laws and processes to increase the success rates of victims seeking assistance from governmental agencies. 

Recommendation 2: Establish a committee for civil society organizations.

The Task Force shall also convene civil society organizations to help form the AOH Hub steering committee and gather a centralized set of resources. Victims will be able to access a centralized hotline and information page, and Hub personnel will then triage reports and direct victims to resources most helpful for their particular situation. This should reduce the burden on those who are targets of harassment campaigns to find the appropriate organizations that can help address their issues by matching incidents to appropriate resources. 

To create the AOH Hub, members of the Task Force can map out civil society stakeholders in the space and solicit applications to achieve comprehensive and equitable representation across sectors. Relevant organizations include organizations/actors working on (but not limited to):

• Combating domestic violence and intimate partner violence
• Addressing technology-facilitated gender based violence (TF-GBV)
• Developing online tools for survivors of harassment to protect themselves
• Conducting policy work to improve policies on harassment
• Providing mental health support for survivors of harassment
• Servicing pro bono or other forms of legal assistance for survivors of harassment
• Connecting tech company representatives with survivors of harassment
• Researching methods to address online harassment and abuse

The Task Force will convene an initial meeting, during which core members will be selected to create an advisory board, act as a liaison across members, and conduct hiring for the personnel needed to redirect victims to needed services. Other secondary members will take part in collaboratively mapping out and sharing available resources, in order to understand where efforts overlap and complement each other. These resources will be consolidated, reviewed, and published as a public database of resources within a year of the group’s formation. 

For secondary members, their primary obligation will be to connect with victims who have been recommended to their services. Core members, meanwhile, will meet quarterly to evaluate gaps in services and assistance provided and examine what more needs to be done to continue growing the robustness of services and aid provided. 

Recommendation 3: Convene committee for industry.

After its formation, the AOH steering committee will be responsible for conducting outreach with industry partners to identify a designated team from each company best equipped to address issues pertaining to online abuse. After the first year of formation, the industry committee will provide operational reporting on existing measures within each company to address online harassment and examine gaps in existing approaches. Committee dialogue should also aim to create standardized responses to harassment incidents across industry actors and understandings of how to best uphold community guidelines and terms of service. This reporting will also create a framework for standardized best practices for data collection, in terms of the information collected on flagged cases of online harassment.

On a day-to-day basis, industry teams will be available resources for the hub, and cases can be redirected to these teams to provide person-to-person support for handling cases of harassment that require a personalized level of assistance and scale. This committee will aim to increase transparency regarding the reporting process and improve equity in responses to online harassment.

Recommendation 4: Gather committees to provide long-term recommendations for policy change.

On a yearly basis, representatives across the three committees will convene and share insights on existing measures and takeaways. These recommendations will be given to the Task Force and other relevant stakeholders, as well as be accessible by the general public. Three years after the formation of these committees, the groups will publish a report centralizing feedback and takeaway from all committees, and provide recommendations of improvement for moving forward. 

Recommendation 5: Create a data-collection mechanism and standard reporting procedures.

The database will be run and maintained by the steering committee with support from the U.S. Digital Service, with funding from the Task Force for its initial development. The data collection mechanism will be informed by the frameworks provided by the committees that compose the Hub to create a trauma-informed and victim-centered framework surrounding the collection, protection, and use of the contained data. The database will be periodically reviewed by the steering committee to ensure that the nature and scope of data collection is necessary and respects the privacy of those whose data it contains. Stakeholders can use this data to analyze and provide evidence of the scale and cross-cutting nature of online harassment and abuse. The database would be populated using a standardized reporting form containing (1) details of the incident; (2) basic demographic data of the victim; (3) platform/means through which the incident occurred; (4) whether it is part of a larger organized campaign; (5) current status of the incident (e.g., whether a message was taken down, an account was suspended, the report is still ongoing); (6) categorization within existing proposed taxonomies indicating the type of abuse. This standardization of data collection would allow advocates to build cases regarding structured campaigns of abuse with well-documented evidence, and the database will archive and collect data across incidents to ensure accountability even if the originals are lost or removed.

The reporting form will be available online through the AOH Hub. Anyone with evidence of online harassment will be able to contribute to the database, including but not limited to victims of abuse, bystanders, researchers, civil society organizations, and platforms. To protect the privacy and safety of targets of harassment, this data will not be publicly available. Access will be limited to: (1) members of the Hub and its committees; (2) affiliates of the aforementioned members; (3) researchers and other stakeholders, after submitting an application stating reasons to access the data, plans for data use, and plans for maintaining data privacy and security. Published reports using data from this database will be nonidentifiable, such as with statistics being published in aggregate, and not be able to be linked back to individuals without express consent.

This database is intended to provide data to inform the committees in and partners of the Hub of the existing landscape of technology-facilitated abuse and violence. The large-scale, cross-domain, and cross-platform nature of the data collected will allow for better understanding and analysis of trends that may not be clear when analyzing specific incidents, and provide evidence regarding disproportionate harms to particular communities (such as women, people of color, LGBTQ+ individuals). Resources permitting, the Hub could also survey those who have been impacted by online abuse and harassment to better understand the needs of victims and survivors. This data aims to provide evidence for and help inform the recommendations made from the committees to the Task Force for policy change and further interventions.

Recommendation 6: Improve law enforcement support.

Law enforcement is often ill-equipped to handle issues of technology-facilitated abuse and violence. To address this, Congress should allocate funding for the Hub to create training materials for law enforcement nationwide. The developed materials will be added to training manuals and modules nationwide, to ensure that 911 operators and officers are aware of how to handle cases of online harassment and how state and federal law can apply to a range of scenarios. As part of the training, operators will also be notified to add records of 911 calls regarding online harassment to the Hub database, with the survivor’s consent. 

Conclusion

As technology-facilitated violence and abuse proliferates, we call for funding to create a steering committee in which experts and stakeholders from civil society, academia, industry, and government can collaborate on monitoring and regulating online harassment across sectors and incidents. The resulting Anti-Online Harassment Hub would maintain a data-collection mechanism accessible to researchers to better understand online harassment as well as provide accountability for social media platforms to address the issue. Finally, the Hub would provide accessible resources for targets of harassment in a fashion that would reduce the burden on these individuals. Implementing these measures would create a safer online space where survivors are able to easily access the support they need and establish a basis for evidence-based, longer-term policy change.

To increase protections for fair work, the U.S. Department of Labor (DOL) should create an Office of the Ombudsman for Fair Work. Gig workers are a category of non-employee contract workers who engage in on-demand work, often through online platforms. They have had historic vulnerabilities in the U.S. economy. A large portion of gig workers are people of color, and the nature of their temporary and largely unregulated work can leave them vulnerable to economic instability and workplace abuse. Currently, there is no federal mechanism to protect gig workers, and state-level initiatives have not offered thorough enough policy redress. Establishing an Office of the Ombudsman would provide the Department of Labor with a central entity to investigate worker complaints against gig employers, collect data and evidence about the current gig economy, and provide education to gig workers about their rights. There is strong precedent for this policy solution, since bureaus across the federal government have successfully implemented ombudsmen that are independent and support vulnerable constituents. To ensure its legal and long-lasting status, the Secretary of Labor should establish this Office in an act of internal agency reorganization.

Challenge and Opportunity

The proportion of the U.S. workforce engaging in gig work has risen steadily in the past few decades, from 10.1% in 2005 to 15.8% in 2015 to roughly 20% in 2018. Since the COVID-19 pandemic began, this trend has only accelerated, and a record number of Americans have now joined the gig economy and rely on its income. In a 2021 Pew Research study, over 16% of Americans reported having made money through online platform work alone, such as on apps like Uber and Doordash, which is merely a subset of gig work jobs. Gig workers in particular are more likely to be Black or Latino compared to the overall workforce.

Though millions of Americans rely on gig work, it does not provide critical employee benefits, such as minimum wage guarantees, parental leave, healthcare, overtime, unemployment insurance, or recourse for injuries incurred during work. According to an NPR survey, in 2018 more than half of contract workers received zero benefits through work. Further, the National Labor Relations Act, which protects employees’ rights to unionize and collectively bargain without retaliation, does not protect gig workers. This lack of benefits, rights, and voice leaves millions of workers more vulnerable than full-time employees to predatory employers, financial instability, and health crises, particularly during emergencies—such as the COVID-19 pandemic. 

Additionally, in 2022, inflation reached a decades-long high, and though the price of necessities has spiked, wages have not increased correspondingly. Extreme inflation hurts lower-income workers without savings the most and is especially dangerous to gig workers, some of whom make less than the federal minimum hourly wage and whose income and work are subject to constant flux.

State-level measures have as yet failed to create protections for all gig workers. In 2020, California passed AB5, legally reclassifying many gig workers as employees instead of independent contractors and thus entitling them to more benefits and protections. But further bills and Proposition 22 reverted several groups of gig workers, including online platform gig workers like Uber and Doordash drivers, to being independent contractors. Ongoing litigation related to Proposition 22 leaves the future status of online platform gig workers in California unclear. In 2022, Washington State passed ESHB 2076 guaranteeing online platform workers—but not all gig workers—the benefits of full-time employees. 

This sparse patchwork of state-level measures, which only supports subgroups of gig workers, could trigger a “race to the bottom” in which employers of gig workers relocate to less strict states. Additionally, inconsistencies between state laws make it harder for gig workers to understand their rights and gain redress for grievances, harder for businesses to determine with certainty their duties and liabilities, and harder for states to enforce penalties when an employer is headquartered in one state and the gig worker lives in another. The status quo is also difficult for businesses that strive to be better employers because it creates downward pressure on the entire landscape of labor market competition. Ultimately, only federal policy action can fully address these inconsistencies and broadly increase protections and benefits for all gig workers. 

The federal ombudsman’s office outlined in this proposal can serve as a resource for gig workers to understand the scope of their current rights, provide a voice to amplify their grievances and harms, and collect data and evidence to inform policy proposals. It is the first step toward a sustainable and comprehensive national solution that expands the rights of gig workers.

Specifically, clarifying what rights, benefits, and means of recourse gig workers do and do not have would help gig workers better plan for healthcare and other emergent needs. It would also allow better tracking of trends in the labor market and systemic detection of employee misclassification. Hearing gig workers’ complaints in a centralized office can help the Department of Labor more expeditiously address gig workers’ concerns in situations where they legally do have recourse and can otherwise help the Department of Labor better understand the needs of and harms experienced by all workers. Collecting broad-ranging data on gig workers in particular could help inform federal policy change on their rights and protections. Currently, most datasets are survey based and often leave out people who were not working a gig job at the time the survey was conducted but typically otherwise do. More broadly, because of its informal and dynamic nature, the gig economy is difficult to accurately count and characterize, and an entity that is specifically charged with coordinating and understanding this growing sector of the market is key.

Lastly, employees who are not gig workers are sometimes misclassified as such and thus lose out on benefits and protections they are legally entitled to. Having a centralized ombudsman office dedicated to gig work could expedite support of gig workers seeking to correct their classification status, which the Wage and Hour Division already generally deals with, as well as help the Department of Labor and other agencies collect data to clarify the scope of the problem.

Plan of Action

The Department of Labor should establish an Office of the Ombudsman for Fair Work. This office should be independent of Department of Labor agencies and officials, and it should report directly to the Secretary of Labor. The Office would operate on a federal level with authority over states.

The Secretary of Labor should establish the Office in an act of internal agency reorganization. By establishing the Office such that its powers do not contradict the Department of Labor’s statutory limitations, the Secretary can ensure the Office’s status as legal and long-lasting, due to the discretionary power of the Department to interpret its statutes.

The role of the Office of the Ombudsman for Fair Work would be threefold: to serve as a centralized point of contact for hearing complaints from gig workers; to act as a central resource and conduct outreach to gig workers about their rights and protections; and to collect data such as demographic, wage, and benefit trends on the labor practices of the gig economy. Together, these responsibilities ensure that this Office consolidates and augments the actions of the Department of Labor as they pertain to workers in the gig economy, regardless of their classification status.

The functions of the ombudsman should be as follows:

1. Establish a clear and centralized mechanism for hearing, collating, and investigating complaints from workers in the gig economy, such as through a helpline or mobile app.
2. Establish and administer an independent, neutral, and confidential process to receive, investigate, resolve, and provide redress for cases in which employers misrepresent to individuals that they are engaged as independent contractors when they’re actually engaged as employees.
3. Commence court proceedings to enforce fair work practices and entitlements, as they pertain to workers in the gig economy, in conjunction with other offices in the DOL.
4. Represent employees or contractors who are or may become a party to proceedings in court over unfair contracting practices, including but not limited to misclassification as independent contractors. The office would refer matters to interagency partners within the Department of Labor and across other organizations engaged in these proceedings, augmenting existing work where possible.
5. Provide education, assistance, and advice to employees, employers, and organizations, including best practice guides to workplace relations or workplace practices and information about rights and protections for workers in the gig economy.
6. Conduct outreach in multiple languages to gig economy workers informing them of their rights and protections and of the Office’s role to hear and address their complaints and entitlements.
7. Serve as the central data collection and publication office for all gig-work-related data. The Office will publish a yearly report detailing demographic, wage, and benefit trends faced by gig workers. Data could be collected through outreach to gig workers or their employers, or through a new data-sharing agreement with the Internal Revenue Service (IRS). This data report would also summarize anonymized trends based on the complaints collected (as per function 1), including aggregate statistics on wage theft, reports of harassment or discrimination, and misclassification. These trends would also be broken down by demographic group to proactively identify salient inequities. The office may also provide separate data on platform workers, which may be easier to collect and collate, since platform workers are a particular subject of focus in current state legislation and litigation.

Establishing an Office of the Ombudsman for Fair Work within the Department of Labor will require costs of compensation for the ombudsman and staff, other operational costs, and litigation expenses. To reflect the need for a reaction to the rapid ongoing changes in gig economy platforms, a small portion of the Office’s budget should be set aside to support the appointment of a chief innovation officer, aimed at examining how technology can strengthen its operations. Some examples of tasks for this role include investigating and strengthening complaint sorting infrastructure, utilizing artificial intelligence to evaluate contracts for misclassification, and streamlining request for proposal processes.

Due to the continued growth of the gig economy, and the precarious status of gig workers in the onset of an economic recession, this Office should be established in the nearest possible window. Establishing, appointing, and initiating this office will require up to a year of time, and will require budgeting within the DOL.

There are many precedents of ombudsmen in federal office, including the Office of the Ombudsman for the Energy Employees Occupational Illness Compensation Program within the Department of Labor. Additionally, the IRS established the Office of the Taxpayer Advocate, and the Department of Homeland Security has both a Citizenship and Immigration Services Ombudsman and an Immigration Detention Ombudsman. These offices have helped educate constituents about their rights, resolved issues that an individual might have with that federal agency, and served as independent oversight bodies. The Australian Government has a Fair Work Ombudsman that provides resources to differentiate between an independent contractor and employee and investigates employers who may be engaging in sham contracting or other illegal practices. Following these examples, the Office of the Ombudsman for Fair Work should work within the Department of Labor to educate, assist, and provide redress for workers engaged in the gig economy.

Conclusion

How to protect gig workers is a long-standing open question for labor policy and is likely to require more attention as post-pandemic conditions affect labor trends. The federal government needs a solution to the issues of vulnerability and instability experienced by gig workers, and this solution needs to operate independently of legislation that may take longer to gain consensus on. Establishing an office of an ombudsman is the first step to increase federal oversight for gig work. The ombudsman will use data, reporting, and individual worker cases to build a clearer picture for how to create redress for laborers that have been harmed by gig work, which will provide greater visibility into the status and concerns of gig workers. It will additionally serve as a single point of entry for gig workers and businesses to learn about their rights and for gig workers to lodge complaints. If made a reality, this office will be an influential first step in changing the entire policy ecosystem regarding gig work. 

Maternal mortality is a crisis in the United States. Yet more than 60% of maternal deaths are preventable—with the right evidence-based interventions. Data is a powerful tool for uncovering best care practices. While healthcare data, including maternal health data, has been generated at a massive scale by the widespread adoption and use of Electronic Health Records (EHR), much of this data remains unstandardized and unanalyzed. Further, while many federal datasets related to maternal health are openly available through initiatives set forth in the Open Government National Action Plan, there is no central coordinating body charged with analyzing this breadth of data. Advancing data harmonization, research, and analysis are foundational elements of the Biden Administration’s Blueprint for Addressing the Maternal Health Crisis. As a data-driven technology, artificial intelligence (AI) has great potential to support maternal health research efforts. Examples of promising applications of AI include using electronic health data to predict whether expectant mothers are at risk of difficulty during delivery. However, further research is needed to understand how to effectively implement this technology in a way that promotes transparency, safety, and equity. The Biden-Harris Administration should establish an AI Center of Excellence to bring together data sources and then analyze, diagnose, and address maternal health disparities, all while demonstrating trustworthy and responsible AI principles.  

Challenge and Opportunity

Maternal deaths currently average around 700 per year, and severe maternal morbidity-related conditions impact upward of 60,000 women annually. Stark maternal health disparities persist in the United States, and pregnancy outcomes are subject to substantial racial/ethnic disparities, including maternal morbidity and mortality. According to the Centers for Disease Control and Prevention (CDC), “Black women are three times more likely to die from a pregnancy-related cause than White women.” Research is ongoing to specifically identify the root causes, which include socioeconomic factors such as insurance status, access to healthcare services, and risks associated with social determinants of health. For example, maternity care deserts exist in counties throughout the country where maternal health services are substantially limited or not available, impacting an estimated 2.2 million women of child-bearing age.

Many federal, public, and private datasets exist to understand the conditions that impact pregnant people, the quality of the care they receive, and ultimate care outcomes. For example, the CDC collects abundant data on maternal health, including the Pregnancy Mortality Surveillance System (PMSS) and the National Vital Statistics System (NVSS). Many of these datasets, however, have yet to be analyzed at scale or linked to other federal or privately held data sources in a comprehensive way. More broadly, an estimated 30% of the data generated globally is produced by the healthcare industry. AI is uniquely designed for data management, including cataloging, classification, and data integration. AI will play a pivotal role in the federal government’s ability to process an unprecedented volume of data to generate evidence-based recommendations to improve maternal health outcomes. 

Applications of AI have rapidly proliferated throughout the healthcare sector due to their potential to reduce healthcare expenditures and improve patient outcomes (Figure 1). Several applications of this technology exist across the maternal health continuum and are shown in the figure below. For example, evidence suggests that AI can help clinicians identify more than 70% of at-risk moms during the first trimester by analyzing patient data and identifying patterns associated with poor health outcomes. Based on its findings, AI can provide recommendations for which patients will most likely be at-risk for pregnancy challenges before they occur. Research has also demonstrated the use of AI in fetal health monitoring. 

Figure 1: Areas Where Artificial Intelligence and Machine Learning Is Used for Women’s Reproductive Health

Source: Davidson and Boland (2021). 

Yet for all of AI’s potential, there is a significant dearth of consumer and medical provider understanding of how these algorithms work. Policy analysts argue that “algorithmic discrimination” and feedback loops in algorithms—which may exacerbate algorithmic bias—are potential risks of using AI in healthcare outside of the confines of an ethical framework. In response, certain federal entities such as the Department of Defense, the Office of the Director of National Intelligence, the National Institute for Standards and Technology, and the U.S. Department of Health and Human Services have published and adopted guidelines for implementing data privacy practices and building public trust of AI. Further, past Day One authors have proposed the establishment of testbeds for government-procured AI models to provide services to U.S. citizens. This is critical for enhancing the safety and reliability of AI systems while reducing the risk of perpetuating existing structural inequities. 

It is vital to demonstrate safe, trustworthy uses of AI and measure the efficacy of these best practices through applications of AI to real-world societal challenges. For example, potential use cases of AI for maternal health include a social determinants of health [SDoH] extractor, which combines AI with clinical notes to more effectively identify SDoH information and analyze its potential role in health inequities. A center dedicated to ethically developing AI for maternal health would allow for the development of evidence-based guidelines for broader AI implementation across healthcare systems throughout the country. Lessons learned from this effort will contribute to the knowledge base around ethical AI and enable development of AI solutions for health disparities more broadly. 

Plan of Action

To meet the calls for advancing data collection, standardization, transparency, research, and analysis to address the maternal health crisis, the Biden-Harris Administration should establish an AI Center of Excellence for maternal health. The AI Center of Excellence for Maternal Health will bring together data sources, then analyze, diagnose, and address maternal health disparities, all while demonstrating trustworthy and responsible AI principles. The Center should be created within the Department of Health and Human Services (HHS) and work closely with relevant offices throughout HHS and beyond, including the HHS Office of the Chief Artificial Intelligence Officer (OCAIO), the National Institutes of Health (NIH) IMPROVE initiative, the CDC, the Veterans Health Administration (VHA), and the National Institute for Standards and Technology (NIST). The Center should offer competitive salaries to recruit the best and brightest talent in AI, human-centered design, biostatistics, and human-computer interaction.

The first priority should be to work with all agencies tasked by the White House Blueprint for Addressing the Maternal Health Crisis to collect and evaluate data. This includes privately held EHR data that is made available through the Qualified Health Information Network (QHIN) and federal data from the CDC, Centers for Medicare and Medicaid (CMS), Office of Personnel Management (OPM), Healthcare Resources and Services Agency (HRSA), NIH, United States Department of Agriculture (USDA), Housing and Urban Development (HUD), the Veterans Health Administration, and Environmental Protection Agency (EPA), all of which contain datasets relevant to maternal health at different stages of the reproductive health journey from Figure 1. The Center should serve as a data clearing and cleaning shop, preparing these datasets using best practices for data management, preparation, and labeling.

The second priority should be to evaluate existing datasets to establish high-priority, high-impact applications of AI-enabled research for improving clinical care guidelines and tools for maternal healthcare providers. These AI demonstrations should be aligned with the White House’s Action Plan and be focused on implementing best practices for AI development, such as the AI Risk Management Framework developed by NIST. The following examples demonstrate how AI might help address maternal health disparities, based on priority areas informed by clinicians in the field:   

1. AI implementation should be explored for analysis of electronic health records from the VHA and QHIN to predict patients who have a higher risk of pregnancy and/or delivery complications. 
2. Drawing on the robust data collection and patient surveillance capabilities of the VHA and HRSA, AI should be explored for the deployment of digital tools to help monitor patients during pregnancy to ensure adequate and consistent use of prenatal care.  
3. Using VHA data and QHIN data, AI should be explored in supporting patient monitoring in instances of patient referrals and/or transfers to hospitals that are appropriately equipped to serve high-risk patients, following guidelines provided by the American College of Obstetricians and Gynecologists.
4. Data on housing from HUD, rural development from the USDA, environmental health from the EPA, and social determinants of health research from the CDC should be connected to risk factors for maternal mortality in the academic literature to create an AI-powered risk algorithm.
5. Understand the power of payment models operated by CMS and OPM for novel strategies to enhance maternal health outcomes and reduce maternal deaths.

The final priority should be direct translation of the findings from AI to federal policymaking around reducing maternal health disparities as well as ethical development of AI tools. Research findings for both aspects of this interdisciplinary initiative should be framed using Living Evidence models that help ensure that research-derived evidence and guidance remain current.

The Center should be able to meet the following objectives within the first year after creation to further the case for future federal funding and creation of more AI Centers of Excellence for healthcare:

1. Conduct a study on the use cases uncovered for AI to help address maternal health disparities explored through the various demonstration projects.
2. Publish a report of study findings, which should be submitted to Congress with recommendations to help inform funding priorities for subsequent research activities.
3. Make study findings available to the public to help build public trust in AI.

Successful piloting of the Center could be made possible by passage of an equivalent bill to S.893 in the current Congress. This is a critical first step in supporting this work. In March 2021, the S.893—Tech to Save Moms Act was introduced in the Senate to fund research conducted by National Academies of Sciences, Engineering, and Medicine to understand the role of AI in maternal care delivery and its impact on bias in maternal health. Passage of an equivalent bill into law would enable the National Academies of Sciences, Engineering, and Medicine to conduct research in parallel with HHS to generate more findings and to broaden potential impact.

Conclusion

The United States has the highest rate of maternal health disparities among all developed countries. Yet more than 60% of pregnancy-related deaths are preventable, highlighting a critical opportunity to uncover the factors impeding more equitable health outcomes for the nation as a whole. Legislative support for research to understand AI’s role in addressing maternal health disparities will affirm the nation’s commitment to ensuring that we are prepared to thrive in a 21^st century influenced and shaped by next-generation technologies such as artificial intelligence.

The unregulated use of algorithmic decision-making systems (ADS)—systems that crunch large amounts of personal data and derive relationships between data points—has negatively affected millions of Americans. These systems impact equitable access to education, housing, employment, and healthcare, with life-altering effects. For example, commercial algorithms used to guide health decisions for approximately 200 million people in the United States each year were found to systematically discriminate against Black patients, reducing, by more than half, the number of Black patients who were identified as needing extra care.

One way to combat algorithmic harm is by conducting system audits, yet there are currently no standards for auditing AI systems at the scale necessary to ensure that they operate legally, safely, and in the public interest. According to one research study examining the ecosystem of AI audits, only one percent of AI auditors believe that current regulation is sufficient. 

To address this problem, the National Institute of Standards and Technology (NIST) should invest in the development of comprehensive AI auditing tools, and federal agencies with the charge of protecting civil rights and liberties should collaborate with NIST to develop these tools and push for comprehensive system audits. 

These auditing tools would help the enforcement arms of these federal agencies save time and money while fulfilling their statutory duties. Additionally, there is a pressing need to develop these tools now, with Executive Order 13985 instructing agencies to “focus their civil rights authorities and offices on emerging threats, such as algorithmic discrimination in automated technology.”

Challenge and Opportunity

The use of AI systems across all aspects of life has become commonplace as a way to improve decision-making and automate routine tasks. However, their unchecked use can perpetuate historical inequities, such as discrimination and bias, while also potentially violating American civil rights.

Algorithmic decision-making systems are often used in prioritization, classification, association, and filtering tasks in a way that is heavily automated. ADS become a threat when people uncritically rely on the outputs of a system, use them as a replacement for human decision-making, or use systems with no knowledge of how they were developed. These systems, while extremely useful and cost-saving in many circumstances, must be created in a way that is equitable and secure. 

Ensuring the legal and safe use of ADS begins with recognizing the challenges that the federal government faces. On the one hand, the government wants to avoid devoting excessive resources to managing these systems. With new AI system releases happening everyday, it is becoming unreasonable to oversee every system closely. On the other hand, we cannot blindly trust all developers and users to make appropriate choices with ADS.

This is where tools for the AI development lifecycle come into play, offering a third alternative between constant monitoring and blind trust. By implementing auditing tools and signatory practices, AI developers will be able to demonstrate compliance with preexisting and well-defined standards while enhancing the security and equity of their systems. 

Due to the extensive scope and diverse applications of AI systems, it would be difficult for the government to create a centralized body to oversee all systems or demand each agency develop solutions on its own. Instead, some responsibility should be shifted to AI developers and users, as they possess the specialized knowledge and motivation to maintain proper functioning systems. This allows the enforcement arms of federal agencies tasked with protecting the public to focus on what they do best, safeguarding citizens’ civil rights and liberties.

Plan of Action

To ensure security and verification throughout the AI development lifecycle, a suite of auditing tools is necessary. These tools should help enable outcomes we care about, fairness, equity, and legality. The results of these audits should be reported (for example, in an immutable ledger that is only accessible by authorized developers and enforcement bodies) or through a verifiable code-signing mechanism. We leave the specifics of the reporting and documenting the process to the stakeholders involved, as each agency may have different reporting structures and needs. Other possible options, such as manual audits or audits conducted without the use of tools, may not provide the same level of efficiency, scalability, transparency, accuracy, or security.

The federal government’s role is to provide the necessary tools and processes for self-regulatory practices. Heavy-handed regulations or excessive government oversight are not well-received in the tech industry, which argues that they tend to stifle innovation and competition. AI developers also have concerns about safeguarding their proprietary information and users’ personal data, particularly in light of data protection laws.

Auditing tools provide a solution to this challenge by enabling AI developers to share and report information in a transparent manner while still protecting sensitive information. This allows for a balance between transparency and privacy, providing the necessary trust for a self-regulating ecosystem.

Solution Technical Requirements

A general machine learning lifecycle. Examples of what system developers at each stage would be responsible for signing off on the use of the security and equity tools in the lifecycle. These developers represent companies, teams, or individuals.

The equity tool and process, funded and developed by government agencies such as NIST, would consist of a combination of (1) AI auditing tools for security and fairness (which could be based on or incorporate open source tools such as AI Fairness 360 and the Adversarial Robustness Toolbox), and (2) a standardized process and guidance for integrating these checks (which could be based on or incorperate guidance such as the U.S. Government Accountability Office’s  Artificial Intelligence: An Accountability Framework for Federal Agencies and Other Entities).¹ 

Dioptra, a recent effort between NIST and the National Cybersecurity Center of Excellence (NCCoE) to build machine learning testbeds for security and robustness, is an excellent example of the type of lifecycle management application that would ideally be developed. Failure to protect civil rights and ensure equitable outcomes must be treated as seriously as security flaws, as both impact our national security and quality of life. 

Equity considerations should be applied across the entire lifecycle; training data is not the only possible source of problems. Inappropriate data handling, model selection, algorithm design, and deployment, also contribute to unjust outcomes. This is why tools combined with specific guidance is essential. 

As some scholars note, “There is currently no available general and comparative guidance on which tool is useful or appropriate for which purpose or audience. This limits the accessibility and usability of the toolkits and results in a risk that a practitioner would select a sub-optimal or inappropriate tool for their use case, or simply use the first one found without being conscious of the approach they are selecting over others.”

Companies utilizing the various packaged tools on their ADS could sign off on the results using code signing. This would create a record that these organizations ran these audits along their development lifecycle and received satisfactory outcomes. 

We envision a suite of auditing tools, each tool applying to a specific agency and enforcement task. Precedents for this type of technology already exist. Much like security became a part of the software development lifecycle with guidance developed by NIST, equity and fairness should be integrated into the AI lifecycle as well. NIST could spearhead a government-wide initiative on auditing AI tools, leading guidance, distribution, and maintenance of such tools. NIST is an appropriate choice considering its history of evaluating technology and providing guidance around the development and use of specific AI applications such as the NIST-led Face Recognition Vendor Test (FRVT).

Areas of Impact & Agencies / Departments Involved

Security & Justice
The U.S. Department of Justice, Civil Rights Division, Special Litigation SectionDepartment of Homeland Security U.S. Customs and Border Protection U.S. Marshals Service 

Public & Social Sector
The U.S. Department of Housing and Urban Development’s Office of Fair Housing and Equal Opportunity

Education
The U.S. Department of Education

Environment
The U.S. Department of Agriculture, Office of the Assistant Secretary for Civil RightsThe Federal Energy Regulatory CommissionThe Environmental Protection Agency

Crisis Response
Federal Emergency Management Agency 

Health & Hunger
The U.S. Department of Health and Human Services, Office for Civil RightsCenter for Disease Control and PreventionThe Food and Drug Administration

Economic
The Equal Employment Opportunity Commission, The U.S. Department of Labor, Office of Federal Contract Compliance Programs

Infrastructure
The U.S. Department of Transportation, Office of Civil RightsThe Federal Aviation AdministrationThe Federal Highway Administration

Information Verification & Validation
The Federal Trade Commission, The Federal Communication Commission, The Securities and Exchange Commission.

Many of these tools are open source and free to the public. A first step could be combining these tools with agency-specific standards and plain language explanations of their implementation process.

Benefits

These tools would provide several benefits to federal agencies and developers alike. First, they allow organizations to protect their data and proprietary information while performing audits. Any audits, whether on the data, model, or overall outcomes, would be run and reported by the developers themselves. Developers of these systems are the best choice for this task since ADS applications vary widely, and the particular audits needed depend on the application. 

Second, while many developers may opt to use these tools voluntarily, standardizing and mandating their use would allow an evaluation of any system thought to be in violation of the law to be easily assesed. In this way, the federal government will be able to manage standards more efficiently and effectively.

Third, although this tool would be designed for the AI lifecycle that results in ADS, it can also be applied to traditional auditing processes. Metrics and evaluation criteria will need to be developed based on existing legal standards and evaluation processes; once these metrics are distilled for incorporation into a specific tool, this tool can be applied to non-ADS data as well, such as outcomes or final metrics from traditional audits.

Fourth, we believe that a strong signal from the government that equity considerations in ADS are important and easily enforceable will impact AI applications more broadly, normalizing these considerations.   

Example of Opportunity

An agency that might use this tool is the Department of Housing and Urban Development (HUD), whose purpose is to ensure that housing providers do not discriminate based on race, color, religion, national origin, sex, familial status, or disability.

To enforce these standards, HUD, which is responsible for 21,000 audits a year, investigates and audits housing providers to assess compliance with the Fair Housing Act, the Equal Credit Opportunity Act, and other related regulations. During these audits, HUD may review a provider’s policies, procedures, and records, as well as conduct on-site inspections and tests to determine compliance. 

Using an AI auditing tool could streamline and enhance HUD’s auditing processes. In cases where ADS were used and suspected of harm, HUD could ask for verification that an auditing process was completed and specific metrics were met, or require that such a process be undergone and reported to them. 

Noncompliance with legal standards of nondiscrimination would apply to ADS developers as well, and we envision the enforcement arms of protection agencies would apply the same penalties in these situations as they would in non-ADS cases.

R&D

To make this approach feasible, NIST will require funding and policy support to implement this plan. The recent CHIPS and Science Act has provisions to support NIST’s role in developing “trustworthy artificial intelligence and data science,” including the testbeds mentioned above. Research and development can be partially contracted out to universities and other national laboratories or through partnerships/contracts with private companies and organizations.

The first iterations will need to be developed in partnership with an agency interested in integrating an auditing tool into its processes. The specific tools and guidance developed by NIST must be applicable to each agency’s use case. 

The auditing process would include auditing data, models, and other information vital to understanding a system’s impact and use, informed by existing regulations/guidelines. If a system is found to be noncompliant, the enforcement agency has the authority to impose penalties or require changes to be made to the system.

Pilot program

NIST should develop a pilot program to test the feasibility of AI auditing. It should be conducted on a smaller group of systems to test the effectiveness of the AI auditing tools and guidance and to identify any potential issues or areas for improvement. NIST should use the results of the pilot program to inform the development of standards and guidelines for AI auditing moving forward.

Collaborative efforts

Achieving a self-regulating ecosystem requires collaboration. The federal government should work with industry experts and stakeholders to develop the necessary tools and practices for self-regulation.

A multistakeholder team from NIST, federal agency issue experts, and ADS developers should be established during the development and testing of the tools. Collaborative efforts will help delineate responsibilities, with AI creators and users responsible for implementing and maintaining compliance with the standards and guidelines, and agency enforcement arms agency responsible for ensuring continued compliance.

Regular monitoring and updates

The enforcement agencies will continuously monitor and update the standards and guidelines to keep them up to date with the latest advancements and to ensure that AI systems continue to meet the legal and ethical standards set forth by the government.

Transparency and record-keeping

Code-signing technology can be used to provide transparency and record-keeping for ADS. This can be used to store information on the auditing outcomes of the ADS, making reporting easy and verifiable and providing a level of accountability to users of these systems.

Conclusion

Creating auditing tools for ADS presents a significant opportunity to enhance equity, transparency, accountability, and compliance with legal and ethical standards. The federal government can play a crucial role in this effort by investing in the research and development of tools, developing guidelines, gathering stakeholders, and enforcing compliance. By taking these steps, the government can help ensure that ADS are developed and used in a manner that is safe, fair, and equitable.

Black, Indigenous, and other people of color (BIPOC) are underrepresented in labor unions. Further, people working in the gig economy, tech supply chain, and other automation-adjacent roles face a huge barrier to unionizing their workplaces. These roles, which are among the fastest-growing segments of the U.S. economy, are overwhelmingly filled by BIPOC workers. In the absence of safety nets for these workers, the racial wealth gap will continue to grow. The Biden-Harris Administration can promote racial equity and support low-wage BIPOC workers’ unionization efforts by creating a National Bargaining in Good Faith Fund.

As a whole, unions lift up workers to a better standard of living, but historically they have failed to protect workers of color. The emergence of labor unions in the early 20th century was propelled by the passing of the National Labor Relations Act (NLRA), also known as the Wagner Act of 1935. Although the NLRA was a beacon of light for many working Americans, affording them the benefits of union membership such as higher wages, job security, and better working conditions, which allowed many to transition into the middle class, the protections of the law were not applied to all working people equally. Labor unions in the 20th century were often segregated, and BIPOC workers were often excluded from the benefits of unionization. For example, the Wagner Act excluded domestic and agricultural workers and permitted labor unions to discriminate against workers of color in other industries, such as manufacturing. 

Today, in the aftermath of the COVID-19 pandemic and amid a renewed interest in a racial reckoning in the United States, BIPOC workers—notably young and women BIPOC workers—are leading efforts to organize their workplaces. In addition to demanding wage equity and fair treatment, they are also fighting for health and safety on the job. Unionized workers earn on average 11.2% more in wages than their nonunionized peers. Unionized Black workers earn 13.7% more and unionized Hispanic workers 20.1% more than their nonunionized peers. But every step of the way, tech giants and multinational corporations are opposing workers’ efforts and their legal right to organize, making organizing a risky undertaking.

A National Bargaining in Good Faith Fund would provide immediate and direct financial assistance to workers who have been retaliated against for attempting to unionize, especially those from historically disadvantaged groups in the United States. This fund offers a simple and effective solution to alleviate financial hardships, allowing affected workers to use the funds for pressing needs such as rent, food, or job training. It is crucial that we advance racial equity, and this fund is one step toward achieving that goal by providing temporary financial support to workers during their time of need. Policymakers should support this initiative as it offers direct payments to workers who have faced illegal retaliation, providing a lifeline for historically disadvantaged workers and promoting greater economic justice in our society.

Challenges and Opportunities

The United States faces several triangulating challenges. First is our rapidly evolving economy, which threatens to displace millions of already vulnerable low-wage workers due to technological advances and automation. The COVID-19 pandemic accelerated automation, which is a long-term strategy for the tech companies that underpin the gig economy. According to a report by an independent research group, self-driving taxis are likely to dominate the ride-hailing market by 2030, potentially displacing 8 million human drivers in the United States alone.

Second, we have a generation of workers who have not reaped the benefits associated with good-paying union jobs due to decades of anti-union activities. As of 2022, union membership has dropped from more than 30% of wage and salary workers in the private sector in the 1950s to just 6.3%. The declining percentage of workers represented by unions is associated with widespread and deep economic inequality, stagnant wages, and a shrinking middle class. Lower union membership rates have contributed to the widening of the pay gap for women and workers of color.

Third, historically disadvantaged groups are overrepresented in nonunionized, low-wage, app-based, and automation-adjacent work. This is due in large part to systemic racism. These structures adversely affect BIPOC workers’ ability to obtain quality education and training, create and pass on generational wealth, or follow through on the steps required to obtain union representation.

Workers face tremendous opposition to unionization efforts from companies that spend hundreds of millions of dollars and use retaliatory actions, disinformation, and other intimidating tactics to stop them from organizing a union. For example, in New York, Black organizer Chris Smalls led the first successful union drive in a U.S. Amazon facility after the company fired him for his activities and made him a target of a smear campaign against the union drive. Smalls’s story is just one illustration of how BIPOC workers are in the middle of the collision between automation and anti-unionization efforts. 

The recent surge of support for workers’ rights is a promising development, but BIPOC workers face challenges that extend beyond anti-union tactics. Employer retaliation is also a concern. Workers targeted for retaliation suffer from reduced hours or even job loss. For instance, a survey conducted at the beginning of the COVID-19 pandemic revealed that one in eight workers perceived possible retaliatory actions by their employers against colleagues who raised health and safety concerns. Furthermore, Black workers were more than twice as likely as white workers to experience such possible retaliation. This sobering statistic is a stark reminder of the added layers of discrimination and economic insecurity that BIPOC workers have to navigate when advocating for better working conditions and wages. 

The time to enact strong policy supporting historically disadvantaged workers is now. Advancing racial equity and racial justice is a focus for the Biden-Harris Administration, and the political and social will is evident. The day one Biden-Harris Administration Executive Order on Advancing Racial Equity and Support for Underserved Communities Through the Federal Government seeks to develop policies designed to advance equity for all, including people of color and others who have been historically underinvested in, marginalized, and adversely affected by persistent poverty and inequality. Additionally, the establishment of the White House  is a significant development. Led by Vice-President Kamala Harris and Secretary of Labor Marty Walsh, the Task Force aims to empower workers to organize and negotiate with their employers through federal government policies, programs, and practices. 

A key focus for the Task Force is to increase worker power in underserved communities by examining and addressing the challenges faced by workers in jurisdictions with restrictive labor laws, marginalized workers, and workers in certain industries. The Task Force is well-timed, given the increased support for workers’ rights demonstrated through the record-high number of petitions filed with the National Labor Relations Board and the rise in strikes over the past two years. The Task Force’s approach to empowering workers and supporting their ability to organize and negotiate through federal government policies and programs offers a promising opportunity to address the unique challenges faced by BIPOC workers in unionization efforts.

The National Bargaining in Good Faith Fund is a critical initiative that can help level the playing field by providing financial assistance to workers facing opposition from employers who refuse to engage in good-faith bargaining, thereby expanding access to unions for Black, Indigenous, and other people of color. In addition, the proposed initiative would reinforce Equal Employment Opportunity Commission (EEOC) and National Labor Relations Board (NLRB) policies regarding employer discrimination and retaliation. The Bargaining in Good Faith Fund will provide direct payments to workers whose employers have retaliated against them for engaging in union organizing activities. The initiative also includes monitoring cases where a violation has occurred against workers involved in union organization and connecting their bargaining unit with relevant resources to support their efforts. With the backing of the Task Force, the fund could make a significant difference in the lives of workers facing barriers to organizing.

Plan of Action

While the adoption of a policy like the Bargaining in Good Faith Fund is unprecedented at the federal level, we draw inspiration from successful state-level initiatives aimed at improving worker well-being. Two notable examples are initiatives enacted in California and New York, where state lawmakers provided temporary monetary assistance to workers affected by the COVID-19 pandemic. Taking a cue from these successful programs, we can develop federal policies that better support workers, especially those belonging to historically disadvantaged groups.

The successful implementation of worker-led, union-organized, and community-led strike assistance funds, as well as similar initiatives for low-wage, app-based, and automation-adjacent workers, indicates that the Bargaining in Good Faith Fund has strong potential for success. For example, the Coworker Solidarity Fund provides legal, financial, and strategic support for worker-activists organizing to improve their companies, while the fund invests in ecosystems that increase worker power and improve economic livelihoods and social conditions across the U.S. South.

New York state lawmakers have also set a precedent with their transformative Excluded Workers Fund, which provided direct financial support to workers left out of pandemic relief programs. The $2.1 billion Excluded Workers Fund, passed by the New York state legislature and governor in April 2021, was the first large-scale program of its kind in the country. By examining and building on these successes, we can develop federal policies that better support workers across the country.

A national program requires multiple funding methods, and several mechanisms have been identified to establish the National Bargaining in Good Faith Fund. First, existing policy needs to be strengthened, and companies violating labor laws should face financial consequences. The labor law violation tax, which could be a percentage of a company’s profits or revenue, would be directed to the Bargaining in Good Faith Fund. Additionally, penalties could be imposed on companies that engage in retaliatory behavior, and the funds generated could also be directed to the Bargaining in Good Faith Fund. New legislation from Congress is required to enforce existing federal policy.

Second, as natural allies in the fight to safeguard workers’ rights, labor unions should allocate a portion of their dues toward the fund. By pooling their resources, a portion of union dues could be directed to the federal fund.

Third, a portion of the fees paid into the federal unemployment insurance program should be redirected to Bargaining in Good Faith Fund. 

Fourth, existing funding for worker protections, currently siloed in agencies, should be reallocated to support the Bargaining in Good Faith Fund more effectively. To qualify for the fund, workers receiving food assistance and/or Temporary Assistance for Needy Families benefits should be automatically eligible once the NLRB and the EEOC recognize the instance of retaliation. Workers who are not eligible could apply directly to the Fund through a state-appointed agency. This targeted approach aims to support those who face significant barriers to accessing resources and protections that safeguard their rights and well-being due to historical labor exploitation and discrimination.

Several federal agencies could collaborate to oversee the Bargaining in Good Faith Fund, including the Department of Labor, the EEOC, the Department of Justice, and the NLRB. These agencies have the authority to safeguard workers’ welfare, enforce federal laws prohibiting employment discrimination, prosecute corporations that engage in criminal retaliation, and enforce workers’ rights to engage in concerted activities for protection, such as organizing a union.

Conclusion

The federal government has had a policy of supporting worker organizing and collective bargaining since the passage of the National Labor Relations Act in 1935. However, the federal government has not fully implemented its policy over the past 86 years, resulting in negative impacts on BIPOC workers, who face systemic racism in the unionization process and on the job. Additionally, rapid technological advances have resulted in the automation of tasks and changes in the labor market that disproportionately affect workers of color. Consequently, the United States is likely to see an increase in wealth inequality over the next two decades.

The Biden-Harris Administration can act now to promote racial equity by establishing a National Bargaining in Good Faith Fund to support historically disadvantaged workers in unionization efforts. Because this is a pressing issue, a feasible short-term solution is to initiate a pilot program over the next 18 months. It is imperative to establish a policy that acknowledges and addresses the historical disadvantage experienced by these workers and supports their efforts to attain economic equity.