Prioritize Student Safety in K-12 Education By Establishing AI Procurement Guardrails
Artificial intelligence (AI) tools are rapidly entering K–12 education, influencing discipline, grading, placement, attendance monitoring, tutoring, and school safety. While these systems claim to promote efficiency and innovation, adoption has outpaced oversight. Opaque and insufficiently tested tools are increasingly shaping student outcomes without consistent transparency, civil rights review, or technical safeguards.
This presents material legal and operational risks. AI systems affecting discipline, eligibility, and monitoring may implicate education civil rights laws such as Title VI, Title IX, Section 504, and the Individuals with Disabilities Education Act (IDEA), particularly where disparate impacts arise from biased historical data. Tools that collect or process sensitive student information also raise compliance concerns under the Family Educational Rights and Privacy Act (FERPA) and related state laws. At the same time, many districts lack the capacity to evaluate vendor efficacy claims or negotiate contracts that protect against bias, privacy breaches, or vendor lock-in.
States and the U.S. Department of Education can address these risks using procurement and oversight tools already within their authority. This memo proposes six actionable steps: (1) establish statewide AI procurement guardrails; (2) require Algorithmic Impact Assessments for high-risk systems; (3) prohibit or strictly limit predictive-policing and law-enforcement-derived analytics in schools; (4) encourage ongoing performance monitoring and incident response; (5) create a state-level technical assistance and vendor accountability programs; and (6) invest in leadership-level capacity building for superintendents and senior administrators. Together, these measures support safer adoption, reduce discrimination and privacy risks, strengthen fiscal stewardship, and build public trust.
Challenge and Opportunity
Recent incidents demonstrate that AI deployment in K–12 settings can create serious risks when implemented without adequate safeguards, transparency, or oversight. In one widely reported example, the Los Angeles Unified School District entered a contract with an education-tech startup that ultimately misused student data and put sensitive information at risk. In another case, an AI-enabled security camera system used in a Baltimore County Public Schools school misidentified a bag of chips as a firearm, illustrating the potential for inaccurate automated threat detection systems to trigger unnecessary panic, disciplinary responses, or law enforcement intervention. These incidents underscore that AI systems deployed in schools can materially affect student privacy, safety, disciplinary outcomes, and civil rights, particularly when systems are introduced without sufficient testing, human oversight, or clear accountability mechanisms.
In some contexts, AI may present genuine opportunities for K–12 education when deployed thoughtfully and within appropriate limits. Certain tools may help educators identify students who need additional academic support, expand access to tutoring, streamline administrative tasks, and improve language accessibility for multilingual learners and families. Used as decision-support rather than decision-making systems, AI can help schools direct limited resources more efficiently and support individualized learning in ways that traditional software often cannot.
At the same time, the benefits of these systems depend heavily on how they are selected, designed, governed, and monitored in practice. Educational institutions are increasingly being asked to evaluate unproven products, some costing in the tens of millions, that make probabilistic inferences, adapt over time, and operate with limited transparency. These are features that differ substantially from conventional education technology. These distinctions matter because systems introduced to improve efficiency may also shape high-stakes educational outcomes in ways that are difficult to detect without structured oversight, and some can even be harmful to students and school communities.
The core challenge is not simply AI adoption, but that it is occurring through procurement systems designed for conventional software—not probabilistic tools that may influence discipline, placement, and safety. Most districts rely on standard ed-tech purchasing processes that rarely require structured review of training data, demographic performance, or long-term equity impacts, leaving high-stakes decisions without proportionate risk analysis.
This governance gap is amplified by fragmentation. Local school boards can adopt procedures to aid decision making. Meanwhile, thousands of districts negotiate independently with sophisticated vendors, often lacking the expertise to assess claims about accuracy, bias, data security, or simply accept “off-the-shelf” AI products and terms of service. Contracts can limit audit rights, assent to harmful data practices, and create vendor lock-in, with smaller districts particularly vulnerable to unverified assurances. In worst case scenarios, oversight begins only after harm occurs, leaving districts reactive rather than preventative. In short, decentralized procurement, uneven capacity, and opaque vendor practices create structural risks that, absent coordinated state standards, may entrench inequities and erode public trust.
While the challenges are significant, states possess clear authority to address them. States retain primary responsibility for K–12 governance, including procurement standards, contracting requirements, and oversight of local education agencies. State legislatures and departments of education can issue guidance, promulgate regulations where authorized, condition funding on compliance, and coordinate with procurement offices, CIOs, and attorneys general to establish uniform contracting expectations.
In practice, states can establish baseline procurement checklists and disclosure requirements; mandate better processes that promote better informed decision making; develop model contract clauses addressing data minimization, audit rights, and termination; require pre-deployment review for high-risk systems; condition technology funding on governance criteria; and provide centralized technical assistance. The U.S. Department of Education plays a complementary role through civil rights enforcement, FERPA guidance, and grant-making authority under relevant statutes.
Importantly, not all AI use in schools is harmful. Tutoring systems that scaffold reasoning, or tools that identify students for additional support, can expand opportunity when transparent and used as decision-support rather than automated decision-making systems. The goal then is not to halt innovation, but to channel it responsibly.
Procurement is a high-leverage intervention point that can foster responsible innovation and technology integration. Rather than framing AI governance as a choice between bans and unregulated adoption, guardrails focus on conditions of purchase and deployment, preventing harms upstream before litigation, remediation, or public controversy arise. A statewide framework reduces fragmentation and strengthens negotiating leverage. Consistent standards lower compliance costs and incentivize vendors to compete on transparency, fairness testing, and privacy protections. By acting now, states and federal education leaders can shape procurement norms before harmful practices become entrenched thereby supporting innovation while safeguarding students’ rights and trust.
Plan of Action
Recommendation 1. Establish Statewide AI Procurement Guardrails for K–12 Purchasing
Statewide procurement guardrails are the most feasible and immediate way to reduce risk in school-based AI adoption. Rather than requiring each district to independently develop technical and legal expertise, state departments of education can establish uniform, scalable standards grounded in existing authority and assist schools with product analysis resources. State departments of education already regulate contracts for textbooks, transportation, and student data systems. AI introduces new technical considerations, but these can be addressed through structured disclosure requirements, risk classification, and standardized contract provisions led by state departments of education.
Centralizing guardrails reduces duplication, strengthens negotiating leverage with vendors, and addresses civil rights and privacy risks upstream. A modest investment in templates and oversight can significantly reduce long-term legal, fiscal, and reputational exposure.
Classify and Define “High-Risk” AI Uses
States should classify AI systems by risk level so that safeguards are proportionate to the stakes involved. High-risk systems would include those that implicate rights found in the 2022 White House “Blueprint for an AI Bill of Rights.” Within an education context that would include decisions that impact discipline or expulsion, affect placement or eligibility decisions (including gifted or special education), generate behavioral risk scores or threat assessments, enable surveillance or facial recognition, impact grading or graduation eligibility, or predict student performance in ways tied to consequential outcomes.
These uses are high-risk because they directly affect educational access, safety, liberty interests, and protected status under federal civil rights law, potentially implicating due process protections and civil rights statutes such as Title VI, Title IX, Section 504, and the IDEA. State guidance should also clarify that systems qualify as high-risk when they materially influence high-stakes decisions or process highly sensitive data such as disability or biometric information. For example, in the K–12 education setting, systems that recommend student discipline, identify students as potential safety threats, determine eligibility for advanced academic programs, monitor student mental health or behavior, or evaluate students for special education interventions can influence school administrator’s decisions and substantially shape student educational opportunities, outcomes, and civil rights protections, and therefore warrant heightened oversight. It is important to underscore that AI that may materially influence high-stakes decisions should always be subject to human oversight and never be the sole basis for a decision.
Require a Pre-Purchase Review Checklist and Minimum Vendor Disclosures
For high-risk systems, states should require a structured pre-purchase review to ensure that key legal, technical, and operational risks are addressed before contracts are executed.
A standardized AI Procurement Review Checklist should require districts to document:
- Purpose and Use Case: The specific problem addressed, whether the system supplements or replaces human decision-making, and which student populations are affected including the risk that certain groups of students may be adversely impacted and how.
- Accuracy and Validation: Identity of the validation tester and any conflicts of interest including financial interest in the sale of the product being evaluated, documented error rates, validation studies (including demographic performance where available), and known system limitations.
- Disparate Impact and Treatment Monitoring: Whether the system has been tested for disparate impact and disparate treatment across protected groups, the methodology used, and plans for ongoing monitoring. Other legal indicia of discrimination would ideally be considered as well, including whether a system has been tested for: selective enforcement, hostile learning environment, proxy discrimination, failures to provide accommodation, amongst others.
- Human Oversight and Contestability: Who reviews outputs, whether AI influences adverse decisions, and the process for student or parent appeals.
- Data Governance and Security: Categories of data used, retention timelines, subcontractor involvement, encryption standards, access controls, and incident response protocols.
Overall Safety Declaration: A declaration that determines whether the product is safe or too dangerous for school access.
Vendors, but ideally independent evaluators, should also provide minimum disclosures, including general descriptions of training data sources, audit summaries (if available), subgroup performance metrics, data retention policies, subprocessor lists, server locations, and cybersecurity certifications (e.g., SOC 2). For example, subgroup performance metrics can help identify whether a tool performs differently for students with disabilities or students from different racial or linguistic backgrounds; data retention and server location disclosures help districts evaluate compliance with student privacy laws and data governance obligations; and cybersecurity certifications and subprocessor information help schools assess risks related to student data security and third-party access. These requirements do not mandate disclosure of proprietary algorithms, but they do require sufficient transparency for schools to conduct procurement diligence, evaluate potential harms, and ensure accountability for systems that may materially affect students’ educational experiences and opportunities.
Establish Standard Contract Clauses
Procurement guardrails are only effective if embedded in enforceable contracts. States should develop model AI contract provisions that districts are required or strongly encouraged to adopt.
Core clauses should include:
- Data Minimization and Use Limits: Collection limited to what is necessary as determined by the school district; prohibition on secondary uses (e.g., model training or resale) without parental consent; clear data deletion timelines upon termination of the contract.
- Security and Breach Notification: Encryption at rest and in transit; defined breach notification timelines; vendor liability for negligence-related breaches.
- Audit and Transparency Rights: District, state, or independent evaluator access to performance and compliance documentation; right to independent audit; annual performance updates.
- Termination and Exit Protections: Termination rights for material failures or civil rights concerns; data portability; certified data deletion or media sanitization upon termination of contract.
- Subcontractor and Data Location Controls: Full subprocessor disclosure; equivalent data protections; restrictions on undisclosed offshore data transfers; U.S.-based storage where feasible.
Embedding these provisions strengthens district leverage, reduces vendor lock-in, and creates enforceable accountability if harms occur.
Recommendation 2. Require an Algorithmic Impact Assessment (AIA) Before Deployment of High-Risk Systems
States, with the assistance of independent evaluators, should require districts to complete and publicly post an Algorithmic Impact Assessment (AIA) before deploying any high-risk AI system. An AIA is a structured evaluation of a system’s purpose, risks, legal implications, and mitigation strategies conducted prior to deployment.
By requiring AIAs, states shift governance upstream and identify civil rights, due process, privacy, and safety risks before systems affect students, rather than responding after harm occurs. This approach reinforces that educational innovation must comply with civil rights law, protect student data, and meet defined safety standards. AIAs do not prohibit AI use. They ensure that AI adoption is deliberate, transparent, and accountable.
When an AIA Is Required
An AIA should be mandatory for AI systems classified as high-risk under Recommendation 1, including systems that:
- Influence discipline or safety interventions
- Affect placement, eligibility, or grading decisions
- Conduct behavioral monitoring or surveillance
- Generate risk scores tied to adverse actions
The AIA must be completed prior to contract execution or system activation.
Core Contents of the AIA
To be meaningful, an AIA must provide sufficient detail to inform decision-makers and the public without requiring disclosure of proprietary source code. States should issue a standardized template to ensure consistency. An AIA should include:
- Purpose and Use: The problem addressed, how AI outputs will be used (advisory or determinative), and which student populations are affected.
- Data Inputs and Governance: Categories of data used; whether protected characteristics or proxies are included; sources of training data; retention timelines; data sharing practices (including subcontractors and cross-border transfers); and whether student data are used for ongoing model training. This enables risk evaluation under FERPA and civil rights law without requiring disclosure of proprietary datasets.
- Accuracy and Validation: Error rates, subgroup performance where feasible, validation methods, and known limitations. Where subgroup testing has not occurred, that must be disclosed.
- Disparate Impact, Disparate Treatment, and Oversight: Results of disparate-impact and treatment testing, ongoing monitoring plans, mitigation strategies, and confirmation that AI will not serve as the sole basis for adverse decisions without human review. The assessment should also identify which agency, department, or designated internal team is responsible for ongoing oversight, whether dedicated staff capacity exists to carry out that responsibility, and who will be accountable for responding if discriminatory outcomes, safety failures, or other material harms emerge after deployment.
- Due Process Protections: Notification procedures, appeal processes, review timelines, and mechanisms for correcting erroneous records.
- Privacy and Security Safeguards: Encryption standards, access controls, incident response protocols, breach notification timelines, and FERPA compliance.
Transparency and Public Posting Requirements
AIAs should balance meaningful transparency with protection of proprietary information. States should require a two-tiered structure:
- Public version: Narrative descriptions of the system’s purpose, data categories, data and privacy protections, performance metrics, limitations, safeguards, and high-level demographic testing results where feasible, accompanied by a plain-language summary that is accessible to parents and guardians.
- Regulator-facing appendix (as needed): More detailed technical documentation, validation studies, and contractual data governance materials.
Raw datasets and proprietary algorithms do not need to be disclosed. However, vague assurances (e.g., “tested for bias”) are insufficient. AIAs must include independent testing, documented metrics, and clear descriptions of methodologies.
Each AIA should be publicly posted on the district website prior to acquisition, including a parent-friendly summary (2–3 pages), contact information for questions or complaints, and a clear explanation of appeal rights. States can reduce administrative burden by providing standardized templates and maintaining a centralized statewide repository.
Public transparency strengthens trust, enables independent review, promotes vendor accountability, and supports cross-district learning.
Recommendation 3. Prohibit or Strictly Limit Predictive-Policing and Law-Enforcement-Derived Analytics in School Settings
Certain AI uses in schools pose heightened civil rights, due process, and safety risks that procurement safeguards alone cannot mitigate. Systems that replicate predictive-policing models or rely on law-enforcement-derived data warrant clear statutory limits.
Define and Prohibit High-Risk Predictive Discipline and Law-Enforcement-Derived Systems
The increasing use of AI-driven behavioral analytics, predictive monitoring tools, and school surveillance technologies raises significant concerns for student privacy, civil rights, due process, educational equity, and student safety. Systems that predict future misconduct, generate behavioral threat scores, or rely on law-enforcement-derived data risk replicating historical patterns of bias, normalizing heightened surveillance, and increasing unnecessary disciplinary or law enforcement intervention, particularly for students of color, students with disabilities, LGBTQ+ students, and other historically marginalized groups. Because these technologies can materially influence disciplinary outcomes states should prohibit or strictly limit AI systems within the education setting that:
- Generate forward-looking risk scores used to justify discipline, suspension, expulsion, or law enforcement referral.
- Produce behavioral threat scores like “aggression” or “threats” to school safety absent specific, individualized evidence.
- Rely primarily on law enforcement datasets or predictive policing models.
- Use facial recognition or other biometric surveillance.
- Identify or label objects as weapons.
- Integrate student data into external law enforcement analytics systems without explicit statutory authorization.
Predictive Discipline Risk Scoring
Due to discrepancies in school discipline, for example, the well-documented research showing Black students are disproportionately disciplined compared to White students for similar behaviors, the same AI risk score may lead to very different interventions, which would exacerbate existing disparities. Therefore, assigning students algorithmic “risk scores” for future misconduct raises serious equity concerns. Systems that aggregate attendance records, prior disciplinary history, or behavioral indicators risk replicating documented racial disparities embedded in historical data. Even if statistically predictive, such tools may institutionalize biased baselines and normalize heightened surveillance of certain students.
Accordingly, states should prohibit AI systems that use forward-looking misconduct predictions to justify disciplinary action or automated referrals without individualized human evaluation.
Law-Enforcement-Derived Analytics
AI systems adapted from policing contexts introduce additional risks. Law enforcement datasets often reflect patterns of over-policing and incorporating them into school decision-making can import external bias into educational settings. States should prohibit systems that integrate criminal justice databases into student risk scoring, share student behavioral data with predictive law enforcement platforms absent a specific incident, or use facial recognition tied to law enforcement watchlists in routine school operations.
Schools are educational institutions—not extensions of the criminal justice system. Clear statutory boundaries are necessary to prevent normalization of predictive surveillance in learning environments.
Allow Narrow Exceptions Only with Heightened Safeguards
There may be limited scenarios where data analytics support school safety planning or student support interventions. In such cases, use should be permitted only under strict conditions:
- AI outputs may not serve as the sole basis for adverse action.
- All outputs must be reviewed by trained personnel.
- Clear documentation of evidentiary basis must accompany any intervention.
- Systems must undergo an Algorithmic Impact Assessment prior to deployment.
- Annual disparate impact and disparate treatment analysis must be conducted.
These safeguards increase the likelihood that technology supplements, not replaces, professional judgment.
Recommendation 4. Governance, Ongoing Performance Monitoring, Public Reporting, and Incident Response
AI systems evolve and update over time, interact with changing student populations, and may degrade in accuracy or fairness after deployment. Effective governance therefore requires a lifecycle oversight model that includes continuous monitoring, transparent reporting, and structured response mechanisms. This recommendation establishes that high-risk AI systems in schools are not “set and forget” technologies. They must be evaluated regularly against performance, equity, and safety benchmarks.
Ongoing Testing and Annual Public Reporting Requirements
States should require districts using AI systems to submit annual public reports summarizing system performance and impact, using a standardized template provided by the state education departments to ensure consistency and reduce burden. Continuous oversight reflects four core principles: accuracy can degrade over time; equity requires ongoing monitoring; transparency builds trust; and accountability must be enforceable through mechanisms such as sunset or reauthorization. This recommendation does not presume failure; rather it ensures responsible innovation through measurable outcomes and structured review.
Annual reporting should include:
- Performance Metrics: Error rates (false positives/negatives), misclassification rates, trends over time, and testing methods. For example, security systems should report alerts versus confirmed threats; grading systems should report educator overrides.
- Demographic Disparities: Disaggregated data on flagging rates across high-impact use cases, AI-linked disciplinary actions, and override rates across race, disability status, English learner status, and other relevant categories. Ongoing disparity monitoring supports compliance with federal civil rights obligations.
- Human Oversight: Frequency of overrides, instances where AI influenced adverse actions, and documentation of review processes to ensure systems are not functioning as automated decision-makers.
- Complaints and Resolution: Volume and type of complaints (accuracy, bias, privacy), resolution timelines, and corrective actions taken, which together serve as an early warning mechanism.
Embedding monitoring and reauthorization into state governance ensures AI systems remain tools for student support rather than unexamined sources of risk.
Rapid Incident Response Protocol
In addition to annual reporting, states should require districts to adopt a rapid incident response protocol for significant AI-related harms, including major student data breaches, unsafe outputs such as erroneous security alerts that trigger law enforcement involvement, systemic bias identified through internal review or complaint, and widespread false positives and negatives affecting multiple students. The protocol should recommend immediate containment—including suspension of system use where necessary—prompt notification to affected families and the state education department within a defined timeframe (such as 48–72 hours), a documented root cause analysis conducted in coordination with the vendor, and a corrective action plan with clear mitigation steps, accompanied by a public summary.
Districts should not hesitate to pause or suspend deployment when student safety, civil rights, or liability risks are implicated. In high-stakes environments, it is prudent to err on the side of intervention rather than adopt a “wait and see” approach. This framework aligns with established cybersecurity incident response standards and minimizes the risk of prolonged or compounded harm.
Sunset and Reauthorization Requirement
To prevent long-term entrenchment of ineffective or harmful systems, states should require periodic reauthorization of AI tools. Authorization should automatically sunset after three years unless renewed based on demonstrated accuracy, absence of unexplained demographic disparities, documented educational benefit, and compliance with reporting and audit requirements. This approach creates accountability without imposing permanent bans and incentivizes continuous system improvement.
Recommendation 5. Create a State-Level Technical Assistance and Vendor Accountability Program
Procurement guardrails, AIAs, and monitoring will only succeed if districts have the capacity to implement them. Many, especially rural districts, lack high speed internet, expertise in AI evaluation, data governance, contract negotiation, and receive less funding than their urban peers. A state-level technical assistance and vendor accountability program, buttressed with support from universities and independent evaluators, can close this gap and shape stronger market standards.
Without practical support, reforms risk becoming procedural rather than protective. Modest centralized investment can reduce duplication, strengthen negotiating leverage, reinforce civil rights and privacy compliance, and promote responsible innovation at far lower cost than reactive remediation after harm occurs.
Statewide Technical Assistance and Training
States should establish targeted training for procurement staff, technology leaders, and administrators overseeing AI adoption. Training should cover risk classification, completion of AIAs, evaluation of vendor claims and accuracy metrics, disparate impact and treatment analysis, contract negotiation best practices, and incident response obligations. Delivery can leverage existing professional development structures such as webinars, regional workshops, online modules, and standardized toolkits. To limit costs, states can partner with public universities, education service agencies, or nonprofit research centers with expertise in education technology and civil rights compliance.
Optional Statewide “Approved Vendor” Pathway
States may establish an optional pre-vetted or “approved vendor” pathway for AI systems, structured as conditional certification tied to transparency and compliance standards. Under this model, vendors voluntarily submit documentation demonstrating compliance with state disclosure, testing, and contract requirements. The state conducts a structured review. If approved, vendors are listed in a public registry. Districts may still procure other vendors but must complete full independent review.
States should also consider pairing any approved-vendor pathway with targeted compliance assistance, particularly for smaller or emerging vendors that may lack dedicated legal or regulatory staff but offer promising educational tools. This could include technical workshops, model disclosure templates, and guidance on meeting state testing, documentation, privacy, and contract expectations. Providing this support helps ensure that approval pathways do not inadvertently favor only large incumbent ed-tech companies with extensive compliance infrastructure, while still preserving rigorous standards for safety, transparency, and civil rights protections.
This approach reduces duplicative review, strengthens bargaining leverage through uniform standards, and incentivizes vendors to compete on transparency and validated performance rather than marketing claims. Approval should remain conditional, subject to ongoing compliance monitoring and revocation if standards are not met.
Independent Evaluation and Privacy-Preserving Audit Options
To strengthen accountability, states should mandate independent evaluation of high-risk AI systems that balance transparency with student privacy and vendor intellectual property protections. Options include secure data enclaves, aggregated performance reviews under confidentiality agreements, de-identified or differential privacy testing environments, and partnerships with public universities for validation studies. Independent evaluation helps test vendor claims, detect disparate impacts and treatment, inform evidence-based policymaking, and build public trust. Where feasible, states may offer grants to support validation of widely used systems.
Ensuring Effective Transmission of State Guidance to Districts
State guidance does not always translate cleanly into local practice. Differences in district capacity, staffing, and procurement autonomy can result in uneven compliance. To improve implementation, states should integrate AI oversight into existing compliance or accreditation cycles, provide standardized templates and model contract language, designate a clear AI governance lead within the State education departments, and phase implementation beginning with high-risk systems. Experience from data privacy, IDEA, and Title IX compliance shows that clear documentation and centralized technical assistance significantly improve consistency between state policy and district practice.
Recommendation 6. Leadership-Level Capacity Building for Superintendents and Senior District Officials
Procurement reform is insufficient without leadership capacity. Superintendents and senior administrators often make AI adoption decisions based on vendor presentations or innovation pressures, yet may lack training in algorithmic risk, civil rights implications, and technology contract governance. States should establish targeted AI governance training for superintendents, cabinet leaders, CTOs, chief academic officers, and school board members.
Core Training Components
Leadership-level training should cover core legal and governance competencies, including civil rights risks under Title VI, IDEA, Section 504, FERPA, and due process standards. Training should include strategic procurement literacy such as avoiding vendor lock-in and understanding critical audit and data provisions. Training should distinguish supportive from punitive AI use cases and build skills to recognize overpromising in vendor marketing. Lastly, training must contain sections on crisis preparedness, including how to respond to discriminatory or unsafe outcomes and communicate transparently with families and the public.
Delivery Mechanisms
States can integrate this leadership capacity-building into existing professional development structures. Some examples could include: annual superintendent conferences, certification renewals, school board association trainings, and regional education service agency programs. Leveraging established forums avoids creating new bureaucratic layers while using trusted professional networks to promote consistent implementation.
Why Leadership Capacity Matters
State guidance does not always translate cleanly into district practice. Implementation is shaped by resource disparities, competing priorities, leadership turnover, and vendor influence. When superintendents understand the governance framework and its rationale, they are more likely to demand compliance with procurement guardrails, resist premature adoption, dedicate staff time to meaningful review, and support transparency and public reporting. Effective leadership now also requires understanding how to manage AI-enabled systems in practice including how automated outputs interact with existing administrative processes, where human judgment must remain central, and how risks may emerge over time after deployment. Because AI governance is inherently cross-functional, district leaders must be prepared to coordinate legal, procurement, technical, and ethical considerations rather than treat AI as a purely technical issue delegated to IT staff alone. The goal is not to turn superintendents into technical specialists, but to ensure they can exercise informed oversight over AI-enabled decision environments. Without leadership buy-in, even well-designed safeguards risk remaining underutilized.
Conclusion
States must ensure AI integration supports student safety and well-being.
This is possible by adopting procurement guardrails, requiring Algorithmic Impact Assessments, limiting high-risk predictive uses, and mandating ongoing oversight. This framework relies on existing state authority over procurement, contracting, and oversight. By leveraging these tools, policymakers can focus on responsible implementation. Modest investments in technical assistance and leadership capacity can create clear, workable standards for districts and predictable expectations for vendors.
Done well, this approach delivers safer technology deployment. Specifically, it lowers discrimination risk and builds stronger data governance. Doing so ensures better stewardship of public funds, and greater public trust. AI in schools should expand opportunity, not erode it. Acting now, while adoption norms are still forming, allows education leaders to ensure innovation and student rights advance together.
This memo does not comprehensively address several important and distinct issues that warrant separate, dedicated analysis. For example, while this memo references students with disabilities, it does not fully examine the unique legal, educational, accessibility, and civil rights implications AI systems may pose for students protected under the IDEA and Section 504. Similarly, this memo does not comprehensively address the particular risks and considerations affecting English language learners, immigrant students, or multilingual families, each of which raises important questions related to language access, equity, and data governance. The memo does not broadly explore AI literacy or training needs for teachers, students, parents, or school administrators, despite the growing importance of ensuring school communities understand how these technologies function and affect educational environments. Given the confines of this memo, the analysis is intentionally focused on procurement guardrails and governance mechanisms for high-risk AI systems in K–12 education. Nevertheless, the issues not addressed here remain critically important and warrant substantial future analysis and policymaker attention.
When properly structured — with specific numeric targets, secured financial obligations, independent monitoring, and meaningful enforcement — CBAs transform data center deals into durable community partnerships.
Protecting the public from the tech industry’s predatory business models and the next wave of AI harms is an enormous challenge, but we have the evidence that trying to build a healthier digital culture is absolutely worth the effort.
Opaque and insufficiently tested tools are increasingly shaping student outcomes without consistent transparency, civil rights review, or technical safeguards. States and the U.S. Department of Education can address these risks using procurement and oversight tools already within their authority.
Commercial artificial intelligence tools have recently emerged that are able to produce police reports. If the resulting reports are inaccurate, incomplete or biased, or if the process leaks confidential information, this could undermine the criminal justice system and harm citizens.