 
    
In Remembrance of Dearly Departed Federal Datasets
On this All Hallow’s Eve, let us recognize the losses of datasets that served the American people
There’s been lots of talk – and some numbers (often in the thousands) – about disappearing federal datasets, especially after many went dark last January when agencies rushed to scrub the perceived spectre of data on gender, DEI, and climate from the public record. Most of those datasets have returned from the dead, some permanently changed by the experience.
Though it’s premature to breathe a sigh of relief – the future of federal data remains in jeopardy – we thought Halloween was an opportune time to ask, which federal datasets have left this mortal realm?
Here’s what we found.
The good news
For the most part, the vast majority of the hundreds of thousands of datasets produced by the federal government are still alive, and have so far escaped mutilation1 or termination. By our rough counts, the datasets that have been truly axed number perhaps in the dozens (not hundreds or thousands).
The bad news
All federal datasets are currently at risk of death by a thousand cuts, weakened by the loss of staff and expertise, contracts, and scientific advisory committees. Just because a given dataset hasn’t been explicitly killed off, doesn’t mean that an agency still has the capacity to collect, protect, process, and publish that data. Also, datasets and variables that do not align with Administration priorities, or might reflect poorly on Administration policy impacts, seem to be especially in the cross-hairs.
The details
We’ve identified three types of data decedents. Examples are below, but visit the Dearly Departed Dataset Graveyard at EssentialData.US for a more complete tally and relevant links.
- Terminated datasets. These are data that used to be collected and published on a regular basis (for example, every year) and will no longer be collected. When an agency terminates a collection, historical data are usually still available on federal websites. This includes the well-publicized terminations of USDA’s Current Population Survey Food Security Supplement, and EPA’s Greenhouse Gas Reporting Program, as well as the less-publicized demise of SAMHSA’s Drug Abuse Warning Network (DAWN). Meanwhile, the Community Resilience Estimates Equity Supplement that identified neighborhoods most socially vulnerable to disasters has both been terminated and pulled from the Census Bureau’s website.
- Removed variables. With some datasets, agencies have taken out specific data columns, generally to remove variables not aligned with Administration priorities. That includes Race/Ethnicity (OPM’s Fedscope data on the federal workforce) and Gender Identity (DOJ’s National Crime Victimization Survey, the Bureau of Prison’s Inmate Statistics, and many more datasets across agencies).
- Discontinued tools. Digital tools can help a broader audience of Americans make use of federal datasets. Departed tools include EPA’s Environmental Justice Screening and Mapping tool – known to friends as “EJ Screen” – which shined a light on communities overburdened by environmental harms, and also Homeland Infrastructure Foundation-Level Data (HIFLD) Open, a digital go-bag of ~300 critical infrastructure datasets from across federal agencies relied on by emergency managers around the country.
Forever in our hearts, and in some cases, given a second life
Another pattern we saw is that some tools have been reincarnated in civil society. Climate Central breathed life back into the U.S. Billion-Dollar Weather and Climate Disasters database, Public Environmental Data Partners did the same for EJScreen, and Fulton Ring brought back FEMA’s Future Risk Index. All of these tools still depend on the federal data underlying them. Beware, without fresh data to feed on, these tools will turn into the walking dead.
Lurking in the shadows
While our focus now is on deceased federal datasets, other threats loom heavy on the horizon. For example, there are a growing number of examples where the primary federal data remain, but the Administration’s interpretation of that data has veered away from science and toward politics (cases in point: Department of Energy’s July report on the impacts of greenhouse gas emissions on the climate or the CDC’s September guidance on COVID vaccines for children). The data systems aren’t dead, but the implications certainly are scary.
We invite you to take a moment to click through the graveyard of the federal datasets, variables, and tools mentioned above, and more, at the Dearly Departed Dataset Graveyard at EssentialData.US. And if we missed a dataset, please let us know.
Happy Halloween 🎃
Huge thanks to colleagues in the Federation of American Scientists and EssentialData.US, The Impact Project, and Public Environmental Data Partners for the collaboration on this project, and for all who submitted obituaries of their dearly departed datasets.
This analysis includes data, variables, and tools that have been terminated or removed before their time. It represents an attempt to capture substantive data losses and changes above baseline. For example, most data collections for evaluation purposes are not intended to go on indefinitely, and their termination is not included. Additionally, administrations often stand up websites and interactive tools to advance specific policies. We are not cataloging the breadth of such tools that disappeared.
CORRECTION: An earlier version of this post described the “System Name” column in the Federal AI Use Case Inventory as missing. In fact, the dataset had been restructured, but still retained that column. We’ll scratch that one off our list of datasets to light a candle for 
Datasets and variables that do not align with Administration priorities, or might reflect poorly on Administration policy impacts, seem to be especially in the cross-hairs.
At a period where the federal government is undergoing significant changes in how it hires, buys, collects and organizes data, and delivers, deeper exploration of trust in these facets as worthwhile.
Using visioning, world-building, scenario planning, and other foresight tools, participants set aside today’s constraints to design blue-sky models of a future American government.
“The first rule of government transformation is: there are a lot of rules. And there should be-ish. But we don’t need to wait for permission to rewrite them. Let’s go fix and build some things and show how it’s done.”