What the Model Has to Forget: Selective Forgetting as a Public Health Tool
Case of the Week — Min Wu, PhD · ai-public-health.com
I took the Gaokao — China's national college entrance exam — many years ago. By any reasonable accounting, I should be done with it. I passed. I went to university. I built a career. The exam ended for me decades ago.
But every now and then, I still wake up from a dream about it. The dream is usually some version of the same scene: I am about to walk into the exam hall and I have forgotten my ID. Or I cannot find the room. Or the test has already started and I am running. The dream does not appear because anything in my current life resembles Gaokao. It appears because the memory was trained into me at a moment when the stakes were enormous, and it does not seem to know that those stakes are gone.
I cannot decide to stop having that dream. I cannot reason my way out of it. The memory is not in a notebook I can close. It is somewhere deeper, in patterns my mind absorbed so completely that it still surfaces them on its own.
I tell that story because it is what came to mind when I was thinking about the question I left open at the end of last week's post.
The previous post was about diagnostic bias in AI tools used in medicine and public health. The argument was that the bias did not start with the model. It was inherited from the data the model learned from — data built decades ago, on a default "universal patient" that was never universal. The detective work was tracing the noise from the kitchen back to the basement, where the actual leak was.
That post diagnosed. This one is about what comes after the diagnosis.
Once you have found the leak, what do you do? You cannot un-train the models that already exist. You cannot un-publish the trial datasets that were built without women, or without dark-skinned patients, or without patients from low-income regions. The data is already there. The models trained on that data are already deployed. Forward-only solutions — better future trials, better future practices — leave the contaminated training corpus in place, and the self-reinforcing loop continues without interruption.
The fix has to be retrospective. The model has to forget. And as the Gaokao dream reminds me, forgetting something that was deeply trained is not a simple act of will. It is a technical problem.
From databases that never forget to models that must
For most of my career in health informatics, the goal of a good data system was simple: do not forget. The database era — roughly the late 1990s through the 2010s — was built on the premise that the value of digital memory was in its perfection. PubMed, CDC WONDER, ClinicalTrials.gov, VAERS, SEER. These projects defined what a good public health data infrastructure looked like, and they all shared the same memory ideal: store everything, lose nothing, retrieve exhaustively.
That ideal made sense in the database era because the database was a library. You walked in, looked something up, walked out. The data sat still while you used it.
AI memory does not sit still. AI memory is closer to a training gym than a library. The same data that gets stored also shapes the model that learns from it — and once it has shaped the model, it is not easy to remove. The model has absorbed the patterns. The weights of the network now encode what was in the data, including what was wrong with it. If the data underrepresented women, the model has learned women as the exception. If the data overrepresented light-skinned patients, the model has learned light skin as the default.
The bias is no longer in a row of a database that you can edit. It is distributed through the weights of a system that has already been trained. It is more like a Gaokao dream than a notebook entry. It surfaces on its own, in patterns we did not consciously decide to keep.
Which means the rules for managing public health data are changing. Forgetting, which the database era treated as a weakness, is becoming a design requirement.
In the database era, the goal of memory was that nothing be forgotten. In the AI era, one of the most important things a public health AI system can do is forget the right thing.
Selective forgetting, named
The technical term for this in AI research is machine unlearning. The applied term I use in the public health literacy work is selective forgetting. Both name the same idea: the deliberate, structured removal of specific data, patterns, or learned associations from a model, without erasing the model wholesale.
Selective forgetting is not data deletion. Deletion takes everything out. Selective forgetting is closer to clinical surgery — you identify what should not be there, and you remove it without destroying the surrounding tissue. The medical knowledge stays. The biased pattern leaves.
There are several places in public health where this kind of forgetting is becoming the right tool — corrective forgetting for health equity, curatorial forgetting for surveillance data, consensual forgetting for patient-facing chatbots. The mechanisms differ. The principle is the same: a public health AI system that cannot selectively forget is a system that cannot be safely maintained. I treat the applied cases at greater length in my textbook; here I want to stay with the core idea.
Why this matters for the diagnostic posture from last week
The previous post I argued that when an AI output looks biased, the diagnostic question is not only "what is wrong with this model?" but "whose absence in the training data is this output revealing?" That posture identifies where the leak is.
Selective forgetting is what you do once you have found it.
For a public health practitioner who is starting to work with AI tools, this changes one practical question. It is no longer enough to ask vendors "what data was your model trained on?" — though that question still matters. The new question is "what mechanism does your system have for removing data once we determine it should not be there?" If the answer is "none," the tool is a database-era tool wearing AI-era clothes— the new bottle running old logic I have written about before. It can store and retrieve. It cannot correct itself. And in the AI era, a system that cannot correct itself is a system that will continue laundering its inheritance forward, model generation after model generation.
Observer's Insight
The Gaokao dream is, in some way, a useful analogy for what AI systems carry. The exam was a moment when the stakes were so high that my mind built deep patterns around it — patterns that still surface, decades later, even though the conditions that produced them are long gone. The patterns are not a notebook I can close. They are part of how I was trained.
AI models trained on biased medical data carry their inheritance the same way. The defaults that shaped twentieth-century medical research are no longer the defaults we want. But they are still in the weights. They are not in a row that can be edited. They surface in outputs the model produces on its own, when the conditions look familiar to what it was trained on, even if the patient in front of the system is not the patient the data was built around.
I am not certain that the field will build the unlearning mechanisms it needs at the scale and speed required. Selective forgetting is technically harder than training. Regulators are still working out what to require, and most procurement contracts do not yet treat unlearning as a feature to specify. The honest uncertainty is whether retrospective fixing can keep pace with the loop it is trying to break.
But the posture from last week and the posture from this week now form a pair. Diagnose by tracing the noise back to the data. Fix by teaching the model to forget the part of the data that should not have been there. Neither move alone is sufficient. Together, they are the beginning of what an era-appropriate public health AI literacy looks like.
If you have encountered an AI tool in your practice that could not be corrected once a problem in its training data was identified — or one that could — I would be interested to hear what you saw. The procurement and contracting side of this question is where most public health practitioners will encounter it first, and your stories are how the series stays grounded.
As a member of the Springer Nature Author Affiliate Program, I may earn a small commission from purchases made through this affiliated link to my book. Support the author by checking out my textbook, Artificial Intelligence in Public Health: https://tidd.ly/4mH9389
