This figure shows whether a past decision may be judged by today's standard. Input: a past decision, the procedure available then. Stage one: the 1962 US law reassessed drugs since 1938, judging evidence and procedure. Stage two: a 1975 study and a 2021 replication, 890 participants, effect 0.60, show memory drifting toward the outcome once known. Stage three: keep what was seen, checked, and left unknown. Stage four: standards keep moving, as Colorado repealed a 2026 duty and the EU AI Act applies from 2027. Bypass: judging by result blames luck and nothing improves. Frame: standards keep shifting backward and forward. Output: repairing procedure, not blame, changes the next decision.
Image abstract — the whole article on one page (click to enlarge)

In 1962, the United States added proof of effectiveness to the conditions for marketing a drug, and required every medicine released since 1938 to be assessed again against the new standard. Material drafted today with generative AI sits on the same side of that reckoning. May I judge the decisions I made in the past by the standard I hold today? What can be judged is the procedure that was available then, not the outcome that surfaced later.

01A past decision can be judged by the procedure that was available, never by the result

When someone brings up an old decision of their own and regrets it, two things are being mixed together. The procedure available at the moment of deciding, and the result that became visible afterwards.

The first can be judged. Which documents did I read that day? Whom did I check with? Which part did I carry forward while still not knowing? Those were all open to me at the time, so the quality of the choosing can be discussed. The second cannot. Take the result as your measure and you are assessing luck rather than judgment.

What the United States did in 1962 was judgment of the first kind. It reached a new standard backwards into the past, and what it looked at there was not the character of the officials who had granted approval, but the evidence and the procedure that had supported them. The same distinction works when a person looks back at their own history. That is my view of it.

02The 1962 US law sent every drug marketed since 1938 back for review

Take one case where judging by procedure was carried out at national scale. The 1938 law required proof of safety before a drug could be sold. Whether it worked was not among the conditions.

The 1962 amendment added proof of effectiveness. Had it stopped there, the story would concern only drugs yet to come. Where it went further was in requiring the agency to assess the effectiveness of every drug introduced since 1938. A condition was added, and the added condition reached backwards.

What the agency looked at when it reached back is the point of this story. It did not reopen the question of whether an official two decades earlier had been right, or diligent, or well intentioned. It read the surviving evidence against the new standard and decided whether that evidence supported effectiveness. The object of assessment was the record, from first to last.

1

What fell within it

For drugs marketed since 1938, the evidence supporting effectiveness and the procedure that had accepted it.

2

What fell outside it

The motives and the character of the officials of the day. The determination was made against the record.

3

The condition added

Alongside safety, proof that the drug worked became a condition of sale.

Figure 1 How the standard moved from 1938 to 1962
1938 proof ofsafetycondition beforesale1962effectiveness…a condition addedReachingbackProcedure andevidence…records, not people1938 proof of safetycondition before sale1962 effectiveness tooa condition addedReaching backProcedure and evidence assessedrecords, not people
What reached backwards was the standard, not a search for culprits. The object of assessment stayed the procedure and the evidence.

03The same reckoning is under way for materials drafted with AI and reviewed later

Standards moving and then reaching back over past decisions is not a story confined to sixty years ago. It is still going on.

Promotional material drafted with generative AI is read under new rules after it has been published. Personnel decisions made with AI screening are in the same position. In Illinois, an amended Human Rights Act took effect on 1 January 2026 that tests AI-assisted employment decisions by the bias in the outcome. Systems already installed and running moved to the side that gets examined under a standard arriving after them.

At that point, what was said at the time does not count as evidence. What remains for the reader is only what was written at the time. Which version of the model received which input, which parts of the draft a person changed, and why the unchanged parts were left alone. Where none of that was written down, the people reading it later see the result and nothing else.

1

Material drafted with AI

Without a record of which model version received what, a later reading cannot be shown the procedure at all.

2

Decisions using AI screening

In Illinois, from 1 January 2026, systems already in place joined those tested by the bias in their outcomes.

3

One's own old decisions

Unlike a workplace record, personal memory carries no version and no date. This is where rewriting happens easily.

04Once people know the outcome, they believe they saw it coming

What happens to the person doing the rereading is worth establishing experimentally. Memory, left alone, does not serve as material.

Fischhoff showed in 1975 that people told an outcome raise their estimate of how likely that outcome was. Hearing that something happened, they report having expected it. In 2021, Chen and colleagues replicated the experiment with 890 participants and reproduced the same bias. The effect size reported was 0.60.

So anyone setting out to judge their past self is working from a memory that has already been rewritten. The recollection of what one expected at the time drifts toward the result at the moment the result is learned. The drift is invisible from inside. That is why the feeling of "you should have known then" arrives whether it is aimed at oneself or at a colleague.

Figure 2 What follows from learning the outcome
The outcome is learnedone piece of informationEstimated likelihoodrevised upwardThe earlier forecastmemory is rewrittenJudgment of otherssame measure appliedThe outcome is learnedone piece of informationEstimated likelihoodrevised upwardThe earlier forecastmemory is rewrittenJudgment of otherssame measure applied
None of the branches is visible from inside. That is why “you should have known” lands on oneself and on others alike.

05Only the record of the evidence seen and kept can be judged

If memory drifts toward the result, the remedy is to keep something that does not drift. Only what was written down at the moment of deciding serves that purpose.

Three items are enough. What was seen. Whom it was checked with. Which parts remained unknown. Write those three down, and when the standard later changes, it can be shown that the change was on the side of the standard.

The writing itself can be plain. In materials review, a line each is enough: the original paper behind this citation was not consulted; the conditions behind this figure could not be read off. No formal apparatus is needed. Write down, with a date, the thing that could not be resolved and the reason for proceeding anyway, and a reader years later can reconstruct the field of view of the day.

The third is the one that goes unwritten. Recording a decision, people set down what they relied on and quietly leave out what they could not resolve. Years later the file reads as a record of confidence, because the hesitation of the day survives nowhere in it.

Figure 3 Four things to leave for whoever reads it later
Keep what wasseenby version and dateWrite theunknownsleave themunresolvedNote thestandard's…Explain by thedifferencethe standard is whatmovedKeep what was seenby version and dateWrite the unknownsleave them unresolvedNote the standard'sversionExplain by the differencethe standard is what moved
The second is the one that drops out. A record with no unknowns in it reads later as a record of confidence.

06Standards move, so keeping the record outlasts assigning blame

Suppose the evidence seen and the questions left open are both written down. What is the file for? Three things can be said at once.

The first is that judging by result means judging luck. The body of argument known as moral luck deals with situations where assessment turns on matters outside the agent's control. If two people follow the same procedure and only the one who drew the worse outcome is blamed, procedure is not what is being blamed.

The second is that the memory serving as material has already moved. That is what the 1975 experiment and the 2021 replication established. Before any session of blame opens, recollection has been tidied to match the result.

The third is that standards are themselves withdrawn. Colorado passed a law in 2024 requiring reasonable care of those handling high-risk AI. Then, on 14 May 2026, the state enacted a replacement removing the broad discrimination duty and the annual impact assessments it had imposed. To hold today's standard against the past as a final measure is to use a measure that was itself rebuilt inside two years.

TestJudging by resultJudging by procedure
Material requiredWhat became known laterWhat was at hand when deciding
Conclusion producedWho was at faultWhat to change
Effect next timeSame luck, same resultChange the procedure, change the result

07Systems being built now will be read under the standards of 2027 and after

If the standard is rebuilt inside two years, it is worth looking at the shape of the next one. At least one of them already has a date.

European rules require AI classified as high-risk to be built so that events over the lifetime of the system are logged automatically. That provision applies from 2 December 2027. Systems being designed now move to the side that will be read again under the standard in force from that day.

What survives then is not the explanation offered at the time. It is the record kept at the time. Everything written here comes back to that single point. One thing is unsettled, though. Who may read those logs, and how far, has not been decided. A duty to create is not a duty to show.

When I look back at my own old decisions, I go in the same order. I do not look at the result first. I set out what was in front of me on the day, and only then separate what was missing: the material, or the procedure. Open a session of blame and the work stops there. Make it a session for repairing the procedure and the next decision changes.

Key Points ── 3 to take away
  1. The 1962 US law added proof of effectiveness and sent every drug marketed since 1938 back for assessment. What it reached back for was the evidence and the procedure, not the people.
  2. A 1975 experiment showed that knowing an outcome raises the probability people assign to it, and a 2021 replication with 890 participants found an effect size of 0.60. Memory has already moved before the judging starts.
  3. Colorado enacted a replacement on 14 May 2026 removing the duty and the impact assessments it had earlier imposed. Today's standard cannot be the final measure, because standards are themselves withdrawn.
Closing

Today's standard may be turned on a past decision only where the procedure is concerned. Judge the outcome and you judge luck, and the next decision changes nothing.

So what deserves keeping is not a record of confidence. It is a record of what could not be resolved. If anyone rereads your decision in a few years it will usually be you, and what helps at that point is finding the reason you hesitated written down.

Sources & references
  1. U.S. Food and Drug Administration. Part III: Drugs and Foods Under the 1938 Act and Its Amendments. (The 1962 amendment added effectiveness as a condition of marketing and required assessment of all drugs introduced since 1938.)
  2. Stanford Encyclopedia of Philosophy. Moral Luck. 20 January 2025. (Assessment turning on factors beyond the agent's control.)
  3. Chen et al. Retrospective and prospective hindsight bias: Replications and extensions of Fischhoff (1975) and Slovic and Fischhoff (1977). Journal of Experimental Social Psychology, 1 September 2021. (Replication with 890 participants; effect size 0.60.)
  4. EU Artificial Intelligence Act (article commentary site). Article 12: Record-Keeping. 12 July 2024. (Automatic logging for high-risk systems, applying from 2 December 2027.)
  5. Hinshaw & Culbertson LLP. Illinois Adopts AI-in-Employment Regulations: What Employers Need to Know for 2026. 26 February 2026. (The amended Human Rights Act took effect on 1 January 2026.)
  6. AI Compliance Atlas. Colorado AI Act (SB 24-205): 2026 Status & SB 26-189. 14 May 2026. (Repeal and reenactment, effective 1 January 2027.)