The Trap of Confirmation Bias and Negative Marking — Putting One's Own Cognitive Habits on the Design Table

"I am making a judgement free of bias" — this is the most universal human conviction that psychology has spent half a century refuting. Confirmation bias, negative marking, framing, assimilation, conformity — these are not the defects of particular individuals but structures embedded in the cognition of every human being. In the field of regulatory review, we frequently render judgements without noticing our own biases. This volume looks at the identity of those biases in light of six traditions — Tversky & Kahneman, Wason, Lord, Asch, Kahneman's dual-process theory, and Zhu Xi's ge wu zhi zhi — and assembles the practice of placing biases that cannot be erased on the design table.

01The Cognitive Habit Is Invisible to Oneself

The hardest part of cognitive bias is the structure that one's own bias is invisible to oneself. The bias of others is easily seen. The moment one evaluates, "that person is in confirmation bias"; "that person is using negative marking," one is at the same time in the same bias. There is research calling this the bias blind spot (Pronin, Lin & Ross, 2002).

In the pharmaceutical field, the moment one feels "I am reviewing fairly" is the moment one should be most cautious. Bias cannot be erased; but the very structure that bias exists can be brought into awareness. What this volume treats is the practice of that awareness and design. Not erasing bias, but assembling judgement on the premise of bias — this is the most realistic response that contemporary cognitive science has reached.

02Tversky & Kahneman — The Discovery of Heuristics and Biases

In the 1970s, the psychologists Amos Tversky (1937–1996) and Daniel Kahneman (1934–2024) showed repeatedly, through experiments, that human judgement relies heavily not on logic but on heuristics. Their 1974 paper "Judgment under Uncertainty" became the starting point that gave birth to behavioural economics.

"People rely on a limited number of heuristics when they intuitively evaluate probabilities and frequencies. These heuristics are useful in many cases but often produce grave and systematic errors." — Tversky & Kahneman, "Judgment under Uncertainty: Heuristics and Biases" (1974)

The three representative heuristics they found: the representativeness heuristic (judging by how much something resembles a typical example); the availability heuristic (judging frequency by ease of recall); anchoring and adjustment (judgement is pulled by the first figure presented). These are evolutionarily useful cognitive shortcuts, but in contemporary complex judgement they bring in systematic errors.

In the field of regulatory review: "this requester had problems before" (availability → judgement is pulled by the past impression); "for this product category, this is typical" (representativeness → exceptions are overlooked); "the first proposed correction becomes the centre" (anchoring → other correction proposals are overlooked). Recognise bias as "a resident living in every heart" — this is the starting point Tversky & Kahneman left.

03Wason's Selection Task — The Discovery of Confirmation Bias

The British psychologist Peter Wason (1924–2003), in the 1960s and 1970s, conducted many experiments illuminating the habits of human reasoning. Especially famous is the four-card problem (Wason selection task). To test the rule "if a card has a vowel on one side, the other side has an even number," which cards should one turn over? He showed that many subjects choose cards that confirm the rule and overlook cards that would falsify the rule.

"Humans overwhelmingly gather information that confirms their hypothesis and rarely gather information that would falsify it. This is a general habit of reasoning, not the defect of particular individuals." — paraphrase of Wason, "Reasoning about a Rule" (1968)

This is the classical discovery of confirmation bias. We unconsciously gather information that supports our hypothesis and unconsciously avoid information that contradicts it. In the field of regulatory review, after one intuitively judges "this case has a problem," one often looks for information confirming the problem and is less apt to notice information falsifying it. Conversely, after one intuits "this case is fine," the same bias operates in the direction of overlooking the problem.

Applying Wason's wisdom to the field: consciously ask "if my hypothesis were wrong, what information should exist?" Turn the eye, just once, in the direction of falsification, not confirmation. This alone changes the quality of judgement.

04Charles Lord, "Biased Assimilation" — The Same Evidence Strengthens Belief

The American social psychologist Charles Lord and colleagues published a shocking experimental result in 1979. Have both supporters and opponents of capital punishment read the same evidence, with both sides represented. The result: supporters became more supportive after reading, and opponents became more opposed — the same evidence polarised both positions further.

"Both sides rated highly the evidence supporting their own position and unduly low the evidence opposing it. After reading the same study, the gap between the two sides actually widened." — paraphrase of Lord, Ross & Lepper, "Biased Assimilation and Attitude Polarization" (1979)

This is called biased assimilation. What Lord's finding shows is that the naive expectation "if shown the evidence, people will think calmly again" is frequently betrayed. Rather, evidence becomes material that strengthens existing belief.

In the pharmaceutical field, when judgement is divided about one's own conclusion, even when both parties read the same regulatory documents, each position can strengthen. This is not the other party's bad faith but the structure of human cognition. Applying Lord's wisdom: ask, before judging, "am I reading this evidence in a way favourable to one side?" The calm reader does not exist from the start. There is only the work of trying to be a calm reader.

05"Negative Marking" as a Japanese Cognitive Culture

From the Eastern lineage, one more cognitive habit. The negative-marking (genten-hō, 減点法) approach, deeply rooted in Japanese organisational culture. This is not a classical academic concept, but as a cognitive pattern of Japanese society it has been widely observed.

"In Japanese organisational culture, evaluation is often 'the remainder after subtracting faults from 100 points of good.' By contrast, additive evaluation is 'the sum built up from the good points.' The former gives rise to a culture of safety and quality, but pushed to excess it produces inhibition and defence." — gist from empirical observations repeatedly made in organisational culture studies

Negative marking itself is not evil. In work like regulatory review, where the cost of missing an error is great, the negative-marking lens is indispensable. The problem is that when negative marking is always operating, the other's good points or understanding of intent automatically drops from the field of view. In an email pointing out five corrections, if not one line touches the other's effort or what went well, the receiver's heart enters defence. Even technically correct findings, when composed only of negative marking, generate within the receiver the interpretation "to this reviewer I am nothing but 'an object to be marked down.'"

One need not discard negative marking. Consciously mix negative-marking findings with additive recognition — this alone greatly changes the quality of transmission. This is a discipline and a practice.

06Kahneman's Dual-Process Theory — System 1 and System 2

Kahneman, in his late major work Thinking, Fast and Slow (2011), organised human thought into two processes.

"System 1 generates impressions and feelings. System 2 carries the critical examination that transforms them. The problem is that System 2 is lazy and often simply ratifies System 1's judgement." — paraphrase of Daniel Kahneman, Thinking, Fast and Slow (2011)

What Kahneman shows is that to reduce bias one has no choice but to consciously activate System 2. In the field of regulatory review, against the System 1 reaction "this is a problem" rendered instantly, System 2 asks again "really? Is there another interpretation?" Activating System 2 requires energy, so it cannot be used always. The habit of consciously calling it forth only at important judgements — this is the practice proposed by contemporary cognitive science.

07Zhu Xi's "Ge Wu Zhi Zhi" — The Round Trip of Observation and Conceptualisation

From the Eastern lineage, a classic that resonates with Kahneman's dual process. The Southern Song Confucian Zhu Xi (1130–1200) perfected the thought of ge wu zhi zhi (格物致知). In his interpretation of The Great Learning, he argued:

To extend knowledge lies in investigating things. When things are investigated, knowledge is then extended.
— gist from Zhu Xi's commentary on the Daxue zhangju: zhi zhi zai ge wu, wu ge er hou zhi zhi

Zhu Xi's ge wu zhi zhi is the epistemology that knowledge can only be obtained by carefully investigating things one by one. Not the instant judgement of System 1 type, but the careful observation of System 2 type. Zhu Xi's wisdom bridges straight from 12th-century China to 21st-century cognitive science. In the field of regulatory review: before issuing one correction request, insert just once the work of "investigating" the object one step deeper (observing and reconceptualising). This alone filters by one step, through System 2's examination, the bias originating in System 1.

08Solomon Asch — Group Conformity Pressure as Another Bias

The American social psychologist Solomon Asch (1907–1996), in a series of experiments from 1951 to 1956, showed the force of group conformity pressure. Even on a simple task of comparing clearly different line lengths, when all confederates around the subject gave wrong answers, about one-third of subjects went along with the wrong answer.

"Even a simple perceptual judgement is greatly distorted under social pressure. The group's judgement frequently overwhelms the individual's accurate cognition." — paraphrase of Asch, "Effects of Group Pressure upon the Modification and Distortion of Judgment" (1951)

In the pharmaceutical field, in a deliberation in which all the other members judge "no problem," even when one's own cognition catches "there is a problem," it becomes hard to voice — this is precisely Asch's discovery. Conformity bias, alongside confirmation bias and negative marking, is a powerful force distorting the quality of organisational judgement. What is important in Asch's experiments is that he also showed that even a single allied dissenter dramatically lowers the conformity rate. The mechanism to secure, organisationally, the "one who asks questions" is the applied form of Asch's wisdom.

09Confirmation Bias, Negative Marking, and Conformity Pressure in the Pharmaceutical Field

Let us bring the abstract down to the field. Typical biases the reviewer meets within a day of judgement.

These are not "the cognitive habits of a particular individual" but the very structure of human cognition. Rather than trying to erase them, understand as structure and place on the design table. This is the core of the volume.

10Four Practices for Placing Cognitive Bias on the Design Table

To bring the same place to which the six traditions pointed down into daily practice, four practices are set here.

Cognitive bias cannot be erased. This is the human structure that cognitive science has confirmed over half a century. But understanding bias as structure and placing it on the design table is possible. Tversky & Kahneman, Wason, Lord, Asch, Kahneman's dual process, Zhu Xi's ge wu zhi zhi — six traditions pointed, in different words, to the same place.

"I am making a judgement free of bias" — the very moment one feels this is the moment to be most cautious. Because that feeling itself is one of the deepest workings of bias. Not to erase bias, but to judge honestly together with bias — this is the realistic practice cognitive science has reached.