The power to catch, and the power not to over-block── Why perfect detection is impossible in principle

Two kinds of regret always wait on my desk. The regret of something I missed, and the regret of something I stopped when I shouldn't have. Neither can ever be zero. This essay looks straight at that "can't."

01Two kinds of regret

Friday evening, two sticky notes sit side by side on my desk. One is about an ad I sent back last week. Reading it again, my objection had gone too far. The writer had it right; I was too cautious and stopped it. The other note is a single line about a drug's effect in an ad I approved. After I stamped it, something small snagged at me. That phrasing, was it really fine?

One is "over-blocking." The other is "almost-missing." These two always sit back to back. Chase one, and the other quietly waits behind you. However long you do this work, the pairing never goes away.

It would be wrong to think the two regrets come from my being bad at this, as if getting better would make one of them vanish. Reviewing needs two separate abilities from the start. One is sensitivity (= the share of truly problematic ads you correctly flag as "problem." The power to catch). The other is specificity (= the share of clean ads you correctly let through. The power not to over-block). Put names on them, and the two notes fall into place.

The left note is a specificity failure: I stopped something with no problem. The right note nearly became a sensitivity failure: I almost passed something that might have had a problem. The same "regret" lives in two different places. Lump them together as "bad day today," and you lose track of what to fix next.

So I make a point of looking at my regrets in two piles. Was it the power to catch that fell short, or the power not to over-block? Split them, and you see that what to blame is not your personality, but where you set the dial.

02Be strict, and you can't pass anything

As a newcomer, I believed something simple: strict looking is good reviewing. Stop anything doubtful. And yes, doing that, my misses dropped. The fear of passing something dangerous did shrink.

But a pile of send-backs grew beside my desk. Writers came over holding their files and slumped: "Again?" Read closely, and among them were things that never needed stopping. However much I raised the power to catch, I was shaving off the power not to over-block. Grip one hand tight, and the other slips through your fingers.

Other months went the opposite way. Not wanting to keep writers waiting, I set myself to pass things easily. The floor went quiet. The send-back pile disappeared. And yet at night, under the covers, my eyes stayed open. That one line, was it really fine to let slide? Raise the power not to over-block, and now the power to catch felt thin.

There is a real mechanism behind this feel. Reviewing has a decision line (= the placement of the standard beyond which you stop; also called the cutoff), the point that separates "problem" from "no problem." Move this line toward the strict side, and you catch the dangerous ones well. In exchange, the harmless ones get caught in the net too. Move it toward the loose side, and the harmless ones pass easily. In exchange, the dangerous ones slip through.

This is not about my hand being unskilled. Turn the single dial, and one side rises while the other falls. You cannot score full marks on both at once. I want to face this balance head-on (= the unavoidable exchange where making one thing better makes the other worse; the trade-off). Strictness is not free. Looseness is not free either. Both are paid for, quietly, behind the scenes.

What the young me lacked was not skill but this one table. I believed in a single road: get stricter, get better. In truth, where you place the line only swaps the size of the two regrets.

03Hearing the real signal inside the noise

One morning at the review desk, I read the same ad three times over. One line snagged. It looked like an overblown claim, and it also looked like something a footnote would make acceptable. I couldn't settle it as black or white. That discomfort stayed with me all day.

Later I learned it has a name. Signal detection theory (= a way of thinking about how to tell a real signal from noise). It grew out of postwar research into telling enemy aircraft from noise on radar. A small point of light appears on a dark screen. The radar operator hesitates. Is that an aircraft closing in, or a meaningless flicker made by wind and waves (= noise)?

What makes the point tricky is that both a real aircraft and mere noise show up at similar brightness on the screen. The spread of brightness when an aircraft is present (= the range of how brightly it shows) and the spread when there is none overlap just a little. A point that falls in that overlap can't be settled either way. The psychologists Green and Swets (= researchers who organized signal detection theory) pointed out this "overlap" and made it possible to treat the standard of judgment (= where you draw the line) separately from sensitivity. The hard part is not the sharpness of your eyes. It is the fact that, as long as an overlap exists, a band where anyone would hesitate remains.

Reviewing ads has the same shape. What I face is not a black-and-white "right answer." Problem ads and clean ads do not split into two clear peaks. Most sit in a gray band in the middle, slightly overlapping. When I saw that, the long self-blame loosened a little. I hesitate not because my ability falls short. I am looking at a band with an overlap, so of course I hesitate.

The ways a judgment slips are always two. The drop, where there really is a problem but I miss it, and the false catch, where there is no problem but I snag it. In the radar operator's words, one is failing to spot an aircraft, the other is calling noise an aircraft, a false alarm. As long as the gray band exists, you cannot zero out both mistakes. The more you brace to cut one, the more the other grows.

So I stopped asking myself, in front of an ad, "is this black or white?" Instead I ask where this point sits in the distribution. The closer a point sits to the middle band, the less I decide alone: I line up the grounds and borrow a colleague's eyes. Not to erase the hesitation, but to name what it is. That was my first step toward hearing the real signal inside the noise.

04One story: where to place the line

Once I asked a long-serving senior straight out: "Isn't there any way to zero out both the misses and the false catches?" He smiled a little and drew a single curved line on the paper in front of him. It rose from the bottom left, arced upward, then flowed off to the right. "All we do is move a point along this line."

That single line has a name. The ROC curve (= a single line showing the balance between few misses and few false catches). One axis is sensitivity (= the share of truly problematic ads you actually pick up). The other is specificity (= the share of clean ads you let through without needlessly snagging them). Sadly, these two do not get along.

Tighten your judgment to cut misses, and clean ads get caught too. Loosen your judgment to avoid false catches, and real breaches slip through. Raise sensitivity and specificity falls; raise specificity and sensitivity falls. Wherever you place yourself on the curve, the spot where both hit full marks lies off the line, out of reach.

Here I saw I had misunderstood something for a long time. When a colleague and I disagreed over a review, part of me felt we were competing over who was more able. But the senior's line made it clear. What we were really arguing about was not high or low ability. It was a difference of position, of where on the same single line we set the standard. One person sets it stricter, another looser. Both are on the line; they just stand in different places.

So where should you stand on the line? The answer is not written inside the ad. The weight of distorted information reaching a patient, against the loss of over-blocking honest, proper information. Only by weighing these two does the place get decided. Where dropping something costs dearly, you accept false catches and shift the standard toward strict. In the reverse case, you loosen it a little. Where to draw the line was set not by ability, but by which mistake we fear more.

After I knew this single line, the color of arguments in meetings changed. "You're too soft," "you're too strict," is usually just talk about the line's position. Then what to ask is not the other person's competence. On this ad, which mistake should we fear more? Once that lines up, the standing position settles on its own. Rather than wearing ourselves out hunting a point where both are perfect, deciding together where to stand on the line got us much further forward.

05A miss or an overreach: which weighs more

One morning, two ads (= the printed drug explanations handed to doctors and patients) sat on my desk. One wrote the effect a little wider than what was approved. The other was just worded somewhat strongly, but its content was correct. I can put a red flag on either. But the flags don't mean the same thing. Miss the first, and a wrong story about the drug's effect flows out to the field. Stop the second, and I only ask the writer, "please fix this once more." This lopsidedness, this difference in weight, sits at the core of my work.

Review judgments miss in two ways. One is a miss (= passing something as no-problem when it really is a problem; a false negative in statistics). The other is an overreach (= stopping something as a problem when it really isn't; a false positive). These two are on a seesaw. Move the line to cut one, and the other grows. You cannot make both zero at once. So the reviewer is always choosing where to put the line.

Then should the line go in the middle? I don't think so. The middle is the right place only when the harms of the two mistakes weigh the same. Here they don't. The harm of a wrong effect line reaching doctors and patients, against the harm of asking a writer for one round of rework. The former is hard to undo; the latter is fixed in a few hours. If the harms aren't equal, putting the line in the middle is, if anything, unfair.

So I set the line slightly toward the catch side. Not neutral, just tilted a little. It's close to how health screening works. A test is tilted toward flagging just in case, rather than overlooking (missing a disease), because you can examine more closely later. Pick up a bit more on the undoable side, thin out the hard-to-undo side. There is a reason for this tilt. Since the two mistakes don't weigh the same, you must not allow them in equal numbers.

But this does not mean "stop anything doubtful." The tilt is only slight. As the next section describes, tilting too far carries a separate price.

06The quiet price of over-blocking

I knew one senior who tilted too far. He sent back nearly everything. The wording is strong, too much of a flat claim, this one line bothers me. His reasons were always right. The ads did become safe. For half a year, not one dangerous sheet left his section. On the numbers, he was a model reviewer.

But after half a year, the writers began letting his objections go in one ear and out the other. "Here comes another one." "He'll say fix it all anyway, so don't bother worrying from the start." His red flags had turned from a warning into background color. Then one day, on a truly important one, a sheet whose way of writing the effect was clearly off, he stuck a flag on it in his usual manner. The writer, in his usual manner, let it slide. The important objection was buried under the pile of his own small objections.

I think of the boy who cried wolf too often. He didn't lie. There were just too many cries. So on the day the wolf really came, no one moved. An overreach looks harmless in itself. Just stopped it, just had it fixed. But stacked up, it slowly wears away an unseen asset called trust. Each objection lowers the weight of the next.

This ties to what I wrote before about "two people covering one judgment," one person's slant covered by another's eyes. The more someone tilts, the harder it is to notice their own tilt. That is exactly why a second pair of eyes is needed. It is continuous with the righteousness sickness I covered in the mind series (= the state of cornering someone too far, using correctness as a shield). Stopping looks like correctness. But when the amount of correctness grows too large, correctness itself grows light.

I tilt the line toward the catch side. That doesn't change. But by the amount I tilt, I count my own red flags. Were today's objections truly the number needed? Not one case may be left buried. So, to keep the important one from being buried, I don't greedily stop the rest. The power to catch and the power not to over-block are the two ends of one single line.

07Searching within a low-problem pool

Ads pile up on my desk every day, explanatory materials and draft advertisements for drugs. I look at each one and check for over-the-top wording or claims with no grounds. Here is something I want to say honestly. Most of them have no problem to begin with. The makers know the rules and write with care.

This fact, that "problems are few from the start," slowly changes how my work looks. Think in numbers. Suppose that out of a hundred sheets, five truly need fixing. The other ninety-five are ads you can simply pass. This five-to-ninety-five ratio is called the base rate (= the "original ratio" of how much of the pool has a problem).

When the base rate is low, an awkward thing happens. However carefully I look, my judgments always mix in drops and jumped guns. Among the things I picked up as "problem," a surprising number turn out, on second look, to have had no problem, false catches. This is not because my hand is bad. The fewer problems in the original pool, the more innocent items the "problem" net scoops up. The nature of the pool makes it so.

A tool for rethinking probability from behind works here. It's Bayesian thinking (= a way of recalculating "how likely something truly is" after seeing a new clue, taking the original ratio into account). It sounds hard, but the core is plain. The chance that a sheet I felt was "suspicious" truly has a problem is not set by the sharpness of my hunch alone. It is strongly dragged by how many problems are mixed into the original pool. In a low-problem pool, the same feeling of "suspicious" tips toward being wrong.

So I don't carry the line-drawing alone. I run what I think is suspicious past a second pair of eyes. When the second person looks, most of my false catches settle there. And sometimes the second person picks back up something I was about to pass. Doubling the net is not a detour. In work where the base rate is low, the second pair of eyes is exactly what cuts both false catches and misses.

Finally, let me put an easily forgotten premise into words. I cannot tell everything apart perfectly on my own. That is not giving up. It is honestly deciding where to draw the line and who to share it with. Letting go of perfection is not the same as throwing the work away. Having accepted the premise of searching within a low-problem pool, I go on, together with one more person, looking at each sheet, one at a time, today too.