
Handing work to a colleague, or using an AI's summary as it stands, a person skips the effort of checking. In exchange, the loss is theirs if the other party turns out to be wrong. Is trusting another person the same as taking a risk? It is a risk, but it is not a bet. Iris Bohnet and colleagues compared, in six countries, outcomes decided by chance with outcomes decided by a person. When a person decided, participants would not accept the risky option unless the odds of success were better.
01Trust Resembles a Bet, but People Weigh Betrayal above Bad Luck
In 2008 Bohnet and her co-authors published experiments run in Brazil, China, Oman, Switzerland, Turkey and the United States. Each participant chose between two options. One was a sure payoff. The other was a risky option that might or might not turn out well.
The researchers set up two conditions. In one, chance determined whether the risky option paid. In the other, a second participant determined it. Prizes and probabilities were the same in both.
When a second participant decided, people refused the risky option unless the probability of success was higher. The money at stake was identical. Even so, participants counted losing through bad luck and losing through another person's choice as two different losses.
Handing work to someone on trust is taking a risk. The money is only part of what the truster stands to lose. So before handing a task to a person or an AI, estimating the chance of success is not enough. If the other party gets it wrong, which losses fall on me? Do I have any means of checking their work? Whoever delegates should write down the answers to both before starting.
02Mayer's Definition: Trust Is the Willingness to Take a Risk, Not the Act
Participants grew cautious once another participant controlled the outcome. Beyond money, which further stake does a person put up by trusting? The definition of the word is the place to look.
The management scholars Mayer, Davis and Schoorman defined trust in 1995. In their account trust is a willingness: the willingness to put oneself in a position to be harmed by another party's actions. The person who trusts expects the other to perform some action that matters to them. Whether the other can be monitored or controlled is not a condition of the definition.
Picture someone who passes a colleague's spreadsheet to a manager without rechecking the totals. If the totals are wrong, the one reprimanded is the one who passed it on. That person has accepted a position in which a colleague's work decides whether they are harmed. The three authors call this being vulnerable.
Making oneself vulnerable, they write, is taking risk. They then separate trust from the act. Trust is not taking risk per se; it is a willingness to take risk.
| Aspect | A bet | Trust |
|---|---|---|
| The outcome is decided by | Chance | The other party's intentions and actions |
| At stake | The amount wagered | The amount entrusted, plus the loss of having been betrayed |
| Means of checking | The probability is known | Monitoring and control are not required |
| In Mayer's terms | The act of taking a risk | The willingness to take a risk |
Separating willingness from act explains more situations. A person ordered by a manager to hand a task to a junior colleague hands it over, trusting or not. And a person who trusts a colleague fully hands over nothing if there is no task to give.
03Pre-Use Review of Promotional Materials Does Not Depend on Trusting the Author
When the other party can be neither monitored nor controlled, the person delegating has nothing to rely on except trust. That is the situation Mayer's definition describes. It follows that in a workplace with a procedure for checking the other party's work, less has to rest on trust. Promotional material for prescription medicines in Japan is a case where the system is built that way.
The Ministry of Health, Labour and Welfare's guideline on sales information provision requires that materials be reviewed by an oversight unit before they are used. Responsibility for approval lies with that unit and with senior management.
Mayer and colleagues wrote that a person who monitors the other party is not taking a risk based on trust. Pre-use review belongs in that category. The reviewer does not have to judge whether the author is an honest person. Whoever the author is, the material passes through the same review.
The existence of review does not mean the author is under suspicion. If faulty material reaches healthcare professionals, a great deal is lost. For work with that much at stake, a checking procedure is put in place by rule, in place of trust.
Material drafted by generative AI goes through the same review. The guideline does not sort materials by who wrote them or by which tool produced them.
04Facing a Person, People Demand Better Odds for the Same Payoff
Promotional materials have pre-use review as their checking procedure. Without any such procedure, a person decides alone whether to trust. Participants in Bohnet's experiment were more cautious when another participant decided the outcome than when placing a bet, and the experiment put a number on that caution.
The method was this. A participant decides whether to give up a sure payoff and accept a risky option. The researchers asked each one: at the very least, which probability of success would make you accept? The answer is called the minimum acceptable probability. The higher the answer, the more cautious the participant.
Across the two conditions, minimum acceptable probabilities were generally higher when another participant, rather than nature, determined the outcome. The four authors named this caution betrayal aversion. Their conclusion: people weigh a loss that results from another person's choice more heavily than the same loss from chance.
I have confirmed this from the paper's abstract only. Country-by-country values and the size of the gap between conditions are therefore absent from this column.
05With AI Too, Trust Guides the Level of Reliance
People weigh betrayal by a person above bad luck. So when the task goes to a machine, which way does reliance move? Research on automation and AI offers the following.
Trust guides reliance
Lee and See: users cannot fully understand complex automation, and in that situation their trust guides the degree to which they rely on it.
Over-reliance
Goddard and colleagues reviewed 74 studies. Users who leaned too heavily on automation failed to recognize new errors the system introduced.
Giving up too early
In Dietvorst's experiments, participants who saw the same mistake lost confidence in an algorithm faster than in a human forecaster.
Lee and See's paper appeared in 2004. As automation grows complex, users cannot completely understand the process inside the machine. At that point the degree of their trust in the machine guides the extent of their reliance. That was the two authors' synthesis.
The studies by Goddard and by Dietvorst report opposite failures. Goddard found over-reliance; Dietvorst found premature rejection. Goddard and colleagues retrieved 13,821 papers and examined the 74 that met their criteria. The systems studied were clinical decision support tools. Users often failed to recognize the new errors those systems introduced.
Dietvorst and colleagues ran five experiments. Participants watched an algorithm and a human forecaster make the same mistake. After seeing the error, they stopped trusting the algorithm, and their confidence fell faster than it did for the human.
People lean too heavily on automation, and people abandon algorithms too soon. Taken together, the two studies show that both failures occur.
06Exposure, the Weight of Betrayal, and the Means to Check Decide the Way We Delegate
Delegating to a machine, a person may rely too much or give up too early. Against both failures there are things the person delegating can check beforehand. I draw them from the work of Mayer and of Bohnet.
The first item to write down is the loss that follows if the other party is wrong. Mayer and colleagues defined trust as the willingness to be in a position to be harmed by another. Guessing the probability of success is no part of that definition. The probability is often unknown even to the person delegating. The possible losses can be listed on paper.
Hesitating before delegating is not necessarily a miscalculation. In Bohnet's experiment, participants demanded better odds simply because a person rather than chance controlled the outcome. People count the loss of being betrayed in addition to the money. When the hesitation is strong, it helps to put into words the loss one senses beyond the money. That adds material for the decision.
If the person delegating has a means of checking, the part that must rest on trust gets smaller. Review by an oversight unit before materials are used is one such means. Mayer and colleagues also describe a weakness of checking. Where control systems are strong, a person who behaves correctly is seen by others as having done so because of the controls, and trust in that person is slow to develop. Checking procedures are best reserved for work in which a mistake costs a great deal. That is my view.
07The More an AI Appears to Act on Its Own, the More Trust Can Rise or Fall
List the possible losses. Count the loss of betrayal. Hold a means of checking. These three steps come out of research on people trusting people. Mayer's definition, too, assumes a specific other party that acts with intentions of its own. When the task goes to an AI, does a person feel betrayed?
Puranam and Vanneste addressed this in a 2021 working paper. As an AI appears more agentic, they argue, people care more about whether it is benevolent toward them. People also anticipate a larger psychological loss should that AI break their trust. Making an AI appear more agentic, they conclude, may increase or decrease the trust humans place in it.
The weight of betrayal
When an AI decides the outcome, do people demand odds as favorable as they demand from a person? The material I have read does not settle it.
Trust in the designer
Where an AI does not appear agentic, Puranam and Vanneste argue, users ask about the ability and benevolence of its designers.
Theory and experiment
The argument is theory from a working paper. Experimental confirmation is a separate matter.
Puranam and Vanneste offer theory, not experimental results. In Bohnet's experiment the outcome was decided either by chance or by a human participant. The odds people would demand when an AI decides cannot be told from the material I have read.
- In 1995 Mayer and colleagues defined trust as the willingness to be vulnerable to another's actions, whether or not one can monitor or control them. Before estimating the odds of success, write down the losses you are exposed to.
- In Bohnet's six-country experiment, participants demanded better odds when another person rather than chance decided the outcome. People count betrayal as a loss separate from the money.
- Japan's guideline on sales information provision requires review by an oversight unit before materials are used. A checking procedure shrinks the part that must rest on trust, and the same procedure works for AI-drafted text.
Trusting another person is being willing to take a risk, and the risk is actually taken once work is handed over on trust. But the risk differs from a bet. The person who hands work over on trust accepts a position in which, without checking, another's actions decide whether they are harmed. And people weigh betrayal above bad luck of equal size.
Whether the task goes to a person or an AI, each means of checking reduces by that much the part that must rest on trust. Compare the AI's summary with the source. Re-add the totals in a colleague's table. Both look like doubting the other party. In practice both reduce the losses I am exposed to.
Before I hand work over, I write down two things: the losses that fall on me if the other party is wrong, and the means of checking I hold. Where a mistake costs a great deal, I use those means. What remains is the part beyond their reach and the work in which little is at stake. That remainder I hand over on trust.
- R. C. Mayer, J. H. Davis, F. D. Schoorman. An Integrative Model of Organizational Trust. Academy of Management Review 20(3), 709–734, 1995. (Definition of trust, its separation from risk-taking behaviour, remarks on monitoring and control systems)
- I. Bohnet, F. Greig, B. Herrmann, R. Zeckhauser. Betrayal Aversion: Evidence from Brazil, China, Oman, Switzerland, Turkey, and the United States. American Economic Review 98(1), 294–310, 2008. (Higher minimum acceptable probabilities when another person decides the outcome; confirmed from the abstract)
- J. D. Lee, K. A. See. Trust in automation: designing for appropriate reliance. Human Factors 46(1), 50–80, 2004. (Trust guides reliance when automation cannot be fully understood; confirmed from the abstract)
- P. Puranam, B. S. Vanneste. Artificial Intelligence, Trust, and Perceptions of Agency. INSEAD Working Paper 2021/42/STR, 2021. (Theory that a more agentic-seeming AI may raise or lower trust)
- B. J. Dietvorst, J. P. Simmons, C. Massey. Algorithm aversion: people erroneously avoid algorithms after seeing them err. Journal of Experimental Psychology: General 144(1), 114–126, 2015. (Confidence in algorithms falls faster than in human forecasters after the same mistake; confirmed from the abstract)
- K. Goddard, A. Roudsari, J. C. Wyatt. Automation bias: a systematic review of frequency, effect mediators, and mitigators. Journal of the American Medical Informatics Association 19(1), 121–127, 2012. (Systematic review selecting 74 of 13,821 papers; over-reliance and failure to recognize new errors; confirmed from the abstract)
- Ministry of Health, Labour and Welfare, Japan. Guideline on Sales Information Provision Activities for Prescription Drugs. 2018-09-25. (Materials must be reviewed by the oversight unit before use)
