AI Safety: Four Parties Asked for Answers at Once── Where rules and court cases overlap, and the records pharma should keep
Download the video (MP4, 14 MB)
Should we leave AI safety to the companies that build it?By the end, you will see why safety is turning into a duty that outsiders check through records. Four very different parties moved. A national office that protects consumers. Regional governments. An official who watches a city's money. And an authority overseas. Pharma has long made life-and-death decisions under strict rules. If we bring AI into that work, we need to know whom we can count on. We will look at money, machines that fail at tasks we give them, deals to find medicines, and the grounds behind image readings. What comes back to us as users?
01Regulation and lawsuits converged
Source Axios ／ Santa Cruz Sentinel ／ Crain's New York ／ Technology Org
We start outside the developers. Until now, builders explained safety in their own words. Here we ask who outside the company began to demand that explanation, and how far.
| Actor | Target | What was reported |
|---|---|---|
| FTC | OpenAI, Anthropic | Demanded documents on recent safety incidents |
| 8 states | OpenAI | Sued, alleging harm to children and deception of adults |
| NYC Comptroller | Anthropic | Urged an IPO delay, citing “myriad safety concerns” |
| EU | OpenAI, Anthropic | Reported first enforcement case under the AI Act |
Axios, Santa Cruz Sentinel, Crain's New York, Technology Org (10/15)
These parties could hardly differ more. A consumer watchdog, regional governments, a city official guarding public money, and an overseas body. Each acted for its own reason, and their timing overlapped. One alone could be dismissed as one firm's lapse. Together, they suggest a shift. Safety is moving from a goal a firm sets itself to a duty enforced from outside.
The heaviest move is the watchdog asking for paperwork. Handing over papers means showing in writing what happened and how it was handled. A spoken promise is no longer enough. Safety work is now judged by whether a trail exists.
For pharma teams, this is not a distant story. I expect work we hand to machines to face the same question one day. Which tool did what, what went wrong, and how was it fixed? Keep that trail now, and you can answer when asked.
Outside scrutiny also changes how investors see these firms. Next, what the figures shown before a stock listing tell us.
02Revenue ahead of the IPO
Source Financial Times ／ Yahoo Finance ／ Investor's Business Daily ／ Bloomberg.com
The very company under scrutiny showed investors strong numbers. How did the market weigh earning power against doubts about safety? That is our question here.
| Item | What was reported | Source |
|---|---|---|
| Quarterly revenue | $16.8B; investors told it rose nearly 50% | Financial Times |
| Cash flow | First positive cash flow among AI model developers | Yahoo Finance |
| AI stocks | Fell even after the revenue reports | Investor's Business Daily |
| Analysts' view | A tailwind for the IPO, but no lift for AI stocks | |
Revenue is for Jul–Sep (Yahoo Finance headline)
Sales grew sharply, and more cash reportedly came in than went out. In a field that pours huge sums into development, that is striking news. Yet shares of related firms did not rise. The market did not read one firm's strength as good news for the whole sector. The numbers and the market's reaction point in different directions.
Why the gap? Proof of earning power has not erased doubts about safety. Some even urge a delay just before the listing. So I think investors are pricing in the risk behind the strong results.
From the pharma side, this offers a clue for choosing tools. When you pick a vendor's machine, comparing performance and price is not enough. How exposed the vendor is to regulators and courts decides whether you can use the same tool on the same terms tomorrow. A tool from a firm being sued may see its terms of use change overnight.
Then, when we let that tool act for us, who answers for its mistakes? Next, a machine trusted to make payments.
03Who is liable when AI shops for you
Source ServiceNow ／ The Verge ／ Cybernews
Machines now press the buttons that people once pressed one by one. When a mistake happens there, who takes the loss? Convenience came first, and the answer is still open.
The path itself is simple. A person hands over the task, and the machine goes as far as paying. Further along, errors or leaks can occur. On that last step, the head of one developer said flatly that the firm would cover the losses. Responsibility is starting to shift from users toward builders.
Yet a case with that machine and a fake email reportedly showed the danger of delegating. A promise to repay losses matters, but it does not prove accidents won't happen. Meanwhile, another firm widened tools that restrain how machines act inside businesses. Protection by promise, or protection by design.
In pharma, machines will one day place orders. In the contract, check two things before any promise of compensation. Can you stop the machine, and is there a log of what it did? Money can repay a loss. It cannot undo the effect on patients or partners.
Words of promise and money that actually moves are different things. Next, how to read the sums attached to a deal to find medicines.
04Three amounts in one AI drug discovery deal
Source AstraZeneca ／ Reuters ／ BioSpace ／ Yahoo Finance
One single deal, yet different reports gave different sums. The question is not how big the sum is, but what it is made of.
| Type of amount | Reported amount | Nature |
|---|---|---|
| Headline amount | $1.86 Billion | Big pharma deal with insitro. Bloomberg and others |
| Ceiling amount | Up to $1.5B | Not a fixed amount. Paid depending on results |
| Upfront amount | $40M | Signing payment. The part certain to be paid |
| Raised in an IPO | $150M | AI drug discoverer Iambic. Separate from partnership payments |
Targets: liver and heart disease (MASH, cardiomyopathy, etc.). Amounts as written in the original reports
The largest figure came from some headlines. Below it sits a cap that is paid only if results come. What will surely change hands looks like only the small first payment. The gap between them is more than an order of magnitude. Money another firm raised by listing is shown too, but partners do not pay it. Remember only the headline figure, and you will misjudge this deal.
The capped sum is paid in pieces, each time the research clears a stage. Miss a stage, and nothing is paid. For the payer, it avoids paying for results that never come. The big headline figure is just the maximum, if everything goes well.
Pharma teams already read deals this way in daily work. Yet when AI is involved, the headline figure tends to spread as is. When you explain an investment inside your company, separate the first payment from the cap. That alone keeps expectations in check.
After money comes the question of how sound the machine's own judgment is. Next, how a machine that reads medical images handles its grounds.
05Evidence in AI that reads X-rays
Source X-Ragent: Tool-Augmented Reinforcement Learning for Evidence-Grounded Chest X-Ray Reasoning
A machine that reads medical images is not trusted just for giving the right answer. If a right answer rests on the wrong grounds, it will miss with the next patient. Here, a team tries to teach a machine to choose its grounds.
The study splits two judgments. One is whether to go looking for support. The other is to leave out values that do not fit the image. Human doctors also distrust a reading that clashes with the picture in front of them. The machine learns that common sense piece by piece.
Gate reward only if correct
Scores for measurement consistency and tool cost go only to trajectories whose diagnosis was correct.
Training material
Built Rad-CoT, combining tool-call logs with report-style reasoning, on top of the Qwen3-VL-8B base model.
Not yet peer reviewed
An arXiv preprint. The abstract does not say which comparison the reported improvements are measured against.
Points for the steps along the way go out only when the final answer is right. Scoring the process behind wrong answers might teach plausible mistakes. Still, outside experts have not yet checked this paper. Its summary also omits what it beat.
For pharma, this overlaps with drug safety work. What matters is less the conclusion than whether the machine can show what it relied on. If its grounds can be traced, that trail helps the people who check it and the regulators who ask about it.
While research asks for grounds, machines are already at work in the field. Next, the places where medicines are found and delivered.
06On the pharma front line
Source Google DeepMind ／ Pulse 2.0 ／ Quiver Quantitative
So far, we have heard from builders and regulators. How far have machines entered the work of pharma itself? Today's news shows part of the answer.
Antimicrobial peptide search
Google DeepMind's official blog described using Google Antigravity to speed up the search for antimicrobial peptides.
AI platform for supply chains
ParkourSC acquired Quartic. The aim is an AI platform for life sciences supply chains; the headline gives no price.
Candidate from a joint program
From an AI-driven collaboration with Recursion, the partner drugmaker reportedly exercised its option on the first development candidate.
Today's moves are spread from the start of drug making to the point where medicines reach patients. Finding candidates, deciding whether to take one over, and moving medicines. The machine is no longer a tool only for the lab. The weightiest is a partner company deciding to take over a candidate born from shared research. The pharma side judged a machine-picked candidate worth its time and money.
Taking it over means the drugmaker uses a choice it secured in the contract. From then on, the development ahead is its own job. So the decision reads as a sign that the machine's proposal is worth the investment. A machine-picked candidate has passed one human test.
Machines entering the delivery stage also deserve attention. If supply stops, medicines stop reaching patients. In research and supply alike, the same question remains. Who checks the machine's judgment, and who can stop it?
Behind the eye-catching news, the question of who stops the machine remains. Last, a few stories that are easy to miss.
07What is being overlooked
Source The New York Times ／ South China Morning Post ／ Android Authority
Our last stories look small next to rules and court cases. Yet they sit closer to users' daily work. Because they hide behind the big news, they may hit tomorrow's work harder.
Wrongdoing by an insider
The NYT reported that an OpenAI engineer charged with planting a bug had been advising the Saudi side.
An AI that stops, and refunds
DeepSeek had another major outage of about 5 hours and gave refunds and compensation. Reportedly at least the 18th this year.
Notice of a 4x price
Google reportedly emailed Gemini subscribers about a change that makes their price 4 times higher.
A flaw planted by an insider, a service that keeps going down, a price that suddenly leaps. None of these is about how smart the machine is. Still, they hurt those who handed over work. Money back after each outage does not restore the work lost while it was down. Wrongdoing from inside cannot be blocked by outer defenses. Users cannot see a builder's internal controls. Safety is more than correct answers. I think it also means staying up, staying loyal, and keeping terms steady.
What if a tool built into pharma work stops one morning? If the price changes, can you move to another tool? How far can you trust the people inside the builder? Users can settle these plans today, without waiting for news on regulation. With answers ready, work goes on. It is the same thinking as preparing for a break in drug supply.
Outside rules and failures in front of us led to the same question. Users, too, cannot give up the job of checking safety.
Will the FTC's scope and the IPO timing be revealed?
Open the full transcript
Intro
Should we leave AI safety to the companies that build it?By the end, you will see why safety is turning into a duty that outsiders check through records. Four very different parties moved. A national office that protects consumers. Regional governments. An official who watches a city's money. And an authority overseas. Pharma has long made life-and-death decisions under strict rules. If we bring AI into that work, we need to know whom we can count on. We will look at money, machines that fail at tasks we give them, deals to find medicines, and the grounds behind image readings. What comes back to us as users?
CH 01 Regulation and lawsuits converged
We start outside the developers. Until now, builders explained safety in their own words. Here we ask who outside the company began to demand that explanation, and how far.These parties could hardly differ more. A consumer watchdog, regional governments, a city official guarding public money, and an overseas body. Each acted for its own reason, and their timing overlapped. One alone could be dismissed as one firm's lapse. Together, they suggest a shift. Safety is moving from a goal a firm sets itself to a duty enforced from outside.The heaviest move is the watchdog asking for paperwork. Handing over papers means showing in writing what happened and how it was handled. A spoken promise is no longer enough. Safety work is now judged by whether a trail exists.For pharma teams, this is not a distant story. I expect work we hand to machines to face the same question one day. Which tool did what, what went wrong, and how was it fixed?
Keep that trail now, and you can answer when asked. Outside scrutiny also changes how investors see these firms. Next, what the figures shown before a stock listing tell us.
CH 02 Revenue ahead of the IPO
The very company under scrutiny showed investors strong numbers. How did the market weigh earning power against doubts about safety?That is our question here.Sales grew sharply, and more cash reportedly came in than went out. In a field that pours huge sums into development, that is striking news. Yet shares of related firms did not rise. The market did not read one firm's strength as good news for the whole sector. The numbers and the market's reaction point in different directions.Why the gap?
Proof of earning power has not erased doubts about safety. Some even urge a delay just before the listing. So I think investors are pricing in the risk behind the strong results.From the pharma side, this offers a clue for choosing tools. When you pick a vendor's machine, comparing performance and price is not enough. How exposed the vendor is to regulators and courts decides whether you can use the same tool on the same terms tomorrow. A tool from a firm being sued may see its terms of use change overnight. Then, when we let that tool act for us, who answers for its mistakes?
Next, a machine trusted to make payments.
CH 03 Who is liable when AI shops for you
Machines now press the buttons that people once pressed one by one. When a mistake happens there, who takes the loss?Convenience came first, and the answer is still open.The path itself is simple. A person hands over the task, and the machine goes as far as paying. Further along, errors or leaks can occur. On that last step, the head of one developer said flatly that the firm would cover the losses. Responsibility is starting to shift from users toward builders.Yet a case with that machine and a fake email reportedly showed the danger of delegating. A promise to repay losses matters, but it does not prove accidents won't happen. Meanwhile, another firm widened tools that restrain how machines act inside businesses. Protection by promise, or protection by design.In pharma, machines will one day place orders. In the contract, check two things before any promise of compensation. Can you stop the machine, and is there a log of what it did?
Money can repay a loss. It cannot undo the effect on patients or partners. Words of promise and money that actually moves are different things. Next, how to read the sums attached to a deal to find medicines.
CH 04 Three amounts in one AI drug discovery deal
One single deal, yet different reports gave different sums. The question is not how big the sum is, but what it is made of.The largest figure came from some headlines. Below it sits a cap that is paid only if results come. What will surely change hands looks like only the small first payment. The gap between them is more than an order of magnitude. Money another firm raised by listing is shown too, but partners do not pay it. Remember only the headline figure, and you will misjudge this deal.The capped sum is paid in pieces, each time the research clears a stage. Miss a stage, and nothing is paid. For the payer, it avoids paying for results that never come. The big headline figure is just the maximum, if everything goes well.Pharma teams already read deals this way in daily work. Yet when AI is involved, the headline figure tends to spread as is. When you explain an investment inside your company, separate the first payment from the cap. That alone keeps expectations in check. After money comes the question of how sound the machine's own judgment is. Next, how a machine that reads medical images handles its grounds.
CH 05 Evidence in AI that reads X-rays
A machine that reads medical images is not trusted just for giving the right answer. If a right answer rests on the wrong grounds, it will miss with the next patient. Here, a team tries to teach a machine to choose its grounds.The study splits two judgments. One is whether to go looking for support. The other is to leave out values that do not fit the image. Human doctors also distrust a reading that clashes with the picture in front of them. The machine learns that common sense piece by piece.Points for the steps along the way go out only when the final answer is right. Scoring the process behind wrong answers might teach plausible mistakes. Still, outside experts have not yet checked this paper. Its summary also omits what it beat.For pharma, this overlaps with drug safety work. What matters is less the conclusion than whether the machine can show what it relied on. If its grounds can be traced, that trail helps the people who check it and the regulators who ask about it. While research asks for grounds, machines are already at work in the field. Next, the places where medicines are found and delivered.
CH 06 On the pharma front line
So far, we have heard from builders and regulators. How far have machines entered the work of pharma itself?Today's news shows part of the answer.Today's moves are spread from the start of drug making to the point where medicines reach patients. Finding candidates, deciding whether to take one over, and moving medicines. The machine is no longer a tool only for the lab. The weightiest is a partner company deciding to take over a candidate born from shared research. The pharma side judged a machine-picked candidate worth its time and money.Taking it over means the drugmaker uses a choice it secured in the contract. From then on, the development ahead is its own job. So the decision reads as a sign that the machine's proposal is worth the investment. A machine-picked candidate has passed one human test.Machines entering the delivery stage also deserve attention. If supply stops, medicines stop reaching patients. In research and supply alike, the same question remains. Who checks the machine's judgment, and who can stop it?
Behind the eye-catching news, the question of who stops the machine remains. Last, a few stories that are easy to miss.
CH 07 What is being overlooked
Our last stories look small next to rules and court cases. Yet they sit closer to users' daily work. Because they hide behind the big news, they may hit tomorrow's work harder.A flaw planted by an insider, a service that keeps going down, a price that suddenly leaps. None of these is about how smart the machine is. Still, they hurt those who handed over work. Money back after each outage does not restore the work lost while it was down. Wrongdoing from inside cannot be blocked by outer defenses. Users cannot see a builder's internal controls. Safety is more than correct answers. I think it also means staying up, staying loyal, and keeping terms steady.What if a tool built into pharma work stops one morning?
If the price changes, can you move to another tool?How far can you trust the people inside the builder?Users can settle these plans today, without waiting for news on regulation. With answers ready, work goes on. It is the same thinking as preparing for a break in drug supply. Outside rules and failures in front of us led to the same question. Users, too, cannot give up the job of checking safety.
Wrap-up
Today's events all pointed to the same place. Safety cannot rest on a builder's promise alone. It holds up when outsiders question it and users keep checking. Tomorrow, we may learn how far the watchdog's request reaches, and how the listing plan moves. Then we can sharpen how we pick tools, and what records we keep inside our companies.
- CH 01Axios「Scoop: The FTC demanded documents from AI giants over recent safety incidents」 axios.com
- CH 01Santa Cruz Sentinel「States sue AI giant OpenAI, saying its popular chatbot is hurting children and deceiving adults」 santacruzsentinel.com
- CH 01Crain's New York「Comptroller urges Anthropic to delay IPO in light of ‘myriad safety concerns’」 crainsnewyork.com
- CH 01Technology Org「EU Opens First AI Act Enforcement Case Against OpenAI and Anthropic」 technology.org
- CH 02Financial Times「Anthropic tells investors quarterly revenue up nearly 50% to $16.8bn ahead of IPO」 ft.com
- CH 02Yahoo Finance「Update: Market Chatter: Anthropic Revenue Hits $16.8 Billion in Q3, AI-Modeler Cash Flow Positive」 finance.yahoo.com
- CH 02Investor's Business Daily「Why Reports About Anthropic's Revenue Are Adding To AI Stock Woes」 investors.com
- CH 02Bloomberg.com「Anthropic Results May Aid IPO but Not AI Trade, Analysts Say」 bloomberg.com
- CH 03ServiceNow「ServiceNow advances its operational harness for enterprise AI with AI Passport and expanded governance capabilities」 newsroom.servicenow.com
- CH 03The Verge「Elon Musk is promising to “make you whole” if Grok messes up a purchase or leaks your credit card.」 theverge.com
- CH 03Cybernews「Grok Bot phishing email catch exposes AI agent risks」 cybernews.com
- CH 04AstraZeneca「AstraZeneca announces new strategic collaboration with insitro to use advanced AI systems to develop transformative therapeutics for difficult to treat cardiometabolic diseases」 astrazeneca.com
- CH 04Reuters「AstraZeneca strikes $40 million AI drug discovery deal with insitro」 reuters.com
- CH 04BioSpace「AstraZeneca bulks up MASH, cardio work in up to $1.5B AI partnership with insitro」 biospace.com
- CH 04Yahoo Finance「Iambic, an AI drug discovery specialist, banks $150M in an IPO」 finance.yahoo.com
- CH 05X-Ragent: Tool-Augmented Reinforcement Learning for Evidence-Grounded Chest X-Ray Reasoning（「X-Ragentはこの二つを分けて学習させる後学習の枠組みで、診断が正しかった軌跡にだけ過程の評価を与える報酬を使う。」）
- CH 06Google DeepMind「Accelerating the search for antimicrobial peptides with Google Antigravity」 deepmind.google
- CH 06Pulse 2.0「ParkourSC Acquires Quartic To Launch Unified AI Platform For Life Sciences Supply Chains」 pulse2.com
- CH 06Quiver Quantitative「Recursion Pharmaceuticals Announces Sanofi Options First Development Candidate from AI-Driven Collaboration」 quiverquant.com
- CH 07The New York Times「OpenAI Engineer, Who Is Charged With Planting Bug, Advised Saudis」 nytimes.com
- CH 07South China Morning Post「DeepSeek suffers another major outage as company offers user refunds」 scmp.com
- CH 07Android Authority「Google is now emailing Gemini subscribers about a change that will cost them 4x more」 androidauthority.com
Articles used
Today's related reports
- AI Daily News October 16, 2026
- AI & Economy News October 16, 2026
- AI & Finance News October 16, 2026
The narration is synthetic speech. The content draws only on this site's articles from the same day. Each edition passes seven plain-language checks before publication. That means known obstacles to comprehension are held below threshold — it is not a guarantee of comprehension.