We need a mechanism for outside parties to verify companies' safety measures

The builders say they are hurrying to put containment in place

On October 15, Nvidia's Jensen Huang said the push to build containment technology into AI should be accelerated. Containment means a mechanism that constrains AI behavior from the outside so that damage does not spread even if the AI behaves in unexpected ways. We should accelerate the development of open shell. We should accelerate the the integration of this containment technology into AI systems.── Jensen Huang (2026-10-15 01:23 UTC · YouTube "Nvidia's Huang Says He Would Not Sell Chips to Irresponsible AI Customers" (Bloomberg Television) at 0:04:57, source)

This statement points to one premise for the companies and individuals who use AI: safety mechanisms are built in by the builders, and users have a hard time seeing how well they are made. An explanation that they are "included" does not tell you whether they actually work. Huang says this because he believes the builders have agency over how the technology advances. On the same day, he also said that because they are the ones building it, they have a chance to move forward safely.

Harm from agents is already showing up in the field

Reports of harm involving agents (AI that goes as far as acting on a person's behalf) continue. A single attacker using an AI agent reportedly broke into the sites of more than 100 companies for about $5,000. A security company has also reported that a Chinese-linked AI agent was behind cyberattacks on Japan. Both are individual cases and are not part of the council's statements.

On October 15, Gary Marcus criticized those who released agents, saying they are not responding correctly with sandboxing and monitoring. A sandbox is a mechanism that limits what an AI can do to an isolated space. they've shown themselves to be kind of incompetent in cybersecurity, not doing the right things around sandboxing and monitoring.── Gary Marcus (2026-10-15 05:48 UTC · YouTube "China’s Rogue Agents Find a Way Around Anti-Bot Curbs" (Bloomberg Television) at 1:14:27, source)

For corporate staff, this means that when adopting an agent, they need to check the substance of the isolation and monitoring themselves. Marcus has long argued that agents that have large language models write and run code would be a security nightmare. He says it has turned out just as he predicted.

If AI does not follow instructions, containment becomes a precondition

Why is containment necessary? Marcus says large language models do not truly follow instructions, and that so-called rogue AI cases are a manifestation of this. LLMs don't really follow instructions. All the so-called rogue AI things are examples of that where they're not really following instructions.── Gary Marcus (2026-10-15 09:08 UTC · YouTube "Bloomberg New Economy | Day 3 | Morning Session" (Bloomberg New Economy) at 1:59:46, source)

From this view, one cannot assume that "if you ask the AI, it will act as asked." Those who delegate work need to restrict its actions from the outside, log them, and keep them in a form that can be verified afterward. Marcus thinks AI's understanding of the world is superficial, and that is why it cannot follow instructions either. Huang, on the other hand, sees the problem as an engineering challenge. In the area of misuse, though, the council's view leans broadly toward those calling for regulation. [Council inference] The difference between the two lies not in how large they judge the danger to be, but in how to design who does the verifying.

If investment cools, spending on safety could be cut too

Safety verification costs money, and its funding depends on business revenue and investment. Marcus pointed out that quarterly growth slowed from about 145% from Q1 to Q2 to about 45% from Q2 to Q3. the growth rate from Q1 to Q2 is about 145%, and the growth rate from Q2 to Q3 was about 45%. It was a deceleration in the growth rate.── Gary Marcus (2026-10-15 12:49 UTC · YouTube "NYU’s Gary Marcus on Building AI We Can Trust" (Bloomberg Television) at 0:17:39, source)

This figure means that if investment slows, safety measures and third-party evaluation could be among the first costs companies cut. Marcus sees the industry as overextended, with profitability unconfirmed. Huang, by contrast, says bubbles occur when supply exceeds demand, and that supply and demand will eventually come into balance. Views on the bubble are also split within the council. [Council inference] This input contains no actual statement linking a pullback in investment to the allocation of resources for safety. This is a gap in the council.

Set side by side, the benefits and burdens call for someone to do the verifying

There are three benefits ahead. Huang said that even in radiology, where AI has spread, the number of radiologists has increased. A policy of building in containment technology and advancing safely was also presented. Marcus also touched on the possibility that a technology different from the current one could fundamentally change medicine.

There are also three burdens. Junior developers cannot get jobs, which cuts off the path by which skills are passed on. There are security risks that come with releasing agents, along with inadequate isolation and monitoring. Slowing growth and excessive investment with unconfirmed returns are also burdens. What links the three benefits and the three burdens is whether companies' explanations can be verified from the outside.

There is also a minority view. It holds that simply scaling up current large language models will not solve the problems of understanding and remembering the world, and that the debate over capabilities is overstated. From this view, investment and doom scenarios based on the performance gains of current models should be discounted.