WEBVTT

00:00:00.000 --> 00:00:04.063
How far should these systems be built, and who draws that line?

00:00:04.063 --> 00:00:08.126
Today, the place where the line gets drawn moved.

00:00:08.126 --> 00:00:12.189
Until now it was drawn inside the firms that build the technology.

00:00:12.189 --> 00:00:16.252
This time it was carried into a room where governments sit.

00:00:16.252 --> 00:00:20.315
On the same day, countries and much smaller units of government began writing their

00:00:20.315 --> 00:00:24.378
own rules, each at its own speed.

00:00:24.378 --> 00:00:28.441
The more places that decide, the harder it becomes to know whose yardstick you must

00:00:28.441 --> 00:00:32.504
meet.

00:00:32.504 --> 00:00:36.567
Today's events put that question in front of you again and again.

00:00:36.567 --> 00:00:40.630
For anyone working with medicines, this is not a distant story.

00:00:40.630 --> 00:00:44.438
The people who make this technology described its dangers from the heaviest seat in

00:00:44.438 --> 00:00:48.246
politics.

00:00:48.246 --> 00:00:52.054
They can slow it down themselves.

00:00:52.054 --> 00:00:55.862
So why ask for a hand from outside?Two rivals sat side by side and pushed in one

00:00:55.862 --> 00:00:59.670
direction.

00:00:59.670 --> 00:01:03.479
The technology moves faster than the reins they hold, and they said so in public.

00:01:03.479 --> 00:01:07.287
Each also had a private problem.

00:01:07.287 --> 00:01:11.095
One had postponed a plan to raise money from investors, with safety given as the

00:01:11.095 --> 00:01:14.903
reason.

00:01:14.903 --> 00:01:18.711
The other faced a competitor abroad that will not ease off.

00:01:18.711 --> 00:01:22.519
When rivals ask for the same thing, they are not asking for goodwill.

00:01:22.519 --> 00:01:26.327
They want everyone held to one set of conditions.Outsiders are invited to explain

00:01:26.327 --> 00:01:30.135
things in that room fairly often.

00:01:30.135 --> 00:01:33.943
What was unusual is that the speakers were the ones who would be bound.

00:01:33.943 --> 00:01:37.751
When an industry asks to be fenced in, the fence tends to follow its own shape.

00:01:37.751 --> 00:01:41.560
Whose language ends up in the text matters for years.

00:01:41.560 --> 00:01:45.368
In medicines that order is familiar: offer a limit from within, before an accident

00:01:45.368 --> 00:01:49.176
forces one on you.

00:01:49.176 --> 00:01:52.984
You need to know whose hand wrote the measure you are asked to meet.

00:01:52.984 --> 00:01:56.792
There is rarely just one.

00:01:56.792 --> 00:02:00.600
So where do these measures take shape?

00:02:00.600 --> 00:02:05.322
There is no longer a single place where this gets decided.

00:02:05.322 --> 00:02:10.045
On one day, several layers moved, and they did not move the same way.One national

00:02:10.045 --> 00:02:14.767
government turned down the idea of moving in step with the world.

00:02:14.767 --> 00:02:19.489
That same day, another promised shared benchmarks at an international table.

00:02:19.489 --> 00:02:24.212
A third pushed a law written for a whole continent into its own paperwork.

00:02:24.212 --> 00:02:28.934
A regional government went further: studying a way to force a halt, and reopening

00:02:28.934 --> 00:02:33.656
the tax breaks it had granted to computing sites.

00:02:33.656 --> 00:02:38.379
Where the layer above steps back, the layer below picks up the pen.Writing a forced

00:02:38.379 --> 00:02:43.101
stop into law says one thing: nobody is counting on makers to police themselves.

00:02:43.101 --> 00:02:47.824
Before you run it, build the brake someone else can pull.

00:02:47.824 --> 00:02:52.546
In medicines that thinking is old.

00:02:52.546 --> 00:02:57.268
You lay out how to pull a product and halt a trial before you talk about benefit.

00:02:57.268 --> 00:03:01.991
When layers disagree, the cost of compliance rises.

00:03:01.991 --> 00:03:06.713
Working out which layer reaches you is not a delay; it is part of the design.

00:03:06.713 --> 00:03:11.435
The burden grows, and the rules still arrive from outside.

00:03:11.435 --> 00:03:16.158
The next question is what happens where no rule has arrived yet, and where the

00:03:16.158 --> 00:03:20.880
things you type end up.

00:03:20.880 --> 00:03:24.915
What you type leaves your hands the moment you send it.

00:03:24.915 --> 00:03:28.950
Where it travels, and where it settles, is something almost nobody using it ever

00:03:28.950 --> 00:03:32.986
looks at.One company published the route in its own words.

00:03:32.986 --> 00:03:37.021
Your files and your questions go into a sealed compartment.

00:03:37.021 --> 00:03:41.056
The exchange is kept on its own machines, and it says even its own staff never look

00:03:41.056 --> 00:03:45.091
inside.

00:03:45.091 --> 00:03:49.127
Convenience preserved, secrecy preserved.

00:03:49.127 --> 00:03:53.162
Yet nobody from outside has checked any of it.

00:03:53.162 --> 00:03:57.197
A promise turns into evidence only when someone else confirms it.

00:03:57.197 --> 00:04:01.232
A diagram shows what was intended, not what was built.This bites in workplaces that

00:04:01.232 --> 00:04:05.268
handle material no single person may release.

00:04:05.268 --> 00:04:09.303
Patient records.

00:04:09.303 --> 00:04:13.338
Documents that go to a regulator before approval.

00:04:13.338 --> 00:04:17.373
Once they are gone, they are gone.

00:04:17.373 --> 00:04:21.409
A pledge not to look and an architecture that cannot look are two different claims.

00:04:21.409 --> 00:04:25.444
Before you adopt it, ask who examined the build from outside.

00:04:25.444 --> 00:04:29.479
If you cannot get an answer, narrow what your people are allowed to put in.

00:04:29.479 --> 00:04:33.514
Assurance drawn on a slide weighs less than assurance someone tested.

00:04:33.514 --> 00:04:37.550
Treat an unchecked claim as a claim, not as proof.

00:04:37.550 --> 00:04:41.585
That stance matters now, because these systems are slipping quietly into ordinary

00:04:41.585 --> 00:04:45.620
work.

00:04:45.620 --> 00:04:49.731
That day, this technology was both a subject for grave meetings and a tool people

00:04:49.731 --> 00:04:53.841
used at their desks.A shopping platform slotted an outside model into the help it

00:04:53.841 --> 00:04:57.952
gives merchants.

00:04:57.952 --> 00:05:02.062
A travel booking firm widened what its staff may reach.

00:05:02.062 --> 00:05:06.173
A money app claimed to be first in its category to wire in a talking assistant.

00:05:06.173 --> 00:05:10.283
A toolkit for programmers added models that run on your own machine.

00:05:10.283 --> 00:05:14.394
This is no longer a separate window you open.

00:05:14.394 --> 00:05:18.504
It is a part inside the task.

00:05:18.504 --> 00:05:22.615
Once the entrance moves inside, nobody gets a moment to choose.The same thing will

00:05:22.615 --> 00:05:26.725
happen where medicines are made and sold.

00:05:26.725 --> 00:05:30.836
The decision to adopt will not arrive as a formal request for approval.

00:05:30.836 --> 00:05:34.946
It will ride in on a routine update.

00:05:34.946 --> 00:05:39.057
By the time anyone notices, a draft, or a reply to an enquiry, may already have

00:05:39.057 --> 00:05:43.167
passed through a machine.

00:05:43.167 --> 00:05:47.278
So you need a way to record which tasks it touched.

00:05:47.278 --> 00:05:51.388
A task nobody can account for afterwards quietly falls out of everything you check.

00:05:51.388 --> 00:05:55.499
Adoption is outrunning verification.

00:05:55.499 --> 00:05:59.609
A record only works if the system can say what it did.

00:05:59.609 --> 00:06:03.720
And what happens inside it is not easy to see.

00:06:03.720 --> 00:06:08.005
Does it not know, or does it know and stay quiet?

00:06:08.005 --> 00:06:12.289
From outside, those two look exactly alike.A research team argues you can tell them

00:06:12.289 --> 00:06:16.574
apart by watching what happens inside, rather than what comes out.

00:06:16.574 --> 00:06:20.859
They describe three situations.

00:06:20.859 --> 00:06:25.144
A polite refusal may sit on top of an answer that is there.

00:06:25.144 --> 00:06:29.428
A weak score may hide real capability.

00:06:29.428 --> 00:06:33.713
Something said to be deleted may only be muffled near the exit.

00:06:33.713 --> 00:06:37.998
None of this is about how clever the thing is.

00:06:37.998 --> 00:06:42.283
It is about whether it is straight with you.

00:06:42.283 --> 00:06:46.567
The write-up has not yet passed review by other researchers.That shifts the

00:06:46.567 --> 00:06:50.852
yardstick for choosing a tool.

00:06:50.852 --> 00:06:55.137
Until now we graded these things on whether the answer was right.

00:06:55.137 --> 00:06:59.422
From here, what happens when the answer is withheld also counts.

00:06:59.422 --> 00:07:03.706
Did the material you asked to have removed really go?

00:07:03.706 --> 00:07:07.991
Without a way to test that, you cannot hand over anything that matters.

00:07:07.991 --> 00:07:12.276
Honesty only counts as a property once someone outside can check it.

00:07:12.276 --> 00:07:16.561
Silence becomes something you read, not something you accept.

00:07:16.561 --> 00:07:20.845
The demand for proof does not belong only to research.

00:07:20.845 --> 00:07:25.130
A workplace full of rules needs that yardstick before it needs the tool.

00:07:25.130 --> 00:07:29.020
The more rules a workplace carries, the later new tools arrive.

00:07:29.020 --> 00:07:32.911
That day the traffic ran the other way.An article written for investors says that

00:07:32.911 --> 00:07:36.801
sectors bound by strict rules are going deeper into rented computing and everyday

00:07:36.801 --> 00:07:40.692
assistance.

00:07:40.692 --> 00:07:44.582
A firm that advises companies says it has folded a model into the work of watching

00:07:44.582 --> 00:07:48.473
for attackers.

00:07:48.473 --> 00:07:52.363
That is defensive work, handed over.

00:07:52.363 --> 00:07:56.254
The third item is a different kind: treating diabetes by transplanting cells was

00:07:56.254 --> 00:08:00.144
talked through in the open.

00:08:00.144 --> 00:08:04.035
What links them is that fields which stayed still out of caution moved.Caution is

00:08:04.035 --> 00:08:07.925
not a goal in itself.

00:08:07.925 --> 00:08:11.816
It is a means for stopping what must stop.

00:08:11.816 --> 00:08:15.707
If defence can be delegated, people move to the judgements that are hard.

00:08:15.707 --> 00:08:19.597
But if you cannot say what happened where you delegated, you have simply let caution

00:08:19.597 --> 00:08:23.488
go.

00:08:23.488 --> 00:08:27.378
The line is not between adopting and refusing.

00:08:27.378 --> 00:08:31.269
It is between being able to narrate the path afterwards and not.

00:08:31.269 --> 00:08:35.159
Writing down how far you handed over is the first step.

00:08:35.159 --> 00:08:39.049
Whether you can tell that story depends on what you have in hand.

00:08:39.049 --> 00:08:42.940
Some of what arrived today is still incomplete.

00:08:42.940 --> 00:08:46.736
A day of news mixes things you can take at face value with things you should

00:08:46.736 --> 00:08:50.532
hold.The story about a very hard problem being cracked has a catch.

00:08:50.532 --> 00:08:54.328
Nobody can read the answer well enough to judge it.

00:08:54.328 --> 00:08:58.124
A second story says tools that decide their own next move were pointed at a large

00:08:58.124 --> 00:09:01.921
set of companies.

00:09:01.921 --> 00:09:05.717
Neither the makers nor the harm has been detailed.

00:09:05.717 --> 00:09:09.513
A third reports a jump in people using a bot, without saying what counts as a

00:09:09.513 --> 00:09:13.309
person.

00:09:13.309 --> 00:09:17.105
In all three, only a slice of the event reached us.Do not treat a missing piece as

00:09:17.105 --> 00:09:20.901
though it were not missing.

00:09:20.901 --> 00:09:24.697
This is exactly what reviewing a document is.

00:09:24.697 --> 00:09:28.493
When a figure appears, ask how it was counted.

00:09:28.493 --> 00:09:32.289
When a claim of effect appears, ask who ran the study.

00:09:32.289 --> 00:09:36.086
There is no reason to drop that discipline because the subject is machines.

00:09:36.086 --> 00:09:39.882
If anything, the faster the current, the more weight a decision to wait carries.

00:09:39.882 --> 00:09:43.678
The test is whether anyone in the room is willing to say the evidence falls short.

00:09:43.678 --> 00:09:47.474
The ability to wait is the most useful preparation today.

00:09:47.474 --> 00:09:51.270
Here, not deciding quickly is itself part of the job.

00:09:51.260 --> 00:09:54.731
More places decide now, and more places use.

00:09:54.731 --> 00:09:58.202
Only our ability to check has failed to keep up.

00:09:58.202 --> 00:10:01.674
Until that gap closes, two questions carry us.

00:10:01.674 --> 00:10:05.145
Whose hand wrote the measure?

00:10:05.145 --> 00:10:08.616
And who tested the claim?

00:10:08.616 --> 00:10:12.087
Ask both every time.

00:10:12.087 --> 00:10:15.559
Keep asking, and however many places start deciding, your footing holds.

00:10:15.559 --> 00:10:19.030
We will carry the same questions into tomorrow.