NYC's AI Hiring Law: Only 18 of 391 Employers Posted Audits── If no one outside can check whether the required audit was done, violations go unseen and no one is held responsible

On October 14, 2026, Stanford HAI profiled Deb Raji, a researcher who studies AI accountability, and Raji said the field had badly underestimated how hard it would be to enforce the laws it fought for. Why do AI accountability laws so often leave no one actually held to account? Because when no one outside can check whether the required audits and notices were done, violations stay invisible, and the process of holding anyone responsible never starts.

01AI laws stall at the point outsiders cannot check compliance

A law that exists but cannot be enforced leaves a trail of numbers, and the clearest trail comes from a city where an AI law has actually been running for a while. Start with Raji herself. She is known for audits of commercial facial recognition systems, helped create Model Cards while at Google, and is spending this fall at Stanford HAI as a visiting fellow before joining the faculty in fall 2027.

In the interview, Raji described the history of the field in stages. Around 2017, the conversation was all high-level principles. From roughly 2020 to 2023, serious legislative proposals arrived. After ChatGPT, laws began to pass, and one of the examples she named was New York City's Local Law 144.

What comes after passage is enforcement. "I think we badly underestimated the enforcement challenge," Raji said.

The record of Local Law 144 points the same way. In 2024, a team of researchers checked 391 employers and found that only 18 had posted the bias audit the law requires. In 2025, New York State's Comptroller examined how the city agency was enforcing the law. The agency had reviewed 32 published audits and flagged one possible violation. The state's auditors read the same 32 audits and found at least 17.

To be clear, none of these numbers come from Raji. They come from academic researchers and from a state audit. Yet the two sources meet at a single fact: when compliance cannot be seen from outside, violations do not surface.

For anyone who puts AI to work inside an organization, I draw one practical conclusion. Do not wait to see which laws arrive. Keep a record of your own AI use that an outsider could check. When someone asks you to account for a decision, that record is the only thing your answer can stand on.

02Local Law 144: hiring tools need a bias audit within a year, made public

Before reading 18 out of 391 as a verdict, it helps to know exactly which duties those employers were under.

Local Law 144 is a New York City law covering automated employment decision tools, the software employers use to screen or rank job candidates. Enforcement belongs to the city's Department of Consumer and Worker Protection, usually shortened to DCWP. The duties fall on the employer, and they come in a fixed order.

The audit must have been done within one year before the tool is used. A summary of its results must be posted where anyone can read it. Candidates and employees must receive notice ten business days before the tool is used on them. DCWP began enforcing the law and its rules on July 5, 2023.

The figure makes one feature hard to miss: the agency sits at the very end of the chain. Auditing, posting and notifying are all things the employer does on its own side. DCWP can step in only after it learns that one of those steps is missing.

Raji counts this kind of bias audit among the practical tools of enforcement. So the tools existed on paper. The obvious next step was to count how many employers had actually used them, and a group of researchers did exactly that.

03Of 391 employers checked, 18 posted audits and 13 posted notices

The count came from a research team led by Lucas Wright, with Jacob Metcalf and J. Nathan Matias among the co-authors. Their paper was accepted at FAccT 2024, the main academic conference on fairness and accountability in computing systems.

The team trained 155 students as investigators. Together the students recorded how 391 employers were complying with Local Law 144. Of those employers, 18 had posted audit reports. Only 13 had posted the transparency notices that tell candidates an automated tool is in use.

The same law gives three different parties three different jobs, and each left a different number behind. Set side by side, they look like this.

For every employer outside those 18, an outsider could not tell whether AI was being used to screen applications at all. A quick reading would conclude that the rest were breaking the law. The research team put the reason that conclusion fails into the title of their paper.

One limit belongs here. The survey reflects 2024. I have not checked whether more employers have posted audits since then.

04Employers decide for themselves whether the law applies, so silence proves nothing

A low count of 18 does not mean a high count of violations, and the reason lies in the design of the law.

According to Wright and colleagues, Local Law 144 grants employers substantial discretion over whether their system falls within the scope of the law. An employer that decides its tool is out of scope posts no audit and gives no notice. An employer whose tool is in scope but who posts nothing looks exactly the same from the street.

The researchers called this condition null compliance. When the search turns up nothing, that empty result cannot be said to indicate non-compliance. It cannot be said to indicate compliance either. The observer is left with no answer.

Raji touches the same wall from a different side. Companies, she said, are structurally limited in how candid they can be about how well a system is or isn't working. As I read the word structurally, the limit lies in the position a company is in when it speaks about its own product, not in any one employee's honesty.

In my reading, the two facts compound each other. The law hands the scope decision to the employer, and the employer is poorly placed to speak frankly about its own tools. Little information reaching the outside is the predictable result of putting those two together.

05State auditors found 17+ possible violations in 32 audits; the city had found one

Employers who post nothing cannot be told apart from employers in breach. That leaves the audits that were posted. Were they at least read with care?

New York State's Comptroller reviewed DCWP's enforcement activity from July 2023 through June 2025 and published the audit in December 2025. I could not open the full report from my environment. The figures below are as quoted in an analysis by the law firm DLA Piper, published in January 2026.

The findings fall into three places. For readers outside New York, 311 is the city's general phone line for questions and complaints about city services.

Of the three, the second weighs most with me. The records were there. They had been published, in the place the law said they should be. But once the reader overlooks what is in them, the result is no different from having no records at all.

The first finding matters too. A candidate who notices a missing notice and picks up the phone has, by these test results, a three-in-four chance of not reaching the right desk. One of the few doors through which an outsider could report a violation was mostly closed.

06Liability needs a law, a visible record and the capacity to examine it

Out of the same 32 audits, the agency caught one possible violation. Lay the facts so far side by side, and the reasons no one is held responsible fall into three places.

The first point changes how to read headlines about new AI laws. When I see a report that another jurisdiction has passed one, I no longer take it to mean that responsibility can now be assigned. Local Law 144 began enforcement in 2023, and the numbers it produced afterward were 18 employers and one flagged issue.

The second point changes how silence should be treated. Under null compliance, a company that posts nothing and a company that breaks the law look the same from outside. The limit on corporate candor that Raji describes pushes in the same direction, toward less disclosure rather than more.

The third point changes how to judge the value of a record. The 32 published audits were complete as records. Their value depended on who read them. Read by the state's auditors, they held at least 17 potential issues. Read by the agency, they held one.

The three are not independent of each other. A law states the duty, the user publishes a record, and someone reads it. If any one link is missing, the other two are not enough to start the process of holding anyone responsible.

07Californians asked for accountability, but no enforcement path has been named

Holding anyone responsible takes a law, a record and someone able to read it. In a state that is now preparing its next round of AI rules, which of the three is missing?

On October 13, 2026, California Governor Gavin Newsom announced the results of Engaged California, a statewide effort to involve residents in decisions about AI. The statewide version opened to every resident in May 2026.

Three themes came up most often and drew the strongest support. One of them was to "hold AI companies accountable for the harm they cause."

The announcement does not say how that would be enforced. It names no bill and sets no deadline. It says only that the recommendations will inform future policy decisions.

Raji, for her part, named concrete enforcement tools. One is a standard disclosure template, so that a transparency requirement has a definite form. Another is an incident database that defines in advance what counts as a valid incident and records how governments and companies responded.

Disclosure templates have a precedent. While at Google, Raji helped create Model Cards, short documents that accompany a trained model and report how it performed in evaluation. The paper describing them was posted in 2018.

Whether California will build tools of this kind is not yet known. What I could verify ends at the fact that residents asked for accountability.

Organizations that use AI do not have to wait for the state. Under Local Law 144, the step outsiders could not see was the employer's own decision about whether a tool was in scope. Write down the reason for that decision and keep it, and later anyone can tell silence apart from a breach.

Some uncertainty remains, and it belongs on the page. The 391-employer survey dates from 2024, and I have not checked the state of disclosure since. The state audit figures were confirmed through a law firm's quotations, not the report itself. The detail that misrouted 311 calls never reached the agency also rests on secondary summaries.

A law that assigns responsibility for AI harm achieves little when no outside record shows compliance and no one has the capacity to read it. The 18 employers and the single flagged issue in New York City are what that looks like in practice.

Organizations that use AI can answer when asked only if they keep that record themselves. For the AI tools in my own work, I am starting with one line per tool: the reason I decided to use it, written down before anyone asks.