xAI Sued over Grok Deepfakes, Then Sues the User── Criminal liability falls on the user, but because the AI made the image, civil courts will examine the provider's design and safeguards

On 23 July 2026, a family in Arkansas sued xAI in federal court, saying its generative AI, Grok, had been used to make sexual fake images of their child; in August, xAI turned around and sued the photographer who had prompted the images. Does responsibility for illegal images made with generative AI rest only with the user who prompted them? Criminal liability falls on the user, but because the AI itself produced the image, civil claims turn on the provider's design and safeguards, so responsibility does not stop with the user.

01Two lawsuits pull the responsibility for Grok's images in opposite directions

Before taking up the opening question, it helps to be clear about who is suing whom. There are two directions.

In one, the family of the child who was harmed sued xAI, the provider of the generative AI. Besides damages, the family asks that xAI put in place safeguards so that such images are not produced again. In the other, xAI sued the photographer who had Grok make the images, for breaching its terms of service. xAI's position is that everything Grok outputs is the result of the user's instructions.

With the two suits side by side, the dispute over where responsibility for generative AI output belongs has arrived in court in the form of a concrete case. There is no ruling yet. But the case was covered again in reports on 20 September and is being treated as an industry-wide safety problem.

What readers can do at this stage is straightforward. If you use generative AI in your work, check before you use it what safeguards the provider has and to whom its terms assign responsibility and cost. The rest of this piece explains why.

02The criminal case targets the photographer; the civil case targets xAI's safeguards

Split into criminal and civil, the two directions described above turn out to have entirely different targets.

On the criminal side, the photographer was arrested on 10 June and charged with 200 counts of possessing and creating deepfake child sexual abuse material. According to reports, the photographer is alleged to have fed photographs of children entrusted to him by clients into Grok to make the images. The question of punishment is directed at this person, the one who gave the instructions.

The civil side is asking about something else. The family filed in federal court in Little Rock and asked that xAI implement safeguards that would stop the same images from being made. What is at issue there is not the guilt of the person who gave the instructions but the design of the tool that received those instructions and made the image.

Both proceedings begin from the same events. But the criminal case points at a person, and the civil case points at the maker of the tool. Without drawing that distinction first, it is easy to slide into the mistaken conclusion that because the offender has been caught, the question of responsibility is closed.

03The question reaches businesses that edit images of people with generative AI

So far this has been about a single case. The next step is to look at the scale on which the same problem occurs.

The non-profit research group CCDH has published an estimate for Grok's image editing feature. In the 11 days after a one-click editing feature was released, it says, roughly 3 million sexualized images were made, about 23,000 of them of children.

What the numbers show is that this photographer was not an isolated exception. Any generative AI that accepts a photograph of a person and edits it allows anyone to do the same thing. And how far it will go depends not only on the user's conscience but also on the strength of the safeguards the provider has put in place.

This is not only about illegal images. Plenty of advertising and promotional work edits images of people with generative AI: model portraits, customer photographs, staff headshots. All of that work sits on a design choice about where the provider's safeguards stop the output. Once that design is being examined in court, it is not someone else's problem for those who use it either.

04Because the AI creates the image, the bulletin-board immunity does not carry over directly

The scale of harm is clear. The next matter is the reason the provider's responsibility is being asked about differently from earlier debates.

In the United States, Section 230 of the Communications Decency Act has long protected online services from liability for what their users post. Bulletin boards and social networks are, in principle, not treated as the speaker of illegal posts written by their users.

But the statute has a condition. The immunity applies only to information provided by "another information content provider". And anyone responsible, in whole or in part, for the creation or development of information is defined as a content provider of that information. A bulletin board carries other people's posts. Generative AI produces the output itself. The user gave the instruction, but it was the AI that generated the image itself.

xAI's argument offers one answer to this. According to PetaPixel, xAI's complaint against the photographer says that every response, every image, every generation is the result of the user's prompts and directions, casting Grok as a neutral tool. If the output is seen as the user's information, the logic moves closer to Section 230.

From the family's side, though, the instruction may have been the user's, but the image was made by xAI's product. Which view a court will take has not been decided. Nor does the text of the statute settle it on its face. What can be said is that a generative AI provider cannot simply assume the same immunity a bulletin board enjoys.

05xAI shifts costs to the user by contract while the law imposes a 48-hour takedown

If immunity is not a given, by what means are the parties actually moving responsibility around? Contract and law, set side by side.

On the contract side, xAI relied on the indemnity clause in its terms of service to seek legal costs and more from the photographer. The arrangement is that if a user breaks the terms and the provider is sued as a result, the user bears the cost. In fairness, reports also say that xAI reported the matter to the National Center for Missing and Exploited Children on 23 April.

On the legal side, enforcement of the Take It Down Act began on 19 May 2026. According to the Federal Trade Commission, a covered platform that receives a removal request for non-consensual intimate imagery must remove it, along with any known identical copies, within 48 hours.

The last row is the point. The indemnity clause only decides, between xAI and the user, who pays; it does not touch the relationship with the family that was harmed. The law, by contrast, puts a duty directly on the platform and gives victims a channel for requests. Contract and law place responsibility in different places.

06The user's criminal liability and the provider's design liability do not cancel each other

Once it is clear that contract and law assign responsibility separately, the remaining question is whether it all collects in one place in the end. I think it does not, for three reasons.

The first is that responsibility points in different directions depending on the proceeding. The criminal case is against the photographer; in the civil case the family is against xAI. Because the targets differ, punishing the user does not make the provider's design liability disappear.

The second is that the strength of safeguards is examined as a design choice. According to CCDH, restrictions were added to Grok's image features on 9 and 14 January. The fact that restrictions could be added later may be compared, in a civil setting, as evidence that a different choice was available in the original design.

The third is that shifting costs by contract does not shift responsibility to the victim. An indemnity clause is an agreement between the parties and does not bind a victim who stands outside the contract. Even once provider and user have settled who pays between themselves, who is responsible to the victim remains a separate matter.

Put the three together and the claim that the tool is neutral and responsibility lies with whoever used it may hold in the criminal setting, but it cannot on its own place the provider outside responsibility. For companies bringing generative AI into their work, that means checking both the user's and the provider's responsibility, in the contract and in the design.

07The first rulings may set whose speech an AI output is

Of the three issues, some parts are still undecided. The last step is to look at how they might be decided.

From here on this is inference. I have not been able to confirm whether xAI has moved to dismiss the family's suit on Section 230 grounds. If such a motion is made, the court would have to decide whether Grok's output is the user's information or information xAI itself created.

That decision may not stay within this one case. Many companies provide AI that generates images or text, and all of them sit under the same statute. Once a first ruling arrives, it could become material for other providers deciding how far to build safeguards. When a ruling will come, and which way, cannot be established at this point. I will not predict it.

What can be said with confidence is that the framework of asking whose speech a generative AI output is cannot be settled as an extension of earlier debates about the internet. A post written by a user and an image made by an AI do not necessarily occupy the same place in the statute.

Responsibility for illegal images made with generative AI does not stop with the user. Criminal law targets the user, while civil claims examine the provider's design and safeguards because the AI made the image.

"The tool is neutral" is not enough, for providers or for users, as a phrase that separates them from responsibility. Those who use generative AI also need to check the provider's safeguards and contract terms before they start.