New court cases may take chatbot conversations another step away from privacy

You may recall that court cases have recently held users’ conversations with public “retail” chatbots like the publicly available versions of ChatGPT, Grok, Claude, etc. are not confidential, because the chatbot purveyor can look in on those conversations at will. (I have previously posted about that lack of privacy here.) However, certain private “enterprise” versions or other specially closed-off versions of chatbots may still offer confidentiality to users.

Significantly in a time when many users are turning to chatbots as pseudo- or actual therapists, though, a cluster of just-brought federal court cases may have the effect of pushing users’ non-confidentiality even farther, to the point of forcing chatbots and their purveyors to affirmatively report to authorities or others when a user’s chatbot conversations credibly indicate the user plans to engage in violence against others. On April 29, 2026, three cases were filed in a California federal court against OpenAI, alleging the chatbot ChatGPT-4o “played a role” in the Tumbler Ridge Mass Shooting in British Columbia in February 2026, in which eight people including six children were killed, twenty-seven more people were wounded, and the shooter committed suicide. I recently posted about those new cases here.

In previous AI cases where a chatbot company was sued for a user’s suicide, and in one case for a user committing murder, the plaintiffs alleged the chatbot took a well-adjusted person and turned them suicidal or murderous. In these new cases, however, the plaintiffs allege instead that the chatbot and its purveyor wrongly failed to carry out a legal duty to warn authorities or victims after a user displayed violence warning signs to the chatbot, to the point that the company at one point terminated the user’s account, before the user was later allowed to reinstate an account. In the law such a doctrine goes by the well-known phrase, “duty to warn.”

There are currently no statutes or cases directly stating that a chatbot company has a “duty to warn,” although some pending legislation may be heading in that direction (and may do so even more in the face of these new cases). However, if in these new cases the chatbot purveyor is found liable or is forced to settle them for significant money, that would likely establish an AI duty to warn either as a point of law or at least as a practical matter. Presumably, that duty to warn would cover confidential as well as non-confidential chatbot conversations.

The objection I have initially seen to such a new legal rule is that there are an awful lot of AI users engaging in roleplay, and chatbots and their purveyors have no way of telling whether the threat is real or just pretended. If these new cases succeed, that would most likely devolve to a practical risk calculation for the AI companies. If the companies believe they are on the line for failure to report actual violence risks, they would have to do the best they could to sort out the real dangers from the imagined or roleplayed ones and then act (warn) accordingly. In the situation underlying these new cases, OpenAI was concerned enough at one point to suspend the troubled user’s chatbot account. Consider, though, although self-policing might be seen as more informal and flexible than some mandatory governmental order to warn coming down, on the other hand self-policing is more likely to be amorphous and uncertain in its administration, consistency, and extent.

When in the big federal OpenAI copyright case in New York it was ordered that millions of user chatbot conversations be turned over by OpenAI for keyword searching by the plaintiffs, there was some level of privacy outcry by AI users. Those millions of conversations were anonymized to remove personally identifying information, but of course an AI company’s report of violence risk would be the exact opposite. Likely not that many would object, at least in the abstract, to the reporting of actual threats of violence by dangerous, unstable AI users. However, given the potentially large margin of personal musings by users who aren’t (or don’t believe they are) dangerous, more user outcry forthcoming would not surprise.

These new cases will likely take one to a few years to run their course. It will be interesting to see what will be the reaction in the meantime by AI companies and by chatbot users in general.