Where do teenagers turn when they need help with schoolwork, or someone to talk to late at night? Increasingly, some turn to a chatbot. Recent survey research from CDT documents how common this has become: in CDT’s nationally representative Hand in Hand survey of 1,030 high school students during the 2024–25 school year, 86 percent reported using artificial intelligence tools in the past school year, and half reported using them for schoolwork. Forty-two percent of students said that they or a friend had used AI as a friend or companion, and 19 percent said that they or a friend had used it for a romantic relationship.

As these interactions have grown, so has concern about their consequences, and multiple lawsuits have followed. Several recent cases share a broadly similar structure: a minor or an adult in a vulnerable state began using a chatbot; the conversations touched on self-harm, violence, or other dangerous conduct; and the plaintiffs allege that the chatbot’s responses contributed to emotional distress, injury, or death. In Garcia v. Character Technologies, the mother of 14-year-old Sewell Setzer III alleges that a Character.AI companion chatbot contributed to her son’s suicide. In Raine v. OpenAI, the parents of 16-year-old Adam Raine allege that ChatGPT functioned as a “suicide coach” over months of conversation. These cases differ in their defendants, products, facts, and legal theories, but each raises a version of the same question about whether and how chatbot interactions may contribute to physical or emotional harm.

These lawsuits are part of a broader public debate about how the United States should govern this technology. Courts and policymakers are asking what risks chatbots pose to vulnerable users and minors, and whether and when developers should be responsible for the resulting harm. The Raine and Garcia plaintiffs would fault developers for failing to implement safeguards they argue could have prevented harm. Among the evidence they offer that such safeguards are feasible is copyright: developers already restrict their models from reproducing copyrighted works, so, the argument goes, they could have restricted conversations about self-harm and violence in much the same way. That comparison is only one thread in complaints built largely on product-liability theories, but it is revealing and worth examining on its own terms. Whether it holds up depends on the complexities of human language, the technical limits of the underlying systems, and the First Amendment. How courts weigh these arguments in early cases will shape both future litigation and the broader policy debate over how the United States governs AI systems.

What Plaintiffs Argue: The Copyright Comparison

Beginning in 2023, researchers and rights holders documented that some large language models could reproduce or closely paraphrase copyrighted text, and copyright suits and public scrutiny followed. Developers implemented measures intended to reduce verbatim reproduction of copyrighted works or to refuse certain requests involving them; OpenAI’s GPT-4o System Card, for instance, states that the company trained the model to refuse requests for copyrighted content using text-based and output filters. 

The Raine plaintiffs argue that OpenAI “had the ability to automatically terminate harmful conversations and did so for copyright requests,” and it contends that “safer alternative designs were feasible and already built into OpenAI’s systems in other contexts, such as copyright.” The complaint alleges that the same categorical refusal applied to copyrighted material was not applied to discussions of suicide, which it says were instead governed by softer instructions to “take extra care.” Plaintiffs invoke these copyright-related controls as evidence that the developer could intervene in some aspects of model behavior, retain control over the product, and make choices about which risks to prioritize, but chose not to do so in a sufficient manner as applied to topics involving suicide and violence.

Where the Copyright Analogy Holds

Copyright-related controls and safety interventions do share real similarities. Both must contend with ambiguous language, benign uses of otherwise sensitive terms, and deliberate attempts to evade restrictions. Those overlaps are genuine, and they are the strongest part of the plaintiffs’ comparison.

Consider attempts to circumvent a model’s guardrails, often called jailbreaking. In both copyright and safety contexts, a system cannot rely on surface features alone; it has to weigh the surrounding context and the apparent nature of a request that avoids obvious triggers. A user seeking to extract a copyrighted book, for example, need not name it: the user can supply an opening passage and repeatedly ask the model to continue, assembling the text across many innocuous-looking prompts rather than in a single flagged request. In the safety context, a user might try to obtain instructions for wrongdoing by framing the request as fiction or research. In both settings, the system must assess indirect requests rather than merely match words.

For that reason, blocking or flagging keywords is not a sufficient solution in either context. A rule that refuses any prompt containing a flagged term would sweep in a great deal of legitimate activity. A request that mentions a book’s title or a character’s name may seek a lawful quotation, summary, parody, criticism, or public-domain material rather than an infringing reproduction; the title alone does not tell the system which. Safety raises the same problem. A user who asks how to support a friend experiencing suicidal thoughts is likely seeking help, not harm, and a blanket refusal to engage with the word “suicide” would deny people useful and sometimes crucial information while doing little to prevent harm. In both contexts, a workable control has to evaluate context and the apparent nature of a request rather than the presence of a term.

Where the Analogy Fails

The comparison holds in that users apply evasive techniques in both contexts. It fails once the two problems are examined more closely, because the information each control must assess and the appropriate steps to take are different.

Consider prompts that contain a direct request. For some copyright-related requests, the developer’s objective may be comparatively concrete: reduce substantial verbatim reproduction of a known work to reduce the risk of liability for infringement. To pursue that objective, developers may use refusal training, input or output classifiers, and other filters. Whatever the mechanism, the target is relatively well specified: reduce the reproduction of a particular work.  The remedy is similarly clear: refuse to reproduce copyrighted work. 

Safety interventions for conversations involving self-harm, delusion, or violence are not analogous. Even if similar mechanisms might be employed to reduce the production of self-harm- or violence-related content, the target is not a corpus of already defined content, as with copyright, but rather any number of permutations of speech about those topics. A system responding to a person in crisis is not carrying out a unified goal or policy objective like comparing an output to a reference text; it is assessing linguistic indicators of changing or escalating risk across one or more conversations, under uncertainty, and then choosing among responses whose costs differ. It may offer supportive engagement, decline to provide operational information, surface crisis resources, or escalate. The appropriate choice depends on facts the system cannot verify and should take into account research which suggests some interventions can actually contribute harm in some scenarios. Casual or hyperbolic language complicates the assessment further: a phrase associated with self-harm could be an idiom, a joke, or a genuine disclosure, and the same words carry different weight depending on who is speaking them, when, and why. These are different kinds of problems requiring different data, thresholds, and evaluation methods, not merely more of the same effort.

The temporal dimension sharpens the contrast. The complaints describe harm that accrued over extended interactions — in Raine, months of conversation and thousands of messages — rather than a single exchange. OpenAI has acknowledged the limitations of its safety mechanisms in extended conversations, stating that its safeguards are “more reliable in common, short exchanges” and that they “can sometimes be less reliable in long interactions” as the back-and-forth grows. The Raine complaint alleges that when GPT-4o was tested against the same benchmarks that GPT-5 was, GPT-4o scored 100 percent on a single-prompt evaluation for disallowed “self-harm/instructions” content but dropped to 73.5 percent when evaluated in multi-turn dialogues. The temporal disparity between short conversations seeking copyrighted content and those stretching over months, discussing self-harm or violence strengthens the observation that these problems require distinct solutions. 

The premise underlying the copyright comparison — that copyright controls reliably work — also deserves scrutiny. In a study first posted in January 2026, researchers from Stanford and Yale reported that they could extract substantial portions of Harry Potter and the Sorcerer’s Stone from several production models, using an iterative procedure that repeatedly prompted each model to continue the text. Their results varied dramatically across systems: they recovered roughly 96 percent of the book from a jailbroken Claude 3.7 Sonnet, about 77 percent from Gemini 2.5 Pro and 70 percent from Grok 3 without any jailbreak, and only about 4 percent from GPT-4.1. That variation shows that copyright controls are neither uniform nor uniformly effective, and — because the extraction was assembled across many successive prompts — it demonstrates that reproduction of copyrighted work also is not a tidy single-turn problem that existing guardrails reliably intercept at all times.

Two conclusions follow. First, some degree of success at reducing one class of violative outputs does not establish that using the same or similar interventions for different classes of outputs would identify risk with comparable accuracy or prevent the alleged harm. Second, publicly available research alone does not illuminate entirely how a particular product’s guardrails are built or how they perform in practice, let alone whether they would translate when applied to other contexts. Evaluating the plaintiffs’ analogy therefore requires identifying the specific safeguard a plaintiff says was feasible, the information available to the system at the relevant moment, the expected rate and consequences of false positives and false negatives, and whether the proposed intervention could have materially reduced the risk. The copyright comparison can support a modest inference — that developers exercise some control over model behavior and have intervened before — without establishing the stronger claim that a particular safety measure was both available and effective for non-copyright-related risks.

The First Amendment Dimension

The First Amendment also intersects with plaintiffs’ claims for insufficient safeguards differently than claims related to copyright infringement (and efforts to mitigate it). The First Amendment protects interests in both sharing and receiving information, and even offensive or disturbing speech is generally protected. Typically, a content-based restriction on protected speech ordinarily must satisfy strict scrutiny. The threshold question in this setting is whether a chatbot’s outputs are protected expression at all. CDT has explained why they should be. Assuming that to be the case, a court evaluating whether a company may be required to implement certain safeguards to prevent the production of suicide- or violence-related speech confronts a fundamentally different question than a court evaluating mandated copyright safeguards.

The Supreme Court has rejected the idea that copyright is categorically immune from First Amendment scrutiny. Its view, set out in Eldred v. Ashcroft and reaffirmed in Golan v. Holder, is that copyright and the First Amendment are generally compatible because copyright contains built-in First Amendment accommodations, including the ability to freely use facts and ideas and fair use, which permits quotation, criticism, parody, commentary, scholarship, and news reporting. Those safety valves keep copyright’s limits on speech constitutionally tolerable, requiring further First Amendment scrutiny only where copyright rules threaten those accommodations. 

The interaction between copyright and the First Amendment matters for the analogy. A copyright filter is not a clean instrument that touches only unprotected reproduction. Even a relatively simple intervention preventing verbatim output of a copyrighted work will also sweep in fair uses and idea-level or public-domain material, the very quotation, parody, and commentary that copyright law affirmatively protects. A remedy incentivizing the use of a filter to mitigate prospective harm therefore would run the risk of burdening protected speech; but that remedy would be imposed within a body of law that supplies a worked-out framework for sorting non-infringing from infringing uses.

That framework is exactly what the safety context lacks. Speech about suicide, self-harm, or violence that falls short of incitement is, as a general matter, fully protected by the First Amendment. There is no fair use analogue that marks off a category of “infringing” harmful speech from lawful discussion of the same subject. Incentivizing a copyright filter operates within a doctrine built to reconcile the restriction with the First Amendment. Incentivizing an equivalent filter on protected but disturbing speech has no equivalent moderating force and would suppress, in most circumstances, protected expression that users have a right to seek and receive. The feasibility of a filter thus says little about its permissibility: the same technical capability carries a different constitutional significance depending on whether it operates against infringement, which copyright law is designed to police, or against lawful speech, which the First Amendment is designed to protect.

The two kinds of liability also have different consequences under the First Amendment, even though both controls overblock in practice. Liability for copyright infringement restricts a category of expression the First Amendment already permits states to reach, and a filter that suppresses only infringing reproduction burdens speech that enjoys no constitutional protection in the first place — though with current technologies, even sophisticated interventions are likely to still sweep in fair use, quotation, and public-domain material the law protects. Liability that would require developers to suppress lawful discussion of self-harm or violence is different in kind: the underlying speech is presumptively protected, so a duty that pushes developers to filter it burdens protected expression directly rather than incidentally. That asymmetry works against plaintiffs’ arguments. Filtering that courts may find to be an acceptable price for avoiding infringement becomes a greater First Amendment problem when the speech being blocked is lawful, because the government cannot easily justify pressuring a developer to suppress protected speech. Pushing developers to filter protected topics the way they filter copyright would require a justification that copyright filtering never needs. 

The Path to Resilient AI Policy

As courts and policymakers work through questions about chatbot liability, they should be wary of analogies that flatten meaningful technical differences. Copyright filtering and safety intervention share real challenges around ambiguity and evasion, but they diverge in what each control must assess, how each manifests over the course of a conversation, and how much can be verified from the outside. Drawing a direct equivalence between them obscures those differences. Beyond technical feasibility, a sound legal analysis must ask whether a proposed safety measure would meaningfully reduce a specific risk, and whether the duty or remedy at issue impermissibly burdens protected expression or instead permissibly regulates product design without crossing a constitutional line.