Think Twice Before You Search: Deterrence Messaging Designs to Prevent Searches for Non-Consensual Intimate Images

Non-consensual distribution of intimate images (NDII) occurs when explicit images or content of a person, whether authentic or synthetic (AI generated), are distributed online or shared with others without their consent. This can lead to significant and long-lasting psychological, financial, privacy, and other harms. The ease with which machine learning tools are becoming accessible and affordable through websites and apps that allow users to produce and share synthetic NDII increases the prevalence and scale of harm.

One way of addressing this problem is through deterrence messaging — notifications or other text aimed at influencing human decision-making and behavior. This approach has been applied with some success to other forms of harmful content and behavior such as the sharing of mis- and disinformation, online gambling, online piracy, and sharing of child sexual and abuse material (CSAM). Indeed, there is also emerging research on the role of deterrence messaging aimed at the sharing of NDII (Prichard, Scanlan, Krone, et al., 2022).

However, one dimension that is under-studied is the use of search to seek out NDII on social media platforms and search engines, which can be a pathway to sharing, creating, or normalizing the practice of NDII. In this report we examine deterrence messaging in search as one strategy for reducing NDII.

Through a series of interviews with platform experts and analysis of how platforms (e.g., social media, adult websites, community/online forums, and search engines) employ deterrence messaging to address different forms of harmful content (including NDII), we developed prototypes of different types of deterrence messaging that could reduce the demand for NDII through search. In addition, we held a workshop in April 2025 with representatives from platforms, civil society groups, academics, and survivor groups to discuss how such prototypes might work in practice.

The three prototypes include:

• Informative mechanisms that provide warnings or educational content to inform users about NDII-related risks and consequences. The prototype consists of a prominent warning banner at the top of the entire search results page when users enter terms likely linked to NDII.
• Reductive or filtering mechanisms which reduce access to harmful content by de-amplifying or removing search results entirely, without deleting content or purging the underlying index. In this prototype, the search still returned results, but NDII-related content was deprioritized or removed from top placements, with a note at the top of the page stating that some results had been filtered or de-amplified.
• Increased Friction mechanisms that introduce significant barriers that slow down user actions, requiring additional effort or interaction. In this prototype, an interstitial pop-up appears before users can access search results for NDII-related terms, requiring them to click either “Learn More” or “Continue” to proceed.

We conclude that platforms generally should invest more in deterrence messaging to reduce the demand for NDII-related searches as a complement to other interventions. The prototypes provided in this report are a starting point for platforms and researchers in that effort. We also make the following recommendations for platforms and for researchers on further issues to pursue.

For Platforms:

• Design interventions specific to platform context. Deterrence messaging should be tailored to each platform’s policies, audience, and content formats. Some platforms may benefit from targeting specific communities or high-risk terms, while others may prioritize individual search behaviors or flagged queries.
• Use multi-layered mechanisms. Platforms should not rely on a single deterrence messaging mechanism. Our findings suggest that combining informative, reductive, and friction mechanisms can better account for varied user intentions and platform affordances.
• Support ongoing monitoring, measurement and refinement. Platforms should collaborate with researchers and civil society to evaluate message effectiveness over time and refine keyword lists, messaging tone, and trigger conditions in response to emerging patterns.
• Ethically partner with NGOs and survivor organizations to include access to their resources. Messages should include links to relevant resources, including support organizations that assist victims or offer behavioral support to potential perpetrators.

For Researchers:

• Explore what search terms should trigger deterrence messaging. There is little consensus on what constitutes an NDII-related query.
• Better understand user intent. It remains unclear how often users knowingly search for NDII versus encountering it unintentionally. Research should examine whether and how intent should affect intervention design.
• Evaluate the role of personalization. While some users may benefit from generic warnings, others may respond better to messages that adapt to behavioral patterns or past interactions. The potential for personalized messaging merits careful, privacy-aware exploration.
• Measure behavioral and attitudinal outcomes over longer time-horizons. Future work should assess whether deterrence messaging leads to longer-term changes in user attitudes, awareness of harm, or demand for viewing or distributing NDII.
• Balance safety with free expression. Efforts to intervene in search must avoid suppressing legitimate content, such as sexual education or survivor storytelling.
• Include intersectional perspectives in the design and testing of deterrence messaging interventions. NDII is disproportionately targeted at women and girls and so research on interventions should also consider variables such as race, sexual orientation, immigration
  status, disability status, etc.

Read the full report.