AI systems may encounter private signs of violent planning before anyone has made a public threat, before law enforcement is aware of the risk, and even before the conduct is clearly illegal.
The report asks what should happen when chatbots surface private signs of violent planning before there is a public threat or clearly illegal conduct.
It proposes a framework separating provider intervention from government reporting: human-reviewed referrals for credible, specific, and imminent threats of serious violence; provider discretion for concerning but not clearly illegal content; and 988 as a possible support pathway for crisis-linked cases.
At its center is a requirement that covered AI chatbot providers maintain, disclose, and follow a violence escalation policy, with privacy protections, no general monitoring duty, and accountability tied to good-faith implementation.
Read the full report by New York University Center on Tech Policy