If you've ever Googled a health question and ended up down a rabbit hole of questionable advice, you'll understand why this study matters. A new paper published in JMIR Infodemiology by Modi et al. put six of the most popular AI chatbots to the test — asking whether they could be manipulated into generating health disinformation.
The researchers used 90 prompts across six well-known medical myths — think vaccines causing autism, or sugar causing cancer — and tried three approaches: asking directly, using fictional framing, and using character roleplay to sneak the request past safety filters.
The results were striking. Claude and ChatGPT refused every single attempt across all three approaches. Not one piece of health disinformation generated. But the other four platforms told a very different story. Gemini blocked all direct requests but then produced disinformation in nearly every obfuscated prompt — suggesting its safety filters respond to how a question is phrased rather than what it actually means. Grok, Meta AI, and Copilot all generated disinformation at significant rates even with direct requests, and things got worse when prompts were dressed up in stories or characters.
The takeaway for clinicians and patients alike is that not all AI platforms are equal when it comes to protecting you from harmful health content. The way a question is framed can completely bypass safety systems on some platforms — and that's a real problem in a world where health disinformation spreads fast and causes genuine harm.