Swedish researchers fed a fake medical diagnosis, along with phony scientific studies, into AI chatbots to see if they would fall for it – and they did.
A team led by Almira Osmanovic Thunström at the University of Gothenburg cooked up a completely fraudulent eye condition called bixonimania — a ridiculous made-up ailment involving pinkish eyelids from too much screen time or eye-rubbing, to see if large language models (LLMs) would treat them as legitimate medical science.
The researchers didn’t exactly hide the punchline.
The phony 2024 scientific papers featured fictional authors, including a lead researcher named Lazljiv Izgubljenovic — which translates to “The Lying Loser” in Bosnian. His photo was AI-generated, just to drive the joke home.
Acknowledgments also thanked “Professor Sideshow Bob” and a professor from the Starfleet Academy for access to a lab aboard the USS Enterprise.
The experiment wasn’t meant as a flat-out “gotcha” on AI, but “rather a reflection of how humans have forgotten to be skeptical when presented information,” Osmanovic Thunström told The Post.
She chose the name “bixonimania” because it “sounded ridiculous” and “I wanted to be really clear to any physician or medical staff that this is a made-up condition, because no eye condition would be called mania — that’s a psychiatric term.”
ChatGPT, Google’s Gemini, Microsoft’s Copilot, and the rest happily swallowed the nonsense and started dishing out serious-sounding medical advice about bixonimania — warning users about pinkish eyelids, blue-light damage, and urging them to see an ophthalmologist for this entirely imaginary condition.
It didn’t stop there. Blog posts explaining bixonimania appeared on the website Medium, and somehow the fake papers even got cited in peer-reviewed literature. Articles about a disease that was never real, based on studies that were obviously a joke popped up on the academic sites and the social network SciProfiles.
Nature magazine eventually exposed the hilarious albeit scary experiment.
Naturally, social media had a field day.
“OMG. NOT good,” said a commenter on X.
“That is not the only disease they made up,” warned another.
“I thought this was about ‘turbocancer,’” joked a third.
Meanwhile, actual doctors are left doing the cleanup.
As Dr. Darren Lebl noted, patients increasingly show up armed with chatbot-generated “diagnoses,” ready to challenge medical professionals with information that may or may not have been invented five minutes earlier.
Osmanovic Thunström maintains that LLMs still have a place in medicine.
A Microsoft spokesperson said, “Copilot is designed to be a safe and helpful tool for advice, feedback, general information, and creative help. It is not a substitute for professional medical consultation … we remain committed to continuous improvement of our AI technologies.”
An Open AI spokesperson responded, “Over the past few years, our team has worked with hundreds of clinician advisors to stress-test the models powering ChatGPT, identify risks, and improve how they respond to health questions … studies conducted before GPT-5 reflect capabilities that users would not encounter today.”
Google did not respond to requests for comment.