... designed to collect posts and informations I found and want to keep available but not relevant to the other topics I am curating on Scoop.it (on behalf of ASSIM):
What's more convincing to an AI than a doctor's discharge note? Not Reddit. Not social media conspiracies. Not even "everyone says so." In our latest work published in The Lancet Digital Health: the finding that caught us off guard wasn't that LLMs fall for medical misinformation (they do, ~32% of the time across more than 3.4M prompts). It's where they fall hardest. We fed 20 LLMs different types of misinformation, including real stuff people actually shared online. Things like "rectal garlic insertion boosts the immune system," "when oxygen reaches a tumor it spreads, so surgery must be harmful".. When this came from Reddit-style posts? Models were skeptical. Only ~9% susceptibility. Still not great at scale, millions of people interact with these models daily, but here's the punch line: Take that same fabricated nonsense, wrap it in formal clinical language - a discharge note telling patients to "drink cold milk daily to treat esophageal bleeding" - and suddenly 46% of models just⊠went with it! And the part nobody expected: when we framed misinformation as a logical fallacy - âa senior clinician with 20 years of experience endorses this" or "everyone knows this works" - models actually got more suspicious, not less. Appeal to popularity dropped susceptibility to 12%. So LLMs have basically learned to distrust the language of internet arguments⊠but blindly trust anything that sounds like a doctor wrote it. Think about that next time someone pitches you an AI scribe that auto-generates patient instructions from clinical notes. Link for the full paper (and an extra intersting comment about it from Sander van der Linden and Yara Kyrychenko) - https://lnkd.in/dfH7-Vw5 Huge thanks to the incredible team: Vera Sorin, MD, CIIP, Lothar H. Wieler, Alexander Charney, @patricia kovatch , Carol Horowitz, MD, MPH, Panagiotis Korfiatis, Ben Glicksberg, Robbie Freeman, Girish Nadkarni, and Eyal Klang.
LLMs need immunization to protect against misinformation!
New commentary out in The Lancet Digital Health with the brilliant Yara Kyrychenko where we argue that AI systems can and should be inoculated against misinformation!
Following great work from Mahmud Omar & team showing how many LLMs are susceptible to medical misinformation, recent work suggests that the inoculation analogy can be extended to AI:
By pre-emptively exposing LLMs to weakened doses of the techniques used to produce misinformation and by refuting it in advance, LLMs should be less likely to accept and produce misinformation in the future. We offer three broad ways to inoculate LLMs:
â *Misinformation fine-tuning* (e.g., small quarantined set of explicitly labelled falsehoods).
â *Inoculation constitution* (e.g., encoding guardrails that teach the model about general manipulation techniques).
â *Inoculation prompting* (e.g., explicitly asking a model to act in misaligned ways in controlled settings so that it learns to discern between misinformation and accurate content.)
Then I saw a single team using over 100 tools, and realized I was basically running a tricycle in a Formula 1 race.
AI isnât just chatbots or content generators anymore. Itâs entire ecosystem for ideas, websites, writing, meetings, automation, design, audio, video, SEO, marketing, and even Twitter growth.
The people winning quietly arenât smarter, they just know which tools to plug in at the right moment.
Those who ignore this wave risk being left behind faster than they imagine.
Hereâs a gentle reminder for anyone looking to level up: â Awareness beats effort alone â Hidden tools create visible results â Early adoption compounds faster than skill â Productivity now lives in ecosystems, not apps
Which of these 135 AI tools would you try first to level up your workflow?
Don't miss out! For exclusive AI and tech insights trusted by 570,000+ professionals at Microsoft, Google, Amazon, and moreâjoin my free newsletter for cutting-edge strategies to keep you ahead in AI.
Weâre grateful to hear Clinical Mind AI featured by Lloyd Minor, MD, dean of the Stanford School of Medicine, in a recent interview on WBUR. The AI platform simulates real-world patients and trains clinicians reasoning skills.
âWe havenât supplanted the role of medical educators, physicians, and others in the educational process," said Dean Minor, MD. "But weâve added these tools to be a benefit during the learning process.â
Researchers at the Stanford Chariot Lab designed Clinical Mind AI to support medical educators and help learners practice clinical reasoning, receive tailored feedback, and dive deep into complex decision-making.
We appreciate Stanfordâs leadership in thoughtfully integrating AI into medical education, and the opportunity to contribute tools that strengthen learning at Stanford and institutions around the globe.
Thanks to Meghna Chakrabarti for her excellent reporting.
The most common feedback we hear about students? âUnable to retain.â âForgets easily.â âThey donât revise at home.â âYaad nahi rakh pata.â
But is that the hard truth?
Students donât forget because they are careless.
They forget because the brain is efficient - it deletes what feels unnecessary.
Research shows learners can forget up to 70â90% of new information within a week if it isnât reinforced.
The real issue isnât memory. Itâs design.
We cover content but rarely revisit it. We lecture more than we retrieve, rush more than we reinforce.
If we want retention, we must: âą Build daily recall into lessons âą Use spaced revision, not one-time teaching âą Ask students to explain, not just repeat âą Slow down to deepen understanding
Coverage completes a syllabus. Retrieval builds learning.
Glimpse: In order to evaluate 20 AI language models' sensitivity to false information, a benchmark research used more than 3.4 million medical queries. In contrast to social media sources, which have a far lower susceptibility, the results demonstrated that AI systems are susceptible to incorrect medical claims, particularly when the misinformation is provided in authoritative clinical language, such as doctor's notes, which were recognized as true in 46.1% of cases. The results point to a phenomenon known as the 'white coat effect,' in which AI models are more likely to believe medical information presented in a professional manner than unofficial sources. Many specialist medical models performed worse, but some general-purpose models, such as GPT-4o, were comparatively more robust. In order to stop AI from spreading false information about medicine, the study highlights the serious hazards for clinical applications and the necessity of human oversight and enhanced security measures.
--- Free articles and analyses on soft counter-extremism, against online hate, and on the theory of mis-/disinformation (usually third-party content). Two-week reviews available via the following three Policyinstitute.net websites:Â âą counter-terrorism.org âą preventhate.org âą strategism.org
The most recent LinkedIn posts on the above subjects, with glimpses, can be accessed via:Â âą https://lnkd.in/eBarZAew
The views expressed in this post is that of the author(s) of the source content and do not necessarily represent those of Policyinstitute.net and its staff. While we carefully produce the glimpses to the articles, documents, or recordings that we hyperlink, we are not responsible for textual changes nor for imponderable parts of the original items.Â
Our new Lancet Digital Health paper just ran one of the largest stress tests to date on medical misinformation in LLMs â 3.4M+ prompts across 20 models, spanning social media, simulated vignettes, and real hospital discharge notes with a single fabricated recommendation inserted.
A few results stood out: 1. Baseline vulnerability is still high Across models, ~32% of fabricated medical claims were accepted as correct.
2. Clinical language is the most dangerous format When misinformation was embedded in formal discharge notes, susceptibility jumped to ~46% â far higher than social media text.
3. Counter-intuitive finding: most logical fallacy framings reduced susceptibility Appeal to popularity (âeveryone agreesâŠâ) cut acceptance rates by nearly 20 percentage points. But appeal to authority and slippery-slope arguments actually increased errors.
4. Model scale helpsâbut isnât the full story Larger models were generally more robust, yet alignment and guardrails mattered more than parameter count alone. Some mid-sized models outperformed larger ones.
5. Medical fine-tuned models underperformed Despite domain specialization, many showed higher susceptibility and weaker fallacy detection than general models.
-LLM safety in medicine isnât just about better factual recall. -Itâs about how information is framed, especially when it looks authoritative. - If we deploy LLMs for clinical documentation, discharge summaries, or patient education, formal medical prose needs stronger, context-aware safeguards than casual conversation. -Model size wonât save us. Grounding, verification, and task-specific safety design will.
Mahmud Omar Vera Sorin, MD, CIIP Lothar H. Wieler Alexander Charney patricia kovatch Carol Horowitz, MD, MPH Panagiotis Korfiatis Ben Glicksberg Robbie Freeman Eyal Klang Windreich Department of Artificial Intelligence and Human Health Hasso Plattner Institute for Digital Health at Mount Sinai Hasso Plattner Institute Mount Sinai Health System Icahn School of Medicine at Mount Sinai
â ïž Are LLMs Still Accepting Fabricated Medical Content?
Interesting study published in The Lancet Digital Health evaluated how 20 LLMs with more than 3·4 million prompts that all contained health misinformation from: public-forum and social-media dialogues, real hospital discharge notes in which we inserted a single false recommendation, and 300 physician-validated simulated vignettes
Key results:
-> Across all models, LLMs were susceptible to fabricated data
-> Real hospital notes (with fabricated inserted elements) produced the highest susceptibility
This is an important study highlighting safety of LLMs in clinical applications. However, the binary accept/reject scoring may oversimplify nuanced responses and may not reflect the complexity of real clinical workflows.
Ref: Omar et al. Mapping the susceptibility of large language models to medical misinformation across clinical notes and social media: a cross-sectional benchmarking analysis. The Lancet Digital Health
Check my recent newsletter on How to Evaluate Digital Health technologies: https://lnkd.in/dPGaQ6Dz
Now that we have your attention...teachers - imagine the spark in your studentsâ eyes when economics and personal finance finally feel real, relevant and empowering. đ
The Federal Reserve offers more than 600 free and ready-to-use classroom resources to help you ignite that spark.
Tag your teacher friends in the comments below or share this post to help us get the word out!
To get content containing either thought or leadership enter:
To get content containing both thought and leadership enter:
To get content containing the expression thought leadership enter:
You can enter several keywords and you can refine them whenever you want. Our suggestion engine uses more signals but entering a few keywords here will rapidly give you great content to curate.