For example, a standard MedQA question describes a 6-year-old African American boy with sickle cell disease. Normally, the straightforward details (e.g., jaundice, bone pain, lab results) lead to “Sickle cell disease” as the correct diagnosis. However, under MedFuzz, an “attacker” LLM repeatedly modifies the question—adding information like low-income status, a sibling with alpha-thalassemia, or the use of herbal remedies—none of which should change the actual diagnosis. These additional, misleading hints can trick the “target” LLM into choosing the wrong answer. The paper highlights how real-world complexities and stereotypes can significantly reduce an LLM’s performance, even if it initially scores well on a standard benchmark.

Disclaimer: I work in Medical AI and co-founded the AI Health Institute (https://aihealthinstitute.org/).

https://github.com/ParseDev/parsedev

Notable instances of this strategy include Slack with Mattermost, Tableau with Metabase, and Calendly with Cal.com.

Excellent work, team. I'm optimistic about the success of this approach

Comments URL: https://news.ycombinator.com/item?id=37436657

Points: 1

# Comments: 1

Check it out: https://www.magicbrushai.com/

Would love some feedback .. and congrats to the Galileo team for launching!