People Innovation Excellence

Bracing Medical AI Systems for Attacks

There’s new advice on how to handle tampering that fools algorithms and enables healthcare fraud

Last June, a team at Harvard Medical School and MIT showed that it’s pretty darn easy to fool an artificial intelligence system analyzing medical images. Researchers modified a few pixels in eye images, skin photos and chest X-rays to trick deep learning systems into confidently classifying perfectly benign images as malignant.

These so-called “adversarial attacks” implement small, carefully designed changes to data—in this case pixel changes imperceptible to human vision—to nudge an algorithm to make a mistake.

That’s not great news at a time when medical AI systems are just reaching the clinic, with the first AI-based medical device approved in April and AI systems besting doctors at diagnosis across healthcare sectors.

Now, in collaboration with a Harvard lawyer and ethicist, the same team is out with an article in the journal Science to offer suggestions about when and how the medical industry might intervene against adversarial attacks.

And their take-home message is—wait, but be ready to defend.

Adversarial attacks against medical AI systems are very likely for two reasons. First, there are “enormous incentives” for doctors and insurers to carry out such attacks, as IEEE reported last June. Second, it’s easy to do, as demonstrated by MIT undergraduates at LabSix. In fact, even just tilting the angle of a camera when taking a picture of a mole can alter an algorithm’s diagnosis from benign beauty mark to malignant skin cancer.

“Those types of things very well could be coming, but are still hypothetical,” emphasizes Samuel Finlayson, a graduate student at Harvard and MIT who co-authored the paper with Harvard biomedical informaticians Andrew Beam and Issac Kohane.

So what’s to be done when the attacks do begin?

Jonathan Zittrain, cofounder of Harvard Law School’s Berkman Klein Center for Internet & Society and author of The Future of the Internet and How to Stop It, had similar questions when he read the team’s paper.

“I was reminded of the time in the early 2000’s when cybersecurity vulnerabilities were readily apparent but not yet often exploited,” Zittrain tells IEEE Spectrum. He reached out to Beam, Kohane and Finlayson to discuss how the field might move forward when dealing with these sorts of attacks. [READ MORE]

Published at : Updated
Leave Your Footprint

    Periksa Browser Anda

    Check Your Browser

    Situs ini tidak lagi mendukung penggunaan browser dengan teknologi tertinggal.

    Apabila Anda melihat pesan ini, berarti Anda masih menggunakan browser Internet Explorer seri 8 / 7 / 6 / ...

    Sebagai informasi, browser yang anda gunakan ini tidaklah aman dan tidak dapat menampilkan teknologi CSS terakhir yang dapat membuat sebuah situs tampil lebih baik. Bahkan Microsoft sebagai pembuatnya, telah merekomendasikan agar menggunakan browser yang lebih modern.

    Untuk tampilan yang lebih baik, gunakan salah satu browser berikut. Download dan Install, seluruhnya gratis untuk digunakan.

    We're Moving Forward.

    This Site Is No Longer Supporting Out-of Date Browser.

    If you are viewing this message, it means that you are currently using Internet Explorer 8 / 7 / 6 / below to access this site. FYI, it is unsafe and unable to render the latest CSS improvements. Even Microsoft, its creator, wants you to install more modern browser.

    Best viewed with one of these browser instead. It is totally free.

    1. Google Chrome
    2. Mozilla Firefox
    3. Opera
    4. Internet Explorer 9