AI Makes Strangely Accurate Predictions From Blurry Medical Scans, Alarming Researchers

Homepage | Forums | Topics In Depth | Science and Environment | AI Makes Strangely Accurate Predictions From Blurry Medical Scans, Alarming Researchers

Viewing 1 reply thread
  • Author
    • #442482
      • Total Posts: 9,978

      New research has found that artificial intelligence (AI) analyzing medical scans can identify the race of patients with an astonishing degree of accuracy, while their human counterparts cannot. With the Food and Drug Administration (FDA) approving more algorithms for medical use, the researchers are concerned that AI could end up perpetuating racial biases. They are especially concerned that they could not figure out precisely how the machine-learning models were able to identify race, even from heavily corrupted and low-resolution images.

      In the study, published on pre-print service Arxiv, an international team of doctors investigated how deep learning models can detect race from medical images. Using private and public chest scans and self-reported data on race and ethnicity, they first assessed how accurate the algorithms were, before investigating the mechanism.

      They found, as previous studies had, that the machine-learning algorithms were able to predict with high accuracy whether the patients were Black, White, or Asian. The team then tested a number of possible ways that the algorithm could glean this information. Among the proposed ideas was that the AI could pick up differences in the density of breast tissue or bone. However, when these factors were masked (by clipping pixel brightness at 60 percent for bone density), the AI was still able to predict with accuracy the self-reported race of the patients.

      Other possibilities included the AI guessing from regional differences in markers on the scan (say one hospital that sees a lot of white patients marks their X-Rays in a specific style, it may be able to guess from demographics), or that there were differences in how high-resolution the scans were when they were taken (for example, deprived areas may not have as good equipment). Again, these factors were controlled for through heavily pixelating, cropping, and blurring the images. The AI could still predict ethnicity and race when humans could not. Even when the resolution of the scan was reduced to 4 x 4 pixels, the predictions were still better than random chance – and by the time resolution was increased to 160 x 160 pixels, accuracy was over 95 percent.

      Jesus: Hey, Dad? God: Yes, Son? Jesus: Western civilization followed me home. Can I keep it? God: Certainly not! And put it down this minute--you don't know where it's been! Tom Robbins in Another Roadside Attraction

    • #442486
      • Total Posts: 2,339

      How well does AI diagnose disease?  That’s what I want to know.

Viewing 1 reply thread
  • You must be logged in to reply to this topic.