In Which Field Has Artificial Intelligence Become BETTER THAN DOCTORS?

On 1 January 2020, Nature published research by Google Health, DeepMind, and Imperial College London naming 31 researchers. Early breast cancer diagnosis was its subject, attracting social media and news coverage. Headlines generally said, “Google's new deep learning algorithm replaces radiologists,” or described how it outperformed doctors.

Artificial Intelligence

What Suggests AI May Outperform or Replace Radiologists?

If a machine answers 10 questions, eight correctly and two incorrectly, while radiologists miss three, is it better? Human decision-making and cognition are complex and incompletely understood. A radiologist uses mammography images to decide whether someone has cancer and, when suspicious but uncertain, can turn to additional clinical tests such as biopsy and other clinical tests, after spending a lifetime training for these decisions.

- Related topic: Is breast cancer still possible after a negative mammogram?

Computer image processing using deep learning extracts meaningful information from pixels. Trained algorithms are assessed on previously unseen test images.

An algorithm is a solution path devised for a problem: “Problem + Algorithm = Solution.” Not medicine-specific, algorithms are used especially in medicine and engineering. Health-serving artificial intelligencesare designed to know thousands or unlimited algorithms.

Evaluating AI Requires More Than Accuracy

A deep-learning algorithm with % 95 accuracy may contain critical information in its %5 errors. Confusion matrices therefore matter. These tables contain four combinations: true positive, true negative, false positive and false negative results.

True values are correct predictions; false values are mistakes. For pregnancy testing, a false positive predicts pregnancy in someone not pregnant, while a false negative is, conversely, described as a nonpregnant person predicted pregnant.

Errors in highly sensitive algorithms contain important information. Falsely diagnosing cancer can cause serious psychological distress.

The recent large-team study compared machines and radiologists using these outcomes.

Details of the Nature Study

Training used UK hospital mammograms from 2012 to 2015 in approximately 25 thousand women; testing used approximately 3 thousand US women. Three neural networks formed the architecture: Facebook's RetinaNet, Google's MobileNetV2, and Microsoft's ResNet V-1 50. A threshold determines cancer status, optimized during training to reduce false positives and negatives.

After training, testing on 500 mammograms showed fewer false positives (%9.4 lower) and false negatives (%5.4 lower), particularly compared with US radiologists. Each comparison involved one radiologist's decision. AI Did Not Show Significant Success Against Decisions Made Through Discussion by Two Radiologists.

It might remove the UK two-radiologist requirement and reduce time and cost, letting one radiologist decide better and faster with AI. But replacement claims remain unrealistic. MIT Technology Review specialist editor Will Knight says not knowing exactly how advanced algorithms do what they do may be problematic.

Neural networks are black boxes, with unclear explanations of predictions. Radiologists use additional tests when uncertain; current AI solutions predict only from images.

National Health Service Professor Ken Young says this study fits real-life scenarios, while earlier studies used more prepared datasets that could introduce bias.

Conclusion This academically and technologically strong study will advance health AI, but it has not yet achieved radiologist replacement as headlines suggest. In the near future, in diagnosis machines appear likely to substantially reduce demand for human experts. For treatment decisions AI still has much further to go.

*

The figure belowshows definitions and differences of 3 related new computing terms: artificial intelligence, machine learning, and deep learning.