A new study has revealed that sensitive health information about individual patients can be extracted from medical artificial intelligence (AI) models more effectively than previous research had indicated.
Diagnostic AI models are used to determine whether some given health data belong to a specific class – for example if an X-ray image shows evidence of pneumonia – and offer huge promise for improving patient outcomes. Earlier studies showed that membership inference attacks (MIAs), which determine whether a particular person’s data was used to train an AI model, yielded results no better than random chance. But these previous studies only evaluated the average risk across all patients in a training dataset.
By contrast, in this latest work doctoral student Moritz Knolle and his supervisor Daniel Rückert at the Technical University of Munich (TUM), together with collaborators at Imperial College London and Georg Kaissis from the Hasso Plattner Institute (HPI), examined the privacy risk for an individual patient. “Privacy that only works on average has little meaning. So I asked myself: would I contribute my data to train a medical AI model?” says Knolle.
As detailed in their recent Nature paper, the researchers attacked AI models based on seven different types of real-life medical datasets, including one comprising 21,799 electrocardiograms and another of 377,110 chest radiographs.
“If a model sees a specific data point during training, it usually becomes more confident at predicting it. So in these MIA attacks I compare the predictive probability number for this data point outputted by the medical AI model with the predictive probability number obtained from my model which has not seen that data point,” explains Knolle, adding that the attack is essentially a statistical test to distinguish between the two models.
Their results revealed that the privacy risk for an individual was far greater than the average risk. The team also found that privacy risk is not equally distributed among subgroups.
“We found that patients from minority groups that are under-represented in the training dataset tend to be over-represented among the 99 percentile of extreme [privacy] risk,” Knolle tells Physics World, warning that as medical AI models become larger and more sophisticated the privacy risk is amplified because their deep analysis creates the small changes in predictive probability that MIAs are good at finding.
While an MIA that reveals someone is a UK citizen poses little security risk, discovering that an individual has received a specific type of cancer treatment reveals personal information that could, for instance, adversely affect their medical insurance premiums. “The more specific the training cohort is, the larger the privacy risk,” states Knolle, crediting his medical doctor parents with inspiring his desire to improve healthcare AI.
Countermeasures include differential privacy (DP), in which a small amount of Gaussian random white noise is added in alongside the data during training, thereby reducing the effectiveness of MIAs while maintaining the model’s diagnostic accuracy. “This gives a mathematically provable upper bound for how much privacy risk any individual in this dataset could face, by limiting the information content that flows to the AI model,” explains Knolle.
Diagnostic AI algorithm focuses on privacy protection
He notes that, unfortunately, such DP tools have not been widely adopted by hospitals or clinics. This, he fears, could undermine patients’ confidence in sharing their data, and thereby not only reduce AI’s ability to help combat rare diseases, but also compound AI discrimination that occurs when there is too little data from minority groups included in training datasets.
“So we are hoping to inspire the medical AI community to do risk assessments before deploying AI models, and use verifiable strategies such as DP or limiting access to the AI model, so that as an individual you have a statistical guarantee that no harm would come to you by being part of a dataset,” concludes Knolle, who now plans to investigate the privacy risks posed by generative AI models for healthcare.
