More robust testing needed for privacy-preserving tools, experts show

Experts show current mechanisms of hiding patient information are inadequate, highlighting a need for more thorough testing of privacy systems.

In a new short paper published in Nature Medicine Matthieu Meeus, Shubham Jain and Dr Yves-Alexandre de Montjoye from the Computational Privacy Group analysed the use of digital masks to safeguard patient privacy in medical research.

The Imperial team showed that current methods of digital masking leave patients vulnerable to re-identification. 

“While sharing data for medical diagnosis is highly important, it should not come at the expense of patient privacy and therefore any anonymization methods need to be extensively and, if possible, adversarially tested to ensure privacy is preserved before the method is deployed and data is shared.” Matthieu Meeus First Author

By replicating the setup used by the original authors, they showed the risk of re-identification was at least 100 times higher than initially reported and that patients are re-identifiable 52.1% of the time.

According to First Author Matthieu Meeus: “This work highlights the importance of thorough evaluation when it comes to ensuring privacy.”

“While sharing data for medical diagnosis is highly important, it should not come at the expense of patient privacy and therefore any anonymization methods need to be extensively and, if possible, adversarially tested to ensure privacy is preserved before the method is deployed and data is shared.”

What is a digital mask?

A digital mask is a method proposed to protect patient privacy in medical research. The idea behind it involves applying a mask to a patient's face in a video to allow a doctor to make a diagnosis whilst withholding the patient's identity.

In theory, the mask retains information relevant for medical diagnosis while withholding any identifiable features, making the data anonymous.

In the evaluation setup used by the original authors, the method is shown to evade AI-powered facial recognition systems which underpins the claim that it preserves privacy. However, according to the Imperial team, these claims do not hold true.

The setup used by the original authors assumes that an attacker attempting to re-identify a patient will try and match a mask to a database of original faces using a readily available facial recognition algorithm.

However, the Imperial team demonstrated that an attacker can re-identify a patient by implementing a simple change to the setup, allowing them to mask the face before matching it to the "anonymous" database, making re-identification possible. 

Ensuring privacy-preserving claims are robust

As digital technologies are increasingly more embedded throughout society in the upcoming years, more and more new methods of preserving privacy are likely to be proposed.

The evaluation of these methods to ensure their privacy-preserving claims hold true is a crucial step in their development. According to Senior Author Dr Yves-Alexandre de Montjoye, this can be achieved through proper adversarial testing.

Some large companies are already doing this, building privacy ‘red teams’ to test the privacy of their systems. Similar to their security counterparts, these red teams are a group of ethical engineers employed by a company to attack the system to extract information about the data it is meant to protect in order to understand whether the system is solid or not. 

The Computational Privacy Group are also researching how AI can help red teams automatically discover vulnerabilities in their systems. Find out more in this Imperial News Story.

-

‘Concerns about using a digital mask to safeguard patient privacy’ by Meeus, Jain and de Montjoye, published in Nature Medicine on 18 July 2023.

Photo by Matthew Yohe, edited by Meeus et al.