This kind of technology is used in systems like Facebook and Google Maps, but this is the first time AI has been used to automatically detect weaknesses in them.
The specialists from Imperial’s Computational Privacy Group looked at assaults on managed interfaces known as query-based systems (QBS), which analysts use to query data and produce helpful global statistics. In order to find QBS assaults, they then developed a brand-new AI-enabled technique dubbed QuerySnout.
Analysts get access to statistics collections created using demographic and location-based personal data thanks to QBS. They are currently utilized in Google Maps to show real-time data on how busy an area is, or in Facebook’s audience measurement function to calculate the size of the audience in a particular region or demographic to support advertising efforts.
The Data Science Institute team, which also includes Ana Maria Cretu, Dr. Florimond Houssiau, Dr. Antoine Cully, and Dr. Yves-Alexandre de Montjoye, found that effective attacks against QBS may be promptly and automatically recognized by clicking a button in their most recent study.
At the 29th ACM Conference on Computer and Communications Security, the paper was presented.
This kind of technology is used in systems like Facebook and Google Maps, but this is the first time AI has been used to automatically detect weaknesses in them.
The specialists from Imperial’s Computational Privacy Group looked at assaults on managed interfaces known as query-based systems (QBS), which analysts use to query data and produce helpful global statistics. In order to find QBS assaults, they then developed a brand-new AI-enabled technique dubbed QuerySnout.
Analysts get access to statistics collections created using demographic and location-based personal data thanks to QBS.
The Need for Query-Based Systems:
We are now much more capable of collecting and storing data than we were ten years ago. Even if a large portion of this data is personal and as such is covered by regulations like the EU’s General Data Protection Regulation, its use nonetheless poses significant privacy issues.
How to allow data to be utilized for benefit while retaining our fundamental right to privacy is thus a topical and crucial question for data scientists and privacy professionals.
Scalable anonymous data analysis that respects privacy may be made possible via QBS. In QBS, curators keep ownership of the data and are able to carefully review and analyze the queries that analysts send to make sure that any information that is returned does not contain personally identifying information.
However, bad attackers might circumvent such systems by designing queries that infer personal information about particular people by exploiting implementation or system weaknesses.
Testing the System:
Due to the dangers of unknown, potent “zero-day” assaults, in which attackers take advantage of security holes in systems, the development and implementation of QBS have been put on hold.
Similar to penetration testing in cyber-security, data breach attacks can be mimicked to uncover information leaks and potential vulnerabilities and evaluate the resiliency of these systems.
However, the human design and execution of these attacks against complicated QBS is a difficult and time-consuming procedure.
The researchers contend that in order to use QBS securely and effectively while preserving individual privacy rights, it is essential to restrict the likelihood of strong, unabated attacks.
QuerySnout:
The Imperial team developed a novel AI-enabled method called QuerySnout that works by teaching the user what questions to ask the system in order to get results. The next step is learning how to automatically integrate the responses in order to identify any potential privacy flaws.
In order to reveal a specific piece of confidential information, the model can create an attack utilizing a sequence of queries and a mixture of responses. Through the use of a fully automated procedure known as “evolutionary search,” the QuerySnout model may learn the appropriate sets of questions to ask.
This happens in a “black-box setting,” which means that in order to uncover the vulnerabilities, the AI just requires access to the system and is not required to comprehend how it works.
We demonstrate that QuerySnout finds more powerful attacks than those currently known on real-world systems. This means our AI model is better than humans at finding these attacks.
Ana-Maria Cretu, Study Co-First Author and PhD Student, Imperial College London
Source: https://www.imperial.ac.uk/
