Differential privacy (DP) is a framework for measuring the privacy guarantees provided by an algorithm. Through the lens of differential privacy, you can design machine learning algorithms that responsibly train models on private data. Learning with differential privacy provides measurable guarantees of privacy, helping to mitigate the risk of exposing sensitive training data in machine learning.
What are the different types of differential privacy?
There are two main types of differential privacy: global and local. Global differential privacy (GDP) applies noise to the output of an algorithm that operates on a dataset, such as a query or a model. Local differential privacy (LDP) applies noise to each individual data point before sending it to an algorithm, such as a survey or a telemetry system.
What are the applications of differential privacy?
Differential privacy has many applications in various domains, such as healthcare, social science, education, and business. For example, differential privacy can be used to protect the privacy of patients’ medical records while enabling researchers to analyze them for insights.
Differential privacy can also be used to protect the privacy of students’ test scores while allowing educators to evaluate their performance. Differential privacy can also be used to protect the privacy of customers’ preferences while allowing businesses to personalize their services.
Why is it called differential privacy?
The term differential privacy was coined by Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith in their seminal paper “Calibrating Noise to Sensitivity in Private Data Analysis” in 2006.
The name reflects the idea that the privacy guarantee of an algorithm should depend on the difference between two neighboring datasets, which differ by only one data point. Intuitively, this means that the presence or absence of any individual in the dataset should not affect the output of the algorithm significantly.
How do you prove differential privacy?
To prove differential privacy, you need to show that the probability of an algorithm producing a certain output is bounded by a function of the difference between two neighboring datasets and a privacy parameter. Mathematically, this can be expressed as follows:
Pr[A(D)=O]≤eϵPr[A(D′)=O]+δ
where A is the algorithm, D and D′ are neighboring datasets, O is the output, ϵ is the privacy parameter, and δ is the probability of failure.
What are the advantages of differential privacy?
Some of the advantages of differential privacy are:
- It provides a rigorous and quantifiable measure of privacy that is independent of the adversary’s prior knowledge and computational power.
- It is robust to composition, meaning that the privacy guarantee of multiple differentially private algorithms can be combined using simple rules.
- It is flexible and adaptable, meaning that it can be applied to various types of data, algorithms, and scenarios.
Who invented differential privacy?
Differential privacy was invented by Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith in their seminal paper “Calibrating Noise to Sensitivity in Private Data Analysis” in 2006. They were motivated by the problem of privacy-preserving data analysis, where the goal is to extract useful information from a dataset without compromising the privacy of the individuals in it.
They proposed a novel definition of privacy that captures the intuition that the output of an algorithm should not reveal much about any individual in the dataset.
What is the difference between Gaussian and Laplace differential privacy?
Gaussian and Laplace differential privacy are two common mechanisms for achieving differential privacy by adding noise to the output of an algorithm. The difference between them is the type and amount of noise they add. Gaussian differential privacy adds Gaussian noise, which follows a normal distribution with mean zero and standard deviation proportional to the sensitivity of the algorithm and the privacy parameter.
Laplace differential privacy adds Laplace noise, which follows a Laplace distribution with mean zero and scale parameter proportional to the sensitivity of the algorithm and the privacy parameter. Gaussian differential privacy provides (ϵ,δ)-differential privacy, where δ is the probability of failure, while Laplace differential privacy provides ϵ-differential privacy, where δ is zero.
What is differential privacy image classification?
Differential privacy image classification is the task of training a machine learning model to classify images while preserving the privacy of the images and their labels. This can be done by applying differential privacy to the training algorithm, such as stochastic gradient descent (SGD), which updates the model parameters using noisy gradients computed on batches of images.
Differential privacy image classification can help protect the privacy of sensitive images, such as faces, biometrics, or medical scans, while enabling useful applications, such as face recognition, biometric authentication, or medical diagnosis.
What is the promise of differential privacy?
The promise of differential privacy is to enable the analysis and use of data for beneficial purposes, such as research, innovation, and personalization, while protecting the privacy and dignity of the individuals in the data.
Differential privacy offers a principled and practical way to balance the trade-off between utility and privacy, by providing a clear and meaningful definition of privacy, a rigorous and quantifiable measure of privacy loss, and a flexible and adaptable framework for designing privacy-preserving algorithms.
Examples of differential privacy in AI
Some examples of successful differential privacy in AI and their track record are:
- Apple: Apple uses differential privacy to collect and analyze data from its users’ devices, such as keyboard usage, emoji preferences, web browsing patterns, and health metrics, while protecting their privacy and identity. Apple claims that it does not see or store the raw data, but only aggregates the noisy data to improve its products and services, such as Siri, Safari, and HealthKit.
- Google: Google uses differential privacy to collect and analyze data from its users’ web and app activity, such as Chrome usage, YouTube views, and Maps searches, while protecting their privacy and choice. Google claims that it does not link or combine the noisy data with other data, but only uses it to improve its products and services, such as Chrome, YouTube, and Maps.
- Microsoft: Microsoft uses differential privacy to collect and analyze data from its customers’ devices, such as Windows usage, Office productivity, and Xbox gaming, while protecting their privacy and security. Microsoft claims that it does not access or store the raw data, but only uses the noisy data to improve its products and services, such as Windows, Office, and Xbox.
Related terms
Some terms related to differential privacy in AI are:
- Privacy parameter: The privacy parameter, denoted by ϵ, is a measure of the privacy loss incurred by a differentially private algorithm. It controls the amount of noise added to the output of the algorithm. A smaller ϵ means more noise and more privacy, while a larger ϵ means less noise and less privacy.
- Sensitivity: The sensitivity, denoted by Δf, is a measure of the maximum change in the output of a function f when applied to two neighboring datasets. It determines the scale of the noise added to the output of the function to achieve differential privacy. A higher sensitivity means more noise and more privacy, while a lower sensitivity means less noise and less privacy.
- Noise: The noise, denoted by N, is a random variable that follows a certain distribution, such as Gaussian or Laplace, and is added to the output of a function f to achieve differential privacy. The noise obscures the influence of any individual data point on the output of the function, making it statistically indistinguishable from the output of the function applied to a neighboring dataset.
Conclusion
Differential privacy in AI is a promising and powerful technique for protecting the privacy of data while enabling its analysis and use for beneficial purposes. By adding noise to the output of an algorithm, differential privacy ensures that the presence or absence of any individual in the data does not affect the output significantly, thus providing a quantifiable and meaningful measure of privacy.
Differential privacy can be applied to various types of data, algorithms, and scenarios, such as image classification, web and app activity, and medical records. Differential privacy can also help optimize web pages for search engines and voice-based queries, by using semantically related words and natural language.
Differential privacy is a principled and practical way to balance the trade-off between utility and privacy, and to foster research, innovation, and personalization in AI.
References
- https://www.microsoft.com/en-us/ai/ai-lab-differential-privacy
- https://www.tensorflow.org/responsible_ai/privacy/tutorials/classification_privacy
- https://blogs.microsoft.com/on-the-issues/2020/12/10/differential-privacy-smartnoise-early-adopter-acceleration-program/
- https://www.apple.com/privacy/docs/Differential_Privacy_Overview.pdf
- https://developers.googleblog.com/2021/01/how-were-helping-developers-with-differential-privacy.html
- https://www.microsoft.com/en-us/ai/ai-lab-differential-privacy