Targeted Machine Unlearning using Gradient Masking – Sharjeel Masood

Machine Unlearning is the process of selectively removing specific knowledge or training data from a deployed ML model. This is crucial for addressing privacy concerns and ensuring model fairness by correcting biases or errors. Simple data deletion is insufficient, as the data's influence remains encoded in the model's weights.

Imagine you have trained a deep learning model for hundreds of hours on a massive dataset. During this extensive training, your model may have learned an undesirable bias towards a specific class, or perhaps you've discovered slight inaccuracies in your dataset labeling. You may now wish for the model to adapt to the corrected data, or feel the need to remove certain specific information from the network due to privacy concerns or other reasons.

While all of these objectives can be achieved by retraining the entire model, this is inefficient—especially for large models that require days of computational resources.

What if there was a way to identify the small parts of the model primarily responsible for the data you wish to remove, and partially retrain only those components without negatively impacting the model's performance on the rest of the data?

In this talk, I will present a gradient masking mechanism capable of identifying and training a specific, small group of neurons while keeping the rest of the model untouched. This mechanism is effective in partially modifying the network but can also be used in various other creative ways, such as identifying the least influential neurons for model pruning and compression.

About Sharjeel Masood