Algorithmic Challenges in Efficient Training of Private (Deep) Language Models
Many attacks have shown that deep learning models trained on private data of users can leak sensitive information of the users. Differential Privacy is a provable way to prevent such attacks. However, training deep learning models using DP introduces several new challenges both in terms of privacy vs accuracy tradeoffs and in the resource cost of the process. In this talk, I will highlight some of the problems we encountered, our solutions for resolving them and mention many important open problems.
Janardhan Kulkarni is a senior researcher in the Algorithms group at MSR, Redmond. His primary research focus is combinatorial optimization and differential privacy. In particular, he is interested in solving optimization problems that arise in building efficient data centers and addressing algorithmic challenges in developing private (deep) learning models. He obtained his Ph.D from Duke University under the supervision of Kamesh Munagala.