The mission of the Software and Systems Division (SSD) at the National Institute of Standards and Technology (NIST) is to work with industry, academia and other government agencies to accelerate the development and adoption of correct, reliable, testable software, leading to increased trust and confidence in deployed software; promulgate methods to develop better standards and testing tools for today's software infrastructures and tomorrow's next-generation software systems; advance the state of the art of software testing by developing scientifically rigorous, breakthrough techniques
Deep Neural Networks (DNNs) enable computers to excel across many different applications such as image classification, speech recognition and robotics control. To accelerate DNN training and serving, parallel computing is widely adopted. System efficiency is a big issue when scaling out. In this talk, I will make three arguments towards better system efficiency in distributed DNN training and serving.