A banner year for Rong Ge, Duke University Associate Professor of Computer Science, in 2019 he has received both a Faculty Early Career Development (CAREER) Award from the National Science Foundation (NSF) and a Sloan Research Fellowship.
The CAREER program is a Foundation-wide activity that offers NSF’s most prestigious awards in support of junior faculty who exemplify the role of teacher-scholars through outstanding research, excellent education and the integration of education and research within the context of the mission of their organizations.
With the five-year grant entitled “Optimization Landscape for Non-convex Functions — Towards Provable Algorithms for Neural Networks”, Ge intends to address optimization problems of deep-learning algorithms with a goal of bringing more theoretical insights into the heuristics for training neural networks. His research project will identify properties of the real-world problems that make these problems special and tractable, and provide new optimization algorithms with theoretical guarantees that are applicable to deep learning. Further, the materials developed in the project will be disseminated in conferences and workshops to connect different research communities, and also used to create new machine learning courses. The algorithms designed in the project will be made publicly available, and implemented in standard deep learning frameworks.
The prestigious Sloan Research Fellowship recognized Ge’s progress to date in making headway with machine learning algorithms’ inherent theoretical problems, which must be solved before the promises of artificial intelligence can happen.
Awarded to 126 scholars each year, the Sloan Fellowships provide support to promising early-career scientists and researchers in the United States and Canada. Candidates are nominated by their peers, and the winners are selected by a panel of senior scholars on the basis of their accomplishments and potential to become leaders in their fields.
“Sloan Research Fellows are the best young scientists working today,” says Adam F. Falk, president of the Alfred P. Sloan Foundation, which awards the fellowships. “Sloan Fellows standout for their creativity, for their hard work, for the importance of the issues they tackle, and the energy and innovation with which they tackle them. To be a Sloan Fellow is to be in the vanguard of twenty-first century science.”
“These awards mean a lot to me,” Ge says. “I’m happy that people like the work I’ve been doing. There are a lot of open problems in it still, and I’m just hoping to continue working on them.”
Ge joined the Department of Computer Science in 2015. Prior to joining Duke, he did post-doctoral research at Microsoft in New England. He received his Bachelor of Science degree in computer science from Tsinghua University in Beijing and his Doctor of Philosophy in computer science from Princeton.
Ge’s research focuses on how neural networks are trained. These networks are essential to machine learning; they are what allow machines to make decisions about new cases without human input. Facial recognition technology is one example. In order to determine whether a photo includes a human face, much less identify whose face it is, the machine needs a framework for analyzing the photo. The neural network provides that framework.
But before that can happen, the neural network must be uncovered. Which way of organizing the network, from all the countless possibilities, will allow the machine to generate the desired prediction about a new case? In facial recognition, the machine could take into account the colors of the photo, the size and shape of the photographed objects, the angle the photo was taken, and more. To guide the machine, computer scientists typically use manually coded examples, providing millions of photos labeled with specific information (like “includes a human face” and “does not include a human face”). As the machine sees more examples, it can create a more accurate neural network.
Finding the right parameters for the network so that it makes the best prediction is an optimization problem. “You want to find the best set of parameters given the data you have,” Ge says. He explains that most research on the issue focuses on a special kind of optimization problem called convex optimization. But machine learning is a non-convex optimization problem, which is more complicated because there can be more than one optimal solution.
In practice, however, very simple algorithms can solve these very complex problems. Ge’s work is focused on understanding why that is the case. Summarizing his work, Ge asks, “Why do we have a problem that is theoretically very difficult that we should not be able to solve, but in practice is solved by a very simple algorithm?”
While he has not yet discovered a complete solution to the dilemma, Ge has found why it works in some cases, like Amazon or Netflix recommendations. In those cases, even though the optimization problems are non-convex, all of the different optimal solutions are equally good, so it doesn’t matter which you find.
Going forward, Ge’s work will focus on expanding our understanding of these problems. Sloan Fellows and NSF Career recipients receive grants to fund their research, which Ge says he will use in part to hire a postdoctoral researcher to help apply his findings to more complicated forms of machine learning.
“We will first try to understand why the current algorithms are working so well, and then hopefully we can design new algorithms that work even better,” he says.
If he succeeds, he will help open the pathway to even more advanced kinds of artificial intelligence technology, like self-driving cars and personalized medicine.