My research currently is focused on planning under uncertainty and reinforcement learning (RL). Unlike classical planning, modern planning tries to deal directly with uncertainty. Uncertainty can appear due to noise, e.g. in a sensor or in a robotic actuator, due to inherent stochasticity in the environment, or due to incomplete information. Planning under uncertainty is naturally modeled using probabilistic models such as Markov decision processes (either fully or partially observable). Planning finds the action policy (or strategy) which maximizes the rewards obtained by an agent given a model of the world. Reinforcement learning accomplishes planning without benefit of the model.
While the basic algorithms for solving Markov decision processes are well understood, solving large problems requires approximation techniques. In particular, with a large (or infinite) number of possible states, an exact representation of a policy and its expected value may be impossible. A well understood, frequently used approach in these cases is to use some form of linear value function approximation (VFA), in which the value is represented as a linear combination of features or basis functions.
An open problem in planning and RL is how to obtain good features for linear VFA. Human-designed features are often compact and effective, but designing good features is difficult and time consuming. Generic, "off the shelf" features such as kernel functions appear to sidestep the design difficulty, but in practice these only shift the problem to one of choosing the type and parameters of the features; worse, generic feature sets are seldom small and compact, which leads to increases in computational costs. My research looks at alternatives to traditional feature design: automating the process of feature discovery or generation; or leveraging existing techniques from supervised learning for selecting compact feature sets from large dictionaries of (typically generic) features.