Ronald Parr

Research Interests

Methods for solving large stochastic planning problems (MDPs) using approximate dynamic programming techniques (reinforcement learning, value function approximation, or policy search). Recently, I have focused on feature selection and regularization for linear value function approximation techniques recently. I also work in game theory, sensing, and robotics.

Education

• Ph.D. in Computer Science, University of California, Berkeley, 1998. Thesis title: Hierarchical Control and Learning in Markov Decision Processes (Advisor: Stuart Russell).
• A.B. in Philosophy cum laude, Princeton University, 1990.

Awards and Honors

• IJCAI-JAIR Best Paper Award, 2007
• DARPA CSSG participant, 2006
• NSF CAREER award, 2006
• Alfred P. Sloan Fellow, 2003

Employment History

• Associate Professor (with tenure), Department of Computer Science, Duke University, 2007-present.
• Assistant Professor, Department of Computer Science, Duke University, 2000-2007.
• Postdoctoral Research Associate, Robotics Laboratory, Stanford University, 1998-2000. Worked with Daphne Koller on research, student advising and research grant-related responsibilities.
• Research Assistant/Independent Research, Department of Electrical Engineering and Computer Science, U.C. Berkeley, 1991-1998. Advisor: Stuart Russell.
• Graduate Student Instructor: Formal Languages and Automata Theory, Department of Electrical Engineering and Computer Science, U.C. Berkeley, 1990-1991.

Professional Service

• IJCAI Senior Program Committee
• Journal of Machine Learning Research (JMLR) Action Editor 2009-
• NIPS reviewer 1999, 2001, 2002, 2003, 2004, 2005, 2008, 2010, 2011
• UAI General Chair 2008
• AAAI 2008 Senior Program Committee
• ISAIM Program Committee 2008, 2010
• ICML Program Committee 2000, 2001, 2003-2006, 2008
• UAI Program Co-chair 2007
• IJCAI 2007 Program Committee
• Machine Learning (MLJ) Editorial Board 2006-2009
• RSS Area Chair 2006
• UAI Area Chair 2002-2006
• IJCAI Reviewer 1999, 2001, 2003
• AI & Math Program Committee 2004, 2006
• AAAI Program Committee 1998, 2000, 2002, 2004, 2005
• IJCAI Program Committee (senior level) 2005
• IJCAI 2005 RUR (Reasonsing with Uncertainty in Robotics) workshop reviewer
• Symposium on Artificial Intelligence and Mathematics Program Committee 2003, 2005
• GAFOS program committee 2004
• ICAPS Program Committee 2003
• Journal of AI Research (JAIR) editorial board member 2000-2002
• UAI program committee 2000, 2001
• AAAI 2000 student poster reviewer
• NASA Intelligent Systems Program reviewer 2000
• Organizer of NIPS 98 workshop on Abstraction and Hierarchy in Reinforcement Learning.
• Review papers for Adaptive Behavior, AIJ, ECAI, IEEE TRO, JAIR, JMLR, MLJ, JAAMAS, RAS

Department Service

• Faculty search committees: 2001-2003, 2005, 2006, 2008, 2010-2011
• Lab Committee: 2004-2011
• Publications Committee: 2004-2011
• Graduate Admissions Committee: 2007
• Strategic Planning Committee: 2002
• Admissions Committee: 2001-2002

Classes Taught

• CPS 271 - Machine Learning (Fall 2003, 2005, 2007, 2009, 2011)
• CPS 170 - Artificial Intelligence (undergraduate) (Spring 2002, 2003, 2004, 2006, 2010, 2011)
• CPS 270 - Artificial Intelligence (graduate) (Fall 2001, 2002, 2004, 2006, 2010)
• CPS 1/296 - Robotics (mixed undergrade and graduate) (Spring 2007)
• CPS 271 - Numeric Artificial Inteligence (Spring 2001)
• CPS 370 - Planning Under Uncertainty (Fall 2000)

Presentations

• U.T. Austin Forum for Artificial Intelligence, 2011.
• Google Distinguished Lecturer, CMU, 2010.
• ARTSI aliance Faculty Workshop presenter, 2010.
• Presenter, 2008 ICML.
• SAMSI Workshop Tutorial on Markov Decision Processes and Reinforcement Learning, 2007.
• Duke MURI Workshop on Adaptive Multi-Sensor Sensing and Waveform Scheduling, 2006.
• NSF Approximate Dynamic Programming Workshop, technical talk and tutorial, 2006.
• Invited talk at ICML Hierarchical Reinforcement Learning Workshop 2005
• Seminars at Princeton, Univ. of Massachusetts, Middlebury, Stony Brook 2005.
• Invited panelist on AFOSR funded workshop on Decision Making in Adversarial Domains
• Seminars at Univ. of Alberta, Carnegie Mellon and NASA Ames 2004.
• Invited participant in GAFOS workshop 2004.
• Daghstuhl 2003 invited tutorial and research talk
• LICS 2003 Workshop on Probability and AI invited talk
• Daghstuhl 2001 invited tutorial and research talk
• Poster Spotlight, 2001 NIPS.
• Duke ISDS Colloquium 2001.
• Presenter, 2000 UAI.
• Presenter, 1999 IJCAI.
• Invited talks at NASA and SRI, 1999.
• Plenary Presentation, 1998 UAI conference.
• Presenter, 1998 SARA Symposium.
• Full Oral presentation, 1997 NIPS conference.
• Presenter, 1997 ICML Workshop on Reinforcement Learning.
• Presenter, 1997 Joint Brazil, US Workshop on Intelligent Robotic Agents.
• Presenter, 1996 AAAI Fall Symposium.
• Presenter, 1996 NSF Reinforcement Learning Workshop.
• Presenter, 1995 IJCAI.
• Invited participant, 1995 Joint AI/OR Workshop.
• Guest lectured for Computer Science and Cognitive Science classes at Berkeley and Stanford.

Journal Papers

• Counting Objects with a Combination of Horizontal and Overhead Sensors, Erik Halvorson and Ronald Parr, The International Journal of Robotics Research, June 2010, 29(7), pp. 840-854.
• Non-Myopic Multi-Aspect Sensing with Partially Observable Markov Decision Processes, Shihao Ji, Ronald Parr, and Lawrence Carin, IEEE Transactions on Signal Processing, June 2007 Volume 55, Issue: 6, Part 1, 2007, pp. 2720-2730.
• Least-Squares Policy Iteration, Michail Lagoudakis and Ronald Parr, Journal of Machine Learning Research (JMLR), Vol. 4, 2003, pp. 1107-1149.
• Efficient Solution Algorithms for Factored MDPs, Carlos Guestrin, Daphne Koller, Ronald Parr and Shobha Venkataraman, Journal of Artificial Intelligence Research (JAIR), Vol. 19, 2003, pp. 399-468. (Recipient of IJCAI-JAIR best paper awards for 2007.)

Books or Edited Volumes

• Proceedings of the 23^rd Conference on Uncertainty in Artificial Intelligence, Ronald Parr and Lind van der Gaag, eds., 2007.

Highly Refereed Papers (also includes WAFR and ICRA)

• Generalized Value Functions for Large Action Sets, Jason Pazis and Ronald Parr, Proceedings of the Twenty-Eighth International Conference on Machine Learning (ICML-2011). [26% acceptance rate]
• Non-parametric Approximate Linear Programming for MDPs, Jason Pazis and Ronald Parr, Proceedings of the Twenty-Fifth AAAI conference on Artificial Intelligence (AAAI-2011). [25% acceptance rate]
• Security Games with Multiple Attacker Resources, Dmytro Korzhyk, Vincent Conitzer, Ronald Parr, Proceedings of the Twenty-second International Joint Conference on Artificial Intelligence (IJCAI-2011). [19% acceptance rate]
• Textured Occupancy Grids for Monocular Localization Without Features, Julian Mason, Susanna Ricco, Ronald Parr, 2011 IEEE International Conference on Robotics and Automation (ICRA 2011). [49% acceptance rate]
• Solving Stackelberg Games with Uncertain Observability, Dmytro Korzhyk, Vincent Conitzer, Ronald Parr, The Tenth International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2011). [22% acceptance rate]
• Linear Complementarity for Regularized Policy Evaluation and Improvement, Jeff Johns, Christopher Painter-Wakefield, Ronald Parr, to appear at NIPS 23. [24% acceptance rate; less than 2% of papers were selected for an oral presentation, of which this was one]
• Feature Selection Using Regularization in Approximate Linear Programs for Markov Decision Processes, Marek Petrik, Gavin Taylor, Ronald Parr, and Shlomo Zilberstein, International Conference on Machine Learning (ICML-2010), pp. 871-878. [26% acceptance rate]
• Complexity of Computing Optimal Stackelberg Strategies in Security Resource Allocation Games, Dmytro Korzhyk, Vincent Conitzer, and Ronald Parr, Proceedings of the 24th National Conference on Artificial Intelligence (AAAI-10) [27% acceptance rate].
• Kernelized Value Function Approximation for Reinforcement Learning, Gavin Taylor and Ronald Parr, International Conference on Machine Learning (ICML-2009) [27% acceptance rate].
• Multi-step Multi-sensor Hider-seeker Games, Erik Halvorson, Vincent Conitzer, and Ronald Parr, Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence (IJCAI-2009), pp. 159-166. [26% acceptance rate]
• Planning Aims for a Network of Horizontal Overhead Sensors, Erik Halvorson and Ronald Parr, Workshop on the Algorithmic Foundations of Robotics 2008 (WAFR-2008). [65% acceptance rate]
• An Analysis of Linear Models, Linear Value-Function Approximation, and Feature Selection for Reinforcement Learning, Ronald Parr, Lihong Li, Gavin Taylor, Christopher Painter-Wakefield, and Michael L. Littman, International Conference on Machine Learning (ICML-2008), pp. 752-759. [27% acceptance rate]
• Point-Based Policy Iteration, Shihao Ji, Ronald Parr, Hui Li, Xuejun Liao, and Lawrence Carin, Proceedings of the Twenty-Second National Conference on Artificial Intelligence (AAAI 2007), pp 1243-1249. [27% acceptance rate]
• Analyzing Feature Generation for Value-Function Approximation, Ronald Parr, Christopher Painter-Wakefield, Lihong Li, and Michael Littman, International Conference on Machine Learning (ICML-2007). [29% acceptance rate]
• Efficient Selection of Disambiguating Actions for Stereo Vision, Monika Schaeffer and Ronald Parr, Proceedings of the Twenty Second Conference on Uncertainty in Artificial Intelligence (UAI-2006), pp. 418-427. [plenary presentation; 32% acceptance overall; 12% of submitted papers were given plenary presentations]
• Hierarchical Linear/Constant Time SLAM using Particle Filters for Dense Maps, Austin I. Eliazar and Ronald Parr, Advances in Neural Information Processing Systems (NIPS-19) 2005. [27% acceptance rate]
• Learning Probabilistic Motion Models for Mobile Robots, Austin I. Eliazar and Ronald Parr, Proceedings of the Twenty First International Conference on Machine Learning (ICML-2004), pp. 249-256. [32% acceptance rate]
• DP-SLAM 2.0, Austin Eliazar and Ronald Parr, IEEE 2004 International Conference on Robotics and Automation (ICRA 2004). [58% acceptance rate]
• Reinforcement Learning as Classification: Leveraging Modern Classifiers, Michail Lagoudakis and Ronald Parr, Proceedings of the Twentieth International Conference on Machine Learning (ICML-2003). [32% acceptance rate]
• DP-SLAM: Fast, Robust Simultaneous Localization and Mapping without Predetermined Landmarks, Austin Eliazar and Ronald Parr, Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI 03). [21% acceptance rate]
• Learning in Zero-Sum Team Markov Games using Factored Value Functions Michail Lagoudakis and Ronald Parr, Advances in Neural Information Processing Systems (NIPS-15) 2002. [30% acceptance rate]
• Value Function Approximation in Zero-Sum Markov Games Michail Lagoudakis and Ronald Parr, Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence (UAI 2002), pp. 283-292, August 2002. [34% acceptance rate]
• Coordinated Reinforcement Learning, Carlos Guestrin, Michail Lagoudakis,and Ronald Parr. Nineteenth International Conference on Machine Learning (ICML-2002), pp. 227-234, July 2002. [33% acceptance rate]
• XPathLearner: An On-Line Self-Tuning Markov Histogram for XML Path Selectivity Estimation, Lipyeow Lim, Min Wang, Sriram Padmanabhan, Jeffrey Scott Vitter, Ronald Parr. Twenty Eighth International Conference on Very Large Databases. (VLDB 2002), pp. 442-453, August 2002. [16% acceptance rate]
• Multiagent Planning with Factored MDPs, Carlos Guestrin, Daphne Koller and Ronald Parr, Advances in Neural Information Processing Systems 2001 (NIPS-14), pp. 1532-1530, December 2001. [Accepted for a full oral presentation. Less than 4% of submitted papers got oral presentations this year.]
• Model-Free Least-Squares Policy Iteration Michail Lagoudakis, Ronald Parr, Advances in Neural Information Processing Systems 2001 (NIPS-14), pp. 1547-1554, August 2001. [Accepted for a poster spotlight. Approximately 10% of submitted papers got poster spotlights this year.]
• Inference in Hybrid Networks: Theoretical Limits and Practical Algorithms, Uri Lerner, Ronald Parr, Uncertainty in Artificial Intelligence, Proceedings of the Seventeenth Conference (UAI 2001), pp. 310-318, August 2000. [Joint winner of best student paper award (student first author).]
• Max-norm Projections for Factored MDPs, Carlos Guestrin, Daphne Koller and Ronald Parr, Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence (IJCAI 2001), pp. 673 - 680, August 2001. [25% acceptance rate]
• Policy Iteration for Factored MDPs, Daphne Koller and Ronald Parr, Uncertainty in Artificial Intelligence, Proceedings of the Sixteenth Conference (UAI 2000), pp. 326-334, June 2000. [45% acceptance rate]
• Bayesian Fault Detection and Diagnosis in Dynamic Systems, Uri Lerner, Ronald Parr, Daphne Koller and Gautam Biswas, Proceedings of the Twelfth National Conference on Artificial Intelligence (AAAI 2000), pp. 531-537, July 2000. [33% acceptance rate]
• Making Rational Decisions Using Adaptive Utility Elicitation, Urszula Chajewska, Daphne Koller, Ronald Parr, Proceedings of the Twelfth National Conference on Artificial Intelligence (AAAI 2000), pp. 363-369, July 2000. [33% acceptance rate]
• Policy Search via Density Estimation, Andrew Y. Ng, Ronald Parr, Daphne Koller, Neural Information Processing Systems 1999 (NIPS 99), pp. 1022-1028, December 1999. [Accepted for a poster spotlight. 10% of submitted papers received poster spotlights.]
• Reinforcement Learning using Approximate Belief States, Andrés Rodgríguez, Ronald Parr, Daphne Koller, Neural Information Processing Systems 1999 (NIPS 99), pp. 1036-1042, December 1999. [32% acceptance rate]
• Computing Factored Value Functions for Policies in Structured MDPs, Daphne Koller, Ronald Parr, Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence (IJCAI 1999), pp. 1332-1339, July 1999. [26% acceptance rate]
• Flexible Decomposition Algorithms for Weakly Coupled Markov Decision Problems, Ronald Parr, Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence (UAI-98), pp. 422-430, July 1998. [45% acceptance rate]
• Reinforcement Learning with Hierarchies of Machines, Ronald Parr, Stuart Russell, Neural Information Processing Systems 1998 (NIPS 98), pp. 1043-1049, December 1998. [Accepted for a full oral presentation. 4.5% of submitted papers were given an oral presentation.]
• Generalized Prioritized Sweeping, David Andre, Nir Friedman, Ronald Parr, Neural Information Processing Systems 1998 (NIPS 98), pp. 1001-1007, December 1998. [31% acceptance rate]
• Approximating Optimal Policies for Partially Observable Stochastic Domains, Ronald Parr, Stuart Russell, Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence (IJCAI-95), pp. 1088-1094, August 1995. [22% acceptance rate]
• Provably Bounded Optimal Agents, Stuart Russell, Devika Subramanian, Ronald Parr, in Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence (IJCAI-93), pp. 338-344, August, 1993. [25% acceptance rate]

Other Papers

• L1 Regularized Linear Temporal Difference Learning, Duke CS Technical Report TR-2012-01 Christopher Painter-Wakefield and Ronald Parr, 2012.
• Planning Aims for a Network of Horizontal and Overhead Sensors, Erik Halvorson and Ronald Parr, International Symposium on Artificial Intelligence and Mathematics 2008 (ISAIM 2008).
• Counting Objects with a Combination of Horizontal and Overhead Sensors, Erik Halvorson and Ronald Parr, Duke CS Technical Report CS-2007-04. (Longer version of the above ISAIM 2008 symposium paper.)
• Efficient Selection of Disambiguating Actions for Stereo Vision, Monika Schaeffer and Ronald Parr, NIPS 2005 Workshop on Value of Information in Inference, Learning and Decision Making.
• Least-Squares Methods in Reinforcement Learning for Control, Michail Lagoudakis, Ronald Parr and Michael L. Littman. Second Hellenic Conference on Artificial Intelligence (SETN-02).
• Model-Free Least-Squares Policy Iteration, Michail Lagoudakis and Ronald Parr, Duke University Technical Report. (Longer version of above NIPS 2001 paper.)
• Coordinated Reinforcement Learning, Carlos Guestrin, Michail Lagoudakis,and Ronald Parr. Proceedings of the 2002 AAAI Spring Symposium Series: Collaborative Learning Agents
• Selecting the Right Algorithm Michail Lagoudakis, Michael L. Littman and Ronald Parr, Proceedings of the 2001 AAAI Fall Symposium Series: Using Uncertainty within Computation, Cape Cod, MA, November 2001.
• Solving Factored POMDPs with Linear Value Functions, Carlos Guestrin, Daphne Koller and Ronald Parr, In the IJCAI-01 workshop on Planning under Uncertainty and Incomplete Information.
• Max-norm Projections for Factored MDPs, Carlos Guestrin, Daphne Koller, and Ronald Parr, AAAI Spring Symposium, Stanford, California, March 2001.
• Adaptive Utility Elicitation using Value of Information, Chajewska, U., M. Kuppermann, R. Parr and D. Koller. (Abstract), Presented at the 22nd Annual Meeting of the Society for Medical Decision Making, 2000).
• Hierarchical Control and Learning for Markov Decision Processes, Ph.D. Dissertation, 1998, University of California, Berkeley.
• A Unifying Framework for Temporal Abstraction in Stochastic Processes, Ronald Parr, Symposium on Abstraction Reformulation and Approximation, 1998 (SARA-98).
• Feasibility Study of Fully Automated Vehicles Using Decision-theoretic Control, Jeffrey Forbes, Nikunj Oza, Ronald Parr, Stuart Russell, California PATH Research Report, UCB-ITS-PRR-97-18.
• Policy Based Clustering in Markov Decision Problems, AAAI-96 Fall Symposium on Learning Complex Behaviors in Adaptive Intelligent Systems.

Funding

• NSF EAGER: Learning in Continuous and High Dimensional Action Spaces, $150,000.
• IGERT: Training Program in Wireless Intelligent Sensor Networks (WISeNet), $3,000,000 (subject to continuing approval), with Silvia Ferrari, Gabriel G. Katul, John D. Albertson, and Pankaj K. Agarwal. Parr partion is difficult to assess since this is a training grant and student participation in the grant is not decided a priori.
• CRA CI Fellow Award, $140,000, funding for postdoc Jeff Johns, (Parr portion depends upon interpretation. The funding process is initiated by and attached to the postdoc, but it is also associated with a particular mentor. For bureaucratic reasons, the funding is associated completely with Parr within Duke because postdocs cannot serve as PI's.) 2009.
• ARO: Efficient Algorithms for Computing Stackelberg Strategies in Security Games, $180,000, Joint with Vincent Conitzer. (This primarily funds one of Conitzer's students, so the Parr portion is neglible.) 2009.
• U.S. Department of Education GAANN grant, $384,000, (Joint with multiple investigators in the department.), Parr portion: $0, (The Department of Education has determined that my students are too wealthy to receive funding from this grant, so the funds have been allocated to other students in the department.) 2007.
• NSF RI: Feature Discovery and Benchmarks for Exportable Reinforcement Learning, $450,000, (Joint with Michael Littman from Rutgers University. Duke/Parr portion $225,000), 2007.
• DARPA CSSG year 2 funding, $500,000, 2007.
• NSF CAREER: Observing to Plan, Planning to Observe, $440,000, 2006.
• DARPA CSSG grant, $80,000, 2006.
• SAIC gift in support of robotics research, $80,000, 2006.
• SAIC gift of 3 research robots, approx. value $50,000, 2006.
• Computing a Semantic View of a Scene for Surveillance from Stereo and Discreet LIDAR, with Carlo Tomasi and Industrial Partner IAI, DARPA SBIR Phase I. Duke portion: $22,000, Parr portion $11,000, 2005-2006. (Duke refused to sign the contract for this grant.)
• A Core Experimental Facility for Computer Vision and Artificial Intelligence, NSF CRI grant (joint with Carlo Tomasi), $300,000, 2005.
• SAIC gift in support of robotics research, $80,000, 2005.
• SAIC gift for graduate student support, $20,000, 2004.
• Sloan Research Fellowship, Alfred P. Sloan Foundation, $40,000, 2003-2004, extended 1 year.
• Prediction and Planning: Bridging the Gap, National Science Foundation, $291,677. Official start date: 9/02, 3 year duration, extended 1 year.

Postdocs Supervised

• Jeff Johns, 2009-2010

Students Advised

• Jason Pazis (Third Year)
• Mac Mason (Sixth Year)
• Christopher Painter-Wakefield (Seventh year)
• Monika Schaeffer (Master's in Spring 2011)
• Gavin Taylor (Ph.D. in Spring 2011, Assistant Professor at the United States Naval Academy)
• Erik Halvorson, Master's in Summer 2008.
• Neeti Wagle, Master's in Spring 2008.
• Austin Eliazar, Ph.D. in Fall 2005, now at Teledyne Scientific.
• Michail Lagoudakis, Ph.D. in Spring 2003, now assistant professor, Technical University of Crete, Greece.

Memberships

AAAI

Citizenship

Natural born U.S. citizen.

Contact Information

Department of Computer Science
Duke University
LSRC / Box 90129
Durham, NC 27708
(919)660-6537
http://www.cs.duke.edu/~parr