Ronald Parr

Research Interests

Methods for solving large stochastic planning problems (MDPs): dynamic programming, reinforcement learning, and policy search. I am interested in techniques for decomposing large problems using hierarchy, problem structure and prior knowledge. I'm also interested in Bayesian networks, tracking, learning probabilistic robotics and, generally, most forms of reasoning under uncertainty.

Education

• Ph.D. in Computer Science, University of California, Berkeley, 1998. Thesis title: Hierarchical Control and Learning in Markov Decision Processes (Advisor: Stuart Russell).
• A.B. in Philosophy cum laude, Princeton University, 1990.

Awards and Honors

• IJCAI-JAIR Best Paper Award, 2007
• DARPA CSSG participant, 2006
• NSF CAREER award, 2006
• Alfred P. Sloan Fellow, 2003

Employment History

• Associate Professor, Duke University, 2007-present.
• Assistant Professor, Duke University, 2000-2007.
• Postdoctoral Research Associate, Stanford University, 1998-2000. Worked with Daphne Koller on research, student advising and research grant-related responsibilities.
• Research Assistant/Independent Research, U.C. Berkeley, 1991-1998. Advisor: Stuart Russell.
• Graduate Student Instructor: Formal Languages and Automata Theory, 1990-1991.

Professional Service

• NIPS reviewer 1999, 2001, 2002, 2003, 2004, 2005, 2008
• UAI General Chair 2008
• AAAI 2008 Senior Program Committee
• ISAIM 2008 Program Committee
• ICML Program Committee 2000, 2001, 2003-2006, 2008
• UAI Program Co-chair 2007
• IJCAI 2007 Program Committee
• Machine Learning (MLJ) Editorial Board 2006-
• RSS Area Chair 2006
• UAI Area Chair 2002-2006
• IJCAI Reviewer 1999, 2001, 2003
• AI & Math Program Committee 2004, 2006
• AAAI Program Committee 1998, 2000, 2002, 2004, 2005
• IJCAI Program Committee (senior level) 2005
• IJCAI 2005 RUR (Reasonsing with Uncertainty in Robotics) workshop reviewer
• Symposium on Artificial Intelligence and Mathematics Program Committee 2003, 2005
• GAFOS program committee 2004
• ICAPS Program Committee 2003
• Journal of AI Research (JAIR) editorial board member 2000-2002
• UAI program committee 2000, 2001
• AAAI 2000 student poster reviewer
• NASA Intelligent Systems Program reviewer 2000
• Organizer of NIPS 98 workshop on Abstraction and Hierarchy in Reinforcement Learning.
• Review papers for Adaptive Behavior, AIJ, ECAI, IEEE TRO, JAIR, JMLR, MLJ, JAAMAS, RAS

Department Service

• Faculty search committees: 2001-2003, 2005, 2006, 2008
• Graduate Admissions Committee: 2007
• Lab Committee: 2004-
• Publications Committee: 2004-
• Strategic Planning Committee: 2002
• Admissions Committee: 2001-2002

Classes Taught

• CPS 271 - Machine Learning (Fall 2003, 2005, 2007)
• CPS 1/296 - Robotics (mixed undergrade and graduate) (Spring 2007)
• CPS 270 - Artificial Intelligence (graduate) (Fall 2001, 2002, 2004, 2006)
• CPS 170 - Artificial Intelligence (undergraduate) (Spring 2002, 2003, 2004, 2006)
• CPS 271 - Numeric Artificial Inteligence (Spring 2001)
• CPS 370 - Planning Under Uncertainty (Fall 2000)

Presentations

• Presenter, 2008 ICML.
• SAMSI Workshop Tutorial on Markov Decision Processes and Reinforcement Learning, 2007.
• Duke MURI Workshop on Adaptive Multi-Sensor Sensing and Waveform Scheduling, 2006.
• NSF Approximate Dynamic Programming Workshop, technical talk and tutorial, 2006.
• Invited talk at ICML Hierarchical Reinforcement Learning Workshop 2005
• Seminars at Princeton, Univ. of Massachusetts, Middlebury, Stony Brook 2005.
• Invited panelist on AFOSR funded workshop on Decision Making in Adversarial Domains
• Seminars at Univ. of Alberta, Carnegie Mellon and NASA Ames 2004.
• Invited participant in GAFOS workshop 2004.
• Daghstuhl 2003 invited tutorial and research talk
• LICS 2003 Workshop on Probability and AI invited talk
• Daghstuhl 2001 invited tutorial and research talk
• Poster Spotlight, 2001 NIPS.
• Duke ISDS Colloquium 2001.
• Presenter, 2000 UAI.
• Presenter, 1999 IJCAI.
• Invited talks at NASA and SRI, 1999.
• Plenary Presentation, 1998 UAI conference.
• Presenter, 1998 SARA Symposium.
• Full Oral presentation, 1997 NIPS conference.
• Presenter, 1997 ICML Workshop on Reinforcement Learning.
• Presenter, 1997 Joint Brazil, US Workshop on Intelligent Robotic Agents.
• Presenter, 1996 AAAI Fall Symposium.
• Presenter, 1996 NSF Reinforcement Learning Workshop.
• Presenter, 1995 IJCAI.
• Invited participant, 1995 Joint AI/OR Workshop.
• Guest lectured for Computer Science and Cognitive Science classes at Berkeley and Stanford.

Journal Papers

• Non-Myopic Multi-Aspect Sensing with Partially Observable Markov Decision Processes, Shihao Ji, Ronald Parr, and Lawrence Carin, IEEE Transactions on Signal Processing, June 2007 Volume 55, Issue: 6, Part 1, 2007, pp. 2720-2730.
• Least-Squares Policy Iteration, Michail Lagoudakis and Ronald Parr, Journal of Machine Learning Research (JMLR), Vol. 4, 2003, pp. 1107-1149.
• Efficient Solution Algorithms for Factored MDPs, Carlos Guestrin, Daphne Koller, Ronald Parr and Shobha Venkataraman, Journal of Artificial Intelligence Research (JAIR), Vol. 19, 2003, pp. 399-468. (Recipient of IJCAI-JAIR best paper awards for 2007.)

Books or Edited Volumes

• Proceedings of the 23^rd Conference on Uncertainty in Artificial Intelligence, Ronald Parr and Lind van der Gaag, eds., 2007.

Highly Refereed Papers

• Kernelized Value Function Approximation for Reinforcement Learning, Gavin Taylor and Ronald Parr, International Conference on Machine Learning (ICML-2009) [27% acceptance rate].
• Multi-step Multi-sensor Hider-seeker Games, Erik Halvorson, Vincent Conitzer, and Ronald Parr, Proceedings of the Twenty-First International Joint Conference on Artificial Intelligence (IJCAI-2009). [26% acceptance rate]
• Planning Aims for a Network of Horizontal Overhead Sensors, Erik Halvorson and Ronald Parr, Workshop on the Algorithmic Foundations of Robotics 2008 (WAFR-2008). [65% acceptance rate (though high self selection rate due to reputation of the venue)]
• An Analysis of Linear Models, Linear Value-Function Approximation, and Feature Selection for Reinforcement Learning, Ronald Parr, Lihong Li, Gavin Taylor, Christopher Painter-Wakefield, and Michael L. Littman, International Conference on Machine Learning (ICML-2008). [27% acceptance rate]
• Point-Based Policy Iteration, Shihao Ji, Ronald Parr, Hui Li, Xuejun Liao, and Lawrence Carin, Proceedings of the Twenty-Second National Conference on Artificial Intelligence (AAAI 2007). [27% acceptance rate]
• Analyzing Feature Generation for Value-Function Approximation, Ronald Parr, Christopher Painter-Wakefield, Lihong Li, and Michael Littman, International Conference on Machine Learning (ICML-2007). [29% acceptance rate]
• Efficient Selection of Disambiguating Actions for Stereo Vision, Monika Schaeffer and Ronald Parr, Uncertainty in Artificial Intelligence (UAI-2006). [plenary presentation; 32% acceptance overall; 12% of submitted papers were given plenary presentations]
• Hierarchical Linear/Constant Time SLAM using Particle Filters for Dense Maps, Austin I. Eliazar and Ronald Parr, Advances in Neural Information Processing Systems (NIPS-19) 2005. [27% acceptance rate]
• Learning Probabilistic Motion Models for Mobile Robots, Austin I. Eliazar and Ronald Parr, Proceedings of the Twenty First International Conference on Machine Learning (ICML-2004). [32% acceptance rate]
• DP-SLAM 2.0, Austin Eliazar and Ronald Parr, IEEE 2004 International Conference on Robotics and Automation (ICRA 2004). [58% acceptance rate; nevertheless, ICRA is considered a top robotics conference.]
• Reinforcement Learning as Classification: Leveraging Modern Classifiers, Michail Lagoudakis and Ronald Parr, Proceedings of the Twentieth International Conference on Machine Learning (ICML-2003). [32% acceptance rate]
• DP-SLAM: Fast, Robust Simultaneous Localization and Mapping without Predetermined Landmarks, Austin Eliazar and Ronald Parr, Proceedings of the Eighteenth International Joint Conference on Artificial Intelligence (IJCAI 03). [21% acceptance rate]
• Learning in Zero-Sum Team Markov Games using Factored Value Functions Michail Lagoudakis and Ronald Parr, Advances in Neural Information Processing Systems (NIPS-15) 2002. [30% acceptance rate]
• Value Function Approximation in Zero-Sum Markov Games Michail Lagoudakis and Ronald Parr, Proceedings of the 18th Conference on Uncertainty in Artificial Intelligence (UAI 2002), pp. 283-292, August 2002. [34% acceptance rate]
• Coordinated Reinforcement Learning, Carlos Guestrin, Michail Lagoudakis,and Ronald Parr. Nineteenth International Conference on Machine Learning (ICML-2002), pp. 227-234, July 2002. [33% acceptance rate]
• XPathLearner: An On-Line Self-Tuning Markov Histogram for XML Path Selectivity Estimation, Lipyeow Lim, Min Wang, Sriram Padmanabhan, Jeffrey Scott Vitter, Ronald Parr. Twenty Eighth International Conference on Very Large Databases. (VLDB 2002), pp. 442-453, August 2002. [16% acceptance rate]
• Multiagent Planning with Factored MDPs, Carlos Guestrin, Daphne Koller and Ronald Parr, Advances in Neural Information Processing Systems 2001 (NIPS-14), pp. 1532-1530, December 2001. [Accepted for a full oral presentation. Less than 4% of submitted papers got oral presentations this year.]
• Model-Free Least-Squares Policy Iteration Michail Lagoudakis, Ronald Parr, Advances in Neural Information Processing Systems 2001 (NIPS-14), pp. 1547-1554, August 2001. [Accepted for a poster spotlight. Approximately 10% of submitted papers got poster spotlights this year.]
• Inference in Hybrid Networks: Theoretical Limits and Practical Algorithms, Uri Lerner, Ronald Parr, Uncertainty in Artificial Intelligence, Proceedings of the Seventeenth Conference (UAI 2001), pp. 310-318, August 2000. [Joint winner of best student paper award (student first author).]
• Max-norm Projections for Factored MDPs, Carlos Guestrin, Daphne Koller and Ronald Parr, Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence (IJCAI 2001), pp. 673 - 680, August 2001. [25% acceptance rate]
• Policy Iteration for Factored MDPs, Daphne Koller and Ronald Parr, Uncertainty in Artificial Intelligence, Proceedings of the Sixteenth Conference (UAI 2000), pp. 326-334, June 2000. [45% acceptance rate]
• Bayesian Fault Detection and Diagnosis in Dynamic Systems, Uri Lerner, Ronald Parr, Daphne Koller and Gautam Biswas, Proceedings of the Twelfth National Conference on Artificial Intelligence (AAAI 2000), pp. 531-537, July 2000. [33% acceptance rate]
• Making Rational Decisions Using Adaptive Utility Elicitation, Urszula Chajewska, Daphne Koller, Ronald Parr, Proceedings of the Twelfth National Conference on Artificial Intelligence (AAAI 2000), pp. 363-369, July 2000. [33% acceptance rate]
• Policy Search via Density Estimation, Andrew Y. Ng, Ronald Parr, Daphne Koller, Neural Information Processing Systems 1999 (NIPS 99), pp. 1022-1028, December 1999. [Accepted for a poster spotlight. 10% of submitted papers received poster spotlights.]
• Reinforcement Learning using Approximate Belief States, Andrés Rodgríguez, Ronald Parr, Daphne Koller, Neural Information Processing Systems 1999 (NIPS 99), pp. 1036-1042, December 1999. [32% acceptance rate]
• Computing Factored Value Functions for Policies in Structured MDPs, Daphne Koller, Ronald Parr, Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence (IJCAI 1999), pp. 1332-1339, July 1999. [26% acceptance rate]
• Flexible Decomposition Algorithms for Weakly Coupled Markov Decision Problems, Ronald Parr, Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence (UAI-98), pp. 422-430, July 1998. [45% acceptance rate]
• Reinforcement Learning with Hierarchies of Machines, Ronald Parr, Stuart Russell, Neural Information Processing Systems 1998 (NIPS 98), pp. 1043-1049, December 1998. [Accepted for a full oral presentation. 4.5% of submitted papers were given an oral presentation.]
• Generalized Prioritized Sweeping, David Andre, Nir Friedman, Ronald Parr, Neural Information Processing Systems 1998 (NIPS 98), pp. 1001-1007, December 1998. [31% acceptance rate]
• Approximating Optimal Policies for Partially Observable Stochastic Domains, Ronald Parr, Stuart Russell, Proceedings of the Fourteenth International Joint Conference on Artificial Intelligence (IJCAI-95), pp. 1088-1094, August 1995. [22% acceptance rate]
• Provably Bounded Optimal Agents, Stuart Russell, Devika Subramanian, Ronald Parr, in Proceedings of the Thirteenth International Joint Conference on Artificial Intelligence (IJCAI-93), pp. 338-344, August, 1993. [25% acceptance rate]

Other Papers

• Planning Aims for a Network of Horizontal and Overhead Sensors, Erik Halvorson and Ronald Parr, International Symposium on Artificial Intelligence and Mathematics 2008 (ISAIM 2008).
• Counting Objects with a Combination of Horizontal and Overhead Sensors, Erik Halvorson and Ronald Parr, Cuke CS Technical Report CS-2007-04. (Longer version of the above ISAIM 2008 symposium paper.)
• Efficient Selection of Disambiguating Actions for Stereo Vision, Monika Schaeffer and Ronald Parr, NIPS 2005 Workshop on Value of Information in Inference, Learning and Decision Making.
• Least-Squares Methods in Reinforcement Learning for Control, Michail Lagoudakis, Ronald Parr and Michael L. Littman. Second Hellenic Conference on Artificial Intelligence (SETN-02).
• Model-Free Least-Squares Policy Iteration, Michail Lagoudakis and Ronald Parr, Duke University Technical Report. (Longer version of above NIPS 2001 paper.)
• Coordinated Reinforcement Learning, Carlos Guestrin, Michail Lagoudakis,and Ronald Parr. Proceedings of the 2002 AAAI Spring Symposium Series: Collaborative Learning Agents
• Selecting the Right Algorithm Michail Lagoudakis, Michael L. Littman and Ronald Parr, Proceedings of the 2001 AAAI Fall Symposium Series: Using Uncertainty within Computation, Cape Cod, MA, November 2001.
• Solving Factored POMDPs with Linear Value Functions, Carlos Guestrin, Daphne Koller and Ronald Parr, In the IJCAI-01 workshop on Planning under Uncertainty and Incomplete Information.
• Max-norm Projections for Factored MDPs, Carlos Guestrin, Daphne Koller, and Ronald Parr, AAAI Spring Symposium, Stanford, California, March 2001.
• Adaptive Utility Elicitation using Value of Information, Chajewska, U., M. Kuppermann, R. Parr and D. Koller. (Abstract), Presented at the 22nd Annual Meeting of the Society for Medical Decision Making, 2000).
• Hierarchical Control and Learning for Markov Decision Processes, Ph.D. Dissertation, 1998, University of California, Berkeley.
• A Unifying Framework for Temporal Abstraction in Stochastic Processes, Ronald Parr, Symposium on Abstraction Reformulation and Approximation, 1998 (SARA-98).
• Feasibility Study of Fully Automated Vehicles Using Decision-theoretic Control, Jeffrey Forbes, Nikunj Oza, Ronald Parr, Stuart Russell, California PATH Research Report, UCB-ITS-PRR-97-18.
• Policy Based Clustering in Markov Decision Problems, AAAI-96 Fall Symposium on Learning Complex Behaviors in Adaptive Intelligent Systems.

Funding

• U.S. Department of Education GAANN grant, $384,000, (Joint with multiple investigators in the department.), Parr portion: $0, (The Department of Education has determined that my students are too wealthy to receive funding from this grant, so the funds have been allocated to other students in the department.) 2007.
• NSF RI: Feature Discovery and Benchmarks for Exportable Reinforcement Learning, $450,000, (Joint with Michael Littman from Rutgers University. Duke/Parr portion $225,000), 2007.
• DARPA CSSG year 2 funding, $500,000, 2007.
• NSF CAREER: Observing to Plan, Planning to Observe, $440,000, 2006.
• DARPA CSSG grant, $80,000, 2006.
• SAIC gift in support of robotics research, $80,000, 2006.
• SAIC gift of 3 research robots, approx. value $50,000, 2006.
• Computing a Semantic View of a Scene for Surveillance from Stereo and Discreet LIDAR, with Carlo Tomasi and Industrial Partner IAI, DARPA SBIR Phase I. Duke portion: $22,000, Parr portion $11,000, 2005-2006. (Duke refused to sign the contract for this grant.)
• A Core Experimental Facility for Computer Vision and Artificial Intelligence, NSF CRI grant (joint with Carlo Tomasi), $300,000, 2005.
• SAIC gift in support of robotics research, $80,000, 2005.
• SAIC gift for graduate student support, $20,000, 2004.
• Sloan Research Fellowship, Alfred P. Sloan Foundation, $40,000, 2003-2004, extended 1 year.
• Prediction and Planning: Bridging the Gap, National Science Foundation, $291,677. Official start date: 9/02, 3 year duration, extended 1 year.

Students Advised

• Mac Mason (Third year)
• Gavin Taylor (Third year)
• Christopher Painter-Wakefield (Fourth year)
• Monika Schaeffer (Fourth year)
• Erik Halvorson, Master's in Summer 2008.
• Neeti Wagle, Master's in Spring 2008.
• Austin Eliazar, Ph.D. in December 2005, now at Signal Innovations Group, Inc., Durham, NC.
• Michail Lagoudakis, Ph.D. in June 2003, now assistant professor, Technical University of Crete, Greece.

Memberships

AAAI

Citizenship

Natural born U.S. citizen.

Contact Information

Department of Computer Science
Duke University
LSRC / Box 90129
Durham, NC 27708
(919)660-6537
http://www.cs.duke.edu/~parr