Ronald Parr
Research Interests
Methods for solving large stochastic planning
problems (MDPs): dynamic programming, reinforcement learning, and policy
search. I am interested in techniques for decomposing large problems using
hierarchy, problem structure and prior knowledge. I'm also interested in
Bayesian networks, tracking, learning probabilistic robotics and,
generally, most forms of reasoning under uncertainty.
Education
- Ph.D. in Computer Science, University of California, Berkeley,
1998. Thesis title: Hierarchical Control and Learning in
Markov Decision Processes (Advisor: Stuart Russell).
- A.B. in Philosophy cum laude, Princeton University, 1990.
Awards and Honors
- IJCAI-JAIR Best Paper Award, 2007
- DARPA CSSG participant, 2006
- NSF CAREER award, 2006
- Alfred P. Sloan Fellow, 2003
Employment History
- Associate Professor, Duke University, 2007-present.
- Assistant Professor, Duke University, 2000-2007.
-
Postdoctoral Research Associate, Stanford University, 1998-2000.
Worked with Daphne Koller on research, student advising and research
grant-related responsibilities.
- Research Assistant/Independent Research, U.C. Berkeley, 1991-1998.
Advisor: Stuart Russell.
- Graduate Student Instructor: Formal Languages and Automata Theory,
1990-1991.
Professional Service
- NIPS reviewer 1999, 2001, 2002, 2003, 2004, 2005, 2008
- UAI General Chair 2008
- AAAI 2008 Senior Program Committee
- ISAIM 2008 Program Committee
- ICML Program Committee 2000, 2001, 2003-2006, 2008
- UAI Program Co-chair 2007
- IJCAI 2007 Program Committee
- Machine Learning (MLJ) Editorial Board 2006-
- RSS Area Chair 2006
- UAI Area Chair 2002-2006
- IJCAI Reviewer 1999, 2001, 2003
- AI & Math Program Committee 2004, 2006
- AAAI Program Committee 1998, 2000, 2002, 2004, 2005
- IJCAI Program Committee (senior level) 2005
- IJCAI 2005 RUR (Reasonsing with Uncertainty in Robotics) workshop reviewer
- Symposium on Artificial Intelligence and Mathematics Program Committee
2003, 2005
- GAFOS program committee 2004
- ICAPS Program Committee 2003
- Journal of AI Research (JAIR) editorial board member 2000-2002
- UAI program committee 2000, 2001
- AAAI 2000 student poster reviewer
- NASA Intelligent Systems Program reviewer 2000
- Organizer of NIPS 98 workshop on Abstraction and Hierarchy in
Reinforcement Learning.
- Review papers for Adaptive Behavior, AIJ, ECAI, IEEE TRO, JAIR, JMLR, MLJ,
JAAMAS, RAS
Department Service
- Faculty search committees: 2001-2003, 2005, 2006, 2008
- Graduate Admissions Committee: 2007
- Lab Committee: 2004-
- Publications Committee: 2004-
- Strategic Planning Committee: 2002
- Admissions Committee: 2001-2002
Classes Taught
- CPS 271 - Machine Learning (Fall 2003, 2005, 2007)
- CPS 1/296 - Robotics (mixed undergrade and graduate) (Spring 2007)
- CPS 270 - Artificial Intelligence (graduate) (Fall 2001, 2002, 2004, 2006)
- CPS 170 - Artificial Intelligence (undergraduate) (Spring 2002, 2003,
2004, 2006)
- CPS 271 - Numeric Artificial Inteligence (Spring 2001)
- CPS 370 - Planning Under Uncertainty (Fall 2000)
Presentations
- Presenter, 2008 ICML.
- SAMSI Workshop Tutorial on Markov Decision Processes and Reinforcement
Learning, 2007.
- Duke MURI Workshop on Adaptive Multi-Sensor Sensing and Waveform
Scheduling, 2006.
- NSF Approximate Dynamic Programming Workshop, technical talk and
tutorial, 2006.
- Invited talk at ICML Hierarchical Reinforcement Learning Workshop 2005
- Seminars at Princeton, Univ. of Massachusetts, Middlebury, Stony Brook
2005.
- Invited panelist on AFOSR funded workshop on Decision Making in
Adversarial Domains
- Seminars at Univ. of Alberta, Carnegie Mellon and NASA Ames 2004.
- Invited participant in GAFOS workshop 2004.
- Daghstuhl 2003 invited tutorial and research talk
- LICS 2003 Workshop on Probability and AI invited talk
- Daghstuhl 2001 invited tutorial and research talk
- Poster Spotlight, 2001 NIPS.
- Duke ISDS Colloquium 2001.
- Presenter, 2000 UAI.
- Presenter, 1999 IJCAI.
- Invited talks at NASA and SRI, 1999.
- Plenary Presentation, 1998 UAI conference.
- Presenter, 1998 SARA Symposium.
- Full Oral presentation, 1997 NIPS conference.
- Presenter, 1997 ICML Workshop on Reinforcement Learning.
- Presenter, 1997 Joint Brazil, US Workshop on Intelligent Robotic
Agents.
- Presenter, 1996 AAAI Fall Symposium.
- Presenter, 1996 NSF Reinforcement Learning Workshop.
- Presenter, 1995 IJCAI.
- Invited participant, 1995 Joint AI/OR Workshop.
- Guest lectured for Computer Science and Cognitive Science classes
at Berkeley and Stanford.
Journal Papers
-
Non-Myopic Multi-Aspect Sensing with
Partially Observable Markov Decision Processes, Shihao Ji, Ronald Parr,
and Lawrence Carin, IEEE
Transactions on Signal Processing, June 2007
Volume 55, Issue: 6, Part 1, 2007, pp. 2720-2730.
-
Least-Squares Policy Iteration, Michail Lagoudakis and Ronald
Parr, Journal of Machine Learning Research (JMLR), Vol. 4,
2003, pp. 1107-1149.
-
Efficient Solution Algorithms for Factored MDPs,
Carlos Guestrin, Daphne Koller,
Ronald Parr and Shobha Venkataraman, Journal of Artificial Intelligence
Research (JAIR), Vol. 19, 2003, pp. 399-468. (Recipient of IJCAI-JAIR
best paper awards for 2007.)
Books or Edited Volumes
-
Proceedings of the 23rd Conference on Uncertainty in Artificial
Intelligence, Ronald Parr and Lind van der Gaag, eds., 2007.
Highly Refereed Papers
- Kernelized Value Function Approximation for
Reinforcement Learning, Gavin Taylor and Ronald Parr, International
Conference on Machine Learning (ICML-2009) [27% acceptance rate].
-
Multi-step Multi-sensor Hider-seeker Games, Erik Halvorson, Vincent
Conitzer, and Ronald Parr, Proceedings of the Twenty-First International
Joint Conference on Artificial Intelligence (IJCAI-2009). [26% acceptance
rate]
-
Planning Aims for a Network of Horizontal Overhead
Sensors, Erik Halvorson and Ronald Parr, Workshop on the Algorithmic
Foundations of Robotics 2008 (WAFR-2008). [65% acceptance rate
(though high self selection rate due to reputation of the venue)]
-
An Analysis of Linear Models, Linear Value-Function
Approximation, and Feature Selection for Reinforcement Learning, Ronald
Parr, Lihong Li, Gavin Taylor, Christopher Painter-Wakefield, and Michael
L. Littman, International Conference on Machine Learning (ICML-2008).
[27% acceptance rate]
-
Point-Based Policy Iteration, Shihao Ji,
Ronald Parr, Hui Li, Xuejun Liao, and Lawrence Carin, Proceedings of the
Twenty-Second National Conference on Artificial Intelligence (AAAI 2007).
[27% acceptance rate]
-
Analyzing Feature Generation for Value-Function
Approximation, Ronald Parr, Christopher Painter-Wakefield, Lihong Li,
and Michael Littman, International Conference on Machine Learning
(ICML-2007). [29% acceptance rate]
-
Efficient Selection of Disambiguating Actions for Stereo Vision,
Monika Schaeffer and Ronald Parr, Uncertainty in Artificial Intelligence
(UAI-2006). [plenary presentation; 32% acceptance overall; 12% of
submitted papers were given plenary presentations]
-
Hierarchical Linear/Constant Time SLAM using Particle Filters for Dense
Maps, Austin I. Eliazar and Ronald Parr, Advances in
Neural Information Processing Systems
(NIPS-19) 2005. [27% acceptance rate]
-
Learning Probabilistic Motion Models for Mobile
Robots, Austin I. Eliazar and Ronald Parr, Proceedings of the Twenty First
International Conference on Machine Learning
(ICML-2004). [32% acceptance rate]
-
DP-SLAM 2.0, Austin Eliazar and Ronald Parr,
IEEE 2004 International Conference on Robotics and Automation (ICRA 2004).
[58% acceptance rate; nevertheless, ICRA is considered a top robotics
conference.]
-
Reinforcement Learning as Classification: Leveraging
Modern Classifiers, Michail Lagoudakis and Ronald Parr,
Proceedings of the Twentieth International Conference on Machine Learning
(ICML-2003). [32% acceptance rate]
-
DP-SLAM: Fast, Robust Simultaneous Localization
and Mapping without Predetermined Landmarks, Austin Eliazar and Ronald
Parr, Proceedings of the Eighteenth International Joint Conference on
Artificial Intelligence (IJCAI 03). [21% acceptance rate]
-
Learning in Zero-Sum Team Markov Games using Factored Value
Functions Michail Lagoudakis and Ronald Parr, Advances in
Neural Information Processing Systems
(NIPS-15) 2002. [30% acceptance rate]
-
Value Function Approximation in Zero-Sum Markov
Games Michail Lagoudakis and Ronald Parr, Proceedings of the 18th
Conference on Uncertainty in Artificial Intelligence (UAI 2002),
pp. 283-292, August 2002. [34% acceptance rate]
-
Coordinated Reinforcement Learning, Carlos
Guestrin, Michail Lagoudakis,and Ronald Parr. Nineteenth International
Conference on Machine Learning (ICML-2002), pp. 227-234, July
2002. [33% acceptance rate]
-
XPathLearner: An On-Line Self-Tuning Markov Histogram for XML Path
Selectivity Estimation, Lipyeow Lim, Min Wang, Sriram Padmanabhan,
Jeffrey Scott Vitter, Ronald Parr. Twenty Eighth International Conference
on Very Large Databases. (VLDB 2002), pp. 442-453, August 2002. [16%
acceptance rate]
-
Multiagent Planning with Factored MDPs, Carlos Guestrin, Daphne Koller
and Ronald Parr, Advances in Neural Information Processing
Systems 2001 (NIPS-14), pp. 1532-1530, December 2001. [Accepted for a full oral
presentation. Less than 4% of submitted papers got oral presentations this
year.]
-
Model-Free Least-Squares Policy Iteration Michail Lagoudakis, Ronald
Parr, Advances in Neural Information Processing Systems
2001 (NIPS-14), pp. 1547-1554, August 2001. [Accepted for a poster spotlight.
Approximately 10% of submitted papers got poster spotlights this year.]
-
Inference in Hybrid Networks: Theoretical Limits and Practical
Algorithms, Uri Lerner, Ronald Parr, Uncertainty in Artificial
Intelligence, Proceedings of the Seventeenth Conference (UAI 2001),
pp. 310-318, August 2000. [Joint winner of best student paper award
(student first author).]
-
Max-norm Projections for Factored MDPs, Carlos
Guestrin, Daphne Koller and Ronald Parr, Proceedings of the Seventeenth
International Joint Conference on Artificial Intelligence (IJCAI 2001),
pp. 673 - 680, August 2001. [25% acceptance rate]
-
Policy Iteration for Factored MDPs, Daphne Koller and Ronald Parr,
Uncertainty in Artificial Intelligence, Proceedings of the Sixteenth
Conference (UAI 2000), pp. 326-334, June 2000. [45% acceptance rate]
- Bayesian Fault Detection and Diagnosis in Dynamic Systems,
Uri Lerner, Ronald Parr, Daphne Koller and Gautam Biswas, Proceedings of
the Twelfth National Conference on Artificial Intelligence (AAAI
2000), pp. 531-537, July 2000. [33% acceptance rate]
- Making Rational Decisions Using Adaptive Utility Elicitation,
Urszula Chajewska, Daphne Koller, Ronald Parr, Proceedings of
the Twelfth National Conference on Artificial Intelligence (AAAI
2000), pp. 363-369, July 2000.
[33% acceptance rate]
- Policy Search via Density Estimation, Andrew Y. Ng, Ronald Parr,
Daphne Koller, Neural Information Processing Systems 1999 (NIPS 99),
pp. 1022-1028, December 1999.
[Accepted for a poster spotlight. 10% of submitted papers received
poster spotlights.]
- Reinforcement Learning using Approximate Belief States,
Andrés Rodgríguez, Ronald Parr, Daphne Koller, Neural
Information Processing Systems 1999 (NIPS 99), pp. 1036-1042, December
1999. [32% acceptance rate]
-
Computing Factored Value Functions for Policies in Structured MDPs,
Daphne Koller, Ronald Parr, Proceedings of the Sixteenth International
Joint Conference on Artificial Intelligence (IJCAI 1999), pp. 1332-1339,
July 1999. [26% acceptance rate]
-
Flexible Decomposition Algorithms for Weakly Coupled Markov Decision
Problems, Ronald Parr, Proceedings of the Fourteenth Conference on
Uncertainty in Artificial Intelligence (UAI-98), pp. 422-430, July 1998. [45%
acceptance rate]
-
Reinforcement Learning with Hierarchies of Machines, Ronald Parr,
Stuart Russell, Neural Information Processing Systems 1998 (NIPS 98),
pp. 1043-1049, December 1998.
[Accepted for a full oral presentation. 4.5% of submitted papers
were given an oral presentation.]
-
Generalized Prioritized Sweeping, David Andre, Nir Friedman, Ronald
Parr, Neural Information Processing Systems 1998 (NIPS 98), pp. 1001-1007,
December 1998. [31% acceptance rate]
-
Approximating Optimal Policies for Partially Observable Stochastic
Domains, Ronald Parr, Stuart Russell,
Proceedings of the Fourteenth
International Joint
Conference on Artificial Intelligence (IJCAI-95), pp. 1088-1094, August
1995. [22% acceptance rate]
- Provably Bounded Optimal Agents,
Stuart Russell, Devika Subramanian, Ronald Parr, in Proceedings
of the Thirteenth International Joint Conference on Artificial
Intelligence (IJCAI-93), pp. 338-344, August, 1993. [25% acceptance rate]
Other Papers
-
Planning Aims for a Network of Horizontal and Overhead Sensors, Erik
Halvorson and Ronald Parr, International Symposium on Artificial
Intelligence and Mathematics 2008 (ISAIM 2008).
-
Counting Objects with a Combination of Horizontal and Overhead
Sensors, Erik Halvorson and Ronald Parr, Cuke CS Technical Report
CS-2007-04. (Longer version of the above ISAIM 2008 symposium paper.)
-
Efficient Selection of Disambiguating Actions for Stereo Vision,
Monika Schaeffer and Ronald Parr, NIPS 2005 Workshop on Value of
Information in Inference, Learning and Decision Making.
-
Least-Squares Methods in Reinforcement Learning for Control, Michail
Lagoudakis, Ronald Parr and Michael L. Littman. Second
Hellenic Conference on Artificial Intelligence (SETN-02).
-
Model-Free Least-Squares Policy Iteration, Michail Lagoudakis and
Ronald Parr, Duke University Technical Report. (Longer version of above
NIPS 2001 paper.)
-
Coordinated Reinforcement Learning, Carlos Guestrin, Michail
Lagoudakis,and Ronald Parr. Proceedings of the 2002
AAAI Spring Symposium Series: Collaborative Learning Agents
-
Selecting the Right Algorithm Michail Lagoudakis, Michael L. Littman and
Ronald Parr, Proceedings of the 2001 AAAI Fall Symposium Series: Using
Uncertainty within Computation, Cape Cod, MA, November 2001.
-
Solving Factored POMDPs with Linear Value Functions, Carlos Guestrin,
Daphne Koller and Ronald Parr, In the IJCAI-01 workshop on Planning under
Uncertainty and Incomplete Information.
-
Max-norm Projections for Factored
MDPs, Carlos Guestrin, Daphne Koller, and Ronald Parr, AAAI Spring
Symposium, Stanford, California, March 2001.
- Adaptive Utility Elicitation using Value of Information,
Chajewska, U., M. Kuppermann, R. Parr and D. Koller. (Abstract),
Presented at the 22nd Annual Meeting of the Society for Medical Decision
Making, 2000).
-
Hierarchical Control and Learning for Markov Decision Processes,
Ph.D. Dissertation, 1998, University of California, Berkeley.
-
A Unifying Framework for Temporal Abstraction in Stochastic Processes,
Ronald Parr, Symposium on Abstraction Reformulation and Approximation,
1998 (SARA-98).
-
Feasibility Study of Fully Automated Vehicles Using Decision-theoretic
Control, Jeffrey Forbes, Nikunj Oza, Ronald Parr, Stuart Russell,
California PATH Research Report, UCB-ITS-PRR-97-18.
-
Policy Based Clustering in Markov Decision Problems, AAAI-96 Fall
Symposium on Learning Complex Behaviors in Adaptive Intelligent Systems.
Funding
-
U.S. Department of Education GAANN grant, $384,000, (Joint with
multiple investigators in the department.), Parr portion: $0, (The
Department of Education has determined that my students are too wealthy to
receive funding from this grant, so the funds have been allocated to
other students in the department.) 2007.
-
NSF RI: Feature Discovery and Benchmarks for Exportable Reinforcement
Learning, $450,000, (Joint with Michael Littman from Rutgers
University. Duke/Parr portion $225,000), 2007.
-
DARPA CSSG year 2 funding, $500,000, 2007.
-
NSF CAREER: Observing to Plan, Planning to Observe, $440,000, 2006.
-
DARPA CSSG grant, $80,000, 2006.
-
SAIC gift in support of robotics research, $80,000, 2006.
- SAIC gift of 3 research robots, approx. value $50,000, 2006.
-
Computing a Semantic View of a Scene for Surveillance from Stereo and
Discreet LIDAR, with Carlo Tomasi and Industrial Partner IAI, DARPA
SBIR Phase I. Duke portion: $22,000, Parr portion $11,000, 2005-2006.
(Duke refused to sign the contract for this grant.)
-
A Core Experimental Facility for Computer Vision and Artificial
Intelligence, NSF CRI grant (joint with Carlo Tomasi), $300,000, 2005.
-
SAIC gift in support of robotics research, $80,000, 2005.
-
SAIC gift for graduate student support, $20,000, 2004.
-
Sloan Research Fellowship, Alfred P. Sloan Foundation, $40,000,
2003-2004, extended 1 year.
-
Prediction and Planning: Bridging the Gap, National Science
Foundation, $291,677. Official start date: 9/02, 3 year duration,
extended 1 year.
Students Advised
- Mac Mason (Third year)
- Gavin Taylor (Third year)
- Christopher Painter-Wakefield (Fourth year)
- Monika Schaeffer (Fourth year)
- Erik Halvorson, Master's in Summer 2008.
- Neeti Wagle, Master's in Spring 2008.
- Austin Eliazar, Ph.D. in December 2005, now at Signal Innovations
Group, Inc., Durham, NC.
- Michail Lagoudakis, Ph.D. in June 2003, now assistant professor, Technical
University of Crete, Greece.
Memberships
AAAI
Citizenship
Natural born U.S. citizen.
Contact Information
Department of Computer Science
Duke University
LSRC / Box 90129
Durham, NC 27708
(919)660-6537
http://www.cs.duke.edu/~parr