This content is not included in
your SAE MOBILUS subscription, or you are not logged in.
Using Reinforcement Learning to Control Life Support Systems
Technical Paper
2004-01-2439
ISSN: 0148-7191, e-ISSN: 2688-3627
Annotation ability available
Sector:
Language:
English
Abstract
Advanced life support systems have many interacting processes and limited resources. Controlling and optimizing advanced life support systems presents unique challenges that are addressed in this paper. We have developed a controller using reinforcement learning [Barto&Sutton], which actively explores the space of possible control strategies, guided by rewards from a user specified long term objective function. We evaluated this controller using Biosim, our discrete event simulation of an advanced life support system. This simulation supports all life support modules including crew, food production, air revitalization, water recovery, solid waste incineration and power. Our algorithm for reinforcement learning discovered unobvious strategies for maximizing mission length. By exploiting nonlinearities in the simulation dynamics, the learned controller outperforms a controller designed by an expert.
Authors
Topic
Citation
Klein, T., Subramanian, D., Kortenkamp, D., and Bell, S., "Using Reinforcement Learning to Control Life Support Systems," SAE Technical Paper 2004-01-2439, 2004, https://doi.org/10.4271/2004-01-2439.Also In
References
- Barto A. Sutton R. Reinforcement learning: an introduction MIT Press 1998
- Davies S. Multi-dimensional triangulation and interpolation in reinforcement learning Proc. AAAI 1998 200 205
- Kortenkamp D. Bell S. Simulating Advanced Life Support Systems for Integrated Controls Research Proc. Third International Conference on Environmental Systems 2003
- Watkins C.J.C.H. Learning from delayed rewards Cambridge University 1989
- Kaelbling L. P. Littman M. L. Moore A. W. Reinforcement learning: A survey Journal of Artificial Intelligence Research 4 23285 May 1996
- Crites R. H. Barto A. G. Improving elevator performance using reinforcement learning Touretzky D. S. Mozer M. C. Hasselmo M. E. Advances in Neural Information Processing Systems 1996
- Sutton Richard S. Generalization in reinforcement learning: successful examples using sparse coarse coding Neural Information processing Systems 1996
- http://advlifesupport.jsc.nasa.gov
- http://advlifesupport.jsc.nasa.gov
- Thrun S. Efficient Exploration In Reinforcement Learning NIPS 1992
- Boyan J. Moore A.W. Generalization in Reinforcement Learning: Safely Approximating the Value Function Neural Information Processing Systems 1995
- Tesauro, G. Practical issues in temporal difference learning Mach. Learning 8 1992 257 277