Normal view MARC view

Dynamic programming and optimal control

Author: Bertsekas, Dimitri P. Publisher: Athena Scientific, 2005 ; 2007Edition: 3rd ed.Language: EnglishDescription: 24 cm.ISBN: 0886529086Type of document: BookNote: Doriot: for 2013-2014 coursesBibliography/Index: Includes bibliographical references and indexContents Note: Vol. 1, 543 p., ISBN 1886529264 ; Vol. 2, , 445 p., ISBN 1886529302
Tags: No tags from this library for this title. Log in to add tags.
Item type Current location Collection Call number Status Date due Barcode Item holds
Book Asia Campus
Textbook Collection (PhD)
Print Ordered
Book Asia Campus
Textbook Collection (PhD)
Print QA402.5 .B47 2005 Vol.1
(Browse shelf)
900222732
Available 900222732
Book Asia Campus
Textbook Collection (PhD)
Print QA402.5 .B47 2012 Vol.2
(Browse shelf)
900222754
Available 900222754
Book Asia Campus
Textbook Collection (PhD)
Print QA402.5 .B47 2005 Vol.1
(Browse shelf)
900199100
Consultation only 900199100
Book Europe Campus
Main Collection
Print QA402.5 .B47 2005 Vol.1
(Browse shelf)
001333373
Available 001333373
Book Digital Library
Main Collection
Print QA402.5 .B47 2005 Vol.1
(Browse shelf)
001211345
Available 001211345
Book Europe Campus
Main Collection
Print QA402.5 .B47 2007 Vol.2
(Browse shelf)
001211352
Available 001211352
Book Europe Campus
Main Collection
Print QA402.5 .B47 2007 Vol.2
(Browse shelf)
001236292
Checked out 03/06/2022 001236292
Total holds: 0

Doriot: for 2013-2014 courses

Includes bibliographical references and index

Vol. 1, 543 p., ISBN 1886529264 ; Vol. 2, , 445 p., ISBN 1886529302

Digitized

Dynamic Programming and Optimal Control Volume I Contents 1. The Dynamic Programming Algorithm 1.1. Introduction 1.2. The Basic Problem .................................................................... 1.3. The Dynamic Programming Algorithm ..................................... 1.4. State Augmentation and Other Reformulations ...................... 1.5. Some Mathematical Issues ...................................................... 1.6. Dynamic Programming and Minimax Control ......................... 1.7. Notes, Sources, and Exercises ................................................ 2. Deterministic Systems and the Shortest Path Problem 2.1. Finite-State Systems and Shortest Paths ............................... p. 64 2.2. Some Shortest Path Applications ............................................ 2.2.1. Critical Path Analysis ................................................... 2.2.2. Hidden Markov Models and the Viterbi Algorithm ....... 2.3. Shortest Path Algorithms ........................................................ 2.3.1. Label Correcting Methods ............................................. 2.3.2. Label Correcting Variations - A* Algorithm . . . ............ 2.3.3. Branch-and-Bound ..................................................... 2.3.4. Constrained and Multiobjective Problems .................... 2.4. Notes, Sources, and Exercises ................................................ 3. Deterministic Continuous-Time Optimal Control 3.1. Continuous-Time Optimal Control ........................................ p. 106 3.2. The Hamilton-Jacobi-Bellman Equation .............................. p. 109 3.3. The Pontryagin Minimum Principle ....................................... p. 115 3.3.1. An Informal Derivation Using the HJB Equation p. 115 3.3.2. A Derivation Based on Variational Ideas . . . . p. 125 3.3.3. Minimum Principle for Discrete-Time Problems . p. 129 3.4. Extensions of the Minimum Principle ................................... p. 131 3.4.1. Fixed Terminal State p. 131 3.4.2. Free Initial State p. 135 v p. p. p. p. p. p. p. p. p. 68 68 70 77 78 87 88 91 97 p. 2 p. p. p. p. p. p. 12 18 35 42 46 51 vi Contents 3.4.3. Free Terminal Time ...................................................... 3.4.4. Time-Varying System and Cost ................................... 3.4.5. Singular Problems ....................................................... 3.5. Notes, Sources, and Exercises ............................................. 4. Problems with Perfect State Information p. p. p. p. 135 138 139 142 4.1. Linear Systems and Quadratic Cost .................................... p. 148 4.2. Inventory Control ................................................................... 4.3. Dynamic Portfolio Analysis .................................................... 4.4. Optimal Stopping Problems .................................................. 4.5. Scheduling and the Interchange Argument ......................... p. p. p. p. 162 170 176 186 4.6. Set-Membership Description of Uncertainty ........................ p. 190 4.6.1. Set-Membership Estimation ....................................... p. 191 4.6.2. Control with Unknown-but-Bounded Disturbances p. 197 4.7. Notes, Sources, and Exercises ............................................. p. 201 5. Problems with Imperfect State Information 5.1. Reduction to the Perfect Information Case ........................... p. 218 5.2. Linear Systems and Quadratic Cost .................................... 5.3. Minimum Variance Control of Linear Systems ..................... 5.4. Sufficient Statistics and Finite-State Markov Chains . ........ 5.4.1. The Conditional State Distribution ............................. 5.4.2. Finite-State Systems .................................................. 5.5. Notes, Sources, and Exercises ............................................. 6. Suboptimal Control 6.1. Certainty Equivalent and Adaptive Control ......................... p. 283 6.1.1. Caution, Probing, and Dual Control ........................... p. 289 6.1.2. Two-Phase Control and Identifiability ......................... p. 291 6.1.3. Certainty Equivalent Control and Identifiability ......... p. 293 6.1.4. Self-Tuning Regulators ................................................ p. 298 6.2. Open-Loop Feedback Control ............................................... p. 300 6.3. Limited Lookahead Policies .................................................... p. 304 6.3.1. Performance Bounds for Limited Lookahead Policies p. 305 6.3.2. Computational Issues in Limited Lookahead ............. p. 310 ... 6.3.3. Problem Approximation - Enforced Decomposition ..... p. 312 6.3.4. Aggregation ................................................................... p. 319 6.3.5. Parametric Cost-to-Go Approximation ........................ p. 325 6.4. Rollout Algorithms ................................................................. p. 335 6.4.1. Discrete Deterministic Problems ................................. p. 342 6.4.2. Q-Factors Evaluated by Simulation ............................ p. 361 6.4.3. Q-Factor Approximation .............................................. p. 363 p. p. p. p. p. p. 229 236 251 252 258 270 Contents 6.5. Model Predictive Control and Related Methods . . . . 6.5.1. Rolling Horizon Approximations ...................................... 6.5.2. Stability Issues in Model Predictive Control . . .......... 6.5.3. Restricted Structure Policies .............................................. 6.6. Additional Topics in Approximate DP .......................................... 6.6.1. Discretization ......................................................................... 6.6.2. Other Approximation Approaches .................................. 6.7. Notes, Sources, and Exercises ....................................................... 7. Introduction to Infinite Horizon Problems 7.1. An Overview ......................................................................................... 7.2. Stochastic Shortest Path Problems .............................................. 7.3. Discounted Problems ........................................................................ 7.4. Average Cost per Stage Problems ................................................. 7.5. Semi-Markov Problems .................................................................... 7.6. Notes, Sources, and Exercises ....................................................... p. p. p. p. p. p. vii p. 366 p. 367 p. 369 p. 376 p. 382 p. 382 p. 384 p. 386 402 405 417 421 435 445 Appendix A: Mathematical Review A.1. Sets ......................................................................................................... A.2. Euclidean Space ................................................................................. A.3. Matrices ................................................................................................. A.4. Analysis ................................................................................................. A.5. Convex Sets and Functions ............................................................ Appendix B: On Optimization Theory B.1. Optimal Solutions .............................................................................. p. 468 B.2. Optimality Conditions ...................................................................... p. 470 B.3. Minimization of Quadratic Forms ................................................ p. 471 Appendix C: On Probability Theory C.1. Probability Spaces ............................................................................. p. 472 C.2. Random Variables ............................................................................. p. 473 C.3. Conditional Probability .................................................................... p. 475 Appendix D: On Finite-State Markov Chains D.1. Stationary Markov Chains .............................................................. D.2. Classification of States .................................................................... D.3. Limiting Probabilities ....................................................................... D.4. First Passage Times .......................................................................... p. p. p. p. 477 478 479 480 p. p. p. p. p. 459 460 461 465 467 viii Appendix E: Kalman Filtering Contents E.1. Least-Squares Estimation ................................................... p. 481 E.2. Linear Least-Squares Estimation ........................................ E.3. State Estimation -- Kalman Filter ........................................ E.4. Stability Aspects .................................................................... E.5. Gauss-Markov Estimators ................................................... E.6. Deterministic Least-Squares Estimation ............................. Appendix F: Modeling of Stochastic Linear Systems F.1. Linear Systems with Stochastic Inputs ............................... p. 503 F.2. Processes with Rational Spectrum ....................................... p. 504 F.3. The ARMAX Model ....................................................................... p. 506 Appendix G: Formulating Problems of Decision Under Uncertainty G.1. The Problem of Decision Under Uncertainty ........................ p. 507 G.2. Expected Utility Theory and Risk ......................................... p. 511 G.3. Stochastic Optimal Control Problems ................................. p. 524 References ............................................................................................ p. 529 Index ........................................................................................................ p. 541 p. p. p. p. p. 483 491 496 499 501 Dynamic Programming and Optimal Control Volume II Contents 1. Infinite Horizon - Discounted Problems 1.1. Minimization of Total Cost - Introduction ....................................... p. 3 1.1.1. The Finite-Horizon DP Algorithm .......................................... p. 4 1.1.2. Shorthand Notation and Monotonicity ................................ p. 6 1.1.3. A Preview of Infinite Horizon Results ................................... p. 8 1.1.4. Randomized and History-Dependent Policies . . . ......... p. 10 1.2. Discounted Problems with Bounded Cost per Stage . . ............ p. 12 1.3. Finite-State Systems - Computational Methods ......................... p. 19 1.3.1. Value Iteration and Error Bounds ...................................... p. 22 1.3.2. Variants of Value Iteration .................................................... p. 30 1.3.3. Policy Iteration .......................................................................... p. 38 1.3.4. Linear Programming ............................................................... p. 51 1.3.5. Limited Lookahead and Rollout Policies .......................... p. 53 1.4. The Role of Contraction Mappings ................................................... p. 56 1.4.1. Sup-Norm Contractions ......................................................... p. 57 1.4.2. m-Stage Sup-Norm Contractions ....................................... p. 62 1.4.3. Discounted Problems - Unbounded Cost per Stage ..... p. 63 1.5. Scheduling and Multiarmed Bandit Problems ............................. p. 66 1.6. Notes, Sources, and Exercises .......................................................... p. 79 2. Stochastic Shortest Path Problems 2.1. Problem Formulation ........................................................................... p. 94 2.2. Bellman's Equation ............................................................................... p. 97 2.3. Value Iteration ...................................................................................... p. 105 2.4. Policy Iteration ..................................................................................... p. 108 2.5. Countable State Problems ............................................................... p. 112 2.6. Notes, Sources, and Exercises ........................................................ p. 114 3. Undiscounted Problems 3.1. Unbounded Costs per Stage ............................................................ p. 124 3.2. Linear Systems and Quadratic Cost ............................................. p. 140 iii iv Contents 3.3. Inventory Control .................................................................. 3.4. Optimal Stopping .................................................................. 3.5. Optimal Gambling Strategies ............................................... 3.6. Nonstationary and Periodic Problems .................................. 3.7. Notes, Sources, and Exercises ............................................. 4. Average Cost per Stage Problems 4.1. Finite-Spaces Average Cost Models ...................................... 4.1.1. Relation with the Discounted Cost Problem . . ......... 4.1.2. Blackwell Optimal Policies ......................................... 4.1.3. Optimality Equations 4.2. Conditions for Equal Average Cost for all Initial States . 4.3. Value Iteration ...................................................................... 4.3.1. Single-Chain Value Iteration ..................................... 4.3.2. Multi-Chain Value Iteration ...................................... 4.4. Policy Iteration ...................................................................... 4.4.1. Single-Chain Policy Iteration ..................................... 4.4.2. Multi-Chain Policy Iteration 4.5 Linear Programming 4.6. Infinite-Spaces Problems 4.6.1. A Sufficient Condition for Optimality 4.6.2. Finite State Space and Infinite Control Space . . 4.6.3. Countable States -- Vanishing Discount Approach 4.6.4. Countable States -- Contraction Approach . . . . 4.6.5. Linear Systems with Quadratic Cost ........................ 4.7. Notes, Sources, and Exercises ............................................. 5. Continuous-Time Problems 5.1. Uniformization ...................................................................... 5.2. Queueing Applications ......................................................... 5.3. Semi-Markov Problems .......................................................... 5.4. Notes, Sources, and Exercises ............................................. 6. Approximate Dynamic Programming p. p. p. p. p. 142 145 150 157 162 p. p. p. p p. p. p. p. p. p. p p p p p. p. p. p. p. 174 178 184 194 198 204 207 222 229 229 235 239 245 253 255 264 267 272 274 p. p. p. p. 288 295 306 317 6.1. Cost Approximation .............................................................. p. 325 6.2. Approximate Policy Iteration -- Direct Policy Evaluation .... p. 329 6.2.1. Gradient Methods for Direct Policy Evaluation . . .............. p. 332 6.2.2. TD(A) ................................................................................ p. 336 6.2.3. Optimistic Policy Iteration ......................................... p. 337 6.2.4. Approximate Policy Iteration Based on Q-Factors ..... p. 338 6.3. Indirect Methods for Policy Evaluation ................................ p. 340 6.3.1. Policy Evaluation by Projected Value Iteration . ....... p. 341 6.3.2. Least Squares Policy Evaluation (LSPE) . . . ............. p. 346 Contents 6.3.3. PVI(A) and LSPE(A) .......................................................... 6.3.4. The LSTD(A) Algorithm ................................................. 6.3.5. The TD(A) Algorithm ...................................................... 6.3.6. Summary and Examples ........................................... 6.4. Q-Learning ............................................................................. 6.4.1. Q-Factor Approximations ........................................... 6.4.2. Q-Learning for Optimal Stopping Problems . . ........... 6.5. Stochastic Shortest Path Problems ...................................... 6.6. Average Cost Problems ........................................................... 6.6.1. Approximate Policy Evaluation ................................... 6.6.2. Approximate Policy Iteration ....................................... 6.6.3. Q-Learning for Average Cost Problems ....................... 6.7. Approximation in Policy Space .............................................. 6.8. Notes, Sources, and Exercises .............................................. Appendix A: Measure-Theoretic Issues in Dynamic Programming p. p. p. p. p. p. p. p. p. p. p. p. p. p. 348 355 357 359 363 364 366 369 375 375 386 389 392 399 A.1. A Two-Stage Example .............................................................. p. 407 A.2. Resolution of the Measurability Issues ................................. p. 412 References ............................................................................................ p. 423 Index ........................................................................................................ p. 443

There are no comments for this item.

Log in to your account to post a comment.
Koha 18.11 - INSEAD Catalogue
Home | Contact Us | What's Koha?