Michael C. Fu

Hyeong Soo Chang

Jiaqiao Hu

Steven I. Marcus

Markov decision process (MDP) models are widely used for modeling sequential decision-making problems that arise in engineering, economics, computer science, and the social sciences. This book brings the state-of-the-art research together for the first time. It provides practical modeling methods for many real-world problems with high dimensionality or complexity which have not hitherto been treatable with Markov decision processes.

Simulation-based Algorithms for Markov Decision Processes

Markov decision process (MDP) models are widely used for modeling sequential decision-making problems that arise in engineering, economics, computer science, and the social sciences. Many real-world problems modeled by MDPs have huge state and/or action spaces, giving an opening to the curse of dimensionality and so making practical solution of the resulting models intractable. In other cases, the system of interest is too complex to allow explicit specification of some of the MDP model parameters, but simulation samples are readily available (e.g., for random transitions and costs). For these settings, various sampling and population-based algorithms have been developed to overcome the difficulties of computing an optimal solution in terms of a policy and/or value function. Specific approaches include adaptive sampling, evolutionary policy iteration, evolutionary random policy search, and model reference adaptive search. This substantially enlarged new edition reflects the latest developments in novel algorithms and their underpinning theories, and presents an updated account of the topics that have emerged since the publication of the first edition. Includes: innovative material on MDPs, both in constrained settings and with uncertain transition properties; game-theoretic method for solving MDPs; theories for developing roll-out based algorithms; and details of approximation stochastic annealing, a population-based on-line simulation-based algorithm. The self-contained approach of this book will appeal not only to researchers in MDPs, stochastic modeling, and control, and simulation but will be a valuable source of tuition and reference for students of control and operations research.

Simulation-Based Algorithms for Markov Decision Processes

Robert J Elliott

Robert A. Jarrow

Ju-Yi Yen

This self-contained volume brings together a collection of chapters by some of the most distinguished researchers and practitioners in the field of mathematical finance and financial engineering. Presenting state-of-the-art developments in theory and practice, the book has real-world applications to fixed income models, credit risk models, CDO pricing, tax rebates, tax arbitrage, and tax equilibrium. It is a valuable resource for graduate students, researchers, and practitioners in mathematical finance and financial engineering.

Michael C. Fu

Books

Your next favorite book is out there