Modern Trends in Controlled Stochastic Processes: Theory and Applications

?

Modern Trends in Controlled Stochastic Processes: Theory and Applications

Liverpool : Luniver Pre, 2011.

Under the general editorship: Piunovskiy A. B.

World leading experts give their accounts of the modern mathematical models in the field: Markov Decision Processes, controlled diffusions, piece-wise deterministic processes etc, with a wide range of performance functionals. One of the aims is to give a general view on the state-of-the-art. The authors use Dynamic Programming, Convex Analytic Approach, several numerical methods, index-based approach and so on. Most chapters either contain well developed examples, or are entirely devoted to the application of the mathematical control theory to real life problems from such fields as Insurance, Portfolio Optimization and Information Transmission. The book will enable researchers, academics and research students to get a sense of novel results, concepts, models, methods, and applications of controlled stochastic processes.

An Optimal Choice of Insurance and Reinsurance Policies in the Risk Process

Golubin A. Y., Gridin V. N., , in: Modern Trends in Controlled Stochastic Processes: Theory and Applications.: Liverpool: Luniver Pre, 2011. P. 216–226.

The chapter studies a dynamic risk model defined on infinite time interval, where both insurance and per-claim reinsurance policies are chosen by the insurer in order to minimize a functional of the form of variation coefficient under constraints imposed with probability one on insured's and reinsurer's risks. We show that the optimum is achieved at ...

Added: April 12, 2012

Language: English

Keywords: Markov Decision Processes controlled diffusions piece-wise deterministic processes

Modern Trends in Controlled Stochastic Processes: Theory and Applications

Mathematical methods of reinforcement learning

Belomestny D., Gasnikov A., Gladin E. et al., Russian Mathematical Surveys 2026 Vol. 81 No. 4(490) P. 3–90

Reinforcement learning (RL) is increasingly grounded in tools from probability, optimization, and operator theory. This survey organizes the mathematical structures that underpin the design and analysis of modern algorithms in RL. We begin from Markov decision processes (MDPs) and the Bellman operators, emphasizing contraction mappings, monotonicity, and fixed-point theory that yield convergence guarantees and rates ...

Added: August 3, 2026

Weighted mesh algorithms for general Markov decision processes: Convergence and tractability

Belomestny D., Schoenmakers J., Zorina V., Journal of Complexity 2025 Vol. 88 Article 101932

We introduce a mesh-type approach for tackling discrete-time, finite-horizon Markov Decision Processes (MDPs) characterized by state and action spaces that are general, encompassing both finite and infinite (yet suitably regular) subsets of Euclidean space. In particular, for bounded state and action spaces, our algorithm achieves a computational complexity that is tractable in the sense of ...

Added: November 10, 2025

Optimal Approximation of Average Reward Markov Decision Processes

Sapronov Y., Yudin N., Computational Mathematics and Mathematical Physics 2025 Vol. 65 No. 3 P. 567–581

We continue to develop the concept of studying the ε-optimal policy for Average Reward Markov Decision Processes (AMDP) by reducing it to Discounted Markov Decision Processes (DMDP). Existing research often stipulates that the discount factor must not fall below a certain threshold. Typically, this threshold is close to one, and as is well-known, iterative methods ...

Added: June 10, 2025

Algorithm for Constrained Markov Decision Process with Linear Convergence

Gladin E., Lavrik-Karmazin M., Zainullina K. et al., Proceedings of Machine Learning Research 2023 Vol. 206 P. 11506–11533

The problem of constrained Markov decision process is considered. An agent aims to maximize the expected accumulated discounted reward subject to multiple constraints on its costs (the number of constraints is relatively small). A new dual approach is proposed with the integration of two ingredients: entropy-regularized policy optimizer and Vaidya’s dual optimizer, both of which ...

Added: November 6, 2024

ON A MIXED SINGULAR/SWITCHING CONTROL PROBLEM WITH MULTIPLE REGIMES

Kelbert M., Moreno-Franco H. A., Advances in Applied Probability 2022 Vol. 54 No. 3 P. 743–782

This paper studies a mixed singular/switching stochastic control problem for a multidimensional diusion with multiple regimes on a bounded domain. Using probabilistic, partial dierential equation (PDE) and penalization techniques, we show that the value function associated with this problem agrees with the solution to a Hamilton-Jacobi-Bellman (HJB) equation. In that way, we see that the regularity of the value function ...

Added: September 26, 2021

Markov Decision Processes and Stochastic Games with Total Effective Payoff

Gurvich V., Boros E., Elbassioni K. et al., , in: 32nd International Symposium on Theoretical Aspects of Computer Science (STACS 2015), Leibniz International Proceedings in Informatics (LIPIcs)Vol. 30.: Dagstuhl: Schloss Dagstuhl--Leibniz-Zentrum fuer Informatik, 2015. P. 103–115.

We consider finite Markov decision processes (MDPs) with undiscounted total effective payoff. We show that there exist uniformly optimal pure stationary strategies that can be computed by solving a polynomial number of linear programs. We apply this result to two-player zero-sum stochastic games with perfect information and undiscounted total effective payoff, and derive the existence ...

Added: October 22, 2016