Deep Reinforcement Learning in VizDoom First-Person Shooter for Health Gathering Scenario

Dmitry Akimov; I. Makarov

?

Deep Reinforcement Learning in VizDoom First-Person Shooter for Health Gathering Scenario

P. 59–64.

In this work, we study the effect of combining existent improvements for Deep Q-Networks (DQN) in Markov Decision Processes (MDP) and Partially Observable MDP (POMDP) settings. Combinations of several heuristics, such as Distributional Learning and Dueling architectures improvements, for MDP are well-studied. We propose a new combination method of simple DQN extensions and develop a new model-free reinforcement learning agent, which works with POMDP and uses well-studied improvements from fully observable MDP. To test our agent we choose the VizDoom environment, which is old first person shooter, and the Health Gathering scenario. We prove that improvements used in MDP setting may be used in POMDP setting as well and our combined agents can converge to better policies. We develop an agent with combination of several improvements showing superior game performance in practice. We compare our agent with Recurrent DQN using Prioritized Experience Replay and Snaphot Ensembling agent and get approximately triple increase in per episode reward.

Язык: английский

Полный текст

Текст на другом сайте

Ключевые слова: first-person shooter Deep Reinforcement Learning VizDoom POMDP глубокое обучение с подкреплением

В книге

Proceedings of 11th International Conference on Advances in Multimedia (MMEDIA'19)

Lansing: ThinkMind, 2019.

MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learned

Никулин А. М., Белоусов Ю. С., Свидченко О. А. и др., , in: Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track.: PMLR, 2022.

Добавлено: 8 октября 2024 г.

Deep Reinforcement Learning with DQN vs. PPO in VizDoom

Anton Zakharenkov, Макаров И. А., , in: Proceedings of IEEE 21st International Symposium on Computational Intelligence and Informatics (CINTI'21), 18-20 Nov. 2021.: NY: IEEE, 2021. P. 000131–000136.

Добавлено: 19 января 2022 г.

Flatland Competition 2020: MAPF and MARL for Efficient Train Coordination on a Grid World

Laurent F., Schneider M., Scheller C. и др., , in: Proceedings of Machine Learning ResearchVol. 133: Proceedings of the NeurIPS 2020: Competition and Demonstration Track.: PMLR, 2021. P. 275–301.

Добавлено: 6 сентября 2021 г.

Deep Reinforcement Learning in VizDoom via DQN and Actor-Critic Agents

Maria Bakhanova, Ilya Makarov, , in: Advances in Computational Intelligence: 16th International Work-Conference on Artificial Neural Networks, IWANN 2021, Virtual Event, June 16–18, 2021, Proceedings, Part I* 1. Vol. 12861.: Springer, 2021. Ch. 12 P. 138–150.

In this work, we study the problem of learning reinforcement learning-based agents in a first-person shooter environment VizDoom. We compare several well-known architectures, such as DQN, DDQN, A3C, and Curiosity-driven model, while highlighting the main differences in learned policies of agents trained via these models. ...

Добавлено: 1 сентября 2021 г.

Balancing Rational and Other-Regarding Preferences in Cooperative-Competitive Environments

Иванов Д. И., Egorov V., Шпильман А. А., , in: AAMAS'2021: Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems.: IFAAMAS, 2021. P. 1536–1538.

Добавлено: 29 мая 2021 г.

AAMAS'2021: Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems

IFAAMAS, 2021.

Добавлено: 29 мая 2021 г.

Workshop on AI for Autonomous Driving (AIAD)

[б.и.], 2020.

Добавлено: 28 декабря 2020 г.

MAGNet: Multi-Agent Graph Network for Deep Multi-Agent Reinforcement Learning

Шпильман А. А., Malysheva A., Kudenko D., , in: Proceedings of 2019 XVI International Symposium "Problems of Redundancy in Information and Control Systems" (REDUNDANCY).: IEEE, 2019. P. 171–176.

Добавлено: 15 июля 2020 г.

Deep Reinforcement Learning Methods in Match-3 Game

Ildar Kamaldinov, Макаров И. А., , in: Analysis of Images, Social Networks and Texts. 8th International Conference AIST 2019.: Springer, 2019. P. 51–62.

A large number of methods are being developed in the deep reinforcement learning area recently, but the scope of their application is limited. The number of environments does not always allow for a comprehensive assessment of a new agent training algorithm. The main purpose of this article is to present another environment for Match-3 game ...

Добавлено: 4 февраля 2020 г.

Artificial Intelligence for Prosthetics: Challenge Solutions

Шпильман А. А., Kidzinski L., Ong C. и др., , in: The NeurIPS '18 Competition: From Machine Learning to Intelligent Conversations.: Springer, 2020. P. 69–128.

Добавлено: 2 декабря 2019 г.

Deep Reinforcement Learning with VizDoom First-Person Shooter

Dmitry Akimov, Макаров И. А., , in: Proceedings of the Fifth Workshop on Experimental Economics and Machine Learning at the National Research University Higher School of Economics co-located with the Seventh International Conference on Applied Research in Economics (iCare7).: Aachen: CEUR Workshop Proceedings, 2019. P. 3–17.

In this work, we study deep reinforcement algorithms for partially observable Markov decision processes (POMDP) combined with Deep Q-Networks. To our knowledge, we are the first to apply standard Markov decision process architectures to POMDP scenarios. We propose an extension of DQN with Dueling Networks and several other model-free policies to training agent using deep ...

Добавлено: 19 ноября 2019 г.

Deep Reinforcement Learning in Match-3 Game

Ildar Kamaldinov, Макаров И. А., , in: Procedings of IEEE Conference on Games (COG'19).: NY: IEEE, 2019. P. 1–4.

An increasing number of algorithms in deep reinforcement learning area creates new challenges for environments, particularly, for their comprehensive analysis and searching application areas. The key purpose of this article is to provide an extensible environment for researches. We consider a Match-3 game, which has simple gameplay, but challenging game design for engaging players. The ...

Добавлено: 30 июля 2019 г.

MAGNet: Multi-agent Graph Network for Deep Multi-agent Reinforcement Learning

Шпильман А. А., Malysheva A., Kudenko D., , in: Adaptive and Learning Agents Workshop at International Joint Conference on Autonomous Agents and Multiagent Systems.: [б.и.], 2019. P. 1–8.

Добавлено: 13 июня 2019 г.

Deep Multi-Agent Reinforcement Learning with Relevance Graphs

Шпильман А. А., Malysheva A., Sung T. T. и др., , in: Deep RL Workshop NeurIPS 2018.: [б.и.], 2018. P. 1–10.

Добавлено: 18 января 2019 г.

Learning to Run with Reward Shaping from Video Data

Malysheva A., Шпильман А. А., Kudenko D., , in: ALA 2018 - Workshop at the Federated AI Meeting 2018.: ALA, 2018. P. 1–7.

Добавлено: 16 октября 2018 г.

Depth Map Interpolation using Perceptual Loss

Макаров И. А., Vladimir Aliev, Gerasimova Olga и др., , in: Adjunct Proceedings of 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct).: NY: IEEE, 2017. P. 93–94.

In this paper, we discuss a semi-dense depth map interpolation method based on convolutional neural network. We propose a compact neural network architecture with loss function defined as Euclidean distance in the feature space of VGG-16 neural network used for deep visual recognition. The suggested solution shows state-of-art performance on synthetic and real datasets. Together ...

Добавлено: 5 августа 2017 г.