?
Deep Reinforcement Learning in VizDoom First-Person Shooter for Health Gathering Scenario
P. 59–64.
Dmitry Akimov, Makarov I.
In this work, we study the effect of combining existent improvements for Deep Q-Networks (DQN) in Markov Decision Processes (MDP) and Partially Observable MDP (POMDP) settings. Combinations of several heuristics, such as Distributional Learning and Dueling architectures improvements, for MDP are well-studied. We propose a new combination method of simple DQN extensions and develop a new model-free reinforcement learning agent, which works with POMDP and uses well-studied improvements from fully observable MDP. To test our agent we choose the VizDoom environment, which is old first person shooter, and the Health Gathering scenario. We prove that improvements used in MDP setting may be used in POMDP setting as well and our combined agents can converge to better policies. We develop an agent with combination of several improvements showing superior game performance in practice. We compare our agent with Recurrent DQN using Prioritized Experience Replay and Snaphot Ensembling agent and get approximately triple increase in per episode reward.
In book
Lansing: ThinkMind, 2019.
Nikulin A. M., Belousov Y., Svidchenko O. et al., , in: Proceedings of the NeurIPS 2021 Competitions and Demonstrations Track.: PMLR, 2022.
Reinforcement learning competitions advance the field by providing appropriate scope and support to develop solutions toward a specific problem. To promote the development of more broadly applicable methods, organizers need to enforce the use of general techniques, the use of sample-efficient methods, and the reproducibility of the results. While beneficial for the research community, these ...
Added: October 8, 2024
Anton Zakharenkov, Makarov I., , in: Proceedings of IEEE 21st International Symposium on Computational Intelligence and Informatics (CINTI'21), 18-20 Nov. 2021.: NY: IEEE, 2021. P. 000131–000136.
Added: January 19, 2022
Laurent F., Schneider M., Scheller C. et al., , in: Proceedings of Machine Learning ResearchVol. 133: Proceedings of the NeurIPS 2020: Competition and Demonstration Track.: PMLR, 2021. P. 275–301.
The Flatland competition aimed at finding novel approaches to solve the vehicle re-scheduling problem (VRSP). The VRSP is concerned with scheduling trips in traffic networks and the re-scheduling of vehicles when disruptions occur, for example the breakdown of a vehicle. While solving the VRSP in various settings has been an active area in operations research ...
Added: September 6, 2021
Maria Bakhanova, Ilya Makarov, , in: Advances in Computational Intelligence: 16th International Work-Conference on Artificial Neural Networks, IWANN 2021, Virtual Event, June 16–18, 2021, Proceedings, Part I* 1. Vol. 12861.: Springer, 2021. Ch. 12 P. 138–150.
In this work, we study the problem of learning reinforcement learning-based agents in a first-person shooter environment VizDoom. We compare several well-known architectures, such as DQN, DDQN, A3C, and Curiosity-driven model, while highlighting the main differences in learned policies of agents trained via these models. ...
Added: September 1, 2021
Ivanov D., Egorov V., Shpilman A., , in: AAMAS'2021: Proceedings of the 20th International Conference on Autonomous Agents and MultiAgent Systems.: IFAAMAS, 2021. P. 1536–1538.
Recent reinforcement learning studies extensively explore the interplay between cooperative and competitive behaviour in mixed environments. Unlike cooperative environments where agents strive towards a common goal, mixed environments are notorious for the conflicts of selfish and social interests. As a consequence, purely rational agents often struggle to maintain cooperation. A prevalent approach to induce cooperative ...
Added: May 29, 2021
IFAAMAS, 2021.
These are the proceedings of the 20th International Conference on Autonomous Agents and Multiagent Systems (AAMAS-2021). They are published by the International Foundation for Autonomous Agents and Multiagent Systems (IFAAMAS). ...
Added: May 29, 2021
[б.и.], 2020.
Self-driving cars and advanced safety features present one of today’s greatest challenges and opportunities for Artificial Intelligence (AI). Despite billions of dollars of investments and encouraging progress under certain operational constraints, there are no driverless cars on public roads today without human safety drivers. Autonomous Driving research spans a wide spectrum, from modular architectures -- ...
Added: December 28, 2020
Shpilman A., Malysheva A., Kudenko D., , in: Proceedings of 2019 XVI International Symposium "Problems of Redundancy in Information and Control Systems" (REDUNDANCY).: IEEE, 2019. P. 171–176.
Over recent years, deep reinforcement learning has shown strong successes in complex single-Agent tasks, and more recently this approach has also been applied to multi-Agent domains. In this paper, we propose a novel approach, called MAGNet, to multi-Agent reinforcement learning that utilizes a relevance graph representation of the environment obtained by a self-Attention mechanism, and ...
Added: July 15, 2020
Ildar Kamaldinov, Makarov I., , in: Analysis of Images, Social Networks and Texts. 8th International Conference AIST 2019.: Springer, 2019. P. 51–62.
A large number of methods are being developed in the deep reinforcement learning area recently, but the scope of their application is limited. The number of environments does not always allow for a comprehensive assessment of a new agent training algorithm. The main purpose of this article is to present another environment for Match-3 game ...
Added: February 4, 2020
Shpilman A., Kidzinski L., Ong C. et al., , in: The NeurIPS '18 Competition: From Machine Learning to Intelligent Conversations.: Springer, 2020. P. 69–128.
Added: December 2, 2019
Dmitry Akimov, Makarov I., , in: Proceedings of the Fifth Workshop on Experimental Economics and Machine Learning at the National Research University Higher School of Economics co-located with the Seventh International Conference on Applied Research in Economics (iCare7).: Aachen: CEUR Workshop Proceedings, 2019. P. 3–17.
In this work, we study deep reinforcement algorithms for partially observable Markov decision processes (POMDP) combined with Deep Q-Networks. To our knowledge, we are the first to apply standard Markov decision process architectures to POMDP scenarios. We propose an extension of DQN with Dueling Networks and several other model-free policies to training agent using deep ...
Added: November 19, 2019
Ildar Kamaldinov, Makarov I., , in: Procedings of IEEE Conference on Games (COG'19).: NY: IEEE, 2019. P. 1–4.
An increasing number of algorithms in deep reinforcement learning area creates new challenges for environments, particularly, for their comprehensive analysis and searching application areas. The key purpose of this article is to provide an extensible environment for researches. We consider a Match-3 game, which has simple gameplay, but challenging game design for engaging players. The ...
Added: July 30, 2019
Shpilman A., Malysheva A., Kudenko D., , in: Adaptive and Learning Agents Workshop at International Joint Conference on Autonomous Agents and Multiagent Systems.: [б.и.], 2019. P. 1–8.
Over recent years, deep reinforcement learning has shown strong successes in complex single-agent tasks, and more recently this approach has also been applied to multi-agent domains. In this paper, we propose a novel approach, called MAGNet, to multi-agent reinforcement learning that utilizes a relevance graph representation of the environment obtained by a self-attention mechanism, and ...
Added: June 13, 2019
Shpilman A., Malysheva A., Sung T. T. et al., , in: Deep RL Workshop NeurIPS 2018.: [б.и.], 2018. P. 1–10.
Over recent years, deep reinforcement learning has shown strong successes in
complex single-agent tasks, and more recently this approach has also been applied
to multi-agent domains. In this paper, we propose a novel approach, called MAGnet,
to multi-agent reinforcement learning (MARL) that utilizes a relevance graph
representation of the environment obtained by a self-attention mechanism, and
a message-generation technique inspired ...
Added: January 18, 2019
Malysheva A., Shpilman A., Kudenko D., , in: ALA 2018 - Workshop at the Federated AI Meeting 2018.: ALA, 2018. P. 1–7.
Learning to produce efficient movement behaviour for humanoid robots from scratch is a hard problem, as has been illustrated by the "Learning to run" competition at NIPS 2017. The goal of this competition was to train a two-legged model of a humanoid body to run in a simulated race course with maximum speed. All submissions ...
Added: October 16, 2018
Makarov I., Vladimir Aliev, Gerasimova Olga et al., , in: Adjunct Proceedings of 2017 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct).: NY: IEEE, 2017. P. 93–94.
In this paper, we discuss a semi-dense depth map interpolation method based on convolutional neural network. We propose a compact neural network architecture with loss function defined as Euclidean distance in the feature space of VGG-16 neural network used for deep visual recognition. The suggested solution shows state-of-art performance on synthetic and real datasets. Together ...
Added: August 5, 2017