Impact of self-learning based high-frequency traders on the stock market

Expert Systems with Applications. 2023. Vol. 232. Article 120567.

Mansurov K., Semenov A., Dmitry Grigoriev, Radionov A., Ibragimov R.

In this paper we investigate the role of self-learning agents in multi-agent models of financial markets. We develop an agent-based simulation model of a financial market and, in addition to the agents with fixed strategies used in previous research, we introduce an agent with a self-learning strategy. To model the behavior of such an agent, we use deep reinforcement learning algorithms, namely deep deterministic policy gradient (DDPG). Next, we conduct a comparative analysis of the results of the constructed model with outcomes of previously proposed models, as well as with the characteristics of real market. To conduct comparative analysis, we use stylized facts of asset returns that allow us to evaluate and compare the characteristics of the markets. Our results show that a model with a self-learning agent gives a better approximation of the real market than a model with classic agents. In particular, unlike the model with classical agents, the model with a self-learning agent turns out to be not so heavy-tailed. Thus, we demonstrate that for a complete understanding of market processes simulation models should take into account self-learning agents that have a significant presence at modern financial markets.

Finally, we present the python package, which was developed by us as part of the research implementation. This package allows to simulate the financial market, as well as create your own agents and evaluate their impact on the market.

Keywords: stylized facts, agent-based model, Reinforcement learning, Market simulations