?
CatBoost: unbiased boosting with categorical features
P. 6638–6648.
Liudmila Prokhorenkova, Gusev G., Vorobev A., Dorogush A. V., Gulin A.
This paper presents the key algorithmic techniques behind CatBoost, a new gradient boosting toolkit. Their combination leads to CatBoost outperforming other publicly available boosting implementations in terms of quality on a variety of datasets. Two critical algorithmic advances introduced in CatBoost are the implementation of ordered boosting, a permutation-driven alternative to the classic algorithm, and an innovative algorithm for processing categorical features. Both techniques were created to fight a prediction shift caused by a special kind of target leakage present in all currently existing implementations of gradient boosting algorithms. In this paper, we provide a detailed analysis of this problem and demonstrate that proposed algorithms solve it effectively, leading to excellent empirical results.
Keywords: Gradient boosting
In book
Neural Information Processing Systems Foundation, 2018.
Lysenok N., International Journal of Computer Information Systems and Industrial Management Applications 2026 Vol. 18 No. 17s P. 533–549
This paper asks which families of volatility forecasting models create economic value in active trading, and through which integration channel that value is transmitted. Seven models drawn from four families — econometric (GJR-GARCH, HAR-J), gradient boosting (XGBoost, LightGBM), recurrent neural networks (LSTM, GRU) and a hybrid combining HAR-J with boosting — are compared on the ...
Added: August 18, 2026
Zelenkov Y., , in: 2023 Ivannikov ISPRAS Open Conference (ISPRAS).: IEEE, 2023. P. 176–182.
We propose a regression ensemble based on a decomposition that separates the weighted average errors of individual learners and the ambiguity of their estimates. This approach is a modification of Gradient Boosting with a variation of the gradient at each step. That allows ensuring explicitly a diversity of base estimators. In addition, the proposed approach ...
Added: May 1, 2024
Bukina T. V., Kashin D., Экономический журнал Высшей школы экономики 2024 Т. 28 № 1 С. 81–107
The paper reveals the forecasts for regional inflation based on the regions of the Privolzhskiy Federal District (PFD). The purpose of the study is to determine the model that most accurately predicts regional inflation. The paper compares the tools of machine learning – support vector machines, gradient boosting, and random forest – with econometric models ...
Added: February 13, 2024
Muratova A., Mitrofanova E., Islam R., , in: Procedia Computer Science: 11th International Young Scientist Conference on Computational ScienceVol. 212.: Elsevier, 2022. P. 358–367.
The article presents a case study on demographic sequences analysis through modern machine learning (ML)
techniques. The studied data contains demographic and socioeconomic events, where the events are presented
as sequences of statuses. The involved demographers are interested in applications of advanced ML techniques
and interpretable patterns for their needs. We show how Shapley value-based explanations can be ...
Added: September 10, 2022
Ustimenko A., Prokhorenkova L., , in: Proceedings of the 38th International Conference on Machine Learning (ICML 2021)Vol. 139.: PMLR, 2021. P. 1–10.
This paper introduces Stochastic Gradient Langevin Boosting (SGLB) - a powerful and efficient machine learning framework that may deal with a wide range of loss functions and has provable generalization guarantees. The method is based on a special form of the Langevin diffusion equation specifically designed for gradient boosting. This allows us to theoretically guarantee ...
Added: August 6, 2021
Малинин А. А., Prokhorenkova L., Ustimenko A., , in: Proceedings of the 9th International Conference on Learning Representations (ICLR 2021). ICLR, 2021.: ICLR, 2021..
Added: August 2, 2021
Ivanov S., Prokhorenkova L., , in: Proceedings of the 9th International Conference on Learning Representations (ICLR 2021). ICLR, 2021.: ICLR, 2021..
Added: August 2, 2021
Liudmila Prokhorenkova, Ustimenko A., , in: International Conference on Machine Learning (ICML 2020)Vol. 119.: PMLR, 2020. P. 9669–9679.
In this paper, we introduce a powerful and efficient framework for direct optimization of ranking metrics. The problem is ill-posed due to the discrete structure of the loss, and to deal with that, we introduce two important techniques: stochastic smoothing and novel gradient estimate based on partial integration. We show that classic smoothing approaches may ...
Added: January 14, 2021
Antipov E. A., Pokryshevskaya E. B., Journal of Revenue and Pricing Management 2020 No. 19 P. 355–364
Forecasting demand and understanding sales drivers are one of the most important tasks in retail analytics. However, traditionally, linear models and/or models with a small number of predictors have been predominantly used in sales modeling. Taking into account that real-world demand is naturally determined by complex substitution and complementation patterns among a large number of ...
Added: October 31, 2020
Timur Kadyrov, Ignatov D. I., , in: Proceedings of the Fifth International Workshop on Experimental Economics and Machine Learning (EEML 2019),Perm, Russia, September 26, 2019Vol. 2479.: CEUR Workshop Proceedings, 2019. P. 77–88.
A multichannel attribution model based on gradient boost- ing over trees is proposed, which was compared with the state of the art models: bagged logistic regression, Markov chains approach, shapely value. Experiments on digital advertising datasets showed that the pro- posed model is better than the solutions considered by ROC AUC metric. In addition, the ...
Added: January 20, 2020
Kuralenok I., Ershov V., Лабутин И. Н., , in: Advances in Neural Information Processing Systems 32 (NeurIPS 2019).: [б.и.], 2019. P. 1–10.
In this work, we introduce a new decision tree ensemble representation framework: instead of using a graph model we transform each tree into a well-known polynomial form. We apply the new representation to three tasks: theoretical analysis, model reduction, and interpretation. The polynomial form of a tree ensemble allows a straightforward interpretation of the original ...
Added: December 27, 2019
Bulychev A., Сомов О. Д., В кн.: Информатика, управление и системный анализ: Труды V Всероссийской научной конференции молодых ученых с международным участием.: Ростов н/Д: Ростовский государственный экономический университет "РИНХ", 2018. С. 94–102.
In the process of developing an information system for logistics transportation, there is a need to determine the initial rating of the new carrier within the parent company. The presence of the rating helps to more accurately carry out the formation of orders and build forecasts of its interaction with the parent company in the ...
Added: September 3, 2019
Kulagin M., Sidorenko V., , in: Proceedings of the Third International Scientific Conference “Intelligent Information Technologies for Industry” (IITI’18) Volume 2Vol. 2.: Springer, 2019. P. 308–316.
This article describes modern methods of data processing regarding the task of assessing activities of transportation employees. The main purpose was to find dependencies in data and construct an algorithm for predicting the probability of transport safety violation by employee. The research was conducted for locomotive drivers. The following algorithms were used: neural networks, gradient boosting over decision trees and ...
Added: January 19, 2019