Efficient numerical algorithms for regularized regression problem with applications to traffic matrix estimations
We study the problem of minimizing the sum of three convex functions: a differentiable, twice-differentiable and a non-smooth term in a high dimensional setting. To this effect we propose and analyze a randomized block cubic Newton (RBCN) method, which in each iteration builds a model of the objective function formed as the sum of the natural models of its three components: a linear model with a quadratic regularizer for the differentiable term, a quadratic model with a cubic regularizer for the twice differentiable term, and perfect (proximal) model for the nonsmooth term. Our method in each iteration minimizes the model over a random subset of blocks of the search variable. RBCN is the first algorithm with these properties, generalizing several existing methods, matching the best known bounds in all special cases. We establish its convergence rates under different assumptions on the component functions. Lastly, we show numerically that our method outperforms the state-of-the-art on a variety of machine learning problems, including cubically regularized least-squares, logistic regression with constraints, and Poisson regression.
The problem of equilibrium distribution of flows in a transportation network in which a part of edges are characterized by cost functions and the other edges are characterized by their capacity and constant cost for passing through them if there is no congestion is studied. Such models (called mixed models) arise, e.g., in the description of railway cargo transportation. A special case of the mixed model is the family of equilibrium distribution of flows over routes—BMW (Beckmann) model and stable dynamics model. The search for equilibrium in the mixed model is reduced to solving a convex optimization problem. In this paper, the dual problem is constructed that is solved using the mirror descent (dual averaging) algorithm. Two different methods for recovering the solution of the original (primal) problem are described. It is shown that the proposed approaches admit efficient parallelization. The results on the convergence rate of the proposed numerical methods are in agreement with the known lower oracle bounds for this class of problems (up to multiplicative constants).
This proceedings publication is a compilation of selected contributions from the “Third International Conference on the Dynamics of Information Systems” which took place at the University of Florida, Gainesville, February 16–18, 2011. The purpose of this conference was to bring together scientists and engineers from industry, government, and academia in order to exchange new discoveries and results in a broad range of topics relevant to the theory and practice of dynamics of information systems. Dynamics of Information Systems: Mathematical Foundation presents state-of-the art research and is intended for graduate students and researchers interested in some of the most recent discoveries in information theory and dynamical systems. Scientists in other disciplines may also benefit from the applications of new developments to their own area of study.