?
Primal-dual fast gradient method with a model
In this work we consider a possibility to use the conception of (δ,L)-model of a function for optimization tasks, whereby solving a primal problem there is a necessity to recover a solution of a dual problem. The conception of (δ,L)-model is based on the conception of (δ,L)-oracle which was proposed by Devolder – Glineur – Nesterov, here with the authors proposed approximate a function with an upper bound using a convex quadratic function with some additive noise δ. They managed to get convex quadratic upper bounds with noise even for nonsmooth functions. The conception of (δ,L)-model continues this idea by using instead of a convex quadratic function a more complex convex function in an upper bound. Possibility to recover the solution of a dual problem gives great benefits in different problems, for instance, in some cases, it is faster to find a solution in a primal problem than in a dual problem. Note that primal-dual methods are well studied, but usually each class of optimization problems has its own primal-dual method. Our goal is to develop a method which can find solutions in different classes of optimization problems. This is realized through the use of the conception of (δ,L)-model and adaptive structure of our methods. Thereby, we developed primal-dual adaptive gradient method and fast gradient method with (δ,L)-model and proved convergence rates of the methods, moreover, for some classes of optimization problems the rates are optimal. The main idea is the following: we find a dual solution to an approximation of a primal problem using the conception of (δ,L)-model. It is much easier to find a solution to an approximated problem, however, we have to do it in each step of our method, thereby the principle of “divide and conquer” is realized.