• A
  • A
  • A
  • ABC
  • ABC
  • ABC
  • А
  • А
  • А
  • А
  • А
Regular version of the site

Working paper

Low-variance Gradient Estimates for the Plackett-Luce Distribution

Bayesian Deep Learning NeurIPS 2019 Workshop. 2019. Bayesian Deep Learning NeurIPS 2019 Workshop, 2019
Gadetsky A., Struminsky K., Robinson C., Quadrianto N., Vetrov D.
Learning models with discrete latent variables using stochastic gradient descent remains a challenge due to the high variance of gradients. Modern variance reduction techniques mostly consider categorical distributions and have limited applicability when the number of possible outcomes becomes large. In this work, we consider models with latent permutations and propose control variates for the Plackett-Luce distribution. Our proof-of-concept experiment recasts optimization over permutations as a variational optimization w.r.t. the Plackett-Luce distribution and solves it using stochastic gradient descent.