Asymptotic analysis of the Renyi, Tsallis and Fisher entropies in a Bayesian problem

P. 614–621.

Mozgunov P., Kelbert M.

Consider a Bayesian problem of estimating of probability of success in a series of trials with binary outcomes. We study the asymp- totic behaviour of weighted differential entropy for posterior probability density function (PDF) conditional on x successes after n trials, when n → ∞. Suppose that one is interested to know whether the coin is fair or not and for large n is interested in true frequency. In other words, one wants to emphasize the parameter value p = 1/2. To do so the concept of weighted differential entropy introduced in [1968] is used when the frequency γ is necessary to emphasize. It was found that the weight in suggested form does not change the asymptotic form of Shannon, Renyi, Tsallis and Fisher entropies, but change the constants. The leading term in weighted Fisher Information is changed by some constant which depend on distance between the true frequency and the value we want to emphasize.

Keywords: теория информации, Fisher information, информация Фишера, information theory, weighted differential entropy, Renyi entropy, Tsallis entropy, взвешенная дифференциальная энтропия, Энтропия Реньи, Энтропия Тцалисса

Publication based on the results of: Стохастические и функционально-аналитические методы в исследовании сложных динамических процессов в экономике (2014)

In book

Proceedings of Information Technology and Systems 2015

Sochi: ., 2015.