Book
Proceedings of Information Technology and Systems 2015
Consider a Bayesian problem of estimating of probability of success in a series of trials with binary outcomes. We study the asymp- totic behaviour of weighted differential entropy for posterior probability density function (PDF) conditional on x successes after n trials, when n → ∞. Suppose that one is interested to know whether the coin is fair or not and for large n is interested in true frequency. In other words, one wants to emphasize the parameter value p = 1/2. To do so the concept of weighted differential entropy introduced in [1968] is used when the frequency γ is necessary to emphasize. It was found that the weight in suggested form does not change the asymptotic form of Shannon, Renyi, Tsallis and Fisher entropies, but change the constants. The leading term in weighted Fisher Information is changed by some constant which depend on distance between the true frequency and the value we want to emphasize.
This book constitutes the refereed proceedings of the 5th International Castle Meeting on Coding Theory and Applications, ICMCTA 2017, held in Vihula, Estonia, in August 2017.
The 24 full papers presented were carefully reviewed and selected for inclusion in this volume. The papers cover relevant research areas in modern coding theory, including codes and combinatorial structures, algebraic geometric codes, group codes, convolutional codes, network coding, other applications to communications, and applications of coding theory in cryptography.
This is the landmark international conference of the IEEE Information Theory Society. The conference will seek original contributions in: Coding theory and practice, Communication theory, Compression, Cryptography and data security, Detection and estimation, Information theory and statistics, Information theory in networks, Multi-terminal information theory, Pattern recognition and learning, Quantum information theory, Sequences and complexity, Shannon theory, Signal processing, Source coding
We investigate one possible generalization of locally recoverable codes (LRC) with all-symbol locality and availability when recovering sets can intersect in a small number of coordinates. This feature allows us to increase the achievable code rate and still meet load balancing requirements. In this paper we derive an upper bound for the rate of such codes and give explicit constructions of codes with such a property. These constructions utilize LRC codes developed by Wang et al.
Understanding the relation between (sensory) stimuli and the activity of neurons (i.e., "the neural code") lies at heart of understanding the computational properties of the brain. However, quantifying the information between a stimulus and a spike train has proven to be challenging. We propose a new (in vitro) method to measure how much information a single neuron transfers from the input it receives to its output spike train. The input is generated by an artificial neural network that responds to a randomly appearing and disappearing "sensory stimulus": the hidden state. The sum of this network activity is injected as current input into the neuron under investigation. The mutual information between the hidden state on the one hand and spike trains of the artificial network or the recorded spike train on the other hand can easily be estimated due to the binary shape of the hidden state. The characteristics of the input current, such as the time constant as a result of the (dis)appearance rate of the hidden state or the amplitude of the input current (the firing frequency of the neurons in the artificial network), can independently be varied. As an example, we apply this method to pyramidal neurons in the CA1 of mouse hippocampi and compare the recorded spike trains to the optimal response of the "Bayesian neuron" (BN). We conclude that like in the BN, information transfer in hippocampal pyramidal cells is non-linear and amplifying: the information loss between the artificial input and the output spike train is high if the input to the neuron (the firing of the artificial network) is not very informative about the hidden state. If the input to the neuron does contain a lot of information about the hidden state, the information loss is low. Moreover, neurons increase their firing rates in case the (dis)appearance rate is high, so that the (relative) amount of transferred information stays constant.
We address the problem of constructing coding schemes for the channels with high-order modulations. It is known, that non-binary LDPC codes are especially good for such channels and significantly outperform their binary counterparts. Unfortunately, their decoding complexity is still large. In order to reduce the decoding complexity, we consider multilevel coding schemes based on non-binary LDPC codes (NB-LDPC-MLC schemes) over smaller fields. The use of such schemes gives us a reasonable gain in complexity. At the same time, the performance of NB-LDPC-MLC schemes is practically the same as the performance of LDPC codes over the field matching the modulation order. In particular, by means of simulations, we showed that the performance of NB-LDPC-MLC schemes over GF(16) is the same as the performance of non-binary LDPC codes over GF(64) and GF(256) in AWGN channel with QAM 64 and QAM 256 accordingly. We also perform a comparison with bit-interleaved coded modulation based on binary LDPC codes.
Consider a Bayesian problem of estimating of probability of success in a series of trials with binary outcomes. We study the asymp- totic behaviour of weighted differential entropy for posterior probability density function (PDF) conditional on x successes after n trials, when n → ∞. Suppose that one is interested to know whether the coin is fair or not and for large n is interested in true frequency. In other words, one wants to emphasize the parameter value p = 1/2. To do so the concept of weighted differential entropy introduced in [1968] is used when the frequency γ is necessary to emphasize. It was found that the weight in suggested form does not change the asymptotic form of Shannon, Renyi, Tsallis and Fisher entropies, but change the constants. The leading term in weighted Fisher Information is changed by some constant which depend on distance between the true frequency and the value we want to emphasize.
A words phonetic decoding method in automatic speech recognition is considered. The properties of Kullback–Leibler divergence are used to synthesize the estimation of the distribution of divergence between minimum speech units (e.g., single phonemes) inside a single class. It is demonstrated that the min imum variance of the intraphonemic divergence is reached when the phonetic database is tuned to the voice of a single speaker. The estimations are proven by experimental results on the recognition of vowel sounds and isolated words of Russian language.
We establish a new upper bound for the Kullback-Leibler divergence of two discrete probability distributions which
are close in a sense that typically the ratio of probabilities is nearly one and the number of outliers is small.
Let k be a field of characteristic zero, let G be a connected reductive algebraic group over k and let g be its Lie algebra. Let k(G), respectively, k(g), be the field of k- rational functions on G, respectively, g. The conjugation action of G on itself induces the adjoint action of G on g. We investigate the question whether or not the field extensions k(G)/k(G)^G and k(g)/k(g)^G are purely transcendental. We show that the answer is the same for k(G)/k(G)^G and k(g)/k(g)^G, and reduce the problem to the case where G is simple. For simple groups we show that the answer is positive if G is split of type A_n or C_n, and negative for groups of other types, except possibly G_2. A key ingredient in the proof of the negative result is a recent formula for the unramified Brauer group of a homogeneous space with connected stabilizers. As a byproduct of our investigation we give an affirmative answer to a question of Grothendieck about the existence of a rational section of the categorical quotient morphism for the conjugating action of G on itself.
Let G be a connected semisimple algebraic group over an algebraically closed field k. In 1965 Steinberg proved that if G is simply connected, then in G there exists a closed irreducible cross-section of the set of closures of regular conjugacy classes. We prove that in arbitrary G such a cross-section exists if and only if the universal covering isogeny Ĝ → G is bijective; this answers Grothendieck's question cited in the epigraph. In particular, for char k = 0, the converse to Steinberg's theorem holds. The existence of a cross-section in G implies, at least for char k = 0, that the algebra k[G]G of class functions on G is generated by rk G elements. We describe, for arbitrary G, a minimal generating set of k[G]G and that of the representation ring of G and answer two Grothendieck's questions on constructing generating sets of k[G]G. We prove the existence of a rational (i.e., local) section of the quotient morphism for arbitrary G and the existence of a rational cross-section in G (for char k = 0, this has been proved earlier); this answers the other question cited in the epigraph. We also prove that the existence of a rational section is equivalent to the existence of a rational W-equivariant map T- - - >G/T where T is a maximal torus of G and W the Weyl group.