Renyi Entropy and Relative alpha-Entropy

Abstract: Renyi entropy was discovered by the Hungarian mathematician Alfred Renyi in 1961 as a measure of information alternative to Shannon entropy when one desires only a weaker form of additivity as opposed to strong additivity. Relative alpha-entropy  is the Renyi analogue of relative entropy (Kullback-Leibler divergence). In this talk I will discuss four problems, namely Campbell’s source coding problem, Massey-Arikan’s guessing problem, Huieihel et al.’s memoryless guessing problem, and Bunte-Lapidoth’s encoding of tasks problem. In all these problems the objective function to optimize is moments of some functions of random variables and Renyi entropy and relative alpha-entropy arise as optimal solutions. I will also talk about a close relationship among these problems and our on-going work on unifying these problems.