Search Results

Turns out $D(p_t||\pi)$ need not always be convex. The following paper demonstrates a counter-example in section $4.2$- http://arxiv.org/abs/0712.2578

Suppose the chain starts with the initial distribution $p$ at time $0$, then at time $t$ the distribution is given by $$p_t = p*e^{Qt}$$ The relative entropy of $p_t$ with respect to the $\pi$ (also known …

Let $H(x) = -x\log(x) - (1-x)\log(1-x)$ be the binary entropy function. …

I think I managed to prove the entire inequality analytically. The whole proof is a bit long to post here (about 7 pages) and involves ugly looking expressions. I'll outline the general strategy I use …

Also intuitively the more concentrated a distribution, the lesser its entropy. So forcing $X$ and $Y$ to be supported in $G$ also ensures $Z$ is supported only on $G$, which is "small". … An exhaustive search of all possible "entropy fixing" distributions of $X$ and $Y$ goes out of hand very fast. …

But consider $$p_X = [0.5, 0, 0, 0.5, 0, 0]$$ and $$p_Y = [0.5 - \delta, \frac{\delta}{2}, \frac{\delta}{2},0.5 - \delta, \frac{\delta}{2}, \frac{\delta}{2}]$$ for a suitable $\delta$ which adjusts the entropy …