Information Theory: Coding Theorems for Discrete Memoryless Systems

a-capacity A₁ achievable rate pair achievable rate region alphabet arbitrary assertion asymptotically average probability binary C₁ capacity region channel coding channel network code f codeword set Coding Theorem conditional entropy consider convex convex function Corollary corresponding dd(x decoder defined definition denote distortion measure distribution DMC's e-achievable e-fidelity encoder f entropy entropy rate equals error exponent error probability Esp(R Ex(R exists f₁ Fano's Inequality feedback finite set follows function given Hence Hint implies inequality information measures Information Theory input constraint M₁ maximum probability message set minimum mutual information n-length block code network of Fig output PD's prefix code probability of error problem proof of Theorem prove R₁ random code resp result satisfying sequences Shannon Show source network stochastic matrix subset sufficiently large Theorem 2.3 transmission variables vertex W₁ X₁ Y₁