• Examples of Joint Entropy

  • The Grouping Property of Entropy

  • Axiomatic Derivation of the Entropy Function

  • Jensen's Inequality Handout

  • Long Jensen's Inequality Handout

  • The Law of Large Numbers

  • Chains and Processes

  • More Chains and Processes

  • Converting Second-Order Process to Markov

  • Simulation of the AEP

  • Some Uses of Calculus in IT

  • The Effect of Averaging a Distribution

  • McMillan's Proof

  • Shannon, The Mathematical Theory of Communication Sources

  • Arithmetic Coding
    • Arithmetic Coding Article

    • Arithmetic Coding Algorithm from Numerical Recipes in C, 20.5

    • More Arithmetic Coding .pdf

  • Types and Fitingof Coding

  • Block Sorting, the Burrows-Wheeler Transform

  • Effros's Analysis of Block Sorting

  • Shannon, TMTC Derivation of the Entropy Function

  • Shannon's class notes, from Peterson & Weldon, 1981, Appendix A
    pp. 466 , 467 , 468 , 469 .

  • A Source whose Uncertainty is not Defined

  • Insufficiency of Columns in the BWT Array

  • Capacity of the Z Channel (.ps)

  • Discrete Noiseless Channels .pdf

  • Behavior of K( n | l(n) ) .pdf

  • Laplacian Estimation .pdf

  • Recurrences and Matrices .pdf

  • Joint and Conditional Distributions and Entropies .pdf

  • Properties of Fundamental Functions .pdf

  • Properties of Markov Chains .pdf

  • Mixing Scotch and Balls .pdf