Information Theory, Inference and Learning Algorithms Reading

1. preface

 binary symmetric channe

l

 Information theory and coding theory oer an alternative (and much more exciting)

approach: we accept the given noisy channel as it is and add communication

systems  to it so that we can detect and correct the errors introduced by

the channel. As shown in gure 1.6, we add an encoder  before the channel and

a decoder  after it.

Majority-vote

decoding algorithm

chapter 4:The Source Coding Theorem