Applebaum - Probability and Information - Integrated Approach 2e (Cambridge, 2008).pdf

(1245 KB) Pobierz
653961086 UNPDF
PROBABILITYAND
INFORMATION
An Integrated Approach
DAVIDAPPLEBAUM
653961086.001.png
CAMBRIDGE UNIVERSITY PRESS
Cambridge, New York, Melbourne, Madrid, Cape Town, Singapore, São Paulo
Cambridge University Press
The Edinburgh Building, Cambridge CB2 8RU, UK
Published in the United States of America by Cambridge University Press, New York
www.cambridge.org
Information on this title: www.cambridge.org/9780521899048
© D. Applebaum 1996, 2008
This publication is in copyright. Subject to statutory exception and to the provision of
relevant collective licensing agreements, no reproduction of any part may take place
without the written permission of Cambridge University Press.
First published in print format
2008
ISBN-13 978-0-511-41424-4
eBook (EBL)
ISBN-13 978-0-521-89904-8
hardback
ISBN-13 978-0-521-72788-4
paperback
Cambridge University Press has no responsibility for the persistence or accuracy of urls
for external or third-party internet websites referred to in this publication, and does not
guarantee that any content on such websites is, or will remain, accurate or appropriate.
653961086.002.png
Contents
Preface to the second edition
p age xi
Preface to the first edition
xiii
1 Introduction
1
1.1 Chance and information
1
1.2 Mathematical models of chance phenomena
2
1.3 Mathematical structure and mathematical proof
5
1.4 Plan of this book
7
2 Combinatorics
10
2.1 Counting
10
2.2 Arrangements
11
2.3 Combinations
13
2.4 Multinomial coefficients
16
2.5 The gamma function
18
Exercises
19
Further reading
21
3 Sets and measures
22
3.1 The concept of a set
22
3.2 Set operations
25
3.3 Boolean algebras
29
3.4 Measures on Boolean algebras
32
Exercises
37
Further reading
40
4 Probability
41
4.1 The concept of probability
41
4.2 Probability in practice
43
vii
viii
Contents
4.3 Conditional probability
48
4.4 Independence
55
4.5 The interpretation of probability
57
4.6 The historical roots of probability
62
Exercises
64
Further reading
68
5 Discrete random variables
70
5.1 The concept of a random variable
70
5.2 Properties of random variables
72
5.3 Expectation and variance
78
5.4 Covariance and correlation
83
5.5 Independent random variables
86
5.6 I.I.D. random variables
89
5.7 Binomial and Poisson random variables
91
5.8 Geometric, negative binomial and hypergeometric
random variables
95
Exercises
99
Furtherreading
104
6 Information and entropy
105
6.1 What is information?
105
6.2 Entropy
108
6.3 Joint and conditional entropies; mutual information
111
6.4 The maximum entropy principle
115
6.5 Entropy, physics and life
117
6.6 The uniqueness of entropy
119
Exercises
123
Further reading
125
7 Communication
127
7.1 Transmission of information
127
7.2 The channel capacity
130
7.3 Codes
132
7.4 Noiseless coding
137
7.5 Coding and transmission with noise – Shannon’s
theorem
143
7.6 Brief remarks about the history of information theory
150
Exercises
151
Further reading
153
Contents
ix
8 Random variables with probability density functions
155
8.1 Random variables with continuous ranges
155
8.2 Probability density functions
157
8.3 Discretisation and integration
161
8.4 Laws of large numbers
164
8.5 Normal random variables
167
8.6 The central limit theorem
172
8.7 Entropy in the continuous case
179
Exercises
182
Further reading
186
9 Random vectors
188
9.1 Cartesian products
188
9.2 Boolean algebras and measures on products
191
9.3 Distributions of random vectors
193
9.4 Marginal distributions
199
9.5 Independence revisited
201
9.6 Conditional densities and conditional entropy
204
9.7 Mutual information and channel capacity
208
Exercises
212
Further reading
216
10 Markov chains and their entropy
217
10.1 Stochastic processes
217
10.2 Markov chains
219
10.3 The Chapman–Kolmogorov equations
224
10.4 Stationary processes
227
10.5 Invariant distributions and stationary Markov chains
229
10.6 Entropy rates for Markov chains
235
Exercises
240
Further reading
243
Exploring further
245
Appendix 1 Proof by mathematical induction
247
Appendix 3 Integration of exp −
2 x 2
249
1
252
Appendix 4 Table of probabilities associated with the standard
normal distribution
254
Appendix 5 A rapid review of matrix algebra
256
Selected solutions
260
Index
268
Appendix 2 Lagrange multipliers

Plik z chomika:

Inne pliki z tego folderu:

Inne foldery tego chomika:

Zgłoś jeśli naruszono regulamin