Institute of Information Science - University of Miskolc

This is an old revision of the document!

Cheatsheet for Math Exercises

Notation	Value	Formula
$$P(A)$$	Probability of event A occuring.	$$P(A) = \frac{\text{Number of favorable outcomes for } A}{\text{Total number of possible outcomes}}$$
$$P(A \mid B)$$	Conditional probability of event A occurring, given that event B has occurred.	$$P(A \mid B) = \frac{P(A \cap B)}{P(B)}$$
$$P(A \cap B)$$	Probability of both events A and B occurring.	In general: $$P(A \cap B) = P(A) \cdot P(B \mid A)$$ If A and B are independent events, then: $$P(A \cap B) = P(A) \cdot P(B)$$
$$P(A \cup B)$$	Probability that event A or event B (or both) occur.	$$P(A \cup B) = P(A) + P(B) - P(A \cap B)$$

Notation	Value	Formula
$$I(A)$$	Information content or self-information of an event A.	$$I(A) = -\log_2 P(A) \text{ [bits]}$$
$$H(X)$$	Entropy, which measures the average amount of information (or uncertainty) in a random variable X.	$$H(X) = -\sum_{x \in X} P(x) \log_2 P(x) \text{ [bits]}$$
$$H_{max}$$	Maximum possible entropy (when all outcomes are equally likely).	$$H_{\text{max}} = \log_2 \|\mathcal{X}\|$$ $$\|\mathcal{X}\| \text{ is the number of possible outcomes in the set } \mathcal{X}$$
$$R(X)$$	Redundancy, which measures the portion of duplicative information within a message.	$$R(X) = 1 - \frac{H(X)}{\log_2 \|X\|}$$ In terms of maximum entropy: $$R = \frac{H_{\text{max}} - H}{H_{\text{max}}}$$