Institute of Information Science - University of Miskolc

This is an old revision of the document!

Cheatsheet for Math Exercises

Notation	Value	Formula
$P(A)$	Probability of event A occuring.	$P(A) = \frac{\text{Number of favorable outcomes for } A}{\text{Total number of possible outcomes}}$
$P(A \mid B)$	Conditional probability of event A occurring, given that event B has occurred.	$P(A \mid B) = \frac{P(A \cap B)}{P(B)}$
$P(A \cap B)$	Probability of both events A and B occurring.	In general: $P(A \cap B) = P(A) \cdot P(B \mid A)$ If A and B are independent events, then: $P(A \cap B) = P(A) \cdot P(B)$
$P(A \cup B)$	Probability that event A or event B (or both) occur.	$P(A \cup B) = P(A) + P(B) - P(A \cap B)$

Notation	Value	Formula
$I(A)$	Information content or self-information of an event A.	$I(A) = -\log_2 P(A) \text{ [bits]}$
$H(X)$	Entropy, which measures the average amount of information (or uncertainty) in a random variable X.	$H(X) = -\sum_{x \in X} P(x) \log_2 P(x) \text{ [bits]}$
$H_{max}$	Maximum possible entropy (when all outcomes are equally likely).	$H_{\text{max}} = \log_2 \|\mathcal{X}\|$ $\|\mathcal{X}\| \text{ is the number of possible outcomes in the set } \mathcal{X}$
$R(X)$	Redundancy, which measures the portion of duplicative information within a message.	$R(X) = 1 - \frac{H(X)}{\log_2 \|X\|}$ In terms of maximum entropy: $R = \frac{H_{\text{max}} - H}{H_{\text{max}}}$

Notation	Value	Formula
	A