Differences

This shows you the differences between two versions of the page.

--- tanszek:oktatas:techcomm:information [2026/10/05 17:15] – [Entropy] knehez
+++ tanszek:oktatas:techcomm:information [2026/10/06 07:01] (current) – [Example: three coin tosses] knehez
@@ Line 55: / Line 55: @@
 If an event space consist of two equal-probability event \(p(E_1) = p(E_2) = 0.5 \) then,
-$$ I_{E_1} = I_{E_2} = \log_2 \frac{1}{0.5} = - \log_2 0.5 = 1 [bit] $$
+$$ I_{E_1} = I_{E_2} = \log_2 \frac{1}{0.5} = - \log_2 2 = 1 [bit] $$
 So the unit of the information means the news value which is connected to the simple, less likely, same probability choice.
@@ Line 94: / Line 94: @@
   * If every letter occurs with equal probability (e.g., random characters), the entropy is maximal → the source is rich in information.
-This concept is crucial in various fields, including //data compression//, //cryptography//, and //machine learning//, where understanding and managing entropy can lead to more efficient algorithms and systems. For example, in data compression, reducing redundancy (and thus reducing entropy) can lead to more compact data representations. Similarly, in cryptography, managing entropy ensures that keys and encrypted messages are less predictable and more secure.
+This concept is crucial in various fields, including //data compression//, //cryptography//, and //machine learning//, where understanding and managing entropy can lead to more efficient algorithms and systems.
 ==== Redundancy ====