Differences

This shows you the differences between two versions of the page.

--- tanszek:oktatas:techcomm:information [2026/10/05 17:15] – [Entropy] knehez
+++ tanszek:oktatas:techcomm:information [2026/10/06 07:01] (current) – [Example: three coin tosses] knehez
@@ Line 55: / Line 55: @@
 If an event space consist of two equal-probability event \(p(E_1) = p(E_2) = 0.5 \) then,
-$$ I_{E_1} = I_{E_2} = \log_2 \frac{1}{0.5} = - \log_2 0.5 = 1 [bit] $$
+$$ I_{E_1} = I_{E_2} = \log_2 \frac{1}{0.5} = - \log_2 2 = 1 [bit] $$
 So the unit of the information means the news value which is connected to the simple, less likely, same probability choice.
@@ Line 94: / Line 94: @@
   * If every letter occurs with equal probability (e.g., random characters), the entropy is maximal → the source is rich in information.
-This concept is crucial in various fields, including //data compression//, //cryptography//, and //machine learning//, where understanding and managing entropy can lead to more efficient algorithms and systems. For example, lossless data compression represents the same information using fewer bits by exploiting redundancy in its representation. It does not reduce the information that must be preserved: the original data can be reconstructed exactly.
+This concept is crucial in various fields, including //data compression//, //cryptography//, and //machine learning//, where understanding and managing entropy can lead to more efficient algorithms and systems.
-For an intuitive example, you can describe a sequence of 1000 identical letters as "1000 copies of A" instead of writing out every letter. The representation becomes shorter, but no information is lost.
 ==== Redundancy ====