Differences

This shows you the differences between two versions of the page.

--- tanszek:oktatas:techcomm:information [2024/10/08 07:23] – [Information] knehez
+++ tanszek:oktatas:techcomm:information [2024/10/15 06:30] (current) – [Example of Entropy calculation] knehez
@@ Line 15: / Line 15: @@
 If an event space consist of two equal-probability event \(p(E_1) = p(E_2) = 0.5 \) then,
-$$ I_{E_1} = I_{E_2} = \log_2 \frac{1}{0.5} = - \log_2 2 = 1 [bit] $$
+$$ I_{E_1} = I_{E_2} = \log_2 \frac{1}{0.5} = - \log_2 0.5 = 1 [bit] $$
 So the unit of the information means the news value which is connected to the simple, less likely, same probability choice.
@@ Line 92: / Line 92: @@
 Let's calculate the maximum entropy:
-$$ H_{\text{max}} = \log_2 n = \log_2 4 = 2 $$
+$$ H_{\text{max}} = \log_2 n = \log_2 4 = 2 [bit]$$
 Then, substituting into the formula for redundancy:
@@ Line 99: / Line 99: @@
 which means the redundancy of the event system is approximately 16%.
+==== Example of Entropy calculation ====
+Why do not store raw password strings in compiled code?
+<sxh c>
+#include <stdio.h>
+#include <math.h>
+float calculateEntropy(unsigned int bytes[], int length);
+char sample[] = "Some poetry types are unique to particular cultures and  genres and respond to yQ%v?FY}ZT=/5cJ.m~A{9^8Lse characteristics of the language in which the poet writes. Readers accustomed to identifying poetry with Dante, Goethe, Mickiewicz, or Rumi may think of it as written in lines based on rhyme and regular meter. There are, however, traditions, such as Biblical poetry and alliterative verse, that use other means to create rhythm and euphony. Much modern poetry reflects a critique of poetic tradition,[6] testing the principle of euphony itself or altogether forgoing rhyme or set rhythm";
+int main()
+{
+    unsigned int byteCounter[256];
+    const int windowWidth = 20;
+    for(int i = 0; i < sizeof(sample) - windowWidth; i++)
+    {
+        memset(byteCounter, 0, sizeof(unsigned int) * 256);
+        char *p = &sample[i];
+        char *end = &sample[i + windowWidth];
+        while(p != end)
+        {
+            byteCounter[(unsigned char)(*p++)]++;
+        }
+        float entropy = calculateEntropy(byteCounter, windowWidth);
+        printf("%d - %.3f\n", i, entropy);
+    }
+}
+float calculateEntropy(unsigned int bytes[], int length)
+{
+    float entropy = 0.0f;
+    for (int i = 0; i < 256; i++)
+    {
+        if (bytes[i] != 0)
+        {
+            float freq = (float) bytes[i] / (float) length;
+            entropy += -freq * log2f(freq);
+        }
+    }
+    return entropy;
+}
+</sxh>