Institute of Information Science - University of Miskolc

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== Conditional Probability and Information Theory Exercises =====

**Related theory: [[tanszek:oktatas:techcomm:information_-_processing|Lecture 2: Information Processing]]**

  - What is the probability of rolling an odd number with a fair dice? How many bits of information does the statement $\text{"we roll an odd number with a fair dice"}$ contain? //(p=0.5; I=1 bit)// \\ \\ 
  - We roll two identical dice twice. What is the probability that we first roll at least one 3, and then immediately roll a 2 and a 3 at the same time (one die shows 2 and the other 3)?  //(p=0.02)// \\ \\
  - An urn contains 2 white and 2 black balls.
    - What is the probability of drawing two white balls consecutively without replacement? //(0.17)//
    - How many bits of information does the statement $\text{"we draw 2 white balls consecutively without replacement from an urn containing 2 white and 2 black balls"}$ contain? //(2.59 bits)//
    - What is the entropy of this set of information (i.e., the average amount of information per news item)? //(1.25 bits)//
    - What is the entropy of the set of information for a single draw? //(1 bit)//
    - What would be the entropy of the set of information for two draws if the four balls were all different colors (e.g., one white, one black, one red, and one green)? //(2.58 bits)// \\ \\
  - An automated machine has a defect rate of 10% (it produces 10 defective units out of 100 units on average).
    - What is the information content of the news that, out of 10 items, 2 are selected and they are defect-free? //(0.32 bit)//
    - What is the information content if we select 5 out of 10 units and all are defect-free? //(1 bit)// \\ \\
  - In a manufacturing process, the expected defect rate is 0.25. After producing 20 units, we randomly and simultaneously select 2 of them and inspect them:
    - How many bits of information has the news that $\text{"we selected 2 defective parts"}$ contain? //(4.25 bits)//
    - How many elements does the news set have (i.e., what are the number of possible outcomes)? What is the entropy of the set? //(n=3; H=1.23 bits)// \\ \\
  - In a communication system, we want to transmit 2-character symbolic words using the characters $A$, $B$, and $C$, with each word having equal probability, using a binary fixed-length code.
    - How many such symbolic code words can be formed? //(9)//
    - What is the minimum number of bits needed for coding? //(4 bits)//
    - What is the average amount of information content for the words transmitted? //(3.17 bits)//
    - What is the redundancy of the code? //(0.21)// \\ \\