To Notes

IT 223 -- May 15, 2024

Review Exercises

1. What is a sample space?
   Answer: The sample space is the set of all possible outcomes for a statistical experiment.
2. Which greek letter is used to represent the sample space?
   Answer: big omega: Ω
3. What is an event?
   Answer: an event is a subspace of the sample space.
4. What are the rules for probabilities?
   Answer: if P(e) is the probability of the event e,
   (a) 0 ≥ P(e)     (b) P(e) ≥ 1     (c) P(e^C) = 1 - P(e)
5. What are the three methods for obtaining probabilities?
   Answer: (a) a priori     (b) empirical      (c) subjective.
6. What is a discrete random variable?
   Answer: a discrete random variable has only a finite number of outcomes.
7. What is a Bernoulli random variable?
   Answer: a bernoulli random variable has only two possible outcomes 0 and 1 defined by this probablity table:
   

   Outcome	Probability
   0	1 - p
   1	p

   
   p is called the probability of success.
8. What is the expected value of a random variable?
   Answer: If
         x₁, x₂, ... , x_n
   are the values of the random variable X and
         p₁, p₂, ... , p_n
   are the corresponding probilities, the expected value of X is
         E(X) = x₁ p₁ + x₂ p₂ + ... + x_n p_n

Expected Value

• The expected value of a random variable is its theoretical average, based on the probabilities of its outcomes.
• The expected value of a random variable x can be calculated using the probability table.
  

  Outcome	Probability
  x1	p1
  x2	p2
  ...	...
  xk	pk

  
  The expected value is:
       E(x) = x₁ p₁ + x₂ p₂ + ... + x_k p_k
• The expected value is the weighted average of the outcomes of the random variable. You don't need to divide by the sum of the weights because the sum of the probabilities is 1.
• The expected value is also called the risk.
  Example 13:   Suppose the probability that the Bears will the Super Bowl next year is 1/20 = 0.05. You make a bet with your friend. If the Bears will you get $100. If the bears do not will, you pay him $10. What is the expected value of this bet?
  

  Payoff	Probability
  -10	0.95
  100	0.05

  
  Note: the payoff of 30 in the table is negative because you are losing $10.

  Expected value = (-10) * 0.95 + 100 * 0.05 = -4.5

  The conclusion is that you will lose $4.50 on the average every time you make this bet.

• Example 14:   Flip a coin 3 times. You win $10 every time 3 heads come up, you lose $1 every time 1 head comes up and you lose $5 every time 0 heads come up.You do do not lose or win anything if 2 heads come up. Compute the expected value.
  Ans: Here is the payoff table:
  

  Payoff	Probablity
  10	1/8
  0	3/8
  -1	3/8
  -5	1/8

  
  E(x) = 10(1/8) + 0(1/8) + (-1)(3/8) + (-5)(1/8) = 2/8 = $0.25
• Example 15:   In the summer, a tropical island has the same probability distribution for the amount of rain every day:
  

  Rainfall	Probability
  0	0.3
  1	0.4
  2	0.2
  3	0.1

  
  What is the expected value for the amount of rain in one day?
  Ans: E(x) = x₁ p₁ + x₂ p₂ + x₃ p₃ + x₄ p₄

                = 0 * 0.3 + 1 * 0.4 + 2 * 0.2 + 3 * 0.1 = 1.1

• Example 16:   You pay $75 per year on a $500,000 house for insurance. The probability that your house is destroyed in a given year is 0.0001.
  
  1. What is the expected value for the insurance company per year?
     Ans: Here is the payout table:
     

     Payout	Probability
     75	0.9999
     -500,000 + 75	0.0001

     
     The expected value is 75 * 0.9999 + (-500,000 + 75) * 0.0001 = 75 * (0.9999 + 0.0001) + (-500,000) * 0.0001 = 75 + (-50) = $25.
  2. What is your risk per year?

     Ans: You pay $75 to the insurance company whether or not your house is destroyed, so the expected value is $-75.

  3. What would your risk be if you did not have insurance?
     Ans: The payout table is
     

     Payout	Probability
     0	0.9999
     -500,000	0.0001

     
     The expected value is 0 * 0.9999 + (-500,000) * 0.0001 = -$50.
  4. What would the "pure premium" to the insurance company be? The pure premium is the premium you would have to pay if the expected value for the insurance company were $0.
     Ans: Solve for p in the following: p (0.9999) + (-500,000 + p) 0.0001 = 0. This gives p = 50.

The Multiplication Rule for Independent Events

• Two events A and B are independent if the occurence of A does not affect the probability that B occurs.
• Example 17:   Flip a coin twice. Define the event A to be getting a head on the first toss. Define the event B to be getting a head on the second toss. The events A and B are independent.
• The Multiplication Rule:   If A and B are independent, then
       P(A and B) = P(A) * P(B)
• Example 18:   Flip a coin 10 times. What is the probability of getting all heads?

  Ans: P(H and H and ... and H (10 heads)) = P(H) * P(H) * ... * P(H) = 0.5¹⁰ = 0.0009766.

• Example 19:   The probability that I will crash on a single plane ride is 0.0001. What is my probability of crashing at least once on 500 independent plane rides?
  Ans: Let A_i be the event of not crashing on the ith plane flight. Then
        P(A₁ and ... and A₅₀₀) = P(A₁) * ... * P(A₅₀₀) = 0.9999⁵⁰⁰ = 0.951227.
  Therefore the probability of crashing on one of the 500 flights is 1 - 0.9512 = 0.04877 = 4.9%.
  Working from the formula gives us
  1 - (1 - p)ⁿ = 1 - (1 - 0.0001)⁵⁰⁰ = 0.04877 = 4.9%
• Example 20:   The probability of contracting AIDS from a single sexual encounter is about 1 out of 1000. What is the probability of contracting AIDS from 20 independent encounters?
  Ans: 1 - (1 - p)ⁿ = 1 - (1 - 1 / 1000)²⁰ = 0.01981 = 1.98%
• In the real world, determining if two events A and B are independent is difficult. There are many lurking variables that can cause A and B to occur or not to occur.

The Addition Rule for Mutually Exclusive Events

• Two events A and B are mutually exclusive if A and B cannot both occur at the same time.
• Example 21:   "Obtaining a 1 with one roll of a die" and "obtaining a 2 with one roll of a die" are mutually excusive.
• Example 22:   In a baseball game, "getting a hit in one at bat" and "getting a base on balls" in one at bat are mutually exclusive.
• The Addition Rule:   If A and B are mutually exclusive events, then
       P(A or B) = P(A) + P(B)
• Example 23:   If event A is "obtaining a 1 with one die" and B is "obtaining a 2 with one die", since A and B are mutually exclusive, P(A or B) = P(A) + P(B) = 1/6 + 1/6 = 1/3.

The Standard Deviation of a Random Variable

• We already know the definition of the expected value:
  E(x) = x₁ p₁ + ... + x_n p_n
• Often the symbol μ_x instead of E(x) for the expected value of the random variable x.
• The variance of a random variable is denoted by Var(x) = σ_x².
  σ_x² = E[(x - E(x))²] = (x₁ - E(x))² p₁ + ... + (x_n - E(x))² p_n
• Many data analysts uses the symbol σ_x² instead of Var(x).
• The standard deviation of a random variable is denoted as σ_x. It is the square root of the variance:

        σ_x = √Var(x) = √(σ_x)

Practice Problems

4. Recall that the expected value for the Rainfall on a Tropical Island example is 1.1 inches. Here is the probability distribution:
   

   Rainfall	Probability
   0	0.3
   1	0.4
   2	0.2
   3	0.1

   
   Compute the variance and standard deviation of this random variable.
   Ans: (0 - 1.1)² 0.3 + (1 - 1.1)² 0.4 + (2 - 1.1)² 0.2 + (3 - 1.1)² 0.1
   = 1.21 * 0.3 + 0.01 * 0.4 + 0.81 * 0.2 + 3.61 * 0.1 = 0.89
   The standard deviation is √0.89 = 0.943
5. Compute the variance and standard deviation of a Bernoulli random variable.
   Ans: The expected value of a Bernoulli random variable is 0(1 - p) + 1p = p.

   Variance = (0 - p)² (1-p) + (1 - p)² p = p²(1-p) + (1 - 2p + p²)p
   = p² - p³ + p - 2p² + p³ = p - p² = p(1 - p)
   The standard deviation is the square root of the variance = √p(1 - p)

6. Use your result from Problem 2 to obtain the mean and standard deviation of the number of heads obtained in a single coin flip.
   Ans: E(x) = p = 0.5; σ_x = √p(1-p) = √0.5(1-0.5) = √0.25 = 0.5

Properties of Random Variables

1. E(cx) = c E(x)
2. E(x + y) = E(x) + E(y)
3. E(x₁ + ... + x_n) = E(x₁) + ... + E(x_n)
4. Var(x) = E(x²) - E(x)²
5. Definition:   x and y are independent if E(xy) = E(x)E(y).
   If x and y are independent, Var(x + y) = Var(x) + Var(y)

Here are the derivations.

Practice Problem

7. Compute the variance and SD of a Bernoulli random variable with the formula Var(x) = E(x²) - E(x)².
        Var(x) = E(x²) - E(x)² = (0² (1 - p) + ... + 1² p) - p²
                 = p - p² = p(1 - p)
   This is the same result that we obtained earlier.

Sums of Random Variables

• Define S = x₁ + ... + x_n, where each x_i has the same expected value and variance, and all of the x_i are independent.
• The expected value of the sum of n outcomes of the random variable x:
  E(S) = E(x₁ + ... + x_n) = E(x₁) + ... + E(x_n) = nE(x)
  

  E(S) = nE(x)

• The variance of the sum of n independent outcomes of the random variable x:
  Var(S) = Var(x₁ + ... + x_n) = Var(x₁) + ... + Var(x_n) = nVar(x)
  

  Var(S) = nVar(x)

• The standard deviation of the sum of n independent outcomes of a random variable:
  σ_S = σ_{x1 + ... + xn} = √[Var(x₁ + ... + x_n)]
      = √[Var(x₁) + ... + Var(x_n) = √nVar(x) = σ_x√n
  

  σS = σx√n

Practice Problems

7. Compute the expected value and standard deviation of the random variable in Practice Problem 1 for 365 days.

   Ans: Recall that E(x) = 1.1 and σ_x = 0.943; E(S) = nE(x) = 365 × 1.1 = 401.5
   σ_S = σ_x√n = 0.943√365 = 18.0

8. Compute the expected value and standard deviation of a Bernoulli random variable (Practice Problem 2) for the sum of n trials.
   Ans: E(S) = nE(x) = np
   σ_S = σ_x √n = √p(1-p) √n = √np(1-p)
9. Suppose that my true percentage of making a free throw is 70%. Out of 100 attempts, what is the expected value and standard deviation of the number of free throws made? Find a 95% confidence interval for the number of free throws made.
   Ans: E(S) = np = 100 × 0.7 = 70.
   Var(S) = np(1 - p) = 100 × 0.7 × (1 - 0.7) = 21.
   σ_S = √21 = 4.58.
   Answer: the 95% confidence interval for S is calculated like this:
   -2 ≤ z_S ≤ 2
   -2 ≤ (S - E(S)) / σ_S ≤ 2
   -2 ≤ (50 - 70) / 4.58 ≤ 2
   -2 * 4.58 ≤ S - 70 ≤ 2 * 4.58 + 9.06 + 70
   -9.06 + 70 ≤ S ≤ 9.06 + 70
   60.94 ≤ S ≤ 79.06.
   This means that the 95% confidence interfal for S is [60.94, 79.06].