To Notes

IT 223 -- May 13, 2024

Review Exercises

1. What is the regression fallacy?
   Ans: In a pretest/posttest situation, the regression fallacy is the mistaken notion that if someone does well on the pretest, he or she should do equally well on the posttest; if that person does poorly on the pretest, he or she should do equally poorly on the posttest. In fact, unless the pretest and posttest scores are perfectly correlated (r = 1), someone that obtains a pretest score k SD_xs above x, will, on the average, obtain a posttest score r × k SD_ys above the average. In other words, the posttest score will be lower than the pretest score, on the average. The situation is the opposite if the person obtains a below average score on the pretest. If the pretest score is k SD_xs below x, then, on the average, the posttest score will be r × k SD_ys below the average. In other words, the posttest score will be higher than the pretest score, on the average.
2. Use this R script to analyze the regression model for predicting weights of Chicago Bears players from their heights: bears.R. Place this script and the data file bears-2024-roster.txt into the folder c:/it223/bears. Delete the top comment line from the data file. Then enter these lines in R to run the bears.R script:
   

   > # Set working directory.
   > setwd("c:/it223/bears"
   >
   > # Run the script.
   > source("bears.R")
   

3. What is the RMSE for a regression model?
   Ans: The root mean squared error (RMSE is the standard deviation of the residuals. In particular, RMSE is the SD of the residuals in a thin rectangle that contains a specific x value. RMSE is defined as SD+_y √(1 - r²).
4. A law school finds this relationship between LSAT scores (independent variable x) and first-year scores (dependent variable y). The data are bivariate normal. Here are the summary variables:
          x = 162    SD_x = 6
          y = 68    SD_y = 10
          r = 0.6
   
   1. About what percentage of the students have first-year scores over 75? Because the data are bivariate normal, the first-year scores are normally distributed, so we can use the normal table.
      Answer: z = (y - y) / SD_y = (75 - 68) / 10 = 0.7. The area of the bin (-∞, 0.7] is 0.7580 = 76%. Therefore the percentage of students with first year score over 75 is 100% - 76% = 24%.
   2. Of those students who scored 165 on the LSAT, about what percentage have first-year scores over 75?  Visualize these scores as lying in a thin vertical rectangle centered at LSAT = 165. The observations in the thin vertical rectangle centered over x=165 are normally distributed.
      Answer: The regression equation is
            y - 68 = (0.6 * 10 / 6) (x - 162)
            y - 68 = 1 (x - 162)
            y = x - 94
      so the predicted value for the students in the thin vertical rectangle centered at x = 165 is y = x - 94 = 165 - 94 = 71.

      The RMSE for those students in the thin vertical rectangle centered at x = 165 is
            RMSE = √1 - r² SD_y = √1 - 0.6² * 10 = 0.8 * 10 = 8
      Then z = (y - y^{^}) / RMSE = (75 - 71) / 8 = 0.5. The area of the bin (-∞, 0.5] is 0.6915 = 69%. Therefore of the students having LSAT score equal to 165, the percentage of students having first year score over 75 is 100% - 69% = 31%.

Learning Outcomes for this Week

• Know the definitions of these terms:
  Sample Space, A Priori Probability, Empirical, Subjective Probability, Probability Distribution, Random Variable, Expected Value, Risk.
• Be able to:
  
  1. Define a probability distribution in three ways: a priori, empirical and subjective.
     
  2. Define a probability distribution, given a sample space.
  3. Compute the expected value of a random variable.
  4. Understand the difference between independent and mutually exclusive events.
  5. Use the multiplication rule for independent events.
  6. Use the addition rule for mutually exclusive events.
  7. Find the expected value of a sum of random variables.
  8. Use n factorial (n!) to find the number or ways to arrange n objects.

Probability

• The sample space is the set of all possible outcomes in a probability situation. We use the greek letter Ω (big omega) to represent the sample space.
• Example 1:   Flip a coin once: Ω = {head, tail} = {H, T}.
• Example 2:   Roll a die once: Ω = {1, 2, 3, 4, 5, 6}.
• Example 3:   An urn contains 3 red balls and 4 green balls. Choose a random ball from the urn: Ω = {red₁, red₂, red₃, green₁, green₂, green₃, green₄}. The subscripts are used to distinguish between the balls of the same color.
• Example 4:   Choose a random part on an assembly line and test it: Ω = {defective, nondefective}.
• An event is a subset of the sample space. The event A occurs if the outcome of a probability experiment is a member of the event subspace A.
  Example 5: Ω = {HHH, THH, HTH, HHT, TTH, THT, HTT, TTT}. Let A be the event that 2 heads are obtained out of 3 coin tosses. In other words, A = {THH, HTH, HHT}.
• The probability that an event occurs is a value between 0 and 1 that indicates how likely that event is to occur; 0 means that the event is impossible, 1 means that the event is certain to occur.
• Three common ways of determining probabilities:
  
  1. Theoretical or A Priori Probability
     Assume that all outcomes are equally likely. Suppose you are interested in the probability that the event A occurs.
     P(A) = # ways A can occur / Total # of events
     Example 6:   What is the probability of getting one head out of two coin flips?
     Sample space Ω = {HH, HT, TH, TT}
     A = Event of getting exactly one head = {HT, TH}
     P(A) = # of events in A / # events in Ω = 2 / 4 = 0.5
  2. Empirical Probability
     Perform an experiment, repeated a large number of times. Count the number of times s the event A occurs and the total number of times n the experiment is repeated.
     P(A) = s / n
     Example 7: Flip a coin 1 million times. 499,523 heads were obtained. The probability of obtaining a head is 499,523 / 1,000,000 = 0.499523.
  3. Subjective Probability
     Each person uses personal intuition or judgement.
     What is the probability that I will need an umbrella today?
     Las Vegas oddsmakers combine subjective probabilities for many sources when determining betting odds.
     Example 8:   What is the probability that the Bears will win the Super Bowl in 2024?
     A subjective probability can be defined as a fair bet, that is a bet for which you would be willing to take either side.
• No matter how probabilities are chosen, a probability distribution is a table that assigns a probability to each outcome in the sample space:
  

  Probability Distribution
  Outcome	Probability
  HH	0.25
  TH	0.25
  HT	0.25
  TT	0.25

  
  When flipping 2 coins, what is the probability of obtaining 1 head?
  Ans: There are two ways in the sample space of obtaining one head: HT, TH. Therefore the probability of obtaining one head is 0.25 + 0.25 = 0.50.
• Rules for Probabilities:
  
  1. Each probability is between 0 and 1, inclusive:
          If s ∈ Ω, then 0 ≤ P(s) ≤ 1.
  2. The probability that some event in the sample space occurs is 1:
          P(Ω) = 1
  3. If p is the probability that the event occurs, then 1 - p is the probability that the event does not occur.
          P(A^c) = 1 - P(A)

Practice Problems

1. What is wrong with this argument? Either the Bears will win the Super Bowl in 2024 or they won't. Therefore the probability that the Cubs will win the World Series is 50%.
   Ans: Just because there are two outcomes doesn't mean they are equally likely. Some outcomes are very likely; they have probabilities close to 1. Some outcomes are unlikely; they have outcomes close to 0. Other outcomes have probabilities close to 0.5. In this case it does not make sense to use the a priori probabilities of 0.5 for winning the superbowl or not winning it in 2024.
2. What is wrong with this strategy? Double down after after each loss. Eventually you win and recoup your losses. For example:
   -1 - 2 - 4 - 8 - 16 + 32 = 1.
   Now start over with 1 and repeat the double down strategy.
   Ans: The problem with this strategy is that, eventually, you will either reach the casino betting limit or you will run out of money.
3. A bookmaker offers 20 to 1 odds that the Bears will win the Super Bowl in 2024. If this is a fair bet, what is the probability p that the Cubs will win the World Series?
   Ans: The expected amount that you win is 20p + (-1)(1 - p); this expression is 0 because we are assuming that is a fair bet. Now solve for p:
        20p + (-1)(1 - p) = 0
        20p - 1 + p = 0
        21p = 1
        p = 1 / 21 = 0.0476 = 4.8%.

Random Variables

• A random variable is a function from the sample space to the set of real numbers.
  
  Example 9:
  

  Outcome	Real Number	Probability
  HH	2	0.25
  TH	1	0.25
  HT	1	0.25
  TT	0	0.25

  
  This table defines a function that assigns HH → 2, TH → 1, HT → 2, TT → 0.
  
  A random variable can also be described directly with a probability distribution:
  

  Outcome	Probability
  x1	P(x1)
  x2	P(x2)
  ...	...
  xk	P(xk)

  
  In the case of a random variable that counts the number of heads obtained by flipping two coins:
  

  x	Probability
  0	0.25
  1	0.50
  2	0.25

  
  
• Example 10:   The random variable x is the number of heads obtained out of three coin flips.
  

  x	Probability
  0	0.125
  1	0.375
  2	0.375
  3	0.125

  
  
• Example 11:   Pick a person at random. That person's height is a random variable.
• Example 12:   In the case of a continuous random variable, where P(z) = 0, for every specific z value, we must use a probability density instead of a probability distribution. Here is the familiar probability density for a standard normal random variable.
  
  
  If z is a normal random variable with mean 0 and variance 1 (z ∼ N(0, 1)), what is the probability that z ∈ [-0.7, 1.3]? Ans: 0.661 = 66%.
• A random variable that has only two outcomes 0 and 1 is called a Bernoulli random variable; 1 denotes success and 0 denotes failure. Here is the probability table for a Bernoulli random variable:
  

  x	Probability
  0	1 - p
  1	p

  
  
• Some examples of Bernoulli random variables: flipping a coin (T = 0, H = 1), rolling an ace with a single die (non-ace = 0, ace), shooting a basketball free throw (failure = 0, success = 1), choosing a part from an assembly line to inspect (non-defective = 0, defective = 1).