May 29, 2024

To Notes

IT 223 -- May 29, 2024

Review Exercises

The probability that a part on an assembly line is defective is 5%. What is the probability that
1. exactly 3 parts are defective out of 15. Use R to check your answers. Answer:
  P(k successes out of n) = nCk p^k (1-p)^n-k
  = 15C3 * 0.05³ * (1-0.05)^15-3
  = 15!/(3!*12!) * 0.05³ * 0.95¹² = 0.03073298 = 3%.
  This probability is computed using R like this:
```
> dbinom(3, 15, 0.05)
[1] 0.03073298
```
2. exactly 2 parts are defective out of 15. Answer:
  P(k successes out of n) = nCk p^k (1-p)^n-k
  = 15C2 * 0.05² * (1-0.05)^15-2
  = 15!/(2!*13!) * 0.05² * 0.95¹³ = 0.1347523 = 13%.
  This probability is computed using R like this:
```
> dbinom(2, 15, 0.05)
[1] 0.1347523
```
3. exactly 1 part is defective out of 15. Answer: P(k successes out of n) = nCk p^k (1-p)^n-k
  = 15C1 * 0.05¹ * (1-0.05)^15-1
  = 15!/(1!*14!) * 0.05¹ * 0.95¹⁴ = 0.3657562 = 37%.
  This probability is computed using R like this:
```
> dbinom(1, 15, 0.05)
[1] 0.3657562
```
4. no parts are defective out of 15. Answer: P(k successes out of n) = nCk p^k (1-p)^n-k
  = 15C0 * 0.05⁰ * (1-0.05)^15-0
  = 15!/(0!*15!) * 0.05⁰ * 0.95¹⁵ = 0.4632912 = 46%.
  This probability is computed using R like this:
```
> dbinom(0, 15, 0.05)
[1] 0.4632912
```
5. 3 or fewer parts are defective. Answer:
  P(3 or fewer defective parts
  0.03073298 + 0.1347523 + 0.3657562 + 0.4632912 = 0.9945327 = 99%.
  To verify this answer with R:
```
> # The following R statement computes the
> # probability of 3 or fewer sucesses out of 15
> # when the probability of success is 0.05.
> pbinom(3, 15, 0.05)
[1] 0.9945327
```
State the Law of Large Numbers (LLN). Who first proved it?
Answer: Jakob Bernoulli.
State the Central Limit Theorem (CLT). Who first proved it?
Answer: Abraham de Moivre proved the CLT in the context of the binomial distribution.
Alexandr Lyapunov proved the modern form of the CLT.

The Average of Random Variables

The expected value of the average of n outcomes of a random variable:
E(x) = E[(x₁ + ... + x_n) / n] = E(x₁ / n + ... + x_n / n)
= E(x₁ / n) + ... + E(x_n / n) = E(x₁) / n + ... + E(x_n) / n
= nE(x) / n = E(x)

E(x) = E(x)
The variance of the average of n independent outcomes of a random variable:
Var(x) = Var((x₁ + ... + x_n) / n) = Var(x₁ / n + ... + x_n / n)
= Var(x₁ / n) + ... + Var(x_n / n) = Var(x₁) / n² + ... + Var(x_n) / n²
= nVar(x) / n² = Var(X) / n

Var(x) = Var(X) / n
The standard error of the average of n independent outcomes of a random variable:
σ_x = √[Var(x)] = √[Var(X) / n] = σ_x / √n

σ_x = σ_x / √n
Practice Problem: a coin is flipped 1,000 times, 510 heads are obtained. Find a 95% confidence interval for the true probability of obtaining a head. Compute this confidence interval using S = the total number of heads obtained, and X = the average number of heads obtained per flip. You should obtain the same answer using either method.

Answer: Method 1 is to compute the confidence interval for the true value of p using S:
n = 1000 S = 510 E(S) = np = 1000*p
p^{^} = S / n = 510 / 1000 = 0.51
σ_S = √n * p * (1-p) = √1000 * 0.5 * (1-0.5) = 15.81139
z_S = (S - E(S)) / σ_S = (510 - 1000*p) / 15.81.
Now -2 ≤ Z_S ≤ 2 95% of the time. Solve for p:
-2 ≤ (510 - 1000*p) / 15.81 ≤ 2
-2 * 15.81 ≤ 510 - 1000 * p ≤ 2 * 15.81
-31.62 - 510 ≤ -1000 * p ≤ 31.62 - 510
-541.62 ≤ -1000 * p ≤ -478.38
-541.62 / (-1000) ≤ p ≤ -478.38 / (-1000)
0.54162 ≥ p ≥ 0.478.38
0.478.38 ≤ p ≤ 0.54162.
The 95% confidence interval for p is [0.47838, 0.54162] = [48%, 54%].
It looks like the coin is a fair coin.

Method 2 is to compute the 95% confidence interval for the true value of p using the average X of the 1,000 outcomes instead of S.
n = 1000 X = 510 / 1000 E(X) = p
p^{^} = X = 510 / 1000 = 0.51
σ_S = √(p * (1-p)) / n = √0.5 * (1-0.5) / 1000 = 0.01581139
z_S = (S - E(S)) / σ_S = (0.51 - p) / 0.01581.
Now -2 ≤ Z_S ≤ 2 95% of the time. Solve for p:
-2 ≤ (0.510 - p) / 0.01581 ≤ 2
-2 * 0.01581 ≤ 0.510 - p ≤ 2 * 0.01581
-0.03162 - 0.510 ≤ -p ≤ 0.03162 - 0.510
-0.54162 ≤ -p ≤ -0.47838
-0.54162 ≤ -p ≤ -0.47838
0.54162 ≥ p ≥ 0.47838
0.478.38 ≤ p ≤ 0.54162.
The 95% confidence interval for p is [0.47838, 0.54162] = [48%, 54%].

Normal Approximation for the Binomial Distribution.

Practice Problems:
1. Flip a fair coin 10,000 times. What is the probability of obtaining exactly 5,000 heads.
  Answer: use the normal approximation. Find the probability that S is between 4999.5 and 5000.5. The standard error of S is
        σ_S = √n*p*(1-p) = √10000*0.5*(1-0.5) 50
  The z scores are
        z_left = (4999.5 - 5000.0) / 50 = -0.001
  and
        z_right = (5000.5 - 5000.0) / 50 = 0.001
        P(z_left ≤ z ≤ z_right) = P(z ≤ z_right) - P(z ≤ z_left) = 0.007978713
  This probability can be computed directly using dbinom:
```
> dbinom(5000, 10000, 0.5)
[1] 0.007978646
```
  The two values 0.007978713 and 0.007978646 are not exactly the same, but the normal approximation is fairly good because n is large at 10,000.

Tests of Hypotheses for Proportions

A test of hypothesis can be used to determine if a coin or die is fair.
For testing if a coin is fair, we test the null hypothesis
H₀: p = 0.5
against the alternative hypothesis
H₁: p ≠ 0.5.
Here are the four steps for testing if the probability of success p for a Bernoulli random variable is equal to p₀:
1. Write down the null and alternative hypothesis:
  H₀: p = p₀
  H₁: p ≠ p₀.
2. Write down the test statistic, assuming the null hypothesis:
  z = (S - E(S)) / σ_S
3. Write down a 95% confidence interval for z:
  I = [-1.96, 1.96]
4. If z ∉ I, reject H₀; if z ∈ I, accept H₁.
Example 1: Flip a coin 100 times and obtain 43 heads. Is the coin fair?
1. H₀: p = 0.5; H₁: p ≠ 0.5
2. S = 43, E(S) = np, and σ_S = √[np(1-p)] = √[100(0.5)(1-0.5)] = 5.0, so
  z = (S - E(S)) / σ_S = (43 - 100(0.5)) / 5.0 = -1.4
3. A 95% confidence interval for z is [-2, 2].
4. -1.4 ∈ [-2, 2], so accept the null hypothesis. Not enough evidence to conclude that the coin is not fair.
Example 2: To test if a four-faced die (shape of a tetrahedron) is fair, such a die is rolled 1600 times. 421 aces are obtained. Perform a test of hypothesis to test whether the die is fair.
Here are the steps to perform the test of hypothesis:
1. Write down the null hypothesis: H₀: p = 1/4
2. Compute the text statistic: z = (S - E(S)) / SE_S
3. Write down a 95% confidence interval: I = (-1.96, 1.96)
4. If z ∈ I, accept the null hypothesis that the die is fair. If z ∉ I reject the null hypothesis.
Answer:
1. H₀: p = 1/4.
2. S = 421 n = 1,600 SE(S) = √1,600 * (1/4) * (1-1/4) = 17.32
  z = (S - np) / SE(S) = (421 - 1,600 (1/4)) / 17.32 = 1.212
3. The 95% confidence interval using the standard normal table is (-1.96, 1.96)
4. 1.212 ∈ (-1.96, 1.96), so accept the null hypothesis that the die is fair; we don't have enough evidence to reject the null hypothesis.
Example 3: Use R to simulate this situation: use the R function call rbinom(1, 500, 0.5) to simulate 500 outcomes of a Bernouilli random variable with p = 0.5. Test whether the random number generator is fair for the given probability.

Tests of Hypothesis in General

The fundamental question that a researcher conducting a test of hypothesis is trying to answer is: Is the result significant, or is it merely due to chance variation?
Flip a coin 10,000 times and obtain 5,038 heads. Is my coin biased, or is it just chance variation?
To reduce the bias, reduce the effect of lurking variables:
If the lurking variable is known and can be controlled:
-- Insure that the effect of the lurking variable is the same on all subjects.
If the lurking variable is known, but cannot be controlled:
      -- Include the lurking variable as an independent variable in the model.
      -- Include the lurking variable as a panel variable.
      -- Only include observations with the same value of the lurking variable.
If the lurking variable is unknown:
-- Randomize the assignment of treatments to subjects to ensure that the effect of lurking variables is equally likely for all subjects.
-- If possible, make the treatments double blind.
The null hypotheses, denoted by H₀, states that the treatment or effect under investigation does not make a difference; the effect is merely due to chance.
Some sample research questions phrased as null hypotheses:
1. The number of heads obtained with a coin being investigated is not significantly different than a fair coin.
2. There is no real difference in the autism rates between children that receive the vaccine and those that do not receive it.
3. The electric and magnetic fields caused by high voltage power lines does not cause significant health risks to those living nearby.
4. Eating irradiated food does not cause significant health risks.
5. There is no real difference in reading scores between the students that use the new reading curriculum and those that do not.
6. A specific die is fair, which means that the probability of any face coming up when rolled is 1/6.
7. There is no real difference in network traffic speed between the old router and the new router.
8. The new tax law is essentially revenue neutral.
The steps for a test of hypothesis. The test is a α%-level z-test:
1. State the null (H₀) and alternative hypotheses (H₁).
2. Compute the test statistic T, assuming the null hypothesis is true.
3. Write down a (1-α)% confidence interval for the test statistic I.
4. If T ∈ I, accept the null hypothesis; if T ∉ I, reject the null hypothesis.
5. (If possible) determine the p-value of the test statistic.
Note: for Project 4, the z-tests are 5%-level tests.
The p-value is the probability of obtaining a test statistic at least as extreme at the test statistic actually obtained, given that the null hypothesis is true.

The z-test

Example 4: To test whether a new tax bill is revenue neutral, test the new tax rules on a random sample 100 tax returns out of 100,000 tax returns on file. For each tax return, compute
x_i = tax under new rules - tax under old rules
Then compute x = -$219 and SD = $725. Let μ be the true change in tax revenue from the old rules to the new rules.
The test of hypothesis:
1. State the null and alternative hypotheses:
  H₀: μ = 0
  H₁: μ ≠ 0
2. Compute the test statistic, assuming H₀ is true:
  z = (x - μ) / SE_ave = (-219 - 0) / (725 / √100) = -3.02
3. Write down a 95% confidence interval for the test statistic, assuming H₀: [-1.96, 1.96].
4. z = -3.02 ∉ (-1.96, 1.96), so reject H₀ and accept H₁; the difference is real and not merely due to chance.
5. Compute the p-value, which is the probability that z is as extreme or more extreme the z which is actually obtained. In our case z = -3.02, and we want to find the probability in the tails. Now the area under the normal curve for the interval (∞, -3.02) is 0.0013. Thus the area in both tails is 2 × 0.0013 = 0.0026.
The general form of the z-score for a z-test is
z = (T - E(T)) / SE_T
Two examples are
z = (S - E(S)) / SE_S and z = (x - E(x)) / SE_x
When performing a z-test for p, the test statistic is z = (S - np) / SE_S, where S is the number of successes from the Binomial Distribution (sum of n Bernouilli random numbers), and p is the true probability of success.
When performing a z-test for μ, from the ideal measurement model, the test statistic is
z = (x - μ) / SE_ave,
where μ is the mu of the null hypothesis.
Only use a z-test when the sample size n is greater than or equal to 30. This insures that, in the case of the ideal measurement model x_i = μ + e_i, the test statistic
z = (x-μ) / SE_ave
is approximately normally distributed according to the CLT, and
SD⁺ in SE_ave = SD⁺ / √n is close to the true standard deviation σ of the population.
In the case of a z-test for a probability p, n ≥ 30 insures that
the test statistic S is approximately normally distributed (thanks to the Central Limit Theorem), and
p^ = S/n in SE_S = √np(1-p) is close to the true value of p.

More about p-values

In the old days (50 or more years ago) statisticians were content to know whether H₀ was accepted or rejected. Now they want to know the p-value, which gives more information.
Recall that the p-value is the probability of obtaining a test statistic as extreme or more extreme that the value actually obtained, given that the null hypothesis is true.
If p is close to zero, the evidence is overwhelming that the result was not due to chance.
If p is slightly less than 0.05, H₀ was just barely rejected. The evidence is borderline as to whether the result is due to chance; more research is needed.
If p is slightly more than 0.05, H₀ was just barely accepted. The evidence is borderline as to whether the result is due to chance; more research is required.
Usually a researcher wants to reject H₀ to prove that the treatment that he or she is investigating is real, not just chance variation.
If H₀ is accepted, it does not necessarily mean that we believe that H₀ is true, it means that there is not enough evidence to reject it.
At the risk of complicating things, our formulation of the z-test is phrased as a two-tailed test, where
H₀: μ = c H₁ ≠ c For a 5% level test (95% confidence), this means that we reject H₀ when the test statistic is not in the confidence interval I = [-1.96, 1.96].
It is also possible to phrase the z-test (and the t-tests we will discuss later) as one tailed tests, for which the null and alternative hypotheses are
H₀: μ = c H₁: μ > c
or
H₀: μ = c H₁: μ < c
In the case of H₁: μ > c, we reject H₀ when the test statistic is not in the interval I = (-∞, 1.645].
Some researchers think that the one-tailed test is an improvement over the two-tailed test because the test of hypothesis is more precise.
However, other researchers think that the one-tailed test is cheating, because it makes rejecting the null hypothesis easier.

Practice Problems

Example 5. In 1999, it was reported that the mean serum cholesterol level for female undergraduates was 168 mg/dl. A recent study at Baylor university collected the following data for cholesterol levels for females:
x = 173.7 SD+ = 27
Is there a real difference between the women in the Baylor study and the reported value in 1999? Perform the test at the 10%-level.
1. H₀: 168 H₁: μ ≠ 168
2. z = (x - μ) / SE_ave = (173.7 - 168) / (27 / √27) = 1.78
3. A 90% confidence interval for z is [-1.64,1.64].
4. 1.78 ∉ [-1.64,1.64], so reject the null hypothesis.
5. Find the area corresponding to the bin [-1.78,1.78]: 2 × 0.0375 = 0.0750.
Claim: if all high school seniors in California took the SAT test, the mean score would be equal to 450. To test this claim, select a random sample of 400 high school seniors and give them the test. Here are the data:
n = 400 x = 461 SD+ = 100
Is this result for the sample significantly different from 450 or is it just chance variation? Perform the test at the 99%-level.
Ans: Here are the steps of the z-test:
1. H₀: μ = 450 H₁: μ ≠ 450
2. z = (x - μ) / SE_ave = (461 - 450) / (100 / √400) = 2.2.
3. A 99% confidence interval for z is [-2.58,2.58].
4. 2.2 ∉ [-2.58, 2.58], so reject the null hypothesis.
5. The p-value is the probability of obtaining a z-value as extreme or more extreme than the one actually obtained. Find the area corresponding to the bin [-2.2,2.2]: 2 × 0.0139 = 0.0278.

The t-test

Use the t-test for μ even when n < 30 and the data are fairly normally distributed.
Construct a normal plot to check if the data are close to normal whenever a t-test is performed.
Because n is small, SD might not be a good approximation of σ This increases the variability, which increases the size of the confidence interval. The t-table is used instead of the z-table to account for the extra variability.
Example 6: A technician makes five measurements of the concentration of carbon monoxide (CO):
78 83 68 72 88
The descriptive statistics are:
n = 5 x = 77.8 SD⁺ = 8.07
Is the average of these concentrations significantly different than 70 or is it just chance variation?
There are two important differences between the t-test and the z-test:
1. The t-table is used instead of the z-table.
2. The p-value is hard to compute so we let R compute it.
The form of the t-statistic is exactly the same as the z-statistic. The only difference is that since n < 30, we can no longer guarentee that t is normally distributed, so we use the t-table instead of the standard normal table.
The tails of the t-density are fatter than the tails of a normal density.
Here is the t-test for Example 2.
1. State the null and alternative hypotheses:
  H₀: μ = 70
  H₁: μ ≠ 70
2. Compute the test statistic:
  t = (x - μ) / (SE / √n) = (77.8 - 70) / (8.07 / √5) = 2.16
3. Look up a 95% confidence interval using the t-table with n - 1 = 4 degrees of freedom. Use the upper-tail probility of 0.025 to obtain a two sided confidence interval of 95% to obtain 2.776.
4. Decide whether to accept or reject the null hypothesis:
  2.16 ∈ [-2.776, 2.776], so accept H₀.
To perform this test using R, create a dataset with variable x containing 78, 83, 68, 72, 88. Move x into the Test Variables box. Set the test variable to μ = 70, click Options and set the confidence level to 95%. Click OK.
A p-value of 0.097 is obtained, which means accept the null hypotheses, since p > 0.05.

Degrees of Freedom

Degrees of Freedom (df) is a technical term that arises when using the t-test. We are using x to estimate μ in SD+. If we are computing the SD+, when we are computing the square of the deviations, once we know the first n - 1 deviations, we automatically know the nth deviation because the sum of the deviations is always zero. n - 1 is called the degrees of freedom because only n - 1 of the deviations are able to vary freely.
The degrees of freedom for the t-test is related to the n - 1 that is used in the denominator of SD+.
Taking df = n - 1 compensates for the additional variation introduced because the true mean μ is unknown and x is used to estimate it.

The Paired Sample t-test

Goal: to test whether there is a significant difference between subjects from two different groups.
Typically, one group is the treatment group and the other group is the control group.
To use the paired sample t-test, each subject in one group is matched with a subject in the other group.
Then compute the differences in the response variable and perform a one-sample t-test on the differences.

Example 7: To test whether a new type of shoe sole material (type B) is better than the old type (type A), manufacture 10 pair of shoes where one shoe is made of type A and the other of type B. Randomly assign the type of material to left or right. Here is the data:

SoleMaterialA	SoleMaterialB
13.2	14.0
8.2	8.8
10.9	11.2
14.3	14.2
10.7	11.8
6.6	6.4
9.5	9.8
10.8	11.3
8.8	9.3
13.3	13.6

Perform the paired-sample t-test to see if there is a real difference between the two sole materials, or if it is just chance variation.
Into R, import the Excel file t-test2.xlsx. Use the Paired Sample t-test Sheet.
Use R to perform the
1. paired-sample t-test.
2. Here are the five steps of the two-sample t-test:
  1. Write down the null and alternative hypothesis:
    H₀: SoleMaterial A = SoleMaterial B
    H₁: SoleMaterial A ≠ SoleMaterial B
3. Obtain the test statistic from R: t = -3.349
4. Using the t-table, obtain a 95% confidence interval with n - 1 = 10 - 1 = 9 degrees of freedom:
  I = [-2.26, 2.26]
5. t ∉ I so reject H₀.
6. Find the p-value from the R output: p = 0.009.
The test statistic for the two-sample t-test obtained by computing the differences
diff = SolematerialA - SoleMaterialB,
then use R to perform a one-sample t-test on the variable diff.
Here are the five steps of the one-sample t-test performed with the diff variable:
1. Write down the null and alternative hypothesis:
  H₀: diff = 0
  H₁: diff ≠ 0
2. Obtain the test statistic from R: t = -3.349
3. Using the t-table, obtain a 95% confidence interval with n - 1 = 10 - 1 = 9 degrees of freedom:
  I = [-2.26, 2.26]
4. t ∉ I so reject H₀.
5. Find the p-value from the R output:
  p = 0.009.
Notice that the test statistic t and the p-value are exactly the same whether a paired two-sample t-test is performed or whether a one-sample t-test on the differences is performed.