To Notes

IT 223 -- Apr 1, 2024

IT 223 Course Website

• See these links on the left side of the Announcements Page:
  
  • Announcements, Prof Info, Lecture Notes, Projects, Documents, Datasets, Exercises, Exam Info, COL Website (submit projects, view grades), Course Syllabus.

Early History of Statistics

• Some early pioneers in statistics:
  

  1654	Blaise Pascal is the first to develop the theory of probability. He is known for Pascal's Triangle, and was the first known person to apply the laws of probability to gambling.
  1662	John Graunt writes Natural and Political Observations upon the Bills of Mortality. He was the first demographer to use statistics to estimate the population of London.
  1718	Abraham de Moivre writes Doctrine of Chances. He is the first to write down the formula for the normal histogram and the first to state the Central Limit Theorem, which states that if a random variable is a sum of many independent influences, then that random variable is approximately normally distributed.
  1722	Roger Cotes published Opera Miscellanea. It is a study of the theory of errors, particularly in astronomy.
  1809	Carl Friedrich Gauss discovers the Method of Least Squares, the standard method for fitting a regression line to data.
  1835	Adolphe Quetelet publishes a study of human measurements that describes "the average man" that nature is trying to produce. He shows that human measurements tend to follow normal histograms. He is the first to introduce the body mass index that is still used today.
  1888	Sir Francis Galton first introduced the concept of correlation, which describes the degree of linear association between two measurements, such as height and weight.
  1890	R. A. Fisher is born. Many call him the father of modern statistics. He popularized the use of the standard deviation to estimate the spread of a dataset, as well as many other statistical techniques.
  1977	John Tukey published the book Exploratory Data Analysis. It popularized many of the techniques we will study in this class. He also coined the terms "bit" and "software." He died in 2000.

Applications of Statistics in the 20th Century

• Reference: Freedman, Pisani, and Purves, Statistics, Fourth Edition, W. W. Norton and Company, 2007.
  Here are the first two chapters of the International Edition that discusses the following four topics:
  1. Experimental Trials for the Salk Polio Vaccine
  2. Forecasting the 1936 Presidential Election
  3. Statistical Experiments vs. Observational Studies
  4. Gender Bias in Berkeley Grad School Admissions

What Does a Data Analyst Do?

• Look at this overview:
  www.careerexplorer.com/careers/data-analyst/

Exploratory Data Analysis

• The term Exploratory Data Analysis or EDA was coined by John Tukey.
• Although EDA is usually associated with techniques like stemplots and boxplots, it is more accurately defined as a philosophy: understand the dataset before deciding which statistical tests to perform.
• Some terminology:
  
  • A univariate dataset is a dataset containing only one variable. For example these hypothetical exam scores form a univariate dataset:
    
    98  93  85  39  91  75  5  93  79  90 
  • A bivariate dataset is a dataset containing two variables. Some examples of bivariate datasets are weight vs. height, price vs. date, and score on exam vs. hours studied for it.
  • A multivariate dataset is a dataset containing two or more variables.
  • A nominal variable (also called a categorical variable) is a variable that contains non-numeric values, such as gender (female, male, non-binary), is-smoker (yes, no), football player position (offense, defense), coin-condition (Poor, Fine, Very-fine, Extremely-fine, Uncirculated, Brilliant-uncirculated).  A numerical grade between 1 and 70 now accompanies the categorical grades.
  • A scale variable (also called a quantitative variable or continuous variable) is a variable that can contain any value in a continuous range. Examples are: age, height, weight, gpa, price.
  • An ordinal variable is a variable that contains discrete numeric values. Examples are: college-year (1, 2, 3, 4), survey-answer (1, 2, 3, 4, 5).
  • To summarize the various value of a univariate dataset, tally marks are traditionally used. Here are the sorted hypothetical exam scores:
    
    5  39  75  79  85  90  91  93  93  98
  • Here is the tally diagram:
    
  • A more informative technique is the stemplot, also called the stem-and-leaf display.
    

    95 | 8
    90 | 0133
    85 | 5
    80 | 
    75 | 59
    70 |  
    65 |
    60 |
    55 |
    50 |
    45 |
    40 |
    35 | 9
    30 |
    25 |
    20 |
    15 |
    10 |
     5 | 5
     0 |
    

    The last digit of each observation is recorded to the right of the vertical line down the middle, which is the "stem."

Using R

• Use R to complete Project 1.

Project 2

• Look at Project 2a.
• Start thinking about what data you want to collect for Project 2.