To Documents

Properties of Random Variables

This document uses summation notation:
      Σⁿ_i=1x_i = x₁ + ... + x_n

• Property 1   If c is a constant and x is a random variable, E(cx) = cE(x).
  
  Derivation: E(cx) = Σⁿ_i=1 (cx_i) P(x = x_i) = c Σⁿ_i=1 x_i P(x = x_i) = c E(x)
• Property 2   If c is a constant, E(c) = c.
  Derivation: E(cx) = Σⁿ_i=1 c P(x = x_i) = c Σⁿ_i=1 P(x = x_i) = c 1 = c
• Property 3   If x and y are random variables, E(x + y) = E(x) + E(y).
  Derivation: E(x + y) = Σⁿ_i=1 Σ^m_j=1 (x_i + y_j) P(x = x_i and y = y_j)

      = Σⁿ_i=1 Σ^m_j=1 [x_i P(x = x_i and y = y_j) + y_j P(x = x_i and y = y_j)]

      = Σⁿ_i=1 Σ^m_j=1 x_i P(x = x_i and y = y_j) + Σⁿ_i=1 Σ^m_j=1 y_j P(x = x_i and y = y_j)

      = Σⁿ_i=1 x_i Σ^m_j=1 P(x = x_i and y = y_j) + Σ^m_j=1 y_j Σⁿ_i=1 P(x = x_i and y = y_j)

      = Σⁿ_i=1 x_i P(x = x_i) + Σ^m_j=1 y_j P(y = y_j) = E(x) + E(y)

• Property 4:   If x_i, ... , x_n are random variables, E(x₁ + ... + x_n) = E(x₁) + ... + E(x_n).
  Derivation: The derivation is for n = 4. Distribute E over the expression one term at a time.

  E(x₁ + x₂ + x₃ + x₄) = E(x₁) + E(x₂ + x₃ + x₄)
      = E(x₁) + E(x₂) + E(x₃ + x₄) = E(x₁) + E(x₂) + E(x₃) + E(x₄)

• Property 5:   Var(x) = E(x²) - E(x)²
  Derivation: Var(x) = E[((x - E(x))²] = E[x² + 2 x E(x) - E(x)²]
      = E(x²) - E[2x E(x)] + E[E(x)²] = E(x²) - 2E(x)E(x) + E(x)² = E(x²) - E(x)²
• Definition:   Two random variables x and y are independent if E(xy) = E(x)E(y).
• Property 6:   If x and y are independent, then Var(x + y) = Var(x) + Var(y).
  Derivation: Var(x + y) = E[(x + y)²] - E(x + y)²
      = E(x² + 2xy + y²) - [E(x) + E(y)]²
      = E(x²) + 2E(xy) + E(y₂) - E(x)² - 2E(x)E(y) - E(y)²
      = E(x²) + 2E(x)E(y) + E(y²) - E(x)² - 2E(x)E(y) - E(y)²
      = E(x²) + E(y²) - E(x)² - E(y)²
      = E(x²) - E(x)² + E(y²) - E(y)²
      = Var(x) + Var(y)