## The Exponential Distribution

Please see a slightly expanded version here.

This is a derivation of the cumulative distribution function, characteristic function, moment generating function, first moment, expected value, second moment, and variance of the exponential distribution given its probability density function.

The probability density function of the exponential distribution is:

$f_X(x) = \lambda e^{-\lambda x}$

Thus the cumulative distribution function is:

$F_X(x \leq a) = \displaystyle\int_{0}^{a} f_X(x) dx = \int_0^a \lambda e^{-\lambda x} dx = -e^{-\lambda x} \Big |_0^a = 1 - e^{-\lambda a}$

And the characteristic function:

$\phi_X(u) = E[e^{i u X}] = \displaystyle\int_{0}^{\infty} e^{i u x} f_X(x) dx = \int_0^{\infty} e^{i u x} \lambda e^{- \lambda x} dx = \lambda \int_0^{\infty} e^{-(\lambda - iu) x} dx = - \frac{\lambda}{\lambda-iu} e^{-(\lambda-iu)x} \Big |_0^{\infty} = \frac{\lambda}{\lambda - iu}$

Now the moment generating function, which is obtained from the characteristic function:

$M_X(t) = \phi_X(-it) = \displaystyle\frac{\lambda}{\lambda - (i(-it))} = \frac{\lambda}{\lambda - t}$

Now the first moment:

$M_X'(t) = \displaystyle\frac{d}{dt} \lambda(\lambda-t)^{-1} = -\lambda (\lambda -t)^{-2} \frac{d}{dt} (\lambda - t) = \frac{\lambda}{(\lambda - t)^2}$

Thus:

$E[X] = M_X'(0) = \displaystyle\frac{1}{\lambda}$

And the second moment:

$M_X''(t) = \displaystyle\frac{d}{dt} M_X'(t) = \frac{d}{dt} \lambda (\lambda - t){-2} = -2 \lambda ( \lambda - t)^{-3} \frac{d}{dt} (\lambda - t) = \frac{2 \lambda}{(\lambda - t)^3}$

So:

$E[X^2] = M_X''(0) = \displaystyle\frac{2}{\lambda^2}$

And finally:

$\displaystyle Var(X) = E[X^2] - E[X]^2 = \frac{2}{\lambda^2} - \Big(\frac{1}{\lambda}\Big)^2 = \frac{1}{\lambda^2}$

Of course the expected value and the variance can be computed by evaluating:

$\displaystyle \int_0^\infty x \lambda e^{-\lambda x} dx$

and

$\displaystyle \int_0^\infty x^2 \lambda e^{-\lambda x} dx$

respectively.

Next we’ll see how the exponential distribution is memoryless, that is:

$\displaystyle P(X > t+s|X>t) = P(X > s)$

First apply Bayes’ Rule:

 $\displaystyle P(X>t +s|X>t)$ $=$ $\displaystyle \frac{P(X>t|X>t+s)P(X>t+s)}{P(X>t)}$ $=$ $\displaystyle \frac{1*P(X>t+s)}{P(X>t)}$ $=$ $\displaystyle \frac{1-(1-e^{-\lambda (t+s)})}{1-(1-e^{-\lambda t})}$ $=$ $\displaystyle \frac{e^{-\lambda (t+s)}}{e^{-\lambda t}}$ $=$ $\displaystyle e^{-\lambda s}$ $=$ $P(X>s)$

Next, Let $X_1, X_2$ be exponential random variables with parameters $\lambda_1$ and $\lambda_2$ respectively, then:

 $\displaystyle P(X_1 < X_2)$ $=$ $\displaystyle \int_0^\infty P(X_1 $=$ $\displaystyle \int_0^\infty P(X_1 $=$ $\displaystyle \int_0^\infty P(x $=$ $\displaystyle \int_0^\infty P(X_2 \geq x)\lambda_1 e^{-\lambda_1 x} dx$ $=$ $\displaystyle \int_0^\infty e^{-\lambda_2 x} \lambda_1 e^{-\lambda_1 x} dx$ $=$ $\displaystyle \lambda_1 \int_0^\infty e^{-(\lambda_1 + \lambda_2)x} dx$ $=$ $\displaystyle \lambda_1 \Big( -\frac{1}{\lambda_1+\lambda_2} e^{-(\lambda_1 + \lambda_2)x} \Big) \Big|_0^\infty$ $=$ $\displaystyle \frac{\lambda_1}{\lambda_1+\lambda_2}$

References:

Ross, Sheldon M. Introduction to Probability Models, 9th edition. Academic Press. 2007.
Bremaud, Pierre. An Introduction to Probabilistic Modeling, 3rd printing. Springer. 1997.