Probability Distribution: Laplace

The Question It Answers

What symmetric model concentrates more sharply at the center yet produces more moderate-to-large deviations than a Gaussian? The Laplace (or double-exponential) distribution answers this: it looks like two Exponential curves placed back-to-back, giving a sharp tent-like peak and tails that are heavier than the Normal’s. Those tails still decay exponentially, so Laplace is not “heavy-tailed” in the power-law sense; it is simply heavier than Gaussian.

Field	Content
Type	Continuous
Random variable	$X$ = a real-valued deviation or error
Support	$-\infty < X < \infty$
Parameters	$\mu$ = location (mean = median = mode; units of $X$); $b$ = scale ($b > 0$; units of $X$)
Mean	$\mu$
Variance	$2b^2$
Signature	Absolute-distance exponential: a sharp peak with heavier-than-Normal tails

Shape Before Formula

The Normal penalizes deviations by their square; the Laplace penalizes them by their absolute value. That single change, $\lvert x - \mu \rvert$ instead of $(x-\mu)^2$, produces the sharp peak and the slower tail decay.

\[f(x) = \frac{1}{2b}\, e^{-\frac{\lvert x - \mu \rvert}{b}}.\]

The comparison above holds variance fixed, so the difference in shape is real and not just a rescaling: at equal variance the Laplace is peakier in the middle and fatter in the tails, trading shoulder mass for both.

CDF, Central Intervals, and Spread

The CDF is piecewise (splitting at $\mu$), which makes symmetric intervals clean:

\[F(x) = \begin{cases} \tfrac{1}{2} e^{(x-\mu)/b} & x < \mu \\[4pt] 1 - \tfrac{1}{2} e^{-(x-\mu)/b} & x \ge \mu \end{cases}, \qquad P(\lvert X - \mu \rvert \le t) = 1 - e^{-t/b}.\]

For example, with $\mu = 0$ and $b = 2$ (a sensor whose errors are usually small but occasionally larger), $P(-3 \le X \le 3) = 1 - e^{-3/2} \approx 0.777$. A robust spread measure is the median absolute deviation, which for the Laplace is exactly $\text{MAD} = b\ln 2$.

Two ML Connections, Kept Separate

The Laplace shows up twice in machine learning, and the two roles are routinely conflated:

• Absolute-residual loss (minimizing $\sum_i \lvert y_i - \hat y_i \rvert$) corresponds to a Laplace likelihood on the residuals and estimates the conditional median. This is least-absolute-deviations regression, the median case of quantile regression, and it is more robust to outliers than squared-error regression.
• An L1 penalty on the coefficients (as in Lasso) corresponds to a Laplace prior on the coefficients in a Bayesian view. Lasso keeps the ordinary squared-residual loss and merely adds $\lambda\sum_j \lvert \beta_j \rvert$; the sharp peak of the Laplace prior at zero is what drives coefficients exactly to zero for feature selection.

One is a claim about the error distribution (median regression); the other is a claim about the coefficients (Lasso). They are different mechanisms that happen to share the Laplace’s absolute-value signature.

Where It Comes From, and When It Fits

The difference of two independent Exponential variables with the same rate is a zero-centered Laplace; adding a location gives the general $\text{Laplace}(\mu, b)$. That is why it appears wherever a quantity is a difference of two waiting-time-like effects.

Assumes	Symmetric errors that are usually small but occasionally moderately large
Common violation	True extreme-value problems (annual maxima, flood peaks), which need dedicated extreme-value distributions (Gumbel/GEV), not merely a curve with more central-tail mass
Closest relatives	Normal (squared vs absolute distance); Exponential (its one-sided half)

A good use case is symmetric error with occasional larger deviations, such as sensor noise or certain signal amplitudes; a poor one is modeling maxima, where a heavier-than-Normal body says nothing about the right model for the extreme tail.

A Minimal Code Check

import numpy as np
from scipy.stats import laplace

mu, b = 0.0, 2.0                       # scipy: loc = mu, scale = b
laplace.cdf(3, mu, b) - laplace.cdf(-3, mu, b)   # P(|X|<=3) -> 0.777
laplace.var(mu, b)                     # 2 b^2 -> 8.0
b * np.log(2)                          # median absolute deviation -> 1.386

---

Appendix: Deriving the Mean and Variance

Shift to $Y = X - \mu$, whose density $f_Y(y) = \frac{1}{2b}e^{-\lvert y\rvert/b}$ is symmetric about 0. Both moments are cleanest in terms of $Y$.

The Mean

The integrand $y\,f_Y(y)$ is odd (an odd function times an even density) and integrable, so it integrates to zero over the symmetric range:

\[E[Y] = \int_{-\infty}^{\infty} y\,\frac{1}{2b}e^{-\lvert y\rvert/b}\,dy = 0.\]

Therefore $E[X] = E[Y] + \mu = \mu$. (Note this is not the same as claiming $x f(x)$ is symmetric about $\mu$, which is false; the symmetry that matters is that of the shifted density $f_Y$ about 0.)

The Variance

Since $E[X] = \mu$, the variance is $E[(X - \mu)^2] = E[Y^2]$. The integrand $y^2 f_Y(y)$ is even, so we integrate over $[0, \infty)$ and double:

\[\text{Var}(X) = 2\int_{0}^{\infty} y^2\,\frac{1}{2b}e^{-y/b}\,dy = \frac{1}{b}\int_{0}^{\infty} y^2 e^{-y/b}\,dy.\]

The remaining integral is a Gamma integral, $\int_{0}^{\infty} y^{n} e^{-y/b}\,dy = n!\,b^{n+1}$, which at $n = 2$ gives $2b^3$. Hence

\[\text{Var}(X) = \frac{1}{b}\cdot 2b^3 = 2b^2.\]

---

Where this sits in the series. Previous concept: Log-Normal. Next concept: Gamma (waiting time for the $k$-th event). Closest cousins: Normal (equal-variance comparison) and Exponential (its one-sided half). Series overview.