Logistic Regression, Part 1: From Log-Odds to Gradient Descent
Derive binary logistic regression from log-odds and Bernoulli likelihood, then train it using cross-entropy and numerical optimization.
Derive binary logistic regression from log-odds and Bernoulli likelihood, then train it using cross-entropy and numerical optimization.
Regularizers are priors in disguise: ridge falls out of a Gaussian prior, lasso out of a Laplace prior. One hand-worked posterior, the geometry of sparsity, posterior and predictive uncertainty, and the nuance most summaries garble.
Evaluate linear regression with R-squared, handle outliers, interpret coefficients and feature importance, and extend the framework with GLMs.
Diagnose multicollinearity and understand how Ridge, Lasso, Elastic Net, and Bayesian priors stabilize or simplify linear regression.
Derive the normal equation and understand the statistical guarantees, assumptions, invariances, and singularity limits of ordinary least squares.
Bias and variance as properties of a learning procedure across datasets: the archery and house-price intuition, a repeated-fits simulation, the squared-error decomposition, and how to diagnose under- vs overfitting in practice.
MCAR/MAR/MNAR as assumptions not labels, what MICE and Rubin's rules really do, and a leakage-safe imputation workflow for prediction and inference.
The foundations post covered how to design and validate a randomized experiment. This one answers the question it left open: how much data, and how much time, do we actually need? That means connec...
Resampling-based A/B tests that build the null distribution from the data: the permutation test and the bootstrap, when to use each, and how many simulations they need.
The model-based A/B tests: ANOVA for three or more groups, the Chi-square test for categorical data, and the Bonferroni correction, with a chi-squared-distribution and KL-divergence appendix.