Boosting, Part 2: The XGBoost Mathematics
Boosting series: Part 1: AdaBoost and Gradient Boosting · Part 2: The XGBoost Mathematics · Part 3: Choosing a Library Part 1 built gradient boosting as gradient descent in function space. XGBoos...
TechScribr
Latest writing
Clear explanations, practical insightsBoosting, Part 1: AdaBoost and Gradient Boosting
Boosting series: Part 1: AdaBoost and Gradient Boosting · Part 2: The XGBoost Mathematics · Part 3: Choosing a Library Boosting answers a deceptively simple question: can a committee of weak lear...
The Wisdom of Trees, Part 2: Random Forests
The Wisdom of Trees series: Part 1: Decision Trees · Part 2: Random Forests In Part 1 we built a single Decision Tree - and saw how readily it overfits. That sets up a natural thought: “If one tr...
The Wisdom of Trees, Part 1: Decision Trees
The Wisdom of Trees series: Part 1: Decision Trees · Part 2: Random Forests Imagine playing a game of “20 Questions.” We are trying to guess an object, and we can only ask yes/no questions. To wi...
Understanding PCA Once and for All
Principal Component Analysis (PCA) is fundamentally a geometric optimization problem: finding the directions that capture the maximum variance in a dataset. This article derives the core equations ...
Connection between KL Divergence & Cross Entropy
Why cross-entropy is the loss for probabilistic classifiers: the additive identity H(P,Q)=H(P)+KL(P||Q), the coin-flip derivation, the bridge to negative log-likelihood and maximum likelihood, binary/multiclass/soft-label forms, and stable logit-based implementation.
A Short Primer on KL/JS Divergence
KL and Jensen-Shannon divergence for drift monitoring: what they measure, why direction and support matter, the divergence-vs-distance and log-base gotchas, and how to turn a drift score into a decision (not an automatic retrain).
Venturing into the Isolation Forest
Most models learn what normal looks like and then measure how far a point sits from it: a distance, a density, a reconstruction error. Isolation Forest takes the opposite route. It never models nor...
Interpretations of ROC-AUC & PR-AUC metrics
What ROC-AUC and PR-AUC (average precision) actually measure, why they diverge under class imbalance, how prevalence moves PR but not ROC, and how to pick a metric from the decision you need to make.
Support Vector Machines, Part 4: One-Class SVM for Novelty Detection
One-Class SVM for novelty detection: the separating-from-the-origin objective, what the RBF unit-sphere property does and does not prove, and the roles of nu and gamma.