Storage for ML Systems, Part 1: Choosing the Right Store
A decision framework for picking data stores across an ML system, from training data in object storage to online features under a p99 budget.
TechScribr
Latest writing
Clear explanations, practical insightsFrom Boring to Brilliant: A Guide to LLM Sampling Techniques
When we ask an LLM a question, it doesn’t just “know” the answer. At each step it produces a score for every possible next token, one step at a time. But how it turns those scores into an actual ch...
Choosing a Boosting Library: XGBoost, LightGBM, and CatBoost
The two previous articles covered the algorithms and the XGBoost mathematics. This final article is practical: the general boosting concepts that apply to any library (feature importance, early sto...
XGBoost Mathematics: The Regularized Second-Order Objective
The previous article built gradient boosting as gradient descent in function space. XGBoost keeps that idea but rebuilds the objective: a regularized loss, a second-order (Newton) approximation, an...
Boosting: From AdaBoost to Gradient Boosting
Boosting answers a deceptively simple question: can a committee of weak learners, each barely better than a coin flip, add up to one strong model? The last three articles of this path work through ...
Random Forests: Bagging, Decorrelated Trees, and Out-of-Bag Error
In the previous article we built a single Decision Tree - and saw how readily it overfits. That sets up a natural thought: “If one tree is good, surely a whole forest is better?” That intuition is...
Decision Trees: Splitting, Impurity, and Why Trees Overfit
Imagine playing a game of “20 Questions.” We are trying to guess an object, and we can only ask yes/no questions. To win quickly, we wouldn’t ask, “Is it a 1994 Ford Taurus?” right away. We’d ask b...
Understanding PCA Once and for All
Principal Component Analysis (PCA) is fundamentally a geometric optimization problem: finding the directions that capture the maximum variance in a dataset. This article derives the core equations ...
Connection between KL Divergence & Cross Entropy
Why cross-entropy is the loss for probabilistic classifiers: the additive identity H(P,Q)=H(P)+KL(P||Q), the coin-flip derivation, the bridge to negative log-likelihood and maximum likelihood, binary/multiclass/soft-label forms, and stable logit-based implementation.
A Short Primer on KL/JS Divergence
KL and Jensen-Shannon divergence for drift monitoring: what they measure, why direction and support matter, the divergence-vs-distance and log-base gotchas, and how to turn a drift score into a decision (not an automatic retrain).