Weight Initialization: Why Random Isn't Random Enough
Part 12 of my "revisiting my AI/ML notes" series. Every post so far in this series has assumed...
Jul 20, 20262 min read0 reactions0 comments
Back to articles Jul 20, 2026 
Jul 20, 2026
Jul 20, 2026
Jul 20, 2026
Jul 19, 2026
Jul 18, 2026
Jul 18, 2026
Jul 18, 2026
Tag archive
#neuralnetworks
Batch, Stochastic, and Mini-Batch Gradient Descent: What's Actually Different
Part 13 of my "revisiting my AI/ML notes" series. Every post in this series so far used gradient...
Jul 20, 20263 min read0 reactions0 comments
Beyond Sigmoid and ReLU: Leaky ReLU, ELU, Swish, and Softmax
Part 11 of my "revisiting my AI/ML notes" series. Earlier in this series I covered sigmoid and plain...
Jul 20, 20263 min read0 reactions0 comments
Dropout and Regularization: How Neural Networks Avoid Overfitting
Part 10 of my "revisiting my AI/ML notes" series. Everything so far has been about getting a network...
Jul 20, 20262 min read0 reactions0 comments
B
Jul 19, 2026Build a Neural Network from Scratch: From First Principles to Backpropagation [Modern C++/ No Libs]
A step-by-step mathematical guide with diagrammatic, clear explanations of all the core...
Jul 19, 202642 min read0 reactions0 comments
The Vanishing Gradient Problem, Explained
Part 8 of my "revisiting my AI/ML notes" series. The last post ended on a warning: multiplying...
Jul 19, 20262 min read0 reactions0 comments
D
Jul 20, 2026DenseNet from scratch: ResNet adds a skip connection, DenseNet concatenates every one — top accuracy at a third of the params
ResNet gave every layer a shortcut to the loss by adding a residual: y = x + F(x). DenseNet takes...
Jul 20, 20263 min read0 reactions0 comments
Backpropagation Explained: How a Neural Network Actually Learns
Part 5 of my "revisiting my AI/ML notes" series. So far this series has covered the ANN structure,...
Jul 18, 20263 min read0 reactions0 comments
Understanding ANN (Artificial Neural Networks): The Building Block of Deep Learning
This is the second post in my "revisiting my AI/ML notes" series — last time I covered the AI vs ML...
Jul 18, 20262 min read0 reactions0 comments
How a Neural Network Actually Computes: Forward Propagation Explained
Part 3 of my "revisiting my AI/ML notes" series. Last post covered the ANN structure — input, hidden,...
Jul 18, 20262 min read0 reactions0 comments
S
Jul 20, 2026Stochastic Depth from scratch: randomly drop whole residual layers to train deeper nets faster
A 100-layer ResNet is slow to train and its gradients still fade on the way to the earliest layers....
Jul 20, 20264 min read0 reactions0 comments
M
Jul 18, 2026Mixup and CutMix: train on images that don't exist by blending two pictures AND their labels into a soft target
Ordinary augmentation transforms one image and keeps its label. Mixup and CutMix go further and, at...
Jul 18, 20264 min read1 reactions0 comments