ML from Scratch

Linear Regression

Linear Regression using gradient descent with methods for training, prediction, scoring (R²), and cost visualization.

Logistic Regression

Binary logistic regression implemented from scratch using gradient descent with prediction, scoring, and 2D decision boundary plotting.

Multimodal Logistic Regression

Advanced logistic regression handling multiple data modalities with early, late, and intermediate fusion using gradient descent.

K-Nearest Neighbors (KNN)

K-Nearest Neighbors classifier implemented from scratch with multiple distance metrics, cross-validation for k selection, and decision boundary visualization.

K-Means Clustering

K‑Means clustering from scratch with random or k‑means++ initialization plus inertia, silhouette, elbow diagnostics, and rich visualization utilities.

DBSCAN

Density-Based Spatial Clustering of Applications with Noise implemented from scratch with batch processing, visualization, and parameter optimization.

Neural Network

Multi-layer neural network implemented from scratch with customizable architectures, activation functions, regularization, and comprehensive visualization capabilities.

Naive Bayes

Gaussian and Multinomial Naive Bayes classifiers implemented from scratch with visualization capabilities for feature distributions and decision boundaries.

Decision Tree

Decision Tree classifier implemented from scratch with support for Gini/Entropy splitting, feature importances, visualization, and boundary plotting.

Support Vector Machine

Support Vector Machine implemented from scratch using Sequential Minimal Optimization with multiple kernel functions and visualization capabilities.

Random Forest

Random Forest classifier implemented from scratch with bootstrap sampling, feature importance analysis, and visualization capabilities.

Gradient Boosting Regressor

Gradient Boosting Regressor implemented from scratch using decision trees as weak learners and mean initialization.

XGBoost

Extreme gradient boosting implementation from scratch with gradient boosting, regularization, subsampling, early stopping, and feature importance analysis.

AdaBoost

Adaptive boosting algorithm that combines weak learners with adaptive sample weighting.

FixMatch

Semi-supervised learning algorithm combining consistency regularization and pseudo-labeling with weak and strong augmentations.

AlexNet

Deep convolutional neural network that revolutionized computer vision with ReLU activations and dropout regularization.

Attention Is All You Need

Complete transformer architecture with multi-head self-attention, positional encoding, and encoder-decoder structure.

DDPM - Diffusion Models

Implementation of DDPM with forward diffusion process, reverse denoising network, and sampling procedures for high-quality image generation.

Auto-Encoding Variational Bayes

Variational autoencoder with encoder-decoder architecture, reparameterization trick, and KL divergence regularization for latent space learning.

Principal Component Analysis

PCA implemented from scratch with variance analysis, visualization, and compression/decompression demonstrations.

t-Distributed Stochastic Neighbor Embedding

t-SNE implemented from scratch with Barnes-Hut approximation, perplexity optimization, and visualization tools.

Stochastic Gradient Descen

SGD optimizer implemented from scratch with momentum, learning rate decay, and gradient clipping options.

Adaptive Moment Estimation

Adam optimizer from scratch with bias correction, exponential moving averages, and convergence monitoring.

Root Mean Square Propagation

RMSProp implementation from scratch featuring moving average of squared gradients and step-size control.

AdamW Optimizer

AdamW optimizer implemented from scratch with decoupled weight decay and improved generalization capabilities.

Adaptive Gradient Algorithm

Adagrad implementation from scratch with per-parameter learning rate adaptation and sparse feature support.