A deep learning library built from first principles.
MyTorch brings together my Fall 2025 deep learning assignments: from sigmoid and backpropagation to convolution, recurrent networks, CTC, and attention. The core runs on NumPy, with SciPy for numerical functions. An optional PyTorch package contains the complete transformer architectures and sequence generation code.
| Area | Implementations |
|---|---|
| Foundations | Linear layers, seven activations, MSE, cross entropy, batch normalization |
| Convolution | 1D/2D convolution, pooling, resampling, assignment transposed convolutions, scanning MLPs, residual blocks |
| Sequences | RNN and GRU cells, stacked GRU, CTC loss, greedy and prefix beam decoding |
| Attention | Scaled dot-product and multi-head attention, including explicit backward passes |
| Bonuses | Reverse-mode autograd, dropout, channel dropout, SGD, Adam, AdamW |
| Transformers | Decoder-only and encoder-decoder models, speech embeddings, greedy/beam/sampling generation |
This is a teaching library: the NumPy implementation makes forward passes and gradients visible. It is intended for small experiments and understanding the algorithms. See the feature map for assignment origins and API conventions for shapes and usage.
git clone https://github.com/baimamboukar/MyTorch.git
cd MyTorch
python -m pip install -e .
python examples/train_xor.pyimport numpy as np
from mytorch import nn, optim
from mytorch.models import MLP
x = np.array([[0.0, 0.0], [0.0, 1.0], [1.0, 0.0], [1.0, 1.0]])
y = np.array([0, 1, 1, 0])
model = MLP([2, 8, 2], seed=7)
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model, lr=0.03)
for _ in range(250):
optimizer.zero_grad()
loss = criterion(model(x), y)
model.backward(criterion.backward())
optimizer.step()
print(model(x).argmax(axis=-1)) # [0 1 1 0]For the complete transformer models:
python -m pip install -e '.[transformer]'
python examples/transformer_smoke.pypython -m pip install -e '.[dev,transformer]'
python -m pytest
ruff check .
ruff format --check .
python -m buildWith uv, use uv sync --extra dev --extra transformer and prefix commands with
uv run. The lockfile records the resolved dependencies. Without the transformer
extra, tests that compare against PyTorch or exercise transformers are skipped.
Tests cover finite-difference gradients, exact CTC path enumeration, PyTorch comparisons, sequence stopping, and small training runs. See validation notes for the verified scope and source-test compatibility changes. Contributions should include a focused numerical or integration test when behavior changes.
Developed through the Fall 2025 Introduction to Deep Learning assignments and bonus exercises. This repository consolidates those implementations into one installable package, with fixes and consistent imports. The source manifest records the original implementation files; course datasets, reference answers, and autograder bundles are not included.