A tiny, pedagogical implementation of a neural network library with a pytorch-like API. The primary use of this library is for education. Use the actual pytorch for more serious deep learning business.
The implementation is complete with tensor-valued autodiff (~100 lines) and a neural network API built off of it (~80 lines).
This little project is actually the result of an article I wrote. Using it, you can learn more about how neural networks work and implement everything in candl yourself from scratch.
pip install candl
First, import candl.
Candl comes with two modules:
nn module contains tools like modules, layers, SGD, MSE, etc. Candl tensors are extensions of numpy ndarrays that can be used to represent data and compute derivatives.
To train a neural net (let’s try to learn XOR), first we can create a model.
nn = candl.nn model = nn.Sequential(nn.Linear(2, 10), nn.ReLU(), nn.Linear(10, 10), nn.ReLU(), nn.Linear(10, 1)) lr = 1e-3 loss_fn = nn.MSE() optimizer = nn.SGD(model.parameters(), lr)
Then, we train:
data = [([0, 0], ), ([0, 1], ), ([1, 0], ), ([1, 1], )] for epoch in range(1000): for sample in data: """ Note that we only allow batches of data, so the shape of the tensor must be n x m, where m is the dimensionality of the input for each batch. """ x = candl.tensor([sample]) y = candl.tensor([sample]) loss = loss_fn(model.forward(x), y) loss.backward() # The `True` argument automatically zeroes the gradients after a step optimizer.step(True)
- Tensors built upon numpy’s ndarrays
- Tensor-valued autograd
- Mean Squared Error Loss Function
- Stochastic Gradient Descent (SGD)
- Blocks (Modules) for putting together neural networks
- Built-in layers: Linear, ReLU, Sigmoid, Tanh