candl

A tiny, pedagogical implementation of a neural network library with a pytorch-like API. The primary use of this library is for education. Use the actual pytorch for more serious deep learning business.

The implementation is complete with tensor-valued autodiff (~100 lines) and a neural network API built off of it (~80 lines).

Learning

This little project is actually the result of an article I wrote. Using it, you can learn more about how neural networks work and implement everything in candl yourself from scratch.

Installation

`pip install candl`

Usage

First, import candl.

`import candl`

Candl comes with two modules: `nn`

and `Tensor`

. The `nn`

module contains tools like modules, layers, SGD, MSE, etc. Candl tensors are extensions of numpy ndarrays that can be used to represent data and compute derivatives.

To train a neural net (let’s try to learn XOR), first we can create a model.

```
nn = candl.nn
model = nn.Sequential(nn.Linear(2, 10),
nn.ReLU(),
nn.Linear(10, 10),
nn.ReLU(),
nn.Linear(10, 1))
lr = 1e-3
loss_fn = nn.MSE()
optimizer = nn.SGD(model.parameters(), lr)
```

Then, we train:

```
data = [([0, 0], [0]), ([0, 1], [1]), ([1, 0], [1]), ([1, 1], [0])]
for epoch in range(1000):
for sample in data:
"""
Note that we only allow batches of data, so the shape of the tensor must be n x m,
where m is the dimensionality of the input for each batch.
"""
x = candl.tensor([sample[0]])
y = candl.tensor([sample[1]])
loss = loss_fn(model.forward(x), y)
loss.backward()
# The `True` argument automatically zeroes the gradients after a step
optimizer.step(True)
```

Features

- Tensors built upon numpy’s ndarrays
- Tensor-valued autograd
- Mean Squared Error Loss Function
- Stochastic Gradient Descent (SGD)
- Blocks (Modules) for putting together neural networks
- Built-in layers: Linear, ReLU, Sigmoid, Tanh