Simple Transformer
An implementation of the “Attention is all you need” paper without extra bells and whistles,
or difficult syntax.
Note: The only extra thing added is Dropout regularization in some layers and option to use GPU.
Install
python -m pip install -r requirements.txt
Toy data
python train_toy_data.py
| Before Training | After 100 Epoch |
|---|---|
 |
 |
English -> German Europarl dataset
python train_translate.py
Training on a small subset of 1000 sentences (Included in this repo)
