Sequence-to-Sequence Learning with Latent Neural Grammars
Code for the paper:
Sequence-to-Sequence Learning with Latent Neural Grammars
Yoon Kim
arXiv Preprint
Dependencies
The code was tested in python 3.7 and pytorch 1.5. We also use a slightly modified version of the Torch-Struct library, which is included in the repo and can be installed via:
cd pytorch-struct
python setup.py install
Data
For convenience we include the datasets used in the paper in the data/ folder. Please cite the original papers when using the data (i.e. Lake and Baroni 2018 for SCAN/MT, and Lyu et al. 2021 for StylePTB).
Training
SCAN
To train the model on (for example) the length split:
python train_scan.py --train_file data/SCAN/tasks_train_length.txt --save_path scan-length.pt
For prediction and evaluation:
python predict_scan.py --data_file data/SCAN/tasks_test_length.txt --model_path scan-length.pt
Style Transfer
To train on (for example) the active-to-passive task:
python train_styleptb.py --train_file data/StylePTB/ATP/train.tsv --dev_file data/StylePTB/ATP/valid.tsv --save_path styleptb-atp.pt
To predict:
python predict_styleptb.py --data_file data/StylePTB/ATP/test.tsv --model_path styleptb-atp.pt
--out_file styleptb-atp-pred.txt
We use the nlg-eval package to calculate the various metrics.
Machine Translation
To train on MT:
python train_mt.py --train_file_src data/MT/train.en --train_file_tgt data/MT/train.fr
--dev_file_src data/MT/dev.en --dev_file_tgt data/MT/dev.fr --save_path mt.pt
To predict on the daxy test set:
python predict_mt.py --data_file data/MT/test-daxy.en --model_path mt.pt --out_file mt-pred-daxy.txt
For the regular test set:
python predict_mt.py --data_file data/MT/test.en --model_path mt.pt --out_file mt-pred.txt
We use the multi-bleu script to calculate BLEU.
Training Stability
We observed training to be unstable and the approach required several runs across different seeds to perform well. For reference we have posted logs of some example runs in the logs/ folder.
License
MIT
