/ Machine Learning

Pointer-generator reinforced seq2seq summarization in PyTorch

Pointer-generator reinforced seq2seq summarization in PyTorch

seq2seq-summarizer

Lightweight PyTorch implementation of a seq2seq text summarizer.

Advantages

  • Simple code structure, easy to understand.
  • Minimal dependencies (Python 3.6, torch, tqdm and matplotlib).

Implemented

  • Batch training/testing on GPU/CPU.
  • Teacher forcing ratio.
  • Initialization with pre-trained word embeddings.
  • Embedding sharing across encoder, decoder input, and decoder output.
  • Attention mechanism (bilinear, aka Luong's "general" type).
  • Pointer network, which copies words (can be out-of-vocabulary) from the source.
  • Visualization of attention and pointer weights:

Visualization

To be implemented

  • Run on longer texts (missing modern hardware $_$).
  • Beam search at test time.

GitHub