Implementation of State-of-the-art Text Classification Models in Pytorch
- fastText: fastText Model from Bag of Tricks for Efficient Text Classification
- TextCNN: CNN for text classification proposed in Convolutional Neural Networks for Sentence Classification
- TextRNN: Bi-direction LSTM network for text classification
- RCNN: Implementation of RCNN Model proposed in Recurrent Convolutional Neural Networks for Text Classification
- CharCNN: Implementation of character-level CNNs as proposed in the paper Character-level Convolutional Networks for Text Classification
- Seq2seq With Attention: Implementation of seq2seq model with attention from Neural Machine Translation by Jointly Learning to Align and Translate, Text Classification Research with Attention-based Recurrent Neural Networks
- Transformer: Implementation of Transformer model proposed in Attention Is All You Need
- Python-3.5.0
- Pandas-0.23.4
- Numpy-1.15.2
- Spacy-2.0.13
- Pytorch-0.4.1.post2
- Torchtext-0.3.1
- Download data into "data/" directory or use already available data
- If using your own data, convert it into the same format as of provided data
- Download Pre-trained word embeddings (Glove/Word2Vec) into "data/" directory
- Go to corresponding model directory
- run following command:
python train.py <path_to_training_file> <path_to_test_file>
- All the models were run on a 14GB machine with 2 Cores and one NVIDIA Tesla K80 GPU.
- Runtime in the table below includes the time to load and process the data and running the model.
- Model parameters are not tuned. So, better performance can be achieved by some parameter tuning.
| Model | Dataset | |||
|---|---|---|---|---|
| AG_News | Query_Well_formedness | |||
| Accuracy (%) | Runtime | Accuracy (%) | Runtime | |
| fastText | 89.46 | 16.0 Mins | 62.10 | 7.0 Mins |
| TextCNN | 88.57 | 17.2 Mins | 67.38 | 7.43 Mins |
| TextRNN | 88.07 (Seq len = 20) 90.43 (Flexible seq len) |
21.5 Mins 36.8 Mins |
68.29 66.29 |
7.69 Mins 7.25 Mins |
| RCNN | 90.61 | 22.73 Mins | 66.70 | 7.21 Mins |
| CharCNN | 87.70 | 13.08 Mins | 68.83 | 2.49 Mins |
| Seq2Seq_Attention | 90.26 | 19.10 Mins | 67.84 | 7.36 Mins |
| Transformer | 88.54 | 46.47 Mins | 63.43 | 5.77 Mins |
[1] Bag of Tricks for Efficient Text Classification
[2] Convolutional Neural Networks for Sentence Classification
[3] Recurrent Convolutional Neural Networks for Text Classification
[4] Character-level Convolutional Networks for Text Classification
[5] Neural Machine Translation by Jointly Learning to Align and Translate
[6] Text Classification Research with Attention-based Recurrent Neural Networks
[7] Attention Is All You Need
[8] Rethinking the Inception Architecture for Computer Vision
[9] Identifying Well-formed Natural Language Questions