1、Attention Is All You Need

2、Language Models are Few-Shot Learners

3、BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

4、Deep contextualized word representations

5、XLNet: Generalized Autoregressive Pretraining for Language Understanding

6、BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

7、RoBERTa: A Robustly Optimized BERT Pretraining Approach

8、CogView: Mastering Text-to-Image Generation via Transformers

9、Improving Language Understanding by Generative Pre-Training

10、Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

11、ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

12、SpanBERT: Improving Pre-training by Representing and Predicting Spans

13、GPT Understands, Too