machine learning / deep learning 比较好的一些topic / trend

Topic:

1. matrix factorization
2. collaborative filtering 
3. CTR预估
4. one shot learning （单样本学习）+ 孪生网络
5. 强化学习 RL
6. 对抗生成网络 GAN
7. GCN/GNN
8. Meta-learning法。从先前的学习经验中提炼出基本的参数和结构配置。
9. Transfer learning法。从先前的学习经验中提炼出可以重用的一些知识
12. machine translation
13. LSTM/GRU/GMU...
14. Attention
15. Transformer
16. encoder-decoder
17. factorization machine
18. auto-encoder
19. sequence to sequence
20. copy net
21. highway
22. conventional/ recurrent / recursive neural network
23. memory network/Neural Turing Machines
24. Momentum

Method:

1. DSSM
2. Wide & Deep
3. GAN各种变形
4. FM各种变形
5. GLU/GTU/GMU/GRU
6. ResNet
7. copynet

实验的trick:

1. dropout
2. L1/L2 正则
3. batch norm 
4. clip by norm/value
5. learning rate decay

预处理语言模型：

word2vec, fasttext, glove, elmo, gpt, bert