1. Welcome
1. Introduction
2. Outline
2. Getting Setup
1. Get Your Hands Dirty, Practical Coding Experience, Data Links
2. How to use Github & Extra Coding Tips (Optional)
3. Where to get the code, notebooks, and data
4. Are You Beginner, Intermediate, or Advanced All are OK!
3. Beginner’s Corner
1. Beginner’s Corner Section Introduction
2. From RNNs to Attention and Transformers - Intuition
最初对状态的解决方案是seq2seq, 引入了编码器和解码器的概念
问题是, 如果句子很长, 但是最终编码器的输出是固定规格的, 于是不可避免地损失上下文信息
3. Sentiment Analysis
pip install transformers
# 查询transformers版本
import transformers
transformers.__version__
from transforms import pipeline
# Create your pipeline (includes tokenization,etc.
classifier = pipeline("sentiment-analysis")
# NO need to convert input into PyTorch Tensor,Numpy array
# Tensorflow Tensor,etc.
# Output is a dictionary
classifier("This is such a great movie!")
classifier("I can't understand anything...prerequisites...")
4. Sentiment Analysis in Python
用py35或2.7试一下
8. Transformers and Attention Theory (Advanced)
1. Theory Section Introduction
2. Basic Self-Attention
3. Self-Attention & Scaled Dot-Product Attention
9. Implement Transformers From Scratch (Advanced)
1. Implementation Section Introduction
2. Encoder Implementation Plan & Outline
