Intro to AI 2: Transformer - Positional encoding

Question

Transformer - Positional encoding

Christian N · Accepted Answer

Key idea: add a new vector containing positional information to current vectors

Checklist: 
1) Unique encoding for each time-step
2) Consistent distance between any two time-steps
3) Should generalize to longer sentences
4) Deterministic

Intro to AI 2

Transformer - Positional encoding

Key idea: add a new vector containing positional information to current vectors Checklist: 1) Unique encoding for each time-step 2) Consistent distance between any two time-steps 3) Should generalize to longer sentences 4) Deterministic

Author