generic encoder - convert something into another representation
(not to be confused with encoder architecture)
text encoder in LLM - tokenizer
- text -> tokenId
- deterministic
image encoder
- pixels -> hidden vectors (learned)
- a neural network
byte pair encoder (BPE) is an advanced example