Conditional Positional Encoding
Conditional Positional Encoding
**Conditional Positional Encoding**, or **CPE**, is a type of positional encoding for [vision transformers](https://paperswithcode.com/methods/category/vision-transformer). Unlike previous fixed or learnable positional encodings, which are predefined and independent of input tokens, CPE is dynamically generated and conditioned on the local neighborhood of the input tokens. As a result, CPE aims to generalize to the input sequences that are longer than what the model has ever seen during training. CPE can also keep the desired translation-invariance in the image classification task. CPE can be implemented with a [Position
Encoding Generator](https://paperswithcode.com/method/positional-encoding-generator) (PEG) and incorporated into the current [Transformer framework](https://paperswithcode.com/methods/category/transformers).
Conditional Positional Encodings for Vision Transformers
https://paperswithcode.com/paper/do-we-really-need-explicit-position-encodings
Conditional Positional Encodings for Vision Transformers