Visformer
Visformer
**Visformer**, or **Vision-friendly Transformer**, is an architecture that combines [Transformer](https://paperswithcode.com/methods/category/transformers)-based architectural features with those from [convolutional neural network](https://paperswithcode.com/methods/category/convolutional-neural-networks) architectures. Visformer adopts the stage-wise design for higher base performance. But [self-attentions](https://paperswithcode.com/method/multi-head-attention) are only utilized in the last two stages, considering that self-attention in the high-resolution stage is relatively inefficient even when the FLOPs are balanced. Visformer employs [bottleneck blocks](https://paperswithcode.com/method/bottleneck-residual-block) in the first stage and utilizes [group 3 × 3 convolutions](https://paperswithcode.com/method/grouped-convolution) in bottleneck blocks inspired by [ResNeXt](https://paperswithcode.com/method/resnext). It also introduces [BatchNorm](https://paperswithcode.com/method/batch-normalization) to patch embedding modules as in CNNs.
Visformer: The Vision-friendly Transformer
https://paperswithcode.com/paper/visformer-the-vision-friendly-transformer
Visformer: The Vision-friendly Transformer