MATE
MATE
**MATE** is a [Transformer](https://paperswithcode.com/method/transformer) architecture designed to model the structure of web tables. It uses sparse attention in a way that allows heads to efficiently attend to either rows or columns in a table. Each attention head reorders the tokens by either column or row index and then applies a windowed attention mechanism. Unlike traditional self-attention, Mate scales linearly in the sequence length.
MATE: Multi-view Attention for Table Transformer Efficiency
https://paperswithcode.com/paper/mate-multi-view-attention-for-table
MATE: Multi-view Attention for Table Transformer Efficiency