Gradient-Based Subword Tokenization
GBST
**GBST**, or **Gradient-based Subword Tokenization Module**, is a soft gradient-based subword tokenization module that automatically learns latent subword representations from characters in a data-driven fashion. Concretely, GBST enumerates candidate subword blocks and learns to score them in a position-wise fashion using a block scoring network.  

GBST learns a position-wise soft selection over candidate subword blocks by scoring them with a scoring network. In contrast to prior tokenization-free methods, GBST learns interpretable latent subwords, which enables easy inspection of lexical representations and is more efficient than other byte-based models.
Charformer: Fast Character Transformers via Gradient-based Subword Tokenization
https://paperswithcode.com/paper/charformer-fast-character-transformers-via
Charformer: Fast Character Transformers via Gradient-based Subword Tokenization