Laya performance on fontlab.org/ornotto
#24
by fontlab - opened
Check Laya results on ornotto. We benchmarked 258 ways to run System One decision models on a Mac against TypeSafe's jev: rune, NeoHorse, Jev-Omni, ollaya, GLiNER and more. The book, the results and the Python package: https://fontlab.org/ornotto
TL;DR: Laya scores poorly, only 52/67, in comparison to Jev 65/67.
try ft
Do you mean fine-tuning? Yes, it helps, but the idea of system one models like Jev is that they should be performant out of the box, without requiring to be fine-tuned first. Anyway, thanks for Laya!
very poor score
Pareto it's not bad: the score isn't great but it's incredibly fast. I'll try a new benchmark where one of the possible answers is "hard to tell" and if the Laya chooses that, the question goes to a bigger model