Why different MTP heads?

by darkbit1001 - opened 12 days ago

Discussion

darkbit1001

12 days ago

What was the reason you chose the base MTP tensors instead of the KAT derived ones?

isogen

Owner 8 days ago

They didn't cotrain MTP or vision, so the base versions should be used for that [1] [2].

For this particular quant, it worked okay in TabbyAPI w/ SillyTavern for me, but I wasn't able to run it with Pi. Maybe the default 4bpw parameters overcooked it. This one worked with Pi as well.

isogen

Owner 8 days ago

I take that back, seems like both MTP exl3 quants are unstable with Pi, it is the non-MTP one that is working.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment