Reddit finetune? 🤔
Would be interesting to see you do a Reddit-based finetune. As a primarily left-leaning platform it might even balance out the worst of Pepe while making it more flexible. I'm already impressed as to how Claude-like this model seems so far, I just think it could use some taming.
(Also as in #4 please do consider moving to Gemma 4 31B one day. That would allow you to also QLoRA the images into the vision component too, I mean 4chan is an imageboard isn't it? :)
Reddit only will likely degrade the model capabilities (similar to twitter, but to a lesser extent, 4chan will likely improve even a great base model though, based on multiple testimonies):
(Original discussion here: https://www.reddit.com/r/LocalLLaMA/comments/1qsrscu/can_4chan_data_really_improve_a_model_turns_out/)
I'm not too surprised at Twitter being so bad at it 🤣
reddit is infested with bots posing as humans, so it's not surprising that the data isn't helpful. even before that, the memes about redditors being out of touch weren't born out of thin air