Whats the maximum context length for this model?
I want to use this with lmstudio and memgpt, and memgpt wants the context length set to the max:
https://memgpt.readthedocs.io/en/latest/lmstudio/#memgpt-lm-studio
it says: Make sure that "context length" is set (inside LM Studio's "Model Configuration" panel) to the max context length of the model you're using (e.g. 8000 for Mistral 7B variants).
Thanks!
@Samvanity > you ca check this for almost each model here on HF - go to the Files and versions tab up there > search for the config.json , click on it > look for the line "max_position_embeddings"
According to openrouter.ai rankings, the 8k context version of this model is exploding!
They refer to it as mythomax-l2-13b-8k, but I couldn't find it anywhere for download? Where is that?
@Gryphe ?
Heya! I believe someone created a 8k version but never published it, only making it available through a service of sorts.
Afraid that's about the extent of my knowledge at this point.
It should be perfectly fine, yeah! I got lotsa reports claiming that MythoMax should be very robust when it comes to RoPE scaling, even pushing it all the way to 16k.