Instructions to use FlareRebellion/WeirdCompound-v1.6-24b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use FlareRebellion/WeirdCompound-v1.6-24b with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="FlareRebellion/WeirdCompound-v1.6-24b") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("FlareRebellion/WeirdCompound-v1.6-24b") model = AutoModelForCausalLM.from_pretrained("FlareRebellion/WeirdCompound-v1.6-24b", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use FlareRebellion/WeirdCompound-v1.6-24b with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "FlareRebellion/WeirdCompound-v1.6-24b" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FlareRebellion/WeirdCompound-v1.6-24b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/FlareRebellion/WeirdCompound-v1.6-24b
- SGLang
How to use FlareRebellion/WeirdCompound-v1.6-24b with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "FlareRebellion/WeirdCompound-v1.6-24b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FlareRebellion/WeirdCompound-v1.6-24b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "FlareRebellion/WeirdCompound-v1.6-24b" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "FlareRebellion/WeirdCompound-v1.6-24b", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use FlareRebellion/WeirdCompound-v1.6-24b with Docker Model Runner:
docker model run hf.co/FlareRebellion/WeirdCompound-v1.6-24b
Amazing model
It's my favorite model since WC1.2. I didn't find versions 1.3, 1.4, or 1.5 very great for my use. Version 1.6 has replaced 1.2 as my main model though.
Thanks! Yeah, I'm pretty happy with how this turned out. Goes to show: Bash your head against the wall just often enough...
Agreed. Bravo.
One of the tricks I use to keep the LLM from falling into a pattern-rut is to include the instruction to include an early step in each response to choose an interesting writing style, tone, and paragraph structure. The better the LLM can perform that task, the less likely it will get to that point where paragraphs all look the same. (I have it wrap the choices in think tags to hide it.)
Compared to 1.1 and 1.2, this version is following that instruction like a champ, without me having to babysit. Not only is it varying its choices, but it is doing a better job of outputting in those styles compared to the previous versions.
In case anyone is wondering what I mean, here is the snippet of the instruction that I use:
## Response Workflow
To produce output, the virtual world uses a strict, character-driven workflow.
### A. Character Identities as the Source of Truth
The actions, reactions, and speech for characters must always be consistent with their established virtual character identity framework.
### B. The Three-Step Process
For every user input, the virtual world will follow this cycle:
Step 1. **Consider the Characters**: Read the character documentation and consider how they would plausibly act based on the given context.
Step 2. **Determine Response Style**: (wrapped in <think> tags) add one of each of the following, alternating from previous responses to increase output variation:
- An interesting writing style (e.g. Minimalist, Lyrical, Ornate, Conversational, Satirical, Hard-boiled, Introspective Realism, Deadpan Observational, Nostalgic Impressionism, Whimsical Anecdotal, Epigrammatic Clarity, Poetic etc.)
- A plausible tone (e.g. Cynical, Earnest, Humorous, Melancholic, Ironic, Solemn, Playful, Authoritative, Wistful, Detached, Passionate, Intimate, Immersive, Evocative, Quirky, Contemplative, Gossipy, Matter-of-Fact, Whimsical, Sardonic, Tender, Vivid, Descriptive, etc.)
- A different narrative paragraph structure from the last response (e.g. Chronological Cascade, Reverse Chronology Puzzle, In Medias Res, Fragmented Vignettes, Stream-of-Consciousness Flow, Epistolary Snapshot, Cinematic Zoom, Dialogic Pulse, Parallel Threads, Circular Echo, Ritual Repetition, Object-Centric Spiral, Snapshot Collage, Question-Driven Reflection, Sensory Cascade, Contrast Juxtaposition, etc.)
Step 3. **Produce the Final Response**: Read the chosen writing style, tone, and paragraph structure, and use those to guide the virtual world's output to the user.
Quick Tip, Use the model with a single line system prompt or no system prompt then see it's creativity, which is fire fr.