Instructions to use coder3101/gpt-oss-20b-heretic with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use coder3101/gpt-oss-20b-heretic with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="coder3101/gpt-oss-20b-heretic")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("coder3101/gpt-oss-20b-heretic")
model = AutoModelForCausalLM.from_pretrained("coder3101/gpt-oss-20b-heretic", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use coder3101/gpt-oss-20b-heretic with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "coder3101/gpt-oss-20b-heretic"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "coder3101/gpt-oss-20b-heretic",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/coder3101/gpt-oss-20b-heretic

SGLang

How to use coder3101/gpt-oss-20b-heretic with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "coder3101/gpt-oss-20b-heretic" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "coder3101/gpt-oss-20b-heretic",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "coder3101/gpt-oss-20b-heretic" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "coder3101/gpt-oss-20b-heretic",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use coder3101/gpt-oss-20b-heretic with Docker Model Runner:
```
docker model run hf.co/coder3101/gpt-oss-20b-heretic
```

coder3101 commited on Jan 6

Commit

7e83054

verified ·

1 Parent(s): 9c516c2

Upload GptOssForCausalLM

Browse files

Files changed (9) hide show

model-00001-of-00009.safetensors +1 -1
model-00002-of-00009.safetensors +1 -1
model-00003-of-00009.safetensors +1 -1
model-00004-of-00009.safetensors +1 -1
model-00005-of-00009.safetensors +1 -1
model-00006-of-00009.safetensors +1 -1
model-00007-of-00009.safetensors +1 -1
model-00008-of-00009.safetensors +1 -1
model-00009-of-00009.safetensors +1 -1

model-00001-of-00009.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8c7f3611231bbbd02bfd260ba8b1c8ebe725310d817e34ed88777d2b26a7576b
 size 4504304664

 version https://git-lfs.github.com/spec/v1
+oid sha256:18d2a39fc7922c952b929629f782e247f961829061661bcf0105e0dcdbb83ebd
 size 4504304664

model-00002-of-00009.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:1f98889e9c01aed2a24ba0e34a27417263470e50d62bd38c30b96489d789d386
 size 4939127656

 version https://git-lfs.github.com/spec/v1
+oid sha256:236867079d998edf13ef0f14c999670609f191aaf37550e498fd2a83b5c1b37d
 size 4939127656

model-00003-of-00009.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0fcdd6fd4215089a6155b3401749ca03690edc8bcab1f1ba444b5e6b0d11ff05
 size 4939127656

 version https://git-lfs.github.com/spec/v1
+oid sha256:eb927f804a74dad6b9a398e38368b781c32b612a138e5bc7d400fe615e0a97b3
 size 4939127656

model-00004-of-00009.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:640a7f3aaeda01fdbcfc44c27a8369c0ce8ee669b6cf3c912d9d020cff619360
 size 4939127680

 version https://git-lfs.github.com/spec/v1
+oid sha256:7afd9ed0264eb5efba2454793e8d8d9f111c42b185660f59002c9ecc6ff45280
 size 4939127680

model-00005-of-00009.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:7aab6e99362b2b5a03b09e2a522d757cfea14142251a1063daf2ccb3af8b0457
 size 4939127704

 version https://git-lfs.github.com/spec/v1
+oid sha256:d749ddf9381820522501ed17b9b4b806c22d542cca98279cd5a4dd59d517754a
 size 4939127704

model-00006-of-00009.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:8bdb0d5602d9d639f0466d24cdfa89c5c673072f53ba566e6a6c34033886b00b
 size 4939127704

 version https://git-lfs.github.com/spec/v1
+oid sha256:03791be25b4308866f4ecd1fbc08fe2ec3cfe6d0115a3a22c6e0381e305411b5
 size 4939127704

model-00007-of-00009.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:f99a33f7623d4985c3f379df8352760098c6235fe3fae3631de09501566d8c58
 size 4939127704

 version https://git-lfs.github.com/spec/v1
+oid sha256:459944236cd36bbe98f0cf4eac8c0d9dd2db5e20bb39b74bbfaa54efa9466024
 size 4939127704

model-00008-of-00009.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:b0a9a0e41ffcf12f4294939454e74b1076e5ade14f45b293f01f298f5c4bcad9
 size 4939127704

 version https://git-lfs.github.com/spec/v1
+oid sha256:ab2e09681c705b5ac49c8b1ed0e6e4c6fb9b4e350d7c37f502d621ad864ad785
 size 4939127704

model-00009-of-00009.safetensors CHANGED Viewed

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:6ed83fdf4bf2a1bd82ed1fd498dc2243cac7f0f33f2102753fea2b80c28d1b7e
 size 2751362856

 version https://git-lfs.github.com/spec/v1
+oid sha256:5c631214091a6187cf6d098a26a97ff464560e3f54e12236836d7d204dd133eb
 size 2751362856