Octopus-Qwen3.5-4B (Flagship Model & Full Checkpoint Zoo)

Octopus-Qwen3.5-4B hosts the trained Rank-64 LoRA policies from Octopus: One Brain, Many Branches for Latent-KV Multi-Agent Reinforcement Learning.

Paper (arXiv): https://arxiv.org/abs/2611.06956
Project Page: https://shatianming5.github.io/octopus/
Code & Serving Runtime: https://github.com/shatianming5/octopus
Training & Evaluation Dataset (OpenResearcher + BrowseComp-Plus BM25): Tommysha/octopus-train
Base Backbone: Qwen/Qwen3.5-4B (24 Gated DeltaNet recurrent layers + 8 Gated Full-Attention layers)

Checkpoint Zoo (Repository Layout)

The repository root (/) contains the flagship Step-45 Recursive DAG-GRPO Adapter ($D_{\max}=\infty$) for direct loading with peft or SGLang (--lora-paths octopus=Tommysha/Octopus-Qwen3.5-4B). All intermediate training checkpoints and experimental variants from the paper are organized under checkpoints/:

Path in Repo	Paper Model / Variant	Max Depth ($D_{\max}$)	Reward / Credit	Val (`500` Q)	BrowseComp-Plus (`830` Q)	Notes & Paper Reference
`/` (Root) & `checkpoints/octopus-recursive/step45/`	Octopus Recursive `DAG-GRPO` (`step45`, Flagship)	$\infty$	Pure Outcome (`0/1`)	`46.6%` (`57.2%` done)	`49.6%`	Main Table 1 flagship model (`329.2K` explored / `199.0K` net surviving; Commander–Mesh emergence)
`checkpoints/octopus-recursive/step{10,20,30,40,45}/`	Octopus Recursive `DAG-GRPO` Trajectory	$\infty$	Pure Outcome (`0/1`)	`41.6% → 46.6%`	`26.3% → 49.6%`	Full recursive `DAG-GRPO` training trajectory
`checkpoints/octopus-depth1/step55/`	Octopus Depth-1 `DAG-GRPO` (`step55`)	$1$	Pure Outcome (`0/1`)	`51.2%`	`62.0%†` (`49.6%` full)	Table 1 & Section 4.3 Depth-1 checkpoint
`checkpoints/octopus-depth1/step{10,20,30,40,50,55}/`	Octopus Depth-1 `DAG-GRPO` Trajectory	$1$	Pure Outcome (`0/1`)	`46.2% → 51.2%`	—	Figure 6 training dynamics curve (`16% → 98%` fork rate)
`checkpoints/single-agent-143k/step40/`	Single-Agent GRPO (`143K` Window) / 40-Step Base	$1$	Pure Outcome (`0/1`)	`43.2%` (`143K` linear) / `46.5%` (fork)	`32.1%` (`143K` linear) / `42.6%` (fork)	Table 1 & Figure 4 budget-matched `143K` Single-Agent Linear evaluation (`OCTOPUS_LINEAR=1`, `OCTOPUS_THREAD_BUDGET=143360`)
`checkpoints/single-agent-143k/step{10,20,30,40}/`	40-Step Pure Outcome Trajectory	$1$	Pure Outcome (`0/1`)	`38.0% → 46.5%`	`30.7% → 42.6%`	40-step training trajectory (`step10–40`)
`checkpoints/octopus-process-gated/step{10,20,30,40,50,60,69}/`	Process-Gated `DAG-GRPO` Ablation	$1$	Process-Gated (`search_gate`)	`45.2%` (`step69`)	`39.5%` (`step69`)	Section 4.3 reward-shaping ablation (`step10–69`)
`checkpoints/init-lora/`	Initial Untrained LoRA (`0`-Shot Baseline)	—	Untrained (`0`-shot)	`40.0%` (linear) / `41.6%` (fork)	`26.0%` (linear) / `26.3%` (fork)	Rank-64 initial adapter before RL updates

Highlights

Trained on Qwen/Qwen3.5-4B with pure binary 0/1 outcome rewards and sparse BM25 (zero process shaping), Octopus coordinates multi-agent episodes over a GPU-resident radix DAG via five execution primitives (fork, rewind, read_thread, wait, and done):

BrowseComp-Plus (830 test queries, sparse BM25): Reaches 49.6% (+17.5 points over 143K Single-Agent GRPO at 32.1%, and +23.6 points over 32K Single-Agent ReAct at 26.0%). Disabling fork at test time on these exact weights drops paired accuracy by -21.7 points (50.8% -> 29.1%), confirming that gains come from live multi-agent execution.
OpenResearcher Validation (500 queries, sparse BM25): Reaches 51.2% (D_max=1, checkpoints/octopus-depth1/step55) and 46.6% (D_max=inf, 57.2% on completed trees).
Attention-Only Zero-Copy KV Mounting (read_thread): Splices sibling KV pages exclusively into the 8 full-attention layers under monotonic RoPE position offsets while all 24 Gated DeltaNet recurrent layers skip mounted spans—retaining 92.2% answer information and cutting cross-branch task hijacking 21x (57.9% -> 2.7%) with 0 extra KV pages.
LoRA Target Modules (r=64, alpha=64): Covers all 10 linear projections across attention, SwiGLU MLP, and Gated DeltaNet layers (q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj, in_proj_qkv, in_proj_z, out_proj).

Benchmark Results on BrowseComp-Plus (`830` Queries)

Method / Agent System	Backbone	Retriever & Tools	Explored / Crit. Path	Gold Doc Open	BrowseComp-Plus Acc.
Single-Agent ReAct (`32K` Window)	`Qwen3.5-4B`	BM25 (`search+open`)	`32.6K` / `1,043`	`12.2%`	`26.0%`
Multi-Agent Fork (0-Shot Untrained)	`Qwen3.5-4B`	BM25 (`search+open+tree`)	`38.1K` / `1,590`	`12.5%`	`26.3%`
Single-Agent GRPO (`143K` Flat Window)	`Qwen3.5-4B`	BM25 (`search+open`)	`40.8K` / `4,075`	`21.1%`	`32.1%`
Context-Summary RL (Shao et al., 2026)	`Qwen3.5-9B`	BM25 (`search+open+sum`)	`<= 32.0K`	—	`42.1%`
CLM + SCR RL (Shao et al., 2026)	`Qwen3.5-9B`	BM25 (`search+open+ret`)	`<= 32.0K`	—	`42.5%`
Octopus-Qwen3.5-4B (`step45`, Ours)	`Qwen3.5-4B`	BM25 (`search+open+tree`)	`329.2K` (`199.0K` net) / `9,919`	`43.3%` (`58.9% >=1`)	`49.6%` (`+17.5`)

Usage with Octopus & SGLang

Install the Octopus runtime and apply the SGLang cursor/mount and LoRA target patches (patches/apply_sglang_*.py, which map HuggingFace PEFT in_proj_qkv + in_proj_z onto SGLang's fused in_proj_qkvz):

git clone https://github.com/shatianming5/octopus.git
cd octopus
SGL=$(python3 -c "import pathlib, sglang; print(pathlib.Path(sglang.__file__).parent)")
for p in patches/apply_sglang_*.py; do python3 "$p" "$SGL"; done

Launch SGLang with Qwen/Qwen3.5-4B and Tommysha/Octopus-Qwen3.5-4B (or any checkpoint subdirectory under checkpoints/):

python3 -m sglang.launch_server \
  --model-path Qwen/Qwen3.5-4B \
  --lora-paths octopus=Tommysha/Octopus-Qwen3.5-4B \
  --max-lora-rank 64 \
  --lora-target-modules all \
  --disable-radix-cache false

Run the Octopus evaluation harness (see scripts/deploy.sh and README.md):
```
bash scripts/deploy.sh /data/octopus check
```

Citation

@article{sha2026octopus,
  title   = {Octopus: One Brain, Many Branches for Latent-KV Multi-Agent Reinforcement Learning},
  author  = {Sha, Tianming and Li, Junnan and Zhao, Siqi},
  journal = {arXiv preprint arXiv:2611.06956},
  year    = {2026},
  url     = {https://arxiv.org/abs/2611.06956}
}