AGILLM-4 / AGILLM-4.md
OpenTransformer's picture
AGILLM4_training_script_and_bounded_uploads
60d9240
|
Raw History Blame Contribute Delete
12.9 kB

AGILLM-4 Starting Point

AGILLM-4 should not be "AGILLM-3 with a larger --block." The next useful version should split context into layers of memory:

  1. Local dense attention for exact nearby reasoning.
  2. Learned anchor tokens that summarize spans while staying trainable end-to-end.
  3. Compressive recurrent memory for information that must persist beyond the dense window.
  4. Retrieval-style recall over previous anchors, so million-token behavior is approached as memory lookup plus local reasoning rather than dense all-pairs attention.

The near-term codebase target is a measurable long-context trainer that can survive on current hardware:

  • RTX 4090: production 24GB lane for the >1B AGILLM-4 floor shape, then block-size growth while preserving AR+SAT+NAT.
  • B200 180GB: serious 2k-16k experiments.
  • B300 262GB: 8k-64k experiments, then memory-augmented tests.
  • Multi-GPU/cluster: ring or sequence parallel experiments later.

Parameter Target

AGILLM-3 is about 700M dense parameters, so AGILLM-4 should start above that rather than treating 700M as the proving tier.

Implemented presets:

Preset Shape Approx params Use
agillm4_floor d=1280, L=28, H=20, rank=160 ~1.21B minimum AGILLM-4 run
agillm4_main d=1536, L=32, H=24, rank=192 ~1.70B main target
agillm4_big d=1792, L=36, H=28, rank=224 ~2.40B stretch target after memory works

Default recommendation: train agillm4_main if B200/B300 availability is good. On a 24GB 4090, start with agillm4_floor so the run is still larger than AGILLM-3 while leaving enough VRAM for AR+SAT+NAT.

4090 Production Long-Block Plan

The RTX 4090 lane is production. Its job is to keep useful AGILLM-4 training moving when only 24GB VRAM is available without accidentally dropping back to the AGILLM-3-sized large preset.

Production first-run recipe on 4090:

AGILLM4_4090_WARMSTART_FROM=/workspace/agillm-4/agillm4_floor_seed_from_v3_v47.pt \
AGILLM4_4090_PRESET=agillm4_floor \
AGILLM4_4090_BLOCK=1280 \
AGILLM4_4090_TOKEN_PARAM_RATIO=100 \
tmux new-session -d -s agillm4_floor \
  /workspace/agillm-4/launch_agillm4_4090_floor_from_v47.sh

Important: --sat_every 1 --nat_every 4 keeps SAT trained every step and NAT active on a cadence that fits 24GB cards. On B200/B300 use --nat_every 1 for full AR+SAT+NAT every step. The AGILLM-4 code now backprops AR, SAT, and NAT sequentially, so the objective remains joint while peak VRAM is lower than holding all activation graphs at once.

Escalation ladder on 4090:

  1. Start block=1280.
  2. If OOM, back off by about 20% at a time (1024, 768, ...), not straight to half.
  3. If stable below 23GB VRAM after real step timing, raise toward 1536.

If 8-bit optimizer is unavailable, install bitsandbytes rather than dropping the long-block target. SAT remains active every step; NAT should stay enabled with a slower cadence or --nat_max_tokens cap on 24GB. The code lowers peak memory by backpropagating AR, SAT, and NAT sequentially, not by deleting heads.

For the year-scale full run, keep checkpoint retention bounded. The 4090 launcher keeps one local full checkpoint and one local delta. The companion uploader publishes status/log tails every 30 minutes, uploads the newest delta at most daily, uploads full checkpoints at most weekly, and prunes remote AGILLM-4 training uploads to the configured rolling window.

Warm Start from AGILLM-3 (function-preserving)

The next AGILLM-4 run does not start from a random init. build_v4_seed.py turns the most recent AGILLM-3 SFT checkpoint into a function-preserving v4 seed:

python /workspace/agillm-4/build_v4_seed.py \
  --from-ckpt /workspace/ckpts_sat_fixed_polish_v47_20260523/final.pt \
  --v4-preset floor \
  --out /workspace/agillm-4/agillm4_floor_seed_from_v3_v47.pt

Construction rules (v3 d=1024 / H=16 / r=128 / L=24 → v4 d=1536 / H=24 / r=192 / L=32):

  • Embedding, AR head, and NAT head pad with zeros in new d_v4 - d_v3 columns. New-dim embeddings are zero — they propagate harmlessly through the rest of the graph because all paths that touch them are zero-padded too.
  • If the v3 checkpoint has no NAT head, build_v4_seed.py initializes NAT from the expanded AR projection rather than random weights.
  • Block params: qkv, proj, ff.0, ff.2 are placed top-left into the larger v4 tensors; the rest is zero. Old heads contribute exactly the same; new heads' attention output is multiplied by zero proj columns, so their contribution is zero at step 0.
  • U rank 128 → 192 by appending zero columns. U @ U.T is unchanged, so the M-fold metric is bit-identical.
  • LayerNorm new dims initialise to weight=1, bias=0.
  • Extra layers 25..32 are identity-init: mha.proj and ff.2 are zero, so the residual stream passes through unchanged. They start contributing only once training shapes them.

Net: at step 0, v4(x) == v3(x) up to floating-point rounding for any input within v3's tokeniser vocabulary. The new capacity has zero impact on day-1 loss; training subsequently shapes it.

Use with --warmstart_from:

python /workspace/agillm-4/nB300_agillm4.py train \
  --preset agillm4_floor \
  --warmstart_from /workspace/agillm-4/agillm4_floor_seed_from_v3_v47.pt \
  --batch_size 1 --block 1280 --amp --grad_checkpoint --sat_every 1 --nat_every 4 \
  --token_param_ratio 100

The seed file is several GB on disk (fp32 tensors). Rebuild it whenever a newer v3 final is preferred. For B200/B300, use --v4-preset main and train agillm4_main; for 4090, use --v4-preset floor.

First Implemented Scaffold

nB300_agillm4.py starts from the current chat-SFT-fixed trainer and adds:

  • --attn_backend manual|sdpa
  • --grad_checkpoint
  • AGILLM-4 defaults for chat SFT compatibility inherited from the patched AGILLM-3 file.
  • A real NAT head is re-added beside AR and SAT. NAT trains with a CTC loss over an expanded non-causal input (--nat_expand 2 by default). --nat_every controls cadence; --nat_max_tokens can cap the NAT slice on 24GB runs without removing the head.

block_sweep_agillm4.py runs synthetic forward/backward sweeps:

python /workspace/agillm-4/block_sweep_agillm4.py \
  --preset agillm4_main \
  --blocks 768,1024,1536,2048,3072,4096 \
  --batch_size 1 \
  --attn_backend sdpa \
  --grad_checkpoint \
  --amp \
  --json_out /workspace/agillm4_block_sweep.json

This gives real VRAM and throughput data before committing to long training.

profile_agillm4.py breaks a synthetic AR+SAT step into timed pieces:

python /workspace/agillm-4/profile_agillm4.py \
  --preset agillm4_floor \
  --block 512 \
  --batch_size 1 \
  --backends sdpa,sublinear \
  --grad_checkpoint \
  --amp \
  --json_out /workspace/agillm4_floor_profile_512.json

Use this before changing architecture. The profiler reports AR core forward, AR head/loss, AR backward, SAT core forward, SAT head/loss, SAT backward, gradient clipping, optimizer step, VRAM peaks, and synthetic tokens/sec.

Sublinear Attention Backend

nB300_agillm4.py now has an opt-in experimental backend:

--attn_backend sublinear \
--sublinear_window 256 \
--sublinear_stride 64 \
--sublinear_max_anchors 256 \
--sublinear_chunk 128

This is still transformer attention, but each query reads a local band plus landmark candidates instead of every token. Candidate count is approximately:

2 * sublinear_window + 1 + min(block / sublinear_stride, sublinear_max_anchors)

So the attention score path scales as O(N * candidates) instead of O(N^2). This does not remove the cost of QKV projections, FFN layers, heads, or the second SAT forward/backward, so the profiler is mandatory. The point is to make long block sizes trainable while preserving exact local reasoning and a learned route to distant evidence.

On a 4090 production lane, first probe:

python /workspace/agillm-4/block_sweep_agillm4.py \
  --preset agillm4_floor \
  --blocks 512,640,768,1024,1280 \
  --batch_size 1 \
  --attn_backend sublinear \
  --sublinear_window 256 \
  --sublinear_stride 64 \
  --sublinear_max_anchors 256 \
  --sublinear_chunk 64 \
  --grad_checkpoint \
  --amp \
  --json_out /workspace/agillm4_sublinear_4090_sweep.json

If it is stable, then compare it against SDPA at the same block size with profile_agillm4.py.

n1.py Harvest

AGILLM-4 is now starting to import exact, proof-backed improvements from C:\Users\Scott\Downloads\n1.py while keeping the AGILLM-4 model-scale and long-context branch intact.

First harvested feature: exact M-fold expansion attention. When rank > d_k, the trainer now uses:

(q @ U) @ (k @ U).T == q @ (U @ U.T) @ k.T

This preserves the function while keeping score/cache key width at d_k instead of rank. The inference path caches U @ U.T; the training path recomputes it so gradients through U remain exact.

Verification:

python /workspace/agillm-4/verify_m_fold_agillm4.py \
  --presets pico_1x,micro_3x \
  --backends manual,sdpa

Second harvested feature: fused QKV projection. AGILLM-4 now uses one Linear(d, 3d) and chunks the result instead of running three separate Linear(d, d) projections. Legacy q.weight/k.weight/v.weight checkpoints load into the fused qkv.weight layout, and matching legacy AdamW moments are remapped on full resume.

Verification:

python /workspace/agillm-4/verify_qkv_agillm4.py \
  --presets pico_1x,micro_3x \
  --backends manual,sdpa,sublinear

See N1_HARVEST.md for the staged port order.

Intelligence per FLOP

Compute reduction is not enough by itself. AGILLM-4 should spend saved FLOPs on training signals that force memory and reasoning:

  • long needle retrieval with many plausible distractors
  • multi-hop chain following over distant records
  • long chat continuity
  • codebase-scale symbol tracking
  • proof dependency recall

long_context_curriculum.py generates JSONL text examples for the first two:

python /workspace/agillm-4/long_context_curriculum.py \
  --out /workspace/agillm-4/data/memory_torture_8k.jsonl \
  --examples 20000 \
  --slots 96 \
  --chain_len 12 \
  --distractor_words 7000

Train it as a small mixed-in source, not as the whole diet. The goal is not memorizing synthetic strings; the goal is punishing models that pretend to use long context while actually relying on local guesses.

AGILLM-4 Research Bets

Anchor Memory

Every anchor_stride tokens, produce an anchor vector from the local block. Future blocks attend to recent anchors in addition to local tokens.

Initial hypothesis:

  • Dense local window: 2k-8k.
  • Anchor stride: 256 or 512.
  • Anchor memory length: 512-4096 anchors.
  • Token attention stays cheap while memory covers hundreds of thousands of tokens.

Status: wired into Encoder as a single AnchorMemoryLayer inserted after a configurable mid-stack block. The module is the existing anchor_memory.AnchorMemoryLayer (attention-pooled compressor + gated cross-attention read from a rolling bank of max_anchors anchors). Enable per-run with:

python /workspace/agillm-4/nB300_agillm4.py train \
  --preset agillm4_floor --batch_size 1 --block 512 --amp --grad_checkpoint \
  --anchor_memory --anchor_stride 256 --anchor_max 2048

--anchor_position N overrides the default mid-stack placement. Verifier: verify_anchor_memory_and_kv_buffer_agillm4.py.

Compressive Recurrence

Keep a recurrent memory bank updated by a learned compressor:

new_memory = f(old_memory, anchor_batch)

The model does not need to carry all old tokens; it carries a trainable summary that is repeatedly tested by retrieval and continuation losses.

Memory Torture Curriculum

Long context must be trained with tasks that punish fake recall:

  • needle retrieval at 8k, 32k, 128k, 512k
  • multi-needle retrieval with distractors
  • "use detail from section A to solve section Z"
  • long chat continuity
  • codebase-scale symbol tracking
  • synthetic theorem/proof dependency chains

Weird But Worth Trying

  • Let SAT heads generate candidate anchor summaries, not just next tokens.
  • Use anchors as a second vocabulary of "thought tokens" with their own loss.
  • Train a small memory critic that predicts whether answer evidence exists in local tokens, anchors, or compressed memory.
  • Make retrieval differentiable during training but replaceable with fast approximate search at inference.

Go / No-Go Gates

AGILLM-4 earns real training only if:

  • Chat SFT improves AGILLM-3 behavior.
  • SDPA + checkpointing produces a clean block sweep.
  • 2k+ block training is stable on B200/B300.
  • Anchor-memory prototype beats plain local context on synthetic long-recall tests.

If chat SFT fails, AGILLM-4 still proceeds, but as an architecture correction rather than a scale-up.