# ABot-ER-9B

ABot-ER unifies embodied question answering, spatial reasoning, point grounding and task planning. This checkpoint uses the **Task-Conditioned Adaptive Training Strategy** described in the technical report: shared backbone projections with sequence-level, task-conditioned low-rank residuals.

## Checkpoint

- Final joint post-training checkpoint: 300.
- Residual layers (zero-based): [19, 23, 27, 31].
- Residual rank and alpha: 16.
- Adapted projections: query and value projections in selected full-attention blocks.
- Tokenizer, route-token IDs and tensor values are preserved from the evaluated checkpoint.

## Loading

Use a clean Python environment, install `huggingface_hub` for downloading, and install the dependencies from `requirements.txt`. Set `HF_REVISION` to this repository's immutable commit SHA. Authentication is needed only if the destination repository requires it. Review the included Python files before enabling custom code. The example below is a text-only loading and routing smoke test, not a visual benchmark example; real image/video inputs must follow the processor and benchmark contracts in the project documentation.

```python
import os
import sys
from huggingface_hub import snapshot_download

path = snapshot_download(repo_id="acvlab/ABot-ER-9B", revision=os.environ["HF_REVISION"])
sys.path.insert(0, path)
from abot_er import load_model, load_processor, route_messages

model = load_model(path, dtype='bfloat16', device_map='auto')
processor = load_processor(path)
messages = route_messages([
    {'role': 'user', 'content': [{'type': 'text', 'text': 'Describe the types of embodied questions you can answer.'}]}
], task_family='2d_qa')
inputs = processor.apply_chat_template(
    messages, tokenize=True, return_dict=True, return_tensors='pt',
    add_generation_prompt=True, enable_thinking=False,
).to(model.device)
output = model.generate(**inputs, max_new_tokens=128, do_sample=False)
print(processor.batch_decode(output[:, inputs['input_ids'].shape[1]:], skip_special_tokens=True))
```

Direct Transformers loading must use `AutoModelForImageTextToText.from_pretrained(path, trust_remote_code=True)`. Loading as an ordinary backbone without the included custom implementation would omit the conditional branches and is unsupported.

## Task routing

The first user message must contain exactly one route token:

- `[2D_QA]`: all embodied QA, including VABench-P and Where2Place point grounding.
- `[TASK_PLAN]`: task planning.

These are the actual tokenizer strings. Preserve them exactly; do not insert both. `route_messages` adds the marker without modifying the benchmark-specific prompt. Use one active request per model instance; concurrent interleaved decoding on a shared instance is not supported by the route cache. Use greedy/sampling generation with `num_beams=1`.

## Scope and terms

This card does not grant a new license or supersede the original backbone's applicable terms. The dataset is maintained separately in `acvlab/ABot-ER-Training-Data`; referenced image and video binaries are not bundled with this model.

See [LICENSE](./LICENSE) for component terms and [CITATION.bib](./CITATION.bib) for the technical report.