aoiandroid/neutts-jp-150m-onnx-float16

neuTTS-JP-150m (ONNX float16)

Japanese TTS language-model backbone exported to ONNX with float16 weights (text-generation-with-past / KV-cache). This repository holds the shrunk ONNX graph (~half the size of FP32 weights) plus tokenizer files copied from the source checkpoint.

Scope: The ONNX file is the causal LM used inside a neuTTS-style pipeline. End-to-end speech also requires a codec (e.g. NeuCodec) and reference audio features outside this repo. For a full PyTorch smoke test pattern, see community scripts that combine AutoModelForCausalLM, tokenizer, and neuphonic/neucodec.

Repository layout

Files on the Hub follow this structure (paths are relative to the repo root):

.
├── README.md                    # Model card (this file)
├── .gitattributes               # Git LFS / Hub pointer rules
├── model_fp16.onnx              # ONNX model (FP16 weights), ~304 MiB
├── config.json                  # Transformers-style backbone metadata (LlamaForCausalLM)
├── generation_config.json       # Default generation settings (if present)
├── tokenizer.json               # Tokenizer vocabulary / merges
├── tokenizer_config.json
├── special_tokens_map.json
└── tokenization_llm_jp_tts.py   # Custom tokenizer module (trust_remote_code)

Not included in this repo

Artifact	Reason
model.onnx (FP32, ~580 MiB)	Kept out to save space; export FP32 locally with optimum.exporters.onnx if needed.

Model summary

Item	Value
Backbone	LlamaForCausalLM (custom Japanese TTS tokenizer)
ONNX task	text-generation-with-past
Weights dtype in ONNX	float16
Hidden size / layers	512 / 12 (see config.json)
Vocab size	99111

Download

huggingface-cli download aoiandroid/neutts-jp-150m-onnx-float16 --local-dir ./neutts-jp-150m-onnx-float16

Or with the Hub API:

from huggingface_hub import snapshot_download

snapshot_download(
    repo_id="aoiandroid/neutts-jp-150m-onnx-float16",
    local_dir="neutts-jp-150m-onnx-float16",
)

ONNX Runtime notes

This graph was produced with Optimum ONNX export, then weights were converted to FP16. When creating an InferenceSession, avoid GraphOptimizationLevel.ORT_ENABLE_ALL on some builds; prefer ORT_ENABLE_EXTENDED or ORT_ENABLE_BASIC if you hit optimizer failures (e.g. layer-norm fusion vs cast nodes).

Example:

import onnxruntime as ort

so = ort.SessionOptions()
so.graph_optimization_level = ort.GraphOptimizationLevel.ORT_ENABLE_EXTENDED
sess = ort.InferenceSession(
    "model_fp16.onnx",
    sess_options=so,
    providers=["CPUExecutionProvider"],
)

Export provenance (local reproducibility)

1. Export ONNX from the FP16 Hugging Face–style folder: optimum.exporters.onnx with --task text-generation-with-past and --dtype fp16.
2. Optional extra shrink / FP16 pass: convert FP32 ONNX to FP16 weights with onnxruntime.transformers.float16.convert_float_to_float16 (e.g. disable_shape_infer=True on Windows-friendly paths).

Citation

If you use this conversion, cite the upstream neuTTS / Neuphonic work and model licenses that apply to the original checkpoint you started from.

Maintainer: aoiandroid