Irfanuruchi/Qwen3-4B-Computer-Science-OpenVINO-FP16

Qwen3-4B Computer Science — OpenVINO FP16

FP16 OpenVINO export of Irfanuruchi/Qwen3-4B-Computer-Science.

This repository provides the fine-tuned Qwen3-4B model in OpenVINO Intermediate Representation (IR) format without INT8 or INT4 weight compression.

The model was converted and benchmarked natively on IULinux using OpenVINO 2026.4 and Optimum Intel.

The source BF16 checkpoint was converted to FP16. This is a deployment conversion, not an additional fine-tuning run.

Model Details

Property	Value
Architecture	Qwen3-4B
Parameters	Approximately 4 billion
Source model	Irfanuruchi/Qwen3-4B-Computer-Science
Source precision	BF16
Export precision	FP16
Weight compression	None
Format	OpenVINO IR
OpenVINO version	2026.4
Model size	Approximately 7.6 GB
License	Apache 2.0

Conversion

Exported using Optimum Intel:

optimum-cli export openvino \
  --model Irfanuruchi/Qwen3-4B-Computer-Science \
  --task text-generation-with-past \
  --weight-format fp16 \
  Qwen3-4B-Computer-Science-OpenVINO-FP16

Installation

pip install openvino optimum-intel transformers

Inference

from transformers import AutoTokenizer
from optimum.intel import OVModelForCausalLM

model_id = "Irfanuruchi/Qwen3-4B-Computer-Science-OpenVINO-FP16"

tokenizer = AutoTokenizer.from_pretrained(model_id)

model = OVModelForCausalLM.from_pretrained(
    model_id,
    device="CPU",
    ov_config={"PERFORMANCE_HINT": "LATENCY"}
)

messages = [
    {
        "role": "user",
        "content": "Explain binary search and implement it in Python."
    }
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False
)

inputs = tokenizer(prompt, return_tensors="pt")

outputs = model.generate(
    **inputs,
    max_new_tokens=256,
    do_sample=False
)

generated = outputs[0][inputs["input_ids"].shape[1]:]

print(tokenizer.decode(generated, skip_special_tokens=True))

Hardware Validation

The exported model was tested natively on IULinux with the following hardware and software configuration:

Component	Specification
Computer	ASUS ROG Strix G16
Processor	Intel Core Ultra 9 275HX
Operating system	IULinux
Python environment	Python 3.14
Runtime	OpenVINO 2026.4
Inference backend	Intel CPU
Power configuration	AC, performance profile

CPU inference was successfully tested.

OpenVINO also detected Intel GPU and NPU devices on the test system. Inference compatibility with those devices has not been validated for this export.

Performance

CPU Benchmark

The following prompt was used:

Write a Python function is_prime(n) using an efficient algorithm. Explain its complexity.

Both versions generated 160 tokens using greedy decoding.

Variant	Model size	Time	Throughput
OpenVINO INT8	3.8 GB	10.17 s	15.73 tok/s
OpenVINO FP16	7.6 GB	20.79 s	7.70 tok/s

Methodology

• Intel Core Ultra 9 275HX
• IULinux
• AC power
• Performance power profile
• OpenVINO CPU backend
• OpenVINO LATENCY performance hint
• 16-token warm-up generation
• 160-token measured generation
• Greedy decoding

Under these test conditions, the INT8 version delivered approximately 2.04 times the average generation throughput of FP16.

Results are single-run measurements and include end-to-end generation overhead. They are not statistically controlled benchmarks.

Functional Validation

The FP16 model successfully generated a Python primality-testing function using square-root-bounded trial division.

The implementation has O(sqrt(n)) time complexity for ordinary supported integer inputs.

This confirms successful inference for the tested prompt. It does not establish broader accuracy or reliability.

Available Precisions

Precision	Repository
BF16 source	Qwen3-4B-Computer-Science
OpenVINO INT4	Qwen3-4B-Computer-Science-OpenVINO-INT4
OpenVINO INT8	Qwen3-4B-Computer-Science-OpenVINO-INT8
OpenVINO FP16	This repository

Intended Use

• Computer science education
• Programming and algorithm assistance
• Software engineering
• Debugging and code explanation
• Local language-model inference
• OpenVINO precision comparisons
• Model deployment research

Limitations

• FP16 conversion changes the original BF16 numerical representation.
• Higher weight precision does not guarantee better responses.
• Model outputs may contain errors or insecure code.
• Generated code should be independently reviewed and tested.
• Comprehensive accuracy and regression evaluations have not been completed.
• Performance depends on hardware, power settings, prompt length, and runtime configuration.
• Intel GPU and NPU execution have not been validated.
• Results from the tested system may not generalize to other hardware or operating systems.

References

• Fine-tuned source checkpoint
• Qwen3
• OpenVINO
• Optimum Intel

License

Apache License 2.0.

Refer to the source model repository for training details and additional limitations.