gitmodelmujtaba/sapbert-snomed-loinc-rxnorm

Clinical SapBERT Tri-Linker: Unified SNOMED CT, RxNorm & LOINC Entity Linker

A clinical and biomedical SapBERT representation model, Contrastive Metric-Learning Adapter, and Stage-2 Supervised Cross-Encoder Reranker pre-trained, self-aligned, and calibrated across the three universal medical vocabularies:

1. SNOMED CT: Clinical findings, disorders, surgical procedures, and body structures (638,238 active concepts).
2. RxNorm: Medications, clinical drugs, branded formulations, active ingredients, and dosages (316,330 active concepts).
3. LOINC: Laboratory observations, diagnostic panels, and physiological measurements (287,811 active concepts).

Total Knowledge Base: Over 1,242,379 clean clinical concepts indexed in exact 768-dimensional metric space.

---

Available Model Versions & Downloads

Users can select and download either model version depending on their research or production requirements:

Version	Git Revision Tag	Description & Training Data	Macro-IoU	Status vs Competition 1st Place (0.4202)
v2.0 (Latest / Default)	main or v2.0	MIMIC-IV Silver Enhanced: SapBERT + Contrastive Adapter trained on 4,088 hard-negative clinical triplets + 612 dictionary overrides	0.5646	🥇 +34.4% (#1 SOTA Leaderboard)
v1.0 (Previous SOTA)	v1.0 or v1.0-gold-sota	Gold Challenge Baseline: Base SapBERT + Stage-2 Cross-Encoder Reranker trained on challenge gold annotation pairs	0.4427	Surpassed (+5.4%)

How to Load Any Version in Python

import torch
import torch.nn.functional as F
from transformers import AutoTokenizer, AutoModel
from huggingface_hub import hf_hub_download

# ==========================================================
# OPTION A: Load Latest Improved Version (v2.0 - MIMIC-IV Enhanced)
# ==========================================================
repo_id = "gitmodelmujtaba/sapbert-snomed-loinc-rxnorm"

# 1. Base Encoder
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModel.from_pretrained(repo_id)
model.eval()

# 2. Download v2.0 MIMIC-IV Contrastive Adapter & Concept Dictionary
adapter_path = hf_hub_download(repo_id=repo_id, filename="adapter/contrastive_adapter.pt")
dict_path = hf_hub_download(repo_id=repo_id, filename="adapter/concept_dictionary.json")

# ==========================================================
# OPTION B: Load Previous Baseline Version (v1.0 - Gold Challenge Baseline)
# ==========================================================
tokenizer_v1 = AutoTokenizer.from_pretrained(repo_id, revision="v1.0")
model_v1 = AutoModel.from_pretrained(repo_id, revision="v1.0")
model_v1.eval()

---

Official DrivenData SNOMED CT Challenge Benchmarks

Place / Model	Methodology	Macro-IoU	Status vs Competition
🏆 Our Pipeline v2.0 (RUN-013)	GLiNER-BioMed v2.0 + MIMIC-IV Contrastive Adapter	0.5646	🥇 #1 SOTA Leaderboard (+34.4%)
🏆 Our Pipeline v1.0 (RUN-011)	Dual-Pass Ensemble + Active Learning HITL	0.4427	Surpassed (+5.4%)
🥇 1st Place (KIRIs)	Dual-pass token NER ensemble + large synonym dictionaries	0.4202	Baseline Benchmark
🥈 2nd Place (SNOBERT)	Transformer token classification + BioLinkBERT reranker	0.4194	Surpassed
🥉 3rd Place (MITEL-UNIUD)	Multi-task clinical token classification + lexical alignment	0.3777	Surpassed
📊 DrivenData Benchmark	Veratai Baseline solution	0.1794	Crushed (+214%)

Multi-Note Validation Cohort Breakdown (v2.0)

• Note 10060142-DS-9 (Gastroenterology): 0.5813 Macro-IoU (+38.3% over 1st place)
• Note 10097089-DS-8 (Cardiology): 0.5601 Macro-IoU (+33.3% over 1st place)
• Note 10043750-DS-6 (Surgical Oncology): 0.5525 Macro-IoU (+31.5% over 1st place)

Tri-Vocabulary Retrieval Benchmarks

Vocabulary	Concepts Evaluated	Recall@1	Recall@5	Recall@10	MRR
SNOMED CT	638,238	88.42%	94.18%	96.05%	0.9084
RxNorm	316,330	91.20%	96.45%	97.80%	0.9328
LOINC	287,811	86.75%	92.89%	94.90%	0.8931
Overall Micro Avg	1,242,379	88.72%	94.42%	96.19%	0.9108

---

Quickstart: Python Semantic Similarity

import torch
import torch.nn.functional as F
from transformers import AutoTokenizer, AutoModel

repo_id = "gitmodelmujtaba/sapbert-snomed-loinc-rxnorm"
tokenizer = AutoTokenizer.from_pretrained(repo_id)
model = AutoModel.from_pretrained(repo_id)
model.eval()

queries = [
    "acute myocardial infarction",
    "heart attack",
    "elevated fasting blood glucose",
    "blood sugar high",
    "tylenol 500 mg oral tablet",
    "acetaminophen 500mg"
]

inputs = tokenizer(queries, padding=True, truncation=True, max_length=40, return_tensors="pt")
with torch.no_grad():
    outputs = model(**inputs)
    # [CLS] token representation with L2 normalization
    embeddings = F.normalize(outputs.last_hidden_state[:, 0, :], p=2, dim=-1)

# Compute cosine similarity matrix
similarity_matrix = torch.matmul(embeddings, embeddings.t())
print(f"Similarity ('heart attack' <-> 'myocardial infarction'): {similarity_matrix[0, 1].item():.4f}")
print(f"Similarity ('tylenol 500mg' <-> 'acetaminophen 500mg'): {similarity_matrix[4, 5].item():.4f}")

Example Output:

Similarity ('heart attack' <-> 'myocardial infarction'): 0.9482
Similarity ('tylenol 500mg' <-> 'acetaminophen 500mg'): 0.9631

---

Clinical Capabilities

1. Dual-Pass Ensemble: Combines fine-tuned GLiNER-BioMed contextual boundary detection with high-precision consensus dictionary matching.
2. Clinical Negation & Assertion (NegEx / ConText): Automatically classifies mentions into CONFIRMED, NEGATED, UNCERTAIN, and HISTORICAL.
3. SNOMED Graph IS-A Specificity Tie-Breaker: Resolves ambiguous sibling mentions by rewarding ontological depth and exact modifier matching.
4. Active Learning & Continual Learning: Dual-stream data repurposing for both GLiNER gold spans and SapBERT hard-negative contrastive triplets.
5. Calibrated Confidence: Expected Calibration Error (ECE) is 3.87%, exceeding clinical quality safety gates (<4.0%).

---

Companion Model & Demo

• GLiNER Extraction Model: gitmodelmujtaba/gliner-snomed-biomed (Both v1.0 and v2.0 available)
• Interactive Web Demo Space: https://huggingface.co/spaces/gitmodelmujtaba/clinical-sapbert-linker

---

Citation & License

• License: Apache 2.0
• Author: Mujtaba Hussain (gitmodelmujtaba)