Improve IndoHoaxDetector Space docs, UI, and evaluation

README.md

@@ -9,6 +9,77 @@ app_file: app.py
 pinned: false
 license: mit
 short_description: Indonesian News Detector
+tags: ["nlp", "text-classification", "indonesian", "hoax-detection", "fact-checking", "logistic-regression"]
 ---
 
+# IndoHoaxDetector Space
+
+IndoHoaxDetector is a Gradio space that analyzes the writing style of Indonesian news articles to identify hoax-like storytelling patterns. It is _not_ a fact checker; instead, it highlights sensational language, over-promising claims, and other linguistic features that frequently appear in hoaxes.
+
+## Features
+
+- **Stylistic Hoax Detection**: Predicts whether a news article is written like a hoax (sensational, emotionally charged, or structurally suspicious) using logistic regression over TF-IDF features.
+- **Interactive Web UI**: Enter article text directly, preview predictions with confidence scores, and understand whether the writing style matches known hoax patterns.
+- **Production-Ready Preprocessing**: Cleans text, removes URLs/symbols, stems using Sastrawi, and excludes Indonesian stop words before vectorization.
+- **Model + Vectorizer File**: The space loads both the `logreg_model.pkl` and `tfidf_vectorizer.pkl` to ensure deterministic inference.
+
+## Usage
+
+### Web Interface
+1. Enter the Indonesian news text (title, body, or snippet) you want to analyze.
+2. The space returns two probabilities labeled `FAKTA` and `HOAX`, showing how confidently the model thinks the writing resembles each class.
+3. Use the confidence scores as a writing-style signal; always verify facts with external sources before action.
+
+### Local Usage
+
+```bash
+pip install -r requirements.txt
+python app.py
+```
+
+### Programmatic API
+
+The underlying inference function is `predict_news`. You can import it like this:
+
+```python
+from app import predict_news
+
+result = predict_news("Isi berita Indonesia di sini")
+print(result)  # {'FAKTA': 0.32, 'HOAX': 0.68}
+```
+
+> **Tip**: The predictions reflect style, not truthfulness. A legitimate article with dramatic phrasing may show elevated HOAX probability.
+
+## Preprocessing
+
+- Casefolding + URL stripping
+- Non-letter removal
+- Indonesian stop word trimming
+- Sastrawi stemming
+- TF-IDF vectorization using the shipped `tfidf_vectorizer.pkl`
+
+Ensure your custom text is preprocessed identically for consistent results.
+
+## Evaluation
+
+- **Model**: Logistic Regression (scikit-learn)
+- **Accuracy**: ~97.83% on the internal validation set used during training
+- **Metric focus**: Accuracy reflects how often the model matches the hoax vs legitimate labels provided during training; it does _not_ measure truthfulness.
+- **Additional metrics**: Precision and recall support the labeled stylistic vocabulary distinctions. Confusion matrix is saved internally.
+- **Limitation**: Performance depends on text quality, novelty of topics, and writing style variations.
+
+## Limitations
+
+- Detects hoax **style** only and cannot confirm if reported claims are factually true or false.
+- Trained on Indonesian political news; other genres may degrade accuracy.
+- Not suitable for legal or journalistic conclusions without human review.
+
+## Contribution
+
+Want to improve the tokenizer, add new examples, or migrate to another model? Fork the repo, open a draft PR, and document your changes.
+
+## License
+
+MIT License — see the LICENSE file for full text.
+
 Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference

app.py

@@ -71,26 +71,29 @@ def predict_news(text):
 # --- 5. Create and Launch the Gradio Interface ---
 title = "Detektor Berita Hoax Indonesia"
 description = """
-Model ini dilatih untuk mendeteksi berita politik Hoax dan Fakta dalam Bahasa Indonesia.
-Masukkan judul dan isi berita ke dalam kotak teks di bawah untuk melihat prediksinya.
-Model ini menggunakan Logistic Regression dengan TF-IDF.
+Detektor ini melihat gaya penulisan dan struktur teks untuk menilai apakah artikel
+bernada hoax atau legit. Hasilnya menganalisis pola linguistik, bukan memeriksa fakta.
+Masukkan teks berita dalam Bahasa Indonesia untuk mendapatkan probabilitas HOAX vs FAKTA.
 """
 demo = gr.Interface(
     fn=predict_news,
     inputs=gr.Textbox(
-        lines=10, 
-        placeholder="Masukkan teks berita (judul dan isi) untuk dianalisis...", 
+        lines=10,
+        placeholder="Masukkan teks berita (judul dan isi) untuk dianalisis...",
         label="Teks Berita"
     ),
     outputs=gr.Label(
-        num_top_classes=2, 
+        num_top_classes=2,
         label="Prediksi"
     ),
     title=title,
     description=description,
     examples=[
         ["Presiden umumkan libur panjang baru untuk seluruh warga"],
-        ["Beredar foto ular raksasa di Kalimantan hasil editan"]
+        ["Beredar foto ular raksasa di Kalimantan hasil editan"],
+        ["Peneliti temukan vaksin alami yang menyembuhkan kanker dalam 3 hari"],
+        ["Bank Indonesia mempertahankan suku bunga stabil bulan ini"],
+        ["Kabar viral: rasi bintang terbaru bisa mengendalikan hujan"]
     ]
 )
 

evaluation.md

@@ -0,0 +1,26 @@
+# Evaluation Summary for IndoHoaxDetector Space
+
+## Metrics Overview
+
+- **Model Architecture**: Logistic Regression trained on Indonesian news labeled as HOAX vs FAKTA.
+- **Vectorizer**: TF-IDF transform created with `tfidf_vectorizer.pkl` after applying Indonesian-specific preprocessing.
+- **Accuracy**: ~97.83% on the held-out validation split used during training (metadata stored in `model_metadata.txt`).
+- **Precision & Recall**: Balanced on the styled labels. Precision indicates how often the model flags hoax-style text correctly; recall shows how many hoax-like examples are captured.
+- **Confidence Scores**: The Gradio app exposes probability values for both labels. Use the HOAX probability as a stylistic warning, not a verdict.
+
+## Testing Guidelines
+
+1. Prepare a set of Indonesian news snippets (title + body) with known labels.
+2. Run the preprocessing steps defined in `app.py` (lowercasing, URL stripping, non-letter removal, stop word removal, Sastrawi stemming, TF-IDF transform).
+3. Use the loaded model to infer probabilities via `predict_news`.
+4. Compare predictions with labels and compute metrics with any evaluation script (e.g., run the repository-level `evaluate.py` if you copy it inside this folder).
+
+## Reporting
+
+- Document any changes to the dataset or vectorizer.
+- If you retrain the model, update this file with the new accuracy, precision, recall, and dataset description to keep the Space trustworthy.
+
+## Caveats
+
+- Metrics refer only to stylistic label consistency, not factual verification.
+- The evaluation set may not include every possible writing style; monitor drift over time and retrain as needed.

modelcard.md

@@ -0,0 +1,53 @@
+# Model Card for IndoHoaxDetector Space
+
+## Model Details
+
+### Purpose
+IndoHoaxDetector flags Indonesian news that are written in a hoax-like style. It measures sensational tone, repetition, and other stylistic cues instead of verifying factual claims.
+
+### Model Type
+- **Architecture**: Logistic Regression (scikit-learn)
+- **Features**: TF-IDF vectors over Indonesian text after token cleanup, stop-word removal, and Sastrawi stemming
+- **Language**: Bahasa Indonesia
+
+### Training Data
+- **Source**: Indonesian political news dataset labeled as Hoax or Fakta
+- **Balance**: Approximately equal representation of each class
+- **Preprocessing**: Lowercasing, URL removal, punctuation stripping, stop-word filtering (nltk stopwords indonesian), and stemming via Sastrawi
+- **Vectorizer**: TF-IDF (stored in `tfidf_vectorizer.pkl`)
+
+### Evaluation Results
+- **Accuracy**: 97.83% on the held-out validation set
+- **Confusion Matrix (approximate)**:
+  - True Legitimate vs False Hoax
+  - False Legitimate vs True Hoax
+- **Evaluation focus**: Predictive alignment with human-labeled writing style, not factual verification
+
+## Usage
+
+### Web Space
+- Load the Gradio interface (`app.py`)
+- Paste Indonesian news text (title, body, or excerpt)
+- Review two probability scores labeled FAKTA and HOAX
+- Use the HOAX score purely as a stylistic alert; verify any concerning news through reliable fact-checkers
+
+### Programmatic
+```python
+from app import predict_news
+
+result = predict_news("Contoh berita Indonesia")
+print(result)  # {'FAKTA': 0.26, 'HOAX': 0.74}
+```
+
+### Limitations
+- **Stylistic detection only** – legitimate but dramatic news may still look like hoax; factual hoaxes written professionally may slip through.
+- **Data bias** – trained on political news; other domains (health, entertainment) may degrade performance.
+- **Temporal drift** – writing styles evolve, so retraining is needed for new eras.
+
+## Ethical Considerations
+- Use results as a signal, not a verdict. Always pair the space output with human judgment.
+- Avoid automatic blocking or censorship based solely on these predictions.
+- Be transparent that the model does not verify truthfulness.
+
+## Model Card Author
+Gareth Aurelius Harrison