---
license: apache-2.0
base_model: upstage/Solar-Open-100B
tags:
- pruned
- compressed
- llm
---

# Solar-Open-100B-pruned-5pct

This model is a pruned version of [upstage/Solar-Open-100B](https://huggingface.co/upstage/Solar-Open-100B).

## Pruning Details

| Property | Value |
|----------|-------|
| Original Model | upstage/Solar-Open-100B |
| Original Parameters | 9.46B |
| Pruned Parameters | 3.09B |
| Compression Ratio | 0.3261 (3.1x smaller) |
| Strategy | Layer + Head + FFN (Recommended) |
| Importance Metric | magnitude |
| Layers Removed | 31 |
| FFN Reduction | 60.00% |

## Usage

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("llaa33219/Solar-Open-100B-pruned-5pct")
tokenizer = AutoTokenizer.from_pretrained("llaa33219/Solar-Open-100B-pruned-5pct")
```

## Notes

This model was created using structured pruning techniques including:
- Layer pruning (removing entire transformer layers)
- Attention head pruning (reducing number of attention heads)
- FFN dimension pruning (reducing intermediate layer sizes)

The pruning was based on magnitude importance scoring to preserve the most important weights.