s1ghhh/Llama-2-13b-Drop8Block
13B • Updated • 15 • 2
LLM-Drop
Model weights of paper "Uncovering the Redundancy in Transformers via a Unified Study of Layer Dropping (TMLR)".
-
-
s1ghhh/Llama-2-13b-Drop4Block
13B • Updated • 2 • 2 -
s1ghhh/Llama-2-13b-Drop4Attn
13B • Updated • 2 • 2 -
s1ghhh/Llama-2-13b-Drop8Attn
13B • Updated • 2 • 2 -
s1ghhh/Llama-2-13b-Drop4MLP
13B • Updated • 2 • 2 -
s1ghhh/Llama-2-13b-Drop8MLP
13B • Updated • 1 • 2 -
s1ghhh/Mistral-7B-v0.1-Drop4Block
7B • Updated • 2 • 2 -
s1ghhh/Mistral-7B-v0.1-Drop8Block
7B • Updated • 2 • 2 -
s1ghhh/Mistral-7B-v0.1-Drop4Attn
7B • Updated • 2 • 2 -
s1ghhh/Mistral-7B-v0.1-Drop8Attn
7B • Updated • 2 • 2 -
s1ghhh/Mistral-7B-v0.1-Drop4MLP
7B • Updated • 1 • 2 -
s1ghhh/Mistral-7B-v0.1-Drop8MLP
7B • Updated • 2 • 2 -
s1ghhh/Llama-2-70b-Drop
Text Generation • Updated • 2 • 2 -
s1ghhh/Llama-3-70b-Drop
Text Generation • 71B • Updated • 14 • 4 -
What Matters in Transformers? Not All Attention is Needed
Paper • 2406.15786 • Published • 32 -
Making Large Language Models Efficient Dense Retrievers
Paper • 2512.20612 • Published • 3 -
Understanding and Harnessing Sparsity in Unified Multimodal Models
Paper • 2512.02351 • Published • 4 -
Demystifying When Pruning Works via Representation Hierarchies
Paper • 2603.24652 • Published • 19