Hugging Face's logo

Debk
/
AgriFair-GRAFT-adapters

PEFT
Safetensors
English
fairness
bias-mitigation
lora
agriculture
india
difference-awareness
Model card Files
xet
 Community
Debk commited on
Commit
3f0ca6c
·
verified ·
1 Parent(s): 3396506

AgriFair GRAFT adapters: 119 adapters

Browse files
Files changed (8) hide show
  1. README.md +3 -3
  2. general-instruct-2/baseline_fairnet/seed_43/README.md +207 -0
  3. general-instruct-2/baseline_fairnet/seed_43/adapter_config.json +548 -0
  4. general-instruct-2/baseline_fairnet/seed_43/adapter_model.safetensors +3 -0
  5. general-instruct/reference_vanilla_qlora/seed_42/README.md +207 -0
  6. general-instruct/reference_vanilla_qlora/seed_42/adapter_config.json +50 -0
  7. general-instruct/reference_vanilla_qlora/seed_42/adapter_model.safetensors +3 -0
  8. manifest.json +33 -1
README.md CHANGED
@@ -29,7 +29,7 @@ census records a real difference, using Integrated Gradients. It places LoRA ada
29
  layers alone with rank proportional to attribution, and trains them with a condition-adaptive,
30
  rationale-aware objective.
31
 
32
- This repository holds **117 adapters** so that every number in the paper can be
33
  recomputed rather than taken on trust.
34
 
35
  ## Layout
@@ -42,8 +42,8 @@ manifest.json base model, revision, rank range and trainable sha
42
  | Tier | Base model | Adapters |
43
  |---|---|---|
44
  | `broad-instruct` | Qwen/Qwen3-4B-Instruct-2507 | 33 |
45
- | `general-instruct` | google/gemma-3-12b-it | 32 |
46
- | `general-instruct-2` | mistralai/Ministral-8B-Instruct-2410 | 19 |
47
  | `small-instruct` | meta-llama/Llama-3.2-3B-Instruct | 33 |
48
 
49
  Methods present: `ablation_no_condition_adaptive`, `ablation_no_rationale_loss`, `ablation_placement_random`, `ablation_placement_random_draw2`, `ablation_placement_random_draw3`, `ablation_placement_uniform`, `ablation_rank_sweep_16`, `ablation_rank_sweep_32`, `ablation_rank_sweep_8`, `baseline_dart`, `baseline_fairnet`, `baseline_igu_lora`, `baseline_lftf`, `baseline_pedal`, `baseline_regift`, `graft_proposed`, `graft_proposed_loao_gender`, `graft_proposed_loao_landholding`, `graft_proposed_loao_social_group`, `graft_proposed_qlora_nf4`, `reference_vanilla_lora`, `reference_vanilla_lora_loao_gender`, `reference_vanilla_lora_loao_landholding`, `reference_vanilla_lora_loao_social_group`, `reference_vanilla_qlora`
 
29
  layers alone with rank proportional to attribution, and trains them with a condition-adaptive,
30
  rationale-aware objective.
31
 
32
+ This repository holds **119 adapters** so that every number in the paper can be
33
  recomputed rather than taken on trust.
34
 
35
  ## Layout
 
42
  | Tier | Base model | Adapters |
43
  |---|---|---|
44
  | `broad-instruct` | Qwen/Qwen3-4B-Instruct-2507 | 33 |
45
+ | `general-instruct` | google/gemma-3-12b-it | 33 |
46
+ | `general-instruct-2` | mistralai/Ministral-8B-Instruct-2410 | 20 |
47
  | `small-instruct` | meta-llama/Llama-3.2-3B-Instruct | 33 |
48
 
49
  Methods present: `ablation_no_condition_adaptive`, `ablation_no_rationale_loss`, `ablation_placement_random`, `ablation_placement_random_draw2`, `ablation_placement_random_draw3`, `ablation_placement_uniform`, `ablation_rank_sweep_16`, `ablation_rank_sweep_32`, `ablation_rank_sweep_8`, `baseline_dart`, `baseline_fairnet`, `baseline_igu_lora`, `baseline_lftf`, `baseline_pedal`, `baseline_regift`, `graft_proposed`, `graft_proposed_loao_gender`, `graft_proposed_loao_landholding`, `graft_proposed_loao_social_group`, `graft_proposed_qlora_nf4`, `reference_vanilla_lora`, `reference_vanilla_lora_loao_gender`, `reference_vanilla_lora_loao_landholding`, `reference_vanilla_lora_loao_social_group`, `reference_vanilla_qlora`
general-instruct-2/baseline_fairnet/seed_43/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: /workspace/Finetune_AgriFair/models/mistralai__Ministral-8B-Instruct-2410
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:/workspace/Finetune_AgriFair/models/mistralai__Ministral-8B-Instruct-2410
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.20.0
general-instruct-2/baseline_fairnet/seed_43/adapter_config.json ADDED
@@ -0,0 +1,548 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {
4
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.down_proj": 42,
5
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.gate_proj": 42,
6
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.k_proj": 42,
7
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.o_proj": 42,
8
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.q_proj": 42,
9
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.up_proj": 42,
10
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.v_proj": 42,
11
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.down_proj": 42,
12
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.gate_proj": 42,
13
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.k_proj": 42,
14
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.o_proj": 42,
15
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.q_proj": 42,
16
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.up_proj": 42,
17
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.v_proj": 42,
18
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.down_proj": 42,
19
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.gate_proj": 42,
20
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.k_proj": 42,
21
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.o_proj": 42,
22
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.q_proj": 42,
23
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.up_proj": 42,
24
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.v_proj": 42,
25
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.down_proj": 42,
26
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.gate_proj": 42,
27
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.k_proj": 42,
28
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.o_proj": 42,
29
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.q_proj": 42,
30
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.up_proj": 42,
31
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.v_proj": 42,
32
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.down_proj": 42,
33
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.gate_proj": 42,
34
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.k_proj": 42,
35
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.o_proj": 42,
36
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.q_proj": 42,
37
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.up_proj": 42,
38
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.v_proj": 42,
39
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.down_proj": 42,
40
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.gate_proj": 42,
41
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.k_proj": 42,
42
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.o_proj": 42,
43
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.q_proj": 42,
44
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.up_proj": 42,
45
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.v_proj": 42,
46
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.down_proj": 42,
47
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.gate_proj": 42,
48
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.k_proj": 42,
49
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.o_proj": 42,
50
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.q_proj": 42,
51
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.up_proj": 42,
52
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.v_proj": 42,
53
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.down_proj": 42,
54
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.gate_proj": 42,
55
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.k_proj": 42,
56
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.o_proj": 42,
57
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.q_proj": 42,
58
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.up_proj": 42,
59
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.v_proj": 42,
60
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.down_proj": 42,
61
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.gate_proj": 42,
62
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.k_proj": 42,
63
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.o_proj": 42,
64
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.q_proj": 42,
65
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.up_proj": 42,
66
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.v_proj": 42,
67
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.down_proj": 42,
68
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.gate_proj": 42,
69
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.k_proj": 42,
70
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.o_proj": 42,
71
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.q_proj": 42,
72
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.up_proj": 42,
73
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.v_proj": 42,
74
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.down_proj": 42,
75
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.gate_proj": 42,
76
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.k_proj": 42,
77
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.o_proj": 42,
78
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.q_proj": 42,
79
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.up_proj": 42,
80
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.v_proj": 42,
81
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.down_proj": 42,
82
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.gate_proj": 42,
83
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.k_proj": 42,
84
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.o_proj": 42,
85
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.q_proj": 42,
86
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.up_proj": 42,
87
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.v_proj": 42,
88
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.down_proj": 42,
89
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.gate_proj": 42,
90
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.k_proj": 42,
91
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.o_proj": 42,
92
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.q_proj": 42,
93
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.up_proj": 42,
94
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.v_proj": 42,
95
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.down_proj": 42,
96
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.gate_proj": 42,
97
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.k_proj": 42,
98
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.o_proj": 42,
99
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.q_proj": 42,
100
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.up_proj": 42,
101
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.v_proj": 42,
102
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.down_proj": 42,
103
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.gate_proj": 42,
104
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.k_proj": 42,
105
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.o_proj": 42,
106
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.q_proj": 42,
107
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.up_proj": 42,
108
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.v_proj": 42,
109
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.down_proj": 42,
110
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.gate_proj": 42,
111
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.k_proj": 42,
112
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.o_proj": 42,
113
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.q_proj": 42,
114
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.up_proj": 42,
115
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.v_proj": 42,
116
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.down_proj": 42,
117
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.gate_proj": 42,
118
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.k_proj": 42,
119
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.o_proj": 42,
120
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.q_proj": 42,
121
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.up_proj": 42,
122
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.v_proj": 42,
123
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.down_proj": 42,
124
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.gate_proj": 42,
125
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.k_proj": 42,
126
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.o_proj": 42,
127
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.q_proj": 42,
128
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.up_proj": 42,
129
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.v_proj": 42,
130
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.down_proj": 42,
131
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.gate_proj": 42,
132
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.k_proj": 42,
133
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.o_proj": 42,
134
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.q_proj": 42,
135
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.up_proj": 42,
136
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.v_proj": 42,
137
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.down_proj": 40,
138
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.gate_proj": 40,
139
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.k_proj": 40,
140
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.o_proj": 40,
141
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.q_proj": 40,
142
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.up_proj": 40,
143
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.v_proj": 40,
144
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.down_proj": 40,
145
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.gate_proj": 40,
146
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.k_proj": 40,
147
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.o_proj": 40,
148
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.q_proj": 40,
149
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.up_proj": 40,
150
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.v_proj": 40,
151
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.down_proj": 40,
152
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.gate_proj": 40,
153
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.k_proj": 40,
154
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.o_proj": 40,
155
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.q_proj": 40,
156
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.up_proj": 40,
157
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.v_proj": 40,
158
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.down_proj": 42,
159
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.gate_proj": 42,
160
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.k_proj": 42,
161
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.o_proj": 42,
162
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.q_proj": 42,
163
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.up_proj": 42,
164
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.v_proj": 42,
165
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.down_proj": 40,
166
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.gate_proj": 40,
167
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.k_proj": 40,
168
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.o_proj": 40,
169
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.q_proj": 40,
170
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.up_proj": 40,
171
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.v_proj": 40,
172
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.down_proj": 40,
173
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.gate_proj": 40,
174
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.k_proj": 40,
175
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.o_proj": 40,
176
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.q_proj": 40,
177
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.up_proj": 40,
178
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.v_proj": 40,
179
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.down_proj": 40,
180
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.gate_proj": 40,
181
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.k_proj": 40,
182
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.o_proj": 40,
183
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.q_proj": 40,
184
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.up_proj": 40,
185
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.v_proj": 40,
186
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.down_proj": 40,
187
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.gate_proj": 40,
188
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.k_proj": 40,
189
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.o_proj": 40,
190
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.q_proj": 40,
191
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.up_proj": 40,
192
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.v_proj": 40,
193
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.down_proj": 40,
194
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.gate_proj": 40,
195
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.k_proj": 40,
196
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.o_proj": 40,
197
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.q_proj": 40,
198
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.up_proj": 40,
199
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.v_proj": 40,
200
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.down_proj": 40,
201
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.gate_proj": 40,
202
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.k_proj": 40,
203
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.o_proj": 40,
204
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.q_proj": 40,
205
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.up_proj": 40,
206
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.v_proj": 40,
207
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.down_proj": 42,
208
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.gate_proj": 42,
209
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.k_proj": 42,
210
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.o_proj": 42,
211
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.q_proj": 42,
212
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.up_proj": 42,
213
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.v_proj": 42,
214
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.down_proj": 42,
215
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.gate_proj": 42,
216
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.k_proj": 42,
217
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.o_proj": 42,
218
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.q_proj": 42,
219
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.up_proj": 42,
220
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.v_proj": 42,
221
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.down_proj": 42,
222
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.gate_proj": 42,
223
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.k_proj": 42,
224
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.o_proj": 42,
225
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.q_proj": 42,
226
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.up_proj": 42,
227
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.v_proj": 42,
228
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.down_proj": 42,
229
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.gate_proj": 42,
230
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.k_proj": 42,
231
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.o_proj": 42,
232
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.q_proj": 42,
233
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.up_proj": 42,
234
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.v_proj": 42,
235
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.down_proj": 42,
236
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.gate_proj": 42,
237
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.k_proj": 42,
238
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.o_proj": 42,
239
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.q_proj": 42,
240
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.up_proj": 42,
241
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.v_proj": 42,
242
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.down_proj": 42,
243
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.gate_proj": 42,
244
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.k_proj": 42,
245
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.o_proj": 42,
246
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.q_proj": 42,
247
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.up_proj": 42,
248
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.v_proj": 42,
249
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.down_proj": 42,
250
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.gate_proj": 42,
251
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.k_proj": 42,
252
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.o_proj": 42,
253
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.q_proj": 42,
254
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.up_proj": 42,
255
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.v_proj": 42
256
+ },
257
+ "arrow_config": null,
258
+ "auto_mapping": null,
259
+ "base_model_name_or_path": "/workspace/Finetune_AgriFair/models/mistralai__Ministral-8B-Instruct-2410",
260
+ "bias": "none",
261
+ "corda_config": null,
262
+ "ensure_weight_tying": false,
263
+ "eva_config": null,
264
+ "exclude_modules": null,
265
+ "fan_in_fan_out": false,
266
+ "inference_mode": true,
267
+ "init_lora_weights": true,
268
+ "layer_replication": null,
269
+ "layers_pattern": null,
270
+ "layers_to_transform": null,
271
+ "loftq_config": {},
272
+ "lora_alpha": 40,
273
+ "lora_bias": false,
274
+ "lora_dropout": 0.05,
275
+ "lora_ga_config": null,
276
+ "megatron_config": null,
277
+ "megatron_core": "megatron.core",
278
+ "modules_to_save": null,
279
+ "monteclora_config": null,
280
+ "peft_type": "LORA",
281
+ "peft_version": "0.20.0",
282
+ "qalora_group_size": 16,
283
+ "r": 20,
284
+ "rank_pattern": {
285
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.down_proj": 21,
286
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.gate_proj": 21,
287
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.k_proj": 21,
288
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.o_proj": 21,
289
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.q_proj": 21,
290
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.up_proj": 21,
291
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.v_proj": 21,
292
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.down_proj": 21,
293
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.gate_proj": 21,
294
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.k_proj": 21,
295
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.o_proj": 21,
296
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.q_proj": 21,
297
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.up_proj": 21,
298
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.v_proj": 21,
299
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.down_proj": 21,
300
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.gate_proj": 21,
301
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.k_proj": 21,
302
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.o_proj": 21,
303
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.q_proj": 21,
304
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.up_proj": 21,
305
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.v_proj": 21,
306
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.down_proj": 21,
307
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.gate_proj": 21,
308
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.k_proj": 21,
309
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.o_proj": 21,
310
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.q_proj": 21,
311
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.up_proj": 21,
312
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.v_proj": 21,
313
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.down_proj": 21,
314
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.gate_proj": 21,
315
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.k_proj": 21,
316
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.o_proj": 21,
317
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.q_proj": 21,
318
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.up_proj": 21,
319
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.v_proj": 21,
320
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.down_proj": 21,
321
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.gate_proj": 21,
322
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.k_proj": 21,
323
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.o_proj": 21,
324
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.q_proj": 21,
325
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.up_proj": 21,
326
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.v_proj": 21,
327
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.down_proj": 21,
328
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.gate_proj": 21,
329
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.k_proj": 21,
330
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.o_proj": 21,
331
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.q_proj": 21,
332
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.up_proj": 21,
333
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.v_proj": 21,
334
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.down_proj": 21,
335
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.gate_proj": 21,
336
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.k_proj": 21,
337
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.o_proj": 21,
338
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.q_proj": 21,
339
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.up_proj": 21,
340
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.v_proj": 21,
341
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.down_proj": 21,
342
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.gate_proj": 21,
343
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.k_proj": 21,
344
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.o_proj": 21,
345
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.q_proj": 21,
346
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.up_proj": 21,
347
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.v_proj": 21,
348
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.down_proj": 21,
349
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.gate_proj": 21,
350
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.k_proj": 21,
351
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.o_proj": 21,
352
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.q_proj": 21,
353
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.up_proj": 21,
354
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.v_proj": 21,
355
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.down_proj": 21,
356
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.gate_proj": 21,
357
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.k_proj": 21,
358
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.o_proj": 21,
359
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.q_proj": 21,
360
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.up_proj": 21,
361
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.v_proj": 21,
362
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.down_proj": 21,
363
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.gate_proj": 21,
364
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.k_proj": 21,
365
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.o_proj": 21,
366
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.q_proj": 21,
367
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.up_proj": 21,
368
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.v_proj": 21,
369
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.down_proj": 21,
370
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.gate_proj": 21,
371
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.k_proj": 21,
372
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.o_proj": 21,
373
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.q_proj": 21,
374
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.up_proj": 21,
375
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.v_proj": 21,
376
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.down_proj": 21,
377
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.gate_proj": 21,
378
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.k_proj": 21,
379
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.o_proj": 21,
380
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.q_proj": 21,
381
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.up_proj": 21,
382
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.v_proj": 21,
383
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.down_proj": 21,
384
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.gate_proj": 21,
385
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.k_proj": 21,
386
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.o_proj": 21,
387
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.q_proj": 21,
388
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.up_proj": 21,
389
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.v_proj": 21,
390
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.down_proj": 21,
391
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.gate_proj": 21,
392
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.k_proj": 21,
393
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.o_proj": 21,
394
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.q_proj": 21,
395
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.up_proj": 21,
396
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.v_proj": 21,
397
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.down_proj": 21,
398
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.gate_proj": 21,
399
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.k_proj": 21,
400
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.o_proj": 21,
401
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.q_proj": 21,
402
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.up_proj": 21,
403
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.v_proj": 21,
404
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.down_proj": 21,
405
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.gate_proj": 21,
406
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.k_proj": 21,
407
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.o_proj": 21,
408
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.q_proj": 21,
409
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.up_proj": 21,
410
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.v_proj": 21,
411
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.down_proj": 21,
412
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.gate_proj": 21,
413
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.k_proj": 21,
414
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.o_proj": 21,
415
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.q_proj": 21,
416
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.up_proj": 21,
417
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.v_proj": 21,
418
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.down_proj": 20,
419
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.gate_proj": 20,
420
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.k_proj": 20,
421
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.o_proj": 20,
422
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.q_proj": 20,
423
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.up_proj": 20,
424
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.v_proj": 20,
425
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.down_proj": 20,
426
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.gate_proj": 20,
427
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.k_proj": 20,
428
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.o_proj": 20,
429
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.q_proj": 20,
430
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.up_proj": 20,
431
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.v_proj": 20,
432
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.down_proj": 20,
433
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.gate_proj": 20,
434
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.k_proj": 20,
435
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.o_proj": 20,
436
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.q_proj": 20,
437
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.up_proj": 20,
438
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.v_proj": 20,
439
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.down_proj": 21,
440
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.gate_proj": 21,
441
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.k_proj": 21,
442
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.o_proj": 21,
443
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.q_proj": 21,
444
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.up_proj": 21,
445
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.v_proj": 21,
446
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.down_proj": 20,
447
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.gate_proj": 20,
448
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.k_proj": 20,
449
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.o_proj": 20,
450
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.q_proj": 20,
451
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.up_proj": 20,
452
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.v_proj": 20,
453
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.down_proj": 20,
454
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.gate_proj": 20,
455
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.k_proj": 20,
456
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.o_proj": 20,
457
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.q_proj": 20,
458
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.up_proj": 20,
459
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.v_proj": 20,
460
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.down_proj": 20,
461
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.gate_proj": 20,
462
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.k_proj": 20,
463
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.o_proj": 20,
464
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.q_proj": 20,
465
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.up_proj": 20,
466
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.v_proj": 20,
467
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.down_proj": 20,
468
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.gate_proj": 20,
469
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.k_proj": 20,
470
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.o_proj": 20,
471
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.q_proj": 20,
472
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.up_proj": 20,
473
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.v_proj": 20,
474
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.down_proj": 20,
475
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.gate_proj": 20,
476
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.k_proj": 20,
477
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.o_proj": 20,
478
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.q_proj": 20,
479
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.up_proj": 20,
480
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.v_proj": 20,
481
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.down_proj": 20,
482
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.gate_proj": 20,
483
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.k_proj": 20,
484
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.o_proj": 20,
485
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.q_proj": 20,
486
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.up_proj": 20,
487
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.v_proj": 20,
488
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.down_proj": 21,
489
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.gate_proj": 21,
490
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.k_proj": 21,
491
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.o_proj": 21,
492
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.q_proj": 21,
493
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.up_proj": 21,
494
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.v_proj": 21,
495
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.down_proj": 21,
496
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.gate_proj": 21,
497
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.k_proj": 21,
498
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.o_proj": 21,
499
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.q_proj": 21,
500
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.up_proj": 21,
501
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.v_proj": 21,
502
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.down_proj": 21,
503
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.gate_proj": 21,
504
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.k_proj": 21,
505
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.o_proj": 21,
506
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.q_proj": 21,
507
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.up_proj": 21,
508
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.v_proj": 21,
509
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.down_proj": 21,
510
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.gate_proj": 21,
511
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.k_proj": 21,
512
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.o_proj": 21,
513
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.q_proj": 21,
514
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.up_proj": 21,
515
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.v_proj": 21,
516
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.down_proj": 21,
517
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.gate_proj": 21,
518
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.k_proj": 21,
519
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.o_proj": 21,
520
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.q_proj": 21,
521
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.up_proj": 21,
522
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.v_proj": 21,
523
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.down_proj": 21,
524
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.gate_proj": 21,
525
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.k_proj": 21,
526
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.o_proj": 21,
527
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.q_proj": 21,
528
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.up_proj": 21,
529
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.v_proj": 21,
530
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.down_proj": 21,
531
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.gate_proj": 21,
532
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.k_proj": 21,
533
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.o_proj": 21,
534
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.q_proj": 21,
535
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.up_proj": 21,
536
+ "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.v_proj": 21
537
+ },
538
+ "revision": null,
539
+ "target_modules": "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*\\.(?:layers|h)\\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32|33|34|35)\\.[^.]+\\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)",
540
+ "target_parameters": null,
541
+ "task_type": "CAUSAL_LM",
542
+ "trainable_token_indices": null,
543
+ "use_bdlora": null,
544
+ "use_dora": false,
545
+ "use_qalora": false,
546
+ "use_rslora": false,
547
+ "velora_config": null
548
+ }
general-instruct-2/baseline_fairnet/seed_43/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:99d328798596dc68b0ed08c62ecb8f8e71c1266c4369adc0f6a3d179ebf6f788
3
+ size 226486472
general-instruct/reference_vanilla_qlora/seed_42/README.md ADDED
@@ -0,0 +1,207 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: /workspace/Finetune_AgriFair/models/google__gemma-3-12b-it
3
+ library_name: peft
4
+ pipeline_tag: text-generation
5
+ tags:
6
+ - base_model:adapter:/workspace/Finetune_AgriFair/models/google__gemma-3-12b-it
7
+ - lora
8
+ - transformers
9
+ ---
10
+
11
+ # Model Card for Model ID
12
+
13
+ <!-- Provide a quick summary of what the model is/does. -->
14
+
15
+
16
+
17
+ ## Model Details
18
+
19
+ ### Model Description
20
+
21
+ <!-- Provide a longer summary of what this model is. -->
22
+
23
+
24
+
25
+ - **Developed by:** [More Information Needed]
26
+ - **Funded by [optional]:** [More Information Needed]
27
+ - **Shared by [optional]:** [More Information Needed]
28
+ - **Model type:** [More Information Needed]
29
+ - **Language(s) (NLP):** [More Information Needed]
30
+ - **License:** [More Information Needed]
31
+ - **Finetuned from model [optional]:** [More Information Needed]
32
+
33
+ ### Model Sources [optional]
34
+
35
+ <!-- Provide the basic links for the model. -->
36
+
37
+ - **Repository:** [More Information Needed]
38
+ - **Paper [optional]:** [More Information Needed]
39
+ - **Demo [optional]:** [More Information Needed]
40
+
41
+ ## Uses
42
+
43
+ <!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
44
+
45
+ ### Direct Use
46
+
47
+ <!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
48
+
49
+ [More Information Needed]
50
+
51
+ ### Downstream Use [optional]
52
+
53
+ <!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
54
+
55
+ [More Information Needed]
56
+
57
+ ### Out-of-Scope Use
58
+
59
+ <!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
60
+
61
+ [More Information Needed]
62
+
63
+ ## Bias, Risks, and Limitations
64
+
65
+ <!-- This section is meant to convey both technical and sociotechnical limitations. -->
66
+
67
+ [More Information Needed]
68
+
69
+ ### Recommendations
70
+
71
+ <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
72
+
73
+ Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
74
+
75
+ ## How to Get Started with the Model
76
+
77
+ Use the code below to get started with the model.
78
+
79
+ [More Information Needed]
80
+
81
+ ## Training Details
82
+
83
+ ### Training Data
84
+
85
+ <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
86
+
87
+ [More Information Needed]
88
+
89
+ ### Training Procedure
90
+
91
+ <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
92
+
93
+ #### Preprocessing [optional]
94
+
95
+ [More Information Needed]
96
+
97
+
98
+ #### Training Hyperparameters
99
+
100
+ - **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
101
+
102
+ #### Speeds, Sizes, Times [optional]
103
+
104
+ <!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
105
+
106
+ [More Information Needed]
107
+
108
+ ## Evaluation
109
+
110
+ <!-- This section describes the evaluation protocols and provides the results. -->
111
+
112
+ ### Testing Data, Factors & Metrics
113
+
114
+ #### Testing Data
115
+
116
+ <!-- This should link to a Dataset Card if possible. -->
117
+
118
+ [More Information Needed]
119
+
120
+ #### Factors
121
+
122
+ <!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
123
+
124
+ [More Information Needed]
125
+
126
+ #### Metrics
127
+
128
+ <!-- These are the evaluation metrics being used, ideally with a description of why. -->
129
+
130
+ [More Information Needed]
131
+
132
+ ### Results
133
+
134
+ [More Information Needed]
135
+
136
+ #### Summary
137
+
138
+
139
+
140
+ ## Model Examination [optional]
141
+
142
+ <!-- Relevant interpretability work for the model goes here -->
143
+
144
+ [More Information Needed]
145
+
146
+ ## Environmental Impact
147
+
148
+ <!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
149
+
150
+ Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
151
+
152
+ - **Hardware Type:** [More Information Needed]
153
+ - **Hours used:** [More Information Needed]
154
+ - **Cloud Provider:** [More Information Needed]
155
+ - **Compute Region:** [More Information Needed]
156
+ - **Carbon Emitted:** [More Information Needed]
157
+
158
+ ## Technical Specifications [optional]
159
+
160
+ ### Model Architecture and Objective
161
+
162
+ [More Information Needed]
163
+
164
+ ### Compute Infrastructure
165
+
166
+ [More Information Needed]
167
+
168
+ #### Hardware
169
+
170
+ [More Information Needed]
171
+
172
+ #### Software
173
+
174
+ [More Information Needed]
175
+
176
+ ## Citation [optional]
177
+
178
+ <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
179
+
180
+ **BibTeX:**
181
+
182
+ [More Information Needed]
183
+
184
+ **APA:**
185
+
186
+ [More Information Needed]
187
+
188
+ ## Glossary [optional]
189
+
190
+ <!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
191
+
192
+ [More Information Needed]
193
+
194
+ ## More Information [optional]
195
+
196
+ [More Information Needed]
197
+
198
+ ## Model Card Authors [optional]
199
+
200
+ [More Information Needed]
201
+
202
+ ## Model Card Contact
203
+
204
+ [More Information Needed]
205
+ ### Framework versions
206
+
207
+ - PEFT 0.20.0
general-instruct/reference_vanilla_qlora/seed_42/adapter_config.json ADDED
@@ -0,0 +1,50 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "alora_invocation_tokens": null,
3
+ "alpha_pattern": {},
4
+ "arrow_config": null,
5
+ "auto_mapping": null,
6
+ "base_model_name_or_path": "/workspace/Finetune_AgriFair/models/google__gemma-3-12b-it",
7
+ "bias": "none",
8
+ "corda_config": null,
9
+ "ensure_weight_tying": false,
10
+ "eva_config": null,
11
+ "exclude_modules": null,
12
+ "fan_in_fan_out": false,
13
+ "inference_mode": true,
14
+ "init_lora_weights": true,
15
+ "layer_replication": null,
16
+ "layers_pattern": null,
17
+ "layers_to_transform": null,
18
+ "loftq_config": {},
19
+ "lora_alpha": 16,
20
+ "lora_bias": false,
21
+ "lora_dropout": 0.05,
22
+ "lora_ga_config": null,
23
+ "megatron_config": null,
24
+ "megatron_core": "megatron.core",
25
+ "modules_to_save": null,
26
+ "monteclora_config": null,
27
+ "peft_type": "LORA",
28
+ "peft_version": "0.20.0",
29
+ "qalora_group_size": 16,
30
+ "r": 8,
31
+ "rank_pattern": {},
32
+ "revision": null,
33
+ "target_modules": [
34
+ "o_proj",
35
+ "gate_proj",
36
+ "v_proj",
37
+ "up_proj",
38
+ "k_proj",
39
+ "q_proj",
40
+ "down_proj"
41
+ ],
42
+ "target_parameters": null,
43
+ "task_type": "CAUSAL_LM",
44
+ "trainable_token_indices": null,
45
+ "use_bdlora": null,
46
+ "use_dora": false,
47
+ "use_qalora": false,
48
+ "use_rslora": false,
49
+ "velora_config": null
50
+ }
general-instruct/reference_vanilla_qlora/seed_42/adapter_model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:51b716d38b4c8d8fdf78f73b53120ea88ffa3ea5ed74f09855dc3cc2e4bac08f
3
+ size 137039712
manifest.json CHANGED
@@ -884,6 +884,26 @@
884
  ],
885
  "trainable_parameter_percentage": 0.2801
886
  },
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
887
  {
888
  "path_in_repo": "general-instruct-2/ablation_no_condition_adaptive/seed_42",
889
  "model_tier": "general-instruct-2",
@@ -992,6 +1012,18 @@
992
  "target_modules": "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*\\.(?:layers|h)\\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17)\\.[^.]+\\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)",
993
  "trainable_parameter_percentage": 0.1359
994
  },
 
 
 
 
 
 
 
 
 
 
 
 
995
  {
996
  "path_in_repo": "general-instruct-2/graft_proposed/seed_42",
997
  "model_tier": "general-instruct-2",
@@ -1589,5 +1621,5 @@
1589
  "trainable_parameter_percentage": null
1590
  }
1591
  ],
1592
- "count": 117
1593
  }
 
884
  ],
885
  "trainable_parameter_percentage": 0.2801
886
  },
887
+ {
888
+ "path_in_repo": "general-instruct/reference_vanilla_qlora/seed_42",
889
+ "model_tier": "general-instruct",
890
+ "base_model": "google/gemma-3-12b-it",
891
+ "base_model_revision": "96b6f1eccf38110c56df3a15bffe176da04bfd80",
892
+ "method": "reference_vanilla_qlora",
893
+ "random_seed": 42,
894
+ "lora_rank_range": 8,
895
+ "lora_alpha": 16,
896
+ "target_modules": [
897
+ "o_proj",
898
+ "gate_proj",
899
+ "v_proj",
900
+ "up_proj",
901
+ "k_proj",
902
+ "q_proj",
903
+ "down_proj"
904
+ ],
905
+ "trainable_parameter_percentage": null
906
+ },
907
  {
908
  "path_in_repo": "general-instruct-2/ablation_no_condition_adaptive/seed_42",
909
  "model_tier": "general-instruct-2",
 
1012
  "target_modules": "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*\\.(?:layers|h)\\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17)\\.[^.]+\\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)",
1013
  "trainable_parameter_percentage": 0.1359
1014
  },
1015
+ {
1016
+ "path_in_repo": "general-instruct-2/baseline_fairnet/seed_43",
1017
+ "model_tier": "general-instruct-2",
1018
+ "base_model": "mistralai/Ministral-8B-Instruct-2410",
1019
+ "base_model_revision": "2f494a194c5b980dfb9772cb92d26cbb671fce5a",
1020
+ "method": "baseline_fairnet",
1021
+ "random_seed": 43,
1022
+ "lora_rank_range": "20-21",
1023
+ "lora_alpha": 40,
1024
+ "target_modules": "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*\\.(?:layers|h)\\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32|33|34|35)\\.[^.]+\\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)",
1025
+ "trainable_parameter_percentage": null
1026
+ },
1027
  {
1028
  "path_in_repo": "general-instruct-2/graft_proposed/seed_42",
1029
  "model_tier": "general-instruct-2",
 
1621
  "trainable_parameter_percentage": null
1622
  }
1623
  ],
1624
+ "count": 119
1625
  }