AgriFair GRAFT adapters: 101 adapters

README.md

@@ -29,7 +29,7 @@ census records a real difference, using Integrated Gradients. It places LoRA ada
 layers alone with rank proportional to attribution, and trains them with a condition-adaptive,
 rationale-aware objective.
 
-This repository holds **95 adapters** so that every number in the paper can be
+This repository holds **101 adapters** so that every number in the paper can be
 recomputed rather than taken on trust.
 
 ## Layout
@@ -41,7 +41,7 @@ manifest.json                 base model, revision, rank range and trainable sha
 
 | Tier | Base model | Adapters |
 |---|---|---|
-| `broad-instruct` | Qwen/Qwen3-4B-Instruct-2507 | 24 |
+| `broad-instruct` | Qwen/Qwen3-4B-Instruct-2507 | 30 |
 | `general-instruct` | google/gemma-3-12b-it | 19 |
 | `general-instruct-2` | mistralai/Ministral-8B-Instruct-2410 | 19 |
 | `small-instruct` | meta-llama/Llama-3.2-3B-Instruct | 33 |

broad-instruct/baseline_dart/seed_42/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /workspace/Finetune_AgriFair/models/Qwen__Qwen3-4B-Instruct-2507
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/workspace/Finetune_AgriFair/models/Qwen__Qwen3-4B-Instruct-2507
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

broad-instruct/baseline_dart/seed_42/adapter_config.json

@@ -0,0 +1,548 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.v_proj": 54
+  },
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/workspace/Finetune_AgriFair/models/Qwen__Qwen3-4B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 54,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 27,
+  "rank_pattern": {
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.v_proj": 27
+  },
+  "revision": null,
+  "target_modules": "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*\\.(?:layers|h)\\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32|33|34|35)\\.[^.]+\\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)",
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

broad-instruct/baseline_dart/seed_42/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1e26df38eccfdf284cb15c79c5ac5d382f5dcb2188e33e58893d8fc3d68b0005
+size 224397392

broad-instruct/baseline_dart/seed_43/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /workspace/Finetune_AgriFair/models/Qwen__Qwen3-4B-Instruct-2507
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/workspace/Finetune_AgriFair/models/Qwen__Qwen3-4B-Instruct-2507
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

broad-instruct/baseline_dart/seed_43/adapter_config.json

@@ -0,0 +1,548 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.v_proj": 54
+  },
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/workspace/Finetune_AgriFair/models/Qwen__Qwen3-4B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 54,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 27,
+  "rank_pattern": {
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.v_proj": 27
+  },
+  "revision": null,
+  "target_modules": "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*\\.(?:layers|h)\\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32|33|34|35)\\.[^.]+\\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)",
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

broad-instruct/baseline_dart/seed_43/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b2134f05156f72ad56d053bbb582c5d3ca6be9a53c4293e36dadf8f5646bcb5a
+size 224397392

broad-instruct/baseline_dart/seed_44/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /workspace/Finetune_AgriFair/models/Qwen__Qwen3-4B-Instruct-2507
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/workspace/Finetune_AgriFair/models/Qwen__Qwen3-4B-Instruct-2507
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

broad-instruct/baseline_dart/seed_44/adapter_config.json

@@ -0,0 +1,548 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.v_proj": 54
+  },
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/workspace/Finetune_AgriFair/models/Qwen__Qwen3-4B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 54,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 27,
+  "rank_pattern": {
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.v_proj": 27
+  },
+  "revision": null,
+  "target_modules": "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*\\.(?:layers|h)\\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32|33|34|35)\\.[^.]+\\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)",
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

broad-instruct/baseline_dart/seed_44/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:94ccd514deb7c6856f51d230d89d6da76e8142fdc05286afd9bb7e1fc7c0da05
+size 224397392

broad-instruct/baseline_lftf/seed_42/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /workspace/Finetune_AgriFair/models/Qwen__Qwen3-4B-Instruct-2507
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/workspace/Finetune_AgriFair/models/Qwen__Qwen3-4B-Instruct-2507
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

broad-instruct/baseline_lftf/seed_42/adapter_config.json

@@ -0,0 +1,254 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.down_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.gate_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.k_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.o_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.q_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.up_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.v_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.down_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.gate_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.k_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.o_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.q_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.up_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.v_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.down_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.gate_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.k_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.o_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.q_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.up_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.v_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.down_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.gate_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.k_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.o_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.q_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.up_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.v_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.down_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.gate_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.k_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.o_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.q_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.up_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.v_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.down_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.gate_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.k_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.o_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.q_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.up_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.v_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.down_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.gate_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.k_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.o_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.q_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.up_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.v_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.down_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.gate_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.k_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.o_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.q_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.up_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.v_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.down_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.gate_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.k_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.o_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.q_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.up_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.v_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.down_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.gate_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.k_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.o_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.q_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.up_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.v_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.down_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.gate_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.k_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.o_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.q_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.up_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.v_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.down_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.gate_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.k_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.o_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.q_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.up_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.v_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.down_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.gate_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.k_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.o_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.q_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.up_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.v_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.down_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.gate_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.k_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.o_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.q_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.up_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.v_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.down_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.gate_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.k_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.o_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.q_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.up_proj": 128,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.v_proj": 128
+  },
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/workspace/Finetune_AgriFair/models/Qwen__Qwen3-4B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 128,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 64,
+  "rank_pattern": {
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.down_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.gate_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.k_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.o_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.q_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.up_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.v_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.down_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.gate_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.k_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.o_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.q_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.up_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.v_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.down_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.gate_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.k_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.o_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.q_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.up_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.v_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.down_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.gate_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.k_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.o_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.q_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.up_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.v_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.down_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.gate_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.k_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.o_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.q_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.up_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.v_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.down_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.gate_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.k_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.o_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.q_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.up_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.v_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.down_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.gate_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.k_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.o_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.q_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.up_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.v_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.down_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.gate_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.k_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.o_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.q_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.up_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.v_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.down_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.gate_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.k_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.o_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.q_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.up_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.v_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.down_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.gate_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.k_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.o_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.q_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.up_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.v_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.down_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.gate_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.k_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.o_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.q_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.up_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.v_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.down_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.gate_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.k_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.o_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.q_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.up_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.v_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.down_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.gate_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.k_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.o_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.q_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.up_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.v_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.down_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.gate_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.k_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.o_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.q_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.up_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.v_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.down_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.gate_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.k_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.o_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.q_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.up_proj": 64,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.v_proj": 64
+  },
+  "revision": null,
+  "target_modules": "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*\\.(?:layers|h)\\.(21|22|23|24|25|26|27|28|29|30|31|32|33|34|35)\\.[^.]+\\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)",
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

broad-instruct/baseline_lftf/seed_42/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3e0ece7169a959789cd003252177b5ed7e0a9eee7f0a21534d3774f77f15320c
+size 220229240

broad-instruct/baseline_regift/seed_42/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /workspace/Finetune_AgriFair/models/Qwen__Qwen3-4B-Instruct-2507
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/workspace/Finetune_AgriFair/models/Qwen__Qwen3-4B-Instruct-2507
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

broad-instruct/baseline_regift/seed_42/adapter_config.json

@@ -0,0 +1,548 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.down_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.gate_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.k_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.o_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.q_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.up_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.v_proj": 56,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.v_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.down_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.gate_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.k_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.o_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.q_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.up_proj": 54,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.v_proj": 54
+  },
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/workspace/Finetune_AgriFair/models/Qwen__Qwen3-4B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 54,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 27,
+  "rank_pattern": {
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.0\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.10\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.11\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.12\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.13\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.14\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.15\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.16\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.17\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.18\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.19\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.1\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.20\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.21\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.22\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.23\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.24\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.25\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.26\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.27\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.28\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.29\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.2\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.30\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.31\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.32\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.33\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.34\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.35\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.3\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.4\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.down_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.gate_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.k_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.o_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.q_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.up_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.5\\.[^.]+\\.v_proj": 28,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.6\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.7\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.8\\.[^.]+\\.v_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.down_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.gate_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.k_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.o_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.q_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.up_proj": 27,
+    "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*(?:layers|h)\\.9\\.[^.]+\\.v_proj": 27
+  },
+  "revision": null,
+  "target_modules": "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*\\.(?:layers|h)\\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32|33|34|35)\\.[^.]+\\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)",
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

broad-instruct/baseline_regift/seed_42/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a54f5e702d38eeda3c413b57e4dfc8ab45b147ecf12b8a25281eb77b49b76c34
+size 224397392

broad-instruct/reference_vanilla_lora/seed_42/README.md

@@ -0,0 +1,207 @@
+---
+base_model: /workspace/Finetune_AgriFair/models/Qwen__Qwen3-4B-Instruct-2507
+library_name: peft
+pipeline_tag: text-generation
+tags:
+- base_model:adapter:/workspace/Finetune_AgriFair/models/Qwen__Qwen3-4B-Instruct-2507
+- lora
+- transformers
+---
+
+# Model Card for Model ID
+
+<!-- Provide a quick summary of what the model is/does. -->
+
+
+
+## Model Details
+
+### Model Description
+
+<!-- Provide a longer summary of what this model is. -->
+
+
+
+- **Developed by:** [More Information Needed]
+- **Funded by [optional]:** [More Information Needed]
+- **Shared by [optional]:** [More Information Needed]
+- **Model type:** [More Information Needed]
+- **Language(s) (NLP):** [More Information Needed]
+- **License:** [More Information Needed]
+- **Finetuned from model [optional]:** [More Information Needed]
+
+### Model Sources [optional]
+
+<!-- Provide the basic links for the model. -->
+
+- **Repository:** [More Information Needed]
+- **Paper [optional]:** [More Information Needed]
+- **Demo [optional]:** [More Information Needed]
+
+## Uses
+
+<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
+
+### Direct Use
+
+<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
+
+[More Information Needed]
+
+### Downstream Use [optional]
+
+<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
+
+[More Information Needed]
+
+### Out-of-Scope Use
+
+<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+
+[More Information Needed]
+
+## Bias, Risks, and Limitations
+
+<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+
+[More Information Needed]
+
+### Recommendations
+
+<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+
+Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+
+## How to Get Started with the Model
+
+Use the code below to get started with the model.
+
+[More Information Needed]
+
+## Training Details
+
+### Training Data
+
+<!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
+
+[More Information Needed]
+
+### Training Procedure
+
+<!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
+
+#### Preprocessing [optional]
+
+[More Information Needed]
+
+
+#### Training Hyperparameters
+
+- **Training regime:** [More Information Needed] <!--fp32, fp16 mixed precision, bf16 mixed precision, bf16 non-mixed precision, fp16 non-mixed precision, fp8 mixed precision -->
+
+#### Speeds, Sizes, Times [optional]
+
+<!-- This section provides information about throughput, start/end time, checkpoint size if relevant, etc. -->
+
+[More Information Needed]
+
+## Evaluation
+
+<!-- This section describes the evaluation protocols and provides the results. -->
+
+### Testing Data, Factors & Metrics
+
+#### Testing Data
+
+<!-- This should link to a Dataset Card if possible. -->
+
+[More Information Needed]
+
+#### Factors
+
+<!-- These are the things the evaluation is disaggregating by, e.g., subpopulations or domains. -->
+
+[More Information Needed]
+
+#### Metrics
+
+<!-- These are the evaluation metrics being used, ideally with a description of why. -->
+
+[More Information Needed]
+
+### Results
+
+[More Information Needed]
+
+#### Summary
+
+
+
+## Model Examination [optional]
+
+<!-- Relevant interpretability work for the model goes here -->
+
+[More Information Needed]
+
+## Environmental Impact
+
+<!-- Total emissions (in grams of CO2eq) and additional considerations, such as electricity usage, go here. Edit the suggested text below accordingly -->
+
+Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700).
+
+- **Hardware Type:** [More Information Needed]
+- **Hours used:** [More Information Needed]
+- **Cloud Provider:** [More Information Needed]
+- **Compute Region:** [More Information Needed]
+- **Carbon Emitted:** [More Information Needed]
+
+## Technical Specifications [optional]
+
+### Model Architecture and Objective
+
+[More Information Needed]
+
+### Compute Infrastructure
+
+[More Information Needed]
+
+#### Hardware
+
+[More Information Needed]
+
+#### Software
+
+[More Information Needed]
+
+## Citation [optional]
+
+<!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
+
+**BibTeX:**
+
+[More Information Needed]
+
+**APA:**
+
+[More Information Needed]
+
+## Glossary [optional]
+
+<!-- If relevant, include terms and calculations in this section that can help readers understand the model or model card. -->
+
+[More Information Needed]
+
+## More Information [optional]
+
+[More Information Needed]
+
+## Model Card Authors [optional]
+
+[More Information Needed]
+
+## Model Card Contact
+
+[More Information Needed]
+### Framework versions
+
+- PEFT 0.20.0

broad-instruct/reference_vanilla_lora/seed_42/adapter_config.json

@@ -0,0 +1,50 @@
+{
+  "alora_invocation_tokens": null,
+  "alpha_pattern": {},
+  "arrow_config": null,
+  "auto_mapping": null,
+  "base_model_name_or_path": "/workspace/Finetune_AgriFair/models/Qwen__Qwen3-4B-Instruct-2507",
+  "bias": "none",
+  "corda_config": null,
+  "ensure_weight_tying": false,
+  "eva_config": null,
+  "exclude_modules": null,
+  "fan_in_fan_out": false,
+  "inference_mode": true,
+  "init_lora_weights": true,
+  "layer_replication": null,
+  "layers_pattern": null,
+  "layers_to_transform": null,
+  "loftq_config": {},
+  "lora_alpha": 16,
+  "lora_bias": false,
+  "lora_dropout": 0.05,
+  "lora_ga_config": null,
+  "megatron_config": null,
+  "megatron_core": "megatron.core",
+  "modules_to_save": null,
+  "monteclora_config": null,
+  "peft_type": "LORA",
+  "peft_version": "0.20.0",
+  "qalora_group_size": 16,
+  "r": 8,
+  "rank_pattern": {},
+  "revision": null,
+  "target_modules": [
+    "o_proj",
+    "gate_proj",
+    "v_proj",
+    "up_proj",
+    "k_proj",
+    "q_proj",
+    "down_proj"
+  ],
+  "target_parameters": null,
+  "task_type": "CAUSAL_LM",
+  "trainable_token_indices": null,
+  "use_bdlora": null,
+  "use_dora": false,
+  "use_qalora": false,
+  "use_rslora": false,
+  "velora_config": null
+}

broad-instruct/reference_vanilla_lora/seed_42/adapter_model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:49f14a832e1b45bb20cd84d645aee67c4ab6dac5a6ffa77e4b221b89bceb434c
+size 66126768

manifest.json

@@ -108,6 +108,42 @@
       "target_modules": "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*\\.(?:layers|h)\\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23)\\.[^.]+\\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)",
       "trainable_parameter_percentage": 0.273
     },
+    {
+      "path_in_repo": "broad-instruct/baseline_dart/seed_42",
+      "model_tier": "broad-instruct",
+      "base_model": "Qwen/Qwen3-4B-Instruct-2507",
+      "base_model_revision": "cdbee75f17c01a7cc42f958dc650907174af0554",
+      "method": "baseline_dart",
+      "random_seed": 42,
+      "lora_rank_range": "27-28",
+      "lora_alpha": 54,
+      "target_modules": "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*\\.(?:layers|h)\\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32|33|34|35)\\.[^.]+\\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)",
+      "trainable_parameter_percentage": null
+    },
+    {
+      "path_in_repo": "broad-instruct/baseline_dart/seed_43",
+      "model_tier": "broad-instruct",
+      "base_model": "Qwen/Qwen3-4B-Instruct-2507",
+      "base_model_revision": "cdbee75f17c01a7cc42f958dc650907174af0554",
+      "method": "baseline_dart",
+      "random_seed": 43,
+      "lora_rank_range": "27-28",
+      "lora_alpha": 54,
+      "target_modules": "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*\\.(?:layers|h)\\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32|33|34|35)\\.[^.]+\\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)",
+      "trainable_parameter_percentage": null
+    },
+    {
+      "path_in_repo": "broad-instruct/baseline_dart/seed_44",
+      "model_tier": "broad-instruct",
+      "base_model": "Qwen/Qwen3-4B-Instruct-2507",
+      "base_model_revision": "cdbee75f17c01a7cc42f958dc650907174af0554",
+      "method": "baseline_dart",
+      "random_seed": 44,
+      "lora_rank_range": "27-28",
+      "lora_alpha": 54,
+      "target_modules": "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*\\.(?:layers|h)\\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32|33|34|35)\\.[^.]+\\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)",
+      "trainable_parameter_percentage": null
+    },
     {
       "path_in_repo": "broad-instruct/baseline_fairnet/seed_42",
       "model_tier": "broad-instruct",
@@ -156,6 +192,18 @@
       "target_modules": "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*\\.(?:layers|h)\\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23)\\.[^.]+\\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)",
       "trainable_parameter_percentage": null
     },
+    {
+      "path_in_repo": "broad-instruct/baseline_lftf/seed_42",
+      "model_tier": "broad-instruct",
+      "base_model": "Qwen/Qwen3-4B-Instruct-2507",
+      "base_model_revision": "cdbee75f17c01a7cc42f958dc650907174af0554",
+      "method": "baseline_lftf",
+      "random_seed": 42,
+      "lora_rank_range": "64-64",
+      "lora_alpha": 128,
+      "target_modules": "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*\\.(?:layers|h)\\.(21|22|23|24|25|26|27|28|29|30|31|32|33|34|35)\\.[^.]+\\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)",
+      "trainable_parameter_percentage": null
+    },
     {
       "path_in_repo": "broad-instruct/baseline_pedal/seed_42",
       "model_tier": "broad-instruct",
@@ -168,6 +216,18 @@
       "target_modules": "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*\\.(?:layers|h)\\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32|33|34|35)\\.[^.]+\\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)",
       "trainable_parameter_percentage": null
     },
+    {
+      "path_in_repo": "broad-instruct/baseline_regift/seed_42",
+      "model_tier": "broad-instruct",
+      "base_model": "Qwen/Qwen3-4B-Instruct-2507",
+      "base_model_revision": "cdbee75f17c01a7cc42f958dc650907174af0554",
+      "method": "baseline_regift",
+      "random_seed": 42,
+      "lora_rank_range": "27-28",
+      "lora_alpha": 54,
+      "target_modules": "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*\\.(?:layers|h)\\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31|32|33|34|35)\\.[^.]+\\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)",
+      "trainable_parameter_percentage": null
+    },
     {
       "path_in_repo": "broad-instruct/graft_proposed/seed_42",
       "model_tier": "broad-instruct",
@@ -252,6 +312,26 @@
       "target_modules": "(?!.*(?:vision_tower|vision_model|visual|image_encoder|multi_modal_projector|audio_tower)).*\\.(?:layers|h)\\.(0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23)\\.[^.]+\\.(q_proj|k_proj|v_proj|o_proj|gate_proj|up_proj|down_proj)",
       "trainable_parameter_percentage": 2.4794
     },
+    {
+      "path_in_repo": "broad-instruct/reference_vanilla_lora/seed_42",
+      "model_tier": "broad-instruct",
+      "base_model": "Qwen/Qwen3-4B-Instruct-2507",
+      "base_model_revision": "cdbee75f17c01a7cc42f958dc650907174af0554",
+      "method": "reference_vanilla_lora",
+      "random_seed": 42,
+      "lora_rank_range": 8,
+      "lora_alpha": 16,
+      "target_modules": [
+        "o_proj",
+        "gate_proj",
+        "v_proj",
+        "up_proj",
+        "k_proj",
+        "q_proj",
+        "down_proj"
+      ],
+      "trainable_parameter_percentage": null
+    },
     {
       "path_in_repo": "broad-instruct/reference_vanilla_lora_loao_gender/seed_42",
       "model_tier": "broad-instruct",
@@ -1269,5 +1349,5 @@
       "trainable_parameter_percentage": null
     }
   ],
-  "count": 95
+  "count": 101
 }