# Inconsistent \_tied\_weights\_keys path in BERT models

**URL:** <https://discuss.huggingface.co/t/inconsistent-tied-weights-keys-path-in-bert-models/125548>\
**Category:** Beginners\
**Created:** [November 18, 2024, 5:12am UTC](https://discuss.huggingface.co/t/inconsistent-tied-weights-keys-path-in-bert-models/125548 "2024-11-18T05:12:09Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![thisisiron](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/thisisiron/32/35874_2.png) [@thisisiron](https://discuss.huggingface.co/u/thisisiron)\
**Post date:** [November 18, 2024, 5:12am UTC](https://discuss.huggingface.co/t/inconsistent-tied-weights-keys-path-in-bert-models/125548/1 "2024-11-18T05:12:09Z")

</div>

I noticed that there are inconsistent paths in `_tied_weights_keys` between different BERT model classes.

In `BertForMaskedLM` [[link]](https://github.com/huggingface/transformers/blob/13493215abceafc1653af88b045120014fb4c1fc/src/transformers/models/bert/modeling_bert.py#L1285):

```python
_tied_weights_keys = ["predictions.decoder.bias", "cls.predictions.decoder.weight"]

```

In `BertForPreTraining` [[link]](https://github.com/huggingface/transformers/blob/13493215abceafc1653af88b045120014fb4c1fc/src/transformers/models/bert/modeling_bert.py#L1178):

```python
_tied_weights_keys = ["cls.predictions.decoder.bias", "cls.predictions.decoder.weight"]

```

Questions:

1. Why are there different paths for accessing the same weights?
2. Is one of these paths incorrect? If so, which one should be used?
3. If both are valid, what’s the rationale behind using different paths?
