# Using Bart for classification task (Fine-Tuning): Input format

**URL:** <https://discuss.huggingface.co/t/using-bart-for-classification-task-fine-tuning-input-format/81488>\
**Category:** Beginners\
**Created:** [April 12, 2024, 8:08pm UTC](https://discuss.huggingface.co/t/using-bart-for-classification-task-fine-tuning-input-format/81488 "2024-04-12T20:08:08Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![animeshk](https://avatars.discourse-cdn.com/v4/letter/a/7feea3/32.png) [@animeshk](https://discuss.huggingface.co/u/animeshk)\
**Post date:** [April 12, 2024, 8:08pm UTC](https://discuss.huggingface.co/t/using-bart-for-classification-task-fine-tuning-input-format/81488/1 "2024-04-12T20:08:08Z")

</div>

I have a custom dataset which is not text document.

Already have a pretrained BART model. For this pretraining purpose, the format of the data for BART model is like this:

For a single input:

```auto
"input_ids": [code1 code2 sep_token_id mask_token_id sep_token_id code5 sep_token_id code10 .... sep_token_id mask_token_id]
"attention_mask": [1 1 1 1 ... 1 1 1] # number of 1s = the length of "input_ids"
"decoder_input_ids": [EOS code1 code2 201 code3 201 code4 201 code5 201 code10 ... 201 code250]

```

Now want to find tune that model for classification purposes.

**For using BartForSequenceClassification model:**  
I am using the BERTTokenizer to load the vocab file and then using the custom datacollator to generate the proper format of the inputs for these “input\_ids”, “attention\_mask”, “decoder\_input\_ids”, “labels”.

I am getting the error on the 2nd line:

> eos\_mask = input\_ids.eq(self.config.eos\_token\_id)  
> sentence\_representation = x[eos\_mask, :].view(x.size(0), -1, x.size(-1))[:, -1, :]

because in my input\_ids, I don’t have any **eos\_token\_id.**

Could you please tell me what is _ **proper format** _ of the **input\_ids, decoder\_input\_ids** and others in the case of BartForSequenceClassification?

- is the format same as above?

---

<div class="post-metadata">

**Author:** ![animeshk](https://avatars.discourse-cdn.com/v4/letter/a/7feea3/32.png) [@animeshk](https://discuss.huggingface.co/u/animeshk)\
**Post date:** [August 28, 2024, 8:36pm UTC](https://discuss.huggingface.co/t/using-bart-for-classification-task-fine-tuning-input-format/81488/3 "2024-08-28T20:36:23Z")

</div>

@ArthurZ could you please tell me this is the correct format?

---

<div class="post-metadata">

**Author:** ![ArthurZ](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/arthurz/32/26972_2.png) [@ArthurZ](https://discuss.huggingface.co/u/ArthurZ)\
**Post date:** [August 29, 2024, 8:42am UTC](https://discuss.huggingface.co/t/using-bart-for-classification-task-fine-tuning-input-format/81488/4 "2024-08-29T08:42:32Z")

</div>

Hey! The eos token can be replace with anything as long as it serves the purpose. For example you can use the sep\_token or the mask\_token to solve the error.  
I am not entirely sure what you are doing and am not a pro at training bert otherwise, would rather go on the courses we have about training encoder-decoders!