# Finetuning BART on a multi-input sequence to sequence task

**URL:** <https://discuss.huggingface.co/t/finetuning-bart-on-a-multi-input-sequence-to-sequence-task/10201>\
**Category:** 🤗Transformers\
**Created:** [September 22, 2021, 1:26am UTC](https://discuss.huggingface.co/t/finetuning-bart-on-a-multi-input-sequence-to-sequence-task/10201 "2021-09-22T01:26:31Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![NR1](https://avatars.discourse-cdn.com/v4/letter/n/ba9def/32.png) [@NR1](https://discuss.huggingface.co/u/NR1)\
**Post date:** [September 22, 2021, 1:26am UTC](https://discuss.huggingface.co/t/finetuning-bart-on-a-multi-input-sequence-to-sequence-task/10201/1 "2021-09-22T01:26:31Z")

</div>

I finetuned bart-base on a sequence to sequence task and I have the following questions:

a) Currently I structured the input and output for the bart model in “t5-style” by adding prefixes in front of each piece of input. For bart how should I give multiple inputs (or train it to return multiple outputs) to the model (is there special token to separate inputs, should I continue the t5-style prefixes, etc.)? Also, how would I do this for gpt-2/gpt-neo?

b) When finetuned with prefixes, the target data is formatted with “output: …”, however, the finetuned-bart returns “outputoutput: …”. Why is this repetiton occurring? Also, does the Bart tokenizer automatically add the eos token?

c) Also does the trainer api automaticallly handle `adjust_logits_during_generation` and `decoder_start_token_id` as discussed in this [post](https://discuss.huggingface.co/t/bart-base-rouge-scores/683)?

@valhalla Could you please help with this? This is my first project training an nlp model, and I would really appreciate any information you can offer regarding my questions.