Conversation
| if instruction: | ||
| user_content.append({"type": "text", "text": str(instruction)}) | ||
| messages.append({"role": "user", "content": user_content}) | ||
| messages.append({"role": "assistant", "content": response}) |
There was a problem hiding this comment.
Hi Stephen, I got a question for instruction. It seems that the current version load the same instruction template for all turns (the audio sources are different but the textual questions are the same). In retrieval cases, I would like to ask the onsite and offsite of specific single events in each audio source. So, may I modify the instruction to be turn specific? Thanks!
There was a problem hiding this comment.
Yeah, this example is set as a base example just to show people how to do LALM training in Auden with a standard hf transformers setup. You can have you own version in terms of manifest preparation and data loading. The suggestion is to prepare the conversation offline and put it in your manifest.
2 participants
Add an example with more native HF transformers interface for large audio language model (LALM) training.