From 53475ecc059cfdd3df1c5daa52508e9fd51cd143 Mon Sep 17 00:00:00 2001 From: Eric Date: Thu, 18 Apr 2024 15:27:58 +0000 Subject: [PATCH 1/2] template info update --- docs/source/examples/DATASETS.md | 13 +- .../supported_conversation_template.md | 124 +++++++++++++++++- 2 files changed, 127 insertions(+), 10 deletions(-) diff --git a/docs/source/examples/DATASETS.md b/docs/source/examples/DATASETS.md index 1efcf061b..f01d1e559 100644 --- a/docs/source/examples/DATASETS.md +++ b/docs/source/examples/DATASETS.md @@ -179,10 +179,15 @@ Tips: Conversations should be formatted before feeding into the model. As of now, we've preset the conversation template for following models: -| Template Name | Filled Example | Detailed Template | -| ------------- | -------------- | ----------------- | -| `llama2` | `[INST] <>`
`You are a chatbot developed by LMFlow team.`
`<>`

`Who are you? [/INST] I am a chatbot developed by LMFlow team.[INST] How old are you? [/INST] I don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.` | [Link](./supported_conversation_template.md#llama-2) | -| `qwen2` | `<\|im_start\|>system`
`You are a Machine Learning expert.<\|im_end\|>`
`<\|im_start\|>user`
`Hi!<\|im_end\|>`
`<\|im_start\|>assistant`
`Hi, nice to meet you!<\|im_end\|>`
`<\|im_start\|>user`
`Who are you?<\|im_end\|>`
`<\|im_start\|>assistant`
`I'm a Machine Learning expert.<\|im_end\|>`
| [Link](./supported_conversation_template.md#qwen-2-qwen-1-5) | +| Template Name | Filled Example | Detailed Template | Preseted | +| ------------- | -------------- | ----------------- | --- | +| `internlm2` | Work in progress :sweat_drops: | [Link](./supported_conversation_template.md#internlm2) | | +| `chatglm3` | Work in progress :sweat_drops: | [Link](./supported_conversation_template.md#chatglm-3) | | +| `llama2` | `[INST] <>`
`You are a chatbot developed by LMFlow team.`
`<>`

`Who are you? [/INST] I am a chatbot developed by LMFlow team.[INST] How old are you? [/INST] I don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.` | [Link](./supported_conversation_template.md#llama-2) | :white_check_mark: | +| `mixtral8x7b` | Work in progress :sweat_drops: | [Link](./supported_conversation_template.md#mixtral-8x7b) | | +| `mixtral8x22b` | Work in progress :sweat_drops: | [Link](./supported_conversation_template.md#mixtral-8x22b) | | +| `qwen2` | `<\|im_start\|>system`
`You are a chatbot developed by LMFlow team.<\|im_end\|>`
`<\|im_start\|>user`
`Who are you?<\|im_end\|>`
`<\|im_start\|>assistant`
`I am a chatbot developed by LMFlow team.<\|im_end\|>`
`<\|im_start\|>user`
`How old are you?<\|im_end\|>`
`<\|im_start\|>assistant`
`I don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.<\|im_end\|>`
| [Link](./supported_conversation_template.md#qwen-2) | :white_check_mark: | +| `yi` | `<\|im_start\|>system`
`You are a chatbot developed by LMFlow team.<\|im_end\|>`
`<\|im_start\|>user`
`Who are you?<\|im_end\|>`
`<\|im_start\|>assistant`
`I am a chatbot developed by LMFlow team.<\|im_end\|>`
`<\|im_start\|>user`
`How old are you?<\|im_end\|>`
`<\|im_start\|>assistant`
`I don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.<\|im_end\|>`
| [Link](./supported_conversation_template.md#yi) | | Passing the template name to the `--conversation_template` argument to apply the corresponding conversation template: ```sh diff --git a/docs/source/examples/supported_conversation_template.md b/docs/source/examples/supported_conversation_template.md index f345c1389..4b20ac57e 100644 --- a/docs/source/examples/supported_conversation_template.md +++ b/docs/source/examples/supported_conversation_template.md @@ -1,8 +1,40 @@ # Supported Conversation Template ## Table of Contents +- [InternLM2](#internlm2) +- [ChatGLM-3](#chatglm-3) - [Llama-2](#llama-2) -- [Qwen-2 (Qwen-1.5)](#qwen-2-qwen-15) +- [Mixtral 8x7B](#mixtral-8x7b) +- [Mixtral 8x22B](#mixtral-8x22b) +- [Qwen-2](#qwen-2) +- [Yi](#yi) + + +## InternLM2 +```{admonition} **Work in Progress** +:class: info + +This template is not preseted in LMFlow currently. We are working on it and will update it soon. +``` +### jinja template +[Reference](https://huggingface.co/internlm/internlm2-chat-20b/blob/477d4748322a8a3b28f62b33f0f6dd353cd0b66d/tokenizer_config.json#L93) +``` +{{ bos_token }}{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %} +``` + + +## ChatGLM-3 +```{admonition} **Work in Progress** +:class: info + +This template is not preseted in LMFlow currently. We are working on it and will update it soon. +``` +### jinja template +[Reference](https://huggingface.co/THUDM/chatglm3-6b/blob/103caa40027ebfd8450289ca2f278eac4ff26405/tokenizer_config.json#L42) +``` +{% for message in messages %}{% if loop.first %}[gMASK]sop<|{{ message['role'] }}|>\n {{ message['content'] }}{% else %}<|{{ message['role'] }}|>\n {{ message['content'] }}{% endif %}{% endfor %}{% if add_generation_prompt %}<|assistant|>{% endif %} +``` + ## Llama-2 ### With a system message @@ -26,8 +58,9 @@ ``` ### jinja template +[Reference](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/blob/main/tokenizer_config.json#L12) ``` -work in progress +{% if messages[0]['role'] == 'system' %}{% set loop_messages = messages[1:] %}{% set system_message = messages[0]['content'] %}{% else %}{% set loop_messages = messages %}{% set system_message = false %}{% endif %}{% for message in loop_messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if loop.index0 == 0 and system_message != false %}{% set content = '<>\\n' + system_message + '\\n<>\\n\\n' + message['content'] %}{% else %}{% set content = message['content'] %}{% endif %}{% if message['role'] == 'user' %}{{ bos_token + '[INST] ' + content.strip() + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ ' ' + content.strip() + ' ' + eos_token }}{% endif %}{% endfor %} ``` ### Filled Example @@ -36,7 +69,45 @@ work in progress ``` -## Qwen-2 (Qwen-1.5) +## Mixtral 8x7B +```{admonition} **Work in Progress** +:class: info + +This template is not preseted in LMFlow currently. We are working on it and will update it soon. +``` + +```{admonition} NOTICE +:class: warning + +The conversation template for Mixtral 8x7B is slightly different from the template for Mixtral 8x22B. +``` +### jinja template +[Reference](https://huggingface.co/mistralai/Mixtral-8x7B-Instruct-v0.1/blob/1e637f2d7cb0a9d6fb1922f305cb784995190a83/tokenizer_config.json#L42) +``` +{{ bos_token }}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ '[INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ message['content'] + eos_token}}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %} +``` + + +## Mixtral 8x22B +```{admonition} **Work in Progress** +:class: info + +This template is not preseted in LMFlow currently. We are working on it and will update it soon. +``` + +```{admonition} NOTICE +:class: warning + +The conversation template for Mixtral 8x22B is slightly different from the template for Mixtral 8x7B. +``` +### jinja template +[Reference](https://huggingface.co/mistralai/Mixtral-8x22B-Instruct-v0.1) +``` +{{bos_token}}{% for message in messages %}{% if (message['role'] == 'user') != (loop.index0 % 2 == 0) %}{{ raise_exception('Conversation roles must alternate user/assistant/user/assistant/...') }}{% endif %}{% if message['role'] == 'user' %}{{ ' [INST] ' + message['content'] + ' [/INST]' }}{% elif message['role'] == 'assistant' %}{{ ' ' + message['content'] + ' ' + eos_token}}{% else %}{{ raise_exception('Only user and assistant roles are supported!') }}{% endif %}{% endfor %} +``` + + +## Qwen-2 ### With a system message ``` <|im_start|>system\n{{system_message}}<|im_end|>\n<|im_start|>user\n{{user_message_0}}<|im_end|>\n @@ -58,11 +129,52 @@ work in progress ``` ### jinja template +[Reference](https://huggingface.co/Qwen/Qwen1.5-72B/blob/93bac0d1ae83d50c43b1793e2d74a00dc43a4c36/tokenizer_config.json#L31) ``` -work in progress +"{% for message in messages %}{% if loop.first and messages[0]['role'] != 'system' %}{{ '<|im_start|>system\nYou are a helpful assistant<|im_end|>\n' }}{% endif %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %} ``` ### Filled Example ``` -<|im_start|>system\nYou are a Machine Learning expert.<|im_end|>\n<|im_start|>user\nIntroduce me the LISA fine-tuning technique.<|im_end|>\n<|im_start|>assistant\nLISA is short for Layerwise Importance Sampled AdamW. It is a novel and memory-efficient training strategy for large language models that outperforms existing methods like LoRA by selectively freezing layers during optimization.<|im_end|>\n<|im_start|>user\nWhere can I find the code?<|im_end|>\n<|im_start|>assistant\nLMFlow supports LISA. For more details, please refer to their official github repository.<|im_end|>\n -``` \ No newline at end of file +<|im_start|>system\nYou are a chatbot developed by LMFlow team.<|im_end|>\n<|im_start|>user\nWho are you?<|im_end|>\n<|im_start|>assistant\nI am a chatbot developed by LMFlow team.<|im_end|>\n<|im_start|>user\nHow old are you?<|im_end|>\n<|im_start|>assistant\nI don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.<|im_end|>\n +``` + + +## Yi +```{admonition} **Work in Progress** +:class: info + +This template is not preseted in LMFlow currently. We are working on it and will update it soon. +``` +### With a system message +``` +<|im_start|>system\n{{system_message}}<|im_end|>\n<|im_start|>user\n{{user_message_0}}<|im_end|>\n +``` + +### Without a system message +``` +<|im_start|>user\n{{user_message_0}}<|im_end|>\n +``` + +### A complete conversation +``` +<|im_start|>system\n{{system_message}}<|im_end|>\n<|im_start|>user\n{{user_message_0}}<|im_end|>\n<|im_start|>assistant\n{{assistant_reply_0}}<|im_end|>\n +``` + +### Multiple rounds +``` +<|im_start|>system\n{{system_message}}<|im_end|>\n<|im_start|>user\n{{user_message_0}}<|im_end|>\n<|im_start|>assistant\n{{assistant_reply_0}}<|im_end|>\n<|im_start|>user\n{{user_message_1}}<|im_end|>\n<|im_start|>assistant\n{{assistant_reply_1}}<|im_end|>\n +``` + +### jinja template +[Reference](https://huggingface.co/01-ai/Yi-34B-Chat/blob/c556c018b58980fb651ff4952d86cd5250a713d0/tokenizer_config.json#L60) +``` +{% if not add_generation_prompt is defined %}{% set add_generation_prompt = false %}{% endif %}{% for message in messages %}{{'<|im_start|>' + message['role'] + '\n' + message['content'] + '<|im_end|>' + '\n'}}{% endfor %}{% if add_generation_prompt %}{{ '<|im_start|>assistant\n' }}{% endif %} +``` + +### Filled Example +``` +<|im_start|>system\nYou are a chatbot developed by LMFlow team.<|im_end|>\n<|im_start|>user\nWho are you?<|im_end|>\n<|im_start|>assistant\nI am a chatbot developed by LMFlow team.<|im_end|>\n<|im_start|>user\nHow old are you?<|im_end|>\n<|im_start|>assistant\nI don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.<|im_end|>\n +``` + + From 2ab5df8250ff9b926b4434b2f7a2ea33af492c30 Mon Sep 17 00:00:00 2001 From: Eric Date: Thu, 18 Apr 2024 15:34:34 +0000 Subject: [PATCH 2/2] remove emoji (doc page doesn't support) --- docs/source/examples/DATASETS.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/source/examples/DATASETS.md b/docs/source/examples/DATASETS.md index f01d1e559..ad663ce6b 100644 --- a/docs/source/examples/DATASETS.md +++ b/docs/source/examples/DATASETS.md @@ -181,12 +181,12 @@ Conversations should be formatted before feeding into the model. As of now, we'v | Template Name | Filled Example | Detailed Template | Preseted | | ------------- | -------------- | ----------------- | --- | -| `internlm2` | Work in progress :sweat_drops: | [Link](./supported_conversation_template.md#internlm2) | | -| `chatglm3` | Work in progress :sweat_drops: | [Link](./supported_conversation_template.md#chatglm-3) | | -| `llama2` | `[INST] <>`
`You are a chatbot developed by LMFlow team.`
`<>`

`Who are you? [/INST] I am a chatbot developed by LMFlow team.[INST] How old are you? [/INST] I don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.` | [Link](./supported_conversation_template.md#llama-2) | :white_check_mark: | -| `mixtral8x7b` | Work in progress :sweat_drops: | [Link](./supported_conversation_template.md#mixtral-8x7b) | | -| `mixtral8x22b` | Work in progress :sweat_drops: | [Link](./supported_conversation_template.md#mixtral-8x22b) | | -| `qwen2` | `<\|im_start\|>system`
`You are a chatbot developed by LMFlow team.<\|im_end\|>`
`<\|im_start\|>user`
`Who are you?<\|im_end\|>`
`<\|im_start\|>assistant`
`I am a chatbot developed by LMFlow team.<\|im_end\|>`
`<\|im_start\|>user`
`How old are you?<\|im_end\|>`
`<\|im_start\|>assistant`
`I don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.<\|im_end\|>`
| [Link](./supported_conversation_template.md#qwen-2) | :white_check_mark: | +| `internlm2` | Work in progress | [Link](./supported_conversation_template.md#internlm2) | | +| `chatglm3` | Work in progress | [Link](./supported_conversation_template.md#chatglm-3) | | +| `llama2` | `[INST] <>`
`You are a chatbot developed by LMFlow team.`
`<>`

`Who are you? [/INST] I am a chatbot developed by LMFlow team.[INST] How old are you? [/INST] I don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.` | [Link](./supported_conversation_template.md#llama-2) | Yes | +| `mixtral8x7b` | Work in progress | [Link](./supported_conversation_template.md#mixtral-8x7b) | | +| `mixtral8x22b` | Work in progress | [Link](./supported_conversation_template.md#mixtral-8x22b) | | +| `qwen2` | `<\|im_start\|>system`
`You are a chatbot developed by LMFlow team.<\|im_end\|>`
`<\|im_start\|>user`
`Who are you?<\|im_end\|>`
`<\|im_start\|>assistant`
`I am a chatbot developed by LMFlow team.<\|im_end\|>`
`<\|im_start\|>user`
`How old are you?<\|im_end\|>`
`<\|im_start\|>assistant`
`I don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.<\|im_end\|>`
| [Link](./supported_conversation_template.md#qwen-2) | Yes | | `yi` | `<\|im_start\|>system`
`You are a chatbot developed by LMFlow team.<\|im_end\|>`
`<\|im_start\|>user`
`Who are you?<\|im_end\|>`
`<\|im_start\|>assistant`
`I am a chatbot developed by LMFlow team.<\|im_end\|>`
`<\|im_start\|>user`
`How old are you?<\|im_end\|>`
`<\|im_start\|>assistant`
`I don't age like humans do. I exist as a piece of software, so I don't have a concept of age in the traditional sense.<\|im_end\|>`
| [Link](./supported_conversation_template.md#yi) | | Passing the template name to the `--conversation_template` argument to apply the corresponding conversation template: