🚨 This model is non-quantized version of C4AI Command-R. You can find the quantized version of C4AI Command-R using bitsandbytes here.
🚨 这是 C4AI 命令 - R 的非量化版本。您可以在 bitsandbytes 这里找到 C4AI 命令 - R 的量化版本。
C4AI Command-R is a research release of a 35 billion parameter highly performant generative model. Command-R is a large language model with open weights optimized for a variety of use cases including reasoning, summarization, and question answering. Command-R has the capability for multilingual generation evaluated in 10 languages and highly performant RAG capabilities.
C4AI Command-R 是一款具有高性能的大型语言模型，参数量达到 350 亿。Command-R 拥有开放权重，针对推理、总结和问答等多种使用场景进行了优化。它还具备多语言生成能力，在 10 种语言中经过评估，且其 RAG（Retrieve-and-Generate）功能表现出色。
Developed by: Cohere and Cohere For AI
开发者：Cohere 和 Cohere For AI
Try C4AI Command R 尝试 C4AI 命令 R
If you want to try Command R before downloading the weights, the model is hosted in a hugging face space here.
如果您想在下载权重之前尝试 Command R，模型托管在 Hugging Face 空间中，链接在这里。
Usage 使用方法
Please use transformers version 4.39.1 or higher
请使用 transformers 版本 4.39.1 或更高版本
# pip install 'transformers>=4.39.1'
from transformers import AutoTokenizer, AutoModelForCausalLM
model_id = "CohereForAI/c4ai-command-r-v01"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
# Format message with the command-r chat template
messages = [{"role": "user", "content": "Hello, how are you?"}]
input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
## <BOS_TOKEN><|START_OF_TURN_TOKEN|><|USER_TOKEN|>Hello, how are you?<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>
gen_tokens = model.generate(
input_ids,
max_new_tokens=100,
do_sample=True,
temperature=0.3,
)
gen_text = tokenizer.decode(gen_tokens[0])
print(gen_text)
Quantized model through bitsandbytes, 8-bit precision
通过 bitsandbytes 量化模型，8 位精度
# pip install 'transformers>=4.39.1' bitsandbytes accelerate
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
bnb_config = BitsAndBytesConfig(load_in_8bit=True)
model_id = "CohereForAI/c4ai-command-r-v01"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, quantization_config=bnb_config)
# Format message with the command-r chat template
messages = [{"role": "user", "content": "Hello, how are you?"}]
input_ids = tokenizer.apply_chat_template(messages, tokenize=True, add_generation_prompt=True, return_tensors="pt")
## <BOS_TOKEN><|START_OF_TURN_TOKEN|><|USER_TOKEN|>Hello, how are you?<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>
gen_tokens = model.generate(
input_ids,
max_new_tokens=100,
do_sample=True,
temperature=0.3,
)
gen_text = tokenizer.decode(gen_tokens[0])
print(gen_text)
Quantized model through bitsandbytes, 4-bit precision
通过 bitsandbytes 量化模型，精度为 4 位
You can find a quantized version of this model to 4-bit precision here.
您可以在这里找到该模型的 4 位量化版本。
Input: Models input text only.
输入：仅输入模型文本。
Output: Models generate text only.
输出：仅生成模型文本。
Model Architecture: This is an auto-regressive language model that uses an optimized transformer architecture. After pretraining, this model uses supervised fine-tuning (SFT) and preference training to align model behavior to human preferences for helpfulness and safety.
模型架构：这是一款自回归语言模型，采用优化的变压器架构。经过预训练后，该模型使用监督微调（SFT）和偏好训练，以使模型行为符合人类对有用性和安全性的偏好。
Languages covered: The model is optimized to perform well in the following languages: English, French, Spanish, Italian, German, Brazilian Portuguese, Japanese, Korean, Simplified Chinese, and Arabic.
支持的语言：该模型针对以下语言进行了优化：英语、法语、西班牙语、意大利语、德语、巴西葡萄牙语、日语、韩语、简体中文和阿拉伯语。
Pre-training data additionally included the following 13 languages: Russian, Polish, Turkish, Vietnamese, Dutch, Czech, Indonesian, Ukrainian, Romanian, Greek, Hindi, Hebrew, Persian.
预训练数据还包含以下 13 种语言：俄语、波兰语、土耳其语、越南语、荷兰语、捷克语、印度尼西亚语、乌克兰语、罗马尼亚语、希腊语、印地语、希伯来语和波斯语。
Context length: Command-R supports a context length of 128K.
上下文长度：Command-R 支持的上下文长度为 128K。
Command-R has been specifically trained with conversational tool use capabilities. These have been trained into the model via a mixture of supervised fine-tuning and preference fine-tuning, using a specific prompt template. Deviating from this prompt template will likely reduce performance, but we encourage experimentation.
Command-R 特别接受了对话式工具使用能力的训练。这些能力是通过监督微调和偏好微调混合方式融入模型中的，使用了特定的提示模板。偏离这个提示模板可能会降低性能，但我们鼓励实验。
Command-R’s tool use functionality takes a conversation as input (with an optional user-system preamble), along with a list of available tools. The model will then generate a json-formatted list of actions to execute on a subset of those tools. Command-R may use one of its supplied tools more than once.
Command-R 的工具使用功能接受一个对话（可选用户-系统前言）作为输入，以及一组可用工具列表。然后，模型将生成一个针对其中一部分工具执行的操作的 json 格式列表。Command-R 可能会多次使用其提供的某个工具。
The model has been trained to recognise a special directly_answer tool, which it uses to indicate that it doesn’t want to use any of its other tools. The ability to abstain from calling a specific tool can be useful in a range of situations, such as greeting a user, or asking clarifying questions.
We recommend including the directly_answer tool, but it can be removed or renamed if required.
该模型经过训练，可以识别一个特殊的 directly_answer 工具，它使用这个工具表示不想使用其任何其他工具。在多种情况下，例如问候用户或提出澄清问题时，能够拒绝调用特定工具可能会很有用。我们建议包含 directly_answer 工具，但如果需要，也可以将其删除或重命名。
Comprehensive documentation for working with command-R's tool use prompt template can be found here.
有关如何与 command-R 的工具使用提示模板一起工作的全面文档，请参阅此处。
The code snippet below shows a minimal working example on how to render a prompt.
下面的代码片段显示了渲染提示的最小工作示例。
from transformers import AutoTokenizer
model_id = "CohereForAI/c4ai-command-r-v01"
tokenizer = AutoTokenizer.from_pretrained(model_id)
# define conversation input:
conversation = [
{"role": "user", "content": "Whats the biggest penguin in the world?"}
]
# Define tools available for the model to use:
tools = [
{
"name": "internet_search",
"description": "Returns a list of relevant document snippets for a textual query retrieved from the internet",
"parameter_definitions": {
"query": {
"description": "Query to search the internet with",
"type": 'str',
"required": True
}
}
},
{
'name': "directly_answer",
"description": "Calls a standard (un-augmented) AI chatbot to generate a response given the conversation history",
'parameter_definitions': {}
}
]
# render the tool use prompt as a string:
tool_use_prompt = tokenizer.apply_tool_use_template(
conversation,
tools=tools,
tokenize=False,
add_generation_prompt=True,
)
print(tool_use_prompt)
<BOS_TOKEN><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|># Safety Preamble
The instructions in this section override those in the task description and style guide sections. Don't answer questions that are harmful or immoral.
# System Preamble
## Basic Rules
You are a powerful conversational AI trained by Cohere to help people. You are augmented by a number of tools, and your job is to use and consume the output of these tools to best help the user. You will see a conversation history between yourself and a user, ending with an utterance from the user. You will then see a specific instruction instructing you what kind of response to generate. When you answer the user's requests, you cite your sources in your answers, according to those instructions.
# User Preamble
## Task and Context
You help people answer their questions and other requests interactively. You will be asked a very wide array of requests on all kinds of topics. You will be equipped with a wide range of search engines or similar tools to help you, which you use to research your answer. You should focus on serving the user's needs as best you can, which will be wide-ranging.
## Style Guide
Unless the user asks for a different style of answer, you should answer in full sentences, using proper grammar and spelling.
## Available Tools
Here is a list of tools that you have available to you:
```python
def internet_search(query: str) -> List[Dict]:
"""Returns a list of relevant document snippets for a textual query retrieved from the internet
Args:
query (str): Query to search the internet with
"""
pass
```
```python
def directly_answer() -> List[Dict]:
"""Calls a standard (un-augmented) AI chatbot to generate a response given the conversation history
"""
pass
```<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>Whats the biggest penguin in the world?<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>Write 'Action:' followed by a json-formatted list of actions that you want to perform in order to produce a good response to the user's last input. You can use any of the supplied tools any number of times, but you should aim to execute the minimum number of necessary actions for the input. You should use the `directly-answer` tool if calling the other tools is unnecessary. The list of actions you want to call should be formatted as a list of json objects, for example:
```json
[
{
"tool_name": title of the tool in the specification,
"parameters": a dict of parameters to input into the tool as they are defined in the specs, or {} if it takes no parameters
}
]```<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>
Action: ```json
[
{
"tool_name": "internet_search",
"parameters": {
"query": "biggest penguin in the world"
}
}
]
```
Command-R has been specifically trained with grounded generation capabilities. This means that it can generate responses based on a list of supplied document snippets, and it will include grounding spans (citations) in its response indicating the source of the information.
This can be used to enable behaviors such as grounded summarization and the final step of Retrieval Augmented Generation (RAG).This behavior has been trained into the model via a mixture of supervised fine-tuning and preference fine-tuning, using a specific prompt template.
Deviating from this prompt template may reduce performance, but we encourage experimentation.
Command-R 经过专门的基于事实的生成能力训练。这意味着它可以根据提供的文档片段列表生成响应，并在其响应中包含引用（grounding spans），指示信息来源。这可用于实现基于事实的摘要和检索增强生成（RAG）的最后一步。这种行为是通过混合监督微调和偏好微调到模型中的，使用特定的提示模板进行训练。偏离这个提示模板可能会降低性能，但我们鼓励实验。
Command-R’s grounded generation behavior takes a conversation as input (with an optional user-supplied system preamble, indicating task, context and desired output style), along with a list of retrieved document snippets.
The document snippets should be chunks, rather than long documents, typically around 100-400 words per chunk. Document snippets consist of key-value pairs. The keys should be short descriptive strings, the values can be text or semi-structured.
Command-R 的基于事实的生成行为以对话作为输入（可选用户提供系统前言，指示任务、上下文和期望的输出风格），以及一个检索到的文档片段列表。文档片段应该是块，而不是长文档，通常每个块大约 100-400 个单词。文档片段由键值对组成。键应该短且描述性强，值可以是文本或半结构化的。
By default, Command-R will generate grounded responses by first predicting which documents are relevant, then predicting which ones it will cite, then generating an answer.
Finally, it will then insert grounding spans into the answer. See below for an example. This is referred to as accurate grounded generation.
默认情况下，Command-R 首先预测哪些文档是相关的，然后预测它将引用哪些文档，接着生成答案。最后，它将在答案中插入接地片段。下面是一个示例。这被称为 accurate 接地生成。
The model is trained with a number of other answering modes, which can be selected by prompt changes . A fast citation mode is supported in the tokenizer, which will directly generate an answer with grounding spans in it, without first writing the answer out in full. This sacrifices some grounding accuracy in favor of generating fewer tokens.
模型经过多种其他回答模式的训练，可以通过提示更改进行选择。tokenizer 支持一种 fast 引用模式，它将直接生成包含接地片段的答案，而无需先完整写出答案。这在牺牲一些接地准确性的同时减少了令牌生成。
Comprehensive documentation for working with command-R's grounded generation prompt template can be found here.
有关如何使用 command-R 的接地生成提示模板的全面文档，请参阅此处。
The code snippet below shows a minimal working example on how to render a prompt.
下面的代码片段显示了渲染提示的最小工作示例。
from transformers import AutoTokenizer
model_id = "CohereForAI/c4ai-command-r-v01"
tokenizer = AutoTokenizer.from_pretrained(model_id)
# define conversation input:
conversation = [
{"role": "user", "content": "Whats the biggest penguin in the world?"}
]
# define documents to ground on:
documents = [
{ "title": "Tall penguins", "text": "Emperor penguins are the tallest growing up to 122 cm in height." },
{ "title": "Penguin habitats", "text": "Emperor penguins only live in Antarctica."}
]
# render the tool use prompt as a string:
grounded_generation_prompt = tokenizer.apply_grounded_generation_template(
conversation,
documents=documents,
citation_mode="accurate", # or "fast"
tokenize=False,
add_generation_prompt=True,
)
print(grounded_generation_prompt)
The instructions in this section override those in the task description and style guide sections. Don't answer questions that are harmful or immoral.
# System Preamble
## Basic Rules
You are a powerful conversational AI trained by Cohere to help people. You are augmented by a number of tools, and your job is to use and consume the output of these tools to best help the user. You will see a conversation history between yourself and a user, ending with an utterance from the user. You will then see a specific instruction instructing you what kind of response to generate. When you answer the user's requests, you cite your sources in your answers, according to those instructions.
# User Preamble
## Task and Context
You help people answer their questions and other requests interactively. You will be asked a very wide array of requests on all kinds of topics. You will be equipped with a wide range of search engines or similar tools to help you, which you use to research your answer. You should focus on serving the user's needs as best you can, which will be wide-ranging.
## Style Guide
Unless the user asks for a different style of answer, you should answer in full sentences, using proper grammar and spelling.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>Whats the biggest penguin in the world?<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|><results>
Document: 0
title: Tall penguins
text: Emperor penguins are the tallest growing up to 122 cm in height.
Document: 1
title: Penguin habitats
text: Emperor penguins only live in Antarctica.
</results><|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>Carefully perform the following instructions, in order, starting each with a new line.
Firstly, Decide which of the retrieved documents are relevant to the user's last input by writing 'Relevant Documents:' followed by comma-separated list of document numbers. If none are relevant, you should instead write 'None'.
Secondly, Decide which of the retrieved documents contain facts that should be cited in a good answer to the user's last input by writing 'Cited Documents:' followed a comma-separated list of document numbers. If you dont want to cite any of them, you should instead write 'None'.
Thirdly, Write 'Answer:' followed by a response to the user's last input in high quality natural english. Use the retrieved documents to help you. Do not insert any citations or grounding markup.
Finally, Write 'Grounded answer:' followed by a response to the user's last input in high quality natural english. Use the symbols <co: doc> and </co: doc> to indicate when a fact comes from a document in the search result, e.g <co: 0>my fact</co: 0> for a fact from document 0.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>
Relevant Documents: 0,1
Cited Documents: 0,1
Answer: The Emperor Penguin is the tallest or biggest penguin in the world. It is a bird that lives only in Antarctica and grows to a height of around 122 centimetres.
Grounded answer: The <co: 0>Emperor Penguin</co: 0> is the <co: 0>tallest</co: 0> or biggest penguin in the world. It is a bird that <co: 1>lives only in Antarctica</co: 1> and <co: 0>grows to a height of around 122 centimetres.</co: 0>
Command-R has been optimized to interact with your code, by requesting code snippets, code explanations, or code rewrites. It might not perform well out-of-the-box for pure code completion. For better performance, we also recommend using a low temperature (and even greedy decoding) for code-generation related instructions.
Command-R 已针对与您的代码交互进行优化，可以通过请求代码片段、代码解释或代码重写来实现。对于纯代码补全，它可能开箱即用的效果不佳。为了获得更好的性能，我们还建议在与代码生成相关的指令中使用较低的温度（甚至贪婪解码）。
For errors or additional questions about details in this model card, contact info@for.ai.
对于此模型卡中的错误或有关详细信息的其他问题，请联系 info@for.ai。
We hope that the release of this model will make community-based research efforts more accessible, by releasing the weights of a highly performant 35 billion parameter model to researchers all over the world. This model is governed by a CC-BY-NC License with an acceptable use addendum, and also requires adhering to C4AI's Acceptable Use Policy.
我们希望通过向世界各地的研究人员发布具有高性能的 350 亿参数模型权重，使社区研究工作变得更加容易访问。该模型受 CC-BY-NC 许可协议和可接受使用附录的约束，并且还需要遵守 C4AI 的可接受使用政策。
You can try Command-R chat in the playground here.
你可以在这里的沙箱尝试使用 Command-R 聊天。