Transformers documentation
提示工程
You are viewing main version, which requires installation from source. If you'd like
regular pip install, checkout the latest stable version (v5.22.1).
提示工程
提示工程（Prompt engineering）或提示（prompting）利用自然语言来提升大型语言模型（LLM）在多种任务上的表现。提示可以引导模型生成期望的输出。在许多情况下，你甚至不需要一个针对特定任务微调的模型，只需要一个好的提示即可。
尝试提示 LLM 对某些文本进行分类。创建提示时，务必提供关于任务以及预期结果形式的非常具体的说明。
from transformers import pipeline
import torch
pipeline = pipeline(task="text-generation", model="mistralai/Mistral-7B-Instruct-v0.1", dtype=torch.bfloat16, device_map="auto")
prompt = """Classify the text into neutral, negative or positive.
Text: This movie is definitely one of my favorite movies of its kind. The interaction between respectable and morally strong characters is an ode to chivalry and the honor code amongst thieves and policemen.
Sentiment:
"""
outputs = pipeline(prompt, max_new_tokens=10)
for output in outputs:
print(f"Result: {output['generated_text']}")
Result: Classify the text into neutral, negative or positive.
Text: This movie is definitely one of my favorite movies of its kind. The interaction between respectable and morally strong characters is an ode to chivalry and the honor code amongst thieves and policemen.
Sentiment:
Positive挑战在于设计能够产生预期结果的提示词，因为语言极其微妙且富有表现力。
本指南介绍了提示工程的最佳实践、技巧和示例，说明如何完成语言和推理任务。
最佳实践
请尽量选择最新的模型以获得最佳性能。请注意，LLM 通常有两种变体：base 和 instruction-tuned（或 chat）。
基础模型在给定初始提示词后补全文本方面表现出色，但在遵循指令方面表现稍逊。指令微调模型是基础模型在指令或对话数据上专门训练得到的版本。这使得指令微调模型更适合用于提示词工程。
现代 LLM 通常是仅解码器（decoder-only）模型，但也有一些编码器-解码器（encoder-decoder）LLM，例如 Flan-T5 或 BART，可用于提示（prompting）。请使用 AutoModelForSeq2SeqLM 类直接加载这些模型（而不是使用 Pipeline），并从模型本身生成输出。
从一个简短而简单的提示开始，并不断迭代以取得更好的效果。
将指令放在提示的开头或结尾。对于较长的提示，模型可能会应用优化以防止注意力机制呈二次方扩展，这会使模型更侧重于提示的开头和结尾部分。
将指令与目标文本明确分开。
请具体且描述性地说明任务及期望的输出，例如格式、长度、风格和语言。避免使用模糊的描述和指令。
指令应侧重于“该做什么”，而非“不该做什么”。
通过编写第一个词甚至第一句话，引导模型生成正确的输出。
尝试使用 few-shot 和 chain-of-thought 等其他技术来提升效果。
使用不同的模型测试你的提示，以评估其鲁棒性。
对提示词性能进行版本管理和跟踪。
技术
仅凭精心设计的提示（也称为零样本提示）可能不足以获得理想的结果。您可能需要尝试几种提示技巧，才能实现最佳性能。
本节介绍几种提示技术。
少样本提示
少样本提示通过在输入中提供模型应生成的具体示例，来提高准确性和性能。明确的示例能让模型更好地理解任务以及你所期望的输出格式。可以尝试使用不同数量的示例（2、4、8 等），观察其对性能的影响。下面的示例为模型提供了 1 个（1-shot）输出格式（MM/DD/YYYY 格式的日期）的示例，作为其应返回的格式参考。
from transformers import pipeline
import torch
pipeline = pipeline(model="mistralai/Mistral-7B-Instruct-v0.1", dtype=torch.bfloat16, device_map="auto")
prompt = """Text: The first human went into space and orbited the Earth on April 12, 1961.
Date: 04/12/1961
Text: The first-ever televised presidential debate in the United States took place on September 28, 1960, between presidential candidates John F. Kennedy and Richard Nixon.
Date:"""
outputs = pipeline(prompt, max_new_tokens=12, do_sample=True, top_k=10)
for output in outputs:
print(f"Result: {output['generated_text']}")
# Result: Text: The first human went into space and orbited the Earth on April 12, 1961.
# Date: 04/12/1961
# Text: The first-ever televised presidential debate in the United States took place on September 28, 1960, between presidential candidates John F. Kennedy and Richard Nixon.
# Date: 09/28/1960少样本提示的缺点在于，你需要构建更长的提示，从而增加计算量和延迟。此外，提示的长度也存在限制。最后，模型可能会从你的示例中学到非预期的模式，并且在复杂的推理任务上表现不佳。
为了提升现代指令微调 LLM 的少样本提示效果，请使用该模型特定的 chat template。这些模型是在包含“用户”与“助手”之间多轮对话的数据集上训练的。将你的提示结构与此对齐，有助于提升性能。
将提示词组织为基于回合的对话，并使用 apply_chat_template 方法进行分词和格式化。
from transformers import pipeline
import torch
pipeline = pipeline(model="mistralai/Mistral-7B-Instruct-v0.1", dtype=torch.bfloat16, device_map="auto")
messages = [
{"role": "user", "content": "Text: The first human went into space and orbited the Earth on April 12, 1961."},
{"role": "assistant", "content": "Date: 04/12/1961"},
{"role": "user", "content": "Text: The first-ever televised presidential debate in the United States took place on September 28, 1960, between presidential candidates John F. Kennedy and Richard Nixon."}
]
prompt = pipeline.tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
outputs = pipeline(prompt, max_new_tokens=12, do_sample=True, top_k=10)
for output in outputs:
print(f"Result: {output['generated_text']}")虽然基础的少样本提示方法将示例嵌入在单个文本字符串中，但聊天模板格式提供了以下优势。
- 由于模型能够更好地识别用户输入和助手输出的预期角色及模式，因此它可能具备更优的理解能力。
- 由于模型在训练期间其结构类似于输入，因此它可能更一致地输出所需的输出格式。
请务必查阅特定指令微调模型的文档，以了解其聊天模板的格式，从而相应地构建您的少样本提示。
思维链
思维链（CoT）通过提供一系列提示，帮助模型更充分地“思考”某个主题，从而生成更具连贯性和逻辑性的输出。
下面的示例为模型提供了多个提示，以引导其完成中间推理步骤。
from transformers import pipeline
import torch
pipeline = pipeline(model="mistralai/Mistral-7B-Instruct-v0.1", dtype=torch.bfloat16, device_map="auto")
prompt = """Let's go through this step-by-step:
1. You start with 15 muffins.
2. You eat 2 muffins, leaving you with 13 muffins.
3. You give 5 muffins to your neighbor, leaving you with 8 muffins.
4. Your partner buys 6 more muffins, bringing the total number of muffins to 14.
5. Your partner eats 2 muffins, leaving you with 12 muffins.
If you eat 6 muffins, how many are left?"""
outputs = pipeline(prompt, max_new_tokens=20, do_sample=True, top_k=10)
for output in outputs:
print(f"Result: {output['generated_text']}")
Result: Let's go through this step-by-step:
1. You start with 15 muffins.
2. You eat 2 muffins, leaving you with 13 muffins.
3. You give 5 muffins to your neighbor, leaving you with 8 muffins.
4. Your partner buys 6 more muffins, bringing the total number of muffins to 14.
5. Your partner eats 2 muffins, leaving you with 12 muffins.
If you eat 6 muffins, how many are left?
Answer: 6与少样本提示类似，CoT 的缺点在于，需要投入更多精力来设计一系列提示，以帮助模型对复杂任务进行推理，并且提示长度的增加会提高延迟。
微调
虽然提示工程是与大语言模型（LLM）交互的强大方式，但在某些场景下，使用微调后的模型，甚至对模型进行微调，效果会更好。
以下是一些微调模型适用的示例场景。
- 您的领域与 LLM 预训练所用的数据存在极大差异，且大量提示词尝试未能达到您期望的效果。
- 您的模型需要在低资源语言中表现良好。
- 您的模型需要在具有严格监管要求的敏感数据上进行训练。
- 由于成本、隐私、基础设施或其他限制，您正在使用一个小模型。
在所有这些场景中，请确保你拥有足够大的领域特定数据集来训练模型，并且有充足的时间和资源，微调的成本也是值得的。否则，你可能更适合尝试优化你的提示词。
示例
以下示例展示了如何提示 LLM 执行不同的任务。
命名实体识别
翻译
摘要
问答
from transformers import pipeline
import torch
pipeline = pipeline(model="mistralai/Mistral-7B-Instruct-v0.1", dtype=torch.bfloat16, device_map="auto")
prompt = """Return a list of named entities in the text.
Text: The company was founded in 2016 by French entrepreneurs Clément Delangue, Julien Chaumond, and Thomas Wolf in New York City, originally as a company that developed a chatbot app targeted at teenagers.
Named entities:
"""
outputs = pipeline(prompt, max_new_tokens=50, return_full_text=False)
for output in outputs:
print(f"Result: {output['generated_text']}")
Result: [Clément Delangue, Julien Chaumond, Thomas Wolf, company, New York City, chatbot app, teenagers]from transformers import pipeline
import torch
pipeline = pipeline(model="mistralai/Mistral-7B-Instruct-v0.1", dtype=torch.bfloat16, device_map="auto")
prompt = """Translate the English text to French.
Text: Sometimes, I've believed as many as six impossible things before breakfast.
Translation:
"""
outputs = pipeline(prompt, max_new_tokens=20, do_sample=True, top_k=10, return_full_text=False)
for output in outputs:
print(f"Result: {output['generated_text']}")
Result: À l'occasion, j'ai croyu plus de six choses impossiblesfrom transformers import pipeline
import torch
pipeline = pipeline(model="mistralai/Mistral-7B-Instruct-v0.1", dtype=torch.bfloat16, device_map="auto")
prompt = """Permaculture is a design process mimicking the diversity, functionality and resilience of natural ecosystems. The principles and practices are drawn from traditional ecological knowledge of indigenous cultures combined with modern scientific understanding and technological innovations. Permaculture design provides a framework helping individuals and communities develop innovative, creative and effective strategies for meeting basic needs while preparing for and mitigating the projected impacts of climate change.
Write a summary of the above text.
Summary:
"""
outputs = pipeline(prompt, max_new_tokens=30, do_sample=True, top_k=10, return_full_text=False)
for output in outputs:
print(f"Result: {output['generated_text']}")
Result: Permaculture is the design process that involves mimicking natural ecosystems to provide sustainable solutions to basic needs. It is a holistic approach that combfrom transformers import pipeline
import torch
pipeline = pipeline(model="mistralai/Mistral-7B-Instruct-v0.1", dtype=torch.bfloat16, device_map="auto")
prompt = """Answer the question using the context below.
Context: Gazpacho is a cold soup and drink made of raw, blended vegetables. Most gazpacho includes stale bread, tomato, cucumbers, onion, bell peppers, garlic, olive oil, wine vinegar, water, and salt. Northern recipes often include cumin and/or pimentón (smoked sweet paprika). Traditionally, gazpacho was made by pounding the vegetables in a mortar with a pestle; this more laborious method is still sometimes used as it helps keep the gazpacho cool and avoids the foam and silky consistency of smoothie versions made in blenders or food processors.
Question: What modern tool is used to make gazpacho?
Answer:
"""
outputs = pipeline(prompt, max_new_tokens=10, do_sample=True, top_k=10, return_full_text=False)
for output in outputs:
print(f"Result: {output['generated_text']}")
Result: A blender or food processor is the modern tool