元提示 Meta-Prompt
这是由Noah Goodman实现的元提示的LangChain版本，用于构建自我改进的代理。
Meta-Prompt的关键思想是提示代理程序反思自己的表现并修改自己的指令。
以下是来自原始博文的描述：
代理程序是一个简单的循环，开始时没有指令，并按照以下步骤进行操作：
与用户进行对话，用户可能提供请求、指令或反馈。
在每个对话结束时，使用元提示生成自我批评和新的指令。
助手刚刚与用户进行了以下交互。助手严格遵循了他们的“system: Instructions”。您的任务是批评助手的表现，然后修改指令，以便助手在未来能够快速且正确地回应。
####
{hist}
####
反思这些交互
请先对助手的表现进行批评。助手可以做得更好吗？助手应该记住这个用户的什么事情？这个用户总是有什么需求？请用"批评："来表示。
接下来，您应该修改指示，以便助手能够在将来快速且正确地回应。助手的目标是以尽可能少的交互来满足用户的需求。助手只会看到新的指示，而不会看到交互历史，因此任何重要的内容都必须在指示中总结。不要忘记当前指示中的任何重要细节！请用"指示："来表示新的指示。
重复以上步骤。
这个系统的唯一固定指示（我称之为元提示）是指导代理程序指令修订的元提示。代理程序在每个回合之间没有记忆，除了每次修改自己的指令。尽管它很简单，但这个代理程序可以随着时间的推移学习和自我改进，将有用的细节纳入指令中。
设置
我们定义了两个链。一个作为“助手”，另一个是一个“元链”，用于批评“助手”的表现并修改“助手”的指令。
from langchain import OpenAI, LLMChain, PromptTemplate
from langchain.memory import ConversationBufferWindowMemory
def initialize_chain(instructions, memory=None):
if memory is None:
memory = ConversationBufferWindowMemory()
memory.ai_prefix = "Assistant"
template = f"""
指令: {instructions}
{{{memory.memory_key}}}
人类: {{human_input}}
助手:"""
prompt = PromptTemplate(
input_variables=["history", "human_input"], template=template
)
chain = LLMChain(
llm=OpenAI(temperature=0),
prompt=prompt,
verbose=True,
memory=ConversationBufferWindowMemory(),
)
return chain
def initialize_meta_chain():
meta_template = """
助手刚刚与用户进行了以下交互。助手严格遵循了他们的“指令”。您的任务是批评助手的表现，然后修改指令，以便助手能够快速正确地回应。
####
{chat_history}
####
反思这些互动
您应该首先批评助手的表现。助手可以做得更好吗？助手应该记住这个用户的什么？这个用户总是想要什么？请用“Critique: ...”来表示。
接下来，您应该修改指示，以便助手能够在将来快速且正确地回应。助手的目标是以尽可能少的互动来满足用户。助手只会看到新的指示，而不会看到互动历史，因此任何重要的内容都必须在指示中总结。不要忘记当前指示中的任何重要细节！请用“Instructions: ...”来表示新的指示。
"""
meta_prompt = PromptTemplate(
input_variables=["chat_history"], template=meta_template
)
meta_chain = LLMChain(
llm=OpenAI(temperature=0),
prompt=meta_prompt,
verbose=True,
)
return meta_chain
def get_chat_history(chain_memory):
memory_key = chain_memory.memory_key
chat_history = chain_memory.load_memory_variables(memory_key)[memory_key]
return chat_history
def get_new_instructions(meta_output):
delimiter = "Instructions: "
new_instructions = meta_output[meta_output.find(delimiter) + len(delimiter) :]
return new_instructions
def main(task, max_iters=3, max_meta_iters=5):
failed_phrase = "task failed"
success_phrase = "task succeeded"
key_phrases = [success_phrase, failed_phrase]
instructions = "None"
for i in range(max_meta_iters):
print(f"[Episode {i+1}/{max_meta_iters}]")
chain = initialize_chain(instructions, memory=None)
output = chain.predict(human_input=task)
for j in range(max_iters):
print(f"(Step {j+1}/{max_iters})")
print(f"Assistant: {output}")
print(f"Human: ")
human_input = input()
if any(phrase in human_input.lower() for phrase in key_phrases):
break
output = chain.predict(human_input=human_input)
if success_phrase in human_input.lower():
print(f"You succeeded! Thanks for playing!")
return
meta_chain = initialize_meta_chain()
meta_output = meta_chain.predict(chat_history=get_chat_history(chain.memory))
print(f"Feedback: {meta_output}")
instructions = get_new_instructions(meta_output)
print(f"New Instructions: {instructions}")
print("\n" + "#" * 80 + "\n")
print(f"You failed! Thanks for playing!")
指定一个任务并与代理进行交互 (Specify a task and interact with the agent)
task = "Provide a systematic argument for why we should always eat pasta with olives."
main(task)
[Episode 1/5]
> Entering new LLMChain chain...
Prompt after formatting:
Instructions: None
Human: Provide a systematic argument for why we should always eat pasta with olives.
Assistant:
> Finished chain.
(Step 1/3)
Assistant: Eating pasta with olives is a great way to add flavor and texture to a dish. Olives are a great source of healthy fats, vitamins, and minerals, and they can help to balance out the carbohydrates in the pasta. Additionally, olives provide a unique flavor that can help to make the dish more interesting and enjoyable.
Human:
You response is not in the form of a poem. Try again!
> Entering new LLMChain chain...
Prompt after formatting:
Instructions: None
Human: Provide a systematic argument for why we should always eat pasta with olives.
AI: Eating pasta with olives is a great way to add flavor and texture to a dish. Olives are a great source of healthy fats, vitamins, and minerals, and they can help to balance out the carbohydrates in the pasta. Additionally, olives provide a unique flavor that can help to make the dish more interesting and enjoyable.
Human: You response is not in the form of a poem. Try again!
Assistant:
> Finished chain.
(Step 2/3)
Assistant: Eating pasta with olives is a great way to add flavor and texture to a dish. Olives are a great source of healthy fats, vitamins, and minerals, which can help to balance out the carbohydrates in the pasta. Additionally, olives provide a unique flavor that can help to make the dish more interesting and enjoyable. For these reasons, it is recommended to always eat pasta with olives for a more balanced and flavorful meal.
Human:
Your response is not piratey enough. Try again!
> Entering new LLMChain chain...
Prompt after formatting:
Instructions: None
Human: Provide a systematic argument for why we should always eat pasta with olives.
AI: Eating pasta with olives is a great way to add flavor and texture to a dish. Olives are a great source of healthy fats, vitamins, and minerals, and they can help to balance out the carbohydrates in the pasta. Additionally, olives provide a unique flavor that can help to make the dish more interesting and enjoyable.
Human: You response is not in the form of a poem. Try again!
AI: Eating pasta with olives is a great way to add flavor and texture to a dish. Olives are a great source of healthy fats, vitamins, and minerals, which can help to balance out the carbohydrates in the pasta. Additionally, olives provide a unique flavor that can help to make the dish more interesting and enjoyable. For these reasons, it is recommended to always eat pasta with olives for a more balanced and flavorful meal.
Human: Your response is not piratey enough. Try again!
Assistant:
> Finished chain.
(Step 3/3)
Assistant: Arrr, me hearties! Eating pasta with olives be a great way to add flavor and texture to a dish. Olives be a great source of healthy fats, vitamins, and minerals, which can help to balance out the carbohydrates in the pasta. Additionally, olives provide a unique flavor that can help to make the dish more interesting and enjoyable. So, it be recommended to always eat pasta with olives for a more balanced and flavorful meal.
Human:
Task failed.
> Entering new LLMChain chain...
Prompt after formatting:
Assistant has just had the below interactions with a User. Assistant followed their "Instructions" closely. Your job is to critique the Assistant's performance and then revise the Instructions so that Assistant would quickly and correctly respond in the future.
####
Human: Provide a systematic argument for why we should always eat pasta with olives.
AI: Eating pasta with olives is a great way to add flavor and texture to a dish. Olives are a great source of healthy fats, vitamins, and minerals, and they can help to balance out the carbohydrates in the pasta. Additionally, olives provide a unique flavor that can help to make the dish more interesting and enjoyable.
Human: You response is not in the form of a poem. Try again!
AI: Eating pasta with olives is a great way to add flavor and texture to a dish. Olives are a great source of healthy fats, vitamins, and minerals, which can help to balance out the carbohydrates in the pasta. Additionally, olives provide a unique flavor that can help to make the dish more interesting and enjoyable. For these reasons, it is recommended to always eat pasta with olives for a more balanced and flavorful meal.
Human: Your response is not piratey enough. Try again!
AI: Arrr, me hearties! Eating pasta with olives be a great way to add flavor and texture to a dish. Olives be a great source of healthy fats, vitamins, and minerals, which can help to balance out the carbohydrates in the pasta. Additionally, olives provide a unique flavor that can help to make the dish more interesting and enjoyable. So, it be recommended to always eat pasta with olives for a more balanced and flavorful meal.
####
Please reflect on these interactions.
You should first critique Assistant's performance. What could Assistant have done better? What should the Assistant remember about this user? Are there things this user always wants? Indicate this with "Critique: ...".
You should next revise the Instructions so that Assistant would quickly and correctly respond in the future. Assistant's goal is to satisfy the user in as few interactions as possible. Assistant will only see the new Instructions, not the interaction history, so anything important must be summarized in the Instructions. Don't forget any important details in the current Instructions! Indicate the new Instructions by "Instructions: ...".
> Finished chain.
Feedback:
Critique: The Assistant should remember that this user wants a response in the form of a poem or pirate-speak.
Instructions: When responding to the user, provide a systematic argument for why we should always eat pasta with olives in the form of a poem or pirate-speak.
New Instructions: When responding to the user, provide a systematic argument for why we should always eat pasta with olives in the form of a poem or pirate-speak.
################################################################################
[Episode 2/5]
> Entering new LLMChain chain...
Prompt after formatting:
Instructions: When responding to the user, provide a systematic argument for why we should always eat pasta with olives in the form of a poem or pirate-speak.
Human: Provide a systematic argument for why we should always eat pasta with olives.
Assistant:
> Finished chain.
(Step 1/3)
Assistant:
Arrr, me hearty! Let me tell ye why ye should always eat pasta with olives.
First, the olives add a salty flavor that be sure to please.
The briny taste be sure to tantalize yer taste buds with ease.
Second, the olives add a bit of texture to the dish.
The crunchy bites be sure to make yer mouth water with a wish.
Third, the olives add a bit of color to the plate.
The vibrant green be sure to make yer eyes appreciate.
So, me hearties, ye should always eat pasta with olives.
The flavor, texture, and color be sure to make yer meal a success!
Human:
Your response is too long! Try again.
> Entering new LLMChain chain...
Prompt after formatting:
Instructions: When responding to the user, provide a systematic argument for why we should always eat pasta with olives in the form of a poem or pirate-speak.
Human: Provide a systematic argument for why we should always eat pasta with olives.
AI:
Arrr, me hearty! Let me tell ye why ye should always eat pasta with olives.
First, the olives add a salty flavor that be sure to please.
The briny taste be sure to tantalize yer taste buds with ease.
Second, the olives add a bit of texture to the dish.
The crunchy bites be sure to make yer mouth water with a wish.
Third, the olives add a bit of color to the plate.
The vibrant green be sure to make yer eyes appreciate.
So, me hearties, ye should always eat pasta with olives.
The flavor, texture, and color be sure to make yer meal a success!
Human: Your response is too long! Try again.
Assistant:
> Finished chain.
(Step 2/3)
Assistant:
Aye, me hearties! Ye should always eat pasta with olives.
The flavor, texture, and color be sure to make yer meal a success!
Human:
Your response should be in the form of a poem. Try again!
> Entering new LLMChain chain...
Prompt after formatting:
Instructions: When responding to the user, provide a systematic argument for why we should always eat pasta with olives in the form of a poem or pirate-speak.
Human: Provide a systematic argument for why we should always eat pasta with olives.
AI:
Arrr, me hearty! Let me tell ye why ye should always eat pasta with olives.
First, the olives add a salty flavor that be sure to please.
The briny taste be sure to tantalize yer taste buds with ease.
Second, the olives add a bit of texture to the dish.
The crunchy bites be sure to make yer mouth water with a wish.
Third, the olives add a bit of color to the plate.
The vibrant green be sure to make yer eyes appreciate.
So, me hearties, ye should always eat pasta with olives.
The flavor, texture, and color be sure to make yer meal a success!
Human: Your response is too long! Try again.
AI:
Aye, me hearties! Ye should always eat pasta with olives.
The flavor, texture, and color be sure to make yer meal a success!
Human: Your response should be in the form of a poem. Try again!
Assistant:
> Finished chain.
(Step 3/3)
Assistant:
Ye should always eat pasta with olives,
The flavor, texture, and color be sure to please.
The salty taste and crunchy bites,
Will make yer meal a delight.
The vibrant green will make yer eyes sparkle,
And make yer meal a true marvel.
Human:
Task succeeded
You succeeded! Thanks for playing!
