Skip to main content

如何使用 SmartLLMChain

SmartLLMChain是一种自我批判链，可以帮助您回答特别复杂的问题。它不是进行单次LLM传递，而是执行以下3个步骤：

  1. 构思：将用户提示通过LLM传递n次，以获得n个输出提案（称为“想法”），其中n是您可以设置的参数
  2. 批判：LLM批判所有想法，找出可能的缺陷并选择最佳想法
  3. 解决：LLM试图改进最佳想法（在批判步骤中选择）并输出它。然后，这就是最终输出。

SmartLLMChain基于SmartGPT工作流程，该工作流程在https://youtu.be/wVzuvf9D9BU中提出。

请注意，SmartLLMChain：

  • 使用更多的LLM传递次数（即n+2次，而不仅仅是1次）
  • 仅在底层LLM具有反思能力时才有效，而较小的模型通常没有这种能力
  • 仅适用于返回精确1个输出而不是多个输出的底层模型

本笔记本演示了如何使用SmartLLMChain。

所有步骤使用相同的LLM​
import os

os.environ["OPENAI_API_KEY"] = "..."

from langchain.prompts import PromptTemplate
from langchain.chat_models import ChatOpenAI
from langchain_experimental.smart_llm import SmartLLMChain

API参考：​

作为示例问题，我们将使用"I have a 12 liter jug and a 6 liter jug. I want to measure 6 liters. How do I do it?"。这是原始SmartGPT视频中的一个示例（https://youtu.be/wVzuvf9D9BU?t=384）。虽然这似乎是一个非常简单的问题，但LLMs在涉及数字和物理推理的问题上往往很难处理。

正如我们将看到的，所有3个初始想法都是完全错误的 - 即使我们使用的是GPT4！只有在使用自我反思时，我们才能得到正确的答案。

hard_question = "I have a 12 liter jug and a 6 liter jug. I want to measure 6 liters. How do I do it?"

因此，我们首先创建一个LLM和提示模板。

prompt = PromptTemplate.from_template(hard_question)
llm = ChatOpenAI(temperature=0, model_name="gpt-4")

现在我们可以创建一个SmartLLMChain。

chain = SmartLLMChain(llm=llm, prompt=prompt, n_ideas=3, verbose=True)

现在我们可以将SmartLLM用作LLM的替代品。例如：

chain.run({})
    
    
    > Entering new SmartLLMChain chain...
    Prompt after formatting:
    I have a 12 liter jug and a 6 liter jug. I want to measure 6 liters. How do I do it?
    Idea 1:
    1. Fill the 6-liter jug completely.
    2. Pour the water from the 6-liter jug into the 12-liter jug.
    3. Fill the 6-liter jug again.
    4. Carefully pour the water from the 6-liter jug into the 12-liter jug until the 12-liter jug is full.
    5. The amount of water left in the 6-liter jug will be exactly 6 liters.
    Idea 2:
    1. Fill the 6-liter jug completely.
    2. Pour the water from the 6-liter jug into the 12-liter jug.
    3. Fill the 6-liter jug again.
    4. Carefully pour the water from the 6-liter jug into the 12-liter jug until the 12-liter jug is full.
    5. Since the 12-liter jug is now full, there will be 2 liters of water left in the 6-liter jug.
    6. Empty the 12-liter jug.
    7. Pour the 2 liters of water from the 6-liter jug into the 12-liter jug.
    8. Fill the 6-liter jug completely again.
    9. Pour the water from the 6-liter jug into the 12-liter jug, which already has 2 liters in it.
    10. Now, the 12-liter jug will have exactly 6 liters of water (2 liters from before + 4 liters from the 6-liter jug).
    Idea 3:
    1. Fill the 6-liter jug completely.
    2. Pour the water from the 6-liter jug into the 12-liter jug.
    3. Fill the 6-liter jug again.
    4. Carefully pour the water from the 6-liter jug into the 12-liter jug until the 12-liter jug is full.
    5. The amount of water left in the 6-liter jug will be exactly 6 liters.
    Critique:
    Idea 1:
    1. Fill the 6-liter jug completely. (No flaw)
    2. Pour the water from the 6-liter jug into the 12-liter jug. (No flaw)
    3. Fill the 6-liter jug again. (No flaw)
    4. Carefully pour the water from the 6-liter jug into the 12-liter jug until the 12-liter jug is full. (Flaw: The 12-liter jug will never be full in this step, as it can hold 12 liters and we are only pouring 6 liters into it.)
    5. The amount of water left in the 6-liter jug will be exactly 6 liters. (Flaw: This statement is incorrect, as there will be no water left in the 6-liter jug after pouring it into the 12-liter jug.)
    
    Idea 2:
    1. Fill the 6-liter jug completely. (No flaw)
    2. Pour the water from the 6-liter jug into the 12-liter jug. (No flaw)
    3. Fill the 6-liter jug again. (No flaw)
    4. Carefully pour the water from the 6-liter jug into the 12-liter jug until the 12-liter jug is full. (Flaw: The 12-liter jug will never be full in this step, as it can hold 12 liters and we are only pouring 6 liters into it.)
    5. Since the 12-liter jug is now full, there will be 2 liters of water left in the 6-liter jug. (Flaw: This statement is incorrect, as the 12-liter jug will not be full and there will be no water left in the 6-liter jug.)
    6. Empty the 12-liter jug. (No flaw)
    7. Pour the 2 liters of water from the 6-liter jug into the 12-liter jug. (Flaw: This step is based on the incorrect assumption that there are 2 liters of water left in the 6-liter jug.)
    8. Fill the 6-liter jug completely again. (No flaw)
    9. Pour the water from the 6-liter jug into the 12-liter jug, which already has 2 liters in it. (Flaw: This step is based on the incorrect assumption that there are 2 liters of water in the 12-liter jug.)
    10. Now, the 12-liter jug will have exactly 6 liters of water (2 liters from before + 4 liters from the 6-liter jug). (Flaw: This conclusion is based on the incorrect assumptions made in the previous steps.)
    
    Idea 3:
    1. Fill the 6-liter jug completely. (No flaw)
    2. Pour the water from the 6-liter jug into the 12-liter jug. (No flaw)
    3. Fill the 6-liter jug again. (No flaw)
    4. Carefully pour the water from the 6-liter jug into the 12-liter jug until the 12-liter jug is full. (Flaw: The 12-liter jug will never be full in this step, as it can hold 12 liters and we are only pouring 6 liters into it.)
    5. The amount of water left in the 6-liter jug will be exactly 6 liters. (Flaw: This statement is incorrect, as there will be no water left in the 6-liter jug after pouring it into the 12-liter jug.)
    Resolution:
    1. Fill the 12-liter jug completely.
    2. Pour the water from the 12-liter jug into the 6-liter jug until the 6-liter jug is full.
    3. The amount of water left in the 12-liter jug will be exactly 6 liters.
    
    > Finished chain.





    '1. Fill the 12-liter jug completely.\n2. Pour the water from the 12-liter jug into the 6-liter jug until the 6-liter jug is full.\n3. The amount of water left in the 12-liter jug will be exactly 6 liters.'
不同步骤使用不同的LLM​

您还可以通过传递ideation_llm、critique_llm和resolve_llm来为不同的步骤使用不同的LLM。您可能希望这样做，以便在构思时使用更具创造性（即高温度）的模型，并在批判和解决时使用更严格（即低温度）的模型。

chain = SmartLLMChain(
    ideation_llm=ChatOpenAI(temperature=0.9, model_name="gpt-4"),
    llm=ChatOpenAI(
        temperature=0, model_name="gpt-4"
    ),  # 如果没有给出特定的LLMs，将用于批判和解决
    prompt=prompt,
    n_ideas=3,
    verbose=True,
)