Skip to main content

流式传输最终代理输出

如果您只想要代理的最终输出进行流式传输，可以使用回调函数FinalStreamingStdOutCallbackHandler。为此，底层的LLM也必须支持流式传输。

from langchain.agents import load_tools
from langchain.agents import initialize_agent
from langchain.agents import AgentType
from langchain.callbacks.streaming_stdout_final_only import (
    FinalStreamingStdOutCallbackHandler,
)
from langchain.llms import OpenAI

API 参考：​

让我们使用streaming = True创建底层的LLM，并传递一个新的FinalStreamingStdOutCallbackHandler实例。

llm = OpenAI(
    streaming=True, callbacks=[FinalStreamingStdOutCallbackHandler()], temperature=0
)

tools = load_tools(["wikipedia", "llm-math"], llm=llm)
agent = initialize_agent(
    tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=False
)
agent.run(
    "It's 2023 now. How many years ago did Konrad Adenauer become Chancellor of Germany."
)

Konrad Adenauer became Chancellor of Germany in 1949, 74 years ago in 2023.

默认情况下，我们假设令牌序列"Final", "Answer", ":"表示代理已经找到了答案。然而，我们也可以传递自定义序列作为答案前缀。

llm = OpenAI(  
    streaming=True,  
    callbacks=[  
        FinalStreamingStdOutCallbackHandler(answer_prefix_tokens=["The", "answer", ":"])  
    ],  
    temperature=0,  
)

为了方便起见，回调函数在与answer_prefix_tokens进行比较时会自动去除空格和换行符。例如，如果answer_prefix_tokens = ["The", " answer", ":"]，那么["\nThe", " answer", ":"]和["The", " answer", ":"]都会被识别为答案前缀。

如果你不知道答案前缀的分词版本，可以使用以下代码确定：

from langchain.callbacks.base import BaseCallbackHandler  

class MyCallbackHandler(BaseCallbackHandler):  
    def on_llm_new_token(self, token, **kwargs) -> None:  
        # 每个令牌都打印在新的一行  
        print(f"#{token}#")  

llm = OpenAI(streaming=True, callbacks=[MyCallbackHandler()])  
tools = load_tools(["wikipedia", "llm-math"], llm=llm)  
agent = initialize_agent(  
    tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=False  
)  
agent.run(  
    "It's 2023 now. How many years ago did Konrad Adenauer become Chancellor of Germany."  
)

API 参考：​

同时流式传输答案前缀​

当参数stream_prefix = True被设置时，答案前缀本身也会被流式传输。当答案前缀本身是答案的一部分时，这将非常有用。例如，当你的答案是一个类似于以下的JSON时：

{ "action": "Final answer", "action_input": "Konrad Adenauer became Chancellor 74 years ago." }

你不仅希望流式传输action_input，还希望整个JSON都被流式传输。