🚅 LiteLLM

LiteLLM 是一个简化了调用 Anthropic、Azure、Huggingface、Replicate 等服务的库。

本文档介绍了如何开始使用 Langchain + LiteLLM I/O 库。

from langchain.chat_models import ChatLiteLLM  
from langchain.prompts.chat import (  
    ChatPromptTemplate,  
    SystemMessagePromptTemplate,  
    AIMessagePromptTemplate,  
    HumanMessagePromptTemplate,  
)  
from langchain.schema import AIMessage, HumanMessage, SystemMessage  

API 参考：

• ChatLiteLLM 来自 langchain.chat_models
• ChatPromptTemplate 来自 langchain.prompts.chat
• SystemMessagePromptTemplate 来自 langchain.prompts.chat
• AIMessagePromptTemplate 来自 langchain.prompts.chat
• HumanMessagePromptTemplate 来自 langchain.prompts.chat
• AIMessage 来自 langchain.schema
• HumanMessage 来自 langchain.schema
• SystemMessage 来自 langchain.schema

chat = ChatLiteLLM(model="gpt-3.5-turbo")  

messages = [  
    HumanMessage(  
        content="Translate this sentence from English to French. I love programming."  
    )  
]  
chat(messages)  

AIMessage(content=" J'aime la programmation.", additional_kwargs={}, example=False)  

ChatLiteLLM 还支持异步和流式功能：

from langchain.callbacks.manager import CallbackManager  
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler  

API 参考：

• CallbackManager 来自 langchain.callbacks.manager
• StreamingStdOutCallbackHandler 来自 langchain.callbacks.streaming_stdout

await chat.agenerate([messages])  

LLMResult(generations=[[ChatGeneration(text=" J'aime programmer.", generation_info=None, message=AIMessage(content=" J'aime programmer.", additional_kwargs={}, example=False))]], llm_output={}, run=[RunInfo(run_id=UUID('8cc8fb68-1c35-439c-96a0-695036a93652'))])  

chat = ChatLiteLLM(  
    streaming=True,  
    verbose=True,  
    callback_manager=CallbackManager([StreamingStdOutCallbackHandler()]),  
)  
chat(messages)  

J'aime la programmation.  

AIMessage(content=" J'aime la programmation.", additional_kwargs={}, example=False)