OctoAI计算服务 (OctoAI Compute Service)

本示例介绍如何使用LangChain与OctoAI LLM端点进行交互

环境设置 (Environment setup)

要运行我们的示例应用程序，需要执行以下四个简单的步骤：

通过访问https://octoai.cloud/templates/mpt-7b-demo并点击"Clone Template"，将MPT-7B演示模板克隆到您的OctoAI账户中。
1. 如果您想使用不同的LLM模型，您还可以将模型容器化，并自己创建一个自定义的OctoAI端点，方法是按照从Python构建容器和从容器创建自定义端点进行操作。
在下面的代码单元格中粘贴您的端点URL。
从您的OctoAI账户页面获取API令牌。
在下面的代码单元格中粘贴您的API密钥。

import os

os.environ["OCTOAI_API_TOKEN"] = "OCTOAI_API_TOKEN"
os.environ["ENDPOINT_URL"] = "https://mpt-7b-demo-kk0powt97tmb.octoai.cloud/generate"

from langchain.llms.octoai_endpoint import OctoAIEndpoint
from langchain import PromptTemplate, LLMChain

template = """以下是描述一个任务的指令。请编写一个适当完成请求的回答。\n 指令:\n{question}\n 回答: """
prompt = PromptTemplate(template=template, input_variables=["question"])

llm = OctoAIEndpoint(
    model_kwargs={
        "max_new_tokens": 200,
        "temperature": 0.75,
        "top_p": 0.95,
        "repetition_penalty": 1,
        "seed": None,
        "stop": [],
    },
)

question = "Who was leonardo davinci?"

llm_chain = LLMChain(prompt=prompt, llm=llm)

llm_chain.run(question)

    '\nLeonardo da Vinci was an Italian polymath and painter regarded by many as one of the greatest painters of all time. He is best known for his masterpieces including Mona Lisa, The Last Supper, and The Virgin of the Rocks. He was a draftsman, sculptor, architect, and one of the most important figures in the history of science. Da Vinci flew gliders, experimented with water turbines and windmills, and invented the catapult and a joystick-type human-powered aircraft control. He may have pioneered helicopters. As a scholar, he was interested in anatomy, geology, botany, engineering, mathematics, and astronomy.\nOther painters and patrons claimed to be more talented, but Leonardo da Vinci was an incredibly productive artist, sculptor, engineer, anatomist, and scientist.'

OctoAI计算服务 (OctoAI Compute Service)

环境设置 (Environment setup)​

环境设置 (Environment setup)