Skip to main content

Clarifai

Clarifai是一个提供完整AI生命周期的AI平台，包括数据探索、数据标注、模型训练、评估和推理。上传输入后，Clarifai应用程序可以用作向量数据库。

本笔记本展示了如何使用与Clarifai向量数据库相关的功能。

要使用Clarifai，您必须拥有一个帐户和个人访问令牌（PAT）密钥。请在此处获取或创建PAT。

依赖项

# 安装所需的依赖项
pip install clarifai

导入

在这里，我们将设置个人访问令牌。您可以在平台的设置/安全性下找到您的PAT。

# 请登录并从https://clarifai.com/settings/security获取您的API密钥
from getpass import getpass

CLARIFAI_PAT = getpass()

我们想要使用OpenAIEmbeddings，所以我们必须获取OpenAI API密钥。

# 导入所需的模块
from langchain.text_splitter import CharacterTextSplitter
from langchain.document_loaders import TextLoader
from langchain.vectorstores import Clarifai

设置

设置将上传文本数据的用户ID和应用程序ID。注意：在创建该应用程序时，请选择适当的基础工作流程来索引您的文本文档，例如语言理解工作流程。

您首先需要在Clarifai上创建一个帐户，然后创建一个应用程序。

USER_ID = "USERNAME_ID"
APP_ID = "APPLICATION_ID"
NUMBER_OF_DOCS = 4

从文本创建Clarifai向量存储​

Create a Clarifai vectorstore from a list of texts.

texts = [
    "我真的很喜欢和你在一起的时间",
    "我讨厌和我的狗在一起的时间",
    "我想去跑步",
    "昨天我去看电影了",
    "我喜欢和朋友们一起踢足球",
]

metadatas = [{"id": i, "text": text} for i, text in enumerate(texts)]
clarifai_vector_db = Clarifai.from_texts(
    user_id=USER_ID,
    app_id=APP_ID,
    texts=texts,
    pat=CLARIFAI_PAT,
    number_of_docs=NUMBER_OF_DOCS,
    metadatas=metadatas,
)
docs = clarifai_vector_db.similarity_search("我很想见到你")
docs
    [Document(page_content='我真的很喜欢和你在一起的时间', metadata={'text': '我真的很喜欢和你在一起的时间', 'id': 0.0}),
     Document(page_content='昨天我去看电影了', metadata={'text': '昨天我去看电影了', 'id': 3.0}),
     Document(page_content='zab', metadata={'page': '2'}),
     Document(page_content='zab', metadata={'page': '2'})]

从文档中创建Clarifai向量存储库​

从文档列表中创建一个Clarifai向量存储库。本节将上传每个文档及其相应的元数据到Clarifai应用程序。然后可以使用Clarifai应用程序进行语义搜索以找到相关文档。

loader = TextLoader("../../../state_of_the_union.txt")
documents = loader.load()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
docs = text_splitter.split_documents(documents)
docs[:4]
    [Document(page_content='Madam Speaker, Madam Vice President, our First Lady and Second Gentleman. Members of Congress and the Cabinet. Justices of the Supreme Court. My fellow Americans.  \n\nLast year COVID-19 kept us apart. This year we are finally together again. \n\nTonight, we meet as Democrats Republicans and Independents. But most importantly as Americans. \n\nWith a duty to one another to the American people to the Constitution. \n\nAnd with an unwavering resolve that freedom will always triumph over tyranny. \n\nSix days ago, Russia’s Vladimir Putin sought to shake the foundations of the free world thinking he could make it bend to his menacing ways. But he badly miscalculated. \n\nHe thought he could roll into Ukraine and the world would roll over. Instead he met a wall of strength he never imagined. \n\nHe met the Ukrainian people. \n\nFrom President Zelenskyy to every Ukrainian, their fearlessness, their courage, their determination, inspires the world.', metadata={'source': '../../../state_of_the_union.txt'}),
     Document(page_content='Groups of citizens blocking tanks with their bodies. Everyone from students to retirees teachers turned soldiers defending their homeland. \n\nIn this struggle as President Zelenskyy said in his speech to the European Parliament “Light will win over darkness.” The Ukrainian Ambassador to the United States is here tonight. \n\nLet each of us here tonight in this Chamber send an unmistakable signal to Ukraine and to the world. \n\nPlease rise if you are able and show that, Yes, we the United States of America stand with the Ukrainian people. \n\nThroughout our history we’ve learned this lesson when dictators do not pay a price for their aggression they cause more chaos.   \n\nThey keep moving.   \n\nAnd the costs and the threats to America and the world keep rising.   \n\nThat’s why the NATO Alliance was created to secure peace and stability in Europe after World War 2. \n\nThe United States is a member along with 29 other nations. \n\nIt matters. American diplomacy matters. American resolve matters.', metadata={'source': '../../../state_of_the_union.txt'}),
     Document(page_content='Putin’s latest attack on Ukraine was premeditated and unprovoked. \n\nHe rejected repeated efforts at diplomacy. \n\nHe thought the West and NATO wouldn’t respond. And he thought he could divide us at home. Putin was wrong. We were ready.  Here is what we did.   \n\nWe prepared extensively and carefully. \n\nWe spent months building a coalition of other freedom-loving nations from Europe and the Americas to Asia and Africa to confront Putin. \n\nI spent countless hours unifying our European allies. We shared with the world in advance what we knew Putin was planning and precisely how he would try to falsely justify his aggression.  \n\nWe countered Russia’s lies with truth.   \n\nAnd now that he has acted the free world is holding him accountable. \n\nAlong with twenty-seven members of the European Union including France, Germany, Italy, as well as countries like the United Kingdom, Canada, Japan, Korea, Australia, New Zealand, and many others, even Switzerland.', metadata={'source': '../../../state_of_the_union.txt'}),
     Document(page_content='We are inflicting pain on Russia and supporting the people of Ukraine. Putin is now isolated from the world more than ever. \n\nTogether with our allies –we are right now enforcing powerful economic sanctions. \n\nWe are cutt