Skip to main content

BagelDB

BagelDB（Open Vector Database for AI）类似于AI数据的GitHub。 它是一个协作平台，用户可以创建、共享和管理向量数据集。它可以支持独立开发者的私有项目、企业的内部协作以及数据DAO的公共贡献。

安装和设置​

pip install betabageldb

从文本创建VectorStore​

from langchain.vectorstores import Bagel

texts = ["hello bagel", "hello langchain", "I love salad", "my car", "a dog"]
# 创建集群并添加文本
cluster = Bagel.from_texts(cluster_name="testing", texts=texts)
# 相似性搜索
cluster.similarity_search("bagel", k=3)
    [Document(page_content='hello bagel', metadata={}),
     Document(page_content='my car', metadata={}),
     Document(page_content='I love salad', metadata={})]
# 分数是一个距离度量，所以越低越好
cluster.similarity_search_with_score("bagel", k=3)
    [(Document(page_content='hello bagel', metadata={}), 0.27392977476119995),
     (Document(page_content='my car', metadata={}), 1.4783176183700562),
     (Document(page_content='I love salad', metadata={}), 1.5342965126037598)]
# 删除集群
cluster.delete_cluster()

从文档创建VectorStore​

from langchain.document_loaders import TextLoader
from langchain.text_splitter import CharacterTextSplitter

loader = TextLoader("../../../state_of_the_union.txt")
documents = loader.load()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
docs = text_splitter.split_documents(documents)[:10]
# 使用文档创建集群
cluster = Bagel.from_documents(cluster_name="testing_with_docs", documents=docs)
# 相似性搜索
query = "What did the president say about Ketanji Brown Jackson"
docs = cluster.similarity_search(query)
print(docs[0].page_content[:102])
    Madam Speaker, Madam Vice President, our First Lady and Second Gentleman. Members of Congress and the

获取集群中的所有文本/文档​

texts = ["hello bagel", "this is langchain"]
cluster = Bagel.from_texts(cluster_name="testing", texts=texts)
cluster_data = cluster.get()
# 所有键
cluster_data.keys()
    dict_keys(['ids', 'embeddings', 'metadatas', 'documents'])
# 所有键和值
cluster_data
    {'ids': ['578c6d24-3763-11ee-a8ab-b7b7b34f99ba',
      '578c6d25-3763-11ee-a8ab-b7b7b34f99ba',
      'fb2fc7d8-3762-11ee-a8ab-b7b7b34f99ba',
      'fb2fc7d9-3762-11ee-a8ab-b7b7b34f99ba',
      '6b40881a-3762-11ee-a8ab-b7b7b34f99ba',
      '6b40881b-3762-11ee-a8ab-b7b7b34f99ba',
      '581e691e-3762-11ee-a8ab-b7b7b34f99ba',
      '581e691f-3762-11ee-a8ab-b7b7b34f99ba'],
     'embeddings': None,
     'metadatas': [{}, {}, {}, {}, {}, {}, {}, {}],
     'documents': ['hello bagel',
      'this is langchain',
      'hello bagel',
      'this is langchain',
      'hello bagel',
      'this is langchain',
      'hello bagel',
      'this is langchain']}
cluster.delete_cluster()

创建具有元数据的集群并使用元数据进行过滤​

texts = ["hello bagel", "this is langchain"]
metadatas = [{"source": "notion"}, {"source": "google"}]

cluster = Bagel.from_texts(cluster_name="testing", texts=texts, metadatas=metadatas)
cluster.similarity_search_with_score("hello bagel", where={"source": "notion"})
    [(Document(page_content='hello bagel', metadata={'source': 'notion'}), 0.0)]
# 删除集群
cluster.delete_cluster()