Skip to main content

DuckDB

DuckDB 是一个内置的 SQL OLAP 数据库管理系统。

使用每行一个文档的方式加载 DuckDB 查询。

#!pip install duckdb
from langchain.document_loaders import DuckDBLoader
Team,Payroll
Nationals,81.34
Reds,82.20
    Writing example.csv
loader = DuckDBLoader("SELECT * FROM read_csv_auto('example.csv')")

data = loader.load()
print(data)
    [Document(page_content='Team: Nationals\nPayroll: 81.34', metadata={}), Document(page_content='Team: Reds\nPayroll: 82.2', metadata={})]

指定哪些列是内容，哪些列是元数据​

loader = DuckDBLoader(
    "SELECT * FROM read_csv_auto('example.csv')",
    page_content_columns=["Team"],
    metadata_columns=["Payroll"],
)

data = loader.load()
print(data)
    [Document(page_content='Team: Nationals', metadata={'Payroll': 81.34}), Document(page_content='Team: Reds', metadata={'Payroll': 82.2})]

将来源添加到元数据中​

loader = DuckDBLoader(
    "SELECT Team, Payroll, Team As source FROM read_csv_auto('example.csv')",
    metadata_columns=["source"],
)

data = loader.load()
print(data)
    [Document(page_content='Team: Nationals\nPayroll: 81.34\nsource: Nationals', metadata={'source': 'Nationals'}), Document(page_content='Team: Reds\nPayroll: 82.2\nsource: Reds', metadata={'source': 'Reds'})]