Skip to main content
Open In ColabOpen on GitHub

SingleStoreLoader

The SingleStoreLoader allows you to load documents directly from a SingleStore database table. It is part of the langchain-singlestore integration package.

Overview​

Integration Details​

ClassPackageJS Support
SingleStoreLoaderlangchain_singlestore❌

Features​

  • Load documents lazily to handle large datasets efficiently.
  • Supports native asynchronous operations.
  • Easily configurable to work with different database schemas.

Setup​

To use the SingleStoreLoader, you need to install the langchain-singlestore package. Follow the installation instructions below.

Installation​

Install langchain_singlestore.

%pip install -qU langchain_singlestore

Initialization​

To initialize SingleStoreLoader, you need to provide connection parameters for the SingleStore database and specify the table and fields to load documents from.

Required Parameters:​

  • host (str): Hostname, IP address, or URL for the database.
  • table_name (str): Name of the table to query. Defaults to embeddings.
  • content_field (str): Field containing document content. Defaults to content.
  • metadata_field (str): Field containing document metadata. Defaults to metadata.

Optional Parameters:​

  • id_field (str): Field containing document IDs. Defaults to id.

Connection Pool Parameters:​

  • pool_size (int): Number of active connections in the pool. Defaults to 5.
  • max_overflow (int): Maximum connections beyond pool_size. Defaults to 10.
  • timeout (float): Connection timeout in seconds. Defaults to 30.

Additional Options:​

  • pure_python (bool): Enables pure Python mode.
  • local_infile (bool): Allows local file uploads.
  • charset (str): Character set for string values.
  • ssl_key, ssl_cert, ssl_ca (str): Paths to SSL files.
  • ssl_disabled (bool): Disables SSL.
  • ssl_verify_cert (bool): Verifies server's certificate.
  • ssl_verify_identity (bool): Verifies server's identity.
  • autocommit (bool): Enables autocommits.
  • results_type (str): Structure of query results (e.g., tuples, dicts).
from langchain_singlestore.document_loaders import SingleStoreLoader

loader = SingleStoreLoader(
    host="127.0.0.1:3306/db",
    table_name="documents",
    content_field="content",
    metadata_field="metadata",
    id_field="id",
)

Load​

docs = loader.load()
docs[0]
print(docs[0].metadata)

Lazy Load​

page = []
for doc in loader.lazy_load():
    page.append(doc)
    if len(page) >= 10:
        # do some paged operation, e.g.
        # index.upsert(page)

        page = []

API reference​

For detailed documentation of all SingleStore Document Loader features and configurations head to the github page: https://github.com/singlestore-labs/langchain-singlestore/