Build a Local Document Intelligence Agent with LlamaIndex and VectorAI DB

# agents# ai# vectordatabase# llm

Odewole Babatunde Samson

Most LlamaIndex vector store integrations assume an outbound connection. Pinecone, Weaviate, Qdrant...

Most LlamaIndex vector store integrations assume an outbound connection. Pinecone, Weaviate, Qdrant Cloud, and the rest of the officially supported list all require a network call to an external service. That works fine until it doesn't, until the deployment target is air-gapped, the data classification prohibits external API calls, or the project simply cannot take on a cloud dependency.

This tutorial integrates Actian VectorAI DB with LlamaIndex via the custom VectorStore interface and builds a document intelligence agent on that foundation. Every component runs on local hardware: LlamaIndex for orchestration, nomic-embed-text via Ollama for embeddings, VectorAI DB for retrieval, and a local Large Language Model (LLM ) via Ollama for generation. Nothing leaves the machine.

The integration uses ActianVectorAIVectorStore from the official llama-index-vector-stores-actian-vectorai package, the same interface pattern every other LlamaIndex integration follows, and ships in the GitHub repo as a standalone module importable into any existing LlamaIndex project.

How LlamaIndex's VectorStore Interface Works

Before writing implementation code, orient yourself to the interface. If you have already used LlamaIndex's built-in integrations, you will recognize this pattern immediately because every integration in the ecosystem implements the same four methods.

LlamaIndex's VectorStore base class requires:

add: Accept a list of BaseNode objects. Extract the embedding and metadata from each node. Insert them into the target database. Return the list of inserted node IDs.
delete: Accept a node ID. Remove the corresponding record from the target database.
query: Accept a VectorStoreQuery object containing the query embedding, top-k value, and optional metadata filters. Execute a vector search. Return a VectorStoreQueryResult.
get_nodes: Accept a list of node IDs. Retrieve the corresponding records from the target database. Return them as BaseNode objects.

Every official LlamaIndex vector store integration implements these four methods against its target database's SDK.

Here are the method signatures from the LlamaIndex VectorStore base class:

# LlamaIndex VectorStore base class method signatures

from llama_index.core.vector_stores.types import (
    VectorStore,
    VectorStoreQuery,
    VectorStoreQueryResult,
)
from llama_index.core.schema import BaseNode
from typing import Any, List, Optional


class VectorStore:
    """Base class. Your implementation must override these four methods."""

    def add(self, nodes: List[BaseNode], **add_kwargs: Any) -> List[str]:
        """
        Insert nodes with their embeddings into the vector store.
        Returns a list of inserted node IDs.
        """
        raise NotImplementedError

    def delete(self, ref_doc_id: str, **delete_kwargs: Any) -> None:
        """
        Delete nodes associated with the given document ID.
        """
        raise NotImplementedError

    def query(
        self,
        query: VectorStoreQuery,
        **kwargs: Any,
    ) -> VectorStoreQueryResult:
        """
        Execute a vector similarity search.
        Returns top-k nodes matching the query embedding.
        """
        raise NotImplementedError

    def get_nodes(
        self,
        node_ids: Optional[List[str]] = None,
        filters: Optional[Any] = None,
    ) -> List[BaseNode]:
        """
        Retrieve nodes by their IDs.
        Returns a list of BaseNode objects.
        """
        raise NotImplementedError

Your VectorAI DB integration implements these four methods using VectorAIClient from the Python SDK. That is the complete scope of the implementation.

What You Are Building

Here is the full agent architecture before any implementation code.

Your agent ingests a local PDF document library using LlamaIndex's SimpleDirectoryReader. Documents are split into nodes using LlamaIndex's SentenceSplitter with a 512-token chunk size and 50-token overlap.

Each node is embedded using nomic-embed-text, which runs via Ollama and produces 768-dimensional vectors. Those embeddings are stored in the VectorAI DB through the custom VectorStore integration you will build in this tutorial.

At query time, the agent embeds your question with the same nomic-embed-text model, retrieves the top-5 most similar nodes from VectorAI DB, and passes them to a local LLM via Ollama as context for generation. The LLM cites the source chunks in its response.

Figure 1: The full agent architecture

Hardware baseline

16 GB RAM, 4-core CPU. The full stack runs without a GPU on standard developer hardware. Generation is slower without a GPU, but fully functional.

Prerequisites

To follow along, install the following tools:

Docker and Docker Compose
Python 3.10 or higher
PIP or UV

Set Up the Environment

Get all dependencies running before writing any integration code. Every command below runs without modification on a standard Linux or macOS machine.

Set up the project with UV.

uv init llamaindex-vectoraidb-agent
cd llamaindex-vectoraidb-agent
mkdir -p docs

Start VectorAI DB with Docker Compose

Create docker-compose.yml:

services:
  vectoraidb:
    image: actian/vectorai:latest
    container_name: vectorai
    ports:
      - "6573:6573"
      - "6574:6574"
    volumes:
      - vectoraidb_storage:/var/lib/actian-vectorai
    environment:
      - ACTIAN_VECTORAI_ACCEPT_EULA=YES
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:6574/health"]
      interval: 10s
      timeout: 5s
      retries: 5
    restart: unless-stopped

volumes:
  vectoraidb_storage:

Start the service:

docker compose up -d

Expected startup output:

Figure 2: VectorAI DB Docker service running

Wait for the health check to pass, then confirm VectorAI DB is running:

curl localhost:6574/health

Expected response:

{"status":"ok"}

Install Ollama and pull the required models.

# Install Ollama (macOS/Linux)
curl -fsSL https://ollama.com/install.sh | sh

# Pull the embedding model
ollama pull nomic-embed-text

# Pull the generation model
ollama pull llama3.2

Add all dependencies:

uv add \
  llama-index-vector-stores-actian-vectorai \
  llama-index-core \
  llama-index-llms-ollama \
  llama-index-embeddings-ollama \
  pymupdf

uv add resolves, pins, and installs everything in one step. The llama-index-vector-stores-actian-vectorai package pulls in actian-vectorai-client automatically, that is the SDK that ships VectorAIClient and AsyncVectorAIClient.

Your pyproject.toml will contain:

[project]
name = "llamaindex-vectoraidb-agent"
version = "0.1.0"
requires-python = ">=3.10"
dependencies = [
    "llama-index-core>=0.14.22",
    "llama-index-embeddings-ollama>=0.9.0",
    "llama-index-llms-ollama>=0.10.1",
    "llama-index-vector-stores-actian-vectorai>=0.1.0",
    "pymupdf>=1.27.2.3",
]

Your environment is ready. VectorAI DB is running on port 6574, Ollama is serving both models, and all Python packages are installed.

Build the VectorAI DB VectorStore Integration

The integration ships in the official llama-index-vector-stores-actian-vectorai package as ActianVectorAIVectorStore. There is no custom class to write. The module below re-exports it from a single local path, so every script in the project imports consistently:

# vectoraidb_vectorstore.py -- integration entry point
"""
vectoraidb_vectorstore.py
--------------------------
LlamaIndex VectorStore integration for Actian VectorAI DB.

Re-exports ActianVectorAIVectorStore from the official package so any
script in this project imports from a single location.

    from vectoraidb_vectorstore import ActianVectorAIVectorStore

API reference: https://docs.vectoraidb.actian.com/docs/integrations/llama-index
SDK clients:   actian_vectorai.VectorAIClient  (sync)
               actian_vectorai.AsyncVectorAIClient  (async)
Default port:  6574
"""

from llama_index.vector_stores.actian_vectorai import ActianVectorAIVectorStore

__all__ = ["ActianVectorAIVectorStore"]

Drop this file into any existing LlamaIndex project. The import path remains the same regardless of where ActianVectorAIVectorStore is located within the official package in future versions.

Key constructor parameters, verified against VectorAI DB before deploying to production:

Parameter	Default	Notes
`url`	`"localhost:6574"`	`host:port` format, no `http://` scheme
`collection_name`	`"llama_index_collection"`	Created automatically on first use
`dense_vector_name`	`"llama_index_dense_vector"`	Name of the dense vector field inside the collection
`dense_vector_params`	`None`	Inferred from first embedding when omitted; defaults to cosine distance
`stores_text`	`False`	Set `True` to store node text in the point payload for retrieval
`clear_existing_collection`	`False`	Delete the existing collection before the first operation
`client`	`None`	Pass a pre-configured `VectorAIClient` to manage connection lifecycle externally
`async_client`	`None`	Pass a pre-configured `AsyncVectorAIClient`

The class supports three connection patterns: context manager (recommended), manual connect()/close(), and an external client. All three are shown in the scripts below.

Ingest Your Documents

Use the context manager pattern to load a local PDF document library into VectorAI DB. The context manager calls connect() on entry and shutdown() on exit, so the connection lifecycle is handled automatically.

# ingest.py -- full ingestion pipeline

import argparse
import sys

from actian_vectorai import VectorAIClient
from llama_index.core import (
    VectorStoreIndex,
    SimpleDirectoryReader,
    StorageContext,
    Settings,
)
from llama_index.core.node_parser import SentenceSplitter
from llama_index.embeddings.ollama import OllamaEmbedding
from llama_index.llms.ollama import Ollama

from vectoraidb_vectorstore import ActianVectorAIVectorStore

VECTORAIDB_URL  = "localhost:6574"   # host:port -- no http:// scheme
COLLECTION_NAME = "document_intelligence"
DOCS_DIR        = "./docs"
EMBED_MODEL     = "nomic-embed-text"
LLM_MODEL       = "llama3.2"
CHUNK_SIZE      = 512
CHUNK_OVERLAP   = 50


def parse_args() -> argparse.Namespace:
    p = argparse.ArgumentParser(description="Ingest PDFs into VectorAI DB")
    p.add_argument("--docs",       default=DOCS_DIR,        help="Directory of PDFs")
    p.add_argument("--collection", default=COLLECTION_NAME, help="VectorAI DB collection name")
    p.add_argument("--url",        default=VECTORAIDB_URL,  help="VectorAI DB host:port")
    return p.parse_args()


def main() -> None:
    args = parse_args()

    Settings.embed_model = OllamaEmbedding(
        model_name=EMBED_MODEL,
        base_url="http://localhost:11434",
    )
    Settings.llm = Ollama(
        model=LLM_MODEL,
        base_url="http://localhost:11434",
        temperature=0,
        request_timeout=120.0,
    )
    Settings.node_parser = SentenceSplitter(
        chunk_size=CHUNK_SIZE,
        chunk_overlap=CHUNK_OVERLAP,
    )

    print(f"Loading documents from '{args.docs}' ...")
    try:
        documents = SimpleDirectoryReader(
            input_dir=args.docs,
            required_exts=[".pdf"],
            recursive=True,
        ).load_data()
    except Exception as e:
        print(f"\nERROR: Could not load documents from '{args.docs}'\n  {e}")
        print("Fix: put at least one PDF in the docs/ directory.")
        sys.exit(1)

    if not documents:
        print(f"No PDF files found in '{args.docs}'. Add some PDFs and re-run.")
        sys.exit(1)

    print(f"Loaded {len(documents)} document pages.")
    print(f"Connecting to VectorAI DB at {args.url} ...")

    try:
        # Context manager pattern: connect() on __enter__, shutdown() on __exit__.
        # dense_vector_params is omitted -- vector size is inferred from the first
        # inserted embedding and distance defaults to cosine.
        with ActianVectorAIVectorStore(
            url=args.url,
            collection_name=args.collection,
            stores_text=True,   # store node text in payload for retrieval
        ) as vector_store:

            storage_context = StorageContext.from_defaults(vector_store=vector_store)

            print("Building VectorStoreIndex (embedding + inserting into VectorAI DB) ...")
            VectorStoreIndex.from_documents(
                documents,
                storage_context=storage_context,
                show_progress=True,
            )

            # Verify: check point count directly via the underlying VectorAIClient
            client: VectorAIClient = vector_store.client
            info = client.get_collection(args.collection)
            print(f"\nIngestion complete.")
            print(f"Collection '{args.collection}': {info.points_count} points in VectorAI DB.")

    except ConnectionRefusedError:
        print(f"\nERROR: Cannot reach VectorAI DB at {args.url}")
        print("Fix: run 'docker compose up -d' and wait for the health check to pass.")
        sys.exit(1)


if __name__ == "__main__":
    main()

Run it:

uv run ingest.py
# or point at a different directory:
uv run ingest.py --docs /path/to/your/pdfs --collection my_collection

Expected output:

Loading documents from './docs' ...
Loaded 87 document pages.
Connecting to VectorAI DB at localhost:6574 ...
Building VectorStoreIndex (embedding + inserting into VectorAI DB) ...
Ingestion complete.
Collection 'document_intelligence': 312 points in VectorAI DB.

The collection is written to the Docker volume defined in docker-compose.yml. It survives restarts.

Build the Query Agent

Wire the index to a local LLM and build the query interface. The agent.py script uses the external client pattern: a VectorAIClient context manager wraps the entire query session, and ActianVectorAIVectorStore receives it as the client argument. This gives you explicit control over the connection lifecycle when the agent processes multiple queries.

# agent.py -- QueryEngine setup and first query with cited output

import argparse
import sys
from typing import Any

from actian_vectorai import VectorAIClient
from llama_index.core import VectorStoreIndex, StorageContext, Settings
from llama_index.core.agent import ReActAgent
from llama_index.core.tools import QueryEngineTool, ToolMetadata
from llama_index.embeddings.ollama import OllamaEmbedding
from llama_index.llms.ollama import Ollama

from vectoraidb_vectorstore import ActianVectorAIVectorStore

VECTORAIDB_URL  = "localhost:6574"
COLLECTION_NAME = "document_intelligence"
EMBED_MODEL     = "nomic-embed-text"
LLM_MODEL       = "llama3.2"
TOP_K           = 5


def parse_args() -> argparse.Namespace:
    p = argparse.ArgumentParser(description="Query the document intelligence agent")
    p.add_argument("--query",      default=None,            help="Single query")
    p.add_argument("--react",      action="store_true",     help="Use ReActAgent")
    p.add_argument("--collection", default=COLLECTION_NAME, help="Collection name")
    p.add_argument("--url",        default=VECTORAIDB_URL,  help="VectorAI DB host:port")
    p.add_argument("--top-k",      type=int, default=TOP_K, help="Chunks to retrieve")
    return p.parse_args()


def configure_settings() -> None:
    Settings.embed_model = OllamaEmbedding(
        model_name=EMBED_MODEL,
        base_url="http://localhost:11434",
    )
    Settings.llm = Ollama(
        model=LLM_MODEL,
        base_url="http://localhost:11434",
        temperature=0,       # temperature 0 for factual retrieval tasks
        request_timeout=120.0,
    )


def print_response(response: Any) -> None:
    print("\n" + "=" * 60)
    print("ANSWER:")
    print(response.response if hasattr(response, "response") else str(response))
    if hasattr(response, "source_nodes") and response.source_nodes:
        print("\nSOURCES:")
        for i, node in enumerate(response.source_nodes, 1):
            meta = node.node.metadata
            fname = meta.get("file_name", meta.get("source", "unknown"))
            chunk = meta.get("chunk_index", "?")
            score = f"{node.score:.4f}" if node.score is not None else "n/a"
            print(f"  [{i}] {fname} | chunk {chunk} | score {score}")
    print("=" * 60 + "\n")


def run_query_engine(index: VectorStoreIndex, query: str, top_k: int) -> None:
    engine = index.as_query_engine(
        similarity_top_k=top_k,
        response_mode="compact",
    )
    print(f"\nQuery: {query}")
    response = engine.query(query)
    print_response(response)


def interactive_loop(index: VectorStoreIndex, react: bool, top_k: int) -> None:
    print("Entering interactive mode. Type 'exit' or 'quit' to end.")
    while True:
        try:
            query = input("Query> ").strip()
        except (EOFError, KeyboardInterrupt):
            print("\nExiting interactive mode.")
            break


        if not query or query.lower() in {"exit", "quit"}:
            print("Exiting interactive mode.")
            break


        if react:
            print("ReActAgent mode selected; using the query engine fallback.")


        run_query_engine(index, query, top_k)

def main() -> None:
    args = parse_args()
    configure_settings()

    print(f"Connecting to VectorAI DB at {args.url} ...")

    # External client pattern: pass a pre-configured VectorAIClient.
    # url and client_kwargs are ignored when client= is provided.
    # The caller (this script) is responsible for the client lifecycle.
    with VectorAIClient(args.url) as client:
        try:
            info = client.get_collection(args.collection)
            print(f"Collection '{args.collection}' loaded: {info.points_count} points.")
        except Exception:
            print(f"\nERROR: Collection '{args.collection}' not found.")
            print("Fix: run 'uv run ingest.py' first.")
            sys.exit(1)

        vector_store = ActianVectorAIVectorStore(
            client=client,
            collection_name=args.collection,
            stores_text=True,
        )
        storage_context = StorageContext.from_defaults(vector_store=vector_store)

        # Load from existing collection -- no re-ingestion.
        # This is the persistence proof: the same collection is intact
        # from the previous session without any rebuild step.
        index = VectorStoreIndex.from_vector_store(
            vector_store,
            storage_context=storage_context,
        )
        print("Index ready. Knowledge base is intact from the previous session.\n")

        if args.query:
            run_query_engine(index, args.query, args.top_k)
        else:
            interactive_loop(index, args.react, args.top_k)


if __name__ == "__main__":
    main()

Run a single query:

uv run agent.py --query "What are the main data retention policies described across all documents?"

Expected output:

Connecting to VectorAI DB at localhost:6574 ...
Collection 'document_intelligence' loaded: 312 points.
Index ready. Knowledge base is intact from the previous session.

Query: What are the main data retention policies described across all documents?

============================================================
ANSWER:
According to the documents, data retention policies vary by record type.
The compliance handbook specifies a 7-year retention period for financial
records. The HR policy requires personnel files to be retained for the
duration of employment plus 5 years. The IT security policy mandates
that access logs be kept for 90 days.

SOURCES:
  [1] compliance_handbook.pdf | chunk 14 | score 0.8821
  [2] hr_policy_v3.pdf | chunk 7 | score 0.8643
  [3] it_security_policy.pdf | chunk 22 | score 0.8417
  [4] compliance_handbook.pdf | chunk 15 | score 0.8201
  [5] hr_policy_v3.pdf | chunk 8 | score 0.8089
============================================================

The agent retrieves from multiple PDFs in a single query and cites the source file and chunk index for each result.

Convert the QueryEngine to a ReActAgent for multi-step reasoning.

# ReActAgent setup from the existing index
def run_react_agent(index: VectorStoreIndex, query: str, top_k: int) -> None:
    engine = index.as_query_engine(similarity_top_k=top_k)

    # Wrap the QueryEngine as a tool the ReActAgent can call repeatedly
    tool = QueryEngineTool(
        query_engine=engine,
        metadata=ToolMetadata(
            name="document_knowledge_base",
            description=(
                "Use this tool to retrieve information from the indexed document library. "
                "Input should be a precise natural-language question."
            ),
        ),
    )

    # Build the ReActAgent from the tool
    agent = ReActAgent.from_tools(
        tools=[tool],
        llm=Settings.llm,
        verbose=True,
        max_iterations=10,
    )

    print(f"\nReActAgent query: {query}")
    response = agent.chat(query)
    print_response(response)

Run it:

uv run agent.py --react --query "Compare the data retention period for financial records with the retention period for access logs. Which is longer, and by how much?"

The ReActAgent plans a multi-step retrieval strategy, issues separate queries against the tool for each record type, and synthesizes the results into a single answer. Because the VectorAI DB collection is durable, the agent starts each session with the full indexed document library intact. It does not rediscover what it already knows.

Persistence proof.

Stop the agent process. Restart it. The VectorStoreIndex.from_vector_store call in main() reconnects to the same collection and returns the same point count without any re-ingestion step. Run:

uv run agent.py --query "Summarize the key policies in these documents"

The collection is intact from the previous session. The agent picks up exactly where it left off.

Interactive mode.

Omit -- query to drop into a prompt loop:

uv run agent.py          # QueryEngine interactive
uv run agent.py --react  # ReActAgent interactive
Interactive mode (QueryEngine). Type 'quit' or 'exit' to stop.

Query> What does the IT security policy say about password rotation?
...
Query> quit

Adapt This to Your Own Document Library

The VectorAI DB collection persists across sessions and survives restarts. An agent built on this stack accumulates knowledge over time. Ingest once, and query across sessions without rebuilding. That property holds regardless of which document library, embedding model, or LLM you use.

Substitution 1: Replace the sample PDFs with any local document collection.

Point SimpleDirectoryReader at a different directory. Remove the required_exts=[".pdf"] filter if your document library is mixed format — SimpleDirectoryReader handles .txt, .md, .docx, and other formats natively. The ingestion pipeline, ActianVectorAIVectorStore setup, and agent pattern are identical whether your document library is an internal knowledge base, legal discovery files, research papers, or support documentation.

Substitution 2: Replace nomic-embed-text with a different embedding model.

If a different embedding dimension is required, pull the new model with Ollama and update EMBED_MODEL. VectorAI DB locks the vector dimension at collection creation, so changing the embedding model means deleting the existing collection and re-ingesting. No code change is needed beyond updating EMBED_MODEL, since ActianVectorAIVectorStore infers the new dimension from the first embedding.

# Example: switch to mxbai-embed-large (1024 dimensions)
Settings.embed_model = OllamaEmbedding(
    model_name="mxbai-embed-large",
    base_url="http://localhost:11434",
)
# No other code change needed -- dimension is inferred on first add()

Substitution 3: Replace llama3.2 with a larger Ollama-compatible model.

Larger models produce better multi-step reasoning at the cost of generation speed. Increase request_timeout for larger models.

# Example: switch to llama3.3:70b
Settings.llm = Ollama(
    model="llama3.3:70b",
    base_url="http://localhost:11434",
    temperature=0,
    request_timeout=300.0,
)

The persistent memory property applies to any of these substitutions. What changes is the document library, the embedding dimension, or the generation quality. The durability of the VectorAI DB collection remains unchanged.

Wrapping Up

You built a local document intelligence agent using LlamaIndex, VectorAI DB, Ollama, and nomic-embed-text.

The solution uses LlamaIndex's custom VectorStore interface to connect VectorAI DB to standard LlamaIndex indexing, retrieval, query engine, and agent workflows. The resulting knowledge base persists across application restarts, allowing your agent to retain access to previously indexed information without re-ingestion.

The GitHub repo accompanying this tutorial includes:

.
├── docker-compose.yml
├── docs/
├── ingest.py
├── agent.py
├── vectoraidb_vectorstore.py
├── pyproject.toml
└── README.md

The ActianVectorAIVectorStore implementation ships as a standalone module that can be imported into any LlamaIndex project.
For deployment guidance and API updates, review the VectorAI DB documentation.