ADR-022: Shared Embedding Infrastructure for Multi-Client RAG

Status

Proposed

Date

2025-12-07

Deciders

• Development Team

Context and Problem Statement

With ADR-021, the CLI has RAG capabilities using pgvector. However, we have multiple clients that could benefit from the same knowledge base:

1. idempiere-cli - CLI with knowledge command and MCP server
2. cloudempiere-backend - Java backend services
3. n8n workflows - Automation via MCP or REST
4. Claude Code - Developer AI assistant via MCP

Each client re-implementing embedding and ingestion would lead to:

• Duplicated embeddings (wasted storage)
• Inconsistent knowledge across clients
• Risk of embedding model mismatch (incompatible vectors)
• Multiple ingestion jobs competing

We need a shared embedding infrastructure where one database serves all clients.

Decision Drivers

• Consistency: All clients query the same knowledge base
• Efficiency: Single ingestion, multiple consumers
• Compatibility: Embedding model MUST be same for all clients
• Simplicity: Minimal coordination between services
• Scalability: Support future clients without changes

Considered Options

1. Shared pgvector + Same Embedding Model - All clients use same DB and embedding model
2. Per-Client Vector DBs - Each client maintains own embeddings
3. Embedding Service (API) - Centralized embedding generation service
4. Hybrid: Shared Read, Separate Write - One writer, multiple readers

Decision Outcome

Chosen option: "Shared pgvector + Same Embedding Model", because:

• Simplest architecture (just database connection sharing)
• Already implemented in CLI (ADR-021)
• No additional services needed
• All clients can use LangChain4j with same model

Confirmation

The decision is confirmed when:

• Backend and CLI both query same cli_embeddings table
• MCP exposes searchKnowledge tool to Claude Code
• Embedding counts match regardless of which client queries

Architecture

Critical Requirement: Same Embedding Model

IMPORTANT: All clients MUST use the same embedding model!

┌─────────────────────────────────────────────────────────────────┐
│                     Embedding Model                              │
│            bge-small-en-q (384 dimensions, ONNX)                │
│                                                                  │
│  Maven: dev.langchain4j:langchain4j-embeddings-bge-small-en-q   │
│                                                                  │
│  ⚠️  Using different model = INCOMPATIBLE VECTORS               │
│  ⚠️  Even same dimensions from different model = INCOMPATIBLE   │
└─────────────────────────────────────────────────────────────────┘

Multi-Client Architecture

┌─────────────────────────────────────────────────────────────────────────┐
│                           Consumers (Read)                               │
├─────────────────────────────────────────────────────────────────────────┤
│                                                                          │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  ┌─────────────┐ │
│  │ Claude Code  │  │    n8n       │  │   Backend    │  │   CLI       │ │
│  │  (via MCP)   │  │ (via MCP)    │  │  Services    │  │  Commands   │ │
│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘  └──────┬──────┘ │
│         │                 │                 │                  │        │
│         │    MCP SSE      │    MCP SSE      │    Direct        │        │
│         └────────┬────────┘                 │                  │        │
│                  │                          │                  │        │
│         ┌────────▼────────┐                 │                  │        │
│         │   MCP Server    │                 │                  │        │
│         │ (idempiere-cli) │                 │                  │        │
│         │                 │                 │                  │        │
│         │ McpKnowledge-   │                 │                  │        │
│         │ Tools           │                 │                  │        │
│         └────────┬────────┘                 │                  │        │
│                  │                          │                  │        │
│                  └──────────┬───────────────┘                  │        │
│                             │                                  │        │
│                  ┌──────────▼──────────────────────────────────▼──────┐ │
│                  │                   RagService                        │ │
│                  │  - search(query, sourceFilter)                      │ │
│                  │  - retrieve(query) → Content[]                      │ │
│                  │  - getContentRetriever() → for LLM augmentation     │ │
│                  └──────────────────────────┬──────────────────────────┘ │
│                                             │                            │
└─────────────────────────────────────────────┼────────────────────────────┘
                                              │
┌─────────────────────────────────────────────┼────────────────────────────┐
│                           Producers (Write)                   │          │
├───────────────────────────────────────────────────────────────┼──────────┤
│                                                               │          │
│  ┌──────────────────────────────────────────────────────────┐ │          │
│  │                    Ingestion Jobs                         │ │          │
│  │                                                           │ │          │
│  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐       │ │          │
│  │  │    Wiki     │  │   K_Entry   │  │ AD Metadata │       │ │          │
│  │  │  Ingestor   │  │  Ingestor   │  │  Ingestor   │       │ │          │
│  │  └─────────────┘  └─────────────┘  └─────────────┘       │ │          │
│  │                                                           │ │          │
│  │  Triggered by: CLI command, CI/CD, cron, backend service  │ │          │
│  └───────────────────────────────────────────────────────────┘ │          │
│                                                                │          │
│  ┌─────────────────────────────────────────────────────────────▼────────┐ │
│  │              EmbeddingStoreProvider (PGVector)                        │ │
│  │  ┌────────────────────────────────────────────────────────────────┐  │ │
│  │  │            EmbeddingModelProvider                               │  │ │
│  │  │         bge-small-en-q (MUST BE SAME FOR ALL)                  │  │ │
│  │  └────────────────────────────────────────────────────────────────┘  │ │
│  └───────────────────────────────────────────────────────────────────────┘ │
│                                              │                             │
└──────────────────────────────────────────────┼─────────────────────────────┘
                                               │
                              ┌────────────────▼────────────────┐
                              │    PostgreSQL + pgvector         │
                              │                                  │
                              │    cli_embeddings table          │
                              │    - embedding vector(384)       │
                              │    - metadata JSONB              │
                              │    - source_type                 │
                              │                                  │
                              │    Shared by all clients!        │
                              └──────────────────────────────────┘

Embedding Model Compatibility

Why This Matters

Embeddings are NOT interchangeable between models:

Even models with same dimensions produce vectors in different semantic spaces.

Our Standard: bge-small-en-q

// ALL clients MUST use this exact model:
import dev.langchain4j.model.embedding.onnx.bgesmallenv15q.BgeSmallEnV15QuantizedEmbeddingModel;

EmbeddingModel model = new BgeSmallEnV15QuantizedEmbeddingModel();
// Produces: 384-dimensional vectors
// Runs: Locally via ONNX (no API calls)

Maven dependency (required in all projects):

<dependency>
    <groupId>dev.langchain4j</groupId>
    <artifactId>langchain4j-embeddings-bge-small-en-q</artifactId>
    <version>${langchain4j.version}</version>
</dependency>

LLM Independence

The embedding model is separate from the chat LLM:

┌─────────────────────────────────────────────────────────────┐
│                    RAG Pipeline                              │
│                                                              │
│  Query: "how to create callout"                              │
│            │                                                 │
│            ▼                                                 │
│  ┌─────────────────────────────────────────────────────┐    │
│  │   Embedding Model (bge-small-en-q)                   │    │
│  │   SAME for all clients - produces query vector       │    │
│  └─────────────────────────────────────────────────────┘    │
│            │                                                 │
│            ▼                                                 │
│  ┌─────────────────────────────────────────────────────┐    │
│  │   pgvector Similarity Search                         │    │
│  │   Returns: relevant document chunks                  │    │
│  └─────────────────────────────────────────────────────┘    │
│            │                                                 │
│            ▼                                                 │
│  ┌─────────────────────────────────────────────────────┐    │
│  │   Chat LLM (ANY - Claude, GPT, Llama, Gemini)       │    │
│  │   Uses retrieved context to generate response        │    │
│  │   Can be DIFFERENT per client!                       │    │
│  └─────────────────────────────────────────────────────┘    │
│                                                              │
└──────────────────────────────────────────────────────────────┘

MCP Integration (ADR-010)

New McpKnowledgeTools

package org.idempiere.cli.mcp.tools;

import dev.langchain4j.rag.content.Content;
import io.quarkiverse.mcp.server.Tool;
import io.quarkiverse.mcp.server.ToolArg;
import jakarta.enterprise.context.ApplicationScoped;
import jakarta.inject.Inject;
import org.idempiere.cli.rag.RagService;

@ApplicationScoped
public class McpKnowledgeTools {

    @Inject
    RagService ragService;

    @Tool(description = "Search the iDempiere knowledge base semantically. " +
          "Includes wiki documentation, KB articles, and Application Dictionary metadata. " +
          "Use for questions about iDempiere development, configuration, or best practices.")
    public String searchKnowledge(
            @ToolArg(description = "Natural language search query") String query,
            @ToolArg(description = "Filter by source: wiki, k_entry, ad_table, ad_window, ad_process, or null for all")
            String sourceFilter) {

        var result = ragService.search(query, sourceFilter);
        return result.toJson();
    }

    @Tool(description = "Get statistics about the knowledge base including document counts by source.")
    public String getKnowledgeStats() {
        var result = ragService.getStatistics();
        return result.toJson();
    }

    @Tool(description = "Retrieve relevant context for augmenting LLM responses. " +
          "Returns raw content chunks suitable for RAG.")
    public String retrieveContext(
            @ToolArg(description = "Query to find relevant context for") String query) {

        var contents = ragService.retrieve(query);
        StringBuilder sb = new StringBuilder();
        for (Content content : contents) {
            sb.append("---\n").append(content.textSegment().text()).append("\n");
        }
        return sb.toString();
    }
}

Claude Desktop / Claude Code Configuration

// ~/.claude/mcp.json
{
  "mcpServers": {
    "idempiere": {
      "url": "http://localhost:8765/mcp/sse",
      "transport": "sse"
    }
  }
}

Now Claude Code can use:

• searchKnowledge("how to create a callout", null)
• getKnowledgeStats()
• retrieveContext("invoice completion process")

Backend Integration

Java Backend Service

package com.cloudempiere.backend.ai;

import dev.langchain4j.data.segment.TextSegment;
import dev.langchain4j.model.embedding.EmbeddingModel;
import dev.langchain4j.model.embedding.onnx.bgesmallenv15q.BgeSmallEnV15QuantizedEmbeddingModel;
import dev.langchain4j.rag.content.retriever.ContentRetriever;
import dev.langchain4j.rag.content.retriever.EmbeddingStoreContentRetriever;
import dev.langchain4j.store.embedding.pgvector.PgVectorEmbeddingStore;

/**
 * Backend service that shares the CLI's embedding store.
 *
 * CRITICAL: Must use same embedding model as CLI (mxbai-embed-large, 1024 dims)
 */
@ApplicationScoped
public class SharedKnowledgeService {

    // MUST match CLI's model exactly
    private final EmbeddingModel embeddingModel = new BgeSmallEnV15QuantizedEmbeddingModel();

    private final PgVectorEmbeddingStore store;

    public SharedKnowledgeService() {
        // Connect to RAG Vector Database (RAG_VECTORDB_PG_* env vars)
        // NOTE: "PG" suffix clarifies this is PostgreSQL with pgvector extension
        this.store = PgVectorEmbeddingStore.builder()
                .host(System.getenv("RAG_VECTORDB_PG_HOST"))
                .port(Integer.parseInt(System.getenv("RAG_VECTORDB_PG_PORT")))
                .database(System.getenv("RAG_VECTORDB_PG_NAME"))
                .user(System.getenv("RAG_VECTORDB_PG_USER"))
                .password(System.getenv("RAG_VECTORDB_PG_PASSWORD"))
                .table("cli_embeddings")  // Same table as CLI
                .dimension(1024)          // mxbai-embed-large dimension
                .createTable(false)       // CLI creates it
                .build();
    }

    public ContentRetriever getRetriever() {
        return EmbeddingStoreContentRetriever.builder()
                .embeddingStore(store)
                .embeddingModel(embeddingModel)  // SAME model
                .maxResults(5)
                .minScore(0.7)
                .build();
    }

    public List<String> search(String query) {
        var embedding = embeddingModel.embed(query).content();
        var results = store.search(EmbeddingSearchRequest.builder()
                .queryEmbedding(embedding)
                .maxResults(5)
                .build());

        return results.matches().stream()
                .map(m -> m.embedded().text())
                .toList();
    }
}

Ingestion Coordination

Single Writer Pattern (Recommended)

To avoid conflicts, designate one ingestion source:

┌─────────────────────────────────────────────────────────────────┐
│                   Ingestion Strategy                             │
├─────────────────────────────────────────────────────────────────┤
│                                                                  │
│  Option A: CLI-Only Ingestion (Recommended)                      │
│  ─────────────────────────────────────────                       │
│  - CI/CD pipeline runs: idempiere-cli knowledge ingest --all     │
│  - Scheduled cron: Daily wiki refresh                            │
│  - Backend and MCP only READ                                     │
│                                                                  │
│  Option B: Distributed Ingestion (Advanced)                      │
│  ──────────────────────────────────────────                      │
│  - Each source has designated writer                             │
│  - Wiki: CI/CD                                                   │
│  - K_Entry: Backend on K_Entry save                              │
│  - AD Metadata: CLI on schema changes                            │
│  - Use source_id to avoid duplicates                             │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

CI/CD Ingestion Example

# .github/workflows/knowledge-sync.yml
name: Sync Knowledge Base

on:
  schedule:
    - cron: '0 2 * * *'  # Daily at 2 AM
  workflow_dispatch:      # Manual trigger

jobs:
  ingest:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Download CLI
        run: |
          wget https://github.com/cloudempiere/idempiere-hub/releases/latest/download/idempiere-hub-runner.jar

      - name: Ingest Knowledge
        env:
          IDEMPIERE_DB_HOST: ${{ secrets.DB_HOST }}
          IDEMPIERE_DB_PORT: 5432
          IDEMPIERE_DB_NAME: idempiere
          IDEMPIERE_DB_USER: ${{ secrets.DB_USER }}
          IDEMPIERE_DB_PASSWORD: ${{ secrets.DB_PASSWORD }}
        run: |
          java -jar idempiere-hub-runner.jar knowledge ingest --source all --force
          java -jar idempiere-hub-runner.jar knowledge status

Configuration

Environment Variables

The CLI uses TWO separate PostgreSQL databases. The naming convention is:

• IDEMPIERE_DB_* - iDempiere application database
• RAG_VECTORDB_PG_* - RAG vector database (PostgreSQL + pgvector)

The "PG" suffix in RAG_VECTORDB_PG_* clarifies this is PostgreSQL with pgvector extension.

# 1. iDempiere Database (IDEMPIERE_DB_*) - Application data
export IDEMPIERE_DB_HOST=localhost      # PostgreSQL host
export IDEMPIERE_DB_PORT=5433           # PostgreSQL port (iDempiere default)
export IDEMPIERE_DB_NAME=idempiere      # Database name
export IDEMPIERE_DB_USER=adempiere      # Database user
export IDEMPIERE_DB_PASSWORD=adempiere  # Database password

# 2. RAG Vector Database (RAG_VECTORDB_PG_*) - Semantic search embeddings
export RAG_VECTORDB_PG_HOST=localhost   # PostgreSQL host
export RAG_VECTORDB_PG_PORT=5432        # PostgreSQL port
export RAG_VECTORDB_PG_NAME=vector      # Database name
export RAG_VECTORDB_PG_USER=postgres    # Database user
export RAG_VECTORDB_PG_PASSWORD=postgres # Database password

Docker Compose

version: '3.8'
services:
  postgres:
    image: pgvector/pgvector:pg16
    environment:
      POSTGRES_DB: idempiere
      POSTGRES_USER: adempiere
      POSTGRES_PASSWORD: adempiere
    ports:
      - "5432:5432"
    volumes:
      - pgdata:/var/lib/postgresql/data

  cli-mcp:
    image: cloudempiere/idempiere-cli:latest
    command: ["mcp-server"]
    environment:
      QUARKUS_PROFILE: mcp
      IDEMPIERE_DB_HOST: postgres
      IDEMPIERE_DB_PORT: 5432
      IDEMPIERE_DB_NAME: idempiere
      IDEMPIERE_DB_USER: adempiere
      IDEMPIERE_DB_PASSWORD: adempiere
    ports:
      - "8765:8765"
    depends_on:
      - postgres

  backend:
    image: cloudempiere/backend:latest
    environment:
      RAG_VECTORDB_PG_HOST: postgres
      RAG_VECTORDB_PG_PORT: 5432
      RAG_VECTORDB_PG_NAME: idempiere
      RAG_VECTORDB_PG_USER: adempiere
      RAG_VECTORDB_PG_PASSWORD: adempiere
    depends_on:
      - postgres

volumes:
  pgdata:

Consequences

Positive

• Single Source of Truth: All clients see same knowledge
• Cost Efficient: One embedding per document, not per client
• Simple Setup: Just database connection, no message queues
• LLM Flexibility: Any LLM works with the shared embeddings

Negative

• Model Lock-in: Changing embedding model requires full re-ingestion
• Coordination Needed: Must ensure all clients use same model
• Single Point of Failure: pgvector DB down = all clients affected

Risks and Mitigations

Implementation Checklist

• [ ] Add McpKnowledgeTools to MCP server
• [ ] Document embedding model requirement in README
• [ ] Add model version metadata to embeddings table
• [ ] Create backend SharedKnowledgeService example
• [ ] Add CI/CD workflow for scheduled ingestion
• [ ] Test multi-client concurrent access

Links

• ADR-021: RAG Architecture - Base RAG implementation
• ADR-010: MCP Server - MCP tool exposure
• ADR-013: LangChain4j Integration - AI framework
• LangChain4j PGVector
• BGE Embedding Models