ADR-057: AWS Bedrock Integration for Multi-Provider LLM Support

Status: Implemented Date: 2025-12-14 Author: Claude Sonnet 4.5 & Norbert Bede Related: ADR-048 (Chat API Server), ADR-056 (Timeouts)

Context

The Chat API server (ADR-048) initially supported Anthropic's direct API and local Ollama models. Users requested support for AWS Bedrock to:

  1. Avoid direct API costs - Use existing AWS enterprise agreements
  2. Regional compliance - Keep data in specific regions (EU/US)
  3. Enterprise features - Leverage AWS security, monitoring, and quotas
  4. Model access - Access Claude Sonnet 4.5 and other models via managed service
  5. Cost optimization - Bedrock pricing can be more favorable for high-volume usage

Problem

Adding AWS Bedrock required:

Initial Attempt - Direct Anthropic API

User initially had an Anthropic API account but ran out of credits. Rather than purchasing more credits, they wanted to use AWS Bedrock (which they already had access to) as a cost-effective alternative.

Decision

Implement AWS Bedrock as a first-class provider alongside Anthropic and Ollama, with Claude Sonnet 4.5 as the default model.

Architecture

┌─────────────────────────────────────────────────────────────┐
│  Chat API Request (OpenAI-compatible)                       │
│  POST /v1/chat/completions                                  │
│  { "model": "eu.anthropic.claude-sonnet-4-5-20250929-v1:0" }│
└────────────────────┬────────────────────────────────────────┘
                     │
                     ▼
         ┌───────────────────────┐
         │   ModelRouter         │
         │   (Pattern Matching)  │
         └───────────┬───────────┘
                     │
         ┌───────────┼───────────┐
         │           │           │
         ▼           ▼           ▼
    ┌────────┐  ┌────────┐  ┌────────┐
    │Bedrock │  │Anthropic│  │Ollama │
    │  EU    │  │ Direct │  │ Local │
    └────────┘  └────────┘  └────────┘
         │
         ▼
┌──────────────────────────────────┐
│  AWS Bedrock Runtime             │
│  Region: eu-west-1 (Ireland)     │
│  Model: claude-sonnet-4-5        │
│  Context: 1M tokens (preview)    │
└──────────────────────────────────┘

Implementation Components

1. ChatAgentSelector.java - Bedrock Provider

@ConfigProperty(name = "quarkus.langchain4j.bedrock.region", defaultValue = "us-east-1")
String bedrockRegion;

@ConfigProperty(name = "quarkus.langchain4j.bedrock.chat-model.model-id",
    defaultValue = "anthropic.claude-3-5-sonnet-20241022-v2:0")
String bedrockModelId;

@ConfigProperty(name = "quarkus.langchain4j.bedrock.timeout", defaultValue = "120s")
Duration bedrockTimeout;

private ChatAgent createBedrockAgent(String model) {
    String modelId = (model != null && !model.isEmpty()) ? model : bedrockModelId;

    return AiServices.builder(ChatAgent.class)
        .chatModel(
            BedrockChatModel.builder()
                .region(Region.of(bedrockRegion))
                .modelId(modelId)  // Bedrock inference profile ID
                .timeout(bedrockTimeout)
                .build()
        )
        .tools(toolProvider)
        .chatMemory(MessageWindowChatMemory.withMaxMessages(10))
        .build();
}

Key Implementation Details:

2. ModelRouter.java - Routing Rules

private static final List<RoutingRule> ROUTING_RULES = List.of(
    // AWS Bedrock - MUST come BEFORE generic Claude patterns
    new RoutingRule("us.anthropic.", "bedrock"),
    new RoutingRule("eu.anthropic.", "bedrock"),
    new RoutingRule("anthropic.", "bedrock"),

    // Anthropic Claude (direct API)
    new RoutingRule("claude-", "anthropic"),
    new RoutingRule("claude-sonnet-4", "anthropic"),

    // Ollama (local)
    new RoutingRule("llama", "ollama"),
    new RoutingRule("qwen", "ollama")
);

Critical: Rule Ordering

3. ProviderRegistry.java - Model Definitions

public static ProviderInfo bedrock() {
    return new ProviderInfo(
        "bedrock",
        "AWS Bedrock",
        "api",
        true,
        List.of(
            // Claude Sonnet 4.5 (September 2025) - Latest & most intelligent
            ModelCapabilities.builder()
                .id("eu.anthropic.claude-sonnet-4-5-20250929-v1:0")
                .name("Claude Sonnet 4.5 (Bedrock EU)")
                .provider("bedrock")
                .ownedBy("anthropic")
                .contextWindow(1000000)  // 1M tokens (preview)
                .maxOutputTokens(16384)
                .supportsVision(true)
                .supportsTools(true)
                .inputPricePerMillion(3.00)
                .outputPricePerMillion(15.00)
                .build(),
            // US variant
            ModelCapabilities.builder()
                .id("us.anthropic.claude-sonnet-4-5-20250929-v1:0")
                .name("Claude Sonnet 4.5 (Bedrock US)")
                .provider("bedrock")
                // ... same capabilities
                .build(),
            // Region-agnostic variant
            ModelCapabilities.builder()
                .id("anthropic.claude-sonnet-4-5-20250929-v1:0")
                .name("Claude Sonnet 4.5 (Bedrock)")
                .provider("bedrock")
                // ... same capabilities
                .build(),
            // Claude 3 Opus, Claude 3 Haiku variants...
        ),
        Map.of("region", "us-east-1")  // Default region hint
    );
}

Model Migration:

4. Configuration (application.properties)

# Enable Bedrock provider
%chat-api.chat-api.providers.bedrock.enabled=true

# AWS region (eu-west-1 for Ireland, us-east-1 for US)
%chat-api.quarkus.langchain4j.bedrock.region=${AWS_REGION:eu-west-1}

# Default model (Claude Sonnet 4.5 EU)
%chat-api.quarkus.langchain4j.bedrock.chat-model.model-id=eu.anthropic.claude-sonnet-4-5-20250929-v1:0

# Timeout (120 seconds default)
%chat-api.quarkus.langchain4j.bedrock.timeout=120s

AWS Bedrock Model IDs (Inference Profiles)

AWS Bedrock uses inference profiles that include region prefixes:

Model ID Region Description
eu.anthropic.claude-sonnet-4-5-20250929-v1:0 EU (Ireland) Claude Sonnet 4.5 EU
us.anthropic.claude-sonnet-4-5-20250929-v1:0 US (Virginia) Claude Sonnet 4.5 US
anthropic.claude-sonnet-4-5-20250929-v1:0 Default Uses account default region
eu.anthropic.claude-3-opus-20240229-v1:0 EU Claude 3 Opus EU
eu.anthropic.claude-3-haiku-20240307-v1:0 EU Claude 3 Haiku EU

Regional Routing:

AWS Credentials

Bedrock uses AWS SDK default credential chain:

  1. Environment Variables (recommended for local dev):
export AWS_ACCESS_KEY_ID="AKIA..."
export AWS_SECRET_ACCESS_KEY="..."
export AWS_REGION="eu-west-1"
  1. AWS Config Files (via aws configure):
~/.aws/credentials
~/.aws/config
  1. IAM Role (for EC2/ECS/Lambda)

Required IAM Permissions:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream"
      ],
      "Resource": "arn:aws:bedrock:*::foundation-model/anthropic.claude-*"
    }
  ]
}

Model Access Enablement

Important: First-time Bedrock users must enable model access in AWS Console:

  1. Go to AWS Bedrock Console → Model access
  2. Click "Modify model access"
  3. Select Anthropic models (Claude Sonnet 4.5, Opus, Haiku)
  4. Submit use case description (required for Anthropic models)
  5. Wait for approval (usually instant for serverless models)

Serverless vs Provisioned:

Claude Sonnet 4.5 Capabilities

Testing

Validation Test

# Test Claude Sonnet 4.5 via Bedrock EU
curl -X POST http://localhost:8081/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "eu.anthropic.claude-sonnet-4-5-20250929-v1:0",
    "messages": [{"role": "user", "content": "Say hello in 3 words"}]
  }'

# Response (successful):
{
  "id": "chatcmpl-69e5c926-319c-429a-90e0-2a500341fdb4",
  "object": "chat.completion",
  "created": 1765740968,
  "model": "eu.anthropic.claude-sonnet-4-5-20250929-v1:0",
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": "Hello there, friend!"
    },
    "finish_reason": "stop"
  }],
  "usage": {
    "input_tokens": 5,
    "output_tokens": 5,
    "total_tokens": 10,
    "estimated_cost_usd": 0.00009000,
    "model": "eu.anthropic.claude-sonnet-4-5-20250929-v1:0"
  },
  "provider": "bedrock"
}

Concurrent Request Test

The implementation is fully concurrent-safe:

Troubleshooting

Common Errors

1. ValidationException: "Model identifier is invalid"

Cause: Model not enabled in AWS Bedrock Console Solution: Enable model access in AWS Console → Bedrock → Model access

2. JSON Parsing Error: "Illegal unquoted character"

Cause: Literal newline in JSON string (multi-line prompts) Solution: Use \n escape or jq for multi-line content

# ❌ WRONG - Literal newline
curl -d '{"messages":[{"content":"Line 1
Line 2"}]}'

# ✅ CORRECT - Escaped newline
curl -d '{"messages":[{"content":"Line 1\nLine 2"}]}'

# ✅ BEST - Use jq
jq -n --arg content "Multi
line
text" '{messages:[{content:$content}]}'

3. Wrong Class Name / Method

Initial Mistakes (corrected):

Consequences

Positive

✅ Multi-provider flexibility - Users can choose Anthropic, Bedrock, or Ollama ✅ Cost optimization - Leverage existing AWS agreements ✅ Regional compliance - EU/US data sovereignty ✅ Enterprise features - AWS monitoring, quotas, security ✅ Latest models - Claude Sonnet 4.5 with 1M context ✅ Concurrent-safe - Handles multiple simultaneous requests ✅ Seamless migration - OpenAI-compatible API unchanged

Negative

⚠️ Additional complexity - Three providers to maintain ⚠️ Model access setup - First-time users must enable in AWS Console ⚠️ AWS credentials required - IAM permissions, region config ⚠️ Regional pricing variance - EU vs US costs may differ ⚠️ Latency - Bedrock adds network hop vs direct API

Neutral

🔵 Model ID format - Bedrock uses inference profiles (region prefixes) 🔵 Credential management - Follows AWS SDK standard patterns 🔵 Multi-line prompts - JSON escaping required (documented in USER_GUIDE.md)

Alternatives Considered

1. Continue with Anthropic Direct API

Rejected: User ran out of credits and preferred AWS Bedrock

2. OpenAI via Azure

Not chosen: User wanted Claude, not GPT

3. Google Vertex AI (Claude via GCP)

Not chosen: User already had AWS infrastructure

4. Self-hosted Claude (vLLM)

Not chosen: Claude weights not publicly available

References

Implementation PR

Files Modified:

Date: 2025-12-14

Path: /docs/developers/architecture/idempiere-hub/057-aws-bedrock-integration