ADR-017: AD_Element Description Management

<!-- MADR 3.0 Template - Markdown Any Decision Records -->

Status

Proposed

Date

2025-12-06

Deciders

• Norbert Bede

Context and Problem Statement

The iDempiere Application Dictionary contains thousands of AD_Element records that define terminology (Name, PrintName, Description, Help) for columns across the system. Many elements have missing or poor-quality Description and Help text. Current state analysis shows:

• 4,035 AD_Element records needing Description updates
• 3,135 AD_Element records needing Help text
• 218 AD_Process + 512 AD_Tab records with missing documentation

Manual documentation is time-consuming and inconsistent. A systematic approach is needed that combines:

1. Pattern-based automatic generation for predictable columns (_UU, _ID, Is*, Qty*, Amt*, Date*, _Acct)
2. Java code analysis to extract business logic context
3. AI-assisted generation for complex descriptions
4. Migration script generation for PostgreSQL and Oracle

Decision Drivers

• Quality: Consistent, professional documentation following iDempiere standards
• Efficiency: Automated generation for pattern-based columns (>50% of elements)
• Intelligence: AI-enhanced descriptions for complex business logic
• Maintainability: Reusable CLI tools that integrate with existing idempiere-cli
• Portability: Adopt proven Python scripts while enabling Java-native implementation
• Standards: Follow ADR-001 (iDempiere Documentation Standards) for output

Considered Options

1. Keep Python scripts standalone - Maintain separate Python tooling
2. Full Java port - Rewrite all functionality in Java within idempiere-cli
3. Hybrid approach - CLI orchestration with Python utilities and AI agent integration

Decision Outcome

Chosen option: "Hybrid approach", because it maximizes reuse of proven Python logic while providing CLI integration, AI enhancement, and future path to Java-native implementation.

Confirmation

• [ ] CLI command element describe generates accurate descriptions for test elements
• [ ] Pattern-based generation matches original Python output
• [ ] AI agent produces descriptions following ADR-001 standards
• [ ] Migration scripts validate against iDempiere seed database

Pros and Cons of the Options

Option 1: Keep Python scripts standalone

Keep Python scripts in .claude/tools/element-docs/ directory.

• Good, because proven working code remains unchanged
• Good, because Python is well-suited for data transformation
• Bad, because requires separate Python environment
• Bad, because no integration with idempiere-cli workflow
• Bad, because duplicates seed database extraction logic

Option 2: Full Java port

Rewrite all Python scripts as Java services.

• Good, because single technology stack
• Good, because tight integration with existing services
• Bad, because significant development effort
• Bad, because Java less suited for rapid text transformation
• Bad, because delays feature availability

Option 3: Hybrid approach (Chosen)

CLI orchestration calling Python scripts + AI agent for complex cases.

• Good, because immediate availability of proven logic
• Good, because AI agent handles edge cases intelligently
• Good, because CLI provides unified interface
• Good, because gradual migration path to Java-native
• Neutral, because requires Python runtime for full functionality
• Bad, because two codebases to maintain initially

Architecture

Component Overview

┌────────────────────────────────────────────────────────────────────────┐
│                        idempiere-cli                                   │
├────────────────────────────────────────────────────────────────────────┤
│  Commands                                                              │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────────┐    │
│  │ element describe│  │ element analyze │  │ element migrate     │    │
│  └────────┬────────┘  └────────┬────────┘  └──────────┬──────────┘    │
│           │                    │                       │               │
├───────────┼────────────────────┼───────────────────────┼───────────────┤
│  Services │                    │                       │               │
│  ┌────────▼────────────────────▼───────────────────────▼──────────┐   │
│  │               ElementDescriptionService                         │   │
│  │  ┌─────────────────┐  ┌──────────────────┐  ┌───────────────┐  │   │
│  │  │ PatternGenerator│  │ JavaCodeAnalyzer │  │ MigrationGen  │  │   │
│  │  └─────────────────┘  └──────────────────┘  └───────────────┘  │   │
│  └────────────────────────────────────────────────────────────────┘   │
│                                    │                                   │
│  ┌─────────────────────────────────▼──────────────────────────────┐   │
│  │                   AI Agent Integration                          │   │
│  │  ┌──────────────────────────────────────────────────────────┐  │   │
│  │  │  idempiere-description-writer (Claude Code Agent)        │  │   │
│  │  │  - Pattern-based generation rules                        │  │   │
│  │  │  - ADR-001 documentation standards                       │  │   │
│  │  │  - Business context analysis                             │  │   │
│  │  └──────────────────────────────────────────────────────────┘  │   │
│  └────────────────────────────────────────────────────────────────┘   │
└────────────────────────────────────────────────────────────────────────┘
                                    │
                                    ▼
┌────────────────────────────────────────────────────────────────────────┐
│                      External Resources                                │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────────┐    │
│  │  Seed Database  │  │  Java Source    │  │  REST API           │    │
│  │  (Adempiere_pg) │  │  (M*.java)      │  │  (AD_Element)       │    │
│  └─────────────────┘  └─────────────────┘  └─────────────────────┘    │
│  ┌─────────────────┐  ┌─────────────────┐  ┌─────────────────────┐    │
│  │  Translation    │  │  GitHub Repos   │  │  2Pack XML          │    │
│  │  XML Export     │  │  (raw files)    │  │  Import/Export      │    │
│  └─────────────────┘  └─────────────────┘  └─────────────────────┘    │
└────────────────────────────────────────────────────────────────────────┘

CLI Commands

element describe

Generate descriptions for AD_Element records.

# Generate descriptions for all missing elements
idempiere-cli element describe --missing

# Generate for specific pattern
idempiere-cli element describe --pattern "_UU"
idempiere-cli element describe --pattern "Is*"

# Generate for specific element
idempiere-cli element describe --column "M_Product_ID"

# Use AI for complex descriptions
idempiere-cli element describe --ai --column "QtyReserved"

# Output formats
idempiere-cli element describe --missing --format json
idempiere-cli element describe --missing --format sql
idempiere-cli element describe --missing --format 2pack

element analyze

Analyze elements and extract context.

# Parse seed database for element context
idempiere-cli element analyze --seed /path/to/Adempiere_pg.dmp

# Extract Java business logic
idempiere-cli element analyze --java /path/to/idempiere/org.adempiere.base

# Find missing descriptions
idempiere-cli element analyze --missing --output element_gaps.json

# Categorize by domain
idempiere-cli element analyze --categorize

element migrate

Generate migration scripts.

# Generate migration for all missing descriptions
idempiere-cli element migrate --missing --output /path/to/migration

# Generate for specific category
idempiere-cli element migrate --category "Accounting"

# Preview without generating files
idempiere-cli element migrate --missing --dry-run

Data Sources

The CLI can extract AD_Element data from multiple sources:

1. REST API (Primary)

Uses iDempiere REST Web Services to query AD_Element and AD_Element_Trl:

# Query elements via REST API
idempiere-cli element analyze --rest --filter "Description IS NULL"

# Export translations via REST
idempiere-cli element describe --rest --lang es_ES

REST endpoints used:

• GET /api/v1/models/ad_element - Query elements
• GET /api/v1/models/ad_element_trl - Query translations
• PUT /api/v1/models/ad_element/{id} - Update element

2. Translation XML Export/Import

Leverages iDempiere's core Translation.java class for XML format:

# Export element translations to XML
idempiere-cli translation export --table AD_Element --lang en_US

# Import updated descriptions from XML
idempiere-cli translation import --file AD_Element_Trl_en_US.xml

Translation XML format (from Translation.java):

<idempiereTrl table="AD_Element_Trl" language="en_US">
  <row id="100" trl="Y">
    <value column="Name">Active</value>
    <value column="PrintName">Active</value>
    <value column="Description">The record is active in the system</value>
    <value column="Help">There are two methods of making records unavailable...</value>
  </row>
</idempiereTrl>

3. Seed Database (Offline)

Parse PostgreSQL seed dump for offline analysis:

# Download and parse seed database
curl -L -o Adempiere_pg.jar https://raw.githubusercontent.com/idempiere/binary.file/master/database/12/Adempiere_pg.jar
unzip Adempiere_pg.jar Adempiere_pg.dmp
idempiere-cli element analyze --seed Adempiere_pg.dmp

4. GitHub Raw Files

Fetch translation files directly from GitHub repositories:

# Fetch from iDempiere core translations
idempiere-cli element fetch --github idempiere/idempiere --path migration/i12.0/oracle/

# Fetch from community translations
idempiere-cli element fetch --github globalqss/idempiere-es --path translation/es_ES/

5. 2Pack XML

Export/import as 2Pack XML for deployment:

# Export elements as 2Pack
idempiere-cli element migrate --format 2pack --output element_descriptions.zip

# Import via existing 2pack command
idempiere-cli 2pack import --file element_descriptions.zip

Pattern-Based Generation Rules

Based on proven Python implementation patterns:

Standard Columns

Business Logic Enhancement

Java code analysis extracts:

1. Validation rules: FillMandatory → "Required field"
2. Read-only conditions: Cannot modify when parent is processed → "Read-only after processing"
3. Calculations: Calculated by multiplication → "Calculated value"
4. Triggers: Triggers recalculation when changed → "Changes trigger related calculations"

AI Agent Definition

The idempiere-description-writer agent is a Claude Code agent for generating AD_Element documentation.

Location: .claude/agents/idempiere-description-writer.md (symlink to cloudempiere-workspace)

Source: cloudempiere-workspace/.claude/agents/idempiere/description-writer.md

The agent provides:

• Templates for all AD element types (Windows, Tabs, Fields, Processes, Reports, Forms)
• Field length guidelines (Name: 60 chars, Description: 255 chars, Help: 2000 chars)
• Writing style rules (present tense, active voice, no abbreviations in Help)
• Standard phrases for Boolean, Date, Amount, and Reference fields
• PO (Purchase Order) variant handling for dual-context elements

Key rules enforced:

• ALWAYS use "iDempiere" as product name, never "Compiere" or "ADempiere"
• NEVER expose class names, method names, or technical implementation details
• NEVER use column names (C_BPartner_ID) in user-facing text - convert to human-readable (Business Partner)
• Description and Help fields must use different vocabulary
• Use complete sentences for Help text
• Follow abbreviation standards: MFA, RFQ, FA, BOM, UOM, PO, SO, GL, AP, AR, etc.

Implementation Plan

Phase 1: CLI Foundation

1. Create ElementDescriptionCommand with subcommands
2. Implement PatternGenerator service for rule-based generation
3. Add JSON/SQL/2Pack output formatters

Phase 2: Python Integration

1. Bundle Python scripts as resources
2. Create PythonBridge utility for script execution
3. Implement seed database parsing via Python

Phase 3: AI Agent Integration

1. Configure idempiere-description-writer agent
2. Add --ai flag to leverage agent for complex cases
3. Implement review workflow for AI-generated content

Phase 4: Java Native (Future)

1. Port parse_seed_dump.py to Java
2. Port generate_descriptions.py patterns to Java
3. Deprecate Python dependencies

More Information

Original Python Scripts

The following Python scripts from the iDempiere repository (.claude/tools/element-docs/) define the original implementation logic:

Data Extraction Scripts

Description Generation Scripts

Migration Scripts

Process/Tab Scripts

Related ADRs

• ADR-001 - iDempiere Documentation Standards
• ADR-008 - Application Dictionary Registry
• ADR-013 - LangChain4j Workflow Routing

References

• iDempiere Application Dictionary
• AD_Element Table
• Translation Export/Import
• 2Pack

Appendix A: Python Script Reference

parse_seed_dump.py

Extracts AD_Element context from PostgreSQL seed database dump:

#!/usr/bin/env python3
"""
Parse iDempiere seed database dump to extract AD_Element context and relationships.
Outputs JSON with element -> windows, processes, tables mappings.
"""

import json
import sys
from collections import defaultdict

def parse_copy_block(lines, start_line, columns):
    """Parse a COPY block from PostgreSQL dump, extracting specified columns."""
    data = []
    col_names = columns.split(', ')

    for i in range(start_line, len(lines)):
        line = lines[i]
        if line.strip() == '\\.':
            break
        if line.startswith('COPY ') or not line.strip():
            continue

        # Tab-separated values
        values = line.rstrip('\n').split('\t')
        if len(values) >= len(col_names):
            row = {}
            for j, col in enumerate(col_names):
                val = values[j] if j < len(values) else ''
                row[col] = val if val != '\\N' else None
            data.append(row)

    return data

# Tables parsed:
# - ad_element: Core element definitions
# - ad_column: Column -> element mapping
# - ad_field: Field -> column -> window mapping
# - ad_tab: Tab -> window mapping
# - ad_window: Window names
# - ad_process_para: Process parameter -> element mapping
# - ad_process: Process names
# - ad_table: Table names

# Output: element_context.json with structure:
# {
#   "element_id": {
#     "id": "...",
#     "uu": "...",
#     "columnname": "...",
#     "name": "...",
#     "description": "...",
#     "help": "...",
#     "entitytype": "D",
#     "windows": ["Window1", "Window2"],
#     "processes": ["Process1"],
#     "tables": ["Table1"]
#   }
# }

generate_descriptions.py

Pattern-based description generation:

#!/usr/bin/env python3
"""
Generate descriptions and help text for AD_Element records with missing documentation.
Uses pattern matching and context analysis.
"""

# Key patterns:
# - *_UU: "Immutable Universally Unique Identifier"
# - *_ID: "{Entity} reference"
# - Is*: "Indicates if {condition}"
# - Qty*: "Quantity of {subject}"
# - Amt*: "Amount for {purpose}"
# - Date*: "Date of {event}"
# - *_Acct: "Account for {purpose}"

# Standard columns dictionary with fixed descriptions
STANDARD_COLUMNS = {
    'Created': 'Date this record was created',
    'Updated': 'Date this record was last updated',
    'CreatedBy': 'User who created this record',
    'UpdatedBy': 'User who last updated this record',
    'IsActive': 'The record is active in the system',
    'Processing': 'Process is currently executing',
    'Processed': 'The document has been processed',
    'Posted': 'Accounting entries have been generated',
    'Value': 'Search key for the record',
    'Name': 'Alphanumeric identifier of the entity',
    'Description': 'Optional short description of the record',
    'DocumentNo': 'Document sequence number of the document',
    'SeqNo': 'Method of ordering records; lowest number comes first',
}

# Categorization by business domain:
# - System - Application Dictionary (AD_*)
# - Business Partner (C_BP*, C_BPartner*)
# - Material Management (M_*)
# - Sales and Purchasing (C_Order*, C_Invoice*)
# - Accounting (GL_*, *_Acct)
# - Manufacturing (PP_*)
# - Human Resources (HR_*)
# - Assets (A_*)
# - Performance Analysis (PA_*)
# - UU Identifiers (*_UU)

integrate_java_context.py

Enhances descriptions with Java business logic:

#!/usr/bin/env python3
"""
Integrate Java business logic into AD_Element descriptions.
Combines pattern-based generation with actual code behavior.
"""

# Business logic patterns detected:
# - "Cannot modify when parent is processed" -> "Read-only after processing"
# - "Triggers recalculation when changed" -> "Triggers recalculation"
# - "Calculated by multiplication/addition" -> "Calculated value"
# - "Currency converted value" -> "Currency-converted amount"
# - "Validation: FillMandatory" -> "Required field"

# Help text enhancement includes:
# - Business logic context from M*.java
# - Model class references for complex logic
# - Window usage context
# - Process usage context

create_final_migration.py

Generates iDempiere-compatible migration scripts:

#!/usr/bin/env python3
"""
Create final consolidated migration script for AD_Element descriptions.
Generates PostgreSQL and Oracle compatible scripts.
"""

# Output paths:
# - migration/iD12/postgresql/{timestamp}_IDEMPIERE-Element-Descriptions.sql
# - migration/iD12/oracle/{timestamp}_IDEMPIERE-Element-Descriptions.sql

# Script format:
# SELECT register_migration_script('{timestamp}_IDEMPIERE-Element-Descriptions.sql') FROM dual;
#
# UPDATE AD_Element SET Description='...', Help='...', Updated=NOW(), UpdatedBy=100
# WHERE AD_Element_UU='...' AND Description IS NULL AND EntityType='D';

# Oracle variant uses SYSDATE instead of NOW()

Appendix B: Documentation Standards (ADR-001 Summary)

Language Rules

1. Product naming: Always "iDempiere", never "Compiere" or "ADempiere"
2. Abbreviations: Uppercase (MFA, RFQ, FA, BOM, UOM, PO, SO, GL, AP, AR, API, PDF, XML, URL)
3. No technical details: Never expose class names, method names, or implementation specifics
4. Human-readable names: Convert column names to readable form (C_BPartner_ID → Business Partner)

Grammar Rules

1. Complete sentences: All Help text must use complete sentences
2. Article usage: "the" for specific, "a/an" for general
3. Singular/plural: Consistent throughout sentences
4. Possessive forms: record's (singular), records' (plural ending in s)

Field-Specific Templates