AI

Apache Hop can advise on pipeline and workflow design through a pluggable AI layer. Completions go through the Language Model Chat transform (hop-transform-languagemodelchat) and langchain4j. No language-model SDK is on the Hop core classpath.

The technology plugin is hop-tech-ai (plugins/tech/ai). AI is disabled until you turn it on.

What ships

  • AI Provider metadata — named model + credentials

  • Configuration perspective → Plugins → AI Assistant — master enable, default provider, extra context notes, context files (none by default), optional full XML, what happens to a running question when its window closes

  • AI Assistant workbench — perspective, floating window, and bottom dock

  • The pipeline and workflow assistants — toolbar and context-menu entry points, including review-and-apply hop_proposals graph edits, clipboard XML/JSON, and metadata saves

  • Language Model Chat can optionally use a named AI Provider for connection fields; inline keys remain as fallback

Privacy

AI is off until you turn it on. A prompt that leaves your machine is billed and may be retained by the provider: send the minimum needed to answer the question.

Keep the project in Git (or similar) and create a feature branch before applying AI-driven changes. Graph Undo is not a substitute: metadata saves and pasted transforms or actions cannot be reverted that way. See Version control on the AI Assistant page.

  • Do not hard-code secrets or personal information in pipelines, workflows, metadata, notes, SQL or sample rows. Put them in environment configuration files as variables, or better in a keystore read with a variable resolver. See Secrets and personal information and Best practices: security.

  • API keys stay on AI Provider metadata and are never copied into advisor prompts. Still prefer ${AI_API_KEY} or a resolver expression instead of a pasted key.

  • Prompts mask every value Hop stores encrypted (password fields), fields named like secrets (password, passphrase, secret key, token, API key, …) and passwords in URLs. That does not catch literals in SQL, notes or logs.

  • Session context inclusion checkboxes control extra payload (transform settings, check results, plugin catalog, execution log, full XML, selected metadata). Use only what the question needs: Context inclusions.

  • Extra context notes and listed context files are sent on every turn as instructions. No context files are listed by default; only add files you wrote for the assistant, and keep secrets out of them.

  • Assistant (advisor) plugins may also ship ai-context.md next to the plugin JAR (Hop’s AI plugin does). Those notes are always sent for that assistant.

  • Full pipeline or workflow XML is not sent unless both the session checkbox and Configuration → Allow sending full XML are on.

  • Delete plugins/tech/ai to remove the GUI, metadata type and advisors.

  • Hide individual widgets with disabledGuiElements.xml.

Extending

Two plugin types are registered from HopEnvironment:

  • @AiProviderPlugin / IAiProvider — another backend (including a future OAuth Copilot plugin)

  • @AiAdvisorPlugin / IAiAdvisor — another advisor. Location ids are free-form strings (pipeline graph, Data Vault graph, lineage view, a metadata editor, …). hopper-edw adds Data Vault, Business Vault, dimensional and lineage advisors this way, with their own toolbar and context-menu plugins.

Keep langchain4j and SWT off the IAiAdvisor / IAiProvider interfaces; those live in hop-core. Third-party advisor plugins omit classLoaderGroup and compile against hop-core + hop-ui only. Plugin authors: see the AI advisor plugins page in the development manual.

Keys that older plugins stored under Configuration → AI Assistant (hopAiConfig with aiApiKey / aiProviderPreset) are imported as an AI Provider the first time the GUI starts with a metadata provider, then removed from hop-config.json.