Apache Hop
Download
Development Documentation 2.20.0

User manual

Edit this page

VFS Plugins

A VFS plugin teaches Apache Hop how to communicate with a new storage system (such as Amazon S3, Google Cloud Storage, Azure Blob, SFTP, or WebDAV) by registering custom URL schemes with the shared Apache Commons VFS manager.

Implementing a VFS Plugin

Annotate the class with @VfsPlugin and implement IVfs:

java
import org.apache.commons.vfs2.provider.FileProvider;
import org.apache.hop.core.variables.IVariables;
import org.apache.hop.core.vfs.plugin.IVfs;
import org.apache.hop.core.vfs.plugin.VfsPlugin;
import java.util.Map;

@VfsPlugin(
    type = "customcloud",
    typeDescription = "Custom Cloud Storage VFS Provider",
    classLoaderGroup = "vfs-customcloud"
)
public class CustomCloudVfsPlugin implements IVfs {

  @Override
  public String[] getUrlSchemes() {
    // Return all URL prefixes claimed by this provider:
    return new String[] {"customcloud", "ccfs"};
  }

  @Override
  public FileProvider getProvider() {
    // Return standard Commons VFS FileProvider implementation:
    return new CustomCloudFileProvider();
  }

  @Override
  public Map<String, FileProvider> getProviders(IVariables variables) {
    // Return named providers based on configured metadata objects:
    // e.g. "my-connection://path/to/file"
    return loadNamedConnectionProviders(variables);
  }
}

Key Architectural Rules

  • Named Connections as URL Schemes: Through getProviders(IVariables), each named metadata connection (such as an Azure or S3 connection object) registers as its own global scheme (my-connection://path).

  • classLoaderGroup is Mandatory:

Always specify classLoaderGroup on @VfsPlugin matching your metadata connection classes and GUI editor plugins. Because named schemes register in a shared process-wide Commons VFS manager, mismatched classloaders will result in ClassCastException when casting configuration metadata.

Concrete Codebase Examples

  • plugins/tech/azure/src/main/java/org/apache/hop/vfs/azure/AzureVfsPlugin.java: Azure Blob and Data Lake Gen2 VFS provider.

  • plugins/tech/google/src/main/java/org/apache/hop/vfs/gs/GoogleStorageVfsPlugin.java: Google Cloud Storage (GCS) provider.

  • plugins/tech/sftp/src/main/java/org/apache/hop/vfs/sftp/SftpVfsPlugin.java: SFTP file system provider.

See also