Plugin samples

Introduction

This document will explain how a developer can add samples to the plugins in the source code so that they will end up in the samples project in the Hop client distribution assembly.

The Samples project

The samples project in the Hop distribution is located in the config/projects/samples folder. Its parent is the default project. This means that it’s not needed to add any metadata objects (like run configurations) to the samples projects that are already present in the default project.

Plugin samples

If a plugin places a samples/ folder under the src/main/ folder its content will be picked up and copied over the samples project folder. The content will end up exactly in the same folder structure. A few examples:

Source Target

plugins/transforms/addsequence/src/main/samples/transforms/add-sequence-add-a-unique-id.hpl

config/projects/samples/transforms/add-sequence-add-a-unique-id.hpl

plugins/engines/beam/src/main/samples/metadata/pipeline-run-configuration/Direct.json

config/projects/samples/metadata/pipeline-run-configuration/Direct.json

As you can see it’s easy to add extra metadata simply by putting it in the metadata/ folder under your samples/ folder in the plugin. The same goes for test data datasets and so on.

Standard project content

If you have metadata objects, sample input files data sets or files that you might want to make available for all plugins, please add them to the following place in the source code:

assemblies/static/src/main/resources/config/projects/samples

Naming standard

When naming samples and folders please stick to the following basic rules:

  • The filenames start with the name of the transform, action or plugin

  • All lowercase

  • No spaces in the filename or underscores: use dashes (-)

  • Put plugins in the right folders: transform samples to in transforms/, action samples go in actions/ and so on. Keep things clean.

  • If you need to add input files for your sample put them in a files/ sub-folder like transforms/files

  • Please use our best practices guide to create the examples.

  • Create notes in pipelines and workflows explaining what happens. Make the note font large enough: 11 or higher.

Set the HOP_LICENSE_HEADER_FILE project variable

All files that are shipped with Apache software need to contain the Apache Software Foundation’s header.

This also applies to all workflows and pipelines that are shipped as integration tests, samples or in any other form.

To spare contributors the pain of manually having to add the ASF license header in workflows and pipeline, Hop Gui supports the HOP_LICENSE_HEADER_FILE project variable.

Set this variable in your project and point it to a file containing the ASF Header’s text, e.g. integration-tests/asf-header.txt to let Hop include the ASF header in all of your workflows and pipelines.

Hop License Header File variable

As with any other piece of code, run mvn apache-rat:check to verify all files contain the correct headers before contributing your test or sample workflows and pipelines.