Logging
Every workflow or pipeline you execute through Hop Gui or Hop Run and a lot of the other tools generate logging as logging text in the most obvious form.
Hop allows data developers to influence how, where and at which level of detail logging information is generated and where it is written to.
Levels
Descriptions
The amount of text logging generated depends on the logging level which is used when executing a workflow or pipeline:
| Level | Description |
|---|---|
NOTHING | No logging is generated |
ERROR | Only errors are reported |
MINIMAL | The bare essential, nothing more |
BASIC | The standard logging which tries to be concise and informative |
DETAILED | More logging information about what happens behind the scenes is reported |
DEBUG | A lot of information is produced, often with specific values being reported |
ROWLEVEL | Values are reported on an individual row level |
Runtime levels
The logging level can be specified when executing using hop-run or in the Hop GUI. It is also possible to use the debug plugin to set custom logging levels on a transform:
You can set a custom logging level for this particular transform and only for certain rows of data:
You can do likewise for a workflow:
Hop GUI logging
The logging file of the Hop GUI is saved in the audit/ folder (or in the folder set by HOP_AUDIT_FOLDER) in the hopui.log file.
Tools logging
If you want to send logging of tools like hop-run or hop-conf to a logging file you need to do this simply by piping the text stream to a file.
Logging plugins
Actions
Write to log
The Write to log acction writes a specific string to the Hop logging system.
Check the Write To Log page for more details.
Metadata Types
Pipeline Log
A pipeline log allows to log the activity of a pipeline with another pipeline.
Check the pipeline log and logging reflection pages for more details.
Workflow Log
Allows to log the activity of a workflow with a pipeline.
Check the workflow log and logging reflection pages for more details.
Perspectives
Neo4j
Hop can write execution logs for workflows and pipelines to a Neo4j database.
Check Neo4j Perspective page for more details.
Transforms
Write to Log
This transform writes information to the Hop logging system.
Check the Write To Log page for more details.
External plugins
The external hop-plugins repository contains the Send information using syslog action and Send message to syslog transforms.