Workflow logging

Description

The Workflow Logging transform receives its input from a Workflow Log metadata item.

The transform requires very little configuration, but provides a lot of information.

Supported Engines

Hop Engine

Supported

Spark

Maybe Supported

Flink

Maybe Supported

Dataflow

Maybe Supported

Related Plugins

Workflow Log

Options

Option	Description
Transform name	Name of this transform
Also log action details	checked: the transform outputs both workflow and action logging and metrics. In this case, the logging will contain one line per action. For each action, the workflow logging and metrics information is repeated. unchecked: the transform only outputs workflow logging and metrics.

Option

Description

Transform name

Name of this transform

Also log action details

checked: the transform outputs both workflow and action logging and metrics. In this case, the logging will contain one line per action. For each action, the workflow logging and metrics information is repeated.
unchecked: the transform only outputs workflow logging and metrics.

Output Fields

Fieldname	Type	Description
Workflow Logging Fields
loggingDate	Date	the date and time this workflow was executed
loggingPhase	String	the phase this workflow is in at the time of logging (e.g. 'end')
workflowName	String	the logged workflow name
workflowFilename	String	the logged workflow file name
workflowStart	Date	execution start date for the logged workflow
workflowEnd	Date	execution end date for the logged workflow
workflowLogChannelId	String	logging channel id for the logged workflow
parentLogChannelId	String	parent logging channel id for the logged workflow (e.g. the channel id of the parent workflow)
workflowLogging	String	full logging text for the logged workflow
workflowErrorCount	Integer	number of errors that occurred in the logged workflow execution
workflowStatusDescription	String	status description for the logged workflow (e.g. 'Finished')
Action Logging Fields
actionName	String	the name of the action in the logged pipeline
actionNr	Integer	the number of the current action
actionResult	Boolean	the result (exit code, Y/N) of the current action
actionLogChannelId	String	log channel id for the current action
actionLoggingText	String	logging text for the current action
actionErrors	Integer	number of errors for the current action
actionLogDate	String	status of the current action (e.g. 'Finished')
actionDuration	Integer	total duration for the current action
actionExitStatus	Integer	exit status of the current action
actionNrFilesRetrieved	Integer	the number of files retrieved by the current transform (if applicable)
actionFilename	String	the filename to use for the current action, e.g. the workflow or pipeline the current action runs.
actionComment	String	an optional comment added to the logs, e.g. 'Workflow execution finished'
actionReason	String	an optional reason provided by the current action

Fieldname

Type

Description

Workflow Logging Fields

loggingDate

Date

the date and time this workflow was executed

loggingPhase

String

the phase this workflow is in at the time of logging (e.g. 'end')

workflowName

String

the logged workflow name

workflowFilename

String

the logged workflow file name

workflowStart

Date

execution start date for the logged workflow

workflowEnd

Date

execution end date for the logged workflow

workflowLogChannelId

String

logging channel id for the logged workflow

parentLogChannelId

String

parent logging channel id for the logged workflow (e.g. the channel id of the parent workflow)

workflowLogging

String

full logging text for the logged workflow

workflowErrorCount

Integer

number of errors that occurred in the logged workflow execution

workflowStatusDescription

String

status description for the logged workflow (e.g. 'Finished')

Action Logging Fields

actionName

String

the name of the action in the logged pipeline

actionNr

Integer

the number of the current action

actionResult

Boolean

the result (exit code, Y/N) of the current action

actionLogChannelId

String

log channel id for the current action

actionLoggingText

String

logging text for the current action

actionErrors

Integer

number of errors for the current action

actionLogDate

String

status of the current action (e.g. 'Finished')

actionDuration

Integer

total duration for the current action

actionExitStatus

Integer

exit status of the current action

actionNrFilesRetrieved

Integer

the number of files retrieved by the current transform (if applicable)

actionFilename

String

the filename to use for the current action, e.g. the workflow or pipeline the current action runs.

actionComment

String

an optional comment added to the logs, e.g. 'Workflow execution finished'

actionReason

String

an optional reason provided by the current action