Workflow logging

Description

the Workflow Logging transform receives its input from a Workflow Log metadata item.

The transform requires very little configuration, but provides a lot of information.

Supported Engines

Hop Engine

Supported

Spark

Maybe Supported

Flink

Maybe Supported

Dataflow

Maybe Supported

Related Plugins

Workflow Log

Options

Option	Description
Transform name	Name of this transform
Also log action details	checked: the transform outputs both workflow and action logging and metrics. In this case, the logging will contain one line per action. For each action, the workflow logging and metrics information is repeated. unchecked: the transform only outputs workflow logging and metrics.

Option

Description

Transform name

Name of this transform

Also log action details

checked: the transform outputs both workflow and action logging and metrics. In this case, the logging will contain one line per action. For each action, the workflow logging and metrics information is repeated.
unchecked: the transform only outputs workflow logging and metrics.

Output Fields

Fieldname Type Description

Fieldname	Type	Description
Workflow Logging Fields
loggingDate	Date	the date and time this workflow was executed
loggingPhase	String	the phase this workflow is in at the time of logging (e.g. 'end')
workflowName	String	the logged workflow name
workflowFilename	String	the logged workflow file name
workflowStart	Date	execution start date for the logged workflow
workflowEnd	Date	execution end date for the logged workflow
workflowLogChannelId	String	logging channel id for the logged workflow
parentLogChannelId	String	parent logging channel id for the logged workflow (e.g. the channel id of the parent workflow)
workflowLogging	String	full logging text for the logged workflow
workflowErrorCount	Integer	number of errors that occurred in the logged workflow execution
workflowStatusDescription	String	status description for the logged workflow (e.g. 'Finished')
Action Logging Fields
actionName	String	the name of the action in the logged pipeline
actionNr	Integer	the number of the current action
actionResult	the result (exit code, Y/N) of the current action	actionLogChannelId
String	log channel id for the current action	actionLoggingText
String	logging text for the current action	actionErrors
Integer	number of errors for the current action	actionLogDate
String	status of the current action (e.g. 'Finished')	actionDuration
Integer	total duration for the current action	actionExitStatus
exit status of the current action	actionNrFilesRetrieved	the number of files retrieved by the current transform (if applicable)
actionFilename	the filename to use for the current action, e.g. the workflow or pipeline the current action runs.	actionComment
an optional comment added to the logs, e.g. `Workflow execution finished`	actionReason	an optional reason provided by the current action

Workflow Logging Fields

loggingDate

Date

the date and time this workflow was executed

loggingPhase

String

the phase this workflow is in at the time of logging (e.g. 'end')

workflowName

String

the logged workflow name

workflowFilename

String

the logged workflow file name

workflowStart

Date

execution start date for the logged workflow

workflowEnd

Date

execution end date for the logged workflow

workflowLogChannelId

String

logging channel id for the logged workflow

parentLogChannelId

String

parent logging channel id for the logged workflow (e.g. the channel id of the parent workflow)

workflowLogging

String

full logging text for the logged workflow

workflowErrorCount

Integer

number of errors that occurred in the logged workflow execution

workflowStatusDescription

String

status description for the logged workflow (e.g. 'Finished')

Action Logging Fields

actionName

String

the name of the action in the logged pipeline

actionNr

Integer

the number of the current action

actionResult

the result (exit code, Y/N) of the current action

actionLogChannelId

String

log channel id for the current action

actionLoggingText

String

logging text for the current action

actionErrors

Integer

number of errors for the current action

actionLogDate

String

status of the current action (e.g. 'Finished')

actionDuration

Integer

total duration for the current action

actionExitStatus

exit status of the current action

actionNrFilesRetrieved

the number of files retrieved by the current transform (if applicable)

actionFilename

the filename to use for the current action, e.g. the workflow or pipeline the current action runs.

actionComment

an optional comment added to the logs, e.g. Workflow execution finished

actionReason

an optional reason provided by the current action