Get records from stream transform Icon Get records from stream

Description

The Get Records From Stream transform returns records that were previously generated by another pipeline in a workflow.

The records were passed to this transform using either the Copy rows to result transform or the Workflow Executor transform.

You can enter the metadata of the fields you are expecting from the previous pipeline in a workflow.

Supported Engines

Hop Engine

Supported

Spark

Maybe Supported

Flink

Maybe Supported

Dataflow

Maybe Supported

Options

Option Description

Transform name

Name of the transform.

Fieldname

Name of the field that contains the records from the previous result.

Type

Type of data. Options include:

  • Number

  • String

  • Date

  • Boolean

  • Integer

  • BigNumber

  • Serializable

  • Binary

  • Timestamp

  • Internet Address

Length

  • For a field with the type of 'Number', enter the total number of significant figures in a number.

  • For a field with the type of 'String', enter the length of the string.

Precision

For a field with the type of 'Number', enter the number of floating point digits. Otherwise, leave blank.

Metadata Injection Support

All fields of this transform support metadata injection. You can use this transform with ETL Metadata Injection to pass metadata to your pipeline at runtime.