Beam Output Icon Beam Output

Description

The Beam Output transform writes files using a file definition with the Beam execution engine.

Supported Engines

Hop Engine

Not Supported

Spark

Supported

Flink

Supported

Dataflow

Supported

Options

Option Description

Transform name

Name of the transform, this name has to be unique in a single pipeline.

Output location

The output location.

File prefix

Text which has to precede the file name.

File suffix

Text which has to succeed the file name.

Windowed writes?

Writes a file per window. Use together with the Beam Window transform.

File definition to use

The file definition to use stored in a metastore.

Edit

Edits an existing file definition.

New

Creates a new file definition.

Manage

Opens the MetaStore Explorer.

File Definition

A file definition can be used to define a file structure definition.

Option Description

Name

The file definition name.

Description

The file definition description.

Field separator

The character separating the fields.

Field enclosure

The character enclosing fields.

Field definitions

A list of fields.

Field name

The field name.

Type

The field data type.

Format

The field format.

Length

The field length.

Precision

The field precisiom.