Split Fields transform Icon Split Fields

Description

The Split Fields transform splits a field into multiple fields based on a specified delimiter.

Supported Engines

Hop Engine

Supported

Spark

Maybe Supported

Flink

Maybe Supported

Dataflow

Maybe Supported

Options

Option Description

Transform name

Name of the transform; this name has to be unique in a single pipeline

Field to split

The name of the field you want to split

Delimiter

Delimiter that determines the field. Special characters (e.g. CHAR ASCII HEX01) can be set with the format $[value], e.g. $[01] or $[6F,FF,00,1F].

Enclosure

You can specify an enclosure string which when placed around a value allows delimiters to be present in it. For example with a comma (,) delimiter: aaaa,"123,456",ccc can be resolved to 3 fields using enclosure ".

Escape string

To include delimiter characters in values sometimes an escape string is used like backslash, double backslash and so on. For example with a comma (,) delimiter: aaaa,123\\,456,ccc can be resolved to 3 fields using escape character \\.

Fields table

This table is where you define the properties for each new field created by the split. For each new field, you must define the field name, data type, and other properties.

Metadata Injection Support

All fields of this transform support metadata injection. You can use this transform with ETL Metadata Injection to pass metadata to your pipeline at runtime.

Examples

Below are examples of split fields:

Example 1

SALES_VALUES field containing: "500,300,200,100"

Use these settings to split the field into four new fields:

  • Delimiter: ,

  • Field: SALES1, SALES2, SALES3, SALES4

  • Id:

  • remove ID no, no, no, no

  • type: Number, Number, Number, Number

  • format: ., ., ., .

  • group:

  • decimal: .

  • currency:

  • length: 3, 3, 3, 3

  • precision: 0, 0, 0, 0

Example 2

SALES_VALUES field containing "Sales2=310.50, Sales4=150.23"

Use these settings to split the field into four new fields:

  • Delimiter: ,

  • Field: SALES1, SALES2, SALES3, SALES4

  • Id: Sales1=, Sales2=, Sales3=, Sales4=

  • remove ID yes, yes, yes, yes

  • type: Number, Number, Number, Number

  • format: ., ., ., .

  • group:

  • decimal: .

  • currency:

  • length: 7, 7, 7, 7

  • precision: 2, 2, 2, 2