Row Normaliser

The Row Normaliser transform converts the columns of an input stream into rows.

You can use this transform to normalize repeating groups of columns.

For every row it reads, the transform writes one row per type. Each input field goes into the new field it names on the row of its type, whatever order the fields are listed in. A type that has no field for one of the new fields leaves that field empty (null) on its rows.

A type can fill each new field from only one input field: listing two fields with the same type and the same new field is an error.

When the Fields table is empty, the transform normalises every field it receives. Each input field is a type of its own, named after the field, and its value goes into the Value field. This works when the input fields are only known when the pipeline runs, for example in a pipeline that profiles any file or table it is given.

Every input field is normalised this way, so no field is passed on unchanged: the output has only the type field and the value field. When the input fields have different data types the value field is a String, as described below. The value field keeps a length or precision only when all input fields have the same one. While you design a pipeline whose input fields are not known yet, the transform shows the value field to the transforms after it as a String.

A new field is filled from a different input field on every row, so its data type depends on all of them:

For example, normalising a String, an Integer and a Number into one field gives a String field holding a, 1 and 1.2. Verify on this transform warns about every new field that is filled from input fields of different data types.