Row Denormaliser
Description
The De-normalizer transform allows you de-normalize data by looking up key-value pairs, with the option to convert data types in the process.
Note: make sure to check the notes on this transform in the Getting started with Beam documentation.
Options
| Transform name | Name of the transform. This name has to be unique in a single pipeline. |
|---|---|
Key field | The field that defined the key of the output row. |
Group fields | Specify the fields that make up the grouping here. |
Target fields | Select the fields to de-normalize by specifying the String value for the key field (see above). Options are provided to convert data types. Strings are most common as key-value pairs so you must often convert to Integer, Number or Date. If you get key-value pair collisions (key is not unique for the group specified) specify the aggregation method to use. |
Metadata Injection Support
You can use the Metadata Injection supported fields with ETL Metadata Injection transform to pass metadata to your pipeline at runtime. All fields can be injected, the values used for the aggregation field are the following
key | value |
TYPE_AGGR_NONE | No Aggregation is done |
TYPE_AGGR_SUM | Sum all values |
TYPE_AGGR_AVERAGE | Calculate the average |
TYPE_AGGR_MIN | Take the minimal value of the group |
TYPE_AGGR_MAX | Take the maximum value of the group |
TYPE_AGGR_COUNT_ALL | Count rows |
TYPE_AGGR_CONCAT_COMMA | Aggragate values separated by comma |
