Repeat Fields
Group fields
In the Group fields section you can specify the group in which to operate. Every new group starts without a previous row.
The fields to repeat
| Option | Description |
|---|---|
Repeat type | Previous row: repeat the field value from the previous row, regardless of its current value. If there is no previous row, take the value from the current row. (ETL Metadata Injection type = Previous when field is null: copy the field value from the previous row if the current source value is null. (ETL Metadata Injection type = Current when indicated: take the value from the current source field if the given indicator matches the value in the indicator field. (ETL Metadata Injection type = |
Source field | The source field to repeat |
Target field | Specify the name of the target field (mandatory) |
Indicator field name | When using the "Current when indicated" type, this is the name of the field that contains the indicator value. |
Indicator value | When using the "Current when indicated" type, this is the indicator value to match with. |
Current when indicated
When we’re dealing with source data coming from multiple sources, for example information about a customer, we want to assemble a record containing all the different fields from the different sources. In the example of our customer, source system A might contain personal data like name, first name, birthdate, and so on. Another source system B might contain the status of the customer. Finally, source system C contains a flag indicating financial status. Each of the sources get changed at a different point in time. In a slowly changing dimension (see: Dimension Lookup/Update ) we want to see the changes when they occur in the different sources. We also want to process the data from the sources as they arrive.
| source | customer_id | timestamp | lastname | firstname | birthdate | status | indicator |
|---|---|---|---|---|---|---|---|
A | 1 | 2025/01/01 12:00:00 | Mouse | Mickey | 1928/11/18 | null | null |
B | 1 | 2025/01/01 13:00:00 | null | null | null | active | null |
C | 1 | 2025/01/01 14:00:00 | null | null | null | null | positive |
What we want to do with this transform is take the appropriate fields from the source data taking into account the source field as an indicator. We want to end up with this result:
| source | customer_id | timestamp | lastname | firstname | birthdate | status | indicator |
|---|---|---|---|---|---|---|---|
A | 1 | 2025/01/01 12:00:00 | Mouse | Mickey | 1928/11/18 | null | null |
B | 1 | 2025/01/01 13:00:00 | Mouse | Mickey | 1928/11/18 | active | null |
C | 1 | 2025/01/01 14:00:00 | Mouse | Mickey | 1928/11/18 | active | positive |
These 3 rows can then be used to create a detailed and correct timeline using the Dimension Lookup/Update transform.
| Add the last record from your target slowly changing dimension to modify existing records. |
| As mentioned above, but worth repeating, make sure your data is sorted on the group key (the customer ID in this example) and ALSO on the timestamp to get a correct timeline result. |