Add Sequence
Description
The Add Sequence transform adds a sequence to the Hop stream. A sequence is an ever-changing integer value with a specific start and increment value.
You can either use a database sequence (if supported, e.g. Oracle, PostgreSQL) to determine the value of the sequence, or have it generated by Hop.
Hop-generated sequence values are unique when used in the same pipeline, and return to the same starting value each time you run the pipeline. Hop sequences are unique only when used in the same pipeline. Also, they are not stored, so the values start back at the same value every time the pipeline is launched.
The start, increment, and maximum can be typed in, or read from another transform. Connect that transform to Add Sequence and select it as the configuration transform. It must return exactly one row. More than one row stops the pipeline with an error. The hop is an info hop, so that row is not part of the main stream. The end field is the maximum value: after it, the sequence returns to the start value. The configuration transform cannot run in multiple copies. If Add Sequence itself runs in multiple copies, copy the configuration row to every copy.
On the Beam engines
With a typed start, increment, and maximum on a bounded input, the Beam engines produce a single sequence across the whole pipeline, with no duplicates, the same as the local engine. To achieve this every row is keyed onto one key, so the transform runs at a parallelism of one: that is the trade-off for a globally increasing value.
A database sequence, a configuration transform, or an unbounded input (for example Kafka or MQTT) runs the transform on each worker instead. A database sequence stays unique, because the database hands out the values. With a configuration transform or an unbounded input each worker keeps its own counter, so the values are not unique across the pipeline.
Options
| Option | Description |
|---|---|
Transform name |
The name of this transform as it appears in the pipeline workspace. This name must be unique within a single pipeline. |
Name of value |
Name of the new sequence value that is added to the stream. |
Use DB to generate the sequence |
Enable if you want the sequence to be driven by a database sequence, then set these parameters: Connection name, Schema name (optional), Sequence name. |
Connection name |
The name of the connection on which the database sequence resides. |
Schema name (optional) |
The table’s schema name. |
Sequence name |
The name of the database sequence. |
Use a pipeline counter to generate the sequence |
Enable if you want the sequence to be generated by Hop, then set these parameters: Counter name (optional), Start at, Increment by, Maximum value. |
Counter name (optional) |
If multiple transforms in a pipeline generate the same value name, this option enables you to specify the name of the counter to associate with. Avoids forcing unique sequencing across multiple transforms. |
Start at |
The value to begin the sequence with. Not used when a configuration transform is selected. |
Increment by |
The amount by which the sequence increases or decreases. Not used when a configuration transform is selected. |
Maximum value |
The value after which the sequence returns to the Start At value. Not used when a configuration transform is selected. |
Configuration transform |
Optional transform that provides exactly one row with the start, end, and increment. Leave this empty to use Start at, Increment by, and Maximum value. |
Start field |
Field in the configuration row that contains the value to begin the sequence with. |
End field |
Field in the configuration row that contains the maximum value. After this value the sequence returns to the start value. |
Increment field |
Field in the configuration row that contains the amount by which the sequence increases or decreases. |