Multi Mapping transform Icon Multi Mapping

Description

The Multi Mapping transform re-uses a child pipeline (a mapping) with zero or more inputs and outputs.

Unlike Simple Mapping, which always has one Mapping input and one Mapping output, Multi Mapping can:

  • start a child that generates its own data (0 inputs)

  • run a child that only writes data (0 outputs)

  • join or look up using several Mapping Input transforms

  • split rows to several Mapping Output transforms

Non-main input mappings become info hops (the same idea as Stream Lookup). Non-main output mappings become target hops (the same idea as Switch / Case).

That lets you wrap a reusable Stream Lookup or Switch / Case style pipeline.

The child pipeline is always executed with a local Hop pipeline run configuration so rows can be streamed in and out of the mapping.

Supported Engines

Hop Engine

Supported

Single Threaded

Supported

Native Spark

Supported

Beam Spark

Supported

Beam Flink

Supported

Beam Dataflow

Supported

Options

  • Transform name: a unique name in your pipeline

  • Pipeline: the mapping (sub-) pipeline to run

  • Run configuration: local Hop run configuration only

  • Parameters tab: variables to pass into the child, plus inherit-all-variables

  • Input tabs (Add Input): one tab per input mapping

    • Input source transform: parent transform that sends rows (empty = auto-detect remaining hops)

    • Mapping Input transform: child Mapping Input that receives those rows

    • Main data path: checked for the primary stream; unchecked for an info hop

    • Update mapped field names downstream: reverse input renames on the way out

    • Field mapping table

  • Output tabs (Add Output): one tab per output mapping

    • Mapping Output transform: child Mapping Output that produces rows

    • Output target transform: parent transform that receives those rows (empty = auto-detect remaining hops)

    • Main data path: checked for the primary stream; unchecked for a target hop

    • Field mapping table

You can close every input or output tab. A mapping with no inputs is a generator. A mapping with no outputs is a sink.

Sample

The samples project contains parent/child examples under transforms/ for a single stream, a generator, a join, an info lookup, and multiple target outputs.