JSON Normalize Input

A record JsonPath selects an array of objects (or scalar elements). Hop emits one output row per array element.

Nested objects under each record are flattened into column paths using a configurable separator (default: a dot, like customer.name for nested keys).

On the Fields tab, each row’s Path is the flattened key (for example customer.name), not a separate JsonPath query into the raw file.

Sibling arrays — One file may hold unrelated lists (for example, in hop-config.json: variables, lifecycleEnvironments, and projectConfigurations). Each list has its own element shape; a single record path cannot return two arrays at once. Normalize each list with its own transform (and JsonPath such as $.variables[*]), or read the file once into string columns and parse those blobs in follow-up steps—see the json-normalize-hop-config-layers.hpl sample.

Wide, sparse tables — It is possible in some cases to point the record path at a very high level (for example the document root) and flatten everything into one enormous row, but unrelated branches then become hundreds of columns with values only on a few of them. That is hard to query and heavy for memory. Prefer a narrow record path per entity type so each hop looks like a real table: mostly dense columns, one business meaning per row.

Wide output — Many distinct paths (especially long field names or deep nesting) mean many columns and larger in-memory row batches. Very large config files with hundreds of top-level branches can create very wide rows if you flatten the whole document in one row.

Large files — Reading big JSON documents is not streaming per record at the JsonPath layer; the engine must materialize matching structures. Prefer narrow Record JsonPath (one list of business records) instead of normalizing an enormous root when possible.

Arrays left as JSON text — Stringified array columns can be large strings; downstream steps (for example filters, calculators) pay the cost of parsing or copying that text.

transforms/json-normalize-orders.hpl with data/json-normalize-orders.json — nested orders array.

transforms/json-normalize-root-array.hpl with data/json-normalize-events.json — root-level array.

transforms/json-normalize-complex-transactions.hpl with data/json-normalize-complex-transactions.json — larger file with many columns and rows.

transforms/json-normalize-hop-config-layers.hpl reads ${HOP_CONFIG_FOLDER}/hop-config.json — wide row from the root object (arrays as JSON text), then parallel branches that normalize variables, lifecycle environments, and project configurations from those strings. Point HOP_CONFIG_FOLDER at your real Hop config folder, or use the fixture under integration-tests/json/resources/hop-config-layers-it/ to try the shape offline.