BigQuery input

Description

The BigQuery input transform reads data from a Google Cloud BigQuery table. It runs on the local Hop engine (using the BigQuery Java client) as well as every Beam engine.

Options

Option Description

Option	Description
Transform name	Name of the transform, this name has to be unique in a single pipeline.
Project ID	The Google Cloud project.
Data set ID	The BigQuery dataset ID.
Table ID	The BigQuery table ID.
Query	The input query, blank means everything from the table.
Query location	The BigQuery dataset location that runs the query, for example `EU` or `US`. Leave blank to use the location of the job itself. Applies to a query only.
Skip query validation	Do not validate the query before the job starts. Applies to a query only. Use this when the query is created or changed by an earlier step, and is therefore not registered when the job is submitted.
Return fields selection	A list of result fields.
BQ Field name	The field name in the BigQuery table.
Rename to… (optional)	The name to be given to a column.
Hop data type	The field data type.

Transform name

Name of the transform, this name has to be unique in a single pipeline.

Project ID

The Google Cloud project.

Data set ID

The BigQuery dataset ID.

Table ID

The BigQuery table ID.

Query

The input query, blank means everything from the table.

Query location

The BigQuery dataset location that runs the query, for example EU or US. Leave blank to use the location of the job itself. Applies to a query only.

Skip query validation

Do not validate the query before the job starts. Applies to a query only. Use this when the query is created or changed by an earlier step, and is therefore not registered when the job is submitted.

Return fields selection

A list of result fields.

BQ Field name

The field name in the BigQuery table.

Rename to… (optional)

The name to be given to a column.

Hop data type

The field data type.