Google Technology

Introduction

The Google technology stack is supported in Hop through a number of plugins. We briefly touch upon them below.

Pipeline Transforms

Check the external plugins for more information on a number of additional Google plugins that can’t be or won’t be included with Apache Hop (Google Sheets Input and Output, Google Analytics)

VFS

Apache VFS Support in Hop allows you to directly read from a multitude of file systems and protocols, including Google:

  • Google Drive: read and write data directly from and to Google Drive files and folders.

  • Google Cloud Storage: read and write data directly from and to files and folders in Google Cloud Storage buckets

Beam vs Google Cloud

When executing your pipeline using a Beam runner which is NOT DataFlow, make sure to pass the default Google cloud project ID by running:

gcloud config set project <project-id>

This affects Google Cloud specific APIs like BigQuery, Pub/Sub and others.