Skip to main content

Google: Big query

Interact with Google Big Query.

Pre-requisite: Install Google Big Query application Profile > {Organization} > Applications to grant Zparse access.

Parameters​

Accountrequired

Select your previously configured Google Big Query application (see pre-requisite)

Project IDrequired

Add the Project ID you want to interact with

Dataset IDrequired

Add the Dataset ID you want to interact with

Endpointrequired

Either select Query Select if you want to retrieve data from your tables or Streaming Insert to publish data to your tables. See below for additional configuration based on selected Endpoint.

Endpoint: Select Query​

Parameters​

Queryrequired

Which SQL query you want to get your data from.

Example: Get data from public big query dataset.
SELECT *  FROM `bigquery-public-data.austin_crime.crime`
Example: Get only first N elements
SELECT *  FROM `bigquery-public-data.austin_crime.crime` LIMIT @limit

Query accept dynamic parameters using @ marker. (ex: @limit)

Query parameters

If you add dynamic parameters in your query, here is where you can create module connector to dynamically inject it.

Parameter: limit of type integer

Input​

Trigger

If you have no dynamic parameters simply trigger the query

{DYNAMICS BASED ON QUERY PARAMETERS}

If you have dynamics parameters simply connect proper objects

Output​

JSONL File
A JSONL file with one row per line.

Endpoint: Streaming Insert​

Parameters​

Tablerequired

On which table you want to insert your data

Input​

JSONL Filerequired

A JSONL file containing one row per line.

Output​

Row inserted
The number of rows which were inserted.