Google: Big query

Interact with Google Big Query.

Pre-requisite: Install Google Big Query application Profile > {Organization} > Applications to grant Zparse access.

Parameters

Account—REQUIRED

Select your previously configured Google Big Query application (see pre-requisite)

Project ID—REQUIRED

Add the Project ID you want to interact with

Dataset ID—REQUIRED

Add the Dataset ID you want to interact with

Endpoint—REQUIRED

Either select Query Select if you want to retrieve data from your tables or Streaming Insert to publish data to your tables. See below for additional configuration based on selected Endpoint.

Endpoint: Select Query

Parameters

Query—REQUIRED

Which SQL query you want to get your data from.

Example: Get data from public big query dataset.

SELECT *  FROM `bigquery-public-data.austin_crime.crime`

Example: Get only first N elements

SELECT *  FROM `bigquery-public-data.austin_crime.crime` LIMIT @limit

Query accept dynamic parameters using @ marker. (ex: @limit)

Query parameters

If you add dynamic parameters in your query, here is where you can create module connector to dynamically inject it.

Parameter: limit of type integer

Input

Trigger

If you have no dynamic parameters simply trigger the query

{DYNAMICS BASED ON QUERY PARAMETERS}

If you have dynamics parameters simply connect proper objects

Output

JSONL File

A JSONL file with one row per line.

Endpoint: Streaming Insert

Parameters

Table—REQUIRED

On which table you want to insert your data

Input

JSONL File—REQUIRED

A JSONL file containing one row per line.

Output

Row inserted

The number of rows which were inserted.