Run

Overview

Transforms are executed via the RUN(run_options) function, deployed in the deployment_dataset_id dataset. This function contains all execution logic and should not be edited.

Arguments

The run_options JSON object enables control over the run_mode at execution time. This object is used to set the following options:

Mode	Type	Description
`run_mode`	`STRING`	Determines the mode of the transformation run. Valid values are `auto`, `full`, `incremental`, `range`, `first` and `last`.
`partition_detection`	`BOOL`	Determines whether partition modifications automatically trigger dependant downstream transformations.
`start_date`	`STRING`	Start date of the partition range to transform for the `range` run mode.
`end_date`	`STRING`	End date of the partition range to transform for the `range` run mode.
`window_days`	`STRING`	Length of window in days for the `first` and `last` run modes, and the number of recent days to transform for the `incremental` run mode when new data is detected.

Run Mode (run_mode)

`auto`

(Default) Transforms any partitions detected in the source tables which are not detected in the destination tables.

Transforms any partitions detected in the source tables which are not detected after the latest partition in the destination tables. If the window_days is set, the latest n date partitions are always refreshed when new data is detected.

`range`

Transforms all partitions between the start_date and end_date.

`first`

Transforms the first n partitions, set by window_days.

`last`

Transforms the last n partitions, set by window_days.

Partition Detection (partition_detection)

Setting the partition_detection boolean as true means that any modifications to upstream partitions detected will trigger an automatic refresh of dependant downstream partitions. The default configuration is to set "partition_detection": true with "run_mode": "auto" for a fully automated, efficient transformation pipeline.

Mode	Description
`auto`
`full`
`incremental`
`range`
`first`
`last`

`run_mode=auto`

Argument	Value	Options

Run