Quality

Quality is a query catalog, not a fifth data pool. Provider tables in providers hold the rows. Quality holds measurements about those tables: statistical process control (SPC), Gate 2 checks, shape stats, and catalog health. Address the latest snapshot across the workspace, or the history of one table:

Surface	Address	Use when
Latest, all provider tables	`providers_quality.summary.{table}`	Workspace-wide health, failing checks, SPC breaches.
History, one table	`providers_quality.{connector}.{table}__{suffix}`	How a check or series changed over runs.

Only main provider tables get quality sidecars. Identity tables (providers.identity.*), artifacts, and parables do not. Run the SQL below over Flight SQL. Clients are on Querying the lake.

What each measurement is

Checks (Gate 2) compare a table to thresholds from the tap’s quality config and schema roles. Each row is one check on one table. Status is PASS, FAIL, or SKIPPED. Skipped is not a pass — the rule lacked a required role (business key, event time, and so on) or window, so it did not evaluate. Business-key columns are x-transformDedupKey on the tap schema; see How taps project into tables.

Check ID	What it measures
`null_rate`	Required-column null rate vs `maxRate`.
`dupe_rate`	Duplicate rate on business-key columns vs `maxRate`.
`timestamp_coverage`	Event-time values inside the expected window vs `minRate`.
`temporal_order`	`effective_from` after `effective_to` vs `maxViolations`.
`no_future_timestamp`	Metadata timestamps in the future vs `maxViolations`.
`timeline_gaps`	Event-time gaps vs `maxGapDays`.
`string_pattern_rate`	Values matching a pattern vs `maxRate`.
`referential_integrity`	Orphan rate to a referenced column vs `maxOrphanRate`.
`row_count_not_suspicious`	Row count is not one of the configured `suspiciousCounts`.

Checks run when a table is promoted and again on a scheduled cadence. The lifecycle column tells you which. SPC is a Shewhart control chart on platform-owned series (row count, column null percent, and similar). It is not a substitute for the checks above. Each series has a rolling mean, standard deviation, 2-sigma warning limits, and 3-sigma control limits.

Series status	Meaning
`IN_BOUNDS`	Latest point is inside the warning limits.
`WARNING`	Outside 2-sigma, inside 3-sigma.
`OUT_OF_BOUNDS`	Outside the 3-sigma control limits.
`INSUFFICIENT_HISTORY`	Fewer points than the configured minimum.

Tracked series kinds: TABLE_ROW_COUNT, COLUMN_NULL_PCT, COLUMN_VALIDATION_FAILURE_PCT, COLUMN_DISTINCT_COUNT, COLUMN_AVG. Stats are the shape snapshot each run uses (row count, column count, bytes). Health is the catalog annotation for that table (state, reason, watermark). Health is computed from annotations, not from the same rollup path as checks and SPC.

Latest across the workspace

Start here. These four tables are the current result per provider table.

Table	Contents
`providers_quality.summary.quality_checks`	Latest Gate 2 check per table.
`providers_quality.summary.spc_results`	Latest SPC series per table.
`providers_quality.summary.table_stats`	Latest row / column / byte counts.
`providers_quality.summary.table_health`	Health state and watermark.

Failing checks for Google Workspace:

SELECT
  connector,
  table_name,
  check_id,
  check_status,
  observed,
  threshold,
  comparator,
  captured_at
FROM providers_quality.summary.quality_checks
WHERE connector = 'google'
  AND check_status = 'FAIL'
ORDER BY captured_at DESC

observed is the measured rate or count. threshold is the configured limit. comparator is how they were compared. Every check that did not pass (fail or skip):

SELECT
  connector,
  table_name,
  check_id,
  check_status,
  observed,
  threshold
FROM providers_quality.summary.quality_checks
WHERE check_status IN ('FAIL', 'SKIPPED')
ORDER BY connector, table_name, check_id

SPC series that have a warning, a control-limit breach, or recorded violations:

SELECT
  connector,
  table_name,
  series_id,
  series_kind,
  latest_value,
  mean,
  stddev,
  series_status,
  overall_status,
  captured_at
FROM providers_quality.summary.spc_results
WHERE has_violations = true
   OR series_status IN ('WARNING', 'OUT_OF_BOUNDS')
ORDER BY captured_at DESC

Largest provider tables by row count:

SELECT
  connector,
  table_name,
  row_count,
  column_count,
  total_byte_size,
  captured_at
FROM providers_quality.summary.table_stats
ORDER BY row_count DESC
LIMIT 20

Stale watermarks:

SELECT
  connector,
  table_name,
  health_state,
  health_reason,
  watermark,
  watermark_stale,
  computed_at
FROM providers_quality.summary.table_health
WHERE watermark_stale = true

History for one table

Suffixes map to a result kind:

Suffix	History of
`__checks`	Gate 2 checks (one row per check per run).
`__spc`	SPC series (one row per series per run).
`__stats`	Shape stats.
`__runs`	Maintenance runs (compaction / vacuum), not quality scores.

Google Workspace directory users:

SELECT
  captured_at,
  check_id,
  check_status,
  observed,
  threshold,
  lifecycle,
  overall_status
FROM providers_quality.google.users__checks
ORDER BY captured_at DESC
LIMIT 20

How row-count SPC for that table moved:

SELECT
  captured_at,
  series_id,
  series_kind,
  latest_value,
  mean,
  upper_control3_sigma,
  lower_control3_sigma,
  series_status
FROM providers_quality.google.users__spc
WHERE series_kind = 'TABLE_ROW_COUNT'
ORDER BY captured_at DESC
LIMIT 20

History tables keep every run. They also expose result_id, run_id, evidence_json (checks), and violations_json (SPC) when you need the payload behind a fail.

Older sidecar files may store status as Fail or OutOfBounds. Newer rows use FAIL and OUT_OF_BOUNDS. Summary tables go through typed decode and use the uppercase values. On __checks / __spc, filter both casings if a query comes back empty.

Getting Started

SDKs

Connectors

Parable Data Engine

What each measurement is

Latest across the workspace

History for one table

​What each measurement is

​Latest across the workspace

​History for one table

What each measurement is

Latest across the workspace

History for one table