How taps project into tables

Each tap lands in the providers pool as one or more tables. The tap’s JSON schema is not only a list of columns. Flags on those fields tell the lake which row is the current one, which version wins, and which fields identify a person or account. You inspect those flags on connector.taps[]. You do not set them.

curl "https://api.askparable.com/api/vendors/tenant-connector-instances" \
  -H "Authorization: Bearer $PARABLE_API_TOKEN"

For a tap, read schema (and outputSchema on your overlay, if you narrowed it), plus dataKind, identityDirectory, and responseExtraction.primaryKey.

From flags to a queryable table

What you see on the tap	What it means when you query
`x-transformDedupKey`	Columns that uniquely identify an entity. After sync, the table has one current row per distinct key combination.
`x-transformOrdering`	The single column that picks the winner when two versions share a dedup key. Higher value wins. Usually the same field as `sync.cursorField`.
`x-transformAccountId`	Provider-native principal id. Join key into `providers.identity.account_associations.account_id`.
`x-transformPersonEmail`	Email that identifies a person. Safe to join to other person-email columns and to `workspace.users.email`.
`x-transformPersonName`	Name used for fuzzy identity matching. Do not treat it as unique.
`x-transformExternalUserId`	Id this row carries for another system. A join hint, not this tap’s primary id.
`dataKind`	How versions collapse: `snapshot`, `changelog`, or `event`.
`identityDirectory`	When true, this tap feeds `providers.identity.*`.

responseExtraction.primaryKey is how ingestion recognizes a fetched record (almost always id). Lake uniqueness is the x-transformDedupKey set. They usually name the same field. Child tables often use a composite (for example type + value on a nested Google user email). If no dedup key is declared, the lake falls back to a column named id.

Names in SQL

Address: providers.{connector}.{table}. {connector} is the connector slug. {table} is the tap name, lowercased. Nested object arrays become extra tables named {tap}__{path} (GitHub issue labels are providers.github.issues__labels). Confirm names in the catalog before you query; not every nested object explodes. Column names are the schema property names, ASCII-lowercased. Nested fields that flatten use __ between segments:

Schema field	SQL column
`primaryEmail`	`primaryemail`
`updatedAt`	`updatedat`
`updated_at`	`updated_at`
`commit.committer.date` (flattened)	`commit__committer__date`

SELECT updatedAt against Linear issues will not find the column. Use updatedat.

Dedup keys

Every current row in a provider table is unique on its dedup key columns. Linear issues and GitHub pull_requests both key on id. You can join or filter on that column and expect one row per issue or PR. When a key is composite, you need every part. A nested Google emails child table keys on address; a nested external-id child keys on value and type. Look at x-transformDedupKey on that nested type, not only on the root. Do not SELECT DISTINCT id to “clean up” a provider table. Dedup already happened. If you see two rows with the same id, you are probably looking at a child table (parent id repeated per nested element) or you joined without the full key.

Ordering keys

A tap may declare at most one x-transformOrdering field. When ingestion writes a newer payload for the same dedup key, the row with the greater ordering value is the one you query. Linear issues: updatedAt (SQL updatedat). GitHub pull requests: updated_at. Directory taps such as Linear users also order on updatedAt so a renamed user replaces the previous row. dataKind is how those versions are treated:

`dataKind`	What the table holds
`snapshot`	Point-in-time state. Latest row per dedup key overwrites. Typical for users, teams, repositories.
`changelog`	Entities that mutate (issues, pull requests). Merged by business key; you still query one current row per key.
`event`	Immutable occurrences. Appended and deduped by id; two events with different ids stay two rows.

deletionSemantics tells you whether a missing source record disappears from the table. undetectable (Linear issues, GitHub PRs) means a deleted item can remain until something else updates that key. by_absence on a snapshot tap means a row gone from the latest extract is gone from the table. tombstone means the source sends an explicit delete marker.

Identity keys

Identity flags mark which columns are about a person or account, not which row is unique. Linear users is the full set on one tap:

{
  "id": { "x-transformDedupKey": true, "x-transformAccountId": true },
  "name": { "x-transformPersonName": true },
  "email": { "x-transformPersonEmail": true },
  "updatedAt": { "x-transformOrdering": true }
}

id is both the row’s dedup key and the Linear account id. email is how you join that person to Google primaryemail or workspace.users.email. name is for matching, not for joins. Taps with identityDirectory: true (Linear users, GitHub members, Google users) contribute to providers.identity.*. That family appears only after at least one directory tap has synced.

Table	Grain	Use
`providers.identity.account`	One row per resolved identity (`id`, `name`).	The person/account entity you join to.
`providers.identity.account_associations`	One row per `(connector, tap, account_id)`.	Maps a provider-native id onto `identity_account_id`.
`providers.identity.account_merges`	Survivor / merged id + `occurred_at`.	When an identity id was replaced.

Association uniqueness is (connector_id, connector_tap_id, account_id). Join Google’s id (account id) through associations; do not join two providers’ id columns to each other and expect a person match.

-- Same person: Linear email to Google primary email
SELECT
  lu.email,
  lu.name AS linear_name,
  gu.orgunitpath
FROM providers.linear.users AS lu
JOIN providers.google.users AS gu
  ON gu.primaryemail = lu.email

-- Same person when emails differ: go through identity
SELECT
  a.name,
  assoc.account_id,
  assoc.account_id_type,
  assoc.match_method
FROM providers.identity.account AS a
JOIN providers.identity.account_associations AS assoc
  ON assoc.identity_account_id = a.id

account_id_type is id, uuid, or email. match_method records how the link was made (canonical, email_exact, email_local_part, email_fuzzy, name_fuzzy). Identity rows are best-effort: service accounts and shared mailboxes get identities too.

Worked tables

Linear issues (dataKind: changelog): one row per id. Order on updatedat. Nested refs (assignee, team) stay on the issue row as flattened or nested columns; they are not a second grain. Filter WHERE id = ... for a single issue. GitHub pull_requests (dataKind: changelog): one row per id. Order on updated_at. Parent context copies full_name onto the PR so you can filter by repo without joining repositories. Labels on GitHub issues explode to providers.github.issues__labels, unique on label id, with the parent issue id repeated. Google users (dataKind: snapshot, identityDirectory: true): one row per directory id (also the account id). primaryemail is the person-email join column. Nested arrays (phones, external ids) are separate tables with their own composite dedup keys.

The catalog lists tables and columns. The tap schema lists which of those columns are keys. Read both before you write a join.

Getting Started

SDKs

Connectors

Parable Data Engine

From flags to a queryable table

Names in SQL

Dedup keys

Ordering keys

Identity keys

Worked tables

​From flags to a queryable table

​Names in SQL

​Dedup keys

​Ordering keys

​Identity keys

​Worked tables

From flags to a queryable table

Names in SQL

Dedup keys

Ordering keys

Identity keys

Worked tables