Skip to main content
Each Tap lands in the Provider Pool as one or more tables. The Tap’s schema is more than a list of columns. Annotations on its fields tell Parable which row is current, which version wins, and which fields identify a person or account. The Provider Plugin declares these annotations. Read them in connector.taps[]:
For each Tap, read schema, dataKind, deletionSemantics, identityDirectory, and responseExtraction.primaryKey. Your Workspace keeps its own copy of each schema as outputSchema. On that copy you can hide fields from SQL, re-point the ordering field, and re-declare identity roles, but not change the keys. See Change a Tap schema in your Workspace. Tap schemas lists every annotation.

From annotations to a table

The Identity family also holds Teams, primary memberships and reporting lines. A Workspace has it when one of its Taps is a directory or lists Teams. Without such a Tap, a query that names its tables reads no rows. Team matching can use names, but people still match only by exact identity keys. See Teams and account flags for mark values, ranking, activity and classification. responseExtraction.primaryKey is how a sync recognizes a fetched record. It is almost always id. The Provider Pool merges a main table on one column:
  1. The first field marked x-transformDedupKey, by field name.
  2. Else the output field of the Tap’s synthetic key.
  3. Else responseExtraction.primaryKey.
  4. Else a field named id.
A Tap marks one root field, and it is the same field as primaryKey.

Names in SQL

The address of a table is providers.{connector}.{table}. {connector} is the plugin’s slug, with each - written as _. {table} is the Tap name. An array of objects can become a child table named {tap}__{path}. GitHub issue labels, for example, are in providers.github.issues__labels. Not every nested object becomes a child table, so check the names in catalog discovery before you query. Column names are the schema field names in lowercase. A flattened object path uses __ between its parts: SQL lowercases a name without quotes, so SELECT updatedAt finds updatedat. A quoted "updatedAt" does not match the column. A struct column keeps the source names of its fields. Write profile."userType" or profile['userType'] to read one.

Dedup keys

A main snapshot or changelog table has one current row for each dedup key. Linear issues and GitHub pull_requests both key on id. You can join or filter on id and expect one row for each issue or pull request. A deleted record can stay in the table. Each row has state columns such as _parable_deleted and _parable_observed. To read current state, filter on _parable_deleted = false. The Provider Pool reference lists every state column. A child table holds one row for each element of the parent’s array. It merges on parent_pk and ordinal: the parent’s key as text, and the element’s position in the array. The parent ID repeats on each child row. Use the complete logical key from the catalog. See Keys and relationships for how to join a child table to its parent. A Tap with child tables also has two views for each child table: a {child}_with_parent view that joins each child row to its parent row, and a {parent}_nested view that puts each child table back into its parent row as an array. GitHub issue labels are in providers.github.issues__labels_with_parent. On an event Tap that stores an event again for each load that reads it, the views pair a child row with the parent row of the same load, on dt and _parable_seal_id. The Provider Pool reference shows how to keep one load’s rows. Do not use SELECT DISTINCT id to clean up a main table. Parable already merged its rows. If you see two rows with the same id, you are probably reading a child table or an event table.

Ordering keys

Each table has at most one x-transformOrdering field. When a sync writes a newer record for the same dedup key, the row with the greater ordering value is the one you query. When the Tap marks no ordering field, its sync.cursorField is the ordering column, if the schema has that field. A child table carries the ordering column of its root row. Linear issues order on updatedAt (SQL updatedat). GitHub pull_requests order on updated_at. Directory Taps such as Linear users also order on updatedAt, so a renamed user replaces the earlier row. dataKind says how Parable treats those versions: deletionSemantics tells you how a deletion in the Provider shows in the table:

Foreign keys

A field marked x-transformForeignKey references a row of another Tap of the same Provider. When the field is an object, the table has one <field>_id column with the referenced row’s ID, not a copy of the row. Notion pages.created_by is the column created_by_id. Join it to the other Tap’s table for the rest:
When the field already holds the key, such as a Slack message’s user, the column keeps its own name. A field whose type is the Tap’s own record type becomes a <field>_id column too, such as a comment’s parent_id. The catalog lists these columns as foreign keys.

Event tables

An event table is laid out by its event time: the field marked x-semantic-role: event_time, or else a timestamp ordering column. Each write is sorted by event time, so a query for one day skips the files of other days. A table that Parable creates this way is partitioned by _parable_event_date, the UTC day of the event time. Filter on it to read whole days:
_parable_event_date is null for a row with no event time. snapshot and changelog tables do not have the column. A child table of an event table carries its parent’s event time under the same name and in the same type, when that field is also the tap’s ordering field. So chrome_activity__events has its own id__time, and a read can filter on it as it filters on the parent.

Identity keys

Identity annotations mark which columns are about a person or account, not which row is unique. Linear users shows every identity annotation on one Tap:
id is both the dedup key of the row and the Linear account ID. email joins that person to Google primaryemail or to workspace.users.email. gitHubUserId is the ID of the same person’s GitHub account. Its annotation names the system (github) and what the value equals (id), so Parable joins the Linear user and the GitHub member with that ID into one identity. A bare "x-transformExternalUserId": true names no system, and Parable joins nothing through it. name labels the person. It never joins. Taps with identityDirectory: true, such as Linear users, GitHub members, and Google users, feed the providers.identity tables. These tables have rows only after at least one directory Tap has synced. Do not join the id columns of two Providers to each other and expect a person match. Join on an email, or go through the identity tables.
account_id_type is id, uuid, or email. match_method records how Parable linked the account. Every current method is exact, and Parable never links accounts on a similar email or an equal name. Identity rows are best-effort: service accounts and shared mailboxes get identities too. An identity ID can change when Parable splits an identity. To keep a stable key for an account, keep its connector_tap_id and account_id. The Provider Pool reference lists every match method.

Worked tables

Linear issues (dataKind: changelog): one row for each id. Order on updatedat. Nested references such as assignee and team stay on the issue row as flattened columns, such as assignee__email and team__key. They are not a second grain. Filter WHERE id = ... for one issue. GitHub pull_requests (dataKind: changelog): one row for each id. Order on updated_at. The sync copies full_name onto each pull request, so you can filter by repository without a join to repositories. GitHub issues (opt-in, dataKind: changelog): labels land in providers.github.issues__labels, one row for each label on each issue. The issue’s key repeats in parent_pk. Google users (dataKind: snapshot, identityDirectory: true): one row for each directory id, which is also the account ID. primaryemail is the column to join people on. When a user leaves the directory, the next full read marks the row _parable_deleted = true.
Catalog discovery lists tables and columns. The Tap schema shows which of those columns are keys. Read both before you write a join.