Skip to main content
Every Provider Plugin is made of Taps. A Tap is one stream of records from the Provider, such as Linear issues, GitHub pull_requests, or Slack channels. The Linear plugin, for example, has Taps from issues and projects to comments and workflow_states. Each Tap lands as its own set of tables in the Provider Pool. The Provider Plugin describes how Parable reads each Tap. Read it in connector.taps of one Provider connection. You do not set the transport, the pagination, or the sync mode, because they follow how the Provider’s API works. How a sync reads a Provider follows these fields through a Linear query and a GitHub parent-child read. A Tap looks like this (other fields are not shown):

Transport

Each Tap has one or more transport options. An option is one way to call the Provider: A sync uses one option for each Tap. A rest option wins over a graphql option. Among options of the same kind, the lowest priority wins, and 1 is highest. A sync does not try the next option when the first one fails. GitHub Taps use rest. Linear Taps use graphql.

Pagination

Providers return results in pages in different ways. The transport option’s pagination.type tells a sync how to request each page until the Provider has no more: You do not choose this. It lets a GitHub Tap and a Linear Tap use the same sync engine with two different paging rules.

Sync mode

sync.mode says how much each run reads: Each run reads a fixed time window. After the first sync of a new connection, scheduled runs usually read the previous complete UTC day. Google Cloud Billing reads hourly export windows. Sync windows says when the first sync runs and what it reads. sync.incrementalParams.filterType says how the window goes to the Provider: Some Providers use a job that a sync creates, polls, and then reads (sync.asyncQuery). You see that block on the Tap when the Provider works that way.

Record shape

schema on the plugin’s Tap is the shape of the records it emits. A sync checks each record against it. Annotations on that schema, such as x-transformDedupKey, x-transformOrdering, and the identity annotations, decide how each row lands in the Provider Pool. Tap schemas lists every annotation, and Tap tables shows the tables that result.

Template Taps

A template Tap is a plugin Tap that has parameters. It tells Parable how to read one table from a database Provider, such as Snowflake, but it does not name the table. A template Tap has no tables of its own. It runs only through the custom Workspace Taps that fill it. Each custom Tap supplies the table, the key, and the cursor in config.source. parameters is a Parable input schema. Each input field names one value that the request reads as a {tap.NAME} placeholder. A placeholder can be anywhere in transportOptions or sync, for example in a Snowflake requestBody.statement:
A parameter can have only one of these names. The value of each comes from the custom Tap: Each parameter must have an x-validatePattern that starts with ^ and ends with $, and the two anchors must hold the whole pattern. Put an alternation inside a group, as ^(?:a|b)$. The value comes from a Workspace, so the pattern stops a value that could change the request. For columns, the pattern applies to each column. A write of a plugin Tap fails with a field error when: A template Tap never runs on its own. Tap selection never turns it on, even when ingestionConfig.enabledTaps names it, and a Workspace override of it fails with WA-TCT-038. Definition rules lists these rules with the other plugin rules. Staff set parameters on POST /api/connectors/{connectorId}/taps and change it on PATCH /api/connectors/{connectorId}/taps/{tapId}. null on the update removes it, and the Tap is then an ordinary Tap. Every answer that returns the plugin Tap includes parameters. Research does not write template Taps, so a promotion keeps the parameters of a stored Tap. A rollback to a revision that has no parameters also keeps the stored value.

Reader text

Each Tap carries text for the people who read its data: Each field of the Tap’s schema has its own text in title, description, and x-purpose. Reader text never changes the stored data.

Required access

requiredAccess says what the Provider credential must allow before the Tap can read its data. Parable derives it from the plugin. You cannot set it. GitHub pull_requests needs the repo scope:
For an operation that requires no additional OAuth scopes, set sync.scopesNotRequired to true and leave sync.scopes empty. Its requiredAccess keeps the oauth_scope model with empty values. Set it only when the Provider’s documentation states that the operation needs no scope. An operation whose scope the source does not state is not proof. Missing scope information still fails catalog validation (scopesRequired). A write that sets the declaration beside nonempty scopes fails with scopesNotRequiredConflict. Research does not set the declaration. An author sets it by hand. requiredAccess is null when the plugin offers more than one authentication strategy, because each strategy can carry different access.

Sensitive Taps

Some Taps read sensitive data: pay and payroll, tax IDs, birth dates, sex, home contact details, bank accounts, or free-text notes. Each kind has its own Tap, and each such Tap is off by default (enabledByDefault: false). The plugin’s main authentication strategy asks only for the access that the default Taps need. A second strategy of the same type, with its own label, adds the access of the sensitive Taps. Justworks, for example, offers “Justworks OAuth 2.0 (company OAuth application)” and “Justworks OAuth 2.0 with pay, payroll and personal data”. Because such a plugin has two strategies, its Taps show requiredAccess: null. To read a sensitive Tap:
  1. Connect with the strategy that adds the sensitive access, or change the connection to it.
  2. Set a Provider Policy on each table of the sensitive Tap. A table that no Provider Policy names is open to every reader who can read Provider data.
  3. Turn the Tap on (see Turn Taps on and off).

Taps in your Workspace

Your Workspace keeps its own record for each Tap of a connection. A new connection starts with the plugin’s default Taps (enabledByDefault: true) and the parent Taps that they need. Each Workspace Tap has these fields:

Turn Taps on and off

The connection’s ingestionConfig.enabledTaps decides which plugin Taps a sync runs:
  • When enabledTaps is absent, a sync runs the Taps with enabledByDefault: true.
  • When enabledTaps is a list, a sync runs the Taps that it names. The list replaces the defaults. An empty list runs no Tap.
  • A sync also runs the parent Taps of each Tap that runs.
  • A Workspace Tap with enabled: false does not run, even when enabledTaps names it.
  • A template Tap (a Tap that declares parameters) never runs on its own, even when enabledTaps names it. It gets no override Workspace Tap, and the catalog serves no table for it. Only its custom Workspace Taps run. A template Tap must have enabledByDefault: false and identityDirectory: false.
  • When the plugin changes a Tap’s enabledByDefault, Parable turns that Tap on or off in each connection that has no enabledTaps list. A Workspace Tap that you turned off stays off. Demo Workspaces keep their Taps. A Tap stays off when its tables would get the SQL name of a custom Tap’s, or when its parent Tap is off.
To turn on an opt-in Tap, send the connection’s whole ingestionConfig, with enabledTaps listing every Tap that you want, to POST /api/connectors/configure. The ingestionConfig that you send replaces the stored one, so include your other settings, such as orgId.
This operation takes an encrypted request body. The SDKs encrypt it when you give them your Workspace’s public key. See Change a connection and Encrypted requests. The call needs permission to manage the connection’s credentials. Parable refuses a change of ingestion settings while the plugin accepts only credentials (Availability). Parable then creates a Workspace Tap for each Tap that the list adds, and disables the Workspace Taps of the Taps that leave the list. Turning a Tap on starts no backfill. The Tap starts with the next scheduled sync. To stop a Tap that runs, set enabled to false on its Workspace Tap:
enabled: false stops a Tap. enabled: true does not add a Tap that enabledTaps leaves out. A child Tap needs its parent. Before you disable a parent Tap, disable its child Taps (WA-TCT-025, WA-TCT-026). The catalog serves only enabled Taps, so a disabled Tap has no table there until you enable it again. Two enabled Taps of one Provider in a Workspace cannot have tables or views with the same SQL name, on one connection or on two. For example, orders and orders_ get the same name. A create, a rename, a schema change or an enable that causes this fails with WA-TCT-036, and nothing is stored. The error names the other Tap. The same rule applies when Parable adds a Provider’s Taps to your Workspace: when the Provider gets a new Tap, and when you save a connection’s enabledTaps. If a Tap would get the SQL name of an enabled custom Tap in your Workspace, Parable does not add it. It adds the other Taps and gives no error. To get the missing Tap, rename or disable the custom Tap, then save the connection again.

Change a Tap schema in your Workspace

The outputSchema of a Workspace Tap of kind override is a full copy of the plugin’s Tap schema. It keeps the same fields and types as the plugin’s schema (WA-TCT-022). You can change only annotations, and only some of them. x-exclude and x-transformEmbed: false stay on your copy when Parable changes the plugin’s Tap. A re-pointed ordering, partition date, fingerprint input, or identity role lasts until Parable next changes the plugin’s Tap. Your copy then takes the plugin’s annotations again. Team, bot and external marks inherit the plugin’s schema on fields the layout retains. Your copy can re-declare them. The manager mark always follows the plugin’s schema. See Teams and account flags. Parable refuses these writes: A WA-TCT-021 message names the rule and the table, such as outputSchema violates promote directive rules: <rule> (root table): <message>. A WA-TCT-035 error lists the fields in its details. Send every x-access* value as the plugin’s copy holds it. Provider Permissions explains why. A change to outputSchema can need a rebuild of the Tap’s stored tables, for example a new ordering field. When a Tap schema changes lists the classes of change and how to approve a rebuild. config also has keys that only Parable writes: materializedDefault, systemDisabled, catalogSlug, storageUri, and source. A write that sets one of them fails with WA-TCT-029. config.selectedFields works only on Salesforce Taps (WA-TCT-024), and never on an Artifact Tap (WA-TCT-028).

Custom Taps on a database connection

A database Provider, such as Snowflake, has a template Tap. A template Tap holds how to reach the database and how to send a query, but it reads no table of its own. Its request has {tap.*} placeholders, and its parameters give a pattern for each one. A custom Tap (kind: custom) on a connection of that Provider reads one table. Its config.source names the table, the key and the cursor. The root fields of its outputSchema are the columns that it reads. Your catalog serves it under its own name. Only Parable staff create custom Taps for now. The operation needs the staff permission admin.tenant.connectors.taps.custom. A Workspace Administrator gets 403.
Parable checks the request before it stores the Tap:
  • The Tap in connectorTapId is a template Tap of the connection’s Provider. If not, the request fails with WA-TCT-038.
  • Each value that fills a {tap.*} placeholder matches the pattern of its parameter. For {tap.columns}, each root field of outputSchema must match. If one does not match, the request fails with WA-TCT-037, and the details name the parameter.
  • The source fills each placeholder that the template reads. The key columns, key.syntheticField and cursorColumn are root fields of outputSchema. A key of more than one column names a syntheticField. The key field (the one key column, or syntheticField) is the one root field with x-transformDedupKey. refreshLookbackDays and maxWindowDays need a cursorColumn. If one of these fails, the request fails with WA-TCT-038.
  • The outputSchema plans with the source’s key and cursor.
  • The Tap’s catalog tables do not get the SQL name of another enabled Tap of the Provider in your Workspace (WA-TCT-036).
access is optional. A custom Tap with no access stores {"roles": ["resources"]}. Only people with All data access read its table until a Provider Policy opens it. See A table that only All data access reads. PATCH and DELETE work on a custom Tap as on any Workspace Tap. A change of config.source, config.access or outputSchema needs the same staff permission and runs the same checks. A config write that leaves out source or access keeps the stored values. A change of source.mode keeps the Tap’s name, so the table keeps its SQL name. config.source and config.access on any other Tap fail with WA-TCT-038 and WA-TCT-028.

Manage Taps with the API

The list returns up to 250 Taps on each page. Use limit and offset to read more. The read of one Tap returns its whole outputSchema. The schema of a large Provider can be more than 50,000 characters. To read it a page at a time, send fieldLimit (1 to 1,000), fieldOffset, or both. The answer then holds one page of the fields of outputSchema: at most fieldLimit fields, and fewer than 40,000 characters in the whole answer, but always one field. The fields of the root type come first, then those of each other type by name. A type is on each page that holds one of its fields, and the enums are on every page. Send nextFieldOffset back as fieldOffset for the next page. The last page has no nextFieldOffset. A fieldLimit outside 1 to 1,000 or a negative fieldOffset fails with 400. PATCH can change enabled, name, description, outputSchema, config, and canonicalPersonSource. It cannot change the connection, the plugin Tap, the sync mode, the pagination, or the transport. A Workspace Tap record does not add a Tap to the Taps that a sync runs. enabledTaps does. The API Reference has the full request and response shapes.