Skip to main content
The connector definition is read-only. Every instance you list includes the nested connector object: taps, traversal, rate limits, addressing, and the schema for ingestionConfig. That object is the map ingestion follows when it talks to the source.
Look at connector.taps, connector.traversal, connector.defaultBaseUrl, connector.addressingConfig, connector.rateLimits, and connector.ingestionConfigSchema.

​
What you set vs what you inspect

Two blobs on the instance are yours. Everything that describes how to walk the API lives on the definition. GitHub’s ingestionConfig requires orgId. Linear’s is an optional baseUrl. Neither lets you change pagination or GraphQL queries.

​
A sync run

Disabled taps are skipped. Child taps whose parent was skipped or empty do not run.

​
Resolve the URL

Ingestion builds every request from:
  1. defaultBaseUrl on the connector (or on the active auth strategy, which wins when set). Linear is https://api.linear.app. GitHub is https://api.github.com.
  2. addressingConfig when the host itself is templated. scheme, host, and basePath can contain {var} placeholders. Each variable lists a source: user_input (from ingestionConfig), oauth_response (from the token payload, for example a Jira cloudId), or static.
  3. Path templates on the tap. REST restConfig.endpoint values such as /orgs/{orgId}/repos or /repos/{full_name}/issues are filled from ingestionConfig or from a parent record, depending on the placeholder.
A GitHub repositories request is therefore GET https://api.github.com/orgs/{orgId}/repos with {orgId} taken from the ingestionConfig you sent when you connected.

​
Walk taps in order

connector.traversal is the graph: On an edge, parentKeyField is read from each parent record and substituted into the child’s endpoint. templateVariable overrides the placeholder name when it differs from the field. parentContextFields copies extra parent fields onto each child record (GitHub copies full_name onto pull requests so later taps still know the repo). parentEligibility can drop a parent before any child request is planned. GitHub is a tree: repositories, members, and teams are roots. repositories.full_name fans out issues, pull_requests, commits, branches, and releases. pull_requests.number then fans out pull_request_comments. Linear is flat: every tap is a root and edges is empty. Each GraphQL query is independent; issues do not have to be listed before comments.

​
One tap: request, page, extract

For each enabled tap, ingestion picks the first usable entry in transportOptions (priority 1 is highest). That option is one way to call the source.

​
Transport

REST placeholders come from ingestionConfig or from the parent record on a traversal edge. GraphQL placeholders usually live in variables and are updated the same way (and by pagination / incremental injection below).

​
Incremental window

sync.mode is incremental or full. Incremental taps keep a watermark on sync.cursorField (for example Linear updatedAt, GitHub updated_at). The first run uses a lookback window; later runs start from the last watermark (minus a buffer for late-arriving rows). maxLookbackDays caps that first window when the source retains only recent data. How the window is sent is sync.incrementalParams.filterType: maxWindowDays splits a long range into sub-windows when the source rejects large spans. defaultLookaheadDays extends the end past “now” for forward-looking APIs such as calendars. If incrementalParams is absent, the watermark is still stored from cursorField, but the source request may not receive a time filter. Some taps use sync.asyncQuery: POST to create a job, poll statusEndpoint until successValue, then page recordsEndpoint. You will see that block when the source works that way (audit-log exports, for example).

​
Pagination

The transport option’s pagination.type is how ingestion walks pages until the stream is exhausted. Details of each type are on Taps and streams. Cursor pagination also names cursorPath / hasMorePath (JMESPath into the body) and either cursorParam (query string) or cursorBodyPath (POST body, including GraphQL variables.after). cursorIsFullUrl is for APIs that return the next request as an absolute URL (OData @odata.nextLink).

​
Records

responseExtraction turns a response body into rows: Those rows are validated against the tap schema and land in the providers pool. Malformed source records stop at the boundary instead of landing in the lake.

​
Two worked paths

​
Linear issues (GraphQL, flat)

The tap is a root. Transport is GraphQL POST /graphql. The query asks for issues(first: $first, after: $after, filter: { updatedAt: { gt: $since } }). Static variables set first to 100.
  1. Ingestion writes the watermark into variables.since (filterType: body_filter, bodyFilterPath: variables.since).
  2. It POSTs the query.
  3. It extracts data.issues.nodes. Each node’s id is the primary key; updatedAt advances the watermark.
  4. If data.issues.pageInfo.hasNextPage is true, it puts pageInfo.endCursor into variables.after and POSTs again.
No parent fan-out. The next Linear tap is a separate query.

​
GitHub pull_requests (REST, child of repositories)

repositories is a root: GET /orgs/{orgId}/repos, Link header pagination, body is an array (responsePath: "@"). Each repository record has full_name. The edge repositories -> pull_requests substitutes that into GET /repos/{full_name}/pulls?per_page=100&state=all&sort=updated. Incremental runs also send since from the last updated_at watermark. The JSON body is again an array. GitHub’s next page is the Link header, not a cursor field. full_name is copied onto each PR record so pull_request_comments can call /repos/{full_name}/pulls/{number}/comments without looking the repo up again.

​
Rate limits, retries, and skipped parents

connector.rateLimits (and optional per-tap rateLimits) cap requests per second and max concurrent calls. retry.respectRetryAfter honors the source’s Retry-After when it is present. GitHub’s definition sits near 1.4 requests/second, so a large backfill is paced on purpose. On fan-out, skipOnHttpStatus (for example [403, 404]) skips that parent and continues. errorBodyRules can reclassify a status by inspecting the body when the source uses one code for several outcomes. perParentAuth switches the OAuth subject per parent record (Google Workspace impersonation). You do not configure these; they explain why one repo 404s and the rest of the sync finishes.

​
Allowlists in ingestionConfig

Some connectors accept parentDataFilters inside ingestionConfig. When the tap’s restConfig has parentDataFilterQueryInject or parentDataFilterBodyInject, those values are written into $filter, a repeated query param, or the JSON body so the source only returns matching rows. If the filter key is absent, the tap runs workspace-wide. If the key is present but empty or the field name does not match, the tap is skipped.
Step-by-step connector setup (OAuth apps, API keys, scopes) lives in the product documentation.