> ## Documentation Index
> Fetch the complete documentation index at: https://developers.askparable.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Parable Data Engine

> Query your workspace lake: providers, artifacts, parables, workspace, and AI SQL functions.

The **Parable Data Engine** is the SQL surface over your workspace lake.
The lake is four **pools**, each a catalog you address by name. Bronze,
silver, and gold are not part of this surface. [AI](/data/ai) functions
(`ponder()`, `embed()`) run on the same connection.

How a [tap](/connectors/taps) authenticates and pages the source is on
the [connector](/connectors/overview) and
[ingestion](/connectors/ingestion). The lake is where queryable tables
live.

```mermaid theme={null}
flowchart LR
  C["Connectors"] --> PR["providers"]
  PR --> Q["providers_quality"]
  U["Uploads"] --> A["artifacts"]
  P["Plots"] --> PA["parables"]
  M["Control plane"] --> W["workspace"]
```

| Pool                             | Address                         | What it holds                                                                                                            |
| -------------------------------- | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| **[providers](/data/providers)** | `providers.{connector}.{table}` | Current data from each connected tool. [Tap projection](/connectors/tap-projection) is how tap keys become those tables. |
| **[artifacts](/data/artifacts)** | `artifacts.{table}`             | Spreadsheets and files you published.                                                                                    |
| **[parables](/data/parables)**   | `parables.{parable}.{plot}`     | Materialized Plot results.                                                                                               |
| **[workspace](/data/workspace)** | `workspace.{table}`             | Live workspace metadata (users, connectors, roles, plots).                                                               |

[Quality](/data/quality) is a separate catalog,
`providers_quality`, not a fifth data pool. It holds SPC series, Gate 2
checks, table stats, and health for provider tables.

[AI](/data/ai) is `ponder()` and `embed()`: LLM and embedding calls in
the same SQL.

References are fully qualified. There is no `USE` or `search_path`. The
workspace on the request selects which tables you see; the catalog name
never encodes the workspace.

## The catalog

The catalog lists every table across the four pools. Per table it
reports:

* **Shape**: row counts and schema.
* **Freshness**: when data last arrived (providers), publish time
  (artifacts), last successful Plot run (parables). Workspace tables
  are current at query time.
* **Quality**: automated checks and SPC. Query them in
  [Quality](/data/quality).
* **Lineage**: which taps or Plots feed the table.

```bash theme={null}
curl "https://api.askparable.com/api/catalog/catalog?includeLineage=true&includeHealth=true" \
  -H "Authorization: Bearer $PARABLE_API_TOKEN"
```

A table can appear in the catalog before it has rows. Probe with
`SELECT COUNT(*) FROM ... LIMIT 1` if you need to know whether data
exists.

## Querying the lake

Lake tables are queryable over **Arrow Flight SQL**. JDBC and ADBC
drivers work, as do DataGrip, DBeaver, and the clients below. You
authenticate with the same API token you use for the REST API.

This example queries the [providers](/data/providers)
pool. Swap the SQL for `artifacts.{table}`, `parables.{parable}.{plot}`,
`workspace.{table}`, or `providers_quality.summary.*` the same way.

<CodeGroup>
  ```python query.py theme={null}
  import os

  import adbc_driver_flightsql.dbapi
  from adbc_driver_flightsql import DatabaseOptions

  token = os.environ["PARABLE_API_TOKEN"]

  with adbc_driver_flightsql.dbapi.connect(
      "grpc+tls://<your-flight-endpoint>",
      db_kwargs={
          DatabaseOptions.AUTHORIZATION_HEADER.value: f"Bearer {token}",
      },
  ) as conn:
      with conn.cursor() as cur:
          cur.execute(
              """
              SELECT
                id,
                primaryemail,
                orgunitpath,
                isadmin,
                lastlogintime
              FROM providers.google.users
              WHERE suspended = false
              LIMIT 10
              """
          )
          print(cur.fetch_arrow_table())
  ```

  ```typescript query.ts theme={null}
  import { tableFromIPC } from "apache-arrow";
  import { createFlightSqlClient } from "@parable/flight-sql-client";

  const token = process.env.PARABLE_API_TOKEN;
  if (!token) {
    throw new Error("Set PARABLE_API_TOKEN");
  }

  const client = await createFlightSqlClient({
    host: "<your-flight-endpoint>",
    port: 443,
    tls: true,
    headers: [["authorization", `Bearer ${token}`]],
  });

  const ipc = await client.query(`
    SELECT
      id,
      primaryemail,
      orgunitpath,
      isadmin,
      lastlogintime
    FROM providers.google.users
    WHERE suspended = false
    LIMIT 10
  `);
  console.log(tableFromIPC(ipc).toArray());
  await client.close();
  ```

  ```go query.go theme={null}
  import (
  	"context"
  	"fmt"
  	"log"
  	"os"

  	"github.com/apache/arrow-go/v18/arrow/flight/flightsql"
  	"google.golang.org/grpc"
  	"google.golang.org/grpc/credentials"
  	"google.golang.org/grpc/metadata"
  )

  ctx := metadata.AppendToOutgoingContext(
  	context.Background(),
  	"authorization", "Bearer "+os.Getenv("PARABLE_API_TOKEN"),
  )

  client, err := flightsql.NewClient(
  	"<your-flight-endpoint>:443",
  	nil,
  	nil,
  	grpc.WithTransportCredentials(credentials.NewTLS(nil)),
  )
  if err != nil {
  	log.Fatal(err)
  }
  defer client.Close()

  info, err := client.Execute(ctx, `
  SELECT
    id,
    primaryemail,
    orgunitpath,
    isadmin,
    lastlogintime
  FROM providers.google.users
  WHERE suspended = false
  LIMIT 10
  `)
  if err != nil {
  	log.Fatal(err)
  }

  reader, err := client.DoGet(ctx, info.Endpoint[0].Ticket)
  if err != nil {
  	log.Fatal(err)
  }
  defer reader.Release()

  for reader.Next() {
  	fmt.Println(reader.Record())
  }
  if err := reader.Err(); err != nil {
  	log.Fatal(err)
  }
  ```

  ```rust query.rs theme={null}
  use arrow_flight::sql::client::FlightSqlServiceClient;
  use futures::TryStreamExt;
  use tonic::transport::Channel;

  let token = std::env::var("PARABLE_API_TOKEN")?;
  let channel = Channel::from_static("https://<your-flight-endpoint>")
      .connect()
      .await?;
  let mut client = FlightSqlServiceClient::new(channel);
  client.set_header("authorization", format!("Bearer {token}"));

  let info = client
      .execute(
          r#"
          SELECT
            id,
            primaryemail,
            orgunitpath,
            isadmin,
            lastlogintime
          FROM providers.google.users
          WHERE suspended = false
          LIMIT 10
          "#
          .to_string(),
          None,
      )
      .await?;
  let ticket = info.endpoint[0].ticket.clone().expect("ticket");
  let mut stream = client.do_get(ticket.into()).await?;
  while let Some(batch) = stream.try_next().await? {
      println!("{batch:?}");
  }
  ```
</CodeGroup>

<Info>
  Flight SQL access is provisioned per workspace. Contact your Parable
  representative to enable it and receive your workspace's query endpoint.
</Info>