Connecting · per-system runbooks

Connecting your systems

Ingestion is the mechanism reference: every door and how the write path treats it. This page is the operator's counterpart: “I have HubSpot / Salesforce / a Google Drive / a Postgres database / a pile of docs / an internal system. Walk me through connecting it and getting my existing data in for the first time.” One runbook per system, grounded in the real connector source.

start simple

Just trying Verity? Do the file drop: verity-cli add ./dir --visibility 1, 60 seconds, no credential. Connecting a real system? Jump straight to yours: HubSpot, Salesforce, Google Drive, Postgres/MySQL, or any internal system. Every runbook is the same five steps.

First time end to end? The 5-minute Get started walks the whole loop.

The universal shape

Verity ships zero OAuth apps. Bring-your-own-token is the only auth mode of the open-source core: you create a first-party credential in your own tenant (a HubSpot private app, a Salesforce Connected App, a Google service account) and paste it into Verity. Your tokens never rest in a third-party cloud. Every runbook below reads the same five steps:

The universal connect shape Create a first-party credential in your own tenant, configure the connector's env vars, run the initial backfill which resolves ACLs before content, verify a scoped recall, then turn on ongoing sync. 1 credential in your tenant 2 configure env vars 3 initial backfill ACL before content 4 verify scoped recall 5 ongoing sync webhook / poll / schedule
Every runbook: create the credential → configure → backfill → verify scoped → turn on ongoing sync.
ACL before content The initial crawl resolves each item's ACL first, and only pulls content if the ACL resolves (SPEC §5a). An item whose visibility cannot be derived is quarantined, never permissively indexed. Backfills are rate-limit-aware and resumable: the cursor is checkpointed after delivery succeeds, so a crash replays the window into deterministic keyed upserts: at-least-once, never silent loss. Data becomes queryable progressively as it lands.

Common prerequisites

Every runbook assumes a running Verity server and a tenant. Create both once:

sh
# local all-in-one: docker Postgres + server + a "dev" tenant + a scope handle
cargo run --release -p verity-cli -- dev

# or, against an existing server, mint a tenant on the trusted admin plane
export VERITY_URL=http://127.0.0.1:7717
export VERITY_ADMIN_TOKEN=...              # the server's admin bearer, if configured
export TENANT=$(curl -s -X POST "$VERITY_URL/v1/admin/tenants" \
  -H 'content-type: application/json' -H "authorization: Bearer $VERITY_ADMIN_TOKEN" \
  -d '{"name":"acme-corp"}' | jq -r .tenant_id)

The Python ingest plane (connectors) reads three shared env vars, plus each connector's own credential:

VERITY_URL
The server base URL the connector's sink POSTs to. Debezium-lane connectors (HubSpot, Salesforce) default to http://127.0.0.1:7717; the Drive connector defaults to http://localhost:8080, so set it explicitly to your server.
VERITY_TENANT_ID
The tenant UUID from above. Required by the Debezium sink; the Drive runner falls back to "default".
VERITY_ADMIN_TOKEN
Sent as the bearer token on the trusted connector plane (/v1/ingest/debezium, /v1/ingest/documents, /v1/admin/principals) when the server requires it.

Install the ingest SDK: pip install verity-ingest (the Drive connector needs the extra: pip install 'verity-ingest[gdrive]' for google-auth).

HubSpot

The flagship connector. HubSpot exposes no per-record ACL API, so it is ACL tier C: the connector cannot mint a faithful ACL envelope and instead requires an admin-assigned visibility_policy with no default. Provenance on every emitted fact is admin-assigned.

1 · Create the credential

In your own HubSpot portal, go to Development → Keys → Service keys, create a Service Key, grant it the CRM read scopes you want mirrored (crm.objects.contacts.read, .companies.read, .deals.read), and copy it; it looks like pat-na1-.... This is BYOT: no vendor-hosted app, ~2 minutes. (HubSpot deprecated private apps in 2026. A legacy private-app token still works via HUBSPOT_PRIVATE_APP_TOKEN; both are used as a bearer token.)

2 · Configure

sh
export HUBSPOT_SERVICE_KEY=pat-na1-...
export VERITY_URL=http://127.0.0.1:7717
export VERITY_TENANT_ID=$TENANT
export VERITY_ADMIN_TOKEN=...              # if the ingest plane requires it

3 · Run the initial backfill

--once runs one truth-lane poll cycle. With no saved cursor this is a full backfill: it searches contacts, companies, and deals on the last-modified property (hs_lastmodifieddate; contacts use lastmodifieddate, a documented HubSpot quirk) from epoch, ascending, paginated at 100/page, honoring the search API's 429 Retry-After. Each non-null property becomes one deterministic keyed L1 fact via POST /v1/ingest/debezium; the last-modified timestamp becomes valid_from. The cursor (max last-modified seen) is written to the state file at the end.

sh
python -m verity_ingest.connectors.hubspot --once --visibility 1,2
# poll: 148 fact event(s), cursor -> 2026-07-09T18:04:57.000Z -> {written:148,...}
# cursor persists to .verity/hubspot_cursor (override: --state-file / $HUBSPOT_STATE_FILE)
tier C: required --visibility is required and has no default. It is the admin-assigned principal-token set every fact carries (HubSpot has no per-record ACL to mirror). Omitting it is a usage error that names the invariant; passing an empty set fails closed.

4 · Verify

Facts land keyed (source="hubspot", table=<object>, id=<record id>, field); the L1 partition is hubspot:<object>. Point-read a field, then confirm scoping: a scope whose principals include your policy tokens sees it; one that does not sees nothing.

bash
# mint a scope handle whose principals include a policy token (e.g. 1)
IN=$(curl -s -X POST "$VERITY_URL/v1/scopes" -H 'content-type: application/json' \
  -d "{\"tenant_id\":\"$TENANT\",\"principals\":[1],\"entity_scope\":[]}" | jq -r .scope_handle)

# the record is queryable at its keyed L1 address
curl -s "$VERITY_URL/v1/records/hubspot:deals/<deal-id>/amount?scope_handle=$IN" | jq .value
# → "84000"

# a scope holding NO policy token sees nothing, fail closed
OUT=$(curl -s -X POST "$VERITY_URL/v1/scopes" -H 'content-type: application/json' \
  -d "{\"tenant_id\":\"$TENANT\",\"principals\":[999],\"entity_scope\":[]}" | jq -r .scope_handle)
curl -s -o /dev/null -w '%{http_code}\n' "$VERITY_URL/v1/records/hubspot:deals/<deal-id>/amount?scope_handle=$OUT"
# → 404  (out-of-scope value is not visible to this principal set)

5 · Ongoing sync

Two options. Re-run --once on a cron (the saved cursor makes it incremental), or push: HubSpot v3 webhook subscriptions are UI-configured under the private app and deliver to a Verity-minted webhook URL. Decode a recorded delivery with --webhook-file (only *.propertyChange events map to facts):

sh
python -m verity_ingest.connectors.hubspot --webhook-file payload.json --visibility 1,2

For a supervised schedule instead of cron, see Ongoing sync (Temporal).

Salesforce

Salesforce is ACL tier A (the *Share tables are readable) but the hardest reconstruction of the 20 surveyed systems. The connector does not reconstruct full effective visibility; enforcement uses the same fail-closed admin-assigned visibility_policy as HubSpot, and share rows ride along as approximated additive metadata.

1 · Create the credential

In your own org, create a Connected App and enable the OAuth client-credentials flow with a run-as integration user. This survived the post-Sept-2025 crackdown that made vendor-distributed apps harder; customer-created stays easy. Copy the consumer key and secret. Note your My Domain (e.g. acmeacme.my.salesforce.com). Salesforce's client-credentials response carries no expires_in and no refresh token, so the token is cached until a 401 (INVALID_SESSION_ID) triggers the shared retry-once hook.

2 · Configure

sh
export SF_MY_DOMAIN=acme                    # or the full acme.my.salesforce.com host
export SF_CLIENT_ID=...                       # Connected App consumer key
export SF_CLIENT_SECRET=...                   # Connected App consumer secret
export VERITY_URL=http://127.0.0.1:7717
export VERITY_TENANT_ID=$TENANT
export VERITY_ADMIN_TOKEN=...

3 · Run the initial backfill

One cycle runs SOQL through GET /services/data/v62.0/query for Account, Contact, and Opportunity with WHERE LastModifiedDate > <cursor> ORDER BY LastModifiedDate ASC. With no cursor that is a full backfill from epoch, following nextRecordsUrl (queryMore) pagination. Each non-null field becomes one keyed L1 fact. For Accounts changed in the window, the connector additionally fetches AccountShare rows and attaches best-effort share principals (user:005…, group:00G…) as approximated metadata. A failed share fetch logs a warning and never gates the facts.

sh
python -m verity_ingest.connectors.salesforce --once --visibility 1,2
# poll: 312 fact event(s), cursor -> 2026-07-09T18:04:57.000+0000 -> {written:312,...}
# --no-shares skips the AccountShare fetch; cursor -> .verity/salesforce_cursor
ACL honesty The share principals are additive, unenforced metadata for the future identity crosswalk, NOT enforced visibility. Full effective visibility (org-wide defaults, role hierarchy, sharing rules, manual/team shares, implicit sharing, territories) is not reconstructed. The enforced policy is the admin-assigned --visibility (admin-assigned), same fail-closed posture as HubSpot.

4 · Verify

Records land at the salesforce:<sobject> partition, keyed by the Salesforce Id. Same scoped check as HubSpot. An in-policy scope sees it; an out-of-policy scope gets a 404:

bash
curl -s "$VERITY_URL/v1/records/salesforce:Account/<account-id>/Name?scope_handle=$IN" | jq .value
# → "Acme Corporation"
curl -s "$VERITY_URL/v1/records/salesforce:Opportunity/<opp-id>/Amount?scope_handle=$IN" | jq .value
# → "84000.0"     (out-of-policy scope → 404, fail closed)

5 · Ongoing sync

Re-run --once on a cron or Temporal schedule; the saved cursor makes each cycle incremental (truncated to whole seconds, so a ≤1s window may replay safely into keyed upserts). The push lane, Salesforce CDC over the Pub/Sub API (gRPC), is a documented no-op in this poll-first connector; the truth lane reconciles everything. Live-org validation awaits a design partner (see Status & honesty).

Google Drive

Drive is ACL tier A and the mirroring proof: permissions.list is best-in-class, so documents carry mirrored provenance. A Verity scope inherits exactly the Drive permissions on each file. This is the one connector with no admin-assigned --visibility: visibility comes from the source ACL, resolved through the principal registry.

1 · Create the credential

In your own Google Cloud project, create a service account, download its JSON key, enable the Google Drive API in that project, and enable domain-wide delegation in your Workspace admin console for the read-only scope https://www.googleapis.com/auth/drive.readonly. (Without DWD, the service account itself must be granted access, e.g. added to shared drives; if the API isn't enabled, the first sync 403s with "API … has not been used in project … or is disabled"; enable it and retry.) No vendor OAuth app, ever.

2 · Configure

sh
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
export VERITY_URL=http://127.0.0.1:7717     # the Drive runner otherwise defaults to :8080
export VERITY_TENANT_ID=$TENANT
export GDRIVE_DELEGATED_SUBJECT=admin@acme.example  # the workspace user to impersonate (DWD)
export GDRIVE_ANYONE_MAPS_TO=org:everyone     # optional; default: anyone-shared quarantines

3 · Run the initial backfill

The first --once bootstraps: with no cursor, poll fetches a start page token via changes.getStartPageToken and returns with no events. History before the token is the reconciliation crawl's job, not the change feed's. The crawl (files.list over trashed = false) is what pulls your existing corpus: per file it fetches metadata, then the ACL via permissions.list, then, only if the ACL resolves, content (Google Docs exported as text/plain; text/* and JSON downloaded directly). Preview the exact request bodies first with --dry-run:

sh
python -m verity_ingest.connectors.gdrive --once --dry-run   # prints POST /v1/ingest/documents bodies
python -m verity_ingest.connectors.gdrive --once             # deliver for real; cursor -> .verity/gdrive_cursor.json
# gdrive: delivered 91 request(s); cursor -> .verity/gdrive_cursor.json

Each Drive permission maps to a canonical principal, resolved to an int token through POST /v1/admin/principals (the HttpRegistry), or a local --principal-map JSON for offline runs:

Drive permissionMaps to
type=useruser:<email>
type=groupgroup:<email> (nested-group closure is the Identity Plane's job)
type=domaindomain:<domain>
type=anyonequarantine unless GDRIVE_ANYONE_MAPS_TO is set
unknown / unmappablequarantine (the whole envelope, fail closed)
tier A: mirrored No --visibility flag: visibility is the real Drive ACL. If every principal on a file is unmappable (or the ACL is unresolvable, e.g. an unmapped anyone-link), the document quarantines: the connector posts it with no visibility field and the server holds it. ACL-before-content means bytes are never pulled for an item that will quarantine.

4 · Verify

Mirrored Drive docs are recall-searchable, pre-filtered by the mirrored ACL. A caller whose resolved principals intersect the file's Drive permissions gets hits; a caller who is not on the file gets zero, because the ACL became an in-index pre-filter (see Permissions → ACL inheritance).

bash
# a scope for a principal on the file → the doc is recalled
curl -s -X POST "$VERITY_URL/v1/recall" -H 'content-type: application/json' \
  -d "{\"scope_handle\":\"$IN\",\"text\":\"onboarding runbook\",\"k\":5}" | jq length
# → 3
# a scope for a principal NOT on the file → 0 hits (mirrored ACL excluded it in the index)
curl -s -X POST "$VERITY_URL/v1/recall" -H 'content-type: application/json' \
  -d "{\"scope_handle\":\"$OUT\",\"text\":\"onboarding runbook\",\"k\":5}" | jq length
# → 0

5 · Ongoing sync

Run without --once to loop at --interval seconds (default 300); each cycle drives changes.list from the saved page-token cursor and re-emits changed files (permission drift shows up as re-emitted envelopes; the server diffs). The changes.watch push lane needs a public HTTPS endpoint and 7-day channel renewal and is a documented no-op today. For a supervised schedule, use Temporal below.

Gmail

Same BYOT service account and domain-wide delegation as Google Drive, with a different scope and command. In your Cloud project enable the Gmail API, and grant the SA the read-only scope https://www.googleapis.com/auth/gmail.readonly in your Workspace admin console (DWD), impersonating the mailbox owner (--subject). Each message is keyed by its RFC822 Message-ID, so the same email seen in two connected mailboxes indexes once; the ACL mirrors the participants: From/To/Cc become user:<email> visibility, mirrored provenance; a message whose participants don't resolve quarantines rather than indexing permissively.

sh
export GOOGLE_APPLICATION_CREDENTIALS=/path/to/service-account.json
python -m verity_ingest.connectors.gmail --backfill --subject you@acme.example   # --once to poll · --dry-run to preview

Google Workspace directory sync Identity Plane

The content connectors above emit group:<email> ACLs but deliberately don't expand them; nested-group closure is this connector's job. Same SA + DWD: enable the Admin SDK API and grant the read-only scopes admin.directory.user.readonly + admin.directory.group.readonly. It reconciles users and groups from the Admin SDK and writes direct membership edges into the SpiceDB ReBAC graph, nesting preserved. SpiceDB owns the transitive closure, and the reconcile interval is the group-membership freshness bound. Proven live against a real Google Workspace directory: a doc shared with a group resolves to its nested members via open_scope, non-members provably dark. Run it supervised as a server-owned worker with verity-cli dev --directory.

sh
python -m verity_ingest.connectors.gdirectory --once --subject admin@acme.example --domain acme.example

Microsoft Entra ID directory sync Identity Plane

The Microsoft analog of the Google sync, for tenants whose groups live in Entra (Azure AD). BYOT app registration: in Entra, register a single-tenant app, add the application permissions User.Read.All + Group.Read.All + GroupMember.Read.All (read-only), and click Grant admin consent. The client secret lives in a 0600 file whose path is passed by env; the server never reads or logs it. It reconciles users, groups, and direct membership edges (SpiceDB owns the transitive closure), keyed on the immutable objectId, never UPN or mail. Freshness rides Graph delta, folded into a persisted full snapshot so a member removed by deleting the user object (a removal Graph's group-delta stream never reports) is still purged. Guests are excluded by a four-part gate (a B2B guest converted to Member is still excluded), and the guest-excluded tenant token is materialized explicitly. Proven live against a real (scratch) Entra tenant: nested closure, guest exclusion, and the delete-a-user edge purge. This is the identity plane the SharePoint/OneDrive content connector below resolves its group grants through.

sh
export ENTRA_TENANT_ID=<directory-tenant-guid> ENTRA_CLIENT_ID=<app-client-id>
export ENTRA_CLIENT_SECRET_FILE=~/.verity/entra-client-secret   # 0600; path only, never logged
python -m verity_ingest.connectors.entra_directory --once   # --dry-run to preview · supervised via /v1/admin/planes/entra-directory/start

SharePoint / OneDrive

The Microsoft content connector, riding the Entra identity plane above. SharePoint is ACL tier A for items whose permissions Graph returns in full: per-item grants carry mirrored provenance, resolved to Entra principals, so a Verity scope inherits the item's real SharePoint permissions including transitive group membership and sharing links. Proven live end to end on a real (scratch) tenant, including deletion-to-dark: delete a document at the source and it stops resolving on the next cycle, with a retire ledger to show for it. No --visibility: visibility is the source ACL.

1 · Create the credential

Reuse the same Entra app registration as the directory sync, with the per-site Sites.Selected posture: in the Azure portal grant the app access only to the specific sites you name, and provide the client secret in a 0600 file whose path is passed by env (the server never reads or logs it). The completeness canary is the proof that Graph returned a full ACL, not the scope name, so a drive without one fails closed rather than trusting a partial permission list.

2 · Configure

sh
export ENTRA_TENANT_ID=contoso.onmicrosoft.com ENTRA_CLIENT_ID=<app-guid>
export ENTRA_CLIENT_SECRET_FILE=~/.verity/entra-client-secret  # 0600; path only, never logged
export SHAREPOINT_SITE_IDS="contoso.sharepoint.com,<site-guid>"  # Sites.Selected: the sites to crawl
export SHAREPOINT_TENANT_GUID=<tenant-guid>               # unset: the everyone-except-external claim poisons
export SHAREPOINT_CANARIES_FILE=~/.verity/sp-canaries.json  # {driveId:{item_id,expected_user_oid}}: G1 completeness
export VERITY_URL=http://127.0.0.1:7717 VERITY_API_KEY=$KEY VERITY_TENANT_ID=$TENANT

3 · Run the initial backfill

--backfill runs the full per-drive delta crawl and stamps the reconcile SLA; run it first, then --once (or the poll loop) for steady state. As with Drive, ACL comes before content: a quarantined item never has its bytes fetched. A drive with no configured canary quarantines wholesale, because Graph returns a caller-filtered partial ACL to under-privileged callers with a 200, and an incomplete mirror we cannot vouch for must be held, not guessed.

sh
python -m verity_ingest.connectors.sharepoint --backfill --dry-run  # preview POST bodies
python -m verity_ingest.connectors.sharepoint --backfill            # first: stamps the reconcile SLA
python -m verity_ingest.connectors.sharepoint --once                # then: incremental delta
tier A: mirrored, with honest edges SharePoint-native site groups currently quarantine rather than resolve (the SP-REST cert-auth lane is not built); change subscriptions are not wired, so this is the poll lane only. Both are over-hide, never leak.

4 · Verify

Same scoped check as Drive: a caller whose resolved Entra principals intersect the item's SharePoint permissions gets hits; a caller who is not on the item gets zero, because the mirrored ACL became an in-index pre-filter. This is the exact chain proven live: a PDF shared with a nested Entra group is recalled by a member and stays dark for a non-member.

5 · Ongoing sync

Run without --once to loop at --interval (default 300s). A detected deletion or a permission narrowing is parked and drained through POST /v1/admin/retire on the next cycle, so the boundary follows the source. Live-org site-group resolution and push subscriptions are the remaining lanes; see Status & honesty.

Slack

Slack is ACL tier B: a channel's membership is the audience. Each message is stored with a single group:slack-channel-<id> visibility, and membership is synced like a mini-directory, so a member who joins can recall the channel's history and someone who leaves loses it, enforced retroactively through the same permission graph. Fixture-tested and live-validated once. No --visibility: it comes from channel membership.

1 · Create the credential

One command, about three minutes. The wizard walks you through creating a Slack app from a manifest (Socket Mode on) and stores the tokens 0600 in ~/.verity/config.toml under [connectors.slack]:

sh
verity-cli connect slack

The app needs the bot scopes channels:history, channels:read, channels:join, groups:history, groups:read, users:read, users:read.email. Invite the bot to the channels you want mirrored (it can self-join public channels; private channels need an invite). This is your consent surface: Verity remembers exactly the channels you add it to.

2 · Configure

The wizard already wrote the tokens, so only the Verity vars remain. To use an env token instead, set SLACK_BOT_TOKEN (xoxb-...), which overrides the config file.

sh
export VERITY_URL=http://127.0.0.1:7717 VERITY_API_KEY=$KEY VERITY_TENANT_ID=$TENANT
python -m verity_ingest.connectors.slack --backfill   # then --once to poll · --dry-run to preview
tier B: membership A user, bot, guest, or deleted account that does not resolve to an email confers nothing (narrowing, never poison), and Slack never mints or redirects a canonical identity another directory sync vouched. Shared and external-org channels quarantine wholesale.

3 · Verify

A caller who is a member of the channel recalls its messages; a caller who is not gets zero, because the channel-group token is an in-index pre-filter. Add or remove the bot, or move a member in and out, and the boundary follows on the next sync.

4 · Ongoing sync

Run without --once to loop at --interval (default 300s). Deletions and channel removals are parked and drained through POST /v1/admin/retire. DMs and private channels the bot is not in are out of scope by design; the Socket-Mode push lane is reserved for a later release, so this is the poll lane today.

Zoom

Zoom mirrors cloud-recording transcripts. Zoom exposes no per-recording audience, so this connector is honest about being a reconstruction: visibility is derived from the recording's share setting plus an operator-declared token, carried as admin-assigned, not a mirrored source ACL. It is fixture-verified and not yet validated against a live Zoom account (that needs a Pro+ seat with cloud recording and audio transcripts on). Fail-closed and over-hiding by construction.

1 · Create the credential

In your own Zoom account, create a Server-to-Server OAuth app (no marketplace review). Note the Account ID, Client ID, and Client Secret; the app needs cloud-recording read, past-participant read, and user:read:user:admin. The client secret lives in a 0600 file whose path is passed by env.

2 · Configure

sh
export ZOOM_ACCOUNT_ID=<acct> ZOOM_CLIENT_ID=<client-id>
export ZOOM_CLIENT_SECRET_FILE=~/.verity/zoom-client-secret  # 0600; path only, never logged
export ZOOM_USER_IDS="host1@acme.com,host2@acme.com"       # whose cloud recordings to mirror
export ZOOM_INTERNAL_MAPS_TO=group:everyone@acme.com        # where "share internally" maps; unset: those quarantine
export VERITY_URL=http://127.0.0.1:7717 VERITY_API_KEY=$KEY VERITY_TENANT_ID=$TENANT

3 · Run the initial backfill

--backfill walks each host's recordings in month-sized windows (Zoom caps the list API at a one-month range) and stamps the reconcile SLA; then --once for incremental. The transcript VTT is parsed into speaker turns; the recording UUID is the stable document key.

sh
python -m verity_ingest.connectors.zoom --backfill   # then --once to poll · --dry-run to preview
reconstructed audience none maps to the host alone, internally to ZOOM_INTERNAL_MAPS_TO plus the host (an admin-assigned policy), publicly quarantines, and anything unrecognized quarantines. Attendance-derived participant audiences are an explicit opt-in and labelled approximated.

4 · Ongoing sync

Run without --once to loop at --interval (default 300s). A recording that disappears from the source list, or a trashed/deleted recording, is parked and drained through POST /v1/admin/retire. Live-account validation is the open item; see Status & honesty.

Postgres / MySQL

Two on-ramps for a relational database, both landing in the same deterministic keyed L1 path. Pick by whether you need history and ordering.

The pg_net trigger (CDC-lite, convenience)

One copy-paste trigger POSTs row changes to a minted webhook URL: no replication slot, no Kafka, no connector process. Works on Supabase (pg_net preinstalled) or any Postgres with pg_net installed. The full snippet is docs/snippets/pg-net-trigger.md; the shape:

Postgres → Verity in three steps

1Mint the webhook URL. Visibility is bound into the URL and can never be widened by a payload. The token in the URL is the credential, shown once.

sh
verity-cli webhook mint pg:orders --visibility 1

2Enable pg_net and install the trigger, substituting the minted URL, your table, and its primary key. It emits Verity's native facts shape: one fact per column, keyed (source="pg:orders", entity_id=<pk>, field). INSERTs send every column; UPDATEs send only what changed.

sql
create extension if not exists pg_net;
-- verity_row_to_facts() + AFTER INSERT OR UPDATE trigger, see the snippet file
perform net.http_post(
  url  := 'https://verity.example.com/wh/<token>',
  body := jsonb_build_object('facts', facts),
  headers := '{"Content-Type": "application/json"}'::jsonb);

3Verify it lands. A changed value supersedes the old row bi-temporally (never UPDATE-in-place); valid_from tracks commit time.

sh
verity-cli query "order 1042"
honest limits The trigger is fire-and-forget (net.http_post is async; gaps possible), sends no DELETEs, and has no backfill: triggers see changes from now on, not history. For existing rows, do a one-time UPDATE … SET id = id to fire the trigger across the table, or use the Debezium lane below. Provenance is admin-assigned (the mint-time visibility).

The Debezium envelope (the truth lane)

Already running CDC, or need ordering, snapshots (backfill), and deletes? Point a Debezium Server HTTP sink (or a small Kafka consumer) at the admin ingest endpoint. Debezium's initial snapshot is your backfill; it accepts a single envelope or a JSON array, upserts row facts through the same bi-temporal path, and handles op:"d" by retiring the entity's facts.

bash
curl -s -X POST "$VERITY_URL/v1/ingest/debezium?tenant_id=$TENANT&pk=id" \
  -H 'content-type: application/json' \
  -H "authorization: Bearer $VERITY_ADMIN_TOKEN" \
  -d '{"op":"u","source":{"connector":"postgresql","table":"orders","ts_ms":1752000000000},
       "after":{"id":"1042","status":"paid","amount":99.00}}'
# → { "facts_inserted":2, "facts_superseded":0, "facts_unchanged":0, "facts_retired":0 }
ACL tier Debezium facts are mirrored when the schema declares principal columns via a source manifest, or admin-assigned under a static policy. The ?pk= query param overrides the primary-key field (default id); the L1 partition is <connector>:<table> (e.g. postgresql:orders). Measured POST-to-queryable: 31 ms on the dev profile.

Bulk documents

Loading an existing document corpus (a folder of markdown, a wiki export, PDFs already extracted to text) is the convenience lane: snapshot-grade, no per-object ACLs, so visibility is always an explicit admin decision (admin-assigned).

verity-cli add

add takes a file, a directory (recursive over .txt/.md/.json/.csv/.html, capped at 200 files), an http(s) URL, or stdin. --visibility is required by the parser and has no default: Verity never guesses who may see a memory. It mints a short-lived scope whose principals are exactly your tokens and uploads under it.

sh
verity-cli add ./handbook --visibility 1                        # a whole directory, org-wide
verity-cli add onboarding.md --visibility 7,11 --entity account:acme  # narrow + entity tag

Framework sinks

If your corpus already loads through a framework, six thin adapters turn its readers into Verity sinks: LlamaIndex, LangChain, LangGraph, CrewAI, Google ADK, OpenAI Agents. Every one takes a required visibility_policy with no default (loaders strip source ACLs by construction, so this lane is always policy-based; missing policy quarantines). Point hundreds of community loaders at permission-aware memory:

python
from verity_llamaindex import VerityVectorStore

store = VerityVectorStore(
    verity_url="http://127.0.0.1:7717",
    tenant_id=TENANT,
    visibility_policy=[1],          # REQUIRED, no default; missing → quarantine
    admin_token=ADMIN_TOKEN,
)
# every LlamaHub reader now writes into permission-aware Verity memory

See Ingestion → framework sinks for all six adapters. For file uploads without a framework, POST /v1/files is a multipart upload under a scope handle (paragraph-chunked, embedded). Verify with a scoped verity-cli query.

Any internal system

A homegrown app, a cron job, an ops tool: anything that can POST JSON becomes a source in ~60 seconds, no connector code and no OAuth. The visibility policy is bound at mint time into the URL; the blast radius is one URL, instantly revocable.

Internal system → source in two calls

1Mint the URL (admin-gated). The raw token is returned once; only its sha256 persists.

bash
curl -s -X POST "$VERITY_URL/v1/webhooks" -H "authorization: Bearer $VERITY_ADMIN_TOKEN" \
  -H 'content-type: application/json' \
  -d '{"tenant_id":"'"$TENANT"'","name":"ops-tool","visibility":[11],
       "entity_scope":["account:acme"]}'
# → { "webhook_id":"0192...", "url":"/wh/n8sQ...raw-token" }

2POST the native payload to the minted URL. No auth header; the path token is the credential. Accepts content/observation (text → a chunk), facts[] (keyed L1 upserts), entities[], and an optional visibility[] that may narrow but never widen the bound set.

bash
curl -s -X POST "$VERITY_URL/wh/n8sQ...raw-token" -H 'content-type: application/json' \
  -d '{"content":"Acme signed the pilot agreement for the fall rollout.",
       "entities":["account:acme"]}'
# → 200 { "episode_id":"...", "chunks_indexed":1, "facts_written":0 }

Provenance is admin-assigned (the visibility was an explicit admin decision at mint). Unknown shapes return 202 {"quarantined":true}: preserved for review, never permissively indexed. Revoke instantly with DELETE /v1/webhooks/{id}. For a reviewable, structured mapping over the same endpoint (with tier-enforced ACL policy), graduate to a source manifest.

Ongoing sync (Temporal)

Once a backfill has landed, promote the connector from ad-hoc --once runs to a supervised schedule. The orchestration plane runs a Temporal worker plus one Schedule per connector; the cursor lives in workflow state instead of a .verity/*_cursor file, and delivery stays at-least-once (the next cursor is returned only after the sink succeeds).

sh
# which connectors sync, and how often, fail closed: EMPTY means nothing syncs
export VERITY_CONNECTORS=hubspot,salesforce,gdrive
export VERITY_SYNC_INTERVAL=300              # default poll seconds
export VERITY_SYNC_INTERVAL_HUBSPOT=60       # per-connector override
# the orchestration runners read each connector's OWN policy env, no default:
export HUBSPOT_VISIBILITY=1,2
export SALESFORCE_VISIBILITY=1,2

# 1) run the worker (scale horizontally by running more on the same task queue)
python -m verity_ingest.orchestration.worker

# 2) apply the schedules (dry-run without --apply; overlap policy SKIP)
python -m verity_ingest.orchestration.schedules --apply
# verity-sync-hubspot: created   verity-sync-salesforce: created   verity-sync-gdrive: created
supervisor semantics One Schedule (verity-sync-<connector>) supervises one forever-looping workflow (connector-sync-<connector>); overlap policy SKIP guarantees never more than one sync chain per connector. A restarted chain resumes with cursor=None: HubSpot/Salesforce replay from epoch into keyed upserts; Drive re-arms the change feed and the reconciliation crawl covers the gap. Temporal needs TEMPORAL_ADDRESS/TEMPORAL_NAMESPACE (defaults localhost:7233 / default).

System comparison

The one-glance table: credential, the initial-backfill command, freshness, and the ACL contract for each system.

SystemCredentialInitial backfillFreshnessACL tiervisibility_policy
HubSpot Service Key (HUBSPOT_SERVICE_KEY) python -m verity_ingest.connectors.hubspot --once --visibility … poll + UI webhooks C · admin-assigned required, no default
Salesforce Connected App, client-credentials (SF_MY_DOMAIN/SF_CLIENT_ID/SF_CLIENT_SECRET) python -m verity_ingest.connectors.salesforce --once --visibility … poll (Pub/Sub later) A · admin-assigned + approximated shares required, no default
Google Drive service account + DWD (GOOGLE_APPLICATION_CREDENTIALS) python -m verity_ingest.connectors.gdrive --once poll (watch later) A · mirrored n/a · ACL is the source
SharePoint / OneDrive Entra app, Sites.Selected (ENTRA_CLIENT_ID/ENTRA_CLIENT_SECRET_FILE) python -m verity_ingest.connectors.sharepoint --backfill poll (subscriptions later) A · mirrored (site groups quarantine) n/a · ACL is the source
Slack verity-cli connect slack (app-from-manifest) python -m verity_ingest.connectors.slack --backfill poll (Socket Mode later) B · mirrored (channel membership) n/a · membership is the source
Zoom S2S OAuth app (ZOOM_ACCOUNT_ID/ZOOM_CLIENT_ID/ZOOM_CLIENT_SECRET_FILE) python -m verity_ingest.connectors.zoom --backfill poll reconstructed · admin-assigned operator token (ZOOM_INTERNAL_MAPS_TO)
Postgres/MySQL (Debezium) admin token + CDC pipeline Debezium snapshot → POST /v1/ingest/debezium ~31 ms to queryable mirrored (manifest) / admin-assigned manifest tier or static
Postgres (pg_net) minted webhook URL one-time UPDATE to fire the trigger (no native backfill) push (fire-and-forget) admin-assigned mint-time visibility
Bulk documents scope handle / admin token verity-cli add <file|dir|url> --visibility … manual / one-shot admin-assigned required, no default
Any internal system minted webhook URL replay history as POST /wh/{token} push (receipt-time) admin-assigned mint-time visibility

Status & honesty

Verity is pre-1.0 and every claim on this site is measured, never quoted. Where a connector is fixture-verified vs live-validated:

next Data's in and correctly scoped. Now shape exactly who can see it: the permission model is where the guarantee lives. Or return to the ingestion mechanism reference.