Connectors & readers

A connector supplies records to a stream. StreamWright ships SDK-backed API connectors and readers for files, object storage and databases — all used the same way: a stream's requests call them, steps shape the result, and exports write it out.

Three families

HTTP / REST APIs — generic REST API connector (restapi) for querying arbitrary web endpoints, webhook payloads, and external APIs without writing custom Python connector packages.
Ads APIs — Google, Microsoft, Meta, LinkedIn, Amazon, Apple, and OpenAI Ads. These use a shared vocabulary so one pipeline spans every network.
Readers — local files, Amazon S3, Google Cloud Storage, Azure Blob Storage, PostgreSQL, MySQL, MongoDB, and Delta Lake.

Family	Providers
HTTP / REST APIs	`restapi`
Ads APIs	`google_ads` · `microsoft_ads` · `meta_ads` · `linkedin_ads` · `amazon_ads` · `apple_ads` · `openai_ads`
Readers	`files` · `s3` · `gcs` · `azure_blob` · `postgres` · `mysql` · `mongodb` · `deltalake` · `google_sheets`

How a connector is used

Every connector follows the same shape in a stream — only provider/sdk and the arguments differ:

source.yaml
auth:
  provider: <connector>        # google_ads | postgres | s3 | ...
  # ...credentials come from secrets, settings from config

streams/example.yaml
requests:
  - sdk: <connector>           # the same provider
    # ...connector-specific arguments
transform:
  mode: page
  steps:
    - sql: SELECT ... FROM records
exports:
  - name: example

Allow-list connectors

Restrict which connectors a run may use with --allow-connector NAME (repeatable). By default any installed connector can be used.

Three families​

How a connector is used​

Three families

How a connector is used