Architecture
StreamWright has a small core: a runtime that reads a source and runs each stream's fetch → shape → write, a set of connectors, pluggable outputs, and an orchestration layer that schedules runs per user and network.
The layers
A run, end to end
Shared vocabulary across networks
One pipeline runs across many networks by mapping canonical node names to each network's streams:
| Canonical node | network_a | network_b | network_c |
|---|---|---|---|
accounts | accounts | customers | accounts |
campaigns | campaigns | campaigns | unsupported |
metrics | daily_metrics | stats | insights |
One pipeline uses the canonical names on the left. Each network's streams: map provides its own stream for each node, or marks it unsupported — so the same pipeline runs across networks that name (or lack) streams differently.
Separation of concerns
- Source = what to pull and how to shape it (declarative).
- Connector = how to talk to a system (sign-in, paging, retries).
- Output = where records go (a run-time choice).
- Orchestration = when and where runs happen (per user and network).
Each layer is independent, so you can add a connector, switch an output, or change scheduling without touching the others.