Context
BSFG fetch semantics are unary paged pull. That means progress over a retained stream must be tracked somewhere durable and replayable. The architecture needs a clear answer to:
- how a fetch client identifies its read position
- whether progress survives restarts and failover
- whether multiple instances can share one logical consumer identity
- how
ConfirmReceiptrelates to stored fetch progress
JetStream consumers are the server-side mechanism that track delivery and acknowledgement state. Durable consumers persist progress on the server, while ephemeral consumers do not persist long-term state and are intended for short-lived use. :contentReference[oaicite:0]{index=0}
Options Considered
| Option | Description | Benefits | Drawbacks |
|---|---|---|---|
| Client-only cursor | Keep offset and progress only in the client or external database. | minimal broker-side setup implementation freedom for clients |
progress semantics become fragmented harder failover and recovery weak alignment with JetStream model |
| Ephemeral consumers | Create short-lived consumers on demand for each fetch session. | simple for ad hoc inspection little persistent state to manage |
progress does not persist reliably poor fit for durable cross-zone replication restart semantics are weak |
| Stateless stream reads only | Read raw stream positions directly with no consumer abstraction. | appears simple conceptually close to a bare log model |
acknowledgement floor is externalized recovery and shared-consumer behavior become custom protocol work gives up useful JetStream primitives |
| Durable named consumers (Selected) | Each logical fetcher uses a stable consumer identity whose progress is persisted server-side. | progress survives restarts clean recovery semantics good fit for pull-based batch processing aligns with server-side acknowledgement tracking |
consumer naming and lifecycle must be governed administrative sprawl is possible if names are unmanaged |
Decision
BSFG fetch progress uses durable named consumers.
FetchFacts(consumer_name, ...)
ConfirmReceipt(consumer_name, ...)
A consumer_name identifies the logical reader whose progress is tracked against the stream. That identity is durable and survives client restarts.
Consequences of this choice:
- fetch position is server-side state, not only client memory
- multiple client instances may share one durable consumer identity when appropriate
- consumer progress and acknowledgement floor remain observable at the log substrate
- ad hoc inspection may still use ephemeral tooling, but it is not the canonical BSFG path
Consequences
Benefits:
- restarts and failover do not lose fetch progress
- boundary replication remains operationally stable
- consumer identity is explicit in the API contract
- the design aligns with pull-based batch consumption in JetStream :contentReference[oaicite:1]{index=1}
Tradeoffs:
- consumer names become part of architecture governance
- unused durable consumers must be cleaned up administratively
- client implementers must understand shared-vs-dedicated consumer identity semantics