Market Data Connectors

What it is

Market data arrives as a stream of snapshots, updates, trades, and heartbeats. A consumer cannot safely use every message just because it passed through a WebSocket. Messages can arrive out of order, a sequence can contain a gap, or an order book can become stale after a connection problem.

This project provides the pieces around that stream: venue connectors, normalized market events, order-book state, integrity checks, and downstream sinks. It supports both direct event processing and a stateful projection that exposes whether an instrument is healthy or stale.

Main features

• Integrity checks: sequence continuity, checksum verification, heartbeat monitoring, timestamp gating, and silence watchdogs.
• Automatic recovery: invalidated instruments become stale, updates are quarantined, and a fresh snapshot can restore a healthy state.
• Multiple venues: WebSocket and REST connectors for cryptocurrency exchanges, FX sources, and reference-rate providers.
• Bounded operation: cancellation-aware workers and explicit byte limits for snapshots, catalogs, and compressed streams.
• Safe Rust: the workspace forbids unsafe code and includes deterministic mock servers and fault injectors for testing.

Why it exists

The difficult part of market data is not opening a connection. It is knowing when the data can no longer support a correct decision. A plain transport adapter usually leaves sequence gaps, stale quotes, and resynchronization rules to each application. That spreads the same failure logic across every consumer.

Market Data Connectors puts those rules into reusable components. Its main difference is that data integrity is treated as part of the event contract, not as an optional check added after the connector. Consumers can choose raw normalized events or use the shared stateful projection when they need healthy, current book data.