DataFlow — Real-Time Streaming Data Platform
DataFlow is a zero-dependency, MIT-licensed real-time streaming data platform for the browser, published by Tekivex. It connects any live data source — WebSocket, Server-Sent Events, HTTP polling, WebTransport, or a built-in simulator — to any UI framework (React 18/19, Vue 3, Svelte 5, or vanilla JavaScript), and handles the hard parts of live data: backpressure, statistical anomaly detection, cell-change tracking, time-travel replay, multi-stream joins, and schema auto-inference.
Install
npm install @gridstorm/dataflow-core @gridstorm/dataflow-react
The headless core has zero runtime dependencies and is about 79 KB gzipped. Framework adapters are thin: React ~12 KB, Vue ~5 KB, Svelte ~4 KB gzipped.
Features
- Five adapters — WebSocket (reconnect + heartbeat + auth), SSE, HTTP polling (fixed / adaptive / long-poll), WebTransport (HTTP/3), and a seeded simulator.
- Backpressure — bounded ring buffer drained by a requestAnimationFrame scheduler, with oldest / newest / sample drop strategies.
- Anomaly detection — Z-score, IQR (Tukey fences), MAD, and static threshold methods, per column, over a rolling window. Plus run-length and burst (sustained) detection.
- Time-travel replay — record every frame, then scrub, seek, and play back at 0.1x to 16x speed.
- Multi-stream join — SQL-style inner / left / outer joins and N-way merge across streams.
- Schema auto-inference — detects number, boolean, timestamp, currency, and percentage column types from live samples.
- Framework adapters — React hooks (
useStream), Vue composables, and Svelte stores on one shared core. - No-code Analyst app — upload Excel / CSV / PDF, connect a live API with a form, and ask a bring-your-own LLM (OpenAI, Anthropic, Gemini, Ollama) to analyze the data. Everything stays in your browser.
Performance
Measured with the reproducible bench/ package on a
13th-gen Intel i3 laptop: without anomaly detection the engine
processes about 24,000–26,000 rows per second, and
2–3× that on Apple M-series hardware. With Z-score and IQR
anomaly detection enabled, throughput is about
1,500 rows per second — the bottleneck is per-row
quantile computation, not the engine itself.
Frequently asked questions
- Is DataFlow free?
- Yes. It is MIT-licensed and free forever. All npm packages are public.
- Does DataFlow need a backend?
- No. The simulated adapter generates realistic data in the browser; for real data, point any adapter at your existing endpoint. A ~250-line Node WebSocket reference server is included for testing.
- Which frameworks are supported?
- React 18/19, Vue 3.3+, and Svelte 5, all on one zero-dependency core.