From 508721bd9735d94c899d9afbee1eb9dd725209c4 Mon Sep 17 00:00:00 2001 From: Emmanuel Ferdman Date: Thu, 2 Nov 2023 21:22:44 +0200 Subject: [PATCH] docs: Update location of `async_query_manager.py` (#25531) Co-authored-by: John Bodley <4567245+john-bodley@users.noreply.github.com> --- superset-websocket/README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/superset-websocket/README.md b/superset-websocket/README.md index a0c7563b1f..097e38506d 100644 --- a/superset-websocket/README.md +++ b/superset-websocket/README.md @@ -33,7 +33,7 @@ This implementation is based on the architecture defined in [SIP-39](https://git ### Streams -Async events are pushed to [Redis Streams](https://redis.io/topics/streams-intro) from the [Superset Flask app](https://github.com/preset-io/superset/blob/master/superset/utils/async_query_manager.py). An event for a particular user is published to two streams: 1) the global event stream that includes events for all users, and 2) a channel/session-specific stream only for the user. This approach provides a good balance of performance (reading off of a single global stream) and fault tolerance (dropped connections can "catch up" by reading from the channel-specific stream). +Async events are pushed to [Redis Streams](https://redis.io/topics/streams-intro) from the [Superset Flask app](https://github.com/preset-io/superset/blob/master/superset/async_events/async_query_manager.py). An event for a particular user is published to two streams: 1) the global event stream that includes events for all users, and 2) a channel/session-specific stream only for the user. This approach provides a good balance of performance (reading off of a single global stream) and fault tolerance (dropped connections can "catch up" by reading from the channel-specific stream). Note that Redis Stream [consumer groups](https://redis.io/topics/streams-intro#consumer-groups) are not used here due to the fact that each group receives a subset of the data for a stream, and WebSocket clients have a persistent connection to each app instance, requiring access to all data in a stream. Horizontal scaling of the WebSocket app requires having multiple WebSocket servers, each with full access to the Redis Stream data.