Shithanshu Mishra — Engineering Noteshttps://shithanshu.com/Backend, streaming and video systems, written down.enScaling WebSockets and SSE to Hundreds of Pods: Where Redis Pub/Sub Breaks and How to Fix Ithttps://shithanshu.com/journal/scaling-websockets-sse-redis-pubsub/https://shithanshu.com/journal/scaling-websockets-sse-redis-pubsub/Thu, 12 Nov 2026 00:00:00 GMTWhy realtime fan-out over Redis Pub/Sub degrades as a WebSocket/SSE gateway grows to hundreds of pods: O(pods) delivery, cluster broadcast, slow-subscriber disconnects, reconnect storms, and the fixes: sharded Pub/Sub, sequence numbers with replay, room affinity, backpressure.WebSocketSSERedisPub/SubScalabilityFrom MySQL to Elasticsearch: Building a Reliable CDC Pipeline That Scales to 1,000 RPShttps://shithanshu.com/journal/cdc-mysql-debezium-kafka-elasticsearch/https://shithanshu.com/journal/cdc-mysql-debezium-kafka-elasticsearch/Wed, 11 Nov 2026 00:00:00 GMTA practical design for streaming MySQL changes into Elasticsearch with Debezium and Kafka: ordering, idempotency, failure recovery, and an honest derivation of what 1,000 RPS means for CDC event rates.KafkaDebeziumMySQLElasticsearchCDCDesigning a Real-Time IoT Data Platform: MQTT, Kafka, ksqlDB and MongoDBhttps://shithanshu.com/journal/iot-mqtt-kafka-ksqldb-mongodb/https://shithanshu.com/journal/iot-mqtt-kafka-ksqldb-mongodb/Wed, 11 Nov 2026 00:00:00 GMTHow device telemetry travels from MQTT through Kafka and ksqlDB into MongoDB: QoS semantics, partitioning, event-time windows, time-series storage, and the failure modes in between.MQTTKafkaksqlDBMongoDBIoTInside an IP Camera: Understanding RTSP, Codecs, FPS, I-Frames and Video Streaminghttps://shithanshu.com/journal/ip-cameras-rtsp-video-streaming/https://shithanshu.com/journal/ip-cameras-rtsp-video-streaming/Wed, 11 Nov 2026 00:00:00 GMTHow an IP camera turns photons into an RTSP stream, what H.264/H.265, GOP and bitrate settings actually change, and how those choices propagate to VMS capacity, latency and ANPR or object-detection accuracy.RTSPH.264H.265ONVIFVideo AnalyticsKafka as a Distributed Log: Partitions, Replication, Consumer Groups and Replayhttps://shithanshu.com/journal/kafka-partitions-replication-consumers/https://shithanshu.com/journal/kafka-partitions-replication-consumers/Wed, 11 Nov 2026 00:00:00 GMTA systems-level guide to Kafka topics, partition ordering, producer acknowledgements, replication, consumer offsets, retention and replay, with interactive partition and consumer-group playgrounds.KafkaDistributed SystemsEvent StreamingConsumer GroupsRabbitMQ Beyond the Queue: Exchanges, Priorities, Persistence and Delayed Workhttps://shithanshu.com/journal/rabbitmq-routing-priority-persistence-delay/https://shithanshu.com/journal/rabbitmq-routing-priority-persistence-delay/Wed, 11 Nov 2026 00:00:00 GMTA practical guide to RabbitMQ routing with direct, topic and fanout exchanges, priority delivery, durable queues, message TTL and delayed retries with dead-letter exchanges.RabbitMQAMQPMessage QueuesDistributed SystemsRedis Beyond the Cache: Persistence, Pub/Sub and Streams, and What Each Actually Guaranteeshttps://shithanshu.com/journal/redis-persistence-pubsub-streams/https://shithanshu.com/journal/redis-persistence-pubsub-streams/Wed, 11 Nov 2026 00:00:00 GMTWhat Redis really promises once it holds data you cannot lose: RDB and AOF durability, async replication and failover loss, Pub/Sub at-most-once, lists as queues, and Streams consumer groups compared with Kafka.RedisStreamsPub/SubAOFPersistence