Message queues with nyx-queue

What is a message queue?

A message queue is a buffer that holds messages until a consumer is ready to process them. Unlike Pub/Sub (where messages are lost if nobody listens), a queue persists messages and guarantees delivery.

Producer ──► ENQUEUE "emails" "welcome:alice" ──► nyx-queue
                                                       │
                                                    ┌──┴──┐
                                                    │Queue│
                                                    │     │
                                                    │msg1 │
                                                    │msg2 │
                                                    │msg3 │
                                                    └──┬──┘
                                                       │ DEQUEUE
                                            ┌──────────┼──────────┐
                                            ▼          ▼          ▼
                                       Consumer 1  Consumer 2  Consumer 3

Key difference from Pub/Sub: each message goes to exactly one consumer (round-robin), not all of them.

Getting started

nyx-queue speaks the RESP protocol, so you use redis-cli to interact with it:

$ redis-cli -p 6381
127.0.0.1:6381> PING
PONG

Producing messages

Send a message to a queue with ENQUEUE:

127.0.0.1:6381> ENQUEUE emails "welcome:alice@example.com"
"msg_1712001234_1"

The command returns a unique message ID. The queue is created automatically if it does not exist.

Consuming messages

DEQUEUE retrieves the next message. It blocks until a message is available:

127.0.0.1:6381> DEQUEUE emails
1) "msg_1712001234_1"
2) "welcome:alice@example.com"

The response is a two-element array: the message ID and the payload. For non-blocking mode:

127.0.0.1:6381> DEQUEUE emails NOWAIT
(nil)

Acknowledging messages

After processing a message, acknowledge it with ACK:

127.0.0.1:6381> ACK emails msg_1712001234_1
OK

If processing fails, send NACK to return the message to the queue for redelivery:

127.0.0.1:6381> NACK emails msg_1712001234_1
OK

Automatic redelivery

If a consumer crashes without sending ACK or NACK, the message is automatically returned to the queue after 30 seconds (configurable with --ack-timeout). This provides at-least-once delivery — every message will be processed at least once.

Queue management

Persistence

Messages survive restarts. nyx-queue saves its state to a queue.ndb binary file using the same background-saver pattern as nyx-kv — every 60 seconds or after 100 changes. On restart, messages that were delivered but not acknowledged are reset to pending.

Example: background email processor

A web application enqueues emails. Three worker processes dequeue and send them:

# Producer (web app)
import redis
q = redis.Redis(port=6381)
q.execute_command("ENQUEUE", "emails", "welcome:alice@example.com")
q.execute_command("ENQUEUE", "emails", "receipt:order_123")
q.execute_command("ENQUEUE", "emails", "reset:bob@example.com")

# Consumer (email worker)
import redis
q = redis.Redis(port=6381)

while True:
    msg = q.execute_command("DEQUEUE", "emails")
    msg_id, payload = msg[0], msg[1]
    try:
        send_email(payload)
        q.execute_command("ACK", "emails", msg_id)
    except Exception:
        q.execute_command("NACK", "emails", msg_id)

Run three workers in parallel. nyx-queue distributes messages round-robin — no message is processed twice (unless NACKed).

Summary

• ENQUEUE queue payload adds a message and returns a unique ID.
• DEQUEUE queue blocks until a message is available.
• ACK marks a message as processed; NACK returns it for retry.
• Automatic redelivery after 30s ensures no messages are lost.
• Messages persist to disk and survive restarts.
• Round-robin delivery across multiple consumers.