Skip to main content

consumer

Kafka consumer for document processing events.

Classes​

KafkaConsumer​

Kafka consumer for document-uploaded events.

Uses a ThreadPoolExecutor to process messages asynchronously, keeping the Kafka listener responsive while document processing runs in parallel.

Constructor:

def __init__(self, message_handler: Callable[[dict], None], max_workers: int | None = None, max_messages: int | None = None) -> None

Methods​

start​

def start(self) -> None

Start listening for messages with async processing.

Initializes the thread pool executor and subscribes to the Kafka topic.

stop​

def stop(self) -> None

Stop the consumer and wait for active tasks.

get_active_task_count​

def get_active_task_count(self) -> int

Get the number of currently active processing tasks.

Returns: Number of tasks currently being processed