PythonKafkaCompose

PythonKafkaCompose is an upgrade of the amazing work done in liveMaps

It is a simple project composed by:

  • an instance of Kafka
  • a Pyhton Consumer
  • a Python Producer

Key Differencies

  • upgraded the infra using docker-compose
  • switched the library used to interact with kafka, from PyKafka (archived) to kafka-python

Getting Started

  1. Spin up Zookeper and Kafka
docker-compose up -d
  1. Setup environment

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
  1. Run the Producer
python producer.py
  1. Run the Consumer (in a new terminal)
python consumer.py

The consumer is a Flask app that expose and endpoint on localhost:5001

so open the browser and go to localhost:5001/topic/topic_test to see a stream of string coming from the topic

Roadmap

[ ] Dockerize Producer

[ ] Dockerize Consumer

[ ] Flask -> FastApi

[ ] Use Poetry

[ ] Think about include https://ntfy.sh/

GitHub

View Github