Category Archives: akka-kafka

Design questions related to kafka

I looked the documents of kafka and I see that there is no way a consume instance can read specific messages from partition.

I have an use case where I need to spawn a topic (single partition) for each user, so there would be 10k online users at a time, there would be very less data per topic.

My questions is, what are limitations of having multiple topics (with 1 partition), I think this situation would cause heavy memory consumption and are they any other limitations? Basically the problem boils down to what are the scalability limitations of having multiple topics (hardware/software)?