[melbourne-pug] Faust - Python Stream Processing

[Sam Bourne]

http://faust.readthedocs.io/en/latest/playbooks/vscelery.html

This overview appears to suggest it's everything celery is plus some. I
can't speak to everyones use of celery, but I believe I could drop this in
as a replacement if it wasn't python3 only.

I love Kafka Streams


Thats good to hear! I haven't used Kafka before and was afraid it could be
difficult to setup and maintain (in the way say an ELK stack is). There are
talks <https://news.ycombinator.com/item?id=17656812> about using
redis-streams <https://redis.io/topics/streams-intro> for the broker or at
least allowing it to be configurable - which adds to my assumption Kafka is
burdensome. It's good to know your experiences have been positive.


On Sat, Aug 4, 2018 at 3:59 PM Rory Hart <hartror at gmail.com> wrote:

> I love Kafka Streams and keep looking for an excuse to deploy them, nice
> to know there is a Python option.
>
> > It appears to be a spiritual successor to celery
>
> Am I right in thinking Faust wouldn't be a idea for the batch processing
> use cases that celery serves?
>
> On Sat, 4 Aug 2018 at 15:44, Sam Bourne <sambvfx at gmail.com> wrote:
>
>> http://faust.readthedocs.io/en/latest/
>>
>> This is super fascinating! It appears to be a spiritual successor to
>> celery <http://www.celeryproject.org/> (it's written by the same guy
>> even), but it's built around kafka
>> <http://faust.readthedocs.io/en/latest/userguide/kafka.html> for the
>> queues and rocksdb
>> <http://faust.readthedocs.io/en/latest/userguide/tables.html#tables> for
>> state.
>>
>> It is used at Robinhood <https://robinhood.com/> to build high
>>> performance distributed systems and real-time data pipelines that process
>>> billions of events every day.
>>
>>
>> -Sam
>> _______________________________________________
>> melbourne-pug mailing list
>> melbourne-pug at python.org
>> https://mail.python.org/mailman/listinfo/melbourne-pug
>>
> _______________________________________________
> melbourne-pug mailing list
> melbourne-pug at python.org
> https://mail.python.org/mailman/listinfo/melbourne-pug
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/melbourne-pug/attachments/20180806/3621414d/attachment.html>