[Web-SIG] Nodejs cluster

[est]

Thank you guys for all the info.

Twisted Web is cool, but there's it's not as interchangeable as WSGI. You
can use whatever framework django/flask/webpy/bottle you like and plug it
in as long as it's PEP-333 compatible. Not so much for Twisted and
Tornado's own ioloop engine.

IPython.parallel

http://ipython.org/ipython-doc/stable/install/install.html#dependencies-for-ipython-parallel-parallel-computing

It's based on ZeroMQ(PyZMQ), and the `ssh` command. I don't think that's
lightweigh enough for busy web clusters.

By QMachine I assume that's

https://github.com/wilkinson/qmachine

For web server cluster it's really not a good idea to amplify HTTP
requests. One client request amplifies several other HTTP requests on
server clusters.

What I propose is something like Zed Shawn's Mongrel2 project (
http://mongrel2.org/), use a very lightweight server-side serialization
protocol as cluster IPC, you can pass states/data between nodes (workers)
easily. It should be agnostic to framework or libraries, the objective is
to unite python modules in the realtime web world. Because for
request-response web world, a synchronized gateway like WSGI is good
enough, between each requests, share
nothing<https://docs.djangoproject.com/en/dev/faq/general/#does-django-scale>
.

But for realtime web, server side state is very much required. There need
to be a fd pool for DBs, external services, and stuff like Server-Side-Push
technologies.

Let's assume the following scenario:

One user submits a blog, his follower gets browser/iOS/Android push
notification. Because users are connected different nodes in one big
cluster, we need some kind of mechanism to broadcast this message.

In such an architecture we can write simpler code like this:

from django.db.models.signals import post_save

@receiver(post_save, sender=BlogPostModel)
def my_handler(sender, **kwargs):
    msg = "User X just posted a new blog, check it out at http://..."
    browser_followers.send(msg)
    ios_followers.send(msg)
    android_followers.send(msg)

Currently this library reall shines.

https://pypi.python.org/pypi/telegraphy/

Telegraphy architecture is like this:

[image: Inline image 1]

What I propose is to merge Web-app part and the AutobahnPython Gateway part
into *one* based on a community honored standard.





On Mon, Mar 17, 2014 at 8:24 PM, anatoly techtonik <techtonik at gmail.com>wrote:

> On Mon, Mar 17, 2014 at 3:53 AM, est <electronixtar at gmail.com> wrote:
> > The best we've got is something like Celery, or Telegraphy for Django
>
> Does IPython suits pattern better?
> http://ipython.org/ipython-doc/stable/parallel/parallel_intro.html
>
> > What do you guys think?
>
> I think not to reinvent the bicycle. Use PythonJS + QMachine to get to
> web client side + distributed processing without abandoning existing
> Python skills.
> --
> anatoly t.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/web-sig/attachments/20140318/2a4786b8/attachment.html>