Share unpickleable object across processes

Israel Brewster israel at ravnalaska.net
Thu Nov 2 14:54:43 EDT 2017


I have a Flask/UWSGI web app that serves up web socket connections. When a web socket connection is created, I want to store a reference to said web socket so I can do things like write messages to every connected socket/disconnect various sockets/etc. UWSGI, however, launches multiple child processes which handle incoming connections, so the data structure that stores the socket connections needs to be shared across all said processes. How can I do this?

Tried so far:

1) using a multiprocessing Manager(), from which I have gotten a dict(). This just gives me "BlockingIOError: [Errno 35] Resource temporarily unavailable" errors whenever I try to access the dict object.
2) Using redis/some other third-party store. This fails because it requires you to be able to pickle the object, and the web socket objects I'm getting are not pickle able.

In C I might do something like store a void pointer to the object, then cast it to the correct object type, but that's not an option in python. So how can I get around this issue?



-----------------------------------------------
Israel Brewster
Systems Analyst II
Ravn Alaska
5245 Airport Industrial Rd
Fairbanks, AK 99709
(907) 450-7293
-----------------------------------------------







More information about the Python-list mailing list