Share unpickleable object across processes

[Israel Brewster]

On Nov 2, 2017, at 12:36 PM, Chris Angelico <rosuav at gmail.com> wrote:
> 
> On Fri, Nov 3, 2017 at 7:35 AM, Israel Brewster <israel at ravnalaska.net <mailto:israel at ravnalaska.net>> wrote:
>> On Nov 2, 2017, at 12:30 PM, Chris Angelico <rosuav at gmail.com> wrote:
>>> 
>>> On Fri, Nov 3, 2017 at 5:54 AM, Israel Brewster <israel at ravnalaska.net> wrote:
>>>> I have a Flask/UWSGI web app that serves up web socket connections. When a web socket connection is created, I want to store a reference to said web socket so I can do things like write messages to every connected socket/disconnect various sockets/etc. UWSGI, however, launches multiple child processes which handle incoming connections, so the data structure that stores the socket connections needs to be shared across all said processes. How can I do this?
>>>> 
>>> 
>>> You're basically going to need to have a single process that manages
>>> all the socket connections. Do you actually NEED multiple processes to
>>> do your work? If you can do it with multiple threads in a single
>>> process, you'll be able to share your socket info easily. Otherwise,
>>> you could have one process dedicated to managing the websockets, and
>>> all the others message that process saying "please send this to all
>>> processes".
>> 
>> Ok, that makes sense, but again: it's UWSGI that creates the processes, not me. I'm not creating *any* processes or threads. Aside from telling UWSGI to only use a single worker, I have no control over what happens where. But maybe that's what I need to do?
>> 
> 
> That's exactly what I mean, yeah. UWSGI should be able to be told to
> use threads instead of processes. I don't know it in detail, but a
> cursory look at the docos suggests that it's happy to use either (or
> even both).

Gotcha, thanks. The hesitation I have there is that the UWSGI config is a user setting. Sure, I can set up my install to only run one process, but what if someone else tries to use my code, and they set up UWSGI to run multiple? I hate the idea of my code being so fragile that a simple user setting change which I have no control over can break it. But it is what it is, and if that's the only option, I'll just put a note in the readme to NEVER, under any circumstances, set UWSGI to use multiple processes when running this app and call it good :-)

> 
> ChrisA
> -- 
> https://mail.python.org/mailman/listinfo/python-list <https://mail.python.org/mailman/listinfo/python-list>