Thread is somehow interfering with a while loop called after the thread is started

Dan McInerney dan.h.mcinerney at gmail.com
Sun Mar 16 09:24:45 EDT 2014


Coming back to this a long time later. I figured it out a little bit after
I posted this. I wasn't aware that q.try_run() within the nfqueue module
was a blocking call
https://www.wzdftpd.net/redmine/projects/nfqueue-bindings/wiki/Examples.
I'm not sure I was even aware of what it meant to be blocking vs
nonblocking. Either way, someone from #python looked at my code and told me
nfqueue is a blocking call and I should look into something like twisted.

Twisted is a complex module but I spent a bunch of time looking at it and
eventually had some working code implementing the absolute bare essentials
as I needed them to go asynchronous. This question eventually progressed
into https://github.com/DanMcInerney/LANs.py.

Speaking of network concurrency, my recent project has been from a web
scraper point of view and I think gevents+requests or gevents+mechanize are
the way to go for that somewhat-related topic. erequests won't import due
to a bug, grequests has no way to handle exceptions in the fetching of
responses, treq can't find and fill out form logins (maybe with a POST,
haven't explored that route), and scrapy is way too big and complex for my
latest project (https://github.com/DanMcInerney/shodan_pharmer), eventlets
I'm sure CAN work with libraries other than urllib2 but it implements and
imports its own urllib2 and it is not a favorable time:effort ratio for me
to try and implement that.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-list/attachments/20140316/45cd349e/attachment.html>


More information about the Python-list mailing list