HowTo send emails reliably in a Python multi-threading application?

[Josiah Carlson]

> 1) How can one send mails reliable in a multi-threaded Python application?  
> Should the email package be used? Should smtplib be used or is it better to 
> call "sendmail" or "mail" directly? Should popen2 or os.system() be 
> preferred?

I don't think this matters one way or another.  It may be convenient to 
use smtplib, if only because it wouldn't rely on sendmail or mail.  The 
problem with smtplib is that if you get rid of threads (which I suggest 
later), smtplib could stall your program.


> 2) How to call an external tool (or many of them) and connect them via pipes 
> with each other and with the Python script in a multithreaded application? I 
> read the discussion http://www.python.org/doc/current/lib/
> popen2-flow-control.html several times now and think I followed the 
> suggestions given there, but it seems still to cause trouble for me.

Connect them the same way you would in a shell (as you seem to be doing 
already), with commands like:
command1 | command2 | command3

For getting the output from command3, I've always been a big fan of:
procs['unique_done'] = os.popen('command1 | command2 | command3 |'
                                 ' special unique_output unique_done')

Where special is a command that passes-though all information received 
from command3, writes it to unique_output, and when done, creates a file 
called unique_done, but doesn't write anything to stdout.

However often you want to, you can check for the existance of 
unique_done (as many of them as you want) and execute the following:

toss = procs['unique_done'].read()
procs['unique_done'].close()
del procs['unique_done']
fil = open('unique_output', 'rb')
output = fil.read()
fil.close()
os.remove('unique_output')
os.remove('unique_done')


Personally, I would toss the multi-threaded portion and stick with 
asynchronous IO.  You don't mention how you are handling new information 
coming into the system, whether this is by sockets or polling for files, 
but basically everything can be wrapped up in an async loop:

while not quit:
     asyncore.poll(0)        #handles socket IO
     poll_and_handle_done()  #would check for all done
                             #outputs, and process them as above

By not using threads, you should find your software moves a few percent 
faster when loaded heavily.


> - - very, very seldom (1 out of 5000 received faxes), one user of me saw the 
> following - at least with Python 2.2.2; I have no feedback about newer 
> versions yet:
> File "/usr/lib/python2.2/site-packages/cs_helpers.py", line 233, in 
> sendMIMEMail
>   sendmail.tochild.write(msg.as_string())
> File "/var/tmp/python-2.2.2-build//usr/lib/python2.2/email/Message.py", line 
> 107, in as_string
> File "/var/tmp/python-2.2.2-build//usr/lib/python2.2/email/Generator.py", line 
> 100, in flatten
> File "/var/tmp/python-2.2.2-build//usr/lib/python2.2/email/Generator.py", line 
> 135, in _write
> File "/var/tmp/python-2.2.2-build//usr/lib/python2.2/email/Generator.py", line 
> 167, in _write_headers
> File "/var/tmp/python-2.2.2-build//usr/lib/python2.2/email/Generator.py", line 
> 191, in _split_header
> File "/var/tmp/python-2.2.2-build//usr/lib/python2.2/email/Generator.py", line 
> 41, in _is8bitstring
> File "/var/tmp/python-2.2.2-build//usr/lib/python2.2/encodings/__init__.py", 
> line 43, in search_function
> AttributeError: 'NoneType' object has no attribute 'get'

I don't know about this one, but it looks like somewhere your software 
is passing a None when it should be passing a dictionary.

Good luck to you,
  - Josiah