[Tutor] Writing unit tests that involve email

[Steven D'Aprano]

On 14/07/13 10:32, Arnel Legaspi wrote:
[...]
> What I'm having difficulty is getting the unit tests to properly run the
> script being tested and send the email, get the proper Maildir message,
> and be able to read it via the unit test script. Right now, even when
> the SMTP server is not running, all the tests for checking the sent
> email pass, which is not what I intended.
>
> I know the SMTP server works, because if I send some email using another
> Python or Powershell script, the script logs on the screen that it did
> receive the email.
>
> How should I approach this?
>
> Here is the unit test script I have (some lines may get wrapped):


Firstly, start with the obvious: are you sure that the unittests are being run? If I have counted correctly, you only have seven test cases at the moment, so your unittest output should show seven dots and then print something like:

Ran 7 tests in 0.01s

OK


Can you confirm that this is the case? If not, then your problem is that the tests aren't running at all.

[...]
>       parser.set('smtp', 'password', ***redacted***)

I hope this isn't a real password to a real mail server visible from the internet, because you've now made it visible to everyone in the world.



> class TestReadingSendingPostFile(unittest.TestCase):
>
>       def setUp(self):
>           """ Prepare a post file for testing. """
>           create_configfile()
>           with open('post.md', 'wb') as postfile:
>               postfile.write(MKDOWN_TEXT)
>           self.mbox = mailbox.Maildir('mail_dir')

Personally, I don't use the setUp and testDown methods, but most of my code avoids storing state so they are rarely relevant to me. But in any case, the setUp method is called before *every* test method. If you want it to be called only once, you should put it inside the class __init__ method. (Don't forget to call the TestCase __init__ as well.)

I see all of your test methods include clear_args() at the start, so you should put that inside the setUp method.


It would help if you tell us which test method you are expecting to fail. My guess is that it is this one:

>       def test_sendpost(self):
>           clear_args()
>           add_testfiles()
>           for message in self.mbox:
>               self.assertEqual(message['subject'], 'post')
>               self.assertEqual(message.get_payload(), HTML_TXT)

I don't believe this actually tries to send any email, and as far as I can see you never actually clear the mailbox, so all you're doing is looking at old emails that happened to exist.

If the mbox happens to be empty, then neither of the assertEqual calls will happen, and the test will just pass. Add this to the test:

           if not self.mbox:
               self.fail("No messages were sent")

or better still, add a separate test to check that email is sent, without caring about the content of the email. That way, if you get a failure, you can immediately see whether the failure is "email wasn't sent", or "email was sent, but contains the wrong stuff".

I suggest you factor out any tests of sending mail into a separate class. Give this class a setUp() that creates a mailbox, and a tearDown that moves everything from that mailbox into another one. That means you have a nice clean mailbox at the start of each test, but you can still inspect the emails by eye if you want. It also means that you can separate tests with an external dependency (the mail server) from those without.



-- 
Steven