[Tutor] urllib2 and tests

[Steven D'Aprano]

On 05/05/13 13:27, RJ Ewing wrote:
> When I run the following test.py, I get the following error:
[...]
> If I run the fetch_file function outside of the test, it works fine. Any
> ideas?

The code you are actually running, and the code you say you are running below, are different. Your error message refers to a file test_filefetcher.py, not the Test.py you show us. As given, Test.py cannot possibly work, since it doesn't define "filefetcher". I can only guess that this is meant to be the module you are trying to test, but since you don't show us what is in that module, I can only guess what it contains.


More comments below:


> RROR: test_fetch_file (__main__.TestFileFetcher)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>    File "test_filefetcher.py", line 12, in test_fetch_file
>      fetched_file = filefetcher.fetch_file(URL)

What's filefetcher? I'm guessing its the module you are testing, which is consistent with the next line showing the file name filefetcher.py:


>    File "/Users/rjewing/Documents/Work/filefetcher.py", line 7, in fetch_file
>      return urllib2.urlopen(url).read()
>    File
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py",
> line 126, in urlopen
>      return _opener.open(url, data, timeout)
>    File
> "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py",
> line 392, in open
>      protocol = req.get_type()
> AttributeError: 'TestFileFetcher' object has no attribute 'get_type'


Somehow, your test suite, the TestFileFetcher object, is being passed down into the urllib2 library. I can only guess that somehow url is not an actual URL. I suggest you add a line:

print(url, type(url))

just before the failing line, and see what it prints.


> ----------------------------------------------------------------------
>
> Test.py:

This cannot be the actual test suite you are running, since it cannot run as shown. It doesn't import unittest or the module to be tested.


> class TestFileFetcher(unittest.TestCase):
>
>      def test_fetch_file(URL):
>          phrase = 'position = support-intern'
>
>          fetched_file = filefetcher.fetch_file(URL)

And here's your error! Just as I thought, URL is not what you think it is, it is the TestFileFetcher instance.

Unittest cases do not take arguments. Since they are methods, they are always defined with a single argument, conventionally called "self", representing the instance that the method is called on. So normally you would define a method like this:

     def test_fetch_file(self, url):

which then takes a single *implicit* argument "self", provided by Python, plus a second *explicit* argument, "url". But because this is a test method, the unittest framework does not expect to pass an argument to the method, so you have to write it like this:

     def test_fetch_file(self):

and get the url some other way.

One common way would be to define an attribute on the test, and store the URL in that:

class TestFileFetcher(unittest.TestCase):
     URL = "some_url_goes_here"  # FIX THIS

     def test_fetch_file(self):
         phrase = 'position = support-intern'
         fetched_file = filefetcher.fetch_file(self.URL)
         ...



>          unittest.assertIsNone(fetched_file,
>                                'The file was not fetched correctly')

This part of the test seems to be wrong to me. It says:

"compare the value of fetched_file to None; if it is None, the test passes; if it is some other value, the test fails with error message 'The file was not fetched correctly'"

But then you immediately go on to use fetched_file:

>          text = filefetcher.add_phrase(fetched_file)

but if the above assertIsNone test passed, then fetched_file is None so this is equivalent to:

         text = filefetcher.add_phrase(None)


which surely isn't right?


>          unittest.assertNotIn(phrase, text, 'The phrase is not in the file')

This test also appears backwards. You're testing:

"check whether phrase is NOT in text; if it is NOT in, then the test passes; otherwise, if it IS in, then fail with an error message 'The phrase is not in the file'"

which is clearly wrong. The message should be:

'The phrase is in the file'


since your test is checking that it isn't in.



-- 
Steven