os.path.walk problem

[Terry Hancock]

From: "Steve Holden" <sholden at holdenweb.com>
> "Terry Hancock" <hancock at anansispaceworks.com> wrote in message
> news:mailman.1029203179.29313.python-list at python.org...
> > What happens is that I get an IOError in the walker
> > which complains that I'm trying to read a directory
> > as a file  (IOError 21 "Is a directory").  A little
> > investigation reveals that this is indeed the case --
> > it's trying to read my "CVS" directory.
> >
> So don't read directories!

Hai hai ... :|

Like I said, I probably know how to fix it.

> > Which is odd, because my understanding is that the
> > walker is supposed to be guaranteed to be passed
> > a directory path and the list of *regular* files in
> > that directory (i.e. the latter list shouldn't
> > contain directories). Now I suppose I can fix it by
> > testing the files before operating on them, but I
> > thought this was pretty odd.  Did I misunderstand
> > the documentation on os.path.walk()? Is there some
> > normal (or even abnormal) condition under which
> > it is expected to let a directory slip through as
> > a regular file? Is it relevant that "CVS" is the
> > *only* directory in this path?
> >
> Yes, otherwise your assumption that os.path.walk() does not pass directores
> through the list would have more clearly been flagged as erroneous.

Heh. Heh. It seems that it wouldn't actually ...

This code runs fine on the FreeBSD box which has *exactly*
the same file structure in those directories. I first
checked this manually, and then I used "cvs diff" to do
an automated comparison (I compared both copies against
the repository, and they both showed no diffs at all).

What's more, this code is preceeded by another bulk loader
for images. Those directories are deeply nested, and they
work fine on both machines, with essentially the same code
(I'll be examining that "essentially" in some detail now,
as you might imagine).

I *do* know that the system failed on the Linux box today
because it's the first time it's been run with the CVS
directory *on that OS*.  It has been run on FreeBSD with
the CVS directories, though. (In fact, that was the motivation
for the code above that tests to see if it's been called on
a CVS directory).

> The docs for walk() clearly say that the list is gooten with os.listdir(),
> whose docs in turn clearly state that it lists the *entries* in the
> directory, not just the plain files.

Okay. I missed that.  Reading it again, I see that you're
right.  That's actually very reassuring, except for the
cases in which it *seems* to work like I expected it to.
Now I have to explain those.

> > To make matters worse, this code *works* on the
> > FreeBSD server at Imeme, but not on my Linux server
> > at home (it's Debian Potato, so kernal 2.2.17, I
> > think).  Curiouser and curiouser.
> >
> The difference wouldn't be the absence of any subdirectories on the FreeBSD
> system, would it?

No it wouldn't. Like I said, I checked that possibility
out very thoroughly as it is the most obvious trouble
point.

> Just insert a test for subdirectories and ignore them?

Yeah. That's what I'm going to do.  I just was surprised
by it, because it worked in the other settings. Now I
perceive that the mystery is why it didn't fail before!
;-)

I suppose it could be that FreeBSD's os.path.walk isn't
the same code, and is perhaps not completely in line
with this spec.  It may also be that my image-loading
code somehow fails silently on directories instead of
raising an exception. (Both would have to be true to
explain what I observe).

Hmm.

Thanks,
Terry

-- 
------------------------------------------------------
Terry Hancock
hancock at anansispaceworks.com       
Anansi Spaceworks                 
http://www.anansispaceworks.com 
P.O. Box 60583                     
Pasadena, CA 91116-6583
------------------------------------------------------