[issue13734] Add a generic directory walker method to avoid symlink attacks

[Charles-François Natali]

Charles-François Natali <neologix at free.fr> added the comment:

> Hmm, sorry, I must have misremembered. I thought openat didn't follow
> symlinks.

OK, I was afraid I had missed something.

 > As for the patch, I think there's a problem with the API

Yes, it was really a proof-of-concept, the directory names are missing.

> Also, walkfd would be easier to use if callable with a str or bytes path
> rather than an int fd.

Agreed.

> Also giving the dirnames without their fds encourages using them
> by name, not by fd ;-)

Well, that's not easy:
- right now, the code uses O(depth of directory hierarchy) FDs
- returning FDs for sub-directories would require O(number of nodes in
the hierarchy), or at least O(max number of child nodes): I fear we'll
run out of FDs quite easily
Also, I don't think it's really a problem, since you have to use the
names anyway. The *at() family accepts a FD as a pointer to the
containing directory, but the target entries are accessed by name.
For example, to perform a safe rmtree, you  would do something like:

for dirfd, dirs, files in os.walkfd(topfd, topdown=False):
    for file in files:
        os.unlinkat(dirfd, file)
    for dir in dirs:
        os.unlinkat(dirfd, dir, os.AT_REMOVEDIR)

> Thanks for that Charles-François - do you mind if I adapt that for walkdir?

Of course not, go ahead. I'll update walkfd() accordingly, and write
doc and test for it.
By the way, do you plan to get walkdir merged in 3.3?
I've been doing a lot of sys-admin scripts lately, and this would be
really helpful.

> I'm currently leaning towards the simple 4-tuple approach

Sounds good to me.

----------

_______________________________________
Python tracker <report at bugs.python.org>
<http://bugs.python.org/issue13734>
_______________________________________