how to get os.py to use an ./ntpath.py instead of Lib/ntpath.py

[ruck]

On Monday, September 10, 2012 1:16:13 PM UTC-7, Steven D'Aprano wrote:
> On Mon, 10 Sep 2012 10:25:29 -0700, ruck wrote:
> 
> 
> 
> > In Python 2.7.2 on Windows 7,
> 
> > 
> 
> > os.walk() uses isdir(),
> 
> > which comes from os.path,
> 
> > which really comes from ntpath.py,
> 
> > which really comes from genericpath.py
> 
> > 
> 
> > I want os.walk() to use a modified isdir() on my Windows 7. Not knowing
> 
> > any better, it seems to me like ntpath.py would be a good place to
> 
> > intercept.
> 
> > 
> 
> > When os.py does "import ntpath as path", how can I get python to process
> 
> > my customized ntpath.py instead of Lib/ntpath.py ?
> 
> 
> 
> import os
> 
> os.path.isdir = my_isdir
> 
> 
> 
> ought to do it.
> 
> 
> 
> This general technique is called "monkey-patching". The Ruby community is 
> 
> addicted to it. Everybody else -- and a goodly number of the more 
> 
> sensible Ruby crowd -- consider it a risky, dirty hack that 99 times out 
> 
> of 100 will lead to blindness, moral degeneracy and subtle, hard-to-fix 
> 
> bugs.
> 
> 
> 
> They are right to be suspicious of it. As a general rule, monkey-patching 
> 
> is not for production code. You have been warned.
> 
> 
> 
> http://www.codinghorror.com/blog/2008/07/monkeypatching-for-humans.html
> 
> 
> 
> 
> 
> [...]
> 
> > Why?  Because the genericpath implementation relies on os.stat() which
> 
> > uses Windows API function that presumes or enforces some naming
> 
> > conventions like "doesn't end with a space or a period". But the NTFS
> 
> > actually supports such filenames and dirnames, and some sw (like cygwin)
> 
> > lets users make files & dirs without restricting. So, cygwin users like
> 
> > me may have file 'voo...\\doo' which os.walk() cannot ordinarily walk. 
> 
> > That is, the isdir('voo...') returns false because the underlying
> 
> > os.stat is assessing 'voo' instead of 'voo...' . 
> 
> 
> 
> Please consider submitting a patch that adds support for cygwin paths to 
> 
> the standard library. You'll need to target 3.4 though, 2.7 is now a 
> 
> maintenance release with no new features allowed.
> 
> 
> 
> 
> 
> > The workaround is to
> 
> > pass os.stat a fullpathname that is prefixed with r'\\?\' so the Windows
> 
> > API recognizes that you do NOT want the name filtered.
> 
> > 
> 
> > Better said by Microsoft:
> 
> > "For file I/O, the "\\?\" prefix to a path string tells the Windows APIs
> 
> > to disable all string parsing and to send the string that follows it
> 
> > straight to the file system.
> 
> 
> 
> That's not so much a workaround as the officially supported API for 
> 
> dealing with the situation you are in. Why don't you just prepend a '?' 
> 
> to paths like they tell you to?
> 
> 
> 
> 
> 
> -- 
> 
> Steven

Steven says:
 That's not so much a workaround as the officially supported API for 
 dealing with the situation you are in. Why don't you just prepend a '?' 
 to paths like they tell you to?

Good idea, but the first thing os.walk() does is a listdir(), and os.listdir() does not like the r'\\?\' prefix.  In other words, 
os.walk(r'\\?\C:Users\john\Desktop\sandbox\goo') 
does not work.

Also, your recipe worked for me -- 
I'm walking 'goo' which contains 'voo.../doo'

    import os

    import genericpath
    def my_isdir(s):
        return genericpath.isdir('\\\\?\\' + os.path.abspath(s + '\\'))

    print 'os.walk(\'goo\') with standard isdir()'
    for root, dirs, files in os.walk('goo'):
        print root, dirs, files

    print 'os.walk(\'goo\') with modified isdir()'
    os.path.isdir = my_isdir
    for root, dirs, files in os.walk('goo'):
        print root, dirs, files

yields

    os.walk('goo') with standard isdir()
    goo [] ['voo...']
    os.walk('goo') with modified isdir()
    goo ['voo...'] []
    goo\voo... [] ['doo']

About monkeypatching, generally -- thanks for the pointer to that discussion.  That sounded like a lot of wisdom and lessons learned being shared.
About me suggesting a patch -- I'll sleep on that :)

Thanks Steven!
John