python 2.7.12 on Linux behaving differently than on Windows

[Steven D'Aprano]

On Wednesday 07 December 2016 03:41, BartC wrote:

> On 06/12/2016 15:44, Michael Torrie wrote:
>> On 12/06/2016 04:43 AM, BartC wrote:
> 
>> Yes shell expansion has it's gotchas.  But those can all be learned,
> 
> Yes, learn to avoid wildcards in command parameters at all costs. But we
> both know that is not satisfactory.

Now you're being ridiculous. Can't you at least debate this sensibly?

There are Windows programs which understand globs, like dir. Do you honestly 
expect us to believe that it is okay to use "dir a*" on Windows because dir 
itself expands the glob, but not okay to use "dir a*" on Linux because the 
shell has expanded the glob before dir sees it?

(Yes, there's a Linux dir. It appears to be almost identical to ls.)

 
> And it's not much help when someone types in:
> 
>   program *

Indeed.

Tell me Bart, what's the difference between these?

    # Linux
    rm *

and 

    # Windows
    del *



If you accidentally say "del *" on Windows, say, when you meant "z*" for 
example, and Windows deletes all the files in the current directory, do you say 
to yourself...?

   "Thank the deities that * was expanded by the del command rather 
   than the shell! Imagine how awful it would have been if the shell
   did the expansion!"


> and the program has to try and clean up the mess, if it can see it is a
> mess. But remember:
> 
>    cp *.c
> 
> There might be some irate users out there if it can't detect a simple
> user error like that.

*shrug* You can blow your foot off with any sufficiently powerful set of tools, 
if you're not paying attention.

I see that the Windows version of copy avoids this *by accident*:

C:\Documents and Settings\Steve>dir a*
 Volume in drive C has no label.
 Volume Serial Number is 9CAB-3B4A

 Directory of C:\Documents and Settings\Steve

07/12/2016  06:05 PM                 4 a1
07/12/2016  06:05 PM                 4 a2
               2 File(s)              8 bytes
               0 Dir(s)   1,672,613,888 bytes free

C:\Documents and Settings\Steve>copy a*
a1
The file cannot be copied onto itself.
        0 file(s) copied.



Instead of accidentally copying a1 over a2, it accidentally tries to copy a1 
over a1, and fails. At least, that's what it looks like to me. Any Windows 
expert like to chime in?


The bottom line is, it doesn't matter where the wildcards are expanded. Whether 
the shell does it, or the application itself, they're dangerous by their very 
nature. You're asking the computer to operate on a set of files you specify 
using fuzzy matching. It is easy to match too much, or too little, if you 
aren't careful.


>> whereas it's much harder to learn and remember all the gotchas and bugs
>> of many individual applications' unique ways of dealing with globs. I'd
>> rather deal with shells.
>>
> 
> OK, I understand. Having a program launcher indiscriminately transform
> input A into A' is /much/ better than having it done by the program,

For the use-cases that guided the design and evolution of Unix, ABSOLUTELY.

That doesn't mean that the same will apply to all uses of the computer command 
line. But it does apply to the great majority of uses by sys admins on Unix 
systems. They designed the system to work the way they want.

I get that its not the way that you like, but fortunately for you, they 
provided not one, but TWO easy ways to get the results you want:

- escaping the metacharacters you want to be left untouched;

- disabling globbing (for either a single command, a session, or permanently).


If *your* design was paramount, it would be up to each and every application to 
choose whether or not to support globbing and environment variable expansion, 
and hard luck if the application I needed to use didn't understand the globs I 
wanted.


> even if the program doesn't want it and it is not meaningful.

*shrug* That's the cost you pay. It's not a big cost, since it costs as little 
as a single character (backslash) to escape something, but it is a cost.


> The fact that you also lose format information (is it three parameters,
> or one parameter transformed into three) is an extra bonus.

Here's a two-line Python program to prove what nonsense that is.

steve at runes:~$ cat argv.py 
import sys
print( sys.argv )


And in use:

steve at runes:~$ python argv.py a\* z\*
['argv.py', 'a*', 'z*']



> This is clearly much better than any other scheme because:
> 
> (1) It's Unix not Windows; everything in Unix is always better and
> always make sense.

[snarky, sarcastic answer to match the tone of your post]

Everything except the tar command.


> (2) There have been 40 years of it working this way and there have never
> been any problems. (That is, 40 years of writing programs with
> stultified command line interfaces to make sure that is always the case.
> [As for 'problems' such as the 'cp *.c' one, that's a feature!]

Don't be absurd.

Any Unix sys admin can tell you stories of accidentally globbing too much and 
screwing something up. And any Windows sys admin can tell you the same stories, 
even though globbing is handled by the individual programs rather than the 
shell.

(At least, any Windows sys admin who uses the command line, or writes batch 
files. I dare say those who use nothing but GUI tools have their own, 
different, horror stories.)

But that's the nature of the job, not the choice of where the globs are 
expanded. Certainly Unix sys admins will cheerfully admit that bash and other 
shells are designed for quick, interactive use, they aren't ideal for writing 
large, complex, maintainable systems.

But Unix systems are the end product of 40 years of directed evolution by 
thousands of sys admins who have designed the shells to work the way they want.

And it looks like Windows is ever-so-slowly converging on the same, or at least 
similar, sets of conventions: by copying Unix globbing metacharacters in the 
first place, then by adding them to the C library to make it easy for 
applications to do their own globbing, then by moving to PowerShell, and the 
latest step is Microsoft's partnership with Canonical (the company behind 
Ubuntu Linux) to bring bash to Windows.



-- 
Steven
"Ever since I learned about confirmation bias, I've been seeing 
it everywhere." - Jon Ronson