need dict to maintain order

Sun Jan 20 03:27:35 EST 2002

Jason Orendorff wrote:
        ...
>> >> try:
>> >>    os.mkdir(self.dirname)
>> >> except OSError:
>> >>    pass

Too raw, of course: you want to suppress only ONE specific kind
of OSError, so:

try:
    os.mkdir(self.dirname)
except OSError, err:
    if err.errno != errno.EEXIST: raise

>> >  if not os.path.isdir(self.dirname):
>> >      os.mkdir(self.dirname)
        ...
> In this *particular* case I think it's unambiguously better.

It has a different defect from the first version, but one that's much
subtler and harder to correct -- the kind of error that will never
show up in tests, but will only crash at the customer site and cause
3 AM calls and emergency (and unsuccessful) attempts to reproduce
and identify.  Don't do this: it's bad practice and eventually it will
catch up with you -- if not in this specific program, then in another
when you have gotten into the habit of using this incorrect idiom.

The defect is a *race condition*.  Two different processes could
be running at the same time (or in some slightly different cases
two threads of the same process) and both be trying to run code
very similar to this one.  If one is switched out and the other
switched in between the if and its body (will happen rarely, which
gives race conditions their elusive nature) then both processes
will see the "not ... isdir" guard satisfied, so both will execute mkdir,
but only one of them will succeed -- the other one crashes with
EEXIST, and it's 3 AM call time.

The Romans said "Experientia docet stultos" -- experience is
fools' teacher; the idea being that the wise can learn from the
experiences of others.  Pretty optimistic: these days, I account it 
unusual enough to be able to learn from one's own errors.  Still,
take a fool's advice on race conditions: *DON'T* have them.

The idiom "look before you leap", LBYL -- if <guard>: <body> -- is
prone to race conditions any time you potentially have more
than one thread or process involved.  "easier to ask forgiveness
than permission", EAFTP -- try: <body>/except... -- is more solid, even
though you DO need, almost invariaby, to be a little bit careful
about the except clause[s], e.g. by checking errno.  Mistakes in 
EAFTP tend to be easier to ferret out by good testing, while race
conditions generally don't show up in tests, making them one of
the most elusive and damaging kind of bugs.

> If the directory exists, it's foolish to try to create it again;

Foolish but innocuous, with a carefully crafted try/except.

> if it *doesn't* exist, and you try to create it, and fail
> (permission denied; or device failure), then it's foolish to
> proceed and start putting files in that directory.  Right?

Wrong.  When you do proceed, you'll almost certainly get an
exception very soon; all that you've lost by using careless
EAFTP is that the error diagnosis comes a little bit later, a little bit 
less clearly.  You still do catch the error if you do decent testing
(and if you don't, you have worse problems than these:-).  It's
nowhere as bad as the race conditions enticed by LBYL.

Alex