all() is slow?

[Devin Jeanpierre]

> I don't expect you to take my word on it (and why should you, I could be
> an idiot or a sock-puppet), but you could always try googling for
> "Raymond Hettinger python" and see what comes up. He is not some fly-by
> Python coder who snuck some dubious n00b code into the standard library
> when no-one was looking :)

Alright, I know *something* about him. I knew he was a core developer,
and that he was responsible for namedtuple. I also discovered (when I
looked up his activestate profile) some other stuff he wrote. I don't
really know anything about him outside of that -- i.e. I have no idea
what parts of Python he's contributed things to in the past that could
make me go, "oh, wow, _he_ did that?" and so on. I don't really feel
like a few minutes research would give me the right feel, it generally
has to come up organically.

Anyway, if we step back, for a trustworthy developer who wrote
something seemingly-crazy, I should be willing to suspend judgement
until I see the relevant facts about something that the developer
might have and I don't. But he did give the facts,
( http://bugs.python.org/issue3974 again) , and I'm not convinced.

Things can go terribly wrong when abusing exec e.g.
http://www.gossamer-threads.com/lists/python/bugs/568206 . That
shouldn't ever happen with a function such as this. exec opens doors
that should not be opened without a really good reason, and those
reasons don't strike me that way.

> The mere fact that it was accepted into the standard library should tell
> you that core Python developers consider it an acceptable technique.

I've seen core developers rail against the namedtuple source code. In
fairness, I don't believe exec was the subject of the rant --
nonetheless its presence isn't evidence of general support, and even
if it were, my tastes have always differed from that of the core
developers.

> That's not to say the technique is uncontroversial. But there are still
> people who dislike "x if flag else y" and @decorator syntax --
> controversy, in and of itself, isn't necessarily a reason to avoid
> certain idioms.

I think there's somewhat a difference in magnitude of objections
between using exec as a hacked-together macro system, and using "x if
flag else y" when if statements would do.

If the exec trick is reasonable, we should normalize it in the form of
a real, useful macro system, that can protect us against exec's many
flaws (code injection, accidental syntax errors, etc.) and tell future
programmers how to do this safely and in a semi-approvable way.

> I would agree that the use of exec is a code smell. But that doesn't mean
> it is wrong or bad, merely that it needs a second look before accepting
> it. There's a world of difference between "You MUST NOT use exec" and
> "You SHOULD NOT use exec".

Do I really need a second look? I see exec, I wonder what it's doing.
It isn't doing anything that couldn't be done subjectively better with
e.g. a dict, so I disapprove of the usage of exec.

> There's nothing inside the template being exec'ed that couldn't be found
> in non-exec code. So if you're having trouble figuring out parts of the
> code, the presence of the exec is not the problem.

There's more overhead going back and forth to the template, and
there's related things that I can't be sure are because of exec or
because of design decisions, etc.  It makes code reading more
challenging, even if it's still possible. That said, sure, some of
these are problems with whatever else he's done.

> Having said that, dynamic code generation is well known for often being
> harder to read than "ordinary" code. But then, pointers are hard too.

And on the other other hand, Python lacks explicit support for both
pointers and code generation (unless you count strings and ctypes).

> Because Python doesn't allow "--" to be an attribute name, and so
> namedtuple doesn't let you try:
>
> t = namedtuple("T", "foo -- bar")(1, 2, 3)
> print(t.foo)
> print(t.--)
> print(t.bar)

'--' is a valid attribute name on virtually any object that supports
attribute setting (e.g. function objects). Of course, you need to use
setattr() and getattr(). Is this really the reason, or is it a
limitation caused primarily by the usage of exec and the need to
prevent code injection? If somebody added this feature later on, would
this create a security vulnerability in certain projects that used
namedtuple in certain ways?

Devin

On Thu, Nov 10, 2011 at 2:48 AM, Steven D'Aprano
<steve+comp.lang.python at pearwood.info> wrote:
> On Wed, 09 Nov 2011 20:26:56 -0500, Devin Jeanpierre wrote:
>
>>> Neither am I. I am less suspicious based on a reputation. Raymond is a
>>> well-known, trusted senior Python developer who knows what he is doing.
>>
>> I don't really know anything about him or why people respect him, so I
>> have no reason to share your faith.
>
> That's fine.
>
> I don't expect you to take my word on it (and why should you, I could be
> an idiot or a sock-puppet), but you could always try googling for
> "Raymond Hettinger python" and see what comes up. He is not some fly-by
> Python coder who snuck some dubious n00b code into the standard library
> when no-one was looking :)
>
> The mere fact that it was accepted into the standard library should tell
> you that core Python developers consider it an acceptable technique.
> That's not to say the technique is uncontroversial. But there are still
> people who dislike "x if flag else y" and @decorator syntax --
> controversy, in and of itself, isn't necessarily a reason to avoid
> certain idioms.
>
>
> Are you familiar with the idea of "code smell"?
>
> http://www.codinghorror.com/blog/2006/05/code-smells.html
> http://www.joelonsoftware.com/articles/Wrong.html
>
> I would agree that the use of exec is a code smell. But that doesn't mean
> it is wrong or bad, merely that it needs a second look before accepting
> it. There's a world of difference between "You MUST NOT use exec" and
> "You SHOULD NOT use exec".
>
> See RFC 2119 if you are unclear on the difference:
>
> http://www.ietf.org/rfc/rfc2119.txt
>
>
>
>>> It reads fine, and the justification is perfectly valid.
>>
>> Well. It reads fine in a certain sense, in that I can figure out what's
>> going on (although I have some troubles figuring out why the heck
>> certain things are in the code). The issue is that what's going on is
>> otherworldly: this is not a Python pattern, this is not a normal
>> approach. To me, that means it does not read fine.
>
> There's nothing inside the template being exec'ed that couldn't be found
> in non-exec code. So if you're having trouble figuring out parts of the
> code, the presence of the exec is not the problem.
>
> Having said that, dynamic code generation is well known for often being
> harder to read than "ordinary" code. But then, pointers are hard too.
>
>
>
>> The use of exec also results in (seemingly) arbitrary constraints on the
>> input. Like, why can't "--" be a name? Because exec? Is there some other
>> reason?
>
> Because Python doesn't allow "--" to be an attribute name, and so
> namedtuple doesn't let you try:
>
> t = namedtuple("T", "foo -- bar")(1, 2, 3)
> print(t.foo)
> print(t.--)
> print(t.bar)
>
>
>
>
> --
> Steven
> --
> http://mail.python.org/mailman/listinfo/python-list
>