lazy (?) evaluation, request for advice

[Alex Martelli]

Tim Peters:

> > ... I'd much rather let the user write the deal-selection
> > function in Python, than in either Tcl, or any custom,
> > ad-hoc "little language".
>
> Against that, you invented ad-hoc semantics as you went along;

Did I?  I used, e.g., spades to mean "the value of a variable named
`spades' in the current scope[s]".  Isn't that normal Python semantics?

What else are the local and global scope arguments to eval/exec for,
if not to let the controlling program determine the current scope[s]
for code being eval'd/exec'd?  Isn't that normal semantics too?

Whether in a given context the `spades' variable contains the number
of spade cards in a player's hand, the number of shovels in a pile of tools,
or the number of 3-year-old stags in the stables, well, yes, that IS
"ad-hoc semantics" -- as is, case by case, the identification of, what
hand, what pile, what stables we're talking about ("the current one",
"the one of interest").  If there is a need to examine multiple stables,
or piles, or hands, then in such contexts using "x.spades" for some
"x" will surely be better -- but you appear to be saying that, even in
contexts where there is only one pile/stables/hand of interest, using
"x.spades" for some one arbitrary "x" is "less ad-hoc"?  I don't see why.

> the only connection to Python appeared to be syntactic.

?"only"?  You call, e.g., the semantics of short-circuit logic a "syntactic"
issue?!  I would appreciate understanding how you define "syntactic" in
this context to justify this extremely peculiar assertion.

> Let the user write real
> Python!  Then as the program gets fancier they'll have the full language
to
> exploit; with "spades" etc as magical global names, what chance is there
to
> extend the scheme to a user-defined function that e.g. wants to examine
> *two* hands at a time?  (I know! "spades1" and "spades2" <wink>)

Where more than one hand is of interest, object notation will be used;
and where more power is needed, statements will be used rather than
just an expression.  The fact that a statement suite is more powerful
(because, e.g., assignments can be done, if/else used rather than just
short-circuit logic for flow control, loops can be variously expressed
rather than just squeezed into map/reduce kinds of molds, etc), is no
reason to outlaw using just an expression where an expression will
suffice; and similarly for object vs plain notation.  Surely "a>2" is
"real Python", just as "x.a>2", or "for b in a: splotz(b)", etc -- a simpler
case, but no less real than the less-simple ones!


> def userfunc(x):  # they get to make up the argument name
>     ... (x.spades >= 6 or x.hearts >= 6) ... etc
>
> Provided your users can pass a mirror test (i.e., you hold a mirror under
> their nose, and they pass if it fogs up <wink>), they can learn this.
After

They can learn, and they will surely appreciate being able to use
more powerful constructs (functions, statements, object notation,
etc) _when needed_ -- but surely will not appreciate having to
use them, willy-nilly, even when their needs are much simpler.

> which, playing __getattr__ tricks with the x object gives you the "lazy"
> behavior you want.

Yep, I said that -- if object notation is used, then lazy evaluation can
be had through __getattr__.  I also said that object notation can be
avoided (for simple cases) if lazy evaluation is not needed.  I would
prefer to offer my users programming interfaces with the "right"
amount of power/complexity, based on my analysis of "right", rather
than making such an interface decision solely on the basis of hassle
of implementation of a "third way" combining simple notation and
lazy evaluation; letting implementation problems constrain interfaces
is not a good general strategy.


> > I've thought about doing the eval repeatedly, within
> > a try/except block to catch the NameError's, inserting
> > the missing names in the dictionary one by one, but
> > I'm suspicious about the performance characteristics
> > of such an approach.
>
> You should be more suspicious of the sheer convolution <0.5 wink>.
>
> Anyway, what's expensive there isn't so much the try/except, but the
> repeated eval *if* the user's expression is carried around as a string
(eval
> will recompile it to byte code each time).  If you do go this route, see
the
> builtin "compile" function for a way to do the compilation to byte code
just
> once (eval accepts compiled code objects too).

Yes, I'm aware of that, although I failed to mention it; I do plan to
compile the user-supplied text before running it over thousands of
candidate deals.

> Note also that if your users
> truly are morons, there's nothing to stop *you* from marching over their
> string and replacing e.g. "spades" with "current_hand.spades" before
> compilation.

Is there a way I could use the existing parsing tools to help me
with this?  I've read through the library reference, but I still find
that part a tad obscure. 3.14.6.1 and 3.14.6.2 are the parts I
am thinking about.  If I could examine in the AST the variable
names used in the expression, matching them with the dictionary
of the hand features I know how to evaluate, I could pre-populate
the dictionary to be used as the scope in the eval; or, I guess,
change the variables as you suggest, to get even lazier (for those
cases where short-circuit logic lets me evaluate a certain feature
only in a subset of cases).  But the logic for examining, and even
more for modifying, the AST, appears to be somewhat complex.
Are there more online examples that I could use?

Or, were you suggesting a similar operation on a purely lexical
level?  E.g., with tokenize?  Or, a string-manipulation approach?

I will appreciate further advice about this, and tolerate the
sneers and name-calling that seem to go with the advice
(although rarely have I seen discussion quite as sneering as
this on this group -- reminds me a lot of the ambiance on
the Perl group -- this one had so far seemed friendlier, though
I guess that may have been an artefact due to a frequentation
started more recently).


> treat-users-like-grownups-and-they-won't-demand-you-give-
>     them-an-allowance<wink>-ly y'rs  - tim

Grown-ups, too, appreciate convenience -- which, inter alia,
means "appropriate level[s] of simplicity/power trade-off[s]".

Rather than offering just one (rich, powerful, complex) way
to express deal selection, I would prefer to offer users the
convenience of choosing different points along the curve of
power vs simplicity according to current needs -- from just
clicking a few radiobuttons on a GUI, to entering a plain and
simple selection expression about one hand, all the way to
subclassing the deal-generator class to tweak and customize
things to the fullest.

The "plain and simple selection expression" appears to me
(from reasonably vast previous experience with this specific
application domain) to be a rather "key" point along the
trade-offs curve, one which will be very often "visited" for
many kinds of simple deal-generation needs.  Therefore, I
judge it worthwhile to expend some thought, and work, in
optimizing usability and convenience at that point in the
curve.


Alex