Friday Finking: Contorted loops

[Avi Gross]

The topic of looping and our current discussion stimulates me to ask if
someone has categorized the uses of loops and it seems something obvious.
Once you know what kinds of loopy looping there are, it can get easier to
decide which, if any, of the methods to set up a loop make more sense.

Computer languages differ in all kinds of ways including more subtle ideas
of when a variable is within view or not and that can impact looping
methods.

Years ago, in many languages, loops were used simply to COPY something
stored in some array-like format. Languages like C might have a character
string stored in a null-terminated array and the accepted way to copy it
might be to use pointers called p (the source) and q(the destination) in
weird and compact code like:

    while (*q++ = *p++);

Now Python does not have some of those operators but copying strings is
fairly trivial with no visible loops but copying some objects (I mean a new
and even deep copy) can use loops, albeit for many objects, there is a copy
functionality available to do it quietly.

If you only want a subset of the values to be taken into a copy, many
languages have a subletting method like var[1:5] and often you have
functions like map/reduce you can pass things to that return an arbitrary
subset based on applying a function.

Python has thus quite a few ways to hide any loops involved in copying so
you will often not see a WHILE or FOR in sight.

For simple (and even sometimes more complex) scenarios, comprehensions of
many types can be used in in-line situations. Generators can supply what can
amount to a sort of deferred and thus distributed loop that is run
interspersed with other things happening.

One reason many applications bring in add-on modules like numpy and pandas
is because they partially fill some gaps and make many vectorized operations
easy. Any operation that works simultaneously on every member in a way that
has no conflict with calculations on other members is an example. If I have
two vector like objects such as A and B, then being able to write 3*A means
modifying all the elements of A to be triple what they were and it does not
necessarily need to happen in any order or even be all done by the same
process. If A and B are the same length (or logically adjusted to be) then
A+B is similarly a vector operation. Quite a few such things are done
routinely with no loops visible in the code that once littered my C code.
Languages like R (and the original S) were built from the ground up so
everything starts with a vector and even a singleton variable is merely a
vector of length 1. Many programs simply do not need loops as it is done for
you.

I have mentioned ways to use objects in Python as a way to hide loops. Of
course that implies an object may use methods that contain small loops, such
as to search and internal list to see if something is already being held, or
to find the end and append another and so on. But you main program may thus
often be written without such looks.

Functional programming techniques, again, can be used to say apply a
function to every item in a list and collect the results in another list or
scalar. You will not see a WHILE or a FOR but a loop happens.

I have sometimes written a program and then after it was working, took
another look and redesigned it in ways that often shorten it substantially
and even speed it up. I admit people reading the code often have no clue
what it does. So especially for teaching Computer Science, many loops remain
a good place to start. 

My guess is that if, like me, you often avoid using loops in trivial cases,
you may end up using them in cases that are more complex. You may end up
with mainly cases where you end up having to BREAK or CONTINUE or RETURN
from within a loop, perhaps with multiple exit points. Some of those cases
may well be much more difficult using the hidden loops.

So I was not in any way being negative with Peter about his admittedly
restricted sample of coding practices, or of others who searched a body of
code by many authors. I am saying that people who write code can evolve and
not go back and change older code.

I once inherited code that had nested loops about 9 units deep. Something
like
	for in in ...
	    for j in ...
	        for k in ...

It was a tad more complex, of course as it tested all variations of
categorical variables. There were additional loops in selected places within
there, as well. It ran slowly in interpreted form. What I ended up doing was
generating a data structure (a data.frame) that contained all combinations
and handed that to a function called pmap that did one row at a time by
setting variables. The problem became way more tractable and quite a bit
faster. And, in this case, I suspect it may have been more readable without
getting lost in all the nesting.

But not all languages and problems are amenable to some approaches, and some
play games with how variables are kept and used. Python has some nice
features that allow a single loop to replace more complex arrangements by
say allowing multiple variables to be instantiated each time around so
something like my deeply nested version can be run in a straightforward way.

I suspect in many cases, a little though of what to feed a loop might be a
great way to simplify the innards of the loop and minimize some of the
concerns about multiple exit methods and so on. But you can add bells and
whistles like the ELSE clause but not get many to actually use it as there
are more standard ways to do that without confusion, especially if it is
confusing to some.

-----Original Message-----
From: Python-list <python-list-bounces+avigross=verizon.net at python.org> On
Behalf Of Peter J. Holzer
Sent: Sunday, September 12, 2021 5:44 AM
To: python-list at python.org
Subject: Re: Friday Finking: Contorted loops

On 2021-09-11 21:38:02 -0400, Avi Gross via Python-list wrote:
> Peter, in your own personal finite sample, I am wondering what you 
> might do TODAY if you looked at your loops again and considered 
> redoing them for an assortment of reasons ranging from using the code 
> for teaching to efficiency to just fitting your mood better?
> 
> I have seen seasoned authors go back to their early work and groan.

Yeah, I do that. (Un)fortunately I also have other people's code to groan
about so I won't despair too much about the stupidity of my younger self.


> My guess is that many of us (meaning myself included) often approach a 
> problem and go with the first thing that comes to mind. If it fits 
> well enough, we move on to the next thing we can do. If not, we may 
> step back and evaluate multiple additional options and try another tack.
> 
> I have seen not of sort-of redundant code because someone did not plan 
> ahead and realize something very similar might be needed later and 
> thus did not make a general function they could re-use. Occasionally 
> they may later go back and re-do but often, not so much and just keep 
> copying lines and making minor modifications. Same general idea.

That certainly happens. I am a bit overly conservative and try to get away
with minimal code changes even if a complete reimplementation of that unit
would be clearly better. Especially if it's someone else's code and there
are no unit tests. But also for my own code.

(As an aside, I notice the same tendency when changing text: Altering an
existing paragraph is hard, especially if someone else wrote it. Also, while
I think I can express myself quite clearly in both German and English, I'm
rarely satisfied when I try to translate between those languages. I always
stick too close to the original).

> And perhaps worse, you may write a loop and later have to keep adding 
> code to deal with new requirements and special cases and rather than 
> pause and analyze and perhaps start again with a cleaner or more 
> easily extendable solution, just keep grafting on things to make the darn
current code work.
> Code that has many ways to exit a loop is often an example of this 
> happening.

That too. Those little C utilities I mentioned are probably a bad example
because they are so small and had little reason to evolve. But I do have
Perl scripts which I originally wrote 20 years ago and which are still in
use and have been adapted to changing business requirements again and again
in that time. Those do contain some gnarly code.


> So if you looked at your own code now, in the context of the rest of 
> your code, would you change things?

Almost certainly. Especially in C I would probably be more cautious about
undefined behaviour now and for different reasons. Back in the 90's I mostly
worried about portability: That code could one day run on a 36-bit
ones-complement machine with 9-bit chars. These I days I worry more about
overly aggressive optimizations: That pointer is accessed here so it can't
be null, so it can't be null here either so that check can be optimized
away.

I started using Python only 7 years ago, when I had already been using Perl
for almost 20 and C for over 25 years. So my older Python code probably
looks a bit "perly". So they use dicts and map and filter but not list
comprehensions for example. Also some of that code was partially inherited
from other Python programmers who adhered to the "a real programmer can
write Fortran in any language" mindset.


> So when counting the various kinds, are you looking for direct or 
> indirect methods too like map/reduce or vectorized operations?

No, because that wasn't the question I was trying to answer. The question
was "do people use do/while loops frequently in languages which provide
them"? I chose C (mostly because it is easier to get useful numbers with
tools like grep and wc than with Perl) and therefore only the types of loops
available in C. (Methodically the main flaw in my approach is that I only
looked at a single language and a single person and only a tinly sample from
that person. To really answer that question you would have to look at a
sizable sample from Github or something like that).

        hp

-- 
   _  | Peter J. Holzer    | Story must make more sense than reality.
|_|_) |                    |
| |   | hjp at hjp.at         |    -- Charles Stross, "Creative writing
__/   | http://www.hjp.at/ |       challenge!"