[Tutor] LLM comparison

[ThreeBlindQuarks]

This is a really hard topic for me as I believe views often differ sharply on what is a good or best way to write software.

Consider some minor task that can be done in some form of explicit loop. It may not be pretty but can be moderately easy to understand and implement, even in nested loops.

Someone then suggests only babies or newbies use loops and suggest a comprehension. Fine, that can be fairly concise but it can also be a tad harder to read and reason about without experience and inherently is just a loop once it is internally transformed.

Someone may then suggest some other vectorized approach such as using a module like numpy that lets you use implicit loops such as A=B+C. Again, great, as long as you trust you know what is going on under the hood.

But some suggest you search the internet and find some module of unknown value that has a function that sounds like it does what you want. You invoke a black box and mumble a few old latin words and hope it did it as you want it.

Arguably, for trusted code, the latter approach may well be best for experts. But the plain loop you wrote is perhaps more likely to work than code you cannot see and verify easily.

Consider another example. You break up your application into many parts and ask an AI-like program how to do each part in isolation. You gets lots of answers but nobody notices that your code does something similar over and over such as sorting something. You might have written one sorting function (or used one in a library) and instead you now reimplement something similar repeatedly. Fixing or changing one now, such as adding logging, does not change the others.

Or, on the flip side, the AI might keep getting you to create a function that you will only use one time and with function call costs. Or, perhaps the AI does not realize the redundancy of calculating the mean of a million numbers over and over again when you could have saved the result once into a variable and referenced the variable each time.

My point is even humans often disagree and give weird advice but an idea program may do better if all the parts are seen as an overall task and programmed accordingly.

I have seen fairly horrible code even in the old days that looks like this.

- Calculate something and put results in VAR which can be something complex like a data.frame containing raw data.
- Starting with XYZ as a data.frame, calculate a subset of the rwos or columns or whatever. Oh, wait, the data is not really in XYZ but in VAR. No problem, COPY it from VAR to XYZ and then continue.

I have seen people do something like this by asking questions like:

- How do I read in data from a file into a data.frame.
- How do I keep only some columns and ignore or delete others?
- How do I apply a condition that filters a data.frame to only keep some rows based on criteria?
- How do I create new columns based on calculations of other columns?
- How do I group my data and then generate a report based on the groups?
- How do I sort it by one column then another, perhaps descending.
- How do I take the highest value among several columns as the value of a new column?

The questions are endless and each may have a reasonable answer. But if each one places the result in the same name like "RESULT" and always looks for the material to be used in MYDATA then fitting all the steps together is painful. You sort of have to keep copying things or other techniques or maybe combine the steps into a pipeline where that is applicable. And it tells you nothing on how to deal with the order you should do the steps to make it run faster, or to deal ith errors like losing all your data so future steps may fail if not done carefully.

Like all tools, used carefully it works.

I had a recent interaction with someone who wanted a problem solved and said their internet search (not an AI) had suggested using a function called "unite()" might do what they want. I had never heard of it and looked into it and at first, it would not. I offered a step by step alternative that was in some sense harder but would work fine. Then I looked again and realized that I could overcome some of the shortcomings and make unite() work by twiddling some parameters such as telling it not to remove the other columns, and making the separator be "" and asking that NA values not be written as "NA" but be ignored. Suddenly, it worked!

If interested, the question was to take two data sets with different columns/names that have been tweaked to hold similar comparable values and merge them so every row now has both values where one of them is an NA, and make a new column containing just the non-NA component. That looks simple, but it takes some careful thought and preparation. My ifelse() approach works fine but is longer. And, no, it was not done in python but as an example, easily could have been.

I have not played much with these supposed AI but my guess is you probably could spell out your need carefully and in length in ways that get it to overcome some of what I said above. If, for example, you specify the names of the input and output of each step or specify you want a pipeline, it might be able to instantiate the code that it was fed on with your names and some of what it produces may be easier or more efficient.

But if we are talking about students learning how to compute, the ONLY valid answers are often based on what has already been taught in class. Fancy and elegant solutions may not get you a good grade. Relying on public modules that can stop being supported or that get changed, can be a trap.







Sent with Proton Mail secure email.

On Thursday, January 18th, 2024 at 8:55 AM, Alan Gauld via Tutor <tutor at python.org> wrote:


> On 15/01/2024 19:12, dn via Tutor wrote:
> 
> > Both of the 'Expert solutions' contained rather 'frilly bits' - perhaps
> > an attempt to convince us of Mastery rather than a true demonstration
> > thereof?
> 
> 
> But remember that none of the current set of AI bots are designed
> to prioritize finding "best" answers, nor even "correct" answers,
> only relevant ones and presenting these in a naturalist way that
> makes it look like a machine didn't do it.
> 
> Judging AI bots on the technical quality of their answers is
> really to judge the internet web sites that they scavenged
> the answers from.
> 
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
> http://www.amazon.com/author/alan_gauld
> Follow my photo-blog on Flickr at:
> http://www.flickr.com/photos/alangauldphotos
> 
> 
> 
> _______________________________________________
> Tutor maillist - Tutor at python.org
> To unsubscribe or change subscription options:
> https://mail.python.org/mailman/listinfo/tutor