From ethanrosenberg414 at gmail.com Mon Jan 1 12:22:08 2024 From: ethanrosenberg414 at gmail.com (Ethan Rosenberg) Date: Mon, 1 Jan 2024 12:22:08 -0500 Subject: [Tutor] uplownumpnct.py Message-ID: Tutor - What is my syntax error? #uplownumpnct.py #To determine uppercase, lowercase, punctuation and special characters in a string up= ['A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P','Q','R','S','T','U','V','W','X','Y','Z'] numm = [1,2,3,4,5,6,7,8,9,0] pnct = [:,;.,,] spec = [!,@,<,#,>,},$,%,&,?,*,(,),{,/,' '] sent = 'THIS is the way we wash 1 2 3 , , !' list2 = list(sent) lgn = len(sent) lgn2 = lgn print(lgn) for i in range(0,lgn-1): { if list2[i] in up: upp+= if list2[i] in low: lww+= if list2[i] in numm: numnum+= if list2[i] in pnct: numpct+= if list2[i] in spec: numspec+= } prnt def prnt: print('Num uppercase ', upp) print('Num lowercase ',lww) print('Num numbers ', numnum) print('Num punctuation ', numpnct) print('Num special characters' , numspec) #uplownumpnct.py #To determine uppercase, lowercase, punctuation and special characters in a string up= ['A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P','Q','R','S','T','U','V','W','X','Y','Z'] numm = [1,2,3,4,5,6,7,8,9,0] pnct = [:,;.,,] spec = [!,@,<,#,>,},$,%,&,?,*,(,),{,/,' '] sent = 'THIS is the way we wash 1 2 3 , , !' list2 = list(sent) lgn = len(sent) lgn2 = lgn print(lgn) for i in range(0,lgn-1): { if list2[i] in up: upp+= if list2[i] in low: lww+= if list2[i] in numm: numnum+= if list2[i] in pnct: numpct+= if list2[i] in spec: numspec+= } prnt def prnt: print('Num uppercase ', upp) print('Num lowercase ',lww) print('Num numbers ', numnum) print('Num punctuation ', numpnct) print('Num special characters' , numspec) Ethan Rosenberg From mats at wichmann.us Mon Jan 1 14:42:14 2024 From: mats at wichmann.us (Mats Wichmann) Date: Mon, 1 Jan 2024 12:42:14 -0700 Subject: [Tutor] uplownumpnct.py In-Reply-To: References: Message-ID: <926a650f-a142-49f7-b5f4-e75b93f3520d@wichmann.us> On 1/1/24 10:22, Ethan Rosenberg wrote: > Tutor - > > What is my syntax error? As people have asked you *multiple* times before, paste the full error message. It will tell you what the syntax error is. If you don't understand it, someone can help interpret - that's a whole lot easier than looking at code (often malformed by the mailer). This is an obvious one: > pnct = [:,;.,,] > spec = [!,@,<,#,>,},$,%,&,?,*,(,),{,/,' '] You can't just toss in a bunch of character that have syntactic meaning to Python. If you want a set of characters, you need to present them as a set of characters - since you're defining these as a list, you should enclose each punctuation/special character in quotes. If one of them is a quote itself, use the other quote mark for it. Also, you can get these from Python >>> import string >>> string.punctuation '!"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~' From alan.gauld at yahoo.co.uk Mon Jan 1 14:43:58 2024 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Mon, 1 Jan 2024 19:43:58 +0000 Subject: [Tutor] uplownumpnct.py In-Reply-To: References: Message-ID: On 01/01/2024 17:22, Ethan Rosenberg wrote: > Tutor - > > What is my syntax error? I don't know, where does Python say it is? Always post the full error message they tell us so much useful information and saves us having to guess! > #uplownumpnct.py > #To determine uppercase, lowercase, punctuation and special characters in a > string > > > up=['A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P','Q','R','S','T','U','V','W','X','Y','Z'] > numm = [1,2,3,4,5,6,7,8,9,0] > pnct = [:,;.,,] > spec = [!,@,<,#,>,},$,%,&,?,*,(,),{,/,' '] > > sent = 'THIS is the way we wash 1 2 3 , , !' > list2 = list(sent) > lgn = len(sent) > lgn2 = lgn > print(lgn) > > for i in range(0,lgn-1): > { > if list2[i] in up: > upp+= I don't know what you expect this to do but, whatever it is, it won't! Same with these below... > > if list2[i] in low: > lww+= > > if list2[i] in numm: > numnum+= > > if list2[i] in pnct: > numpct+= > > if list2[i] in spec: > numspec+= > > } > > prnt > > def prnt: This might be it? def defines a function. A function has a parameter enclosed parameter list(possibly empty) > print('Num uppercase ', upp) > print('Num lowercase ',lww) > print('Num numbers ', numnum) > print('Num punctuation ', numpnct) > print('Num special characters' , numspec) > #uplownumpnct.py -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Mon Jan 1 14:49:26 2024 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Mon, 1 Jan 2024 19:49:26 +0000 Subject: [Tutor] uplownumpnct.py In-Reply-To: References: Message-ID: On 01/01/2024 19:43, Alan Gauld via Tutor wrote: >> def prnt: > > This might be it? def defines a function. > A function has a parameter enclosed parameter list(possibly empty) Doh! "...a parentheses enclosed parameter list..."! sorry. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From mats at wichmann.us Mon Jan 1 15:04:54 2024 From: mats at wichmann.us (Mats Wichmann) Date: Mon, 1 Jan 2024 13:04:54 -0700 Subject: [Tutor] uplownumpnct.py In-Reply-To: References: Message-ID: On 1/1/24 10:22, Ethan Rosenberg wrote: > Tutor - > > What is my syntax error? > > #uplownumpnct.py > #To determine uppercase, lowercase, punctuation and special characters in a > string > > > up= > ['A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P','Q','R','S','T','U','V','W','X','Y','Z'] > numm = [1,2,3,4,5,6,7,8,9,0] > pnct = [:,;.,,] > spec = [!,@,<,#,>,},$,%,&,?,*,(,),{,/,' '] > > sent = 'THIS is the way we wash 1 2 3 , , !' > list2 = list(sent) > lgn = len(sent) > lgn2 = lgn > print(lgn) > > for i in range(0,lgn-1): > { > if list2[i] in up: > upp+= > > if list2[i] in low: > lww+= > > if list2[i] in numm: > numnum+= > > if list2[i] in pnct: > numpct+= > > if list2[i] in spec: > numspec+= > > } > > prnt > > def prnt: > print('Num uppercase ', upp) > print('Num lowercase ',lww) > print('Num numbers ', numnum) > print('Num punctuation ', numpnct) > print('Num special characters' , numspec) > #uplownumpnct.py > #To determine uppercase, lowercase, punctuation and special characters in a > string > > > up= > ['A','B','C','D','E','F','G','H','I','J','K','L','M','N','O','P','Q','R','S','T','U','V','W','X','Y','Z'] > numm = [1,2,3,4,5,6,7,8,9,0] > pnct = [:,;.,,] > spec = [!,@,<,#,>,},$,%,&,?,*,(,),{,/,' '] in fact, you can also get uppercase, etc. from the string module. >>> import string >>> string.digits '0123456789' >>> string.ascii_lowercase 'abcdefghijklmnopqrstuvwxyz' >>> string.ascii_uppercase 'ABCDEFGHIJKLMNOPQRSTUVWXYZ' >>> string.punctuation '!"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~' >>> string.whitespace ' \t\n\r\x0b\x0c' >>> > sent = 'THIS is the way we wash 1 2 3 , , !' > list2 = list(sent) > lgn = len(sent) you don't need any of that > lgn2 = lgn > print(lgn) > > for i in range(0,lgn-1): > { That is C/javascript/etc. syntax. Python delimits a code block following a : using required indentation, no braces involved. Or allowed. And... you can (and should) just iterate over the string directly, like: for c in sent: if c in up: upp += 1 Indeed, Python can also make that check for you using a string method: if c.isupper(): upp += 1 there's islower() and isdigit() and others. From PythonList at DancesWithMice.info Mon Jan 1 17:05:25 2024 From: PythonList at DancesWithMice.info (dn) Date: Tue, 2 Jan 2024 11:05:25 +1300 Subject: [Tutor] Class definitions Message-ID: <141e8ed4-1945-4c2f-a490-daa5073f5ee5@DancesWithMice.info> Are you training folk, or requiring in Code Reviews, the use of parentheses in stand-alone class definitions (or not)? The non-responsive fellow wrote a function-definition as: def prnt: (twice!) which is plainly incorrect because the def, the parentheses, and the colon are all compulsory [as discussed, elsewhere] - even if the parameter-list is empty. funcdef ::= [decorators] "def" funcname [type_params] "(" [parameter_list] ")" ["->" expression] ":" suite Whereas, a class-definition requires only the class and a colon. In this case, parentheses are only needed to delimit an inheritance argument-list. classdef ::= [decorators] "class" classname [type_params] [inheritance] ":" suite inheritance ::= "(" [argument_list] ")" That said, and IIRC carrying-over from Python2, the inheritance argument-list is separately optional. In other words, one may express an empty set of parentheses. Accordingly, this old-timer's personal habit/muscle-memory(?) is to type them quite unconsciously. - or maybe some vague-idea that they just might be needed in-future in the same way that we add commas after the last element of lists in case of later additions-and-alterations? Conversely: over the years, I've occasionally noted newcomers becoming confused between function-definitions and class-definitions. Not between function and class as constructs, but between (function) parameter-lists and (class) inheritance argument-lists. Accordingly, wondered if you feel there is virtue in writing class-names without parentheses, and only moving to such syntax upon reaching more advanced levels involving inheritance (and multiple-inheritance)? Web.Ref: https://docs.python.org/3/reference/compound_stmts.html -- Regards, =dn From alan.gauld at yahoo.co.uk Mon Jan 1 18:53:16 2024 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Mon, 1 Jan 2024 23:53:16 +0000 Subject: [Tutor] Class definitions In-Reply-To: <141e8ed4-1945-4c2f-a490-daa5073f5ee5@DancesWithMice.info> References: <141e8ed4-1945-4c2f-a490-daa5073f5ee5@DancesWithMice.info> Message-ID: On 01/01/2024 22:05, dn via Tutor wrote: > Are you training folk, or requiring in Code Reviews, the use of > parentheses in stand-alone class definitions (or not)? > I'm not quite sure who you are addressing here? > Accordingly, wondered if you feel there is virtue in writing class-names > without parentheses, and only moving to such syntax upon reaching more > advanced levels involving inheritance (and multiple-inheritance)? My personal take is that it would have been consistent with Python's "explicit is better" approach if class definitions always required the parens with object where appropriate class MyClass(object): Even though it meant more typing. But Guido obviously thought different and we are stuck with the definition structures. Since the idiomatic way of writing it is to omit the parens that's what I do and encourage others to do too. Idioms are powerful tools. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From PythonList at DancesWithMice.info Mon Jan 1 20:08:38 2024 From: PythonList at DancesWithMice.info (dn) Date: Tue, 2 Jan 2024 14:08:38 +1300 Subject: [Tutor] Class definitions In-Reply-To: References: <141e8ed4-1945-4c2f-a490-daa5073f5ee5@DancesWithMice.info> Message-ID: On 2/01/24 12:53, Alan Gauld via Tutor wrote: > On 01/01/2024 22:05, dn via Tutor wrote: >> Are you training folk, or requiring in Code Reviews, the use of >> parentheses in stand-alone class definitions (or not)? >> > > I'm not quite sure who you are addressing here? When helping folk start-out, the preference is to go with the one and only one way to do things. Code Review is about (organisational) convention. >> Accordingly, wondered if you feel there is virtue in writing class-names >> without parentheses, and only moving to such syntax upon reaching more >> advanced levels involving inheritance (and multiple-inheritance)? > > My personal take is that it would have been consistent with > Python's "explicit is better" approach if class definitions > always required the parens with object where appropriate > > class MyClass(object): > > Even though it meant more typing. A useful concept to understand - but often rather too theoretical for first-timers. Which has been true only since 'new style types' were brought-in (somewhere back in the mists of Python 2.n history). Thus, one of several changes to the way we define custom-classes. > But Guido obviously thought different and we are stuck with > the definition structures. Since the idiomatic way of writing > it is to omit the parens that's what I do and encourage others > to do too. Idioms are powerful tools. That's what I was wondering. Is there some authoritative statement for this? OTOH it seems a little odd, that some classes are defined with parentheses and others not. Hence the 'full' parentheses indicate a sub-class, whereas the 'empty' ones indicate a super-class or something that stands-alone and outside of any hierarchy. Consulting PEP-008 (latest update, last month) there is considerable discussion about "Designing for Inheritance" (much of which curls the hair of SOLID OOP-ers) but no mention of parentheses (that I noticed). Although, the one code-example does feature the ubiquitous Cartesian Co-ordinate as "class Point:"!? -- Regards, =dn From threesomequarks at proton.me Tue Jan 2 00:31:17 2024 From: threesomequarks at proton.me (ThreeBlindQuarks) Date: Tue, 02 Jan 2024 05:31:17 +0000 Subject: [Tutor] Class definitions In-Reply-To: References: <141e8ed4-1945-4c2f-a490-daa5073f5ee5@DancesWithMice.info> Message-ID: Dave and Alan and others, I think this discussion is relevant but widespread in many programming languages and environments. Often there is a formal way to completely spell-out exactly what you want to do but then someone feels it involves too much typing or attention to detail and comes up with a second or third way to do it that may be simpler in common cases. I am reminded of an annoyance in another language called R. The core language has been extended by many packages and quite a few packages decided to implement some version of piping by creating new operators. The details are not important and I will just use the word %PIPE% here to denote the many methods in pseudocode. The main point will be whether empty parentheses are allowed. So imagine a need to open up some data (in a data.frame format) and hand it to a function that selects only certain rows and pass the result to another that filters rows that satisfy a condition and perhaps call some functions that only need a single argument like say print(). The way some of these pipes worked was to sort of interpolate what was coming down the pipe into the first argument of the next function. So if only one argument was needed, the function needed no specified parameters and thus parentheses would be empty. Since the language has functional aspects, an object that is considered a function can be passed along without parentheses, normally. So you could say: data %PIPE% select(...) %PIPE% filter(...) %PIPE% print() or leave out parentheses for any function that does not need an argument or has a default as in: data %PIPE% select(...) %PIPE% filter(...) %PIPE% head %PIPE% print After years, some parts of the R community found these pipes so useful that the powers that be decided to add a native pipe, using a new symbol, to the language. So instead of: temp1 <- select(data, ...) temp2 <- filter(temp1, ...) temp3 <- head(temp2) print(temp3) You can just do:: data |> select(...) |> filter(...) |> head() |> print() But the kicker is this nice fast implementation was not made fully compatible with the most popular variant many had been using. They required a set of empty parentheses or the code will generate errors. They also made it much more complex to pass the piped result to anything other than as a first argument. But the tendency in many programming environments tends to be to keep adding features and python is no exception. One common feature for some kind of convenience is to allow less typing or less chance of errors. So another example might be how older languages often insisted on an exact number of arguments to many functions in exactly the same order. Sometimes you could leave some "blank" but had to leave the commas in places as in func(arg1,,arg3,,,,) and that could easily generate odd errors. Python allows quite a bit of flexibility in this regard and has a combination of positional parameters and named parameters and you can often just include options you want, perhaps named, so it is less ambiguous. The problem is that users disagree on what is really a better choice for them. Creating a class in which you have nothing more to say on the first line, is equivalent to providing no arguments and accepting the defaults, if applicable. Much of the actual initialization in those cases is done a bit differently than with a function declaration. But you can argue that it is far easier to parse and evaluate code if it contains patterns that can be counted on. A name followed immediately by something (or nothing) in parentheses is easier to look for than evaluating the object more deeply to see what it is or should be. But that only applies if your language has a well-thought-out rationale and frankly, languages like Python can be quite horrible in this regard as the number of uses in distinct ways of paired parentheses is astounding. Sometimes, as with curried code, you encounter rather monstrous constructions with dangling pairs of parentheses to indicate you are calling a function that returns a function that returns a function and providing arguments to various levels. I end by noting a reality. No matter what your personal style and choices, if you want others to read your code, or are working in a team that has made some rules/expectations, consider doing it the hard way rather than what makes it easier for you. Sent with Proton Mail secure email. On Monday, January 1st, 2024 at 8:08 PM, dn via Tutor wrote: > On 2/01/24 12:53, Alan Gauld via Tutor wrote: > > > On 01/01/2024 22:05, dn via Tutor wrote: > > > > > Are you training folk, or requiring in Code Reviews, the use of > > > parentheses in stand-alone class definitions (or not)? > > > > I'm not quite sure who you are addressing here? > > > When helping folk start-out, the preference is to go with the one and > only one way to do things. > > Code Review is about (organisational) convention. > > > > Accordingly, wondered if you feel there is virtue in writing class-names > > > without parentheses, and only moving to such syntax upon reaching more > > > advanced levels involving inheritance (and multiple-inheritance)? > > > > My personal take is that it would have been consistent with > > Python's "explicit is better" approach if class definitions > > always required the parens with object where appropriate > > > > class MyClass(object): > > > > Even though it meant more typing. > > > A useful concept to understand - but often rather too theoretical for > first-timers. > > Which has been true only since 'new style types' were brought-in > (somewhere back in the mists of Python 2.n history). Thus, one of > several changes to the way we define custom-classes. > > > But Guido obviously thought different and we are stuck with > > the definition structures. Since the idiomatic way of writing > > it is to omit the parens that's what I do and encourage others > > to do too. Idioms are powerful tools. > > > That's what I was wondering. Is there some authoritative statement for this? > > OTOH it seems a little odd, that some classes are defined with > parentheses and others not. > > Hence the 'full' parentheses indicate a sub-class, whereas the 'empty' > ones indicate a super-class or something that stands-alone and outside > of any hierarchy. > > Consulting PEP-008 (latest update, last month) there is considerable > discussion about "Designing for Inheritance" (much of which curls the > hair of SOLID OOP-ers) but no mention of parentheses (that I noticed). > Although, the one code-example does feature the ubiquitous Cartesian > Co-ordinate as "class Point:"!? > > -- > Regards, > =dn > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor From threesomequarks at proton.me Tue Jan 2 17:30:20 2024 From: threesomequarks at proton.me (ThreeBlindQuarks) Date: Tue, 02 Jan 2024 22:30:20 +0000 Subject: [Tutor] Debugging skills Message-ID: <_yrI7GYi2qa0aw0XP6MBCPsTj_3f2vJktj_eJME6mXz9eRxCEJyb5OKlSgK2ugZeeQvGIpgJUHtIKmWpZ3kmcaL0M1fAJekhyx-Ho9-MAfU=@proton.me> After some recent discussions here, I think the best gift for some learners is learning debugging skills. As an example Mr. Rosenberg probably could benefit from writing small amounts of code then stopping and testing the results up to that point to see if it was what was expected. You can use Python very interactively including various methods of cutting and pasting or a line or region at a time if using RSTUDIO to run it. So it can be useful to people to run some code, pause and check things like what is the current content of a variable, what "type" python thinks it is or if it is a composite object like a class, examine some of the contents. I am not talking about a formal debugger, just fairly basic stuff. When you encounter an error message, you definitely should not proceed with more code until it has been dealt with as some errors make the rest of your code be treated as nonsense. And, certainly, there are debuggers you can learn and use but mostly there is no need. If you do things like this and then still hit a wall, you can then ask a question here more intelligently and using the minimal amount of code that reproduces the error. Ages ago, I used to work and teach with programs written on punch cards and handed in and received back hours or days later. Those were primitive times where you felt you had to write a whole progra before submitting it. That is not the case with Python and incremental work is often a decent approach as long as you have a good editor or setup and know how to make sure to write modular code that can be tested and debugged in small chunks. The other thing to keep remembering is that when programming in a language, ONLY the rules for that language apply. Sometimes people forget that python uses indentation quite precisely and although extra parentheses and other kinds of brackets may seem harmless, sometimes they really aren't. Because of the indentation issues, unlike some other languages, copying code from one place to another, or just moving it within a program generally causes problems as it may not be aligned as expected. And some symbols simply do not mean the same thing as in some other language such as curly braces generally are reserved for a set or dictionary and not for grouping. I think people have a tendency to jump ahead and do more challenging things before they have mastered the basics. I know I often do but I also know enough to back away and examine my environment when things are not working or even switch to other methods I know already. My guess is some people are using a resource, such as an online list of programming exercises that are not really designed as a teaching experience where each section extends what has already been learned. They may be better off with some more formal course such as some of the free course on coursera or perhaps paid versions that do better grading and offer a certificate: https://www.coursera.org/search?query=python& Of course, if you are someone like me who mainly learns languages these days as a sort of hobby, ... From alan.gauld at yahoo.co.uk Tue Jan 2 18:27:36 2024 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Tue, 2 Jan 2024 23:27:36 +0000 Subject: [Tutor] Debugging skills In-Reply-To: <_yrI7GYi2qa0aw0XP6MBCPsTj_3f2vJktj_eJME6mXz9eRxCEJyb5OKlSgK2ugZeeQvGIpgJUHtIKmWpZ3kmcaL0M1fAJekhyx-Ho9-MAfU=@proton.me> References: <_yrI7GYi2qa0aw0XP6MBCPsTj_3f2vJktj_eJME6mXz9eRxCEJyb5OKlSgK2ugZeeQvGIpgJUHtIKmWpZ3kmcaL0M1fAJekhyx-Ho9-MAfU=@proton.me> Message-ID: On 02/01/2024 22:30, ThreeBlindQuarks via Tutor wrote: > ...the best gift for some learners is learning debugging skills. Interestingly the paper book version of my tutorial has a chapter on debugging. However, I've never got round to adding that chapter to the online version. I probably should... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From mats at wichmann.us Tue Jan 2 18:56:23 2024 From: mats at wichmann.us (Mats Wichmann) Date: Tue, 2 Jan 2024 16:56:23 -0700 Subject: [Tutor] Debugging skills In-Reply-To: References: <_yrI7GYi2qa0aw0XP6MBCPsTj_3f2vJktj_eJME6mXz9eRxCEJyb5OKlSgK2ugZeeQvGIpgJUHtIKmWpZ3kmcaL0M1fAJekhyx-Ho9-MAfU=@proton.me> Message-ID: On 1/2/24 16:27, Alan Gauld via Tutor wrote: > On 02/01/2024 22:30, ThreeBlindQuarks via Tutor wrote: >> ...the best gift for some learners is learning debugging skills. > > Interestingly the paper book version of my tutorial has a chapter on > debugging. However, I've never got round to adding that chapter > to the online version. I probably should... That would be cool. I think the topic of which debugging technique to choose is quite interesting. Inserting prints, logging events, adding decorators which emit useful information, actual debuggers - both the quite basic pdb and some more advanced variants, various sorts of profiling and other external inspection, and... test-driven-development is really a kind of debugging technique: you write a failing test, and then work until you can get the test to pass, that's really another form of debugging, as long as you keep the unit of test simple enough that you don't just go "I wrote the code, and I have no idea why it's failing my test". There's probably more things that I didn't call out explicitly. Different approaches are usually needed for differing situations. From PythonList at DancesWithMice.info Thu Jan 4 16:58:13 2024 From: PythonList at DancesWithMice.info (dn) Date: Fri, 5 Jan 2024 10:58:13 +1300 Subject: [Tutor] Debugging skills In-Reply-To: References: <_yrI7GYi2qa0aw0XP6MBCPsTj_3f2vJktj_eJME6mXz9eRxCEJyb5OKlSgK2ugZeeQvGIpgJUHtIKmWpZ3kmcaL0M1fAJekhyx-Ho9-MAfU=@proton.me> Message-ID: <8ac348f9-feca-4609-9220-ef10c0399ca1@DancesWithMice.info> Prefacing these remarks with the comment that I don't (currently) teach Python-from-scratch, nor for that matter, procedural coding from-scratch (and why would I, given the wide-availability of materials?). Many of the issues-raised and under-lying themes, apply at a variety of 'levels' of learning... On 3/01/24 12:56, Mats Wichmann wrote: > On 1/2/24 16:27, Alan Gauld via Tutor wrote: >> On 02/01/2024 22:30, ThreeBlindQuarks via Tutor wrote: >>> ...the best gift for some learners is learning debugging skills. >> >> Interestingly the paper book version of my tutorial has a chapter on >> debugging. However, I've never got round to adding that chapter >> to the online version. I probably should... > > That would be cool. +1 > I think the topic of which debugging technique to choose is quite > interesting.? Inserting prints, logging events, adding decorators which > emit useful information, actual debuggers - both the quite basic pdb and > some more advanced variants, various sorts of profiling and other > external inspection, and... test-driven-development is really a kind of > debugging technique: you write a failing test, and then work until you > can get the test to pass, that's really another form of debugging, as > long as you keep the unit of test simple enough that you don't just go > "I wrote the code, and I have no idea why it's failing my test". There's > probably more things that I didn't call out explicitly. Different > approaches are usually needed for differing situations. The over-bearing issue (in both senses) is that starting-from-scratch presents a significant 'learning-curve'. The challenge with any (properly-structured) course is less 'what to teach' but how to control the learning into manageable 'chunks'. I think we've previously discussed or touched-on the question of choosing an editor (quite aside from the 'real men use NotePad' meme). Do we recommend neophytes start with a simple tool which does not add significantly to the cognitive-load; or should we 'start as we mean to go on' with a professional-level tool, eg PyCharm; and save 'extra effort' later? This ties-in with the above discussion because a casual look-around reveals very few courses or books which start-off in the Python REPL despite its characteristics of immediate-feedback and (better) line-by-line (syntax) error-revelation. (cf its impermanence!) Yet, the REPL is such a useful go-to tool to track-down issues, as illustrated in the OP (and so-frequently making Tutor responses more difficult than they need-be). Perhaps, those of use with experience of compiled-languages can more readily appreciate these advantages? (whereas to quote Joni Mitchell, others don't know what they've got 'til it's gone?) Which led me to wonder if an intro-level session might eschew a local-editor altogether. There are a number of REPL-its, Fiddles, "on-line compilers" (sic), all the way 'up' to full-fat on-line IDEs, eg PythonAnywhere and Cloud9. They all save the hassles of an installation step (and the inevitable consequent question from mystified MS-Win users). Yet this is not the real-topic! At this point I'd like to digress to mention another phenomenon which we (in this discussion) have to consciously stop to consider: when we learned 'software' back in the ?good old days, there was also a fair-amount of parallel instruction in 'hardware'. I recall building circuits with microprocessors and learning about "registers" cf stacks, interactions between memory and ALUs, and so-on. Today's learner seldom enjoys such a more-rounded background - and many courses eschew same simply to 'make progress' or because of a perception that it is "not necessary" (ref: OP-comments about "jump ahead"). Similarly, back-then, many came from a math-background. Accordingly, once the assignment-operator was 'disconnected' from the mathematical "equals", learning something like FORTRAN or BASIC was much closer to an exercise in mechanisation than learning something 'new'. Again, today's learners seldom 'enter' through that portal and often struggle with the concepts behind 'a variable is like an algebraic quantity' in the way it "stands for" a value - which may change. Change? Huh? My name doesn't change, why do these 'names' have different values/meanings??? Accordingly, I wondered if one could commence training with Phil Guo's excellent PythonTutor? This combination of text-expressed code, graphical representation, and the opportunity to limit execution to its line-by-line progression, neatly attacks many of the issues and concerns raised. (to 'close the circle', if an installation-step is not regarded as an over-large potential pot-hole, consider Thonny - the "Python IDE for beginners", which also offers the above advantages) The impatience of learners has always been a factor. We have a 'new generation' of folk who have the idea to 'get into' AI. Rather than Python being seen as a tool, some even have the idea that learning Python 'first' is an unnecessary block to their goal and want to learn 'just the basics' and move-on. Sadly, many (training colleagues) have caved-in to the idea that 'the customer is always right'; and the over-sold enthusiasm of many a YouTuber who offers 'training' which depends on (their) plethora of assumptions (chief amongst which is that nothing will ever go-wrong - hence instructions commencing with "simply..."). Even basic psychological research reveals the fallacy of "students are the best judges of what [they need to] know" (in both present and future tenses!), eg Kruger and Dunning. This has even been used by an idiot-politician attempting to avoid questioning and the use of actual-facts (to justify what became a futile and fatal endeavor). That said, the most important aspect of training is to encourage rather than discourage. Thus, curiosity (all too rare) is to be kindled (and directed), yet enactment kept achievable! This is where the 'project approach', which certainly has virtue, "such as an online list of programming exercises that are not really designed as a teaching experience where each section extends what has already been learned." [OP] is a faint-hope or a 'silver bullet', and not a 'fast-track' to 'real programming' (whatever that is?). It is the combination of a good understanding (which comes from a designed curriculum) and practical proof of learning ("mastery") which leads to long-term success. There have been many trainees who have tackled an early-challenge of writing code to convert temperature between Fahrenheit and Celsius (Centigrade). It's a further step of meta-cognition to realise that once-solved, the techniques-learned can be applied to similar problems, eg converting between financial-currencies, or between metric and 'imperial' measures. (the same psychology powering the idea of 'programming patterns'!). Accordingly, unless 'made' to stop and think, many trainees will solve that little assignment and then want to move-on quickly - without realising the wider picture. Similarly, the process of breaking things down, or building them up ("step-wise refinement" - learned by many of us at the feet of Niklaus Wirth's books, RiP!) is not an obvious problem-solving approach to many (inside or outside of IT). Accordingly, a need to train - but when? (before writing any code/I just want to get on with it; or after trainees get themselves into a jam/"why didn't you explain this to begin with?"). Hind-sight is such a wonderful thing! However, in all cases, such 'received wisdom' is built upon 'learning' - and the cost/value of that learning is seldom factored into such debate/criticism! There are numbers of web-pages which advise 'how to post' or 'how to express your question', but 'the bottom line' is that very few read them - and even fewer BEFORE their first post. We see this in training courses, with all aspects of 'admin' - trainees prefer to skip such 'fluff' (polite re-wording) because they just want to 'get on with it'. Few trainees have been introduced to the 'rubber duck'. Consequently, we face questions which can't be answered without further questions - which, in-turn, often causes the OP to grumble. However, if unable to coherently ask questions of humans, how do such questioners expect to be able to communicate with a (pedantic) machine? Back to the theme: How many accelerated code-camp style courses or 'simply...' YouTube series, include 'soft skills' in their content? When post-ers fail to follow-up, but persist in asking similar questions, ignoring advice, the easiest course is to copy-paste. Alternately, because we (on this list) are all volunteers, ignore them - we all have better uses for our time (a tactic not available when I'm at-work!). My research-area encompasses the high drop-out rate from Internet-based courses (actually, any "distance course" short of legal-compulsion, eg Australia's historic "School of the air"). Accordingly, I've not looked-at a slightly-similar concern: the many people 'start' in IT/programming but quit-the-business within a few years. Why does this happen? Unreasonable demands and 'stress' are obvious factors. I wonder if the type (and depth) of training somehow contributes? Have you looked at this end of things? (hah! ,,, couldn't resist the pun) Web.Refs Tool: https://pythontutor.com/ Paper for Tutor-consideration: articles/java-visualizer.html Thonny: https://thonny.org/ Known unknowns: https://en.wikipedia.org/wiki/There_are_unknown_unknowns A reasonably-readable article: https://www.learning-mind.com/known-unknowns-unknown-unknowns/ Rubber-ducking: https://en.wikipedia.org/wiki/Rubber_duck_debugging -- Regards, =dn From alan.gauld at yahoo.co.uk Thu Jan 4 20:46:55 2024 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Fri, 5 Jan 2024 01:46:55 +0000 Subject: [Tutor] Debugging skills In-Reply-To: <8ac348f9-feca-4609-9220-ef10c0399ca1@DancesWithMice.info> References: <_yrI7GYi2qa0aw0XP6MBCPsTj_3f2vJktj_eJME6mXz9eRxCEJyb5OKlSgK2ugZeeQvGIpgJUHtIKmWpZ3kmcaL0M1fAJekhyx-Ho9-MAfU=@proton.me> <8ac348f9-feca-4609-9220-ef10c0399ca1@DancesWithMice.info> Message-ID: On 04/01/2024 21:58, dn via Tutor wrote: > Prefacing these remarks with the comment that I don't (currently) teach > Python-from-scratch, nor for that matter, procedural coding from-scratch > (and why would I, given the wide-availability of materials?). > > Many of the issues-raised and under-lying themes, apply at a variety of > 'levels' of learning... I'm hoping to respond to this more thoroughly later (busy, busy!) but it strikes me that this (and several more of your recent posts) might be of greater interest on the Python Edu SIG? Are you a member of that community? They have a mailing list which is active but quiet. I don't mean you should not post this stuff here, just that the SIG has members who might be actively interested and are not on Tutor.... >>> Interestingly the paper book version of my tutorial has a chapter on >>> debugging. However, I've never got round to adding that chapter >>> to the online version. I probably should...> +1 OK, I'll add it to my list. I'm already working on a new book project(or more accurately resurrecting an old book project from 20 years ago!) > This ties-in with the above discussion because a casual look-around > reveals very few courses or books which start-off in the Python REPL > despite its characteristics of immediate-feedback Me, me! :-) The first 5 chapters/topics in my tutorial are all from the REPL. (Although 3 of those are really about concepts and getting Python installed ec!) I introduce the idea of an editor and saving your work in a file in chapter 6. But I do regularly refer back to the REPL as a way of testing functions and classes etc in later topics. > (whereas to quote Joni Mitchell, others don't know what they've got 'til > it's gone?) Now I'm humming along to "...put up a parking lot!" > Known unknowns: https://en.wikipedia.org/wiki/There_are_unknown_unknowns > A reasonably-readable article: > https://www.learning-mind.com/known-unknowns-unknown-unknowns/ I always felt Rumsfeld got unfair criticism for that comment. He was absolutely correct in what he said. (Albeit using the statement of fact to blow a smokescreen over the Iraq question of the day) But it seems the journalists at the time just didn't get the concept! -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From threesomequarks at proton.me Thu Jan 4 22:51:41 2024 From: threesomequarks at proton.me (ThreeBlindQuarks) Date: Fri, 05 Jan 2024 03:51:41 +0000 Subject: [Tutor] Debugging skills In-Reply-To: References: <_yrI7GYi2qa0aw0XP6MBCPsTj_3f2vJktj_eJME6mXz9eRxCEJyb5OKlSgK2ugZeeQvGIpgJUHtIKmWpZ3kmcaL0M1fAJekhyx-Ho9-MAfU=@proton.me> <8ac348f9-feca-4609-9220-ef10c0399ca1@DancesWithMice.info> Message-ID: <5yA0C5dVbnkTiNIPz-DUPMHAqB9gd7mUPpPbEKQTyPyrsJXXxeUl_w37ZG27fo3PwHNdvWK-t2vus0Tcpo_RpMiPoM_qXDfsLcU47WU_u2U=@proton.me> I would like to point out a debugging aspect above the REPL that I rarely see people use. Take a programming environment like RSTUDIO that actually supports python and other languages too. It lets you split the screen into multiple zones that include files you are editing and a window that shows the console they are running in that you can interact with. But an interesting one shows all your variables and their current values! When you stop a python script at some level (and you can run it one expression at a time easily) you can pause and look at the variables of variables right there OR go to the console and type commands including ones that show the contents whatever way you want. It is one thing to teach a LANGUAGE and another to teach an environment. But many environments are made precisely to make some aspects of programming easier. Another example would be variants of a Jupiter notebook. You create chunks of code. You can run them individually or run all of them up to a point. Obviously you can thus stop at various points and add a chunk to evaluate the state up to there. And, of course, running under various debuggers and setting breakpoints, can produce valuable info. A student of the language can then learn techniques that allow them to more rapidly figure out which small chunks of code are not doing what is expected and perhaps fix them. The concept of writing all your code first and then debugging should be relegated to history. Sent with Proton Mail secure email. On Thursday, January 4th, 2024 at 8:46 PM, Alan Gauld via Tutor wrote: > On 04/01/2024 21:58, dn via Tutor wrote: > > > Prefacing these remarks with the comment that I don't (currently) teach > > Python-from-scratch, nor for that matter, procedural coding from-scratch > > (and why would I, given the wide-availability of materials?). > > > > Many of the issues-raised and under-lying themes, apply at a variety of > > 'levels' of learning... > > > I'm hoping to respond to this more thoroughly later (busy, busy!) > but it strikes me that this (and several more of your recent posts) > might be of greater interest on the Python Edu SIG? Are you a > member of that community? They have a mailing list which is > active but quiet. > > I don't mean you should not post this stuff here, just that > the SIG has members who might be actively interested and are > not on Tutor.... > > > > > Interestingly the paper book version of my tutorial has a chapter on > > > > debugging. However, I've never got round to adding that chapter > > > > to the online version. I probably should...> +1 > > > OK, I'll add it to my list. I'm already working on a new book > project(or more accurately resurrecting an old book project > from 20 years ago!) > > > This ties-in with the above discussion because a casual look-around > > reveals very few courses or books which start-off in the Python REPL > > despite its characteristics of immediate-feedback > > > Me, me! :-) > The first 5 chapters/topics in my tutorial are all from the REPL. > (Although 3 of those are really about concepts and getting Python > installed ec!) I introduce the idea of an editor and saving your > work in a file in chapter 6. > > But I do regularly refer back to the REPL as a way of testing > functions and classes etc in later topics. > > > (whereas to quote Joni Mitchell, others don't know what they've got 'til > > it's gone?) > > > Now I'm humming along to "...put up a parking lot!" > > > Known unknowns: https://en.wikipedia.org/wiki/There_are_unknown_unknowns > > A reasonably-readable article: > > https://www.learning-mind.com/known-unknowns-unknown-unknowns/ > > > I always felt Rumsfeld got unfair criticism for that comment. > He was absolutely correct in what he said. (Albeit using the > statement of fact to blow a smokescreen over the Iraq question > of the day) But it seems the journalists at the time just > didn't get the concept! > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.amazon.com/author/alan_gauld > Follow my photo-blog on Flickr at: > http://www.flickr.com/photos/alangauldphotos > > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor From mats at wichmann.us Fri Jan 5 08:56:31 2024 From: mats at wichmann.us (Mats Wichmann) Date: Fri, 5 Jan 2024 06:56:31 -0700 Subject: [Tutor] Debugging skills In-Reply-To: <8ac348f9-feca-4609-9220-ef10c0399ca1@DancesWithMice.info> References: <_yrI7GYi2qa0aw0XP6MBCPsTj_3f2vJktj_eJME6mXz9eRxCEJyb5OKlSgK2ugZeeQvGIpgJUHtIKmWpZ3kmcaL0M1fAJekhyx-Ho9-MAfU=@proton.me> <8ac348f9-feca-4609-9220-ef10c0399ca1@DancesWithMice.info> Message-ID: <39c95ffa-1962-4d64-8f2b-ce5918b163f7@wichmann.us> On 1/4/24 14:58, dn via Tutor wrote: >>> Interestingly the paper book version of my tutorial has a chapter on >>> debugging. However, I've never got round to adding that chapter >>> to the online version. I probably should... >> >> That would be cool. while we're nattering on a whole range of sort-of-debugging topics, here's something I've occasionally wondered about, when a beginner has written something that makes sense only to them, not to the Python interpreter. Regexes are confusing, so there are several websites (see for example regex101.com) where you can enter your "code" (regex) and as it's interpreted, will show you an explanation in words in a separate pane. Does anyone know if there's a similar thing for Python code? "This is what you wrote actually means to the interpreter". Ones I know about (like at Programiz, w3schools, online-python) don't attempt the explanation part. It's not something that's likely to be useful for very many cases, but I could see where in the beginning there might be some benefit... From alan.gauld at yahoo.co.uk Fri Jan 5 12:34:24 2024 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Fri, 5 Jan 2024 17:34:24 +0000 Subject: [Tutor] Debugging skills In-Reply-To: <39c95ffa-1962-4d64-8f2b-ce5918b163f7@wichmann.us> References: <_yrI7GYi2qa0aw0XP6MBCPsTj_3f2vJktj_eJME6mXz9eRxCEJyb5OKlSgK2ugZeeQvGIpgJUHtIKmWpZ3kmcaL0M1fAJekhyx-Ho9-MAfU=@proton.me> <8ac348f9-feca-4609-9220-ef10c0399ca1@DancesWithMice.info> <39c95ffa-1962-4d64-8f2b-ce5918b163f7@wichmann.us> Message-ID: On 05/01/2024 13:56, Mats Wichmann wrote: > interpreter. Regexes are confusing, so there are several websites (see > for example regex101.com) where you can enter your "code" (regex) and as > it's interpreted, will show you an explanation in words in a separate > pane. Does anyone know if there's a similar thing for Python code? > "This is what you wrote actually means to the interpreter". Ones I know > about (like at Programiz, w3schools, online-python) don't attempt the That would be good but I think it would be difficult to code, especially when dealing with issues like recursion - which a lot of newbies stumble upon by accident! I certainly don't know of anything like that in any general programming language let alone Python. (I could probably code it for assembler but anything beyond that boggles my brain!) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From threesomequarks at proton.me Fri Jan 5 17:01:28 2024 From: threesomequarks at proton.me (ThreeBlindQuarks) Date: Fri, 05 Jan 2024 22:01:28 +0000 Subject: [Tutor] Debugging skills In-Reply-To: <39c95ffa-1962-4d64-8f2b-ce5918b163f7@wichmann.us> References: <_yrI7GYi2qa0aw0XP6MBCPsTj_3f2vJktj_eJME6mXz9eRxCEJyb5OKlSgK2ugZeeQvGIpgJUHtIKmWpZ3kmcaL0M1fAJekhyx-Ho9-MAfU=@proton.me> <8ac348f9-feca-4609-9220-ef10c0399ca1@DancesWithMice.info> <39c95ffa-1962-4d64-8f2b-ce5918b163f7@wichmann.us> Message-ID: Mats, In theory, regular expressions were intended to be largely language independent and initially fairly simple. So any stand-alone program could take a regular expression and try to tell you in other more detailed words what it might be doing. In practice, languages like PERL made their own extensions or interpretations and there are often flags external to the RE string that also guide what it does and those are far from standardized. Python has multiple variations including asking for some PERL compatibility. Still, it may be possible to have a tool that takes in a string containing the RE, perhaps sanitizes it by removing for example the comments allowed in some versions, asks some questions on what setting you are using, and gives a half decent answer. In practice, I have often found it best NOT to make a humongously complex RE. Instead, some purposes are better done in stages, including some not directly using regular expressions and the stages together perform the required result with each step being able to be somewhat understood and results checked. I suspect a really complex RE, with any tool, might be as hard to understand as some legalese that keep mumbling about the party of the second part until you forget what the topic is and start over. Sent with Proton Mail secure email. On Friday, January 5th, 2024 at 8:56 AM, Mats Wichmann wrote: > On 1/4/24 14:58, dn via Tutor wrote: > > > > > Interestingly the paper book version of my tutorial has a chapter on > > > > debugging. However, I've never got round to adding that chapter > > > > to the online version. I probably should... > > > > > > That would be cool. > > > while we're nattering on a whole range of sort-of-debugging topics, > here's something I've occasionally wondered about, when a beginner has > written something that makes sense only to them, not to the Python > interpreter. Regexes are confusing, so there are several websites (see > for example regex101.com) where you can enter your "code" (regex) and as > it's interpreted, will show you an explanation in words in a separate > pane. Does anyone know if there's a similar thing for Python code? > "This is what you wrote actually means to the interpreter". Ones I know > about (like at Programiz, w3schools, online-python) don't attempt the > explanation part. It's not something that's likely to be useful for > very many cases, but I could see where in the beginning there might be > some benefit... > > > > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor From roel at roelschroeven.net Fri Jan 5 17:14:56 2024 From: roel at roelschroeven.net (Roel Schroeven) Date: Fri, 5 Jan 2024 23:14:56 +0100 Subject: [Tutor] Debugging skills In-Reply-To: <39c95ffa-1962-4d64-8f2b-ce5918b163f7@wichmann.us> References: <_yrI7GYi2qa0aw0XP6MBCPsTj_3f2vJktj_eJME6mXz9eRxCEJyb5OKlSgK2ugZeeQvGIpgJUHtIKmWpZ3kmcaL0M1fAJekhyx-Ho9-MAfU=@proton.me> <8ac348f9-feca-4609-9220-ef10c0399ca1@DancesWithMice.info> <39c95ffa-1962-4d64-8f2b-ce5918b163f7@wichmann.us> Message-ID: <20e9f427-f5e2-46be-93bc-75bc2973521a@roelschroeven.net> Mats Wichmann schreef op 5/01/2024 om 14:56: > while we're nattering on a whole range of sort-of-debugging topics, > here's something I've occasionally wondered about, when a beginner has > written something that makes sense only to them, not to the Python > interpreter. Regexes are confusing, so there are several websites (see > for example regex101.com) where you can enter your "code" (regex) and as > it's interpreted, will show you an explanation in words in a separate > pane. Does anyone know if there's a similar thing for Python code? > "This is what you wrote actually means to the interpreter". Ones I know > about (like at Programiz, w3schools, online-python) don't attempt the > explanation part. It's not something that's likely to be useful for > very many cases, but I could see where in the beginning there might be > some benefit... Python Tutor goes a bit in that direction, though not nearly as far as those regex websites. https://pythontutor.com/python-compiler.html I think it's mostly useful to visualize how recursion works. (As an aside, not that despite the name it can also work for Java, C, C++, and JavaScript.) -- "This planet has - or rather had - a problem, which was this: most of the people living on it were unhappy for pretty much of the time. Many solutions were suggested for this problem, but most of these were largely concerned with the movement of small green pieces of paper, which was odd because on the whole it wasn't the small green pieces of paper that were unhappy." -- Douglas Adams From mats at wichmann.us Sat Jan 6 10:36:21 2024 From: mats at wichmann.us (Mats Wichmann) Date: Sat, 6 Jan 2024 08:36:21 -0700 Subject: [Tutor] Debugging skills In-Reply-To: <20e9f427-f5e2-46be-93bc-75bc2973521a@roelschroeven.net> References: <_yrI7GYi2qa0aw0XP6MBCPsTj_3f2vJktj_eJME6mXz9eRxCEJyb5OKlSgK2ugZeeQvGIpgJUHtIKmWpZ3kmcaL0M1fAJekhyx-Ho9-MAfU=@proton.me> <8ac348f9-feca-4609-9220-ef10c0399ca1@DancesWithMice.info> <39c95ffa-1962-4d64-8f2b-ce5918b163f7@wichmann.us> <20e9f427-f5e2-46be-93bc-75bc2973521a@roelschroeven.net> Message-ID: <8deddd0e-9a37-4065-b0b2-0d5a97c612f3@wichmann.us> On 1/5/24 15:14, Roel Schroeven wrote: > Mats Wichmann schreef op 5/01/2024 om 14:56: >> while we're nattering on a whole range of sort-of-debugging topics, >> here's something I've occasionally wondered about, when a beginner has >> written something that makes sense only to them, not to the Python >> interpreter.? Regexes are confusing, so there are several websites (see >> for example regex101.com) where you can enter your "code" (regex) and as >> it's interpreted, will show you an explanation in words in a separate >> pane.? Does anyone know if there's a similar thing for Python code? >> "This is what you wrote actually means to the interpreter".? Ones I know >> about (like at Programiz, w3schools, online-python) don't attempt the >> explanation part.? It's not something that's likely to be useful for >> very many cases, but I could see where in the beginning there might be >> some benefit... > > Python Tutor goes a bit in that direction, though not nearly as far as > those regex websites. > > https://pythontutor.com/python-compiler.html > > I think it's mostly useful to visualize how recursion works. hey thanks - there are actually cases where that could be useful. yes, you have to step through it like in a debugger, but since it shows all the interesting variables updating as you go (pdb is somewhat limited in this regard), it makes a nice visual of what's going on. I'll keep track of that one for my toolbox of recommendations :-) From jjhartley at gmail.com Tue Jan 9 13:06:22 2024 From: jjhartley at gmail.com (James Hartley) Date: Tue, 9 Jan 2024 12:06:22 -0600 Subject: [Tutor] import issues Message-ID: I admit that I have not internalized how Python's import system works yet. Given the following directory structure: +-project | +-src | | | +-__init__.py | +-foo.py | +-tests | +-test_foo.py Currently, __init__.py is empty. What needs to be placed in __init__.py and test_foo.py in order for pytest to execute all tests in test_foo.py? pytest is being executed while in the project directory. With: from foo import Foo --or-- from src.foo import Foo defined in test_foo.py, pytest reports "ModuleNotFoundError: No module named 'src'". Any and all comments would be appreciated. Any links to online sources would also be appreciated. Thanks! j From PythonList at DancesWithMice.info Tue Jan 9 17:29:52 2024 From: PythonList at DancesWithMice.info (dn) Date: Wed, 10 Jan 2024 11:29:52 +1300 Subject: [Tutor] import issues In-Reply-To: References: Message-ID: <6ed59339-6103-41ed-8dec-dd7ef2ef58ce@DancesWithMice.info> On 10/01/24 07:06, James Hartley wrote: > I admit that I have not internalized how Python's import system works yet. > Given the following directory structure: > > +-project > | > +-src > | | > | +-__init__.py > | +-foo.py > | > +-tests > | > +-test_foo.py > > Currently, __init__.py is empty. What needs to be placed in __init__.py Nothing - it's not relevant to this discussion. > and test_foo.py in order for pytest to execute all tests in test_foo.py? > pytest is being executed while in the project directory. With: > > from foo import Foo > > --or-- > > from src.foo import Foo > > defined in test_foo.py, pytest reports "ModuleNotFoundError: No module > named 'src'". > > Any and all comments would be appreciated. Any links to online sources > would also be appreciated. Thanks! I use the latter, except: - using PyCharm's auto-testing feature. However, 'translating' those commands back to manual-typing, would be something like: python -m pytest /path/project/tests/test_test_foo.py --no-header --no-summary -q - prepending "python -m" which configures the sys.path to include current/project directory (and hence the "src.foo" relative-addressing) which likely accounts for the issues outlined (above) Web.Refs: Specifics: https://docs.pytest.org/en/7.4.x/explanation/pythonpath.html#pytest-vs-python-m-pytest and the rest of the online-manual (although I have to say, even after years of use, I find it somewhat overwhelming) Somewhat easier is the 'book of words', Brian Okken's: https://www.amazon.com.au/Python-Testing-pytest-Effective-Scalable/dp/1680508601/ref=sr_1_1?keywords=brian+okken&qid=1704838720&sr=8-1 (disclaimer: there are other book-sellers. No commission claimed) Coursera on-line training: (amongst many 'Python' courses) Testing and Debugging Python https://www.coursera.org/projects/testing-and-debugging-python - is a $paid project approach Possibly slightly off (your) topic is LearnQuest's 'TDD Specialisation': https://www.coursera.org/learn/test-driven-development-overview (which I think can be "audited" for $free) NB have not completed either, and thus can't comment on quality, nor degree of pytest-content. -- Regards, =dn From mats at wichmann.us Tue Jan 9 18:01:17 2024 From: mats at wichmann.us (Mats Wichmann) Date: Tue, 9 Jan 2024 16:01:17 -0700 Subject: [Tutor] import issues In-Reply-To: <6ed59339-6103-41ed-8dec-dd7ef2ef58ce@DancesWithMice.info> References: <6ed59339-6103-41ed-8dec-dd7ef2ef58ce@DancesWithMice.info> Message-ID: <4f679bf2-2c4c-4088-8083-13286e59e943@wichmann.us> On 1/9/24 15:29, dn via Tutor wrote: > On 10/01/24 07:06, James Hartley wrote: >> ? I admit that I have not internalized how Python's import system >> works yet. >> Given the following directory structure: >> >> +-project >> ???? | >> ???? +-src >> ???? |?? | >> ???? |?? +-__init__.py >> ???? |?? +-foo.py >> ???? | >> ???? +-tests >> ???????? | >> ???????? +-test_foo.py >> >> Currently, __init__.py is empty.? What needs to be placed in __init__.py > > Nothing - it's not relevant to this discussion. > > >> and test_foo.py in order for pytest to execute all tests in test_foo.py? >> pytest is being executed while in the project directory.? With: >> >> from foo import Foo >> >> --or-- >> >> from src.foo import Foo >> >> defined in test_foo.py, pytest reports "ModuleNotFoundError: No module >> named 'src'". >> >> Any and all comments would be appreciated.? Any links to online sources >> would also be appreciated. Thanks! > > I use the latter, except: > > - using PyCharm's auto-testing feature. However, 'translating' those > commands back to manual-typing, would be something like: > > python -m pytest /path/project/tests/test_test_foo.py --no-header > --no-summary -q > > - prepending "python -m" > which configures the sys.path to include current/project directory (and > hence the "src.foo" relative-addressing) which likely accounts for the > issues outlined (above) > > > Web.Refs: > Specifics: > https://docs.pytest.org/en/7.4.x/explanation/pythonpath.html#pytest-vs-python-m-pytest > and the rest of the online-manual (although I have to say, even after > years of use, I find it somewhat overwhelming) > > Somewhat easier is the 'book of words', Brian Okken's: > https://www.amazon.com.au/Python-Testing-pytest-Effective-Scalable/dp/1680508601/ref=sr_1_1?keywords=brian+okken&qid=1704838720&sr=8-1 > (disclaimer: there are other book-sellers. No commission claimed) > > Coursera on-line training: (amongst many 'Python' courses) > Testing and Debugging Python > https://www.coursera.org/projects/testing-and-debugging-python > - is a $paid project approach > Possibly slightly off (your) topic is LearnQuest's 'TDD Specialisation': > https://www.coursera.org/learn/test-driven-development-overview > (which I think can be "audited" for $free) > NB have not completed either, and thus can't comment on quality, nor > degree of pytest-content. Lotsa good stuff here, too: https://www.obeythetestinggoat.com/ From PythonList at DancesWithMice.info Tue Jan 9 19:01:33 2024 From: PythonList at DancesWithMice.info (dn) Date: Wed, 10 Jan 2024 13:01:33 +1300 Subject: [Tutor] import issues In-Reply-To: <4f679bf2-2c4c-4088-8083-13286e59e943@wichmann.us> References: <6ed59339-6103-41ed-8dec-dd7ef2ef58ce@DancesWithMice.info> <4f679bf2-2c4c-4088-8083-13286e59e943@wichmann.us> Message-ID: <4992066d-7cf1-46ec-8c6c-64db1d58a5c9@DancesWithMice.info> On 10/01/24 12:01, Mats Wichmann wrote: > On 1/9/24 15:29, dn via Tutor wrote: >> On 10/01/24 07:06, James Hartley wrote: >>> ? I admit that I have not internalized how Python's import system >>> works yet. >>> Given the following directory structure: >>> >>> +-project >>> ???? | >>> ???? +-src >>> ???? |?? | >>> ???? |?? +-__init__.py >>> ???? |?? +-foo.py >>> ???? | >>> ???? +-tests >>> ???????? | >>> ???????? +-test_foo.py >>> >>> Currently, __init__.py is empty.? What needs to be placed in __init__.py >> >> Nothing - it's not relevant to this discussion. >> >> >>> and test_foo.py in order for pytest to execute all tests in test_foo.py? >>> pytest is being executed while in the project directory.? With: >>> >>> from foo import Foo >>> >>> --or-- >>> >>> from src.foo import Foo >>> >>> defined in test_foo.py, pytest reports "ModuleNotFoundError: No module >>> named 'src'". >>> >>> Any and all comments would be appreciated.? Any links to online sources >>> would also be appreciated. Thanks! >> >> I use the latter, except: >> >> - using PyCharm's auto-testing feature. However, 'translating' those >> commands back to manual-typing, would be something like: >> >> python -m pytest /path/project/tests/test_test_foo.py --no-header >> --no-summary -q >> >> - prepending "python -m" >> which configures the sys.path to include current/project directory >> (and hence the "src.foo" relative-addressing) which likely accounts >> for the issues outlined (above) >> >> >> Web.Refs: >> Specifics: >> https://docs.pytest.org/en/7.4.x/explanation/pythonpath.html#pytest-vs-python-m-pytest >> and the rest of the online-manual (although I have to say, even after >> years of use, I find it somewhat overwhelming) >> >> Somewhat easier is the 'book of words', Brian Okken's: >> https://www.amazon.com.au/Python-Testing-pytest-Effective-Scalable/dp/1680508601/ref=sr_1_1?keywords=brian+okken&qid=1704838720&sr=8-1 >> (disclaimer: there are other book-sellers. No commission claimed) >> >> Coursera on-line training: (amongst many 'Python' courses) >> Testing and Debugging Python >> https://www.coursera.org/projects/testing-and-debugging-python >> - is a $paid project approach >> Possibly slightly off (your) topic is LearnQuest's 'TDD >> Specialisation': >> https://www.coursera.org/learn/test-driven-development-overview >> (which I think can be "audited" for $free) >> NB have not completed either, and thus can't comment on quality, nor >> degree of pytest-content. > > Lotsa good stuff here, too: > > https://www.obeythetestinggoat.com/ OK, now you've made me feel bad! Having broadened the OP's scope, this book/web-site should indeed have been mentioned! I met Harry at a London-PUG meeting. Wow, must be a solid 15 years back - started playing with pythonanywhere.com when they were still setting it up/converting from the idea of a Python-based 'better Excel'... (Harry has since moved-on to work on other things) The testing-goat has a lot of grey in his beard these days. However, what other tome covers Python+testing+pytest? [have a 'gathering' on Friday*, to talk about 'testing'. Thus, will be interested in any and all comments, advice, sources...] IIRC the main project-example was web/Django, which some Python-folk may find unnecessarily complicated/unfamiliar. * 1500 NZDT so not a particularly-friendly hour for folk in EUR/UK https://www.meetup.com/nzpug-auckland/events/298338908/ part of our Vacation Exception Handlers gatherings (cf formal PUG-meetings) NB it is summer, as well as the festive/holiday-period down-under. https://danceswithmice.info/Python/2024/VacExcHndlrs.html. -- Regards, =dn From alexkleider at gmail.com Wed Jan 10 16:18:39 2024 From: alexkleider at gmail.com (Alex Kleider) Date: Wed, 10 Jan 2024 13:18:39 -0800 Subject: [Tutor] importing modules to be tested Message-ID: It's been my impression that tests are usually placed in their own separate directory. The problem then arises how to import the modules that are to be tested. My solution has been to use the following code: """ import os import sys sys.path.insert(0, os.path.split(sys.path[0])[0]) """ Is there a "better way?" I'm using Debian/GNU/Linux and python 3.9 Thanks in advance for any advice you might have. Sincerely, Alex Kleider The following may or may not be helpful as background: $ pwd /home/alex/Git/Lib/Question $ ls -lA total 28 -rwxr-xr-x 1 alex alex 100 Jan 10 12:25 code2test.py drwxr-xr-x 2 alex alex 4096 Jan 10 12:26 tests $ cat code2test.py #!/usr/bin/env python3 # file: code2test.py def code(): return "code2test.code() has been run" $ cat tests/test_code.py #!/usr/bin/env python3 # File: tests/test_code.py import os import sys sys.path.insert(0, os.path.split(sys.path[0])[0]) import code2test assert code2test.code() == "code2test.code() has been run" print("Test passes") $ ./tests/test_code.py Test passes $ cat tests/test_code.py #!/usr/bin/env python3 # File: tests/test_code.py """ import os import sys sys.path.insert(0, os.path.split(sys.path[0])[0]) """ import code2test assert code2test.code() == "code2test.code() has been run" print("Test passes") $ ./tests/test_code.py Traceback (most recent call last): File "/home/alex/Git/Lib/Question/./tests/test_code.py", line 8, in import code2test ModuleNotFoundError: No module named 'code2test' -- alex at kleider.ca (he/him) (sent from my current gizmo) From PythonList at DancesWithMice.info Wed Jan 10 17:09:51 2024 From: PythonList at DancesWithMice.info (dn) Date: Thu, 11 Jan 2024 11:09:51 +1300 Subject: [Tutor] importing modules to be tested In-Reply-To: References: Message-ID: <5b4f8627-a4ca-4ff0-97ba-3bfb1fedabcd@DancesWithMice.info> Alex, On 11/01/24 10:18, Alex Kleider wrote: > It's been my impression that tests are usually placed in their own Yes, there are two popular directory-layouts. Either: project -- src -- script -- tests -- test-script or: project -- script -- tests -- test-script > separate directory. The problem then arises how to import the > modules that are to be tested. Well spotted! - and yes, both of the above suffer from such. > My solution has been to use the following code: Again, you've 'taken the bull by the horns' and bravely attempted to solve the problem. However, this is 're-inventing the wheel' and not 'standing on the shoulders of giants' (who have trod this path before). There are three popular automated-testing frameworks: - doctest - unittest - pytest NB there are others, as well as plug-ins/extensions to the above - the world is your oyster! I use pytest, and answered this exact question in (yesterday's) discussion: "Import Issues". As you have no-doubt anticipated, the business of managing the PYTHONPATH is pure 'boilerplate' and would have to be repeated in every single test-script. Boring! The other advantage is that pytest's reporting can be abbreviated to a simple "P" or "F" (pass or fail) for each test, which saves you from having to add reporting print() calls! Automation is a huge advantage. Thus, it is possible to build test-runs into CI/CD pipelines, eg tests being run at every git-commit. When I first started with pytest I didn't learn that one can select only certain tests be run/code be tested - the system was so fast to test 'everything' that by the time I'd taken the opportunity to stretch my neck and loosen my shoulders, the results were in! Using PyCharm, pytest (or other) can be configured into the IDE, and many features made even easier to use. Web.Refs: https://docs.pytest.org/en/7.4.x/getting-started.html https://www.jetbrains.com/help/pycharm/pytest.html Blatant advert: Friday 1500 (NZDT=UTC+13) VacExcHndlrs: Testing: https://www.meetup.com/nzpug-auckland/events/298338908/ (also, in a few hours (!) Boston, USA) Python Study Group: using mocks in your tests: https://www.meetup.com/bostonpython/events/298315254/ - which is several steps beyond where you (seem to be) now, but may be of-interest, serve to inspire... -- Regards, =dn From alexkleider at gmail.com Wed Jan 10 18:50:32 2024 From: alexkleider at gmail.com (Alex Kleider) Date: Wed, 10 Jan 2024 15:50:32 -0800 Subject: [Tutor] importing modules to be tested In-Reply-To: <5b4f8627-a4ca-4ff0-97ba-3bfb1fedabcd@DancesWithMice.info> References: <5b4f8627-a4ca-4ff0-97ba-3bfb1fedabcd@DancesWithMice.info> Message-ID: Thanks "dn" for your response. I couldn't find an explicit answer to my question but am I to understand that if one uses one of the testing frameworks then the problem goes away? Alex > Again, you've 'taken the bull by the horns' and bravely attempted to > solve the problem. > > However, this is 're-inventing the wheel' and not 'standing on the > shoulders of giants' (who have trod this path before). > > There are three popular automated-testing frameworks: > > - doctest > - unittest > - pytest > > NB there are others, as well as plug-ins/extensions to the above - the > From PythonList at DancesWithMice.info Wed Jan 10 19:10:36 2024 From: PythonList at DancesWithMice.info (dn) Date: Thu, 11 Jan 2024 13:10:36 +1300 Subject: [Tutor] importing modules to be tested In-Reply-To: References: <5b4f8627-a4ca-4ff0-97ba-3bfb1fedabcd@DancesWithMice.info> Message-ID: On 11/01/24 12:50, Alex Kleider wrote: > Thanks "dn" for your response. > I couldn't find an explicit answer to my question but am I to understand that > if one uses one of the testing frameworks then the problem goes away? Yes, don't waste your time by doing things the hard-way, unnecessarily! > Alex > >> Again, you've 'taken the bull by the horns' and bravely attempted to >> solve the problem. >> >> However, this is 're-inventing the wheel' and not 'standing on the >> shoulders of giants' (who have trod this path before). >> >> There are three popular automated-testing frameworks: >> >> - doctest >> - unittest >> - pytest >> >> NB there are others, as well as plug-ins/extensions to the above - the >> -- Regards, =dn From alexkleider at gmail.com Wed Jan 10 19:43:09 2024 From: alexkleider at gmail.com (Alex Kleider) Date: Wed, 10 Jan 2024 16:43:09 -0800 Subject: [Tutor] importing modules to be tested In-Reply-To: References: <5b4f8627-a4ca-4ff0-97ba-3bfb1fedabcd@DancesWithMice.info> Message-ID: I've done what I think 'dn' is recommending but still things aren't working. There is still the same import error. Again, any advice would of course be appreciated. $ ls -lA total 16 -rwxr-xr-x 1 alex alex 100 Jan 10 12:25 code2test.py drwxr-xr-x 3 alex alex 4096 Jan 10 16:19 tests $ ls -lA tests total 8 -rwxr-xr-x 1 alex alex 233 Jan 10 16:19 test_code.py $ cat code2test.py #!/usr/bin/env python3 # file: code2test.py def code(): return "code2test.code() has been run" $ cat tests/test_code.py #!/usr/bin/env python3 # File: tests/test_code.py import code2test def test_code(self): assert code2test.code() == "code2test.code() has been run" $ pytest ============================== test session starts ============================== platform linux -- Python 3.9.2, pytest-7.2.0, pluggy-1.0.0 rootdir: /home/alex/Git/Lib/Question collected 0 items / 1 error ==================================== ERRORS ===================================== ______________________ ERROR collecting tests/test_code.py ______________________ ImportError while importing test module '/home/alex/Git/Lib/Question/tests/test_code.py'. Hint: make sure your test modules/packages have valid Python names. Traceback: /usr/lib/python3.9/importlib/__init__.py:127: in import_module return _bootstrap._gcd_import(name[level:], package, level) tests/test_code.py:8: in import code2test E ModuleNotFoundError: No module named 'code2test' ============================ short test summary info ============================ ERROR tests/test_code.py !!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!! =============================== 1 error in 0.06s ================================ On Wed, Jan 10, 2024 at 4:10?PM dn wrote: > > On 11/01/24 12:50, Alex Kleider wrote: > > Thanks "dn" for your response. > > I couldn't find an explicit answer to my question but am I to understand that > > if one uses one of the testing frameworks then the problem goes away? > > Yes, don't waste your time by doing things the hard-way, unnecessarily! > > > > Alex > > > >> Again, you've 'taken the bull by the horns' and bravely attempted to > >> solve the problem. > >> > >> However, this is 're-inventing the wheel' and not 'standing on the > >> shoulders of giants' (who have trod this path before). > >> > >> There are three popular automated-testing frameworks: > >> > >> - doctest > >> - unittest > >> - pytest > >> > >> NB there are others, as well as plug-ins/extensions to the above - the > >> > > -- > Regards, > =dn > -- alex at kleider.ca (he/him) (sent from my current gizmo) From PythonList at DancesWithMice.info Wed Jan 10 19:46:50 2024 From: PythonList at DancesWithMice.info (dn) Date: Thu, 11 Jan 2024 13:46:50 +1300 Subject: [Tutor] importing modules to be tested In-Reply-To: References: <5b4f8627-a4ca-4ff0-97ba-3bfb1fedabcd@DancesWithMice.info> Message-ID: <84c30b74-afe9-4b36-82d2-e4664a5f53df@DancesWithMice.info> Sorry Alex, am in the middle of a conference. Please see 'yesterday's' conversation, as referenced earlier - should solve the problem... On 11/01/24 13:43, Alex Kleider wrote: > I've done what I think 'dn' is recommending but still things > aren't working. > There is still the same import error. > Again, any advice would of course be appreciated. > > $ ls -lA > total 16 > -rwxr-xr-x 1 alex alex 100 Jan 10 12:25 code2test.py > drwxr-xr-x 3 alex alex 4096 Jan 10 16:19 tests > > $ ls -lA tests > total 8 > -rwxr-xr-x 1 alex alex 233 Jan 10 16:19 test_code.py > > $ cat code2test.py > #!/usr/bin/env python3 > # file: code2test.py > def code(): > return "code2test.code() has been run" > > $ cat tests/test_code.py > #!/usr/bin/env python3 > # File: tests/test_code.py > import code2test > > def test_code(self): > assert code2test.code() == "code2test.code() has been run" > > $ pytest > ============================== test session starts > ============================== > platform linux -- Python 3.9.2, pytest-7.2.0, pluggy-1.0.0 > rootdir: /home/alex/Git/Lib/Question > collected 0 items / 1 error > > ==================================== ERRORS > ===================================== > ______________________ ERROR collecting tests/test_code.py > ______________________ > ImportError while importing test module > '/home/alex/Git/Lib/Question/tests/test_code.py'. > Hint: make sure your test modules/packages have valid Python names. > Traceback: > /usr/lib/python3.9/importlib/__init__.py:127: in import_module > return _bootstrap._gcd_import(name[level:], package, level) > tests/test_code.py:8: in > import code2test > E ModuleNotFoundError: No module named 'code2test' > ============================ short test summary info > ============================ > ERROR tests/test_code.py > !!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection > !!!!!!!!!!!!!!!!!!!!! > =============================== 1 error in 0.06s > ================================ > > On Wed, Jan 10, 2024 at 4:10?PM dn wrote: >> >> On 11/01/24 12:50, Alex Kleider wrote: >>> Thanks "dn" for your response. >>> I couldn't find an explicit answer to my question but am I to understand that >>> if one uses one of the testing frameworks then the problem goes away? >> >> Yes, don't waste your time by doing things the hard-way, unnecessarily! >> >> >>> Alex >>> >>>> Again, you've 'taken the bull by the horns' and bravely attempted to >>>> solve the problem. >>>> >>>> However, this is 're-inventing the wheel' and not 'standing on the >>>> shoulders of giants' (who have trod this path before). >>>> >>>> There are three popular automated-testing frameworks: >>>> >>>> - doctest >>>> - unittest >>>> - pytest >>>> >>>> NB there are others, as well as plug-ins/extensions to the above - the >>>> >> >> -- >> Regards, >> =dn >> > > -- Regards, =dn From leamhall at gmail.com Wed Jan 10 20:15:38 2024 From: leamhall at gmail.com (Leam Hall) Date: Wed, 10 Jan 2024 19:15:38 -0600 Subject: [Tutor] importing modules to be tested In-Reply-To: <84c30b74-afe9-4b36-82d2-e4664a5f53df@DancesWithMice.info> References: <5b4f8627-a4ca-4ff0-97ba-3bfb1fedabcd@DancesWithMice.info> <84c30b74-afe9-4b36-82d2-e4664a5f53df@DancesWithMice.info> Message-ID: Hey Alex, I'm not sure if this will help, but you can look at the bp_tracker repo we worked on for an example: https://github.com/LeamHall/bp_tracker/blob/master/ With that, there are a couple of things to make it work. 1. In the test/ directory there's an empty file "__init__.py". 2. From the same directory as bp_tracker.py, run: python -m unittest That should let it discover your tests, and the imports you have should work. As to using a test framework, I usually use unittest for simplicity. I've done pytest, and will probably do it again, but that's more want than need. Like git, being able to use unittest or pytest is a skill you want to learn. Using doctest is okay, but seems much more limited. Leam On 1/10/24 18:46, dn via Tutor wrote: > Sorry Alex, am in the middle of a conference. > Please see 'yesterday's' conversation, as referenced earlier - should solve the problem... > > On 11/01/24 13:43, Alex Kleider wrote: >> I've done what I think 'dn' is recommending but still things >> aren't working. >> There is still the same import error. >> Again, any advice would of course be appreciated. >> >> $ ls -lA >> total 16 >> -rwxr-xr-x 1 alex alex? 100 Jan 10 12:25 code2test.py >> drwxr-xr-x 3 alex alex 4096 Jan 10 16:19 tests >> >> $ ls -lA tests >> total 8 >> -rwxr-xr-x 1 alex alex? 233 Jan 10 16:19 test_code.py >> >> $ cat code2test.py >> #!/usr/bin/env python3 >> # file: code2test.py >> def code(): >> ???? return "code2test.code() has been run" >> >> $ cat tests/test_code.py >> #!/usr/bin/env python3 >> # File: tests/test_code.py >> import code2test >> >> def test_code(self): >> ???? assert code2test.code() ==? "code2test.code() has been run" >> >> $ pytest >> ============================== test session starts >> ============================== >> platform linux -- Python 3.9.2, pytest-7.2.0, pluggy-1.0.0 >> rootdir: /home/alex/Git/Lib/Question >> collected 0 items / 1 error >> >> ==================================== ERRORS >> ===================================== >> ______________________ ERROR collecting tests/test_code.py >> ______________________ >> ImportError while importing test module >> '/home/alex/Git/Lib/Question/tests/test_code.py'. >> Hint: make sure your test modules/packages have valid Python names. >> Traceback: >> /usr/lib/python3.9/importlib/__init__.py:127: in import_module >> ???? return _bootstrap._gcd_import(name[level:], package, level) >> tests/test_code.py:8: in >> ???? import code2test >> E?? ModuleNotFoundError: No module named 'code2test' >> ============================ short test summary info >> ============================ >> ERROR tests/test_code.py >> !!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection >> !!!!!!!!!!!!!!!!!!!!! >> =============================== 1 error in 0.06s >> ================================ >> >> On Wed, Jan 10, 2024 at 4:10?PM dn wrote: >>> >>> On 11/01/24 12:50, Alex Kleider wrote: >>>> Thanks "dn" for your response. >>>> I couldn't find an explicit answer to my question but am I to understand that >>>> if one uses one of the testing frameworks then the problem goes away? >>> >>> Yes, don't waste your time by doing things the hard-way, unnecessarily! >>> >>> >>>> Alex >>>> >>>>> Again, you've 'taken the bull by the horns' and bravely attempted to >>>>> solve the problem. >>>>> >>>>> However, this is 're-inventing the wheel' and not 'standing on the >>>>> shoulders of giants' (who have trod this path before). >>>>> >>>>> There are three popular automated-testing frameworks: >>>>> >>>>> - doctest >>>>> - unittest >>>>> - pytest >>>>> >>>>> NB there are others, as well as plug-ins/extensions to the above - the >>>>> >>> >>> -- >>> Regards, >>> =dn >>> >> >> > -- Software Engineer (reuel.net/resume) Scribe: The Domici War (domiciwar.net) General Ne'er-do-well (github.com/LeamHall) From alexkleider at gmail.com Wed Jan 10 21:02:38 2024 From: alexkleider at gmail.com (Alex Kleider) Date: Wed, 10 Jan 2024 18:02:38 -0800 Subject: [Tutor] importing modules to be tested In-Reply-To: References: <5b4f8627-a4ca-4ff0-97ba-3bfb1fedabcd@DancesWithMice.info> <84c30b74-afe9-4b36-82d2-e4664a5f53df@DancesWithMice.info> Message-ID: thanks, Leam. The answer was $ python -m pytest and to 'dn's credit that was in his thread but I failed to 'get it!' Now I have a better understanding of what he wrote Problem solved. Thank you to the both of you. a On Wed, Jan 10, 2024 at 5:18?PM Leam Hall wrote: > > Hey Alex, > > I'm not sure if this will help, but you can look at the bp_tracker repo we worked on for an example: > https://github.com/LeamHall/bp_tracker/blob/master/ > > With that, there are a couple of things to make it work. > > 1. In the test/ directory there's an empty file "__init__.py". > 2. From the same directory as bp_tracker.py, run: > python -m unittest > > That should let it discover your tests, and the imports you have should work. > > As to using a test framework, I usually use unittest for simplicity. I've done pytest, and will probably do it again, but that's more want than need. Like git, being able to use unittest or pytest is a skill you want to learn. Using doctest is okay, but seems much more limited. > > Leam > > > On 1/10/24 18:46, dn via Tutor wrote: > > Sorry Alex, am in the middle of a conference. > > Please see 'yesterday's' conversation, as referenced earlier - should solve the problem... > > > > On 11/01/24 13:43, Alex Kleider wrote: > >> I've done what I think 'dn' is recommending but still things > >> aren't working. > >> There is still the same import error. > >> Again, any advice would of course be appreciated. > >> > >> $ ls -lA > >> total 16 > >> -rwxr-xr-x 1 alex alex 100 Jan 10 12:25 code2test.py > >> drwxr-xr-x 3 alex alex 4096 Jan 10 16:19 tests > >> > >> $ ls -lA tests > >> total 8 > >> -rwxr-xr-x 1 alex alex 233 Jan 10 16:19 test_code.py > >> > >> $ cat code2test.py > >> #!/usr/bin/env python3 > >> # file: code2test.py > >> def code(): > >> return "code2test.code() has been run" > >> > >> $ cat tests/test_code.py > >> #!/usr/bin/env python3 > >> # File: tests/test_code.py > >> import code2test > >> > >> def test_code(self): > >> assert code2test.code() == "code2test.code() has been run" > >> > >> $ pytest > >> ============================== test session starts > >> ============================== > >> platform linux -- Python 3.9.2, pytest-7.2.0, pluggy-1.0.0 > >> rootdir: /home/alex/Git/Lib/Question > >> collected 0 items / 1 error > >> > >> ==================================== ERRORS > >> ===================================== > >> ______________________ ERROR collecting tests/test_code.py > >> ______________________ > >> ImportError while importing test module > >> '/home/alex/Git/Lib/Question/tests/test_code.py'. > >> Hint: make sure your test modules/packages have valid Python names. > >> Traceback: > >> /usr/lib/python3.9/importlib/__init__.py:127: in import_module > >> return _bootstrap._gcd_import(name[level:], package, level) > >> tests/test_code.py:8: in > >> import code2test > >> E ModuleNotFoundError: No module named 'code2test' > >> ============================ short test summary info > >> ============================ > >> ERROR tests/test_code.py > >> !!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection > >> !!!!!!!!!!!!!!!!!!!!! > >> =============================== 1 error in 0.06s > >> ================================ > >> > >> On Wed, Jan 10, 2024 at 4:10?PM dn wrote: > >>> > >>> On 11/01/24 12:50, Alex Kleider wrote: > >>>> Thanks "dn" for your response. > >>>> I couldn't find an explicit answer to my question but am I to understand that > >>>> if one uses one of the testing frameworks then the problem goes away? > >>> > >>> Yes, don't waste your time by doing things the hard-way, unnecessarily! > >>> > >>> > >>>> Alex > >>>> > >>>>> Again, you've 'taken the bull by the horns' and bravely attempted to > >>>>> solve the problem. > >>>>> > >>>>> However, this is 're-inventing the wheel' and not 'standing on the > >>>>> shoulders of giants' (who have trod this path before). > >>>>> > >>>>> There are three popular automated-testing frameworks: > >>>>> > >>>>> - doctest > >>>>> - unittest > >>>>> - pytest > >>>>> > >>>>> NB there are others, as well as plug-ins/extensions to the above - the > >>>>> > >>> > >>> -- > >>> Regards, > >>> =dn > >>> > >> > >> > > > > -- > Software Engineer (reuel.net/resume) > Scribe: The Domici War (domiciwar.net) > General Ne'er-do-well (github.com/LeamHall) > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor -- alex at kleider.ca (he/him) (sent from my current gizmo) From PythonList at DancesWithMice.info Wed Jan 10 21:05:26 2024 From: PythonList at DancesWithMice.info (dn) Date: Thu, 11 Jan 2024 15:05:26 +1300 Subject: [Tutor] importing modules to be tested In-Reply-To: References: <5b4f8627-a4ca-4ff0-97ba-3bfb1fedabcd@DancesWithMice.info> <84c30b74-afe9-4b36-82d2-e4664a5f53df@DancesWithMice.info> Message-ID: <2f673e87-2eec-4281-a807-4c36aa26fd0c@DancesWithMice.info> On 11/01/24 13:59, Alex Kleider wrote: > I've studied that thread; if the solution is there, I've not > understood what it is. > In fact it was that thread that prompted me to bring up this (same?) > issue that has been on my mind for a long time. > I can be patient and continue to use my clumsy solution for the time being. > Again, thank you for all your contributions. The issue described below IS similar to referenced thread: pytest is running but is unable to 'see' the script from a test-script in the test sub-directory. <<< python -m pytest /path/project/tests/test_test_foo.py --no-header --no-summary -q - prepending "python -m" which configures the sys.path to include current/project directory (and hence the "src.foo" relative-addressing) which likely accounts for the issues outlined (above) >>> > On Wed, Jan 10, 2024 at 4:46?PM dn wrote: >> >> Sorry Alex, am in the middle of a conference. >> Please see 'yesterday's' conversation, as referenced earlier - should >> solve the problem... >> >> On 11/01/24 13:43, Alex Kleider wrote: >>> I've done what I think 'dn' is recommending but still things >>> aren't working. >>> There is still the same import error. >>> Again, any advice would of course be appreciated. >>> >>> $ ls -lA >>> total 16 >>> -rwxr-xr-x 1 alex alex 100 Jan 10 12:25 code2test.py >>> drwxr-xr-x 3 alex alex 4096 Jan 10 16:19 tests >>> >>> $ ls -lA tests >>> total 8 >>> -rwxr-xr-x 1 alex alex 233 Jan 10 16:19 test_code.py >>> >>> $ cat code2test.py >>> #!/usr/bin/env python3 >>> # file: code2test.py >>> def code(): >>> return "code2test.code() has been run" >>> >>> $ cat tests/test_code.py >>> #!/usr/bin/env python3 >>> # File: tests/test_code.py >>> import code2test >>> >>> def test_code(self): >>> assert code2test.code() == "code2test.code() has been run" >>> >>> $ pytest >>> ============================== test session starts >>> ============================== >>> platform linux -- Python 3.9.2, pytest-7.2.0, pluggy-1.0.0 >>> rootdir: /home/alex/Git/Lib/Question >>> collected 0 items / 1 error >>> >>> ==================================== ERRORS >>> ===================================== >>> ______________________ ERROR collecting tests/test_code.py >>> ______________________ >>> ImportError while importing test module >>> '/home/alex/Git/Lib/Question/tests/test_code.py'. >>> Hint: make sure your test modules/packages have valid Python names. >>> Traceback: >>> /usr/lib/python3.9/importlib/__init__.py:127: in import_module >>> return _bootstrap._gcd_import(name[level:], package, level) >>> tests/test_code.py:8: in >>> import code2test >>> E ModuleNotFoundError: No module named 'code2test' >>> ============================ short test summary info >>> ============================ >>> ERROR tests/test_code.py >>> !!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection >>> !!!!!!!!!!!!!!!!!!!!! >>> =============================== 1 error in 0.06s >>> ================================ >>> >>> On Wed, Jan 10, 2024 at 4:10?PM dn wrote: >>>> >>>> On 11/01/24 12:50, Alex Kleider wrote: >>>>> Thanks "dn" for your response. >>>>> I couldn't find an explicit answer to my question but am I to understand that >>>>> if one uses one of the testing frameworks then the problem goes away? >>>> >>>> Yes, don't waste your time by doing things the hard-way, unnecessarily! >>>> >>>> >>>>> Alex >>>>> >>>>>> Again, you've 'taken the bull by the horns' and bravely attempted to >>>>>> solve the problem. >>>>>> >>>>>> However, this is 're-inventing the wheel' and not 'standing on the >>>>>> shoulders of giants' (who have trod this path before). >>>>>> >>>>>> There are three popular automated-testing frameworks: >>>>>> >>>>>> - doctest >>>>>> - unittest >>>>>> - pytest >>>>>> >>>>>> NB there are others, as well as plug-ins/extensions to the above - the >>>>>> >>>> >>>> -- >>>> Regards, >>>> =dn >>>> >>> >>> >> >> -- >> Regards, >> =dn >> > > -- Regards, =dn From alexkleider at gmail.com Wed Jan 10 21:05:34 2024 From: alexkleider at gmail.com (Alex Kleider) Date: Wed, 10 Jan 2024 18:05:34 -0800 Subject: [Tutor] importing modules to be tested In-Reply-To: References: <5b4f8627-a4ca-4ff0-97ba-3bfb1fedabcd@DancesWithMice.info> <84c30b74-afe9-4b36-82d2-e4664a5f53df@DancesWithMice.info> Message-ID: I should have mentioned that the presence of the __init__.py file seems to be crucial. You mentioned it but I don't think that was mentioned in 'dn's thread. On Wed, Jan 10, 2024 at 5:18?PM Leam Hall wrote: > > Hey Alex, > > I'm not sure if this will help, but you can look at the bp_tracker repo we worked on for an example: > https://github.com/LeamHall/bp_tracker/blob/master/ > > With that, there are a couple of things to make it work. > > 1. In the test/ directory there's an empty file "__init__.py". > 2. From the same directory as bp_tracker.py, run: > python -m unittest > > That should let it discover your tests, and the imports you have should work. > > As to using a test framework, I usually use unittest for simplicity. I've done pytest, and will probably do it again, but that's more want than need. Like git, being able to use unittest or pytest is a skill you want to learn. Using doctest is okay, but seems much more limited. > > Leam > > > On 1/10/24 18:46, dn via Tutor wrote: > > Sorry Alex, am in the middle of a conference. > > Please see 'yesterday's' conversation, as referenced earlier - should solve the problem... > > > > On 11/01/24 13:43, Alex Kleider wrote: > >> I've done what I think 'dn' is recommending but still things > >> aren't working. > >> There is still the same import error. > >> Again, any advice would of course be appreciated. > >> > >> $ ls -lA > >> total 16 > >> -rwxr-xr-x 1 alex alex 100 Jan 10 12:25 code2test.py > >> drwxr-xr-x 3 alex alex 4096 Jan 10 16:19 tests > >> > >> $ ls -lA tests > >> total 8 > >> -rwxr-xr-x 1 alex alex 233 Jan 10 16:19 test_code.py > >> > >> $ cat code2test.py > >> #!/usr/bin/env python3 > >> # file: code2test.py > >> def code(): > >> return "code2test.code() has been run" > >> > >> $ cat tests/test_code.py > >> #!/usr/bin/env python3 > >> # File: tests/test_code.py > >> import code2test > >> > >> def test_code(self): > >> assert code2test.code() == "code2test.code() has been run" > >> > >> $ pytest > >> ============================== test session starts > >> ============================== > >> platform linux -- Python 3.9.2, pytest-7.2.0, pluggy-1.0.0 > >> rootdir: /home/alex/Git/Lib/Question > >> collected 0 items / 1 error > >> > >> ==================================== ERRORS > >> ===================================== > >> ______________________ ERROR collecting tests/test_code.py > >> ______________________ > >> ImportError while importing test module > >> '/home/alex/Git/Lib/Question/tests/test_code.py'. > >> Hint: make sure your test modules/packages have valid Python names. > >> Traceback: > >> /usr/lib/python3.9/importlib/__init__.py:127: in import_module > >> return _bootstrap._gcd_import(name[level:], package, level) > >> tests/test_code.py:8: in > >> import code2test > >> E ModuleNotFoundError: No module named 'code2test' > >> ============================ short test summary info > >> ============================ > >> ERROR tests/test_code.py > >> !!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection > >> !!!!!!!!!!!!!!!!!!!!! > >> =============================== 1 error in 0.06s > >> ================================ > >> > >> On Wed, Jan 10, 2024 at 4:10?PM dn wrote: > >>> > >>> On 11/01/24 12:50, Alex Kleider wrote: > >>>> Thanks "dn" for your response. > >>>> I couldn't find an explicit answer to my question but am I to understand that > >>>> if one uses one of the testing frameworks then the problem goes away? > >>> > >>> Yes, don't waste your time by doing things the hard-way, unnecessarily! > >>> > >>> > >>>> Alex > >>>> > >>>>> Again, you've 'taken the bull by the horns' and bravely attempted to > >>>>> solve the problem. > >>>>> > >>>>> However, this is 're-inventing the wheel' and not 'standing on the > >>>>> shoulders of giants' (who have trod this path before). > >>>>> > >>>>> There are three popular automated-testing frameworks: > >>>>> > >>>>> - doctest > >>>>> - unittest > >>>>> - pytest > >>>>> > >>>>> NB there are others, as well as plug-ins/extensions to the above - the > >>>>> > >>> > >>> -- > >>> Regards, > >>> =dn > >>> > >> > >> > > > > -- > Software Engineer (reuel.net/resume) > Scribe: The Domici War (domiciwar.net) > General Ne'er-do-well (github.com/LeamHall) > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor -- alex at kleider.ca (he/him) (sent from my current gizmo) From alexkleider at gmail.com Wed Jan 10 21:12:07 2024 From: alexkleider at gmail.com (Alex Kleider) Date: Wed, 10 Jan 2024 18:12:07 -0800 Subject: [Tutor] importing modules to be tested In-Reply-To: <2f673e87-2eec-4281-a807-4c36aa26fd0c@DancesWithMice.info> References: <5b4f8627-a4ca-4ff0-97ba-3bfb1fedabcd@DancesWithMice.info> <84c30b74-afe9-4b36-82d2-e4664a5f53df@DancesWithMice.info> <2f673e87-2eec-4281-a807-4c36aa26fd0c@DancesWithMice.info> Message-ID: I stand corrected: the __init__.py file is not needed. $ python -m pytest works all by itself. Thank you for your patience. On Wed, Jan 10, 2024 at 6:05?PM dn wrote: > > On 11/01/24 13:59, Alex Kleider wrote: > > I've studied that thread; if the solution is there, I've not > > understood what it is. > > In fact it was that thread that prompted me to bring up this (same?) > > issue that has been on my mind for a long time. > > I can be patient and continue to use my clumsy solution for the time being. > > Again, thank you for all your contributions. > > > The issue described below IS similar to referenced thread: pytest is > running but is unable to 'see' the script from a test-script in the test > sub-directory. > > <<< > python -m pytest /path/project/tests/test_test_foo.py --no-header > --no-summary -q > > - prepending "python -m" > which configures the sys.path to include current/project directory (and > hence the "src.foo" relative-addressing) which likely accounts for the > issues outlined (above) > >>> > > > > > On Wed, Jan 10, 2024 at 4:46?PM dn wrote: > >> > >> Sorry Alex, am in the middle of a conference. > >> Please see 'yesterday's' conversation, as referenced earlier - should > >> solve the problem... > >> > >> On 11/01/24 13:43, Alex Kleider wrote: > >>> I've done what I think 'dn' is recommending but still things > >>> aren't working. > >>> There is still the same import error. > >>> Again, any advice would of course be appreciated. > >>> > >>> $ ls -lA > >>> total 16 > >>> -rwxr-xr-x 1 alex alex 100 Jan 10 12:25 code2test.py > >>> drwxr-xr-x 3 alex alex 4096 Jan 10 16:19 tests > >>> > >>> $ ls -lA tests > >>> total 8 > >>> -rwxr-xr-x 1 alex alex 233 Jan 10 16:19 test_code.py > >>> > >>> $ cat code2test.py > >>> #!/usr/bin/env python3 > >>> # file: code2test.py > >>> def code(): > >>> return "code2test.code() has been run" > >>> > >>> $ cat tests/test_code.py > >>> #!/usr/bin/env python3 > >>> # File: tests/test_code.py > >>> import code2test > >>> > >>> def test_code(self): > >>> assert code2test.code() == "code2test.code() has been run" > >>> > >>> $ pytest > >>> ============================== test session starts > >>> ============================== > >>> platform linux -- Python 3.9.2, pytest-7.2.0, pluggy-1.0.0 > >>> rootdir: /home/alex/Git/Lib/Question > >>> collected 0 items / 1 error > >>> > >>> ==================================== ERRORS > >>> ===================================== > >>> ______________________ ERROR collecting tests/test_code.py > >>> ______________________ > >>> ImportError while importing test module > >>> '/home/alex/Git/Lib/Question/tests/test_code.py'. > >>> Hint: make sure your test modules/packages have valid Python names. > >>> Traceback: > >>> /usr/lib/python3.9/importlib/__init__.py:127: in import_module > >>> return _bootstrap._gcd_import(name[level:], package, level) > >>> tests/test_code.py:8: in > >>> import code2test > >>> E ModuleNotFoundError: No module named 'code2test' > >>> ============================ short test summary info > >>> ============================ > >>> ERROR tests/test_code.py > >>> !!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection > >>> !!!!!!!!!!!!!!!!!!!!! > >>> =============================== 1 error in 0.06s > >>> ================================ > >>> > >>> On Wed, Jan 10, 2024 at 4:10?PM dn wrote: > >>>> > >>>> On 11/01/24 12:50, Alex Kleider wrote: > >>>>> Thanks "dn" for your response. > >>>>> I couldn't find an explicit answer to my question but am I to understand that > >>>>> if one uses one of the testing frameworks then the problem goes away? > >>>> > >>>> Yes, don't waste your time by doing things the hard-way, unnecessarily! > >>>> > >>>> > >>>>> Alex > >>>>> > >>>>>> Again, you've 'taken the bull by the horns' and bravely attempted to > >>>>>> solve the problem. > >>>>>> > >>>>>> However, this is 're-inventing the wheel' and not 'standing on the > >>>>>> shoulders of giants' (who have trod this path before). > >>>>>> > >>>>>> There are three popular automated-testing frameworks: > >>>>>> > >>>>>> - doctest > >>>>>> - unittest > >>>>>> - pytest > >>>>>> > >>>>>> NB there are others, as well as plug-ins/extensions to the above - the > >>>>>> > >>>> > >>>> -- > >>>> Regards, > >>>> =dn > >>>> > >>> > >>> > >> > >> -- > >> Regards, > >> =dn > >> > > > > > > -- > Regards, > =dn > -- alex at kleider.ca (he/him) (sent from my current gizmo) From PythonList at DancesWithMice.info Wed Jan 10 21:23:01 2024 From: PythonList at DancesWithMice.info (dn) Date: Thu, 11 Jan 2024 15:23:01 +1300 Subject: [Tutor] importing modules to be tested In-Reply-To: References: <5b4f8627-a4ca-4ff0-97ba-3bfb1fedabcd@DancesWithMice.info> <84c30b74-afe9-4b36-82d2-e4664a5f53df@DancesWithMice.info> <2f673e87-2eec-4281-a807-4c36aa26fd0c@DancesWithMice.info> Message-ID: On 11/01/24 15:12, Alex Kleider wrote: > I stand corrected: the __init__.py file is not needed. People often include. It is/was part of "packaging". However, nothing that really affects testing. > $ python -m pytest > works all by itself. Yes, it's all about project-directory and sub-directory structuring; and consequent implications for the import system. You detected the problem, and figured-out how to combat same. When you have time to dive into pytest (or ??? framework), you'll likely find all manner of other useful aids which will add quality to your work... NB @Leam's solution uses PSL's own unittest framework, cf pytest, which can be found in Pypi/'the Cheese Shop'. > Thank you for your patience. As long as it works (and will work next project...!) > On Wed, Jan 10, 2024 at 6:05?PM dn wrote: >> >> On 11/01/24 13:59, Alex Kleider wrote: >>> I've studied that thread; if the solution is there, I've not >>> understood what it is. >>> In fact it was that thread that prompted me to bring up this (same?) >>> issue that has been on my mind for a long time. >>> I can be patient and continue to use my clumsy solution for the time being. >>> Again, thank you for all your contributions. >> >> >> The issue described below IS similar to referenced thread: pytest is >> running but is unable to 'see' the script from a test-script in the test >> sub-directory. >> >> <<< >> python -m pytest /path/project/tests/test_test_foo.py --no-header >> --no-summary -q >> >> - prepending "python -m" >> which configures the sys.path to include current/project directory (and >> hence the "src.foo" relative-addressing) which likely accounts for the >> issues outlined (above) >> >>> >> >> >> >>> On Wed, Jan 10, 2024 at 4:46?PM dn wrote: >>>> >>>> Sorry Alex, am in the middle of a conference. >>>> Please see 'yesterday's' conversation, as referenced earlier - should >>>> solve the problem... >>>> >>>> On 11/01/24 13:43, Alex Kleider wrote: >>>>> I've done what I think 'dn' is recommending but still things >>>>> aren't working. >>>>> There is still the same import error. >>>>> Again, any advice would of course be appreciated. >>>>> >>>>> $ ls -lA >>>>> total 16 >>>>> -rwxr-xr-x 1 alex alex 100 Jan 10 12:25 code2test.py >>>>> drwxr-xr-x 3 alex alex 4096 Jan 10 16:19 tests >>>>> >>>>> $ ls -lA tests >>>>> total 8 >>>>> -rwxr-xr-x 1 alex alex 233 Jan 10 16:19 test_code.py >>>>> >>>>> $ cat code2test.py >>>>> #!/usr/bin/env python3 >>>>> # file: code2test.py >>>>> def code(): >>>>> return "code2test.code() has been run" >>>>> >>>>> $ cat tests/test_code.py >>>>> #!/usr/bin/env python3 >>>>> # File: tests/test_code.py >>>>> import code2test >>>>> >>>>> def test_code(self): >>>>> assert code2test.code() == "code2test.code() has been run" >>>>> >>>>> $ pytest >>>>> ============================== test session starts >>>>> ============================== >>>>> platform linux -- Python 3.9.2, pytest-7.2.0, pluggy-1.0.0 >>>>> rootdir: /home/alex/Git/Lib/Question >>>>> collected 0 items / 1 error >>>>> >>>>> ==================================== ERRORS >>>>> ===================================== >>>>> ______________________ ERROR collecting tests/test_code.py >>>>> ______________________ >>>>> ImportError while importing test module >>>>> '/home/alex/Git/Lib/Question/tests/test_code.py'. >>>>> Hint: make sure your test modules/packages have valid Python names. >>>>> Traceback: >>>>> /usr/lib/python3.9/importlib/__init__.py:127: in import_module >>>>> return _bootstrap._gcd_import(name[level:], package, level) >>>>> tests/test_code.py:8: in >>>>> import code2test >>>>> E ModuleNotFoundError: No module named 'code2test' >>>>> ============================ short test summary info >>>>> ============================ >>>>> ERROR tests/test_code.py >>>>> !!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection >>>>> !!!!!!!!!!!!!!!!!!!!! >>>>> =============================== 1 error in 0.06s >>>>> ================================ >>>>> >>>>> On Wed, Jan 10, 2024 at 4:10?PM dn wrote: >>>>>> >>>>>> On 11/01/24 12:50, Alex Kleider wrote: >>>>>>> Thanks "dn" for your response. >>>>>>> I couldn't find an explicit answer to my question but am I to understand that >>>>>>> if one uses one of the testing frameworks then the problem goes away? >>>>>> >>>>>> Yes, don't waste your time by doing things the hard-way, unnecessarily! >>>>>> >>>>>> >>>>>>> Alex >>>>>>> >>>>>>>> Again, you've 'taken the bull by the horns' and bravely attempted to >>>>>>>> solve the problem. >>>>>>>> >>>>>>>> However, this is 're-inventing the wheel' and not 'standing on the >>>>>>>> shoulders of giants' (who have trod this path before). >>>>>>>> >>>>>>>> There are three popular automated-testing frameworks: >>>>>>>> >>>>>>>> - doctest >>>>>>>> - unittest >>>>>>>> - pytest >>>>>>>> >>>>>>>> NB there are others, as well as plug-ins/extensions to the above - the >>>>>>>> >>>>>> >>>>>> -- >>>>>> Regards, >>>>>> =dn >>>>>> >>>>> >>>>> >>>> >>>> -- >>>> Regards, >>>> =dn >>>> >>> >>> >> >> -- >> Regards, >> =dn >> > > -- Regards, =dn From mk1853387 at gmail.com Fri Jan 12 17:32:15 2024 From: mk1853387 at gmail.com (marc nicole) Date: Fri, 12 Jan 2024 23:32:15 +0100 Subject: [Tutor] best tool to extract domain hierarchy from a dimension in an OLAP dataset (csv) Message-ID: Hi all, I have a csv OLAP dataset that I want to extract the domain hierarchies from each of its dimensions. Anybody could recommend a Python tool that could manage this properly? Thanks From PythonList at DancesWithMice.info Sat Jan 13 21:51:39 2024 From: PythonList at DancesWithMice.info (dn) Date: Sun, 14 Jan 2024 15:51:39 +1300 Subject: [Tutor] Mtg: Object-Oriented VacExcHndlrs (UTC+13) Message-ID: Let's meet on Wednesday (17Jan, 1600 NZDT (UTC+13), wearing a head-set) to talk about Object-Oriented everything. Is O-O worthwhile, or does is it just a load of guys running around and getting no-where? NB this is not a formal PUG-meeting. It's part of the "Vacation Exception Handlers" series (https://danceswithmice.info/Python/2024/VacExcHndlrs.html) - virtual-gatherings for folk left-behind to keep the wheels turning, whilst everyone else swans-off sunning themselves... (non-Kiwis please remember: it's not just school vacation, but summer-time down-under. Wish you were here?) Caf?-style approach, so there will be no formal presentation. All welcome. No presumption of knowledge/skill. This gathering is for everyone, from Beginner to Python-Master. Is Python an Object-Oriented language? Why does Python use (what appear to be) procedural constructs for so many of its core functions, eg len(a_list) rather than a_list.length() and sqrt(a_number) rather than a_number.sqrt()? Why do pythonista say "everything in Python is an object"? Is it faster to write in an OOP-style and/or does OOP-code run faster? If not, why bother? - insert your question here: What do you want to know? What has been bothering you about OOP (or O-O in Python) that you'd like to settle? To join us (we don't bite!), please RSVP at https://www.meetup.com/nzpug-auckland/events/298536620/ -- Regards, =dn From PythonList at DancesWithMice.info Sun Jan 14 23:16:35 2024 From: PythonList at DancesWithMice.info (dn) Date: Mon, 15 Jan 2024 17:16:35 +1300 Subject: [Tutor] Debugging skills In-Reply-To: <39c95ffa-1962-4d64-8f2b-ce5918b163f7@wichmann.us> References: <_yrI7GYi2qa0aw0XP6MBCPsTj_3f2vJktj_eJME6mXz9eRxCEJyb5OKlSgK2ugZeeQvGIpgJUHtIKmWpZ3kmcaL0M1fAJekhyx-Ho9-MAfU=@proton.me> <8ac348f9-feca-4609-9220-ef10c0399ca1@DancesWithMice.info> <39c95ffa-1962-4d64-8f2b-ce5918b163f7@wichmann.us> Message-ID: <6f38c279-53b6-4fc4-9696-1afcd7afda1a@DancesWithMice.info> On 6/01/24 02:56, Mats Wichmann wrote: > On 1/4/24 14:58, dn via Tutor wrote: > >>>> Interestingly the paper book version of my tutorial has a chapter on >>>> debugging. However, I've never got round to adding that chapter >>>> to the online version. I probably should... >>> >>> That would be cool. > > while we're nattering on a whole range of sort-of-debugging topics, > here's something I've occasionally wondered about, when a beginner has > written something that makes sense only to them, not to the Python > interpreter.? Regexes are confusing, so there are several websites (see > for example regex101.com) where you can enter your "code" (regex) and as > it's interpreted, will show you an explanation in words in a separate > pane.? Does anyone know if there's a similar thing for Python code? > "This is what you wrote actually means to the interpreter".? Ones I know > about (like at Programiz, w3schools, online-python) don't attempt the > explanation part.? It's not something that's likely to be useful for > very many cases, but I could see where in the beginning there might be > some benefit... It's a lot easier to build such 'checkers' when working with a small and bounded system, than with something as broad as Python. Am thinking that the best tool to diagnose when a trainee's mental-model diverges from Python's way of working, is the interpreter. Work has been invested in recent versions to improve err.msgs/diagnostics. Of course, this will never be perfect, and it is a bug-bear of learners of all languages that the diagnosis may be incorrect, and thus a red-herring. The concern is valid, and particularly in a self-teach/remote-learner situation where asking someone for help may be problematic. Reprising earlier conversation, I fear that the attempt to go too far, too fast, will lead to, or certainly more-frequently evidence this problem. My solution (in other topics) is to show multiple examples of (correct) working-code and discuss how each works, in order to create and reinforce the requisite mental-model. Then ask the trainees to solve a (very) similar problem - for which many have the wit to copy a suitable example, and edit from there. Seems to work! As previously-mentioned, and echoed by @Roel, PythonTutor does go some way to illustrating what's happening under-the-hood. No, pdb is not so good (IMHO) and thus even less-so in a training environment. PyCharm's debugger does a reasonable job, by stepping-through and showing values as they change in a dashboard-like manner - but it doesn't draw the 'pretty pictures' per PythonTutor or Thonny. -- Regards, =dn From PythonList at DancesWithMice.info Mon Jan 15 04:27:19 2024 From: PythonList at DancesWithMice.info (dn) Date: Mon, 15 Jan 2024 22:27:19 +1300 Subject: [Tutor] Debugging skills In-Reply-To: <6f38c279-53b6-4fc4-9696-1afcd7afda1a@DancesWithMice.info> References: <_yrI7GYi2qa0aw0XP6MBCPsTj_3f2vJktj_eJME6mXz9eRxCEJyb5OKlSgK2ugZeeQvGIpgJUHtIKmWpZ3kmcaL0M1fAJekhyx-Ho9-MAfU=@proton.me> <8ac348f9-feca-4609-9220-ef10c0399ca1@DancesWithMice.info> <39c95ffa-1962-4d64-8f2b-ce5918b163f7@wichmann.us> <6f38c279-53b6-4fc4-9696-1afcd7afda1a@DancesWithMice.info> Message-ID: <6c63208b-6293-4577-a1f8-80fa0a6c182f@DancesWithMice.info> On 15/01/24 17:16, dn via Tutor wrote: > On 6/01/24 02:56, Mats Wichmann wrote: >> while we're nattering on a whole range of sort-of-debugging topics, >> here's something I've occasionally wondered about, when a beginner has >> written something that makes sense only to them, not to the Python >> interpreter.? Regexes are confusing, so there are several websites >> (see for example regex101.com) where you can enter your "code" (regex) >> and as it's interpreted, will show you an explanation in words in a >> separate pane.? Does anyone know if there's a similar thing for Python >> code? "This is what you wrote actually means to the interpreter". >> Ones I know about (like at Programiz, w3schools, online-python) don't >> attempt the explanation part.? It's not something that's likely to be >> useful for very many cases, but I could see where in the beginning >> there might be some benefit... Re-reading recent post - when copy came [back] from the reflector, I wondered about LLMs. If you come across someone who lands in just such a problem, where his/her mental model is considerably at-odds with the interpreter; it would be very interesting to see if/how an LLM helps them through. Alternately, if you can recall such circumstance, I'll be keen to try to reproduce and to experiment... -- Regards, =dn From mats at wichmann.us Mon Jan 15 11:16:23 2024 From: mats at wichmann.us (Mats Wichmann) Date: Mon, 15 Jan 2024 09:16:23 -0700 Subject: [Tutor] Debugging skills In-Reply-To: <6c63208b-6293-4577-a1f8-80fa0a6c182f@DancesWithMice.info> References: <_yrI7GYi2qa0aw0XP6MBCPsTj_3f2vJktj_eJME6mXz9eRxCEJyb5OKlSgK2ugZeeQvGIpgJUHtIKmWpZ3kmcaL0M1fAJekhyx-Ho9-MAfU=@proton.me> <8ac348f9-feca-4609-9220-ef10c0399ca1@DancesWithMice.info> <39c95ffa-1962-4d64-8f2b-ce5918b163f7@wichmann.us> <6f38c279-53b6-4fc4-9696-1afcd7afda1a@DancesWithMice.info> <6c63208b-6293-4577-a1f8-80fa0a6c182f@DancesWithMice.info> Message-ID: <411e59ef-f015-4391-a16a-b1c56c4a6242@wichmann.us> On 1/15/24 02:27, dn via Tutor wrote: > On 15/01/24 17:16, dn via Tutor wrote: >> On 6/01/24 02:56, Mats Wichmann wrote: >>> while we're nattering on a whole range of sort-of-debugging topics, >>> here's something I've occasionally wondered about, when a beginner >>> has written something that makes sense only to them, not to the >>> Python interpreter.? Regexes are confusing, so there are several >>> websites (see for example regex101.com) where you can enter your >>> "code" (regex) and as it's interpreted, will show you an explanation >>> in words in a separate pane.? Does anyone know if there's a similar >>> thing for Python code? "This is what you wrote actually means to the >>> interpreter". Ones I know about (like at Programiz, w3schools, >>> online-python) don't attempt the explanation part.? It's not >>> something that's likely to be useful for very many cases, but I could >>> see where in the beginning there might be some benefit... > > Re-reading recent post - when copy came [back] from the reflector, I > wondered about LLMs. > > If you come across someone who lands in just such a problem, where > his/her mental model is considerably at-odds with the interpreter; it > would be very interesting to see if/how an LLM helps them through. > > Alternately, if you can recall such circumstance, I'll be keen to try to > reproduce and to experiment... Well, I can name one very simple one: if you've programmed only in, for example C, your mental model is arrays are pre-allocated (and hopefully you remember the fixed size so you don't go out of bounds), so you might try to use a Python list the same way - the syntax looks similar, after all... data = [] for i in range(5): data[i] = i IndexError: list assignment index out of range Wait, what? Obviously you then quickly learn, so I'm not sure this was a particularly useful example, but the brain wants to follow patterns it knows. From PythonList at DancesWithMice.info Mon Jan 15 13:38:24 2024 From: PythonList at DancesWithMice.info (dn) Date: Tue, 16 Jan 2024 07:38:24 +1300 Subject: [Tutor] Debugging skills In-Reply-To: <411e59ef-f015-4391-a16a-b1c56c4a6242@wichmann.us> References: <_yrI7GYi2qa0aw0XP6MBCPsTj_3f2vJktj_eJME6mXz9eRxCEJyb5OKlSgK2ugZeeQvGIpgJUHtIKmWpZ3kmcaL0M1fAJekhyx-Ho9-MAfU=@proton.me> <8ac348f9-feca-4609-9220-ef10c0399ca1@DancesWithMice.info> <39c95ffa-1962-4d64-8f2b-ce5918b163f7@wichmann.us> <6f38c279-53b6-4fc4-9696-1afcd7afda1a@DancesWithMice.info> <6c63208b-6293-4577-a1f8-80fa0a6c182f@DancesWithMice.info> <411e59ef-f015-4391-a16a-b1c56c4a6242@wichmann.us> Message-ID: <1a62f62d-c3cd-4b95-ae56-d2ca83c8eb8a@DancesWithMice.info> On 16/01/24 05:16, Mats Wichmann wrote: ... >> If you come across someone who lands in just such a problem, where >> his/her mental model is considerably at-odds with the interpreter; it >> would be very interesting to see if/how an LLM helps them through. >> >> Alternately, if you can recall such circumstance, I'll be keen to try >> to reproduce and to experiment... > > Well, I can name one very simple one:? if you've programmed only in, for > example C, your mental model is arrays are pre-allocated (and hopefully > you remember the fixed size so you don't go out of bounds), so you might > try to use a Python list the same way - the syntax looks similar, after > all... > > data = [] > for i in range(5): > ??? data[i] = i > > IndexError: list assignment index out of range > > Wait, what? > > Obviously you then quickly learn, so I'm not sure this was a > particularly useful example, but the brain wants to follow patterns it > knows. Not at all. the phrase "brain...patterns" is the cognitive issue - although this disconnect has been framed in the context of someone who already knows how to program[me] (in another language) and is adding Python to those skills. (slightly different situation to someone programming for the first time!) However, don't most languages which allow such code, also require the array to be pre-defined, eg def data[ 5 ]? If so, the coder has (apparently) casually disregarded the implications of removing that! Regardless, have 'interviewed' Anthropic's Claude2 and OpenAI's ChatGPT 3.5, and will send their responses in separate messages to facilitate side-by-side viewing. The experiment is compromised! I asked the question on behalf of A.N.Other, for fear of the system habitually down-grading the level/quality of answer it will give to me in other experiments. Thus, felt the need to add an extra request for explanation, which improved the response in both cases - but required more work from 'me'. Would a trainee's personal phrasing elicit a better response, earlier? What do you think of the way the LLMs have responded to this scenario? (am asking you, gentle reader, not only Mats as 'challenger'!) -- Regards, =dn From PythonList at DancesWithMice.info Mon Jan 15 13:39:10 2024 From: PythonList at DancesWithMice.info (dn) Date: Tue, 16 Jan 2024 07:39:10 +1300 Subject: [Tutor] Debugging skills (Claude2 experiment) In-Reply-To: <411e59ef-f015-4391-a16a-b1c56c4a6242@wichmann.us> References: <_yrI7GYi2qa0aw0XP6MBCPsTj_3f2vJktj_eJME6mXz9eRxCEJyb5OKlSgK2ugZeeQvGIpgJUHtIKmWpZ3kmcaL0M1fAJekhyx-Ho9-MAfU=@proton.me> <8ac348f9-feca-4609-9220-ef10c0399ca1@DancesWithMice.info> <39c95ffa-1962-4d64-8f2b-ce5918b163f7@wichmann.us> <6f38c279-53b6-4fc4-9696-1afcd7afda1a@DancesWithMice.info> <6c63208b-6293-4577-a1f8-80fa0a6c182f@DancesWithMice.info> <411e59ef-f015-4391-a16a-b1c56c4a6242@wichmann.us> Message-ID: On 16/01/24 05:16, Mats Wichmann wrote: ... >> If you come across someone who lands in just such a problem, where >> his/her mental model is considerably at-odds with the interpreter; it >> would be very interesting to see if/how an LLM helps them through. >> >> Alternately, if you can recall such circumstance, I'll be keen to try >> to reproduce and to experiment... > > Well, I can name one very simple one:? if you've programmed only in, for > example C, your mental model is arrays are pre-allocated (and hopefully > you remember the fixed size so you don't go out of bounds), so you might > try to use a Python list the same way - the syntax looks similar, after > all... > > data = [] > for i in range(5): > ??? data[i] = i > > IndexError: list assignment index out of range > > Wait, what? > > Obviously you then quickly learn, so I'm not sure this was a > particularly useful example, but the brain wants to follow patterns it > knows. From Claude2:- DN You are a Master of programming computers in Python. A trainee's code shows the following error: ```python data = [] for i in range(5): data[i] = i IndexError: list assignment index out of range ``` What is the correct Python idiom to use? Please correct the code. Claude2 The error occurs because the trainee is trying to assign values to indexes in the data list that don't exist yet. Python lists need to be resized first before assigning to an index. The idiomatic way to build up a list in Python is to use the append() method: python Copy code data = [] for i in range(5): data.append(i) This dynamically grows the list as new elements are added. The corrected code should use append() instead of directly accessing indexes that are out of bounds. This avoids the IndexError and builds up the list correctly. DN Please re-express the answer in simple terms to suit someone who is just starting to learn Python. Claude2 Here is how I would explain it to a beginning Python learner: When you create a new list in Python like: python Copy code data = [] It starts out empty. So if you try to store values in the list by referring to index numbers: python Copy code data[0] = 1 data[1] = 2 Python will give an error because you are trying to store data in slots that don't exist yet in the empty list. The right way is to use the append() method which will automatically grow the list: python Copy code data = [] data.append(1) data.append(2) Now data contains [1, 2] even though it started empty. append() adds each new value to the end of the list, one by one. This builds up the list dynamically instead of assuming it already has slots assigned. So in the original code, use append() inside the loop instead of directly accessing indexes. This will fix the error and build up the list properly. DN Why aren't the slots assigned, or at least accessible, when create a new list in Python? Having created the list, why not write data[ 0 ] = 1? Claude2 Great question! When you create a new list in Python with data = [], it initializes a data structure that holds references to object values stored in the list. But since the list is empty, there are no values referenced yet by any indexes. The key is that in Python, lists are dynamic arrays under the hood. When you write data[0], Python checks if a 0th element exists yet to store that value. If not, it raises the index error. In languages like C, arrays have fixed pre-allocated sizes, so data[0] would work there even on a new array, by assuming index 0 exists. But Python handles sizing and memory allocation for you automatically. It grows lists dynamically as you append objects. So append() builds out those referenced slots one by one, while accessing indexes directly assumes those slots already were set up. In summary, Python lists start empty and unsupported indexes aren't pre-created. Appending objects builds out the referenced structure. So new Python programmers should reach for append() rather than assume lists have default pre-made slots on creation. shift + ? to add a new line, Ctrl + K to create a new chat Claude.ai is in beta release and may display incorrect or harmful information -- Regards, =dn From PythonList at DancesWithMice.info Mon Jan 15 13:39:29 2024 From: PythonList at DancesWithMice.info (dn) Date: Tue, 16 Jan 2024 07:39:29 +1300 Subject: [Tutor] Debugging skills (ChatGPT experiment) In-Reply-To: <411e59ef-f015-4391-a16a-b1c56c4a6242@wichmann.us> References: <_yrI7GYi2qa0aw0XP6MBCPsTj_3f2vJktj_eJME6mXz9eRxCEJyb5OKlSgK2ugZeeQvGIpgJUHtIKmWpZ3kmcaL0M1fAJekhyx-Ho9-MAfU=@proton.me> <8ac348f9-feca-4609-9220-ef10c0399ca1@DancesWithMice.info> <39c95ffa-1962-4d64-8f2b-ce5918b163f7@wichmann.us> <6f38c279-53b6-4fc4-9696-1afcd7afda1a@DancesWithMice.info> <6c63208b-6293-4577-a1f8-80fa0a6c182f@DancesWithMice.info> <411e59ef-f015-4391-a16a-b1c56c4a6242@wichmann.us> Message-ID: <0d198eff-d81f-4a13-8437-adcb6d88297e@DancesWithMice.info> On 16/01/24 05:16, Mats Wichmann wrote: ... >> If you come across someone who lands in just such a problem, where >> his/her mental model is considerably at-odds with the interpreter; it >> would be very interesting to see if/how an LLM helps them through. >> >> Alternately, if you can recall such circumstance, I'll be keen to try >> to reproduce and to experiment... > > Well, I can name one very simple one:? if you've programmed only in, for > example C, your mental model is arrays are pre-allocated (and hopefully > you remember the fixed size so you don't go out of bounds), so you might > try to use a Python list the same way - the syntax looks similar, after > all... > > data = [] > for i in range(5): > ??? data[i] = i > > IndexError: list assignment index out of range > > Wait, what? > > Obviously you then quickly learn, so I'm not sure this was a > particularly useful example, but the brain wants to follow patterns it > knows. From ChatGPT 3.5:- User You are a Master of programming computers in Python. A trainee's code shows the following error: ```python data = [] for i in range(5): data[i] = i IndexError: list assignment index out of range ``` What is the correct Python idiom to use? Please correct the code. ChatGPT Certainly. The issue in the provided code is that you are trying to assign values to indices in an empty list, which will result in an "IndexError" because the list has no elements initially. To fix this, you can use the "append" method to add elements to the list. Here's the corrected code: python Copy code data = [] for i in range(5): data.append(i) This way, each iteration of the loop will add the value of i to the end of the list, preventing the "IndexError" you encountered. User Please re-express the answer in simple terms to suit someone who is just starting to learn Python. ChatGPT Certainly! In the original code, there was a mistake because the list called data was empty, and the code was trying to put values into specific positions that didn't exist yet. To fix this, we use the append method, which adds elements to the end of the list. Here's the corrected version: python Copy code data = [] for i in range(5): data.append(i) Now, it will add each number (i) to the end of the list, and you won't encounter the "IndexError" anymore. It's like building the list step by step. User Why aren't the specific positions available, or at least accessible, when create a new list in Python? Having created the list, why not write data[ 0 ] = 1? ChatGPT When you create a new list in Python, it starts out as an empty list with no elements. Unlike some other programming languages, Python doesn't automatically allocate space for specific indices when you create an empty list. Therefore, if you try to access or assign a value to a specific index that doesn't exist yet, you'll encounter an "IndexError." In your original code: python Copy code data = [] for i in range(5): data[i] = i You're attempting to assign values to specific indices (0 to 4) in an empty list, but there are no elements at those positions initially. On the other hand, when you use the append method: python Copy code data = [] for i in range(5): data.append(i) It dynamically adds elements to the end of the list during each iteration, avoiding the need to access indices that don't exist. This is a common and efficient way to build a list in Python. Once the list is populated, you can access specific positions like data[0] = 1 without any issues. Message ChatGPT? ChatGPT can make mistakes. Consider checking important information. -- Regards, =dn From mats at wichmann.us Mon Jan 15 13:45:16 2024 From: mats at wichmann.us (Mats Wichmann) Date: Mon, 15 Jan 2024 11:45:16 -0700 Subject: [Tutor] Debugging skills (ChatGPT experiment) In-Reply-To: <0d198eff-d81f-4a13-8437-adcb6d88297e@DancesWithMice.info> References: <_yrI7GYi2qa0aw0XP6MBCPsTj_3f2vJktj_eJME6mXz9eRxCEJyb5OKlSgK2ugZeeQvGIpgJUHtIKmWpZ3kmcaL0M1fAJekhyx-Ho9-MAfU=@proton.me> <8ac348f9-feca-4609-9220-ef10c0399ca1@DancesWithMice.info> <39c95ffa-1962-4d64-8f2b-ce5918b163f7@wichmann.us> <6f38c279-53b6-4fc4-9696-1afcd7afda1a@DancesWithMice.info> <6c63208b-6293-4577-a1f8-80fa0a6c182f@DancesWithMice.info> <411e59ef-f015-4391-a16a-b1c56c4a6242@wichmann.us> <0d198eff-d81f-4a13-8437-adcb6d88297e@DancesWithMice.info> Message-ID: On 1/15/24 11:39, dn via Tutor wrote: > User > Please re-express the answer in simple terms to suit someone who is just > starting to learn Python. > > ChatGPT > Certainly! In the original code, there was a mistake because the list > called data was empty, and the code was trying to put values into > specific positions that didn't exist yet. To fix this, we use the append > method, which adds elements to the end of the list. Here's the corrected > version: > > python > Copy code > data = [] > > for i in range(5): > ??? data.append(i) > > Now, it will add each number (i) to the end of the list, and you won't > encounter the "IndexError" anymore. It's like building the list step by > step. > > User > Why aren't the specific positions available, or at least accessible, > when create a new list in Python? Having created the list, why not write > data[ 0 ] = 1? > > ChatGPT > When you create a new list in Python, it starts out as an empty list > with no elements. Unlike some other programming languages, Python > doesn't automatically allocate space for specific indices when you > create an empty list. Therefore, if you try to access or assign a value > to a specific index that doesn't exist yet, you'll encounter an > "IndexError." > > In your original code: > > python > Copy code > data = [] > for i in range(5): > ??? data[i] = i > > You're attempting to assign values to specific indices (0 to 4) in an > empty list, but there are no elements at those positions initially. > > On the other hand, when you use the append method: > > python > Copy code > data = [] > for i in range(5): > ??? data.append(i) > > It dynamically adds elements to the end of the list during each > iteration, avoiding the need to access indices that don't exist. This is > a common and efficient way to build a list in Python. Once the list is > populated, you can access specific positions like data[0] = 1 without > any issues. Well, that's not terrible... maybe there's something to that approach. From PythonList at DancesWithMice.info Mon Jan 15 14:12:42 2024 From: PythonList at DancesWithMice.info (dn) Date: Tue, 16 Jan 2024 08:12:42 +1300 Subject: [Tutor] LLM comparison Message-ID: <587c8659-eb4e-4ff6-bee8-8d41e1593d74@DancesWithMice.info> The latest branch of the "Debugging skills" thread involved a casual experiment in using ChatGPT3.5 and Claude2 to help a programmer debug his/her code after Python rejected lines which (s)he reasonably-expected to work. If you haven't been following that thread, and this topic piques your interest, perhaps you'd like to review the former and add your reactions or comments? Meantime, I approached the problem in (what seems to me) the more common mode ('what the kids are doing today'): by presenting the LLM with the problem itself, and asking the 'AI' to generate code. The wording is in a slightly pompous, academic style, in order to extract comparative-responses:- Question: The requirement is a Python function which given an integer-number, will return a list populated with values equal to that value's index or relative-position in the list. Please provide three answers: (1) code likely to be produced by a Python-Beginner, (2) code likely to be produced by an Intermediate-level programmer, and (3) code from a Python-Master. ChatGPT3.5: Certainly! Here are three versions of the Python function as per your request: 1. **Beginner Level:** ```python def beginner_function(number): result_list = [] for i in range(number): result_list.append(i) return result_list ``` 2. **Intermediate Level:** ```python def intermediate_function(number): return [i for i in range(number)] ``` 3. **Python Master Level:** ```python from itertools import range def master_function(number): return list(range(number)) ``` These implementations achieve the same result, creating a list with values equal to their index or relative position in the list, but differ in terms of simplicity, conciseness, and utilization of advanced features of Python. Claude2: Here are three potential solutions to the given problem at different Python experience levels: 1. Beginner solution: ```python def populate_list(num): new_list = [] for i in range(num): new_list.append(i) return new_list ``` 2. Intermediate solution: ```python def populate_list(num): return list(range(num)) ``` 3. Expert solution: ```python populate_list = lambda num: list(range(num)) ``` The beginner creates an empty list first and uses a for loop to append each index. The intermediate programmer leverages the built-in range() function and typecasts it to a list directly. The expert uses a lambda function to achieve the same result as the intermediate solution, but in a simple one-liner expression. In summary, the solutions showcase Pythonic techniques getting more advanced from basic procedural logic to built-in functions to functional programming. Thus, the question elicited expected responses. What do you think? Both of the 'Expert solutions' contained rather 'frilly bits' - perhaps an attempt to convince us of Mastery rather than a true demonstration thereof? The first/Beginner-level responses are essentially-identical. Would you expect that most folk improving the Beginner's 'traditional' for-loop approach would graduate to a list-comprehension? There was no need for ChatGPT (response 3) to import itertools (it's not used anyway) - the built-in function does the job. That Claude2 failed to suggest a list-comprehension was a surprise. What do you think? Do you agree that Claude2's proposition of a lambda as "simple one-liner" demonstrates mastery? Indeed is that 'better' than its own 'Intermediate solution'? Between the two LLMs, compare their choices of names. How do they contribute to a reader's comprehension (ignoring the lack of docstrings, or indeed imagining that such are present). What do you think (particularly if was part of a much larger code-base)? Other thoughts? -- Regards, =dn From mats at wichmann.us Mon Jan 15 15:00:17 2024 From: mats at wichmann.us (Mats Wichmann) Date: Mon, 15 Jan 2024 13:00:17 -0700 Subject: [Tutor] LLM comparison In-Reply-To: <587c8659-eb4e-4ff6-bee8-8d41e1593d74@DancesWithMice.info> References: <587c8659-eb4e-4ff6-bee8-8d41e1593d74@DancesWithMice.info> Message-ID: On 1/15/24 12:12, dn via Tutor wrote: > There was no need for ChatGPT (response 3) to import itertools (it's not > used anyway) - the built-in function does the job. in fact, as written, it's incorrect: from itertools import range that will cause an import error. > That Claude2 failed to suggest a list-comprehension was a surprise. What > do you think? "Modern" Python lore tends to favor comprehensions and deemphasize lambdas, so yes, this is a bit strange. > The intermediate programmer leverages the built-in range() function and typecasts it to a list directly. I also am not a favorite of using the term "type cast" for Python. Maybe it's just me, but you're not reinterpreting the type of an object, you're creating a new one from the value of the original one. However, I've been known to get hung up on language usage, it does seem to be a widely used term. From alan.gauld at yahoo.co.uk Thu Jan 18 08:55:17 2024 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Thu, 18 Jan 2024 13:55:17 +0000 Subject: [Tutor] LLM comparison In-Reply-To: <587c8659-eb4e-4ff6-bee8-8d41e1593d74@DancesWithMice.info> References: <587c8659-eb4e-4ff6-bee8-8d41e1593d74@DancesWithMice.info> Message-ID: On 15/01/2024 19:12, dn via Tutor wrote: > Both of the 'Expert solutions' contained rather 'frilly bits' - perhaps > an attempt to convince us of Mastery rather than a true demonstration > thereof? But remember that none of the current set of AI bots are designed to prioritize finding "best" answers, nor even "correct" answers, only relevant ones and presenting these in a naturalist way that makes it look like a machine didn't do it. Judging AI bots on the technical quality of their answers is really to judge the internet web sites that they scavenged the answers from. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From PythonList at DancesWithMice.info Thu Jan 18 15:24:40 2024 From: PythonList at DancesWithMice.info (dn) Date: Fri, 19 Jan 2024 09:24:40 +1300 Subject: [Tutor] LLM comparison In-Reply-To: References: <587c8659-eb4e-4ff6-bee8-8d41e1593d74@DancesWithMice.info> Message-ID: <9c6b43d3-d50e-4e4e-a371-7c2b81ae250b@DancesWithMice.info> On 19/01/24 02:55, Alan Gauld via Tutor wrote: > On 15/01/2024 19:12, dn via Tutor wrote: > >> Both of the 'Expert solutions' contained rather 'frilly bits' - perhaps >> an attempt to convince us of Mastery rather than a true demonstration >> thereof? > > But remember that none of the current set of AI bots are designed > to prioritize finding "best" answers, nor even "correct" answers, > only relevant ones and presenting these in a naturalist way that > makes it look like a machine didn't do it. Indeed, often expressed with a great deal of confidence. (conversely, the apologies when errors are pointed-out are often amusing) > Judging AI bots on the technical quality of their answers is > really to judge the internet web sites that they scavenged > the answers from. +1 Many of us will have code-repos of incomplete projects - including code which won't execute (in its current state of development). Yet, GitHub was used as source-material. I like the description "stochastic parrot". An LLM can only repeat words (tokens) it has been taught, using advanced statistics to associate these in a combination that could?should be meaningful. The relevance 'here', is that too many learners are using these tools without being aware of such limitations, traps, and snares - often seduced by the confident air, and by successful early experience (at the intro-level). Like the previous tactic of copying from StackOverflow, the value of an LLM's particular contribution can only be assessed if one first possesses sufficient knowledge 'between the ears'! Will a trainee gain such competence, using a GAI? -- Regards, =dn From threesomequarks at proton.me Thu Jan 18 20:43:47 2024 From: threesomequarks at proton.me (ThreeBlindQuarks) Date: Fri, 19 Jan 2024 01:43:47 +0000 Subject: [Tutor] LLM comparison In-Reply-To: References: <587c8659-eb4e-4ff6-bee8-8d41e1593d74@DancesWithMice.info> Message-ID: <4_9jsRFmzvuQ7rzVctO-93M3SRaEAdDJT6zVX3KWD8lSAyiI_Yaygdxke0sP5V-LTySsFAwqdLXXB2EMYgE4ze_T1WWYsYEsjxwLmA5MPeY=@proton.me> This is a really hard topic for me as I believe views often differ sharply on what is a good or best way to write software. Consider some minor task that can be done in some form of explicit loop. It may not be pretty but can be moderately easy to understand and implement, even in nested loops. Someone then suggests only babies or newbies use loops and suggest a comprehension. Fine, that can be fairly concise but it can also be a tad harder to read and reason about without experience and inherently is just a loop once it is internally transformed. Someone may then suggest some other vectorized approach such as using a module like numpy that lets you use implicit loops such as A=B+C. Again, great, as long as you trust you know what is going on under the hood. But some suggest you search the internet and find some module of unknown value that has a function that sounds like it does what you want. You invoke a black box and mumble a few old latin words and hope it did it as you want it. Arguably, for trusted code, the latter approach may well be best for experts. But the plain loop you wrote is perhaps more likely to work than code you cannot see and verify easily. Consider another example. You break up your application into many parts and ask an AI-like program how to do each part in isolation. You gets lots of answers but nobody notices that your code does something similar over and over such as sorting something. You might have written one sorting function (or used one in a library) and instead you now reimplement something similar repeatedly. Fixing or changing one now, such as adding logging, does not change the others. Or, on the flip side, the AI might keep getting you to create a function that you will only use one time and with function call costs. Or, perhaps the AI does not realize the redundancy of calculating the mean of a million numbers over and over again when you could have saved the result once into a variable and referenced the variable each time. My point is even humans often disagree and give weird advice but an idea program may do better if all the parts are seen as an overall task and programmed accordingly. I have seen fairly horrible code even in the old days that looks like this. - Calculate something and put results in VAR which can be something complex like a data.frame containing raw data. - Starting with XYZ as a data.frame, calculate a subset of the rwos or columns or whatever. Oh, wait, the data is not really in XYZ but in VAR. No problem, COPY it from VAR to XYZ and then continue. I have seen people do something like this by asking questions like: - How do I read in data from a file into a data.frame. - How do I keep only some columns and ignore or delete others? - How do I apply a condition that filters a data.frame to only keep some rows based on criteria? - How do I create new columns based on calculations of other columns? - How do I group my data and then generate a report based on the groups? - How do I sort it by one column then another, perhaps descending. - How do I take the highest value among several columns as the value of a new column? The questions are endless and each may have a reasonable answer. But if each one places the result in the same name like "RESULT" and always looks for the material to be used in MYDATA then fitting all the steps together is painful. You sort of have to keep copying things or other techniques or maybe combine the steps into a pipeline where that is applicable. And it tells you nothing on how to deal with the order you should do the steps to make it run faster, or to deal ith errors like losing all your data so future steps may fail if not done carefully. Like all tools, used carefully it works. I had a recent interaction with someone who wanted a problem solved and said their internet search (not an AI) had suggested using a function called "unite()" might do what they want. I had never heard of it and looked into it and at first, it would not. I offered a step by step alternative that was in some sense harder but would work fine. Then I looked again and realized that I could overcome some of the shortcomings and make unite() work by twiddling some parameters such as telling it not to remove the other columns, and making the separator be "" and asking that NA values not be written as "NA" but be ignored. Suddenly, it worked! If interested, the question was to take two data sets with different columns/names that have been tweaked to hold similar comparable values and merge them so every row now has both values where one of them is an NA, and make a new column containing just the non-NA component. That looks simple, but it takes some careful thought and preparation. My ifelse() approach works fine but is longer. And, no, it was not done in python but as an example, easily could have been. I have not played much with these supposed AI but my guess is you probably could spell out your need carefully and in length in ways that get it to overcome some of what I said above. If, for example, you specify the names of the input and output of each step or specify you want a pipeline, it might be able to instantiate the code that it was fed on with your names and some of what it produces may be easier or more efficient. But if we are talking about students learning how to compute, the ONLY valid answers are often based on what has already been taught in class. Fancy and elegant solutions may not get you a good grade. Relying on public modules that can stop being supported or that get changed, can be a trap. Sent with Proton Mail secure email. On Thursday, January 18th, 2024 at 8:55 AM, Alan Gauld via Tutor wrote: > On 15/01/2024 19:12, dn via Tutor wrote: > > > Both of the 'Expert solutions' contained rather 'frilly bits' - perhaps > > an attempt to convince us of Mastery rather than a true demonstration > > thereof? > > > But remember that none of the current set of AI bots are designed > to prioritize finding "best" answers, nor even "correct" answers, > only relevant ones and presenting these in a naturalist way that > makes it look like a machine didn't do it. > > Judging AI bots on the technical quality of their answers is > really to judge the internet web sites that they scavenged > the answers from. > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.amazon.com/author/alan_gauld > Follow my photo-blog on Flickr at: > http://www.flickr.com/photos/alangauldphotos > > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor From mats at wichmann.us Fri Jan 19 12:54:35 2024 From: mats at wichmann.us (Mats Wichmann) Date: Fri, 19 Jan 2024 10:54:35 -0700 Subject: [Tutor] LLM comparison In-Reply-To: <4_9jsRFmzvuQ7rzVctO-93M3SRaEAdDJT6zVX3KWD8lSAyiI_Yaygdxke0sP5V-LTySsFAwqdLXXB2EMYgE4ze_T1WWYsYEsjxwLmA5MPeY=@proton.me> References: <587c8659-eb4e-4ff6-bee8-8d41e1593d74@DancesWithMice.info> <4_9jsRFmzvuQ7rzVctO-93M3SRaEAdDJT6zVX3KWD8lSAyiI_Yaygdxke0sP5V-LTySsFAwqdLXXB2EMYgE4ze_T1WWYsYEsjxwLmA5MPeY=@proton.me> Message-ID: On 1/18/24 18:43, ThreeBlindQuarks via Tutor wrote: > This is a really hard topic for me as I believe views often differ sharply on what is a good or best way to write software. long thoughtful post mostly deleted... > > Consider some minor task that can be done in some form of explicit loop... > Someone then suggests only babies or newbies use loops and suggest a comprehension. > Someone may then suggest some other vectorized approach ... > But some suggest you search the internet and find some module... Well, problem solving always has contexts. What's right for a beginner in a structured learning situation (as mentioned in this post somewhere) is to "make use of the tools you've already been presented" - such exercises are designed to reinforce what has been presented, not send you off on searches. Trying to solve a problem for work is always going to have different considerations, which include things like "can I actually use this existing code considering [list-of-possible-issues]" vs "is it just better to write our own". The wide success of open source software, which is now part of just about everything, is largely down to working out that it's generally just too expensive to write your own if someone already has a solution. > My point is even humans often disagree and give weird advice but an idea program may do better if all the parts are seen as an overall task and programmed accordingly. My belief is this depends a little on the scenario, but essentially understanding the entire problem, or understanding the architecture, or understanding the scope and interfaces of the subsystem you're programming within, are crucial to be able to be effective. The TDD approach tries to come at things from another direction: do very small things, first describing the desired behavior through a test, so you have reliable components others can depend on - but you still have to understand how to compartmentalize the problem to be able to do that, which means you need to understand the problem. So the trend I'm seeing - more through social media commentary than through being in the trenches myself, to be honest- is that there's already a subset of the community that believes that the traits of a programmer are morphing to "be able to effectively ask a copilot a question and then be a really good code reviewer" - much moreso than being a coder yourself. How in the world newcomers are going to develop those two abilities without understanding the big picture, which you're going to be even less taught to do if "writing the code" is not much of the equation any longer, I have no idea at all. From pablo.fleitas at gmail.com Fri Jan 19 11:24:25 2024 From: pablo.fleitas at gmail.com (Pablo Fleitas Zeni) Date: Fri, 19 Jan 2024 16:24:25 +0000 Subject: [Tutor] Crystal Reports with Python. Message-ID: Hi nice people, I wanto to load crystla report from "*.rpt" and export to ".pdf" I have Crystal Runtime version 13.0 . Someone knows about it? For any informationm, Thanks a lot. From alan.gauld at yahoo.co.uk Fri Jan 19 18:41:37 2024 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Fri, 19 Jan 2024 23:41:37 +0000 Subject: [Tutor] Crystal Reports with Python. In-Reply-To: References: Message-ID: On 19/01/2024 16:24, Pablo Fleitas Zeni wrote: > Hi nice people, I wanto to load crystla report from "*.rpt" and export to ".pdf" > I have Crystal Runtime version 13.0 . Someone knows about it? > For any informationm, Thanks a lot. Wow! That's a blast from the past. I didn't know Crystal Reports was still being used, I haven't seen it in over 20 years! This is going to be really tricky to do by hand so you will need to hope that somebody has written a module or package to work with Crystal. (There are a few PDF writing packages so that might be doable from scratch if not supported in your module.) I'd start with some Google type searches. A very quick search suggests that there is a module that can run the Crystal Reports executable robotically from Python. So if you are running this on a machine with CR installed it might be possible. But I didn't see anything that suggests there is a native .rpt file reader. Could you get the data in another format such as CSV? That would make life much easier! -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Sat Jan 20 19:32:22 2024 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sun, 21 Jan 2024 00:32:22 +0000 Subject: [Tutor] Crystal Reports with Python. In-Reply-To: References: Message-ID: On 19/01/2024 23:41, Alan Gauld via Tutor wrote: > On 19/01/2024 16:24, Pablo Fleitas Zeni wrote: >> Hi nice people, I wanto to load crystla report from "*.rpt" and export to ".pdf" >> I have Crystal Runtime version 13.0 . Someone knows about it? >> For any informationm, Thanks a lot. > > Wow! That's a blast from the past. I didn't know Crystal Reports > was still being used, I haven't seen it in over 20 years! It seems they have moved up market since I last saw them and are now marketed as a BI tool by SAP. I didn't delve deep but I'd expect a tool in that sector to have many export options, at least one of which should suit Python (eg. CSV, XML, JSON, etc) But a native binary format such as rpt is usually the hardest to work with. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From mk1853387 at gmail.com Sun Jan 28 13:16:10 2024 From: mk1853387 at gmail.com (marc nicole) Date: Sun, 28 Jan 2024 19:16:10 +0100 Subject: [Tutor] How to create a binary tree hierarchy given a list of elements as its leaves Message-ID: So I am trying to build a binary tree hierarchy given numerical elements serving for its leaves (last level of the tree to build). From the leaves I want to randomly create a name for the higher level of the hierarchy and assign it to the children elements. For example: if the elements inputted are `0,1,2,3` then I would like to create firstly 4 elements (say by random giving them a label composed of a letter and a number) then for the second level (iteration) I assign each of 0,1 to a random name label (e.g. `b1`) and `2,3` to another label (`b2`) then for the third level I assign a parent label to each of `b1` and `b2` as `c1`. An illustation of the example is the following tree: [image: tree_exp.PNG] For this I use numpy's `array_split()` to get the chunks of arrays based on the iteration needs. for example to get the first iteration arrays I use `np.array_split(input, (input.size // k))` where `k` is an even number. In order to assign a parent node to the children the array range should enclose the children's. For example to assign the parent node with label `a1` to children `b1` and `b2` with range respectively [0,1] and [2,3], the parent should have the range [0,3]. All is fine until a certain iteration (k=4) returns parent with range [0,8] which is overlapping to children ranges and therefore cannot be their parent. My question is how to evenly partition such arrays in a binary way and create such binary tree so that to obtain for k=4 the first range to be [0,7] instead of [0,8]? My code is the following: #!/usr/bin/python # -*- coding: utf-8 -*- import string import random import numpy as np def generate_numbers_list_until_number(stop_number): if str(stop_number).isnumeric(): return np.arange(stop_number) else: raise TypeError('Input should be a number!') def generate_node_label(): return random.choice(string.ascii_lowercase) \ + str(random.randint(0, 10)) def main(): data = generate_numbers_list_until_number(100) k = 1 hierarchies = [] cells_arrays = np.array_split(data, data.size // k) print cells_arrays used_node_hierarchy_name = [] node_hierarchy_name = [generate_node_label() for _ in range(0, len(cells_arrays))] used_node_hierarchy_name.extend(node_hierarchy_name) while len(node_hierarchy_name) > 1: k = k * 2 # bug here in the following line cells_arrays = list(map(lambda x: [x[0], x[-1]], np.array_split(data, data.size // k))) print cells_arrays node_hierarchy_name = [] # node hierarchy names should not be redundant in another level for _ in range(0, len(cells_arrays)): node_name = generate_node_label() while node_name in used_node_hierarchy_name: node_name = generate_node_label() node_hierarchy_name.append(node_name) used_node_hierarchy_name.extend(node_hierarchy_name) print used_node_hierarchy_name hierarchies.append(list(zip(node_hierarchy_name, cells_arrays))) From alan.gauld at yahoo.co.uk Sun Jan 28 14:05:47 2024 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sun, 28 Jan 2024 19:05:47 +0000 Subject: [Tutor] How to create a binary tree hierarchy given a list of elements as its leaves In-Reply-To: References: Message-ID: On 28/01/2024 18:16, marc nicole via Python-list wrote: > So I am trying to build a binary tree hierarchy given numerical elements > serving for its leaves (last level of the tree to build). From the leaves I > want to randomly create a name for the higher level of the hierarchy and > assign it to the children elements. For example: if the elements inputted > are `0,1,2,3` then I would like to create firstly 4 elements (say by random > giving them a label composed of a letter and a number) then for the second > level (iteration) I assign each of 0,1 to a random name label (e.g. `b1`) > and `2,3` to another label (`b2`) then for the third level I assign a > parent label to each of `b1` and `b2` as `c1`. > > An illustation of the example is the following tree: > > > [image: tree_exp.PNG] Unfortunately the Python mail servers strip out binary attachments so we can't see the example. Which is a pity because I don't really understand your explanation, particularly about the random labelling of parents. Also you are using an ordered set of inputs which is usually a bad case for a binary tree - it turns into a linear list... However, I notice you've included several other mailing lists so maybe someone else will comprehend your intention. Meanwhile, for a basic Python binary tree implementation example you could try here: https://www.askpython.com/python/examples/binary-tree-implementation That may or may not help! I'm actually surprised it's not already in the standard library somewhere, but I couldn't find one. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From mk1853387 at gmail.com Sun Jan 28 18:11:47 2024 From: mk1853387 at gmail.com (marc nicole) Date: Mon, 29 Jan 2024 00:11:47 +0100 Subject: [Tutor] How to create a binary tree hierarchy given a list of elements as its leaves In-Reply-To: References: Message-ID: Here's the tree example I attached earlier which shows how to go from leaf nodes (0,1,2,3) to the root node (the range[0,3]) my problem (bug) in the code is that the generated intermediate intervals for the nodes don't fit each other because of the way I binary split the input array C1[0,3] ??? B1[0,1] ? ??? A0(0) ? ??? A1(1) ??? B2[2,3] ??? A2(2) ??? A3(3) Le dim. 28 janv. 2024 ? 20:12, Alan Gauld via Tutor a ?crit : > On 28/01/2024 18:16, marc nicole via Python-list wrote: > > So I am trying to build a binary tree hierarchy given numerical elements > > serving for its leaves (last level of the tree to build). From the > leaves I > > want to randomly create a name for the higher level of the hierarchy and > > assign it to the children elements. For example: if the elements inputted > > are `0,1,2,3` then I would like to create firstly 4 elements (say by > random > > giving them a label composed of a letter and a number) then for the > second > > level (iteration) I assign each of 0,1 to a random name label (e.g. `b1`) > > and `2,3` to another label (`b2`) then for the third level I assign a > > parent label to each of `b1` and `b2` as `c1`. > > > > An illustation of the example is the following tree: > > > > > > [image: tree_exp.PNG] > > Unfortunately the Python mail servers strip out binary > attachments so we can't see the example. > > Which is a pity because I don't really understand your > explanation, particularly about the random labelling > of parents. > > Also you are using an ordered set of inputs which is usually > a bad case for a binary tree - it turns into a linear list... > > However, I notice you've included several other mailing > lists so maybe someone else will comprehend your intention. > > Meanwhile, for a basic Python binary tree implementation > example you could try here: > > https://www.askpython.com/python/examples/binary-tree-implementation > > That may or may not help! > > I'm actually surprised it's not already in the standard > library somewhere, but I couldn't find one. > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.amazon.com/author/alan_gauld > Follow my photo-blog on Flickr at: > http://www.flickr.com/photos/alangauldphotos > > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From threesomequarks at proton.me Sun Jan 28 22:09:06 2024 From: threesomequarks at proton.me (ThreeBlindQuarks) Date: Mon, 29 Jan 2024 03:09:06 +0000 Subject: [Tutor] How to create a binary tree hierarchy given a list of elements as its leaves In-Reply-To: References: Message-ID: I too am not clear on exactly what is wanted and just want to say that you need to come up first with which of the many ways you can construct a binary tree of sorts in a way that makes the rest of what you want straightforward or at least doable. Numpy does not come to mind as a way to do a binary tree. I am sure it can be used in some way to get a result but my first thoughts include using nested lists that contain no more than two items at each level. But that can have some issues if you also want to store labels in some way so I would suspect using a customized object, or some module that already supplies what you want, as easier. If using objects, you might have an object that points to or contains two similar objects with a name like left/right and when one is not is use, puts in a stub like a null instead. You can have other contents to hold names, or how many time sit has been visited, maybe pointers back up the tree and so on and add the member functions needed to do additions or deletions or searches or whatever makes sense, and some may best be written in a sort of recursive way. Of course, if your other functionality wants results in another format, such as a nested tuple implementation, you may want functions that take a root pointing to the binary tree parts and traverse them while building the tuple or list or vice versa. You need not use one fixed structure for every kind of calculation if they are inter-convertible. It depends on your needs, such as will you ever be asked to identify and return a sub-tree rather than a leaf? If this is akin to homework, you may want to design your own version from scratch. If not, this is a very common thing often used and often traversed. You need to decide if it has to be very fast or just easier to construct and use and so on. Note another of many possibilities could be using nested dictionaries with each containing a left and a right. It may make sense to experiment with which data structure you feel a bit more comfortable. And, as Alan pointed out, binary trees tend to not be at all balanced if the data fed is is already in order. There are algorithms people use to take a bad tree and rebalance it better so it can be accessed and manipulated faster on average. Sent with Proton Mail secure email. On Sunday, January 28th, 2024 at 6:11 PM, marc nicole wrote: > Here's the tree example I attached earlier which shows how to go from leaf > nodes (0,1,2,3) to the root node (the range[0,3]) my problem (bug) in the > code is that the generated intermediate intervals for the nodes don't fit > each other because of the way I binary split the input array > > C1[0,3] > ??? B1[0,1] > ? ??? A0(0) > ? ??? A1(1) > ??? B2[2,3] > ??? A2(2) > ??? A3(3) > > Le dim. 28 janv. 2024 ? 20:12, Alan Gauld via Tutor tutor at python.org a > > ?crit : > > > On 28/01/2024 18:16, marc nicole via Python-list wrote: > > > > > So I am trying to build a binary tree hierarchy given numerical elements > > > serving for its leaves (last level of the tree to build). From the > > > leaves I > > > want to randomly create a name for the higher level of the hierarchy and > > > assign it to the children elements. For example: if the elements inputted > > > are `0,1,2,3` then I would like to create firstly 4 elements (say by > > > random > > > giving them a label composed of a letter and a number) then for the > > > second > > > level (iteration) I assign each of 0,1 to a random name label (e.g. `b1`) > > > and `2,3` to another label (`b2`) then for the third level I assign a > > > parent label to each of `b1` and `b2` as `c1`. > > > > > > An illustation of the example is the following tree: > > > > > > [image: tree_exp.PNG] > > > > Unfortunately the Python mail servers strip out binary > > attachments so we can't see the example. > > > > Which is a pity because I don't really understand your > > explanation, particularly about the random labelling > > of parents. > > > > Also you are using an ordered set of inputs which is usually > > a bad case for a binary tree - it turns into a linear list... > > > > However, I notice you've included several other mailing > > lists so maybe someone else will comprehend your intention. > > > > Meanwhile, for a basic Python binary tree implementation > > example you could try here: > > > > https://www.askpython.com/python/examples/binary-tree-implementation > > > > That may or may not help! > > > > I'm actually surprised it's not already in the standard > > library somewhere, but I couldn't find one. > > > > -- > > Alan G > > Author of the Learn to Program web site > > http://www.alan-g.me.uk/ > > http://www.amazon.com/author/alan_gauld > > Follow my photo-blog on Flickr at: > > http://www.flickr.com/photos/alangauldphotos > > > > _______________________________________________ > > Tutor maillist - Tutor at python.org > > To unsubscribe or change subscription options: > > https://mail.python.org/mailman/listinfo/tutor > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor From mk1853387 at gmail.com Mon Jan 29 02:30:26 2024 From: mk1853387 at gmail.com (marc nicole) Date: Mon, 29 Jan 2024 08:30:26 +0100 Subject: [Tutor] How to create a binary tree hierarchy given a list of elements as its leaves In-Reply-To: References: Message-ID: great comments! but could you help debug the code I provided? Le lun. 29 janv. 2024 ? 04:09, ThreeBlindQuarks a ?crit : > I too am not clear on exactly what is wanted and just want to say that you > need to come up first with which of the many ways you can construct a > binary tree of sorts in a way that makes the rest of what you want > straightforward or at least doable. > > Numpy does not come to mind as a way to do a binary tree. I am sure it can > be used in some way to get a result but my first thoughts include using > nested lists that contain no more than two items at each level. But that > can have some issues if you also want to store labels in some way so I > would suspect using a customized object, or some module that already > supplies what you want, as easier. > > If using objects, you might have an object that points to or contains two > similar objects with a name like left/right and when one is not is use, > puts in a stub like a null instead. You can have other contents to hold > names, or how many time sit has been visited, maybe pointers back up the > tree and so on and add the member functions needed to do additions or > deletions or searches or whatever makes sense, and some may best be written > in a sort of recursive way. > > Of course, if your other functionality wants results in another format, > such as a nested tuple implementation, you may want functions that take a > root pointing to the binary tree parts and traverse them while building the > tuple or list or vice versa. You need not use one fixed structure for every > kind of calculation if they are inter-convertible. It depends on your > needs, such as will you ever be asked to identify and return a sub-tree > rather than a leaf? > > If this is akin to homework, you may want to design your own version from > scratch. If not, this is a very common thing often used and often > traversed. You need to decide if it has to be very fast or just easier to > construct and use and so on. > > Note another of many possibilities could be using nested dictionaries with > each containing a left and a right. It may make sense to experiment with > which data structure you feel a bit more comfortable. > > And, as Alan pointed out, binary trees tend to not be at all balanced if > the data fed is is already in order. There are algorithms people use to > take a bad tree and rebalance it better so it can be accessed and > manipulated faster on average. > > > > > > Sent with Proton Mail secure email. > > On Sunday, January 28th, 2024 at 6:11 PM, marc nicole > wrote: > > > Here's the tree example I attached earlier which shows how to go from > leaf > > nodes (0,1,2,3) to the root node (the range[0,3]) my problem (bug) in the > > code is that the generated intermediate intervals for the nodes don't fit > > each other because of the way I binary split the input array > > > > C1[0,3] > > ??? B1[0,1] > > ? ??? A0(0) > > ? ??? A1(1) > > ??? B2[2,3] > > ??? A2(2) > > ??? A3(3) > > > > Le dim. 28 janv. 2024 ? 20:12, Alan Gauld via Tutor tutor at python.org a > > > > ?crit : > > > > > On 28/01/2024 18:16, marc nicole via Python-list wrote: > > > > > > > So I am trying to build a binary tree hierarchy given numerical > elements > > > > serving for its leaves (last level of the tree to build). From the > > > > leaves I > > > > want to randomly create a name for the higher level of the hierarchy > and > > > > assign it to the children elements. For example: if the elements > inputted > > > > are `0,1,2,3` then I would like to create firstly 4 elements (say by > > > > random > > > > giving them a label composed of a letter and a number) then for the > > > > second > > > > level (iteration) I assign each of 0,1 to a random name label (e.g. > `b1`) > > > > and `2,3` to another label (`b2`) then for the third level I assign a > > > > parent label to each of `b1` and `b2` as `c1`. > > > > > > > > An illustation of the example is the following tree: > > > > > > > > [image: tree_exp.PNG] > > > > > > Unfortunately the Python mail servers strip out binary > > > attachments so we can't see the example. > > > > > > Which is a pity because I don't really understand your > > > explanation, particularly about the random labelling > > > of parents. > > > > > > Also you are using an ordered set of inputs which is usually > > > a bad case for a binary tree - it turns into a linear list... > > > > > > However, I notice you've included several other mailing > > > lists so maybe someone else will comprehend your intention. > > > > > > Meanwhile, for a basic Python binary tree implementation > > > example you could try here: > > > > > > https://www.askpython.com/python/examples/binary-tree-implementation > > > > > > That may or may not help! > > > > > > I'm actually surprised it's not already in the standard > > > library somewhere, but I couldn't find one. > > > > > > -- > > > Alan G > > > Author of the Learn to Program web site > > > http://www.alan-g.me.uk/ > > > http://www.amazon.com/author/alan_gauld > > > Follow my photo-blog on Flickr at: > > > http://www.flickr.com/photos/alangauldphotos > > > > > > _______________________________________________ > > > Tutor maillist - Tutor at python.org > > > To unsubscribe or change subscription options: > > > https://mail.python.org/mailman/listinfo/tutor > > > > _______________________________________________ > > Tutor maillist - Tutor at python.org > > To unsubscribe or change subscription options: > > https://mail.python.org/mailman/listinfo/tutor > From threesomequarks at proton.me Mon Jan 29 18:17:57 2024 From: threesomequarks at proton.me (ThreeBlindQuarks) Date: Mon, 29 Jan 2024 23:17:57 +0000 Subject: [Tutor] How to create a binary tree hierarchy given a list of elements as its leaves In-Reply-To: References: Message-ID: <0ps2iUKgI3wJXk_XFPWSAm6DJEb2F46LDj__c23HT2bcC1Rmq2ZZAMBahIjYKnUookwznvFuQg4fdiIMVRXiaTw-kih2RJcgtCs1pkci0C0=@proton.me> Marc, You have asked quite a few questions on this forum including some where I kept wondering WHY you want something. Speaking for myself, I like to be motivated to help someone and at a time when I can afford to volunteer. Your code takes time and effort to copy and try as it is not written as intuitively or commented as well as you think. I won't start a project till I have time from an otherwise busy life, and besides, it does not look at all the way my instincts would design it and I said so. I had hoped others would chime in ;-) In my experience, the first step in debugging is to understand your own code well by commenting it properly. So, what does this do: def generate_numbers_list_until_number(stop_number): if str(stop_number).isnumeric(): return np.arange(stop_number) else: raise TypeError('Input should be a number!') I have to look further to see how it is invoked and perhaps guess the purpose. It may turn out to be irrelevant to finding out what is wrong. But the code is simple enough so it simply seems to be a way of making a sort of sequence while checking if the input for highest (minus one) number is actually numeric. No big deal but it already makes me wonder if your code has an off-by-one error as what should be returned is 0..stop_number as a numpy array >>> generate_numbers_list_until_number(5) array([0, 1, 2, 3, 4]) Your next function is a bit opaque without explanation: def generate_node_label(): return random.choice(string.ascii_lowercase) \ + str(random.randint(0, 10)) It takes no argument and generates two-part strings like s9 and w8. So far, purpose unknown but perhaps part of building a simulation. They do get evaluated and run fine but the main() does not. What version of python are you using? Any recent version does not have a print command but more of a print function. I replaced uour: print cells_arrays with print (cells_arrays) I made the same update for other print statements. Otherwise, the code has issues but can continue running as these seem to be debug statements. Now what was the point of this line? Motivate us: cells_arrays = np.array_split(data, data.size // k) In your example, k is 100 and cells_arrays.size is not an array but a native python list also of length 100 containing what? >>> print (cells_arrays) [array([0]), array([1]), array([2]), array([3]), array([4]), array([5]), array([6]), array([7]), array([8]), array([9]), array([10]), array([11]), array([12]), array([13]), array([14]), array([15]), array([16]), array([17]), array([18]), array([19]), array([20]), array([21]), array([22]), array([23]), array([24]), array([25]), array([26]), array([27]), array([28]), array([29]), array([30]), array([31]), array([32]), array([33]), array([34]), array([35]), array([36]), array([37]), array([38]), array([39]), array([40]), array([41]), array([42]), array([43]), array([44]), array([45]), array([46]), array([47]), array([48]), array([49]), array([50]), array([51]), array([52]), array([53]), array([54]), array([55]), array([56]), array([57]), array([58]), array([59]), array([60]), array([61]), array([62]), array([63]), array([64]), array([65]), array([66]), array([67]), array([68]), array([69]), array([70]), array([71]), array([72]), array([73]), array([74]), array([75]), array([76]), array([77]), array([78]), array([79]), array([80]), array([81]), array([82]), array([83]), array([84]), array([85]), array([86]), array([87]), array([88]), array([89]), array([90]), array([91]), array([92]), array([93]), array([94]), array([95]), array([96]), array([97]), array([98]), array([99])] I see it as a list of numpy singleton arrays containing the same info in 100 chunks rather than one array containing all 100. If that is what you want, I need to read on to see what purpose this has as an explanation of your algorithm in some human language would have been useful. I am quite flexible on the language. As a guess, you may want to make your binary tree by extending some of those arrays, perhaps several times at greater depth. I repeat, not my idea of an appropriate data structure. A bit later, the following code fails for me with an error: The next lines generate what may be too many strings unless you wanted names ending with eleven choices from a0 to a10. Moving on, this two-line combo floors me. while len(node_hierarchy_name) > 1: k = k * 2 This is either an infinite loop or does nothing. The body of the loop never changes the variable so the length remains the same. If all you want is to calculate a large number just calculate k^n where n is the size. I assume you meant to consume some items in the list, perhaps only in some cases would you do the doubling. Until you submit code with the changes needed, I cannot follow your code further except on paper. My bottom line remains that I do not see you providing enough to motivate help here. Yes, iot likely is possible to use numpy arrays embedded some way to arbitrary levels or other techniques but do not be shocked if doing what is in some sense a recursive process does not easily work this way. Good Luck. Sent with Proton Mail secure email. On Monday, January 29th, 2024 at 2:30 AM, marc nicole wrote: > great comments! but could you help debug the code I provided? > > Le lun. 29 janv. 2024 ? 04:09, ThreeBlindQuarks threesomequarks at proton.me > > a ?crit : > > > I too am not clear on exactly what is wanted and just want to say that you > > need to come up first with which of the many ways you can construct a > > binary tree of sorts in a way that makes the rest of what you want > > straightforward or at least doable. > > > > Numpy does not come to mind as a way to do a binary tree. I am sure it can > > be used in some way to get a result but my first thoughts include using > > nested lists that contain no more than two items at each level. But that > > can have some issues if you also want to store labels in some way so I > > would suspect using a customized object, or some module that already > > supplies what you want, as easier. > > > > If using objects, you might have an object that points to or contains two > > similar objects with a name like left/right and when one is not is use, > > puts in a stub like a null instead. You can have other contents to hold > > names, or how many time sit has been visited, maybe pointers back up the > > tree and so on and add the member functions needed to do additions or > > deletions or searches or whatever makes sense, and some may best be written > > in a sort of recursive way. > > > > Of course, if your other functionality wants results in another format, > > such as a nested tuple implementation, you may want functions that take a > > root pointing to the binary tree parts and traverse them while building the > > tuple or list or vice versa. You need not use one fixed structure for every > > kind of calculation if they are inter-convertible. It depends on your > > needs, such as will you ever be asked to identify and return a sub-tree > > rather than a leaf? > > > > If this is akin to homework, you may want to design your own version from > > scratch. If not, this is a very common thing often used and often > > traversed. You need to decide if it has to be very fast or just easier to > > construct and use and so on. > > > > Note another of many possibilities could be using nested dictionaries with > > each containing a left and a right. It may make sense to experiment with > > which data structure you feel a bit more comfortable. > > > > And, as Alan pointed out, binary trees tend to not be at all balanced if > > the data fed is is already in order. There are algorithms people use to > > take a bad tree and rebalance it better so it can be accessed and > > manipulated faster on average. > > > > Sent with Proton Mail secure email. > > > > On Sunday, January 28th, 2024 at 6:11 PM, marc nicole mk1853387 at gmail.com > > wrote: > > > > > Here's the tree example I attached earlier which shows how to go from > > > leaf > > > nodes (0,1,2,3) to the root node (the range[0,3]) my problem (bug) in the > > > code is that the generated intermediate intervals for the nodes don't fit > > > each other because of the way I binary split the input array > > > > > > C1[0,3] > > > ??? B1[0,1] > > > ? ??? A0(0) > > > ? ??? A1(1) > > > ??? B2[2,3] > > > ??? A2(2) > > > ??? A3(3) > > > > > > Le dim. 28 janv. 2024 ? 20:12, Alan Gauld via Tutor tutor at python.org a > > > > > > ?crit : > > > > > > > On 28/01/2024 18:16, marc nicole via Python-list wrote: > > > > > > > > > So I am trying to build a binary tree hierarchy given numerical > > > > > elements > > > > > serving for its leaves (last level of the tree to build). From the > > > > > leaves I > > > > > want to randomly create a name for the higher level of the hierarchy > > > > > and > > > > > assign it to the children elements. For example: if the elements > > > > > inputted > > > > > are `0,1,2,3` then I would like to create firstly 4 elements (say by > > > > > random > > > > > giving them a label composed of a letter and a number) then for the > > > > > second > > > > > level (iteration) I assign each of 0,1 to a random name label (e.g. > > > > > `b1`) > > > > > and `2,3` to another label (`b2`) then for the third level I assign a > > > > > parent label to each of `b1` and `b2` as `c1`. > > > > > > > > > > An illustation of the example is the following tree: > > > > > > > > > > [image: tree_exp.PNG] > > > > > > > > Unfortunately the Python mail servers strip out binary > > > > attachments so we can't see the example. > > > > > > > > Which is a pity because I don't really understand your > > > > explanation, particularly about the random labelling > > > > of parents. > > > > > > > > Also you are using an ordered set of inputs which is usually > > > > a bad case for a binary tree - it turns into a linear list... > > > > > > > > However, I notice you've included several other mailing > > > > lists so maybe someone else will comprehend your intention. > > > > > > > > Meanwhile, for a basic Python binary tree implementation > > > > example you could try here: > > > > > > > > https://www.askpython.com/python/examples/binary-tree-implementation > > > > > > > > That may or may not help! > > > > > > > > I'm actually surprised it's not already in the standard > > > > library somewhere, but I couldn't find one. > > > > > > > > -- > > > > Alan G > > > > Author of the Learn to Program web site > > > > http://www.alan-g.me.uk/ > > > > http://www.amazon.com/author/alan_gauld > > > > Follow my photo-blog on Flickr at: > > > > http://www.flickr.com/photos/alangauldphotos > > > > > > > > _______________________________________________ > > > > Tutor maillist - Tutor at python.org > > > > To unsubscribe or change subscription options: > > > > https://mail.python.org/mailman/listinfo/tutor > > > > > > _______________________________________________ > > > Tutor maillist - Tutor at python.org > > > To unsubscribe or change subscription options: > > > https://mail.python.org/mailman/listinfo/tutor > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor From alan.gauld at yahoo.co.uk Tue Jan 30 17:23:06 2024 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Tue, 30 Jan 2024 22:23:06 +0000 Subject: [Tutor] How to create a binary tree hierarchy given a list of elements as its leaves In-Reply-To: <0ps2iUKgI3wJXk_XFPWSAm6DJEb2F46LDj__c23HT2bcC1Rmq2ZZAMBahIjYKnUookwznvFuQg4fdiIMVRXiaTw-kih2RJcgtCs1pkci0C0=@proton.me> References: <0ps2iUKgI3wJXk_XFPWSAm6DJEb2F46LDj__c23HT2bcC1Rmq2ZZAMBahIjYKnUookwznvFuQg4fdiIMVRXiaTw-kih2RJcgtCs1pkci0C0=@proton.me> Message-ID: On 29/01/2024 23:17, ThreeBlindQuarks via Tutor wrote: > Your code takes time and effort to copy and try I gave up when I saw it was Python v2 code and I don't have v2 interpreter installed. > I had hoped others would chime in ;-) I had planned to, but it all became a bit to convoluted to comprehend without putting in more time than I have available. Especially since I still don't understand the objective. The label thing makes no sense to me. > def generate_node_label(): > return random.choice(string.ascii_lowercase) \ > + str(random.randint(0, 10)) > > It takes no argument and generates two-part strings The "random" labels for the nodes. Although it does allow duplicates which are then removed by later code(perhaps) so not truly random. But not clear what their purpose or value is. > Moving on, this two-line combo floors me. > > while len(node_hierarchy_name) > 1: > k = k * 2 > > This is either an infinite loop or does nothing. I initially thought that too but the indented code after the exdented comment is actually inside the loop and does change node_heirarchy_name -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From threesomequarks at proton.me Tue Jan 30 21:59:05 2024 From: threesomequarks at proton.me (ThreeBlindQuarks) Date: Wed, 31 Jan 2024 02:59:05 +0000 Subject: [Tutor] How to create a binary tree hierarchy given a list of elements as its leaves In-Reply-To: References: <0ps2iUKgI3wJXk_XFPWSAm6DJEb2F46LDj__c23HT2bcC1Rmq2ZZAMBahIjYKnUookwznvFuQg4fdiIMVRXiaTw-kih2RJcgtCs1pkci0C0=@proton.me> Message-ID: Duh! Sorry Alan, I got misled by empty lines and a comment dedented all the way and stopped looking below to see the rest of the code. So, yes, the variable gets emptied rapidly and then refilled. My major concerns with such designs are that without a decent explanation of intent, it is hard to see if it does what is intended. Often, to report a problem, you both narrow the range so a small example can be run where you show what you expected alongside what you got. ANd it seems my interpreter gets confused by comments not indented at the level of the code. I had to move them to the right to get it to see the full loop too or it went into that infinite loop on the first part. Ideally code should not be edited after and we should get an exact copy of what was run, albeit I too have no interest in using python 2.0 and I have to ask why anyone is except for legacy code. I want to ask a serious question. Are there two programming assignments here with one being smashed together with another? I mean if you have an assignment to create functions or classes to represent a binary tree with insertions and deletions and traversals, then the payload is not the main point. You then exercise the algorithm with something like root.insert(node) and so on. You do not need random names albeit to really test it, sure, you can make some up. But note that if the algorithm sorts nodes in alphabetical order, then a10 may come ahead of a9. You may need a special order function that takes the two parts apart, or insist on using a01 and a02 and so on so a10 is more than a09. What the code I am seeing does not really seem to be doing any of what I am expecting. I have not really been shown what is expected albeit in another forum, some of the design is shown. Having said that, I can imagine ways the above could make a binary tree representation but perhaps one it would be difficult to traverse. Can you do a depth-first search versus a breadth first search on this and apply recursive algorithms? If nothing else, the OP perhaps should consider explaining what something like this does that I wonder if they just copied from somewhere: cells_arrays = list(map(lambda x: [x[0], x[-1]], np.array_split(data, data.size // k))) I can run it and see the results but by itself it is formidable. It creates an anonymous lambda function that simply return a list of the first and last item in what it is applied to. It applies that to what looks like the numbers modulo 100, which in this case may be meaningless. Then it converts what I think is a list to being a list. Now here would be an alternative I can imagine where the names grew as you went deeper. If you looked at a name like "a5.b6.c7" and wanted to insert "b9" which would be after b6 but before b7, perhaps you could adjust names as you went along to reflect some kind of hierarchy. I would not likely do that either and go with actual objects as classes which point to other such nodes and leaves and which can easily be traversed. Without explanations, I am not seeing why I would want to go on. Sorry. My advice sometimes to people is to do things another way rather than pound on to a design they may not have thought out well enough. Amazingly often, that is not seen as advice and they merrily keep trying to patch ... Moving on ... Sent with Proton Mail secure email. On Tuesday, January 30th, 2024 at 5:23 PM, Alan Gauld via Tutor wrote: > On 29/01/2024 23:17, ThreeBlindQuarks via Tutor wrote: > > > Your code takes time and effort to copy and try > > > I gave up when I saw it was Python v2 code and I > don't have v2 interpreter installed. > > > I had hoped others would chime in ;-) > > > I had planned to, but it all became a bit to > convoluted to comprehend without putting in > more time than I have available. Especially > since I still don't understand the objective. > The label thing makes no sense to me. > > > def generate_node_label(): > > return random.choice(string.ascii_lowercase) \ > > + str(random.randint(0, 10)) > > > > It takes no argument and generates two-part strings > > > The "random" labels for the nodes. Although it does > allow duplicates which are then removed by later > code(perhaps) so not truly random. But not clear > what their purpose or value is. > > > Moving on, this two-line combo floors me. > > > > while len(node_hierarchy_name) > 1: > > k = k * 2 > > > > This is either an infinite loop or does nothing. > > > I initially thought that too but the indented code after > the exdented comment is actually inside the loop and > does change node_heirarchy_name > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.amazon.com/author/alan_gauld > Follow my photo-blog on Flickr at: > http://www.flickr.com/photos/alangauldphotos > > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor From simon.n.connah at protonmail.com Wed Jan 31 08:06:48 2024 From: simon.n.connah at protonmail.com (Simon Connah) Date: Wed, 31 Jan 2024 13:06:48 +0000 Subject: [Tutor] Cryptography Message-ID: Hi, I need to implement a way to securely sign a value in a cookie to make sure that it has not been tampered with. I've been playing around with the cryptography library and it does what I need but I'm stuck on secure locations to store the private key and the password required to decrypt it. Having said that I have experience with GnuPG because I use it for other things and was wondering if that would be a more sensible choice. I mean it doesn't matter too much if they get exposed as it is only to ensure that cookie data is correct. I'm not using it for secrecy. So I thought I'd ask here about what the best practice is for generating, storing and encrypting key pairs in Python. I'm using the Flask framework and the MongoDB database. Simon. From PythonList at DancesWithMice.info Wed Jan 31 15:34:34 2024 From: PythonList at DancesWithMice.info (dn) Date: Thu, 1 Feb 2024 09:34:34 +1300 Subject: [Tutor] MTG: pytest (NZPUG, Auckland, VacExcHndlrs) Message-ID: <232586bd-f3dd-4e56-ab07-e500cd8d3b46@DancesWithMice.info> Wed 7 Feb (evening NZDT) will be the last virtual gathering in the current Vacation Exception Handlers (VacExcHndlrs) series (https://danceswithmice.info/Python/2024/VacExcHndlrs.html). You are cordially-invited to join us to investigate the pytest Python testing framework. "The pytest framework makes it easy to write small, readable tests, and can scale to support complex functional testing for applications and libraries." Pre-requisites: 1 Head-set to ask questions and make contributions 2 Lab-/Log-/Note-book 3 Python 3.8+ installed 4 Ability to write intro-level Python program[me]s (at least) 5 Credentials to install from PyPi ("The Cheese Shop") 6 Preparedness to research, experiment, work with others... As before, we will follow a caf? style of meeting. So, come prepared with a list of objectives you would like to achieve and a list of achievements you will be ready to share. Possible (but far too many) topics: - installing pytest (individual, system-wide, PyCharm, VSCodium, ...) - TDD's (Test-Driven Development) red-green refactoring - the Python assert-statement - the project directory-tree and auto-discovery - adding simple-tests - running pytest - reporting - structuring the code-base to facilitate testing - advantages/disadvantages of automated testing - using a GAI to suggest/generate tests - fixtures and dependencies (set-up and/or tear-down) - scopes (function, class, module, session) - parametrizing - monkey-patching - CI/CD chaining you've built - plug-ins you're finding helpful - coverage - testing strategies - other testing frameworks and aids (open-ended - what would you like to add?) Come to participate, learn-from, and help others! Please RSVP at https://www.meetup.com/nzpug-auckland/events/298901851/ -- Regards, =dn