From s.molnar at sbcglobal.net Tue Aug 3 09:08:16 2021 From: s.molnar at sbcglobal.net (Stephen P. Molnar) Date: Tue, 3 Aug 2021 09:08:16 -0400 Subject: [Tutor] Extract Names from string References: <61093FC0.1080200.ref@sbcglobal.net> Message-ID: <61093FC0.1080200@sbcglobal.net> I have a project which generates a large number of data files and have pyhon scrip;t that does most of the pre- and post-processing. However, I have one problem which has not succumbed to Googling. I have a list of chemical names which I can load as a string of the form: "name1","name2", . . . Unfortunately, the names are not the same length which precludes extracting each name by indexing. My question is ho w to extract individual names? Thanks in advance. -- Stephen P. Molnar, Ph.D. 614.312.7528 (c) Skype: smolnar1 From nathan-tech at hotmail.com Tue Aug 3 09:29:08 2021 From: nathan-tech at hotmail.com (nathan Smith) Date: Tue, 3 Aug 2021 14:29:08 +0100 Subject: [Tutor] Extract Names from string In-Reply-To: <61093FC0.1080200@sbcglobal.net> References: <61093FC0.1080200.ref@sbcglobal.net> <61093FC0.1080200@sbcglobal.net> Message-ID: So if I've understood this right you'd have a list of elements like this: st="\"hydrogen\", \"helium\"" That being the case could you not just do something like: splitted=st.split(",") for x in range(len(splitted)): ?splitted[x]=splitted[x][1:-1] And there you have it? A list of names without quotes? Nathan On 03/08/2021 14:08, Stephen P. Molnar wrote: > I have a project which generates a large number of data files and have > pyhon scrip;t that does most of the pre- and post-processing. However, > I have one problem which has not succumbed to Googling. > > I have a list of chemical names which I can load as a string of the form: > > "name1","name2", . . . > > Unfortunately, the names are not the same length which precludes > extracting each name by indexing. > > My question is ho w to extract individual names? > > Thanks in advance. > From joel.goldstick at gmail.com Tue Aug 3 09:31:25 2021 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Tue, 3 Aug 2021 09:31:25 -0400 Subject: [Tutor] Extract Names from string In-Reply-To: <61093FC0.1080200@sbcglobal.net> References: <61093FC0.1080200.ref@sbcglobal.net> <61093FC0.1080200@sbcglobal.net> Message-ID: On Tue, Aug 3, 2021 at 9:09 AM Stephen P. Molnar wrote: > > I have a project which generates a large number of data files and have > pyhon scrip;t that does most of the pre- and post-processing. However, I > have one problem which has not succumbed to Googling. > > I have a list of chemical names which I can load as a string of the form: > > "name1","name2", . . . > > Unfortunately, the names are not the same length which precludes > extracting each name by indexing. > > My question is ho w to extract individual names? > you need the csv module: https://docs.python.org/3/library/csv.html > Thanks in advance. > > -- > Stephen P. Molnar, Ph.D. > 614.312.7528 (c) > Skype: smolnar1 > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor -- Joel Goldstick From s.molnar at sbcglobal.net Tue Aug 3 10:52:12 2021 From: s.molnar at sbcglobal.net (Stephen P. Molnar) Date: Tue, 3 Aug 2021 10:52:12 -0400 Subject: [Tutor] Extract Names from string In-Reply-To: References: <61093FC0.1080200.ref@sbcglobal.net> <61093FC0.1080200@sbcglobal.net> Message-ID: <6109581C.7050205@sbcglobal.net> I appreciate your answer, but I'm afraid that I was not clear in what I want to do. I read the list of names using: f = open('Ligand.list_1.csv', 'r') # 'r' = read lines = f.read() print(lines), f.close() Which results in lines str 116: "CaffeicAcid","Cannflavin-A","Cannflavin-B","Cannflavin-C","Diosmetin","Echinacoside","Hesperetin","L-CichoricAcid" My problem is that I want to iteratively extract each name which I need for an inner loop which out carries out the next calculation in my computational pipeline. Sorry for the confusion. On 08/03/2021 09:29 AM, nathan Smith wrote: > So if I've understood this right you'd have a list of elements like this: > > st="\"hydrogen\", \"helium\"" > > > That being the case could you not just do something like: > > splitted=st.split(",") > > for x in range(len(splitted)): > > splitted[x]=splitted[x][1:-1] > > > And there you have it? > > A list of names without quotes? > > Nathan > > On 03/08/2021 14:08, Stephen P. Molnar wrote: >> I have a project which generates a large number of data files and >> have pyhon scrip;t that does most of the pre- and post-processing. >> However, I have one problem which has not succumbed to Googling. >> >> I have a list of chemical names which I can load as a string of the >> form: >> >> "name1","name2", . . . >> >> Unfortunately, the names are not the same length which precludes >> extracting each name by indexing. >> >> My question is ho w to extract individual names? >> >> Thanks in advance. >> > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor -- Stephen P. Molnar, Ph.D. 614.312.7528 (c) Skype: smolnar1 From wlfraed at ix.netcom.com Tue Aug 3 11:34:28 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Tue, 03 Aug 2021 11:34:28 -0400 Subject: [Tutor] Extract Names from string References: <61093FC0.1080200.ref@sbcglobal.net> <61093FC0.1080200@sbcglobal.net> <6109581C.7050205@sbcglobal.net> Message-ID: <3doiggdqn5l38vdqrernjbg6nbf4aqaeh0@4ax.com> On Tue, 3 Aug 2021 10:52:12 -0400, "Stephen P. Molnar" declaimed the following: >I appreciate your answer, but I'm afraid that I was not clear in what I >want to do. > >I read the list of names using: > >f = open('Ligand.list_1.csv', 'r') # 'r' = read DON'T DO THAT. Python has a module just for reading/writing CSV (or similar) file types. Read the library documentation for the csv module: https://docs.python.org/3/library/csv.html -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From david at graniteweb.com Tue Aug 3 11:38:29 2021 From: david at graniteweb.com (David Rock) Date: Tue, 3 Aug 2021 10:38:29 -0500 Subject: [Tutor] Extract Names from string In-Reply-To: <6109581C.7050205@sbcglobal.net> References: <61093FC0.1080200.ref@sbcglobal.net> <61093FC0.1080200@sbcglobal.net> <6109581C.7050205@sbcglobal.net> Message-ID: <20210803153829.GK28069@graniteweb.com> * Stephen P. Molnar [2021-08-03 10:52]: > I appreciate your answer, but I'm afraid that I was not clear in what I want > to do. > > I read the list of names using: > > f = open('Ligand.list_1.csv', 'r') # 'r' = read > lines = f.read() > print(lines), > f.close() > > Which results in lines str 116: > > "CaffeicAcid","Cannflavin-A","Cannflavin-B","Cannflavin-C","Diosmetin","Echinacoside","Hesperetin","L-CichoricAcid" > > My problem is that I want to iteratively extract each name which I need for > an inner loop which out carries out the next calculation in my computational > pipeline. As already mentioned, look into the CSV module. After all, you are already reading from a CSV file, so you might as well leverage the tools available in it to parse the line. https://docs.python.org/3/library/csv.html It will also help if you are dealing with a file that has multiple lines later on. -- David Rock david at graniteweb.com From edwinconnell at gmail.com Tue Aug 3 14:55:41 2021 From: edwinconnell at gmail.com (Ed Connell) Date: Tue, 3 Aug 2021 13:55:41 -0500 Subject: [Tutor] Copying one text file into another Message-ID: Suppose you have two text files, A.txt and B.txt. You want to copy A into B at some designated place .Of course there is copy and paste, but I got interested in how you could do this with Python. Can you help? Ed Connell -- I have a right and a left brain, but there is nothing right in the left one and there is nothing left in the right one! From wlfraed at ix.netcom.com Tue Aug 3 19:02:26 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Tue, 03 Aug 2021 19:02:26 -0400 Subject: [Tutor] Copying one text file into another References: Message-ID: On Tue, 3 Aug 2021 13:55:41 -0500, Ed Connell declaimed the following: >Suppose you have two text files, A.txt and B.txt. You want to copy A into >B at some designated place .Of course there is copy and paste, but I got >interested in how you could do this with Python. Can you help? > If small, you can load both files into internal lists, split the B-list at the "designated place", insert A-list at the split, then write the list to your output file. If too long to load into memory, you'll have to open a "b.tmp" (since you state "copy A into B") for output, open B for input, copy lines (read line, write line) until you hit the "designated place", open A for input, copy lines from A to output file, at end resume copying B input to B output file. Finally, delete B.txt and rename B.tmp to B.txt. -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From PyTutor at DancesWithMice.info Tue Aug 3 18:55:27 2021 From: PyTutor at DancesWithMice.info (dn) Date: Wed, 4 Aug 2021 10:55:27 +1200 Subject: [Tutor] Copying one text file into another In-Reply-To: References: Message-ID: On 04/08/2021 06.55, Ed Connell wrote: > Suppose you have two text files, A.txt and B.txt. You want to copy A into > B at some designated place .Of course there is copy and paste, but I got > interested in how you could do this with Python. Can you help? Ed, Is this an homework assignment? What code do you have already? Show us what you have written (and any errors-reported) and we will be able help you... Meantime: https://duckduckgo.com/?q=python+copy+files&t=brave&atb=v269-1&ia=web -- Regards, =dn From mats at wichmann.us Tue Aug 3 19:07:59 2021 From: mats at wichmann.us (Mats Wichmann) Date: Tue, 3 Aug 2021 17:07:59 -0600 Subject: [Tutor] Copying one text file into another In-Reply-To: References: Message-ID: <22a55a84-d09a-f488-8a56-1b90602bec81@wichmann.us> On 8/3/21 12:55 PM, Ed Connell wrote: > Suppose you have two text files, A.txt and B.txt. You want to copy A into > B at some designated place .Of course there is copy and paste, but I got > interested in how you could do this with Python. Can you help? File systems don't provide an insert method for files, so you are going to have to rewrite the target file. As long as that's okay, there are various approaches. If the files aren't massive, and you can do it in memory then it's basically: read file A into an appropriate Python data structure X, read file B into an appropriate data structure Y, insert X into Y, write Y out to file B. ("appropriate data structure" - it could be a list of lines if the operation is to be line-based, bytes, bytearray, etc. depending on what's in the files) From learn2program at gmail.com Tue Aug 3 19:12:29 2021 From: learn2program at gmail.com (Alan Gauld) Date: Wed, 4 Aug 2021 00:12:29 +0100 Subject: [Tutor] Copying one text file into another In-Reply-To: References: Message-ID: <85359458-684c-ef45-332c-d9df5d19bd73@yahoo.co.uk> This sounds like it could be homework so I'll suggest a method but not show you any code. If it doesn't work show us ypiur code and we will try to help. On 03/08/2021 19:55, Ed Connell wrote: > Suppose you have two text files, A.txt and B.txt. You want to copy A into > B at some designated place .Of course there is copy and paste, but I got > interested in how you could do this with Python. Can you help? Assume File A: A1 A2 A3 A4 And file B B1 B2 B3 So you want to insert B into A after line A2. open A.txt for reading open OUT.txt for writing for each line in A ???? write line to OUT.txt. ???? if line is A2: ???????? open B.txt for reading ???????? for each line in B ???????????? write line to OUT.txt ???????? close B.txt close A.txt close OUT.txt rename A.txt to A.bak?? (or just delete it if you feel brave!) rename OUT.txt to A.TXT? Try that in Python. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From mmssdd1920 at gmail.com Wed Aug 4 09:16:13 2021 From: mmssdd1920 at gmail.com (Msd De) Date: Wed, 4 Aug 2021 18:46:13 +0530 Subject: [Tutor] Question about matrix multiplication Message-ID: I have to take product of three matrices Q = Nn.P1.M in the attached code where M = M2.M3.M4...............Mn I would like to know whether the logic is correct. The output is not what I expected. I am attaching the file to retain the indentation. Thanks in advance From PyTutor at DancesWithMice.info Wed Aug 4 16:02:40 2021 From: PyTutor at DancesWithMice.info (dn) Date: Thu, 5 Aug 2021 08:02:40 +1200 Subject: [Tutor] Question about matrix multiplication In-Reply-To: References: Message-ID: On 05/08/2021 01.16, Msd De wrote: > I have to take product of three matrices > Q = Nn.P1.M in the attached code > where M = M2.M3.M4...............Mn > I would like to know whether the logic is correct. > The output is not what I expected. > I am attaching the file to retain the indentation. Regret that this list does not pass attachments. -- Regards, =dn From phillor9 at gmail.com Thu Aug 5 02:31:19 2021 From: phillor9 at gmail.com (Phil) Date: Thu, 5 Aug 2021 16:31:19 +1000 Subject: [Tutor] Magic numbers Message-ID: <89999336-d925-7da5-cbe7-2ab6646662ec@gmail.com> I'm in the process of tidying up a recent project which was littered with magic numbers. This line of code is called approximately 80 times per second: if variable-name < 128: I'm doing bit shifting calculations and so? < 2 ** 7 does make sense or I could use a pre-calculated global constant. I'm thinking that, for Python, a global constant might be the better option or doesn't it matter and only adds an unnecessary complication making the code less readable. It's not a matter of life or death, just wondering. -- Regards, Phil From PyTutor at DancesWithMice.info Thu Aug 5 02:43:55 2021 From: PyTutor at DancesWithMice.info (dn) Date: Thu, 5 Aug 2021 18:43:55 +1200 Subject: [Tutor] Magic numbers In-Reply-To: <89999336-d925-7da5-cbe7-2ab6646662ec@gmail.com> References: <89999336-d925-7da5-cbe7-2ab6646662ec@gmail.com> Message-ID: On 05/08/2021 18.31, Phil wrote: > I'm in the process of tidying up a recent project which was littered > with magic numbers. This line of code is called approximately 80 times > per second: > > if variable-name < 128: > > I'm doing bit shifting calculations and so? < 2 ** 7 does make sense or > I could use a pre-calculated global constant. I'm thinking that, for > Python, a global constant might be the better option or doesn't it > matter and only adds an unnecessary complication making the code less > readable. > > It's not a matter of life or death, just wondering. It would certainly be more efficient to use a constant, in preference to re-calculating each time. What is the "128" against which you are comparing the calculation? If you can call it by some name, then give it that name, eg fitting_in_seven_bits. Is there some significance to the "128", eg maximum number of seats at a conference venue? Then name it. A name is a form of documentation because it contributes to (reader) understanding! -- Regards, =dn From phillor9 at gmail.com Thu Aug 5 03:48:02 2021 From: phillor9 at gmail.com (Phil) Date: Thu, 5 Aug 2021 17:48:02 +1000 Subject: [Tutor] Magic numbers In-Reply-To: References: <89999336-d925-7da5-cbe7-2ab6646662ec@gmail.com> Message-ID: On 5/8/21 4:43 pm, dn via Tutor wrote: > On 05/08/2021 18.31, Phil wrote: > It would certainly be more efficient to use a constant, in preference to > re-calculating each time. Thank you dn, I'll go with a global constant. > What is the "128" against which you are comparing the calculation? 128 is bit 7, in this example, of an 8 bit number. > If you can call it by some name, then give it that name, eg > fitting_in_seven_bits. I always have difficulty coming up with descriptive names and that's why I always end up with magic numbers that can make the code difficult to follow a fortnight later. -- Regards, Phil From alan.gauld at yahoo.co.uk Thu Aug 5 05:32:19 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Thu, 5 Aug 2021 10:32:19 +0100 Subject: [Tutor] Magic numbers In-Reply-To: <89999336-d925-7da5-cbe7-2ab6646662ec@gmail.com> References: <89999336-d925-7da5-cbe7-2ab6646662ec@gmail.com> Message-ID: On 05/08/2021 07:31, Phil wrote: > I'm in the process of tidying up a recent project which was littered > with magic numbers. This line of code is called approximately 80 times > per second: > > if variable-name < 128: Using a named constant can make things more readable. But in this case 128 is such a well known marker that I'd argue it doesn't need to be changed. If it was a slightly different number like 132 or something then we'd like an explanation, but 128 is a commonly used power of two. Its like time calculations where we multiply by 60 or 24 - we all know how many minutes are in an hour and hours in days so it isn't really necessary to have constants called HOURS_IN_DAY etc. On the other hand, if 128 is not being used as a power-of-two marker and actually represents some application value then is should definitely be given a symbolic name. > bit shifting calculations and so? < 2 ** 7 does make sense No it doesn't because you'd be doing the calculation every time (although the optimiser might be clever enough to spot that and create its own constant!) which would be slow.... It would certainly be enough to wipe out any theoretical advantage of using bit shifting... > I could use a pre-calculated global constant. You could, but for this specific case, 128 is so well known that would not really be necessary. > Python, a global constant might be the better option Either global or local if only used in one function. Also you could have a module of constants that you import. If they are constants they should be upper case so that is one case where from myapp_constants import * is safe to use. Incidentally, why are you using bit shifting rather than regular arithmetic? That should be a last resort optimisation. It will make the code far less readable than magic numbers! So unless you have tested the code using regular arithmetic and found it runs slow and profiling the code has identified that it is the arithmetic that is slowing it down, then there is no sane reason to use bit-shifting! Of course, if you are actually reading and processing the individual bits of a piece of data then bit shifting is entirely appropriate. But using it for arithmetic is usually a fault-prone waste of everyone's time. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From wlfraed at ix.netcom.com Thu Aug 5 11:11:51 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Thu, 05 Aug 2021 11:11:51 -0400 Subject: [Tutor] Magic numbers References: <89999336-d925-7da5-cbe7-2ab6646662ec@gmail.com> Message-ID: On Thu, 5 Aug 2021 17:48:02 +1000, Phil declaimed the following: >128 is bit 7, in this example, of an 8 bit number. But what is /significant/ about bit-7 (and are you counting bits starting from 0, or from 1. It matters for documentation since when counting from 0, bit-7 is the /eighth/ bit in the byte). -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From learn2program at gmail.com Thu Aug 5 11:59:07 2021 From: learn2program at gmail.com (Alan Gauld) Date: Thu, 5 Aug 2021 16:59:07 +0100 Subject: [Tutor] Magic numbers In-Reply-To: References: <89999336-d925-7da5-cbe7-2ab6646662ec@gmail.com> Message-ID: <7ee5b96a-468d-155e-fab8-8be0da25d76f@yahoo.co.uk> On 05/08/2021 16:11, Dennis Lee Bieber wrote: > On Thu, 5 Aug 2021 17:48:02 +1000, Phil declaimed the > following: > >> 128 is bit 7, in this example, of an 8 bit number. > But what is /significant/ about bit-7 (and are you counting bits > starting from 0, or from 1. It matters for documentation since when > counting from 0, bit-7 is the /eighth/ bit in the byte). And this is exactly why bit-shifting is a fault-prone technique only to be used in extremis! :-) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From wlfraed at ix.netcom.com Thu Aug 5 13:34:07 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Thu, 05 Aug 2021 13:34:07 -0400 Subject: [Tutor] Magic numbers References: <89999336-d925-7da5-cbe7-2ab6646662ec@gmail.com> <7ee5b96a-468d-155e-fab8-8be0da25d76f@yahoo.co.uk> Message-ID: <8r7oggphraljhacv43h4jhi2k9a0keprk7@4ax.com> On Thu, 5 Aug 2021 16:59:07 +0100, Alan Gauld declaimed the following: > >On 05/08/2021 16:11, Dennis Lee Bieber wrote: >> On Thu, 5 Aug 2021 17:48:02 +1000, Phil declaimed the >> following: >> >>> 128 is bit 7, in this example, of an 8 bit number. >> But what is /significant/ about bit-7 (and are you counting bits >> starting from 0, or from 1. It matters for documentation since when >> counting from 0, bit-7 is the /eighth/ bit in the byte). > >And this is exactly why bit-shifting is a fault-prone technique only to >be used in extremis! > >:-) Doesn't help that the OP isn't using bit shift operators, but instead is using an exponentiation operator -- which may invoke heavy (relatively speaking) math operations vs a direct shift. 1 << 0 feels more explicit than 2 ** 0 >>> 1 << 7 128 -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From phillor9 at gmail.com Thu Aug 5 21:08:35 2021 From: phillor9 at gmail.com (Phil) Date: Fri, 6 Aug 2021 11:08:35 +1000 Subject: [Tutor] Magic numbers In-Reply-To: References: <89999336-d925-7da5-cbe7-2ab6646662ec@gmail.com> Message-ID: <3243ceea-7e64-55be-7a1a-9fa15f327a3c@gmail.com> On 5/8/21 7:32 pm, Alan Gauld via Tutor wrote: Thank you everyone for your replies, I didn't expect so much interest. > Incidentally, why are you using bit shifting rather > than regular arithmetic? I'm simulating a shift register that can shift either left or right. I start with bit 0 equal to one and shift it to the right until I hit the most significant bit , for example 128 for an 8 bit shift register, and then reverse the shift until I return to bit zero. Looping forever to create a Cylon effect (everyone remembers Lorne Green in Battlestar Galactica, don't they?). An and mask is applied to determine the status of each bit which in turn controls the state of my array of on-screen LEDs. It all works as expected and I cannot think of any more enhancements, at the moment, so now I'm looking for a new project. -- Regards, Phil From cs at cskk.id.au Thu Aug 5 22:14:09 2021 From: cs at cskk.id.au (Cameron Simpson) Date: Fri, 6 Aug 2021 12:14:09 +1000 Subject: [Tutor] Magic numbers In-Reply-To: <3243ceea-7e64-55be-7a1a-9fa15f327a3c@gmail.com> References: <3243ceea-7e64-55be-7a1a-9fa15f327a3c@gmail.com> Message-ID: On 06Aug2021 11:08, Phil wrote: >Looping forever to create a Cylon effect (everyone remembers Lorne >Green in Battlestar Galactica, don't they?). Aye. The PDP-11s at uni had a front panel display which could show any memory address (as a row of bits, of course). We ran a kernel which updated a Cylon display in a memory address in the kernel's idle loop. When the system was idle or lightly loaded the effect was smooth and when the system was busy the effect was jerky. As handy as a flame graph for load :-) Cheers, Cameron Simpson From alan.gauld at yahoo.co.uk Fri Aug 6 04:02:39 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Fri, 6 Aug 2021 09:02:39 +0100 Subject: [Tutor] Magic numbers In-Reply-To: <3243ceea-7e64-55be-7a1a-9fa15f327a3c@gmail.com> References: <89999336-d925-7da5-cbe7-2ab6646662ec@gmail.com> <3243ceea-7e64-55be-7a1a-9fa15f327a3c@gmail.com> Message-ID: On 06/08/2021 02:08, Phil wrote: > On 5/8/21 7:32 pm, Alan Gauld via Tutor wrote: > > Thank you everyone for your replies, I didn't expect so much interest. >> Incidentally, why are you using bit shifting rather >> than regular arithmetic? > > I'm simulating a shift register that can shift either left or right. Ah! Ok, that's not really "doing bit shifting calculations" it is doing actual bit shifting! It makes complete sense in that context. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From edwinconnell at gmail.com Thu Aug 5 16:47:11 2021 From: edwinconnell at gmail.com (Ed Connell) Date: Thu, 5 Aug 2021 15:47:11 -0500 Subject: [Tutor] Copying one text file into another In-Reply-To: References: Message-ID: Thanks for your suggestions. Though I can see why you thought of HOMEWORK, that is funny since I will turn 80 in a few days. :) I relaxed my "requirements" a bit (sorry), but I am interested in your suggestions. Recently I noticed that the same thing occurred in programs over and over - like getting an integer in a certain range or confirming something - so I developed some short functions to handle those situations. I am attaching a printout that shows my code on the current topic as well as several of those functions. I am just having fun. Thanks for all you do, Ed Connell Later I tried my hand at the rest and came up with the following. Again, I'm just having fun! Here the destination is a temp file as was suggested by someone. def insert_file( dest, lookfor, filea, fileb ): with open( dest, 'w' ) as fout: with open( filea, 'r' ) as fin: while True: lin = fin.readline() print( lin[:-1], file = fout ) if len( lin ) == 0: print( '\n\n\n', file = fout ) return if lookfor in lin: break with open( fileb, 'r' ) as fin2: while True: lin = fin2.readline() print( lin[:-1], file = fout ) if len( lin ) == 0: print( '\n\n\n', file = fout ) break while True: lin = fin.readline() print( lin[:-1], file = fout ) if len( lin ) == 0: print( '\n\n\n', file = fout ) return if __name__ == '__main__': insert_file( 'teds.py', 'XXXXX', 'eds.py', 'test.py' ) On Tue, Aug 3, 2021 at 1:55 PM Ed Connell wrote: > Suppose you have two text files, A.txt and B.txt. You want to copy A into > B at some designated place .Of course there is copy and paste, but I got > interested in how you could do this with Python. Can you help? > > Ed Connell > > -- > I have a right and a left brain, but there is nothing right in the left > one and there is nothing left in the right one! > > -- I have a right and a left brain, but there is nothing right in the left one and there is nothing left in the right one! -------------- next part -------------- ============================================================ # 8/5/21 print( 'One for the money...' ) def combine( destination, *source ): with open( destination, 'w' ) as fout: print( '\n\n\n\n============================================================\n\n\n', file = fout ) print( '\n\n\n\n============================================================\n\n\n' ) for src in source: with open( src, 'r' ) as fin: while True: lin = fin.readline() if len( lin ) == 0: break print( lin[ : -1 ], file = fout ) print( lin[ : -1 ] ) print( '\n\n\n\n============================================================\n\n\n', file = fout ) print( '\n\n\n\n============================================================\n\n\n' ) print( "That's all folks..." ) if __name__ == '__main__': combine( 'tmp.txt', 'filework.py', 'binary_search.py', 'get_char.py', 'get_int_float.py',\ 'int_pair.py', 'pick_from_list.py', 'yes_ok.py' ) ============================================================ # 8/5/21 import random probes = 1 thesize = 100 def binary_search( left, right, thelist, target ): global probes # just used to see resulting operation print( probes, ' ', end='' ) probes += 1 mid = ( left + right ) // 2 if right - left <= 3: if right == len( thelist ): right -= 1 for x in range( left, right ): if thelist[ x ] == target: print( 'Alternate: ', end='' ) return x return -1 found = thelist[ mid ] if found == target: return mid elif found < target: left = mid + 1 elif found > target: right = mid return binary_search( left, right, thelist, target ) if __name__ == '__main__': input( '\n\nFirst with integers. Press ENTER to start ...' ) right = thesize thelist = sorted( [ random.randint( 0, 99 ) for _ in range( right ) ] ) print( '\n\n', thelist ) target = random.randint( 0, 99 ) print( '\n\ntarget: ', target, '\nprobes: ', end='' ) position = binary_search( 0, right, thelist, target ) if position >=0: print( '\n\nposition: ', position ) start = position - 10 if position >= 10 else 0 print( '\n\n', thelist[ start : position ] ) print( '\n', thelist[ position ] ) print( '\n', thelist[ position+1: position+11 ] ) else: print( '\n\nNot in the list...' ) input( '\n\nPress ENTER to continue to a list of floats...' ) probes = 1 right = thesize thelist = sorted( [ round( random.random() * 10, 2 ) for _ in range( right ) ] ) print( '\n\n', thelist ) target = round( random.random() * 10, 2 ) print( '\n\ntarget: ', target, '\nprobes: ', end='' ) position = binary_search( 0, right, thelist, target ) if position >=0: print( '\n\nposition: ', position ) start = position - 10 if position >= 10 else 0 print( '\n\n', thelist[ start : position ] ) print( '\n', thelist[ position ] ) print( '\n', thelist[ position+1: position+11 ] ) else: print( '\n\nNot in the list...' ) ============================================================ # 8/2/21 from msvcrt import getch def getchar(): return chr( getch()[0] ) if __name__ == '__main__': print( '\n\nPress some ', end = '' ) print( 'keys.\n\n q or Q to quit' ) # this works fine while True: k = getchar() print( '+', k, 'ordinal =', ord(k), end = '' ) # it seems that if the end='' is followed by another call to getchar() there are problems # try putting a # before the next statement ( the print() ) print( ' last' ) if k == 'q' or k == 'Q': break print( '\n\nTrying down ', end = '' ) print( 'here.\n' ) # again - works fine ============================================================ # 8/2/21 def getint( s, low = 0, high = None, show_range = False ): low_limit = str(low) if low != None else '-inf' high_limit = str(high) if high != None else 'inf' if show_range: s += ' ** range: ' + low_limit + ' thru ' + high_limit while True: result = input( s + ': ' ) if result == '': return None try: result = int( result ) except: print( 'Must be an integer...' ) continue if (low != None) and (low > result ): print( "Can't be less than ", low ) continue if (high != None) and (high < result ): print( "Can't be greater than ", high ) continue return result def getfloat( s, low = 0, high = None, show_range = False ): low_limit = str(low) if low != None else '-inf' high_limit = str(high) if high != None else 'inf' if show_range: s += ' ** range: ' + low_limit + ' thru ' + high_limit while True: result = input( s + ': ' ) if result == '': return None try: result = float( result ) except: print( 'Must be an float...' ) continue if (low != None) and (low > result ): print( "Can't be less than ", low ) continue if (high != None) and (high < result ): print( "Can't be greater than ", high ) continue return result if __name__ == '__main__': a = getint( '\n\nEnter an integer', low = 9, high = 11, show_range = True ) print( 'Got ', a ) b = getint( '\n\nEnter a non-negative intger ' ) print( 'Got ', b ) c = getint( '\n\nEnter an integer less than zero ', low = None, high = -1 ) print( 'Got ', c ) a = getfloat( '\n\nEnter a number', low = 9, high = 11, show_range = True ) print( 'Got ', a ) b = getfloat( '\n\nEnter a non-negative number ' ) print( 'Got ', b ) c = getfloat( '\n\nEnter a number less than zero ', low = None, high = -1 ) print( 'Got ', c ) ============================================================ # 8/3/21 def intpair( s, low=0, high=None ): while True: try: a = input( s + ' Enter two integers separated by a period - (like: 15.7 or 9.34): ' ) if a == '': return None, None elif '.' not in a: print( 'No period...' ) continue a, b = a.split( '.' ) a, b = map( int, [ a, b ] ) if low != None and (a < low or b < low): print( "Can't be less than ", low ) continue if high != None and (a > high or b > high): print( "Can't be greater than ", high ) continue return a, b except: print( 'Invalid, must be two nonnegative integers seperated by a period ...' ) print( ' four examples: 6.8 : 34.21 : 12 . 23 : 0.15 ' ) print( ' four bad examples - not: 32 : 9-7 : 23.67.21 : 6.8 7' ) continue if __name__ == '__main__': x, y = intpair( s = '\n\nSomething - ' ) print( '\n\nx =', x, 'y =', y ) if x == '': print( 'null str is True' ) ============================================================ # 8/2/21 def getint( s: str, low = 0, high = None ): while True: try: res = input( s ) if res == '': return '' res = int( res ) except: print( 'Must be an integer...' ) continue if (low != None) and (low > res): print( "Can't be less than ", low ) continue if (high != None) and (high < res): print( "Can't be greater than ", high ) continue return res def pick_from_list( thelist: list ) -> str: print() thelist.sort() newlist = [ 'None' ] + thelist for i, nm in enumerate( newlist ): # None gets paired with 0 print( i, nm ) choice = getint( 'Choice? ', low = 0, high = len( newlist ) - 1 ) if choice == 0 or choice == '': return None else: return newlist[ choice ] if __name__ == '__main__': mylist = [ 'top', 'bottom', 'side' ] pick = pick_from_list( mylist ) print( pick, 'chosen' ) ============================================================ # 8/2/21 def yes( s, default_yes = False ): if default_yes: s += ' (Y or y or null = yes): ' else: s += ' (only Y or y for yes): ' result = input( s ).upper() return True if result == 'Y' or (default_yes and result == '') else False if __name__ == '__main__': if yes( '\n\nDo something. ' ): print( '\n\nTrue ' ) else: print( '\n\nFalse ' ) if yes( '\n\nDo something. ', default_yes = True ): print( '\n\nTrue ' ) else: print( '\n\nFalse ' ) ============================================================ From mmssdd1920 at gmail.com Fri Aug 6 05:19:14 2021 From: mmssdd1920 at gmail.com (Msd De) Date: Fri, 6 Aug 2021 14:49:14 +0530 Subject: [Tutor] matrix multiplication Message-ID: I have to take product of three matrices Q = Nn.P1.M in the code below where M = M2.M3.M4...............Mn I would like to know whether the dot is appropriately used. The output is not what I expected. Thanks in advance d_list=[2.0,2.5,3.0,3.5,4.0,4.5,5.0,6.0,8.0,10.0,12.0,14.0,16.0,18.0,20.0] #15 values *for id in range(len(d_list)):* d=d_list[id] R_list = [20] #for test purpose for r in range(len(R_list)): R=R_list[r] etatip=np.sqrt(d/(d+R)) UB=etatip evb_list = [3.0] #for test purpose ######################### for v in range(len(evb_list)): evb=evb_list[v] ............. DEx_11 = []; DF = [] Fj1prxj = [];Fj2prxj = []; Fjmin11prxj =[];Fjmin12prxj = []; Integrand_11 = [] dist=d_list[id] M = [[1,1],[1,1]] #initialise M matrix m=40 *for j in range(1,m+1):* ak1 = np.sqrt(2*FCON*Ex) ak2 = np.sqrt(2*FCON*(Ex- deltaprime)) ............ ................................... det_albe = alfajmin1xj*betajmin1xj - betajmin1xj*alfajmin1xjp1 Fjmin11prxj.append(Xprjmin1j*(alfaprjmin1xj*betajmin1xj - alfajmin1xjp1*betaprjmin1xj )/det_albe) Fjmin12prxj.append(Xprjmin1j*(alfajmin1xj*betaprjmin1xj - betajmin1xj*alfaprjmin1xj )/det_albe) G11_2 = Fj2prxj[0] G11_1 = Fj1prxj[0] P111 =1.0 P112 = 1.0 P121 =(1j*ak1-G11_1)/G11_2 P122 = (G11_1 - 1j*ak1)/G11_2 *P1 =[[P111,P112],[P121,P122]]* GNNp1_1 =Fj1prxj[m-1] #cos number starts fron zero GNNp1_2 = Fj2prxj[m-1] Nn11 =GNNp1_1/1j*ak2 Nn12 = GNNp1_2/1j*ak2 Nn21 = 0.0 Nn22 = 1.0 *Nn =[[Nn11,Nn12],[Nn21,Nn22]]* for n in range(1,m): #M_2,M_3,............M_N Gjj2 = Fj2prxj[n] Gjmin1j1 =Fjmin11prxj[n] Gjmin1j2 =Fjmin12prxj[n] Mj11 = 1.0 Mj12 = 0.0 Mj21 = (Gjmin1j1-Gjj2)/Gjj2 Mj22 = (Gjmin1j2)/Gjj2 Mj = [[Mj11,Mj12],[Mj21,Mj22]] *M = np.dot(M,Mj)* Q = np.dot(Nn,np.dot(M,P1)) Q11=Q[0][0] Q12=Q[0][1] Q21=Q[1][0] Q22=Q[1][1] T = (Q11*Q22 - Q12*Q21)*np.exp(-1j*ak2*d)/(Q22-Q12) DEx_11 = ak2*abs(T)**2/ak1 DF=(np.log(1+np.exp(-BETA*(Ex-eta1)))/BETA) - (np.log(1+np.exp(-BETA*(Ex-eta1+evb)))/BETA) Integrand_11=DEx_11*DF*CONST print Integrand_11 From wlfraed at ix.netcom.com Fri Aug 6 15:04:43 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Fri, 06 Aug 2021 15:04:43 -0400 Subject: [Tutor] Magic numbers References: <89999336-d925-7da5-cbe7-2ab6646662ec@gmail.com> <3243ceea-7e64-55be-7a1a-9fa15f327a3c@gmail.com> Message-ID: <180rggldq7ltsonkmn0qmhdhi3to7r2d16@4ax.com> On Fri, 6 Aug 2021 11:08:35 +1000, Phil declaimed the following: >I'm simulating a shift register that can shift either left or right. I >start with bit 0 equal to one and shift it to the right until I hit the Most people would be shifting a 1 LEFT to reach that 128, than shift right to get back to the first bit. >most significant bit , for example 128 for an 8 bit shift register, and >then reverse the shift until I return to bit zero. Looping forever to >create a Cylon effect (everyone remembers Lorne Green in Battlestar The common name for that is a "Larson Scanner"; he used the same technique on KITT (Knight Rider). https://learn.adafruit.com/larson-scanner-shades >Galactica, don't they?). An and mask is applied to determine the status >of each bit which in turn controls the state of my array of on-screen LEDs. > Well, no LEDs emulated here, just 0/1, but the core of a Larson Scanner import time LED = 1 print(format(LED, "08b"), end="\r", flush=True) while True: for i in range(8): print(format(LED, "08b"), end="\r", flush=True) LED = LED << 1 time.sleep(0.125) for i in range(8): LED = LED >> 1 print(format(LED, "08b"), end="\r", flush=True) time.sleep(0.125) -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From phillor9 at gmail.com Fri Aug 6 17:57:46 2021 From: phillor9 at gmail.com (Phil) Date: Sat, 7 Aug 2021 07:57:46 +1000 Subject: [Tutor] Magic numbers In-Reply-To: <180rggldq7ltsonkmn0qmhdhi3to7r2d16@4ax.com> References: <89999336-d925-7da5-cbe7-2ab6646662ec@gmail.com> <3243ceea-7e64-55be-7a1a-9fa15f327a3c@gmail.com> <180rggldq7ltsonkmn0qmhdhi3to7r2d16@4ax.com> Message-ID: On 7/8/21 5:04 am, Dennis Lee Bieber wrote: > Most people would be shifting a 1 LEFT to reach that 128 Most people know the difference between their left and right hands,? however, on this occasion I do shift bit 0 left to reach bit 7. I only have two hands so it's a 50-50 chance that I pick the correct one. > Well, no LEDs emulated here, just 0/1, but the core of a Larson Scanner I'll have a look at this, no doubt it'll be an improvement on what I've done. -- Regards, Phil From alan.gauld at yahoo.co.uk Sat Aug 7 08:03:41 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sat, 7 Aug 2021 13:03:41 +0100 Subject: [Tutor] Copying one text file into another In-Reply-To: References: Message-ID: On 05/08/2021 21:47, Ed Connell wrote: > Thanks for your suggestions. Though I can see why you thought of HOMEWORK, > that is funny since I will turn 80 in a few days. :) Even 80 year olds can be doing homework! :-) > Here the destination is a temp file as was suggested by someone. > > def insert_file( dest, lookfor, filea, fileb ): > with open( dest, 'w' ) as fout: > with open( filea, 'r' ) as fin: > while True: > lin = fin.readline() > print( lin[:-1], file = fout ) I'm not sure why you are using print() to write to the file. It will work, but it is not expressing clearly the purpose of your code, namely to write to the file. It would be clearer (and hence more maintainable, long term) to use fout.write() [ Bonus points.... Alternatively, just use print() but not to a file. That will display the output on stdout. You can then redirect stdout to a file when you run the program: C:\> python insertfile.py file1.txt file2.txt XXX > results.txt Of course you'd need to write some code to read the arguments from sys.argv when you run the code. This makes your program work like many of the Unix/Linux OS utilities and is easy to debug/test. But it does require a wee bit extra work collecting the arguments from the command line. For bonus points you could even have it read from stdin so that you only specify the marker and the file to insert on the command line. You would then use input (and a while loop!) to read the input and print to display the output. To make it use files you would type: C:\> python insertfileat infile.txt XXX fileout.txt Just for some more fun! :-) ...] > if len( lin ) == 0: > print( '\n\n\n', file = fout ) > return If you used a for loop this block would go in the else block since it is executed when you reach the end of the file: for line in filea: process the line else: #only executed if you reach the end of file. process end of file. > if lookfor in lin: > break You don't really need the break or the three while loops. If you use a for loop to iterate over filea that will guarantee to process all of the file. And you can just insert the processing of the second file at this point. > with open( fileb, 'r' ) as fin2: > while True: > lin = fin2.readline() > print( lin[:-1], file = fout ) > if len( lin ) == 0: > print( '\n\n\n', file = fout ) > break Again a for loop will do the job using the same pattern as above. > while True: > lin = fin.readline() > print( lin[:-1], file = fout ) > if len( lin ) == 0: > print( '\n\n\n', file = fout ) > return And the for loop above would remove the need to duplicate the end of file processing. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From manpritsinghece at gmail.com Sun Aug 8 00:52:55 2021 From: manpritsinghece at gmail.com (Manprit Singh) Date: Sun, 8 Aug 2021 10:22:55 +0530 Subject: [Tutor] Query on sqlite3 module Message-ID: Dear Sir , Consider a table "stocks" being made in a sqlite3 database, whose entries are as follows : date trans symbol qty price ('2006-01-05', 'BUY', 'RHAT', 100, 35.14) ('2006-01-05', 'BUY', 'RHAT', 100, 35.14) ('2006-04-08', 'BUY', 'CPQ', 160, 40.56) ('2006-03-27', 'BUY', 'IBM ', 120, 45.34) 1) I have to update price = 55.62 where symbol is "RHAT": Is the following the correct way to do it ? cur.execute('UPDATE stocks set price=55.62 where symbol= ?', ("RHAT",)) where cur is cursor. 2) I have to print all matching rows where price is 55.62: Is the following the correct way to do it ? cur.execute("select * from stocks where price = ?", (55.62,)) print(cur.fetchall()) Kindly guide Regards Manprit Singh From cs at cskk.id.au Sun Aug 8 01:54:59 2021 From: cs at cskk.id.au (Cameron Simpson) Date: Sun, 8 Aug 2021 15:54:59 +1000 Subject: [Tutor] Query on sqlite3 module In-Reply-To: References: Message-ID: On 08Aug2021 10:22, Manprit Singh wrote: >Consider a table "stocks" being made in a sqlite3 database, whose >entries >are as follows : >date trans symbol qty price > >('2006-01-05', 'BUY', 'RHAT', 100, 35.14) >('2006-01-05', 'BUY', 'RHAT', 100, 35.14) >('2006-04-08', 'BUY', 'CPQ', 160, 40.56) >('2006-03-27', 'BUY', 'IBM ', 120, 45.34) > >1) I have to update price = 55.62 where symbol is "RHAT": > >Is the following the correct way to do it ? > >cur.execute('UPDATE stocks set price=55.62 where symbol= ?', ("RHAT",)) >where cur is cursor. That looks ok to me. I would make 55.62 also a parameter though - almost every value you use in an SQL statement should be a parameter so that the driver can correctly pass it to the database. Whenever you embed a value (particularly strings) in SQL directly there is a risk a mistake can lead to injection, where what you intended to be a simple scalar value was misused by the SQL because it's actually SQL syntax. >2) I have to print all matching rows where price is 55.62: >Is the following the correct way to do it ? > >cur.execute("select * from stocks where price = ?", (55.62,)) >print(cur.fetchall()) That will work just fine. Did you try it? I'd probably write a little loop to print each row, because we usually expect listings to have one row per line. Cheers, Cameron Simpson From s.molnar at sbcglobal.net Sun Aug 8 09:48:00 2021 From: s.molnar at sbcglobal.net (Stephen P. Molnar) Date: Sun, 8 Aug 2021 09:48:00 -0400 Subject: [Tutor] Extract Field from List References: <610FE090.4060902.ref@sbcglobal.net> Message-ID: <610FE090.4060902@sbcglobal.net> I have hit a roadblock in a Python script I have been developing to process a large number of text files of fixed format. I have a list of names of chemicals, which I can read into a list.csv: CaffeicAcid,Cannflavin-A,Cannflavin-B,Cannflavin-C,Diosmetin,Echinacoside,Hesperetin,L-CichoricAcid My code is; import csv with open("Ligand.list_r.txt", newline='') as csvfile: rows = csv.reader(csvfile, delimiter = ',') data = [] for rows in rows: data.append(rows) print(data) This results in: [['CaffeicAcid\tCannflavin-A\tCannflavin-B\tCannflavin-C\tDiosmetin\tEchinacoside\tHesperetin\tL-CichoricAcid']] when run. What I need to do is iterate through the list and generate names of the format of chemicalname+'.i.log'. This is where I can seem to hit a show stopper. How do I extract fields form the list generated by the script that works? What am I missing? -- Stephen P. Molnar, Ph.D. 614.312.7528 (c) Skype: smolnar1 From bouncingcats at gmail.com Sun Aug 8 10:36:33 2021 From: bouncingcats at gmail.com (David) Date: Mon, 9 Aug 2021 00:36:33 +1000 Subject: [Tutor] Extract Field from List In-Reply-To: <610FE090.4060902@sbcglobal.net> References: <610FE090.4060902.ref@sbcglobal.net> <610FE090.4060902@sbcglobal.net> Message-ID: On Sun, 8 Aug 2021 at 23:49, Stephen P. Molnar wrote: > > I have hit a roadblock in a Python script I have been developing to > process a large number of text files of fixed format. > > I have a list of names of chemicals, which I can read into a list.csv: > > CaffeicAcid,Cannflavin-A,Cannflavin-B,Cannflavin-C,Diosmetin,Echinacoside,Hesperetin,L-CichoricAcid > > My code is; > > import csv > > with open("Ligand.list_r.txt", newline='') as csvfile: > rows = csv.reader(csvfile, delimiter = ',') > data = [] > for rows in rows: > data.append(rows) > > print(data) > > This results in: > > [['CaffeicAcid\tCannflavin-A\tCannflavin-B\tCannflavin-C\tDiosmetin\tEchinacoside\tHesperetin\tL-CichoricAcid']] > > when run. > > What I need to do is iterate through the list and generate names of the > format of chemicalname+'.i.log'. > > This is where I can seem to hit a show stopper. How do I extract fields > form the list generated by the script that works? What am I missing? There's several reasons why your code does not give the result you want. You also don't give any clue which version of Python you are using. Python behaviour does differ between versions, so it is always important to mention that. Just run 'python --version' if you are on Linux. Debugging this code might be tricky for you because modern csv.reader() returns a generator object, not a list. So, I suggest as a learning exercise you replace the line: rows = csv.reader(csvfile, delimiter = ',') with rows = list(csv.reader(csvfile, delimiter = ',')) That means that rows will become a list object, which is what older versions of Python used to do, and doing that will make it much easier for you to see what is happening in your code. After that, you can debug your code by sprinkling a few print statements around it so that you can see the data in the variables. For example: print("data = ", data) Then, you will be able to see what the code is doing. And you might be able to debug it yourself. Let us know how it goes. From __peter__ at web.de Sun Aug 8 10:32:06 2021 From: __peter__ at web.de (Peter Otten) Date: Sun, 8 Aug 2021 16:32:06 +0200 Subject: [Tutor] Extract Field from List In-Reply-To: <610FE090.4060902@sbcglobal.net> References: <610FE090.4060902.ref@sbcglobal.net> <610FE090.4060902@sbcglobal.net> Message-ID: On 08/08/2021 15:48, Stephen P. Molnar wrote: > I have hit a roadblock in a Python script I have been developing to > process a large number of text files of fixed format. > > I have a list of names of chemicals, which I can read into a list.csv: > > CaffeicAcid,Cannflavin-A,Cannflavin-B,Cannflavin-C,Diosmetin,Echinacoside,Hesperetin,L-CichoricAcid > > > My code is; > > import csv > > with open("Ligand.list_r.txt", newline='') as csvfile: > ??? rows = csv.reader(csvfile, delimiter = ',') > ??? data = [] > ??? for rows in rows: > ??????? data.append(rows) > > print(data) > > This results in: > > [['CaffeicAcid\tCannflavin-A\tCannflavin-B\tCannflavin-C\tDiosmetin\tEchinacoside\tHesperetin\tL-CichoricAcid']] > > > when run. > > What I need to do is iterate through the list and generate names of the > format of chemicalname+'.i.log'. > > This is where I can seem to hit a show stopper. How do I extract fields > form the list generated by the script that works? What am I missing? (1) Your CSV is tab-delimited (2) There is only one row Try with open(...) as csvfile: rows = csv.reader(csvfile, delimiter='\t') [data] = rows This ensures that there is only one row so that no data can be lost accidentally. Note that data will be a tuple; convert it if necessary. From s.molnar at sbcglobal.net Sun Aug 8 11:23:44 2021 From: s.molnar at sbcglobal.net (Stephen P. Molnar) Date: Sun, 8 Aug 2021 11:23:44 -0400 Subject: [Tutor] Extract Field from List In-Reply-To: References: <610FE090.4060902.ref@sbcglobal.net> <610FE090.4060902@sbcglobal.net> Message-ID: <610FF700.5020409@sbcglobal.net> Python v-3.7.3 The Script returns a one element list: 0 str 99 CaffeicAcid Cannflavin-A Cannflavin-B Cannflavin-C Diosmetin Echinacoside Hesperetin L-CichoricAcid In the next step I want to iteratively insert each name in the list in: for i in range(1,11): name_in = name+.i'.log' data+.i = np.genfromtxt(name_in, skip_header=28, skip_footer=1) data+.i = data+.i[0, 1] Most likely, it would probably be simpler to put these lines of code into a function. Now I'm not certain about formatting the insertion the value of i. Coming out of the loop I want to have a series of files, data1 to data10 to use as arguments for np.vstack(... I would be most grateful for assistance in solving this problem. Thanks in advance. On 08/08/2021 10:36 AM, David wrote: > On Sun, 8 Aug 2021 at 23:49, Stephen P. Molnar wrote: >> I have hit a roadblock in a Python script I have been developing to >> process a large number of text files of fixed format. >> >> I have a list of names of chemicals, which I can read into a list.csv: >> >> CaffeicAcid,Cannflavin-A,Cannflavin-B,Cannflavin-C,Diosmetin,Echinacoside,Hesperetin,L-CichoricAcid >> >> My code is; >> >> import csv >> >> with open("Ligand.list_r.txt", newline='') as csvfile: >> rows = csv.reader(csvfile, delimiter = ',') >> data = [] >> for rows in rows: >> data.append(rows) >> >> print(data) >> >> This results in: >> >> [['CaffeicAcid\tCannflavin-A\tCannflavin-B\tCannflavin-C\tDiosmetin\tEchinacoside\tHesperetin\tL-CichoricAcid']] >> >> when run. >> >> What I need to do is iterate through the list and generate names of the >> format of chemicalname+'.i.log'. >> >> This is where I can seem to hit a show stopper. How do I extract fields >> form the list generated by the script that works? What am I missing? > There's several reasons why your code does not give the result you want. > > You also don't give any clue which version of Python you are using. > Python behaviour does differ between versions, so it is always > important to mention that. Just run 'python --version' if you are on Linux. > > Debugging this code might be tricky for you because > modern csv.reader() returns a generator object, not a list. > > So, I suggest as a learning exercise you replace the line: > rows = csv.reader(csvfile, delimiter = ',') > with > rows = list(csv.reader(csvfile, delimiter = ',')) > That means that rows will become a list object, which is what older > versions of Python used to do, and doing that will make it much > easier for you to see what is happening in your code. > > After that, you can debug your code by sprinkling a few print statements > around it so that you can see the data in the variables. > For example: > print("data = ", data) > Then, you will be able to see what the code is doing. And you might be > able to debug it yourself. Let us know how it goes. > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > -- Stephen P. Molnar, Ph.D. 614.312.7528 (c) Skype: smolnar1 From wlfraed at ix.netcom.com Sun Aug 8 12:13:28 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Sun, 08 Aug 2021 12:13:28 -0400 Subject: [Tutor] Query on sqlite3 module References: Message-ID: On Sun, 8 Aug 2021 10:22:55 +0530, Manprit Singh declaimed the following: >Consider a table "stocks" being made in a sqlite3 database, whose entries >are as follows : >date trans symbol qty price > >('2006-01-05', 'BUY', 'RHAT', 100, 35.14) >('2006-01-05', 'BUY', 'RHAT', 100, 35.14) >('2006-04-08', 'BUY', 'CPQ', 160, 40.56) >('2006-03-27', 'BUY', 'IBM ', 120, 45.34) > >1) I have to update price = 55.62 where symbol is "RHAT": > You appear to have an unusual Use Case there. The database indicates that it is a time-ordered list of market transactions. A change in stock price normally does not get applied retroactively -- the price in the transaction is presumed to be the price that was paid AT THE TIME OF THE TRANSACTION. If there is a new price, it would be entered as part of a new transaction, on a new date. The only justification I can see for modifying historical prices would be if a data entry clerk made the mistake when entering the day's transactions. >Is the following the correct way to do it ? > >cur.execute('UPDATE stocks set price=55.62 where symbol= ?', ("RHAT",)) >where cur is cursor. As has already been mentioned, price should also be a parameter... >cur.execute('UPDATE stocks set price=? where symbol= ?', (price, "RHAT")) > >2) I have to print all matching rows where price is 55.62: >Is the following the correct way to do it ? > >cur.execute("select * from stocks where price = ?", (55.62,)) Given the sample database, I'd suggest adding an ORDER BY (RDBMs are not required -- by definition of a relation -- to return results in any particular order. >cur.execute("select * from stocks where price = ? ORDER BY date", (55.62,)) >Kindly guide https://en.wikipedia.org/wiki/SQL_syntax https://en.wikipedia.org/wiki/Database_normalization https://www.sqlite.org/docs.html https://www.sqlitetutorial.net/ -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From wlfraed at ix.netcom.com Sun Aug 8 12:31:41 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Sun, 08 Aug 2021 12:31:41 -0400 Subject: [Tutor] Extract Field from List References: <610FE090.4060902.ref@sbcglobal.net> <610FE090.4060902@sbcglobal.net> Message-ID: On Sun, 8 Aug 2021 09:48:00 -0400, "Stephen P. Molnar" declaimed the following: How many mistakes can I see... >with open("Ligand.list_r.txt", newline='') as csvfile: > rows = csv.reader(csvfile, delimiter = ',') This returns an object configured to read the file -- it does not return the rows en-mass. > data = [] > for rows in rows: This statement replaces the reader object by the first row of the data -- hence it does not process the data, only the header line > data.append(rows) This appends the "list" of the header fields into a list, thereby making a list containing one element -- itself a list. Since the FOR statement trashed the CSV reader object, I'm not sure what happens on the second pass -- either it falls out as if at end-of-input (as your output suggests); or it pulls out the first header field, appends that as a second list, then (given the replacement problem) pulls the first character, and then ... >This results in: > >[['CaffeicAcid\tCannflavin-A\tCannflavin-B\tCannflavin-C\tDiosmetin\tEchinacoside\tHesperetin\tL-CichoricAcid']] > >when run. > >What I need to do is iterate through the list and generate names of the >format of chemicalname+'.i.log'. > The headers are the first LIST WITHIN "data"... >This is where I can seem to hit a show stopper. How do I extract fields >form the list generated by the script that works? What am I missing? Indexing to get to the first list within a list? for field in data[0]: I suspect I'm going to be a bit offensive in my closing... Please read the Python tutorial... Your name shows up nearly monthly with essentially the same type of simple Python questions. You should be able to learn Python well enough to not need to ask a question for every variation of your "ligands" processing. This is not, to my knowledge, a forum to do your work for you; your questions always seem to be at the level of high-school (or even earlier) homework, which this forum does not do either. Show us what you've tried, where it seems to go wrong, etc.; and we provide suggestions to solve the problem. At the least, walk through your simple code snippets with a pencil and paper -- eg: pretend to be the computer, for each statement, write down what the contents of variables are, etc. -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From wlfraed at ix.netcom.com Sun Aug 8 12:59:25 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Sun, 08 Aug 2021 12:59:25 -0400 Subject: [Tutor] Extract Field from List References: <610FE090.4060902.ref@sbcglobal.net> <610FE090.4060902@sbcglobal.net> <610FF700.5020409@sbcglobal.net> Message-ID: On Sun, 8 Aug 2021 11:23:44 -0400, "Stephen P. Molnar" declaimed the following: >Python v-3.7.3 > >The Script returns a one element list: > >0 str 99 CaffeicAcid Cannflavin-A Cannflavin-B Cannflavin-C >Diosmetin Echinacoside Hesperetin L-CichoricAcid > Because, as was mentioned, your data is not a Comma SV -- it is a Tab SV! Change the format of your CSV reader to use tab separation. Then you will get a list of lists. >In the next step I want to iteratively insert each name in the list in: > >for i in range(1,11): > name_in = name+.i'.log' Where is "name" defined? What is ".i" supposed to be doing -- "i" is an integer, not a string, and you can't add integers to strings. > data+.i = np.genfromtxt(name_in, skip_header=28, skip_footer=1) > data+.i = data+.i[0, 1] You are creating whatever you think "data+.i" produces using a numpy call... and then immediately replacing it with what I interpret as the first row, second column of itself. > >Most likely, it would probably be simpler to put these lines of code >into a function. > >Now I'm not certain about formatting the insertion the value of i. >Coming out of the loop I want to have a series of files, data1 to data10 >to use as arguments for np.vstack(... If you expect a series of files "data1" through "data10", what is all that rigmarole with "name"? If you mean you want 1 through 10, say so. "data"x is not a file -- it is a numpy array containing the contents of a file... Note -- you can not dynamically create variable names in Python (at least, not without using very advanced methods that get into how Python itself operates). > >I would be most grateful for assistance in solving this problem. data = [] #main results list for name in : data_i = [] #result list for one "ligand"/name for i in range(1,11): full_name = "%s.%s.log" % (name, i) #create file name data_i.append(np.genfromtxt(full_name, skip_header=28, skip_footer=1) #fetch contents of said file, append to result list data.append(data_i) #append "name" results list to main results To access the data for, say, name-2, and file 3, you would use data[1][2] #Python lists start at 0, not one, so you have to #adjust the indices used. "data[1]" gets you the list for the second "name", then applying [2] to that gets you the third numpy array. -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From wlfraed at ix.netcom.com Sun Aug 8 13:08:46 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Sun, 08 Aug 2021 13:08:46 -0400 Subject: [Tutor] Extract Field from List References: <610FE090.4060902.ref@sbcglobal.net> <610FE090.4060902@sbcglobal.net> <610FF700.5020409@sbcglobal.net> Message-ID: On Sun, 08 Aug 2021 12:59:25 -0400, Dennis Lee Bieber declaimed the following: If you want to access using the "name" you initially read, change the outer "data" from a list to a dictionary... > >data = [] #main results list data = {} >for name in : > data_i = [] #result list for one "ligand"/name > for i in range(1,11): > full_name = "%s.%s.log" % (name, i) #create file name > data_i.append(np.genfromtxt(full_name, > skip_header=28, skip_footer=1) > #fetch contents of said file, append to result list > data.append(data_i) #append "name" results list to main results data[name] = data_i > > To access the data for, say, name-2, and file 3, you would use > > data[1][2] #Python lists start at 0, not one, so you have to > #adjust the indices used. data["name-2"][2] -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From lucas at labcodes.com.br Sun Aug 8 10:02:55 2021 From: lucas at labcodes.com.br (Lucas Cavalcante) Date: Sun, 8 Aug 2021 17:02:55 +0300 Subject: [Tutor] Extract Field from List In-Reply-To: <610FE090.4060902@sbcglobal.net> References: <610FE090.4060902.ref@sbcglobal.net> <610FE090.4060902@sbcglobal.net> Message-ID: Hey Stephan, It is not quite clear what is the desired output you expect to obtain. Can you draft how would you like the output to be from this example you shared? Kind regards, Lucas C Em dom, 8 de ago de 2021 16:49, Stephen P. Molnar escreveu: > I have hit a roadblock in a Python script I have been developing to > process a large number of text files of fixed format. > > I have a list of names of chemicals, which I can read into a list.csv: > > > CaffeicAcid,Cannflavin-A,Cannflavin-B,Cannflavin-C,Diosmetin,Echinacoside,Hesperetin,L-CichoricAcid > > My code is; > > import csv > > with open("Ligand.list_r.txt", newline='') as csvfile: > rows = csv.reader(csvfile, delimiter = ',') > data = [] > for rows in rows: > data.append(rows) > > print(data) > > This results in: > > > [['CaffeicAcid\tCannflavin-A\tCannflavin-B\tCannflavin-C\tDiosmetin\tEchinacoside\tHesperetin\tL-CichoricAcid']] > > when run. > > What I need to do is iterate through the list and generate names of the > format of chemicalname+'.i.log'. > > This is where I can seem to hit a show stopper. How do I extract fields > form the list generated by the script that works? What am I missing? > > -- > Stephen P. Molnar, Ph.D. > 614.312.7528 (c) > Skype: smolnar1 > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From alexkleider at gmail.com Sun Aug 8 22:05:23 2021 From: alexkleider at gmail.com (Alex Kleider) Date: Sun, 8 Aug 2021 19:05:23 -0700 Subject: [Tutor] Terminating "\M" characters with regard to csv.DictWriter Message-ID: Using the csv module's DictReader and DictWriter to update data in a file one of my functions is as follows: def dict_write(f, fieldnames, iterable): """ Writes all records received from into a new csv file named . defines the record keys. Code writen in such a way that could be a generator function. """ with open(f, 'w', newline="") as outfile_obj: print("Opening {} for output...".format(outfile_obj.name)) dict_writer = csv.DictWriter(outfile_obj, fieldnames) dict_writer.writeheader() for record in iterable: dict_writer.writerow(record) All works as I would like except that the newly generated csv file has a '\M' at the end of each line. I only became aware of this when doing a diff between the original file and the one produced by my function. When examining the files using vim (my text editor of choice) they look the same (except for the changes I meant to make) so things seem to be working fine and the system doesn't seem to mind. Should I be setting the named 'newline' parameter to something other than an empty string? From where do the '\M' characters come??? I'd very much like to be able to use the 'diff' command to check if the correct changes are being made but at present, diff marks every line as being different! TIA Alex Kleider (Using python 7 on Debian GNU/Linux) From wlfraed at ix.netcom.com Sun Aug 8 22:25:16 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Sun, 08 Aug 2021 22:25:16 -0400 Subject: [Tutor] Terminating "\M" characters with regard to csv.DictWriter References: Message-ID: On Sun, 8 Aug 2021 19:05:23 -0700, Alex Kleider declaimed the following: >All works as I would like except that the newly generated csv file has a >'\M' at the end of each line. I only became aware of this when doing a Are you sure it's a \M? Capital M? Standard line endings are one of \r\n, \n, \r (Windows, Linux/UNIX, old Macintosh/TRS-80, respectively) >(Using python 7 on Debian GNU/Linux) No such snake exists in nature. Obsolete Python is 2.x, current Python is 3.x. -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From robertvstepp at gmail.com Sun Aug 8 22:57:21 2021 From: robertvstepp at gmail.com (boB Stepp) Date: Sun, 8 Aug 2021 21:57:21 -0500 Subject: [Tutor] Terminating "\M" characters with regard to csv.DictWriter In-Reply-To: References: Message-ID: On 21/08/08 07:05PM, Alex Kleider wrote: >Using the csv module's DictReader and DictWriter to update data in a file >one of my functions is as follows: > >def dict_write(f, fieldnames, iterable): > """ > Writes all records received from into a new csv > file named . defines the record keys. > Code writen in such a way that could be > a generator function. > """ > with open(f, 'w', newline="") as outfile_obj: > print("Opening {} for output...".format(outfile_obj.name)) > dict_writer = csv.DictWriter(outfile_obj, fieldnames) > dict_writer.writeheader() > for record in iterable: > dict_writer.writerow(record) > >All works as I would like except that the newly generated csv file has a >'\M' at the end of each line. I only became aware of this when doing a In a comment to the answer at https://unix.stackexchange.com/questions/82536/showing-type-of-newline-character-in-emacs it says: And if you set the buffer to Unix line endings when the file contains DOS line endings, it will show the CRs as \Ms at the end of each line ? Michael Mrozek Jul 10 '13 at 19:53 I imagine emacs and vim display things similarly and I know you use Linux. Could this be it? -- Wishing you only the best, boB Stepp From bouncingcats at gmail.com Sun Aug 8 23:18:25 2021 From: bouncingcats at gmail.com (David) Date: Mon, 9 Aug 2021 13:18:25 +1000 Subject: [Tutor] Terminating "\M" characters with regard to csv.DictWriter In-Reply-To: References: Message-ID: On Mon, 9 Aug 2021 at 12:58, boB Stepp wrote: > On 21/08/08 07:05PM, Alex Kleider wrote: > >All works as I would like except that the newly generated csv file has a > >'\M' at the end of each line. I only became aware of this when doing a It would help us to answer you if you would explain exactly what method you are using that shows you this '\M'. > I imagine emacs and vim display things similarly and I know you use Linux. > Could this be it? The carriage return character is ASCII decimal 13 which can be generated by holding the [ctrl] key and pressing the [M] key, sometimes written as ctrl-M or ^M. Python represents this character as '\r'. If a file contains this character, run cat -A to display its contents will show this character as '^M', and line-feed character ASCII decimal 10 will be shown as '$' at the end of the line. To see a hex representation of the characters in a file, run od -A x -t x1z From cs at cskk.id.au Sun Aug 8 23:04:18 2021 From: cs at cskk.id.au (Cameron Simpson) Date: Mon, 9 Aug 2021 13:04:18 +1000 Subject: [Tutor] Terminating "\M" characters with regard to csv.DictWriter In-Reply-To: References: Message-ID: On 08Aug2021 21:57, boB Stepp wrote: >>All works as I would like except that the newly generated csv file has >>a >>'\M' at the end of each line. I only became aware of this when doing a > >In a comment to the answer at >https://unix.stackexchange.com/questions/82536/showing-type-of-newline-character-in-emacs > >it says: > > >And if you set the buffer to Unix line endings when the file contains DOS line >endings, it will show the CRs as \Ms at the end of each line ? Michael Mrozek >Jul 10 '13 at 19:53 > > >I imagine emacs and vim display things similarly and I know you use Linux. >Could this be it? And for reference, the "M" is likely because \r (carriage return) is character 13, also known as control-M because you can often enter it that way, and M is the thirteenth letter of the ASCII alphabet, thus the convention. So yes, Alex needs tighter control of the line ending. Cheers, Cameron Simpson From __peter__ at web.de Mon Aug 9 03:58:34 2021 From: __peter__ at web.de (Peter Otten) Date: Mon, 9 Aug 2021 09:58:34 +0200 Subject: [Tutor] Terminating "\M" characters with regard to csv.DictWriter In-Reply-To: References: Message-ID: On 09/08/2021 04:05, Alex Kleider wrote: > Using the csv module's DictReader and DictWriter to update data in a file > one of my functions is as follows: > > def dict_write(f, fieldnames, iterable): > """ > Writes all records received from into a new csv > file named . defines the record keys. > Code writen in such a way that could be > a generator function. > """ > with open(f, 'w', newline="") as outfile_obj: > print("Opening {} for output...".format(outfile_obj.name)) > dict_writer = csv.DictWriter(outfile_obj, fieldnames) > dict_writer.writeheader() > for record in iterable: > dict_writer.writerow(record) > > All works as I would like except that the newly generated csv file has a > '\M' at the end of each line. As explained by others the ^M or chr(13) or "\r" is part of a windows line ending "\r\n". I'd like to clarify that it is *not* inserted by the file object but by the csv.writer(). Its default dialect is "excel" which writes "\r\n" as the line separator regardless of the operating system's line terminator (os.linesep). To avoid it you can either override the line terminator with # excel dialect, but with newlines only writer = csv.writer(outstream, lineterminator="\n") or pick a different dialect writer = csv.writer(outstream, dialect="unix") >>> f = open("tmp.csv", "w", newline="") >>> csv.writer(f).dialect.lineterminator '\r\n' >>> csv.writer(f, lineterminator="\n").dialect.lineterminator '\n' >>> csv.writer(f, dialect="unix").dialect.lineterminator '\n' I only became aware of this when doing a > diff between the original file and the one produced by my function. When > examining the files using vim (my text editor of choice) they look the same > (except for the changes I meant to make) so things seem to be working fine > and the system doesn't seem to mind. > Should I be setting the named 'newline' parameter to something other than > an empty string? From where do the '\M' characters come??? I'd very much > like to be able to use the 'diff' command to check if the correct changes > are being made but at present, diff marks every line as being different! > TIA > Alex Kleider > > (Using python 7 on Debian GNU/Linux) > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From alexkleider at gmail.com Mon Aug 9 23:47:13 2021 From: alexkleider at gmail.com (Alex Kleider) Date: Mon, 9 Aug 2021 20:47:13 -0700 Subject: [Tutor] Terminating "\M" characters with regard to csv.DictWriter In-Reply-To: References: Message-ID: On Sun, Aug 8, 2021 at 7:26 PM Dennis Lee Bieber wrote: > > Are you sure it's a \M? Capital M? > Yes, it is a capital M, here's the first line of the diff command output: < first,last,phone,address,town,state,postal_code,country,email,dues,dock,kayak,mooring,status^M > > >(Using python 7 on Debian GNU/Linux) > > No such snake exists in nature. Obsolete Python is 2.x, current > Python > is 3.x. > Should have been 3.7- major error on my part. Also perhaps should have mentioned it's Debian 10 (since I hear Debian 11 is soon to be the stable release.) > > > From alexkleider at gmail.com Mon Aug 9 23:53:04 2021 From: alexkleider at gmail.com (Alex Kleider) Date: Mon, 9 Aug 2021 20:53:04 -0700 Subject: [Tutor] Terminating "\M" characters with regard to csv.DictWriter In-Reply-To: References: Message-ID: On Sun, Aug 8, 2021 at 8:20 PM David wrote: > > It would help us to answer you if you would explain exactly what method > you are using that shows you this '\M'. > I'm using the unix/Linux diff command's output redirected to a file which I then examine with my editor 'vim'. > > The carriage return character is ASCII decimal 13 which can be generated > by holding the [ctrl] key and pressing the [M] key, sometimes written as > ctrl-M or ^M. Python represents this character as '\r'. > > If a file contains this character, run > cat -A > to display its contents will show this character as '^M', and line-feed > character ASCII decimal 10 will be shown as '$' at the end of the line. > > To see a hex representation of the characters in a file, run > od -A x -t x1z > Thanks for these tips. From alexkleider at gmail.com Tue Aug 10 00:03:53 2021 From: alexkleider at gmail.com (Alex Kleider) Date: Mon, 9 Aug 2021 21:03:53 -0700 Subject: [Tutor] Terminating "\M" characters with regard to csv.DictWriter In-Reply-To: References: Message-ID: On Mon, Aug 9, 2021 at 1:14 AM Peter Otten <__peter__ at web.de> wrote: > As explained by others the ^M or chr(13) or "\r" is part of a windows > line ending "\r\n". > I'd like to clarify that it is *not* inserted by the file object but by > the csv.writer(). Its default dialect is "excel" which writes "\r\n" as > the line separator regardless of the operating system's line terminator > (os.linesep). To avoid it you can either override the line terminator with > > # excel dialect, but with newlines only > writer = csv.writer(outstream, lineterminator="\n") > This solved my problem; Thank you very much for the suggestion. > > or pick a different dialect > > writer = csv.writer(outstream, dialect="unix") > > > This got rid of the '\M' at the end of each line but created a new problem in that each field was in quotes!! Thanks again for taking the trouble of helping. Cheers, Alex PS I need to ponder the following a bit more:-) >>> f = open("tmp.csv", "w", newline="") > >>> csv.writer(f).dialect.lineterminator > '\r\n' > >>> csv.writer(f, lineterminator="\n").dialect.lineterminator > '\n' > >>> csv.writer(f, dialect="unix").dialect.lineterminator > '\n' > From cs at cskk.id.au Tue Aug 10 00:39:22 2021 From: cs at cskk.id.au (Cameron Simpson) Date: Tue, 10 Aug 2021 14:39:22 +1000 Subject: [Tutor] Terminating "\M" characters with regard to csv.DictWriter In-Reply-To: References: Message-ID: On 09Aug2021 20:47, Alex Kleider wrote: >On Sun, Aug 8, 2021 at 7:26 PM Dennis Lee Bieber >wrote: >> Are you sure it's a \M? Capital M? > >Yes, it is a capital M, here's the first line of the diff command output: >< >first,last,phone,address,town,state,postal_code,country,email,dues,dock,kayak,mooring,status^M That's "^M", not "\M". ^x is a common shorthand for control-x. And is also what the "control" key does on your keyboard. ^J => newline, ^M => carriage return, etc. Cheers, Cameron Simpson From cs at cskk.id.au Tue Aug 10 00:37:02 2021 From: cs at cskk.id.au (Cameron Simpson) Date: Tue, 10 Aug 2021 14:37:02 +1000 Subject: [Tutor] Terminating "\M" characters with regard to csv.DictWriter In-Reply-To: References: Message-ID: On 09Aug2021 21:03, Alex Kleider wrote: >PS I need to ponder the following a bit more:-) > >>> f = open("tmp.csv", "w", newline="") >> >>> csv.writer(f).dialect.lineterminator >> '\r\n' I'd imagine the empty string looks false, and falls back to the default. Cheers, Cameron Simpson From alexkleider at gmail.com Tue Aug 10 02:29:19 2021 From: alexkleider at gmail.com (Alex Kleider) Date: Mon, 9 Aug 2021 23:29:19 -0700 Subject: [Tutor] Terminating "\M" characters with regard to csv.DictWriter In-Reply-To: References: Message-ID: On Mon, Aug 9, 2021 at 9:39 PM Cameron Simpson wrote: > On 09Aug2021 20:47, Alex Kleider wrote: > >On Sun, Aug 8, 2021 at 7:26 PM Dennis Lee Bieber > >wrote: > >> Are you sure it's a \M? Capital M? > > > >Yes, it is a capital M, here's the first line of the diff command output: > >< > > >first,last,phone,address,town,state,postal_code,country,email,dues,dock,kayak,mooring,status^M > > That's "^M", not "\M". ^x is a common shorthand for control-x. And is > also what the "control" key does on your keyboard. ^J => newline, ^M => > carriage return, etc. > > Cheers, > Cameron Simpson > From alexkleider at gmail.com Tue Aug 10 02:33:17 2021 From: alexkleider at gmail.com (Alex Kleider) Date: Mon, 9 Aug 2021 23:33:17 -0700 Subject: [Tutor] Terminating "\M" characters with regard to csv.DictWriter In-Reply-To: References: Message-ID: On Mon, Aug 9, 2021 at 9:39 PM Cameron Simpson wrote: > On 09Aug2021 20:47, Alex Kleider wrote: > >On Sun, Aug 8, 2021 at 7:26 PM Dennis Lee Bieber > >wrote: > >> Are you sure it's a \M? Capital M? > > > >Yes, it is a capital M, here's the first line of the diff command output: > >< > > >first,last,phone,address,town,state,postal_code,country,email,dues,dock,kayak,mooring,status^M > > That's "^M", not "\M". ^x is a common shorthand for control-x. And is > also what the "control" key does on your keyboard. ^J => newline, ^M => > carriage return, etc. > > Cheers, > Cameron Simpson > Ah! Staring me in the face but I managed to miss it! Thanks for the clarification. Alex From __peter__ at web.de Tue Aug 10 02:28:32 2021 From: __peter__ at web.de (Peter Otten) Date: Tue, 10 Aug 2021 08:28:32 +0200 Subject: [Tutor] Terminating "\M" characters with regard to csv.DictWriter In-Reply-To: References: Message-ID: On 10/08/2021 06:37, Cameron Simpson wrote: > On 09Aug2021 21:03, Alex Kleider wrote: >> PS I need to ponder the following a bit more:-) >>>>> f = open("tmp.csv", "w", newline="") >>> >>> csv.writer(f).dialect.lineterminator >>> '\r\n' > > I'd imagine the empty string looks false, and falls back to the default. I'm not sure I understand that remark. If you talk about the newline="" argument -- "" means no translation, None means "convert '\n' to the os-specific line separator", and False is illegal. On windows: >>> with open("tmp.txt", "w", newline="") as f: f.write("foo\nbar\r\n") 9 >>> open("tmp.txt", "rb").read() b'foo\nbar\r\n' >>> with open("tmp.txt", "w", newline=None) as f: f.write("foo\nbar\r\n") 9 >>> open("tmp.txt", "rb").read() b'foo\r\nbar\r\r\n' >>> with open("tmp.txt", "w", newline=False) as f: f.write("foo\nbar\r\n") Traceback (most recent call last): File "", line 1, in with open("tmp.txt", "w", newline=False) as f: f.write("foo\nbar\r\n") TypeError: open() argument 'newline' must be str or None, not bool From __peter__ at web.de Tue Aug 10 02:57:26 2021 From: __peter__ at web.de (Peter Otten) Date: Tue, 10 Aug 2021 08:57:26 +0200 Subject: [Tutor] Terminating "\M" characters with regard to csv.DictWriter In-Reply-To: References: Message-ID: On 10/08/2021 06:03, Alex Kleider wrote: > On Mon, Aug 9, 2021 at 1:14 AM Peter Otten <__peter__ at web.de> wrote: >> or pick a different dialect >> >> writer = csv.writer(outstream, dialect="unix") >> >> >> This got rid of the '\M' at the end of each line but created a new problem > in that each field was in quotes!! The line terminator is not the only thing controlled by a dialect. You can override the quoting spec with writer = csv.writer( outstream, dialect="unix", quoting=csv.QUOTE_MINIMAL ) or whatever quoting style you prefer. From wlfraed at ix.netcom.com Tue Aug 10 10:57:36 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Tue, 10 Aug 2021 10:57:36 -0400 Subject: [Tutor] Terminating "\M" characters with regard to csv.DictWriter References: Message-ID: On Mon, 9 Aug 2021 20:47:13 -0700, Alex Kleider declaimed the following: > >Yes, it is a capital M, here's the first line of the diff command output: >< >first,last,phone,address,town,state,postal_code,country,email,dues,dock,kayak,mooring,status^M > Ah... NOT \M but ^M; totally different critters. ^M signifies -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From alexkleider at gmail.com Tue Aug 10 11:08:26 2021 From: alexkleider at gmail.com (Alex Kleider) Date: Tue, 10 Aug 2021 08:08:26 -0700 Subject: [Tutor] Terminating "\M" characters with regard to csv.DictWriter In-Reply-To: References: Message-ID: On Mon, Aug 9, 2021 at 11:59 PM Peter Otten <__peter__ at web.de> wrote: > The line terminator is not the only thing controlled by a dialect. You > can override the quoting spec with > > writer = csv.writer( > outstream, dialect="unix", quoting=csv.QUOTE_MINIMAL > ) > > or whatever quoting style you prefer. > > Thanks, Peter, for pointing that out. There's a lot in the 'csv' module documentation that I hadn't appreciated. Cheers, Alex From alexkleider at gmail.com Tue Aug 10 11:14:34 2021 From: alexkleider at gmail.com (Alex Kleider) Date: Tue, 10 Aug 2021 08:14:34 -0700 Subject: [Tutor] Terminating "\M" characters with regard to csv.DictWriter In-Reply-To: References: Message-ID: On Tue, Aug 10, 2021 at 7:59 AM Dennis Lee Bieber wrote: > > Ah... NOT \M but ^M; totally different critters. ^M signifies > > > Yes, I finally recognized the embarrassing over site. Thanks to all who lead me down a better path! > > -- > Wulfraed Dennis Lee Bieber AF6VN > wlfraed at ix.netcom.com > http://wlfraed.microdiversity.freeddns.org/ > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From shariesk at sundaelectronics.com Tue Aug 10 14:31:30 2021 From: shariesk at sundaelectronics.com (Shari Eskenas) Date: Tue, 10 Aug 2021 11:31:30 -0700 Subject: [Tutor] Learn Python through nursery rhymes & fairy tales Message-ID: Hey everyone, I recently released my first Python book for beginners, called *A Day in Code- Python*: https://www.amazon.com/Day-Code-Python-Illustrated-Beginners/dp/1735907944 This book teaches Python basics through a story that is described with Python programs. It has a castles and dragons theme and I thought of another unique book idea to complement this. My upcoming book *Learn Python through Nursery Rhymes and Fairy Tales *translates the classic tales into Python programs. The code is explained below the programs and every program teaches one or more new Python concepts. I just launched it on Kickstarter this morning: https://www.kickstarter.com/projects/914595512/learn-python-through-nursery-rhymes-and-fairy-tales It's discounted today. You'll see some sample pages on the Kickstarter page- *Goldilocks and the Three Bears* and *The Muffin Man* as Python programs! I'd love to hear any feedback you have. Best, Shari -- Shari Eskenas Founder and CEO | Sundae Electronics LLC www.sundaelectronics.com From nzbzxx at gmail.com Thu Aug 12 03:23:40 2021 From: nzbzxx at gmail.com (nzbz xx) Date: Thu, 12 Aug 2021 15:23:40 +0800 Subject: [Tutor] Sorting a list manually Message-ID: I am trying to create a def function where I can sort a list in ascending order by starting with the first item in the list and comparing it to the number next to it. The evolution of the list [4, 3, 2, 1] would look as such: [*4*, 3, 2, 1] [3, *4*, 2, 1] [3, 2, *4*, 1] [3, 2, 1, *4*] [*3*, 2, 1, 4] [2, *3*, 1, 4] [2, 1, *3*, 4] [2, 1, 3, *4*] [*2*, 1, 3, 4] [1, *2*, 3, 4] [1, 2, *3*, 4] [1, 2, 3, *4*] [*1*, 2, 3, 4] [1, *2*, 3, 4] [1, 2, *3*, 4] [1, 2, 3, *4*] There's a total of 12 steps in the sorting above. So far, I've managed to solve the same list using the code below: def sorting_function(k): steps_count = 0 for j in range(len(k)): for i in range(len(k)-1): if k[i] > k[i+1]: k[i], k[i+1] = k[i+1], k[i] steps_count = steps_count + 1 print(k) print(steps_count) k = [4,3,2,1] sorting_function(k) And got the following output: [3, 4, 2, 1] [3, 2, 4, 1] [3, 2, 1, 4] [2, 3, 1, 4] [2, 1, 3, 4] [2, 1, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4] 12 However, when I tested the code using a different list e.g. [1, 2, 3, 4], the output is: [1, 2, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4] 12 The output should only have a total of 4 steps. What's wrong with my code? From alan.gauld at yahoo.co.uk Thu Aug 12 04:49:21 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Thu, 12 Aug 2021 09:49:21 +0100 Subject: [Tutor] Sorting a list manually In-Reply-To: References: Message-ID: On 12/08/2021 08:23, nzbz xx wrote: > I am trying to create a def function where I can sort a list in ascending > order by starting with the first item in the list and comparing it to the > number next to it. The evolution of the list [4, 3, 2, 1] would look as > such: > > [*4*, 3, 2, 1] [3, *4*, 2, 1] [3, 2, *4*, 1] [3, 2, 1, *4*] > [*3*, 2, 1, 4] [2, *3*, 1, 4] [2, 1, *3*, 4] [2, 1, 3, *4*] > [*2*, 1, 3, 4] [1, *2*, 3, 4] [1, 2, *3*, 4] [1, 2, 3, *4*] > [*1*, 2, 3, 4] [1, *2*, 3, 4] [1, 2, *3*, 4] [1, 2, 3, *4*] > > There's a total of 12 steps in the sorting above. So far, I've managed to > solve the same list using the code below: > > def sorting_function(k): > steps_count = 0 > for j in range(len(k)): > for i in range(len(k)-1): > if k[i] > k[i+1]: > k[i], k[i+1] = k[i+1], k[i] > steps_count = steps_count + 1 > print(k) > print(steps_count) > > k = [4,3,2,1] > sorting_function(k) > > And got the following output: > 12 > > However, when I tested the code using a different list e.g. [1, 2, 3, 4], > the output is: > 12 > > The output should only have a total of 4 steps. What's wrong with my code? What makes you think it should only have 4 steps? What part of your code makes it stop after 4 steps? You always complete both loops so 4x3=12 steps regardless of input. Hint: To do it in less you would need to detect when the list was sorted. One way to do so would be to mark whether the list was changed during the inner loop. If there were no changes then break out of the outer loop. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From PyTutor at DancesWithMice.info Thu Aug 12 05:49:34 2021 From: PyTutor at DancesWithMice.info (dn) Date: Thu, 12 Aug 2021 21:49:34 +1200 Subject: [Tutor] Sorting a list manually In-Reply-To: References: Message-ID: <11176d18-d508-f3ac-4f2e-c9a5f69a499a@DancesWithMice.info> On 12/08/2021 19.23, nzbz xx wrote: > I am trying to create a def function where I can sort a list in ascending > order by starting with the first item in the list and comparing it to the > number next to it. The evolution of the list [4, 3, 2, 1] would look as > such: > > [*4*, 3, 2, 1] [3, *4*, 2, 1] [3, 2, *4*, 1] [3, 2, 1, *4*] > [*3*, 2, 1, 4] [2, *3*, 1, 4] [2, 1, *3*, 4] [2, 1, 3, *4*] > [*2*, 1, 3, 4] [1, *2*, 3, 4] [1, 2, *3*, 4] [1, 2, 3, *4*] > [*1*, 2, 3, 4] [1, *2*, 3, 4] [1, 2, *3*, 4] [1, 2, 3, *4*] > > There's a total of 12 steps in the sorting above. So far, I've managed to > solve the same list using the code below: > > def sorting_function(k): > steps_count = 0 > for j in range(len(k)): > for i in range(len(k)-1): > if k[i] > k[i+1]: > k[i], k[i+1] = k[i+1], k[i] > steps_count = steps_count + 1 > print(k) > print(steps_count) > > k = [4,3,2,1] > sorting_function(k) > > And got the following output: > [3, 4, 2, 1] > [3, 2, 4, 1] > [3, 2, 1, 4] > [2, 3, 1, 4] > [2, 1, 3, 4] > [2, 1, 3, 4] > [1, 2, 3, 4] > [1, 2, 3, 4] > [1, 2, 3, 4] > [1, 2, 3, 4] > [1, 2, 3, 4] > [1, 2, 3, 4] > 12 > > However, when I tested the code using a different list e.g. [1, 2, 3, 4], > the output is: > [1, 2, 3, 4] > [1, 2, 3, 4] > [1, 2, 3, 4] > [1, 2, 3, 4] > [1, 2, 3, 4] > [1, 2, 3, 4] > [1, 2, 3, 4] > [1, 2, 3, 4] > [1, 2, 3, 4] > [1, 2, 3, 4] > [1, 2, 3, 4] > [1, 2, 3, 4] > 12 > > The output should only have a total of 4 steps. What's wrong with my code? This is a form of the "Bubble Sort". You will find plenty of coverage on the web, eg https://duckduckgo.com/?q=bubble+sort Per your marvellously-helpful illustration, looking at the first line you will notice that the effect is to move the largest value (4) out to the end of the list. How about if the data was [ 2, 3, 4, 1 ], would the 4 still make it to the end? So, can it be said that the effect of the first time through the outer-loop, is that the largest element will always be in its correct place, wrt the final sorted list? That realised, let's examine what happens the second time around the outer-loop. It will (eventually) pick up the 3 (no matter where the 3 started in the original_list) and move it to the second-last position. At that point, we know the 4 is in-position AND that the 3 has already been shown to be smaller than the 4 (and thus won't bubble/swap position with it), so what is the point of that comparison? Thus, the inner loop can be shortened by one, every time an element is moved to its final position. Perhaps something like: for i in range( len( k ) - 1 - j): This will have the effect of reducing the steps_count - slightly. With k == 4, the compute-time is negligible. If however the list were massively-long, then you may like to consider computing len( k ) - 1 only once per outer loop (cf every time). Also compute i + 1 once (cf three times per inner loop. The code to 'swap' is well written! Do you know about: steps_count += 1? Another nit-pick is that you'll notice that in order to discuss the topic sensibly, English words are being used - the meaning of i, j, and k is less meaningful! Beyond that short-cut/improvement, there's little that can be done. The algorithm must consider every element on the line (excepting as-above), ie just because two 'neighbors' don't swap early along the line, doesn't mean that the inner-loop can "break", eg [ 2, 1, 4, 3 ] The 2 and the 1 must swap, but the 2 and the 4 must not. At this point, you can't pack-up and go home, because the 4 must be compared against the 3 because that will realise a swap! So, the Bubble Sort is a beautiful introductory algorithm and one that can be 'seen' as it happens (particularly with diagrams such as yours). However, it is very laborious and thus inefficient. The way to speed-up sorting is to choose a better algorithm. (one of which is to use Python's sort() - but should we assume it may not be used?) Just tossing-around an idea: Another way to consider the bubble sort, would be to 'bubble' the smallest number to the 'left'/top (instead of the largest number to the right/bottom), then the next smallest, etc. So, using the min(), index(), and pop() or remove() list methods (functions), could one: - take the original_list as input, and create an empty destination_list - find the min() value and append() it to the destination_list - index() that value in the original_list and pop() it, or perhaps just remove() it (?speed comparison) - rinse and repeat (the latter two steps) until the original_list is "exhausted". This may realise a speed advantage over the above, because it is processing with (efficient) built-in methods, and eschewing complicated and multiple index-manipulations. Python does the work of gradually reducing the length of one list and increasing that of the output, with each pass. (apologies, no time to code it myself - if you do, will be interested to hear your conclusions) Web.Refs: List functions: https://docs.python.org/3/library/stdtypes.html?highlight=list%20method#sequence-types-list-tuple-range Python's timer function: https://docs.python.org/3/library/timeit.html -- Regards, =dn From wlfraed at ix.netcom.com Thu Aug 12 12:38:50 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Thu, 12 Aug 2021 12:38:50 -0400 Subject: [Tutor] Sorting a list manually References: Message-ID: On Thu, 12 Aug 2021 15:23:40 +0800, nzbz xx declaimed the following: >I am trying to create a def function where I can sort a list in ascending >order by starting with the first item in the list and comparing it to the >number next to it. The evolution of the list [4, 3, 2, 1] would look as >such: > >[*4*, 3, 2, 1] [3, *4*, 2, 1] [3, 2, *4*, 1] [3, 2, 1, *4*] >[*3*, 2, 1, 4] [2, *3*, 1, 4] [2, 1, *3*, 4] [2, 1, 3, *4*] >[*2*, 1, 3, 4] [1, *2*, 3, 4] [1, 2, *3*, 4] [1, 2, 3, *4*] >[*1*, 2, 3, 4] [1, *2*, 3, 4] [1, 2, *3*, 4] [1, 2, 3, *4*] > >There's a total of 12 steps in the sorting above. So far, I've managed to >solve the same list using the code below: Ancient Bubble-Sort... > >def sorting_function(k): > steps_count = 0 > for j in range(len(k)): > for i in range(len(k)-1): j runs 4 iterations (presuming a four element input list), i runs 3 iterations, 4 * 3 => 12 total passes. To reduce the passes, you have to consider that, once the first pass has completed, the largest value is at the end of the list. Subsequent passes do not need to look at that item. So the second pass only looks at list[0:3], after that pass, the two largest values have been sorted, so you now only have to look at list[0:2], etc. That's the first optimization -- but still runs a full count of passes regardless of the data. def bubble_sort(aList): steps_count = 0 for j in reversed(range(len(aList))): for i in range(j): if aList[i] > aList[i+1]: aList[i], aList[i+1] = aList[i+1], aList[i] steps_count += 1 print(aList) print(steps_count) return aList bubble_sort([4, 3, 2, 1]) bubble_sort([1, 2, 3, 4]) [3, 4, 2, 1] [3, 2, 4, 1] [3, 2, 1, 4] [2, 3, 1, 4] [2, 1, 3, 4] [1, 2, 3, 4] 6 [1, 2, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4] [1, 2, 3, 4] 6 The next optimization is to detect if a swap took place during the pass; if no swap, list is in order, exit. def bubble_sort(aList): steps_count = 0 for j in reversed(range(len(aList))): swap = False for i in range(j): if aList[i] > aList[i+1]: aList[i], aList[i+1] = aList[i+1], aList[i] swap = True steps_count += 1 print(aList) if not swap: break if not swap: break print(steps_count) return aList bubble_sort([4, 3, 2, 1]) bubble_sort([1, 2, 3, 4]) [3, 4, 2, 1] [3, 2, 4, 1] [3, 2, 1, 4] [2, 3, 1, 4] [2, 1, 3, 4] [1, 2, 3, 4] 6 [1, 2, 3, 4] 1 For a possibly cleaner version (I haven't checked if it exits early -- adding instrumentation output I'd leave up to you) look at https://www.geeksforgeeks.org/bubble-sort/ The text states that it needs one pass with no swap to know it is done -- but I don't see any such test in the solutions -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From wlfraed at ix.netcom.com Thu Aug 12 13:17:19 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Thu, 12 Aug 2021 13:17:19 -0400 Subject: [Tutor] Sorting a list manually References: <11176d18-d508-f3ac-4f2e-c9a5f69a499a@DancesWithMice.info> Message-ID: On Thu, 12 Aug 2021 21:49:34 +1200, dn via Tutor declaimed the following: > >Just tossing-around an idea: > >Another way to consider the bubble sort, would be to 'bubble' the >smallest number to the 'left'/top (instead of the largest number to the >right/bottom), then the next smallest, etc. > I vaguely recall coding a double-ended bubble sort some 40+ years ago -- it essentially moved the largest item in the active range to the end, then reversed indexing to move the smallest item to the start; then reduced the active range by one element at each end. >So, using the min(), index(), and pop() or remove() list methods >(functions), could one: >- take the original_list as input, and create an empty destination_list >- find the min() value and append() it to the destination_list >- index() that value in the original_list and pop() it, or perhaps just >remove() it (?speed comparison) >- rinse and repeat (the latter two steps) until the original_list is >"exhausted". > An "insertion sort" where all insertions are at the end of the sorted-list? Granted, using the built-ins on the input list may be faster than scanning the growing output list for the position, then splitting the list to allow inserting the element at that place. -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From go.umang108 at gmail.com Fri Aug 13 06:53:52 2021 From: go.umang108 at gmail.com (Umang Goswami) Date: Fri, 13 Aug 2021 16:23:52 +0530 Subject: [Tutor] Regarding inability of Python Module Winsound to produce beep in decimal frequency Message-ID: Hi There, Hope you find this mail in good health. I am Umang Goswami, a Python developer and student working on a huge project for automation of music instruments. I am producing the musical notes using the Beep function of Winsound Module( https://docs.python.org/3/library/winsound.html) by passing frequency as a argument to the function. Now whenever i provide frequency of any note in decimal(for example 277.1826 for C4 note) it shows following error: Traceback (most recent call last): File "C:\Users\Umang Goswami\Desktop\Umang Goswami\test.py", line 2, in winsound.Beep(111.11,111111) TypeError: integer argument expected, got float Now I have to round up the frequencies. This is hurting the quality, accuracy ,authenticity and future of the project. Almost all the notes have the frequencies in decimal parts. Rounding up means changing semitones and quatertones thus whole note itself. This problem is technically making my program useless. Its my humble request to you all, I beg you, Please tell me how to overcome this issue. I have consulted many sources both online and offline but I remained unsatisfied. I can not make audio files of each note because there are many many notes and so practically making so many files of different time length wont help. Please suggest to me the way to resolve this issue or is there any other module to produce the sound of decimal frequency. Waiting in your reply, Umang Goswami From nzbzxx at gmail.com Fri Aug 13 03:14:03 2021 From: nzbzxx at gmail.com (nzbz xx) Date: Fri, 13 Aug 2021 15:14:03 +0800 Subject: [Tutor] Replacing a value in a list Message-ID: I came across this problem: Given that -999 is a missing value in a dataset, replace this value with the mean of its adjacent values. For example [1, 10, -999, 4, 5] should yield 7. This is what i got so far. The problem is that i don't know what to do when there are consecutive -999 in the list e.g. [1, 10, -999, -999, -999] def clean_data(case): data_cleaned_count = 0 for data in range(len(case)-1): if case[data] == -999: ave = (case[data-1] + case[data+1])/2 case[data] = ave data_cleaned_count = data_cleaned_count + 1 print(case, data_cleaned_count) From alan.gauld at yahoo.co.uk Fri Aug 13 08:50:22 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Fri, 13 Aug 2021 13:50:22 +0100 Subject: [Tutor] Replacing a value in a list In-Reply-To: References: Message-ID: On 13/08/2021 08:14, nzbz xx wrote: > This is what i got so far. The problem is that i don't know what to do when > there are consecutive -999 in the list e.g. [1, 10, -999, -999, -999] This is a specification/design issue not a coding problem. You eiyther: 1) ask the client what they want you to do (and update the specification as required) 2) ask the architect for a design decision 3) make an executive decision for yourself and *document it in the code* Some possibilities are: 1) use the average value from either side to populate all the spaces eg [1,2,3,-999,-999,-999,-999,8,9] becomes: [1, 2, 3, 5.5, 5.5, 5.5, 5.5, 8, 9] 2) create a fill pattern using the average value to fill the mid points then take the average of the end point and average to fill the next points and so on. (this yields a smoother curve). eg [1,2,3,-999,-999,-999,-999,8,9] becomes: [1, 2, 3, 4.25, 5.5, 5.5, 6.75, 8, 9] But it's a lot more complex to code. But there are many other interpolation techniques, including looking at the slopes of the curve on ether side. These 2 options are just the easiest. Ideally, your client will know exactly what they want! -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From mats at wichmann.us Fri Aug 13 08:51:29 2021 From: mats at wichmann.us (Mats Wichmann) Date: Fri, 13 Aug 2021 06:51:29 -0600 Subject: [Tutor] Replacing a value in a list In-Reply-To: References: Message-ID: <7e2c791e-332f-82d6-3ddf-e678e9675467@wichmann.us> On 8/13/21 1:14 AM, nzbz xx wrote: > I came across this problem: Given that -999 is a missing value in a > dataset, replace this value with the mean of its adjacent values. For > example [1, 10, -999, 4, 5] should yield 7. > > This is what i got so far. The problem is that i don't know what to do when > there are consecutive -999 in the list e.g. [1, 10, -999, -999, -999] For the problem as stated, you probably want to raise an error. But since the problem statement doesn't say, there isn't really an "answer" we can give. It's not actually a Python problem. This is data science... in the real world, collected data may have missing points, and what you do about missing points affects your conclusions, and the choices are almost certainly colored by an understanding of the data. I think that lecture (which I'm not qualified to give anyway) probably belongs in an entirely different forum. One of the approaches is to "impute" a value for a missing element, and one way to do that is to use a mean of nearest neighbors as in your problem. Even when you've chosen that specific approach there are variances: the size of window used for nearest-neighbor may smooth too much if too large or too little if too small. If the neighbors are missing too, you would probably conclude that missing-neighbors was not a good choice for imputation. Maybe there's a different algorithm to try? And just maybe, that the dataset might be of such low quality that you can't draw any reasonable conclusions from it From wlfraed at ix.netcom.com Fri Aug 13 16:20:02 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Fri, 13 Aug 2021 16:20:02 -0400 Subject: [Tutor] Regarding inability of Python Module Winsound to produce beep in decimal frequency References: Message-ID: On Fri, 13 Aug 2021 16:23:52 +0530, Umang Goswami declaimed the following: >I am Umang Goswami, a Python developer and student working on a huge >project for automation of music instruments. I am producing the musical >notes using the Beep function of Winsound Module( >https://docs.python.org/3/library/winsound.html) by passing frequency as a >argument to the function. > "Automation" of musical instruments already exists for MIDI compatible equipment (keyboard-synthesizers and sound cards). And pitch-riders have existed for things like flutes and other winds (which produce only one note at a time). Strings are more difficult, unless played one note only. The Winsound module was never meant to be "pure" for music. It's more of an alarm feature invoked by applications to notify the use of events they may need to investigate. As documented: """ The winsound module provides access to the basic sound-playing machinery provided by Windows platforms. It includes functions and several constants. winsound.Beep(frequency, duration) Beep the PC?s speaker. The frequency parameter specifies frequency, in hertz, of the sound, and must be in the range 37 through 32,767. The duration parameter specifies the number of milliseconds the sound should last. If the system is not able to beep the speaker, RuntimeError is raised. """ If you want /music/ you should be looking at MIDI control of the computer's sound card, rather than a software (CPU intensive?) generation of a square wave. Just one library https://mido.readthedocs.io/en/latest/intro.html Of course, MIDI notes are not specified by frequency, but by "note" (0-127 I believe is the range). There /are/ "pitch-bend" controllers that allow one to blur the core note to sharp or flat, but nothing precise in terms of raw frequency. >Now whenever i provide frequency of any note in decimal(for example >277.1826 for C4 note) it shows following error: >Traceback (most recent call last): > File "C:\Users\Umang Goswami\Desktop\Umang Goswami\test.py", line 2, in > > winsound.Beep(111.11,111111) >TypeError: integer argument expected, got float > >Now I have to round up the frequencies. This is hurting the quality, I wouldn't "round UP" the frequencies... Round to nearest integer would be better (round nearest: 277 vs round up: 278). You might want to peruse https://wiki.python.org/moin/PythonInMusic as it lists more packages than I want to explore. There are plenty for playing/recording WAV data, plenty of MIDI, etc. -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From nzbzxx at gmail.com Sat Aug 14 00:23:15 2021 From: nzbzxx at gmail.com (nzbz xx) Date: Sat, 14 Aug 2021 12:23:15 +0800 Subject: [Tutor] Replacing a value in a list In-Reply-To: <7e2c791e-332f-82d6-3ddf-e678e9675467@wichmann.us> References: <7e2c791e-332f-82d6-3ddf-e678e9675467@wichmann.us> Message-ID: Assuming that when there are consecutive missing values, they should be replaced with adjacent valid values e.g [1,2,-999,-999,5] should give [1,2,2,5,5]. And given [1,-999,-999,-999,5], the middle missing value would take the average of index 1 & 3. This should get an output of [1, 2 , 2, 3.5, 5]. How should it be coded for it to solve from the outer elements first to the inner elements? On Fri, Aug 13, 2021 at 8:52 PM Mats Wichmann wrote: > On 8/13/21 1:14 AM, nzbz xx wrote: > > I came across this problem: Given that -999 is a missing value in a > > dataset, replace this value with the mean of its adjacent values. For > > example [1, 10, -999, 4, 5] should yield 7. > > > > This is what i got so far. The problem is that i don't know what to do > when > > there are consecutive -999 in the list e.g. [1, 10, -999, -999, -999] > > For the problem as stated, you probably want to raise an error. But > since the problem statement doesn't say, there isn't really an "answer" > we can give. It's not actually a Python problem. > > This is data science... in the real world, collected data may have > missing points, and what you do about missing points affects your > conclusions, and the choices are almost certainly colored by an > understanding of the data. I think that lecture (which I'm not > qualified to give anyway) probably belongs in an entirely different > forum. One of the approaches is to "impute" a value for a missing > element, and one way to do that is to use a mean of nearest neighbors as > in your problem. Even when you've chosen that specific approach there > are variances: the size of window used for nearest-neighbor may smooth > too much if too large or too little if too small. If the neighbors are > missing too, you would probably conclude that missing-neighbors was not > a good choice for imputation. Maybe there's a different algorithm to > try? And just maybe, that the dataset might be of such low quality that > you can't draw any reasonable conclusions from it > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From alan.gauld at yahoo.co.uk Sat Aug 14 08:09:25 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sat, 14 Aug 2021 13:09:25 +0100 Subject: [Tutor] Replacing a value in a list In-Reply-To: References: <7e2c791e-332f-82d6-3ddf-e678e9675467@wichmann.us> Message-ID: On 14/08/2021 05:23, nzbz xx wrote: > Assuming that when there are consecutive missing values, they should be > replaced with adjacent valid values e.g [1,2,-999,-999,5] should give > [1,2,2,5,5]. And given [1,-999,-999,-999,5], the middle missing value would > take the average of index 1 & 3. This should get an output of [1, 2 , 2, > 3.5, 5]. How should it be coded for it to solve from the outer elements > first to the inner elements? That still leaves the question of what happens when there are 4 or 5 blank pieces? For 4 you can duplicate the average, but what about 5? Is there a point at which ou decide the data is too damaged to continue and throw an error? As for coding it I'd write a small helper function to find the start/stop indices of the blanks (or the start and length if you prefer) Something like: def findBlanks(seq,blank=-999): start = seq.index(blank) end = start+1 while seq[end] == blank: end+=1 return start,end For the simple case of 3 blanks you can do if start != 0: seq[start] = seq[start-1] if end != len(seq)-1: end = seq[end+1] gapsize = end-start if gapsize >= 3: seq[start+1]=(seq[start]+seq[end])/2 Now what you do if end-start>3 is up to you. And what you do if the gap is at the start or end of seq is also up to you... It's all in the specification. What is supposed to happen? Once you know the complete algorithm the code should practically write itself. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From nzbzxx at gmail.com Mon Aug 16 13:15:19 2021 From: nzbzxx at gmail.com (nzbz xx) Date: Tue, 17 Aug 2021 01:15:19 +0800 Subject: [Tutor] Replacing a value in a list In-Reply-To: References: <7e2c791e-332f-82d6-3ddf-e678e9675467@wichmann.us> Message-ID: I have the codes as such now but it doesn't yield the same number of input and output e.g. given the raw data is [-999,-999, 3, 4, -999 ], the output generated is [3, 3, 4]. I'm not sure where the problem lies when I've already done a for-loop and it should give me the same number of elements in the list. def clean_data(dataset): data_cleaned_count = 0 clean_list = [] # Identifying all valid data in the dataset for data in range(len(dataset)): if dataset[data] != -999: clean_list.append(dataset[data]) # Replacing missing data for md in range(len(dataset)): if dataset[md] == -999: consecutive_invalid_count = 0 for i in range(len(dataset)-1): if dataset[i] == dataset[i+1]: consecutive_invalid_count +=1 start = dataset.index(-999) end = start + consecutive_invalid_count left_idx = start-1 # Finding the adjacent valid data right_idx = end + 1 if abs(md - left_idx) > abs(md - right_idx) or md >= len(dataset)-1: # Locating the nearest valid data clean_list.insert(md,dataset[left_idx] ) data_cleaned_count += 1 if abs(md - left_idx) < abs(md - right_idx) or md == 0: clean_list.insert(md, dataset[right_idx]) data_cleaned_count += 1 On Sat, Aug 14, 2021 at 8:10 PM Alan Gauld via Tutor wrote: > On 14/08/2021 05:23, nzbz xx wrote: > > Assuming that when there are consecutive missing values, they should be > > replaced with adjacent valid values e.g [1,2,-999,-999,5] should give > > [1,2,2,5,5]. And given [1,-999,-999,-999,5], the middle missing value > would > > take the average of index 1 & 3. This should get an output of [1, 2 , 2, > > 3.5, 5]. How should it be coded for it to solve from the outer elements > > first to the inner elements? > > That still leaves the question of what happens when there are 4 or 5 > blank pieces? For 4 you can duplicate the average, but what about 5? > Is there a point at which ou decide the data is too damaged to continue > and throw an error? > > As for coding it I'd write a small helper function to find the > start/stop indices of the blanks (or the start and length if you prefer) > > Something like: > > def findBlanks(seq,blank=-999): > start = seq.index(blank) > end = start+1 > while seq[end] == blank: > end+=1 > return start,end > > For the simple case of 3 blanks you can do > > if start != 0: > seq[start] = seq[start-1] > if end != len(seq)-1: > end = seq[end+1] > > gapsize = end-start > if gapsize >= 3: > seq[start+1]=(seq[start]+seq[end])/2 > > Now what you do if end-start>3 is up to you. > And what you do if the gap is at the start or > end of seq is also up to you... > > It's all in the specification. > What is supposed to happen? Once you know the > complete algorithm the code should practically > write itself. > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.amazon.com/author/alan_gauld > Follow my photo-blog on Flickr at: > http://www.flickr.com/photos/alangauldphotos > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From mebalts at gmail.com Mon Aug 16 11:54:29 2021 From: mebalts at gmail.com (Mebale Tsige Araya) Date: Mon, 16 Aug 2021 08:54:29 -0700 Subject: [Tutor] Request for help Message-ID: Hi, Is there anybody who can help me, please? Can I run Python codes on Browser, please? If so could you have any idea how to run my codes on Browser? -- Best Regards, Mebale Tsige Araya Addis Ababa, Ethiopia From deepakdixit0001 at gmail.com Mon Aug 16 15:22:51 2021 From: deepakdixit0001 at gmail.com (Deepak Dixit) Date: Tue, 17 Aug 2021 00:52:51 +0530 Subject: [Tutor] Request for help In-Reply-To: References: Message-ID: If you are asking to generate outputs using Python then yes, there are many frameworks for easy development like Flask, Django etc but if you want to replace JavaScript with Python then I don't think so. What exactly are you looking for? On Tue, Aug 17, 2021 at 12:45 AM Mebale Tsige Araya wrote: > Hi, > > Is there anybody who can help me, please? Can I run Python codes on > Browser, please? If so could you have any idea how to run my codes on > Browser? > > -- > Best Regards, > Mebale Tsige Araya > Addis Ababa, Ethiopia > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > -- *With Regards,* *Deepak Kumar Dixit* From alan.gauld at yahoo.co.uk Mon Aug 16 15:28:06 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Mon, 16 Aug 2021 20:28:06 +0100 Subject: [Tutor] Replacing a value in a list In-Reply-To: References: <7e2c791e-332f-82d6-3ddf-e678e9675467@wichmann.us> Message-ID: On 16/08/2021 18:15, nzbz xx wrote: > I have the codes as such now but it doesn't yield the same number of input > and output e.g. given the raw data is [-999,-999, 3, 4, -999 ], the output > generated is [3, 3, 4]. I'm not sure where the problem lies when I've > already done a for-loop and it should give me the same number of elements > in the list. > > def clean_data(dataset): > > data_cleaned_count = 0 > clean_list = [] > > # Identifying all valid data in the dataset > for data in range(len(dataset)): > if dataset[data] != -999: > clean_list.append(dataset[data]) First, you shouldn't need the index. Just use the data items directly: for data in dataset: if data != -999: clean_list.append(data) Much more pythonic. However its also wrong! Because it builds clean list without any elements where the -999 occurs. You need the exception processing embedded inside the loop. And you probably need to use enumerate() to get the index for accessing the earlier or later elements. Something like: for index, item in enumerate(dataset): if item != -999: clean_list.append(item) else: # count markers n = 1 while dataset[index+n] = -999: n += 1 if n == 1: # do something with item if n == 2: do another thing etc... clean_list.append(item) > # Replacing missing data > for md in range(len(dataset)): > if dataset[md] == -999: > > consecutive_invalid_count = 0 > for i in range(len(dataset)-1): > if dataset[i] == dataset[i+1]: > consecutive_invalid_count +=1 This loop looks wrong to me, although I haven't worked through it in detail. But it looks like you are iterating over most of dataset each time? This will count adjacent equal values not just marker values? > start = dataset.index(-999) > end = start + consecutive_invalid_count > > left_idx = start-1 # Finding the adjacent valid data > right_idx = end + 1 > > if abs(md - left_idx) > abs(md - right_idx) or md >= > len(dataset)-1: # Locating the nearest valid data > clean_list.insert(md,dataset[left_idx] ) > data_cleaned_count += 1 > > if abs(md - left_idx) < abs(md - right_idx) or md == 0: > clean_list.insert(md, dataset[right_idx]) > data_cleaned_count += 1 > > On Sat, Aug 14, 2021 at 8:10 PM Alan Gauld via Tutor > wrote: > >> On 14/08/2021 05:23, nzbz xx wrote: >>> Assuming that when there are consecutive missing values, they should be >>> replaced with adjacent valid values e.g [1,2,-999,-999,5] should give >>> [1,2,2,5,5]. And given [1,-999,-999,-999,5], the middle missing value >> would >>> take the average of index 1 & 3. This should get an output of [1, 2 , 2, >>> 3.5, 5]. How should it be coded for it to solve from the outer elements >>> first to the inner elements? >> >> That still leaves the question of what happens when there are 4 or 5 >> blank pieces? For 4 you can duplicate the average, but what about 5? >> Is there a point at which ou decide the data is too damaged to continue >> and throw an error? >> >> As for coding it I'd write a small helper function to find the >> start/stop indices of the blanks (or the start and length if you prefer) >> >> Something like: >> >> def findBlanks(seq,blank=-999): >> start = seq.index(blank) >> end = start+1 >> while seq[end] == blank: >> end+=1 >> return start,end >> >> For the simple case of 3 blanks you can do >> >> if start != 0: >> seq[start] = seq[start-1] >> if end != len(seq)-1: >> end = seq[end+1] >> >> gapsize = end-start >> if gapsize >= 3: >> seq[start+1]=(seq[start]+seq[end])/2 >> >> Now what you do if end-start>3 is up to you. >> And what you do if the gap is at the start or >> end of seq is also up to you... >> >> It's all in the specification. >> What is supposed to happen? Once you know the >> complete algorithm the code should practically >> write itself. >> >> -- >> Alan G >> Author of the Learn to Program web site >> http://www.alan-g.me.uk/ >> http://www.amazon.com/author/alan_gauld >> Follow my photo-blog on Flickr at: >> http://www.flickr.com/photos/alangauldphotos >> >> >> _______________________________________________ >> Tutor maillist - Tutor at python.org >> To unsubscribe or change subscription options: >> https://mail.python.org/mailman/listinfo/tutor >> > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Mon Aug 16 15:39:14 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Mon, 16 Aug 2021 20:39:14 +0100 Subject: [Tutor] Request for help In-Reply-To: References: Message-ID: On 16/08/2021 16:54, Mebale Tsige Araya wrote: > Is there anybody who can help me, please? Can I run Python codes on > Browser, please? If so could you have any idea how to run my codes on > Browser? Your question could mean several different things. I'll give very brief answers to each below but you will need to give more specific information for more specific answers. 1) Can I embed Python code in an HTML page and have the browser execute it like JavaScript. No. But you can get Python to JavaScript translators or languages that look like Python (CoffeeScript) that are converted to JavaScript by the server. There are at least 2 very old browsers that do run Python internally but they are used by nobody in the realm world! 2) Is there a web site that lets me enter python code without Python being installed on my device? Yes. There are a few sites that present a virtual terminal online that lets you code python inside a browser. The code actually runs on the server but the input and results appear in the browser. The caveat is that they usually have limited support for modules and they don't allow you to install third party add-ons beyond what the site owner has installed (numpy may be there for example) 3) Can I write web site code in Python that will generate web pages that can be seen in a browser? Yes, there are many, many web frameworks for python from the super sophisticated (eg Zope) to substantial but practical for humans (eg Django) to simple ones for quick and dirty developments (eg Flask) And you can use the cgi module in the standard library for very basic web development. If that's not enough or you meant something different then repost with more details, perhaps even describing what you want to do with it. HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From PyTutor at DancesWithMice.info Mon Aug 16 15:39:36 2021 From: PyTutor at DancesWithMice.info (dn) Date: Tue, 17 Aug 2021 07:39:36 +1200 Subject: [Tutor] Request for help In-Reply-To: References: Message-ID: <6623f557-8721-420b-37c3-ffa234418de2@DancesWithMice.info> On 17/08/2021 03.54, Mebale Tsige Araya wrote: > Hi, > > Is there anybody who can help me, please? Can I run Python codes on > Browser, please? If so could you have any idea how to run my codes on > Browser? Do you mean that you are unable to run Python in/on your own computer and would like to be able to use your web-browser to access a Python environment 'in the cloud'? You may like to try Python Anywhere: https://www.pythonanywhere.com/ If you are learning Python, there is a very helpful on-line tool at http://www.pythontutor.com/ Does this answer address the actual need? Please respond with an example of what you'd like to achieve... -- Regards, =dn From jankenin at gmx.de Tue Aug 17 12:37:22 2021 From: jankenin at gmx.de (Jan Kenin) Date: Tue, 17 Aug 2021 18:37:22 +0200 Subject: [Tutor] exec('a=1') in functions Message-ID: <8a8a7795-e6a7-763e-833f-a39883226dba@gmx.de> Hello, in python 2.7 the following code works: >>> def f( b ): ...???? exec( 'a=%s'%b ) ...???? print 'a=', a ... >>> f( 1 ) a= 1 in python3.6 this works in the interpreter: >>> b=1 >>> exec('a=%s'%b) >>> a 1 but not in a function: >>> def f( b ): ...???? exec( 'a=%s'%b ) ...???? print( 'a=', a ) ... >>> f( 1 ) Traceback (most recent call last): ? File "", line 1, in ? File "", line 3, in f NameError: name 'a' is not defined How can I get the latter working? Thanks for all advice! Jan From manpritsinghece at gmail.com Wed Aug 18 00:13:53 2021 From: manpritsinghece at gmail.com (Manprit Singh) Date: Wed, 18 Aug 2021 09:43:53 +0530 Subject: [Tutor] Need to discuss about a class that can be used to print number of vowels, consonants, uppercase and lowercase letters and spaces in a string Message-ID: Dear sir, I have written below given program using a class that will print number of vowels, consonants, uppercase and lowercase letters and spaces in a string, provided string contains only alphabets and space class StringStats: def __init__(self): self.str_seq = None self.uppercase = None self.lowercase = None self.vowel = None self.consonants = None self.spaces = None def set_string(self, seq): self.str_seq = seq def count_uppercase(self): self.uppercase = sum(ch.isupper() for ch in self.str_seq) def count_lowercase(self): self.lowercase = sum(ch.islower() for ch in self.str_seq) def count_vowels(self): self.vowel = sum(ch in "AEIOUaeoiu" for ch in self.str_seq) def count_consonants(self): self.consonants = sum(ch not in "AEIOUaeoiu " for ch in self.str_seq) def count_space(self): self.spaces = self.str_seq.count(" ") def compute_stats(self): self.count_uppercase() self.count_lowercase() self.count_vowels() self.count_consonants() self.count_space() def show_stats(self): print("String is-", self.str_seq) print("Count of vowels-", self.vowel) print("Count of consonants-", self.consonants) print("Count of uppercase letters-", self.uppercase) print("Count of lowercase letters", self.lowercase) print("Count of spaces", self.spaces) s_seq = "Amar Singh" st1 = StringStats() st1.set_string(s_seq) st1.compute_stats() st1.show_stats() Gives the answer as below : String is- Amar Singh Count of vowels- 3 Count of consonants- 6 Count of uppercase letters- 2 Count of lowercase letters 7 Count of spaces 1 Just need your comments on this class against following points : 1) The way i have initialized all instance variables inside __init__(), later on assigning values to each instance variable in different methods . 2) I have made different methods to set each instance variable, can't all these instance variables be set in a single method ? 3) Any further improvement in the class Regards Manprit Singh From PyTutor at DancesWithMice.info Wed Aug 18 01:10:02 2021 From: PyTutor at DancesWithMice.info (dn) Date: Wed, 18 Aug 2021 17:10:02 +1200 Subject: [Tutor] Need to discuss about a class that can be used to print number of vowels, consonants, uppercase and lowercase letters and spaces in a string In-Reply-To: References: Message-ID: On 18/08/2021 16.13, Manprit Singh wrote: > Dear sir, > > I have written below given program using a class that will print number of > vowels, consonants, uppercase and lowercase letters and spaces in a > string, provided string contains only alphabets and space > > class StringStats: > def __init__(self): > self.str_seq = None > self.uppercase = None > self.lowercase = None > self.vowel = None > self.consonants = None > self.spaces = None > > def set_string(self, seq): > self.str_seq = seq > > def count_uppercase(self): > self.uppercase = sum(ch.isupper() for ch in self.str_seq) > > def count_lowercase(self): > self.lowercase = sum(ch.islower() for ch in self.str_seq) > > def count_vowels(self): > self.vowel = sum(ch in "AEIOUaeoiu" for ch in self.str_seq) > > def count_consonants(self): > self.consonants = sum(ch not in "AEIOUaeoiu " for ch in > self.str_seq) > > def count_space(self): > self.spaces = self.str_seq.count(" ") > > def compute_stats(self): > self.count_uppercase() > self.count_lowercase() > self.count_vowels() > self.count_consonants() > self.count_space() > > def show_stats(self): > print("String is-", self.str_seq) > print("Count of vowels-", self.vowel) > print("Count of consonants-", self.consonants) > print("Count of uppercase letters-", self.uppercase) > print("Count of lowercase letters", self.lowercase) > print("Count of spaces", self.spaces) > > s_seq = "Amar Singh" > st1 = StringStats() > st1.set_string(s_seq) > st1.compute_stats() > st1.show_stats() > > Gives the answer as below : > > String is- Amar Singh > Count of vowels- 3 > Count of consonants- 6 > Count of uppercase letters- 2 > Count of lowercase letters 7 > Count of spaces 1 > > Just need your comments on this class against following points : > > 1) The way i have initialized all instance variables inside __init__(), later > on assigning values to each instance variable in different methods . You didn't explain why... What is the purpose of an instantiated object before it is given a source-string? If none, why not pass the source to __init__() and save a step/an extra method? > 2) I have made different methods to set each instance variable, can't all > these instance variables be set in a single method ? Yes. Again, you didn't explain why... What is the reason for separate calculation, cf computing them all at once ... and even in __init__()? > 3) Any further improvement in the class In attempting to answer the above, I assumed that the class could be used to selectively count vowels, consonants, or whatever - despite the example call(s) methodically executing each in-turn. However, show_stats() blindly assumes that all of the counts have been computed. Aside from the lack of clarity in the class's purpose, the whole thing seems 'over-done' and probably twice as long as it needs to be. Would a shorter form be less readable - don't think so, but... Critique: 1 I locate I/O outside of classes. In this case, if I wanted to apply the class within a French application/user-group, I would have to disregard the method or sub-class and over-ride. Given that any post-processing of these counts would see direct-access to st1.vowel, st1.consonants, etc, why not regard the same mechanism as appropriate for printing? 2 I was going to criticise the attribute-names as failing to indicate that they are 'counts'. However, the purpose of the class is that, so maybe not. That said, does the class-name meet PEP-008 conventions? Similarly, the fact that "vowels" and "consonants" have been expressed in the plural form, but the other counters have not, is an inconsistency that is likely to cause (unnecessary) difficulties. 3 Assuming the printing method is necessary, and that you wish to keep the separate computation methods, and especially if users are expected to use some counts (but not others), then consider using @property. 4 Contrarily, if all of the computations will always be carried-out, consider the number of loops coded, and whether it might be better/faster to count them all within a single loop. 5 Question: should a class which basically only has one purpose - to take a string and print 'the' five statistics; really be a (single) function? 6 I'm curious about a series of sub-counts without a reference-point - the number of characters in the source-string. 7 I prefer to shift 'constants', eg "AEIOUaeoiu"; out of the body of the code and up to the top of the module. This makes them easier to modify *when* the need arises (again, think use in another language). -- Regards, =dn From __peter__ at web.de Wed Aug 18 03:11:01 2021 From: __peter__ at web.de (Peter Otten) Date: Wed, 18 Aug 2021 09:11:01 +0200 Subject: [Tutor] exec('a=1') in functions In-Reply-To: <8a8a7795-e6a7-763e-833f-a39883226dba@gmx.de> References: <8a8a7795-e6a7-763e-833f-a39883226dba@gmx.de> Message-ID: On 17/08/2021 18:37, Jan Kenin wrote: > Hello, > > in python 2.7 the following code works: > >>> def f( b ): > ...???? exec( 'a=%s'%b ) > ...???? print 'a=', a > ... > >>> f( 1 ) > a= 1 > > in python3.6 this works in the interpreter: > >>> b=1 > >>> exec('a=%s'%b) > >>> a > 1 > > but not in a function: > >>> def f( b ): > ...???? exec( 'a=%s'%b ) > ...???? print( 'a=', a ) > ... > >>> f( 1 ) > Traceback (most recent call last): > ? File "", line 1, in > ? File "", line 3, in f > NameError: name 'a' is not defined > > How can I get the latter working? You can't. In Python 2 the exec statement modified the byte code, with sometimes surprising consequences. In Python 3 exec() is an ordinary function; you have to copy data from its namespace into the function explicitly. Example: >>> def f(b): ns = dict(b=b) exec("a = b", ns) a = ns["a"] print(f"{a=}") >>> f(42) a=42 Less magic, but much cleaner ;) From alan.gauld at yahoo.co.uk Wed Aug 18 05:04:59 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Wed, 18 Aug 2021 10:04:59 +0100 Subject: [Tutor] Need to discuss about a class that can be used to print number of vowels, consonants, uppercase and lowercase letters and spaces in a string In-Reply-To: References: Message-ID: On 18/08/2021 05:13, Manprit Singh wrote: > Dear sir, > > I have written below given program using a class that will print number of > vowels, consonants, uppercase and lowercase letters and spaces in a > string, provided string contains only alphabets and space In general its better to keep printing separate from calculation. You have done that by creating a separate method but it would be better still to simply create a __str__() method and you can then just print the object. This males the class more usable in different contexts - such as a GUI or Web application. > class StringStats: > def __init__(self): > self.str_seq = None > self.uppercase = None > self.lowercase = None > self.vowel = None > self.consonants = None > self.spaces = None These initializations are not type compatible with the expected values. It would be better to use an empty string and zeros. Otherwise your new instance will be incompatible with code that uses it - imagine an app with a list of StringStats objects and that extracts all the values for calculation. Any object which has not had a string assigned and an explicit call to compute() will return Nones instead of numbers. Why not add the seq to the constructor and then analyse it immediately? class StringStats: def __init__(self), seq="") self.seq = seq self.compute_stats() That does everything your method does and more. You can still use the seq assignment directly if you need to change the string later. > def set_string(self, seq): > self.str_seq = seq > > def count_uppercase(self): > self.uppercase = sum(ch.isupper() for ch in self.str_seq) > > def count_lowercase(self): > self.lowercase = sum(ch.islower() for ch in self.str_seq) > > def count_vowels(self): > self.vowel = sum(ch in "AEIOUaeoiu" for ch in self.str_seq) > > def count_consonants(self): > self.consonants = sum(ch not in "AEIOUaeoiu " for ch in > self.str_seq) > > def count_space(self): > self.spaces = self.str_seq.count(" ") Given that you need to loop over the entire string for each method why not just have one method that sets all the counters in one pass of the string? The extra work over just setting one is minimal. > def compute_stats(self): > self.count_uppercase() > self.count_lowercase() > self.count_vowels() > self.count_consonants() > self.count_space() So this would become: def compute_stats(self): for ch in self.seq: if ch in string.vowels self.vowels =+=1 elif ch.islower(): self.lower += 1 elif ch.isupper(): self.upper == 1 etc. > def show_stats(self): > print("String is-", self.str_seq) > print("Count of vowels-", self.vowel) > print("Count of consonants-", self.consonants) > print("Count of uppercase letters-", self.uppercase) > print("Count of lowercase letters", self.lowercase) > print("Count of spaces", self.spaces) And this could become your __str__() method, returning the string instead of printing it. So the modified class creates neater client code: > s_seq = "Amar Singh" > st1 = StringStats() > st1.set_string(s_seq) > st1.compute_stats() > st1.show_stats() Becomes s_seq = "Amar Singh" st1 = StringStats(s_seq) print(st1) > Just need your comments on this class against following points : > > 1) The way i have initialized all instance variables inside __init__(), later > on assigning values to each instance variable in different methods . See above. The use of inconsistent types and the deferred assignment of the string leaves the class in an inconsistent state compared to its peers. This could cause issues in the processing of collections of Stringstat objects. At the very least it will require extra checks to see if None is returned. Much better to initialize the string in the init as a parameter. > 2) I have made different methods to set each instance variable, can't all > these instance variables be set in a single method ? Yes, it would be better that way. it seems very unlikely that a user would want to only initialise one statistic. > 3) Any further improvement in the class Does it need to be a class at all? This could just be a function that takes a string and returns a dict or tuple of results. It's the old rule that a class that only has an init() plus one method is really a function. (getter/setters and _str_() don't really count as methods in that context) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Wed Aug 18 05:09:54 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Wed, 18 Aug 2021 10:09:54 +0100 Subject: [Tutor] exec('a=1') in functions In-Reply-To: <8a8a7795-e6a7-763e-833f-a39883226dba@gmx.de> References: <8a8a7795-e6a7-763e-833f-a39883226dba@gmx.de> Message-ID: On 17/08/2021 17:37, Jan Kenin wrote: > in python3.6 this works in the interpreter: > >>> b=1 > >>> exec('a=%s'%b) > >>> a > 1 > > but not in a function: > How can I get the latter working? Peter has already explained the changes in v3. But I assume you fully understand the dangers of using exec() in real code? It is an advanced technique that should only be used as a last resort. Certainly not for the trivial example shown, but even in more complex cases there is nearly always a way to code it directly rather than using exec(). I apologize if this is patronising but the tutor list is read by many beginners and they may not realize what dangerous tools exec/eval etc are. So the warning is almost obligatory whenever they get mentioned. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From breamoreboy at gmail.com Tue Aug 17 20:52:32 2021 From: breamoreboy at gmail.com (Mark Lawrence) Date: Wed, 18 Aug 2021 01:52:32 +0100 Subject: [Tutor] exec('a=1') in functions In-Reply-To: <8a8a7795-e6a7-763e-833f-a39883226dba@gmx.de> References: <8a8a7795-e6a7-763e-833f-a39883226dba@gmx.de> Message-ID: On 17/08/2021 17:37, Jan Kenin wrote: > Hello, > > in python 2.7 the following code works: > >>> def f( b ): > ...???? exec( 'a=%s'%b ) > ...???? print 'a=', a > ... > >>> f( 1 ) > a= 1 > > in python3.6 this works in the interpreter: > >>> b=1 > >>> exec('a=%s'%b) > >>> a > 1 > > but not in a function: > >>> def f( b ): > ...???? exec( 'a=%s'%b ) > ...???? print( 'a=', a ) > ... > >>> f( 1 ) > Traceback (most recent call last): > ? File "", line 1, in > ? File "", line 3, in f > NameError: name 'a' is not defined > > How can I get the latter working? > > Thanks for all advice! > Jan Quite frankly I've no idea, but even if I did know I'd advise you not to use it. See e.g. https://stackoverflow.com/questions/1933451/why-should-exec-and-eval-be-avoided -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From julkhami at gmail.com Wed Aug 18 14:39:42 2021 From: julkhami at gmail.com (Julius Hamilton) Date: Wed, 18 Aug 2021 20:39:42 +0200 Subject: [Tutor] Installing Python 2 with Python 3 already installed Message-ID: I would like to run the tool YouTube-DL (in Ubuntu 20.04). After installing, when I try to run it, it says "python: no such file or directory". I have python3 installed and working. I assume the tool is written with Python 2. I tried to install Python 2 with "apt-get install python", and got this response: Some packages could not be installed. This may mean that you have requested an impossible situation or if you are using the unstable distribution that some required packages have not yet been created or been moved out of Incoming. The following information may help to resolve the situation: The following packages have unmet dependencies: python : PreDepends: python-minimal (= 2.7.16-1) but it is not going to be installed Depends: libpython-stdlib (= 2.7.16-1) but it is not going to be installed Depends: python2 (= 2.7.16-1) but 2.7.17-2ubuntu4 is to be installed E: Unable to correct problems, you have held broken packages. How can I install the necessary version of Python in my situation and make it available to YouTube-DL? Thanks very much, Julius From mats at wichmann.us Wed Aug 18 18:50:35 2021 From: mats at wichmann.us (Mats Wichmann) Date: Wed, 18 Aug 2021 16:50:35 -0600 Subject: [Tutor] Installing Python 2 with Python 3 already installed In-Reply-To: References: Message-ID: <6a93c30c-5863-7505-53ff-db6fc16d49e4@wichmann.us> On 8/18/21 12:39 PM, Julius Hamilton wrote: > I would like to run the tool YouTube-DL (in Ubuntu 20.04). After > installing, when I try to run it, it says "python: no such file or > directory". > > I have python3 installed and working. I assume the tool is written with > Python 2. That's not it: https://packages.ubuntu.com/focal/youtube-dl It's clearly listed as a Python 3 package. From alan.gauld at yahoo.co.uk Wed Aug 18 18:56:29 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Wed, 18 Aug 2021 23:56:29 +0100 Subject: [Tutor] Installing Python 2 with Python 3 already installed In-Reply-To: References: Message-ID: On 18/08/2021 19:39, Julius Hamilton wrote: > I would like to run the tool YouTube-DL (in Ubuntu 20.04). After > installing, when I try to run it, it says "python: no such file or > directory". How exactly ae you running it? A GUI icon? A terminal command line? Something else? If a command line show us the exact command you type and the exact error - cut n paste from the terminal please. > I tried to install Python 2 with "apt-get install python", You probably need to specify the exact python version. On modern linux boxes python tends to mean python 3. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From cs at cskk.id.au Wed Aug 18 19:27:59 2021 From: cs at cskk.id.au (Cameron Simpson) Date: Thu, 19 Aug 2021 09:27:59 +1000 Subject: [Tutor] Installing Python 2 with Python 3 already installed In-Reply-To: References: Message-ID: On 18Aug2021 20:39, Julius Hamilton wrote: >I would like to run the tool YouTube-DL (in Ubuntu 20.04). After >installing, when I try to run it, it says "python: no such file or >directory". > >I have python3 installed and working. I assume the tool is written with >Python 2. Nay. It is Python 3. (Maybe it works in Python 2 also, but I use it with Python 3.) I recommend installing it on a personal basis, using pip: pip3 install --user youtube-dl That installs as you and does not need playing guessing games with the system python packages. Cheers, Cameron Simpson From cs at cskk.id.au Wed Aug 18 19:36:16 2021 From: cs at cskk.id.au (Cameron Simpson) Date: Thu, 19 Aug 2021 09:36:16 +1000 Subject: [Tutor] Installing Python 2 with Python 3 already installed In-Reply-To: References: Message-ID: On 19Aug2021 09:27, Cameron Simpson wrote: >I recommend installing it on a personal basis, using pip: > > pip3 install --user youtube-dl > >That installs as you and does not need playing guessing games with the >system python packages. Bear in mind that that requires installing the "python3-pip" package first. But then you're good to install youtube-dl as yourself. Cheers, Cameron Simpson From manpritsinghece at gmail.com Thu Aug 19 10:54:19 2021 From: manpritsinghece at gmail.com (Manprit Singh) Date: Thu, 19 Aug 2021 20:24:19 +0530 Subject: [Tutor] Need to discuss about a class that can be used to print number of vowels, consonants, uppercase and lowercase letters and spaces in a string In-Reply-To: References: Message-ID: Dear Sir, Based on the guidance given by all of you I have rewritten the class . Now the problem is a little bit modified. The class should print only number of spaces, uppercase letters and lowercase letters .The class is given below class StringStats: def __init__(self, seq = ""): self.str_seq = seq self.cnt_lowercase = 0 self.cnt_space= 0 self.cnt_uppercase = 0 self.compute_stats() def compute_stats(self): for ch in self.str_seq: if ch == " ": self.cnt_space += 1 elif ch.isupper(): self.cnt_uppercase += 1 elif ch.islower(): self.cnt_lowercase += 1 def __str__(self): return (f"String is- {self.str_seq}\n" f"Count of uppercaseletters-{self.cnt_uppercase}\n" f"Count of lowercaseletters-{self.cnt_lowercase}\n" f"Count of spaces -{self.cnt_space}") s_seq = "Amar Singhji " st1 = StringStats(s_seq) print(st1) Gives the desired output as follows : String is- Amar Singhji Count of uppercaseletters-2 Count of lowercaseletters-9 Count of spaces -2 Still i have following questions : !) How can i place self.compute_stats() inside __init__ without placing all those instance variables that are initialized to 0. 2) The way I have used f strings inside __str__ for multiple lines is correct ? On Wed, Aug 18, 2021 at 2:35 PM Alan Gauld via Tutor wrote: > On 18/08/2021 05:13, Manprit Singh wrote: > > Dear sir, > > > > I have written below given program using a class that will print number > of > > vowels, consonants, uppercase and lowercase letters and spaces in a > > string, provided string contains only alphabets and space > > In general its better to keep printing separate from calculation. > You have done that by creating a separate method but it would be > better still to simply create a __str__() method and you can > then just print the object. This males the class more usable > in different contexts - such as a GUI or Web application. > > > class StringStats: > > def __init__(self): > > self.str_seq = None > > self.uppercase = None > > self.lowercase = None > > self.vowel = None > > self.consonants = None > > self.spaces = None > > These initializations are not type compatible with the expected > values. It would be better to use an empty string and zeros. > Otherwise your new instance will be incompatible with code > that uses it - imagine an app with a list of StringStats > objects and that extracts all the values for calculation. > Any object which has not had a string assigned and an explicit > call to compute() will return Nones instead of numbers. > > Why not add the seq to the constructor and then analyse > it immediately? > > class StringStats: > def __init__(self), seq="") > self.seq = seq > self.compute_stats() > > That does everything your method does and more. > You can still use the seq assignment directly if you > need to change the string later. > > > def set_string(self, seq): > > self.str_seq = seq > > > > def count_uppercase(self): > > self.uppercase = sum(ch.isupper() for ch in self.str_seq) > > > > def count_lowercase(self): > > self.lowercase = sum(ch.islower() for ch in self.str_seq) > > > > def count_vowels(self): > > self.vowel = sum(ch in "AEIOUaeoiu" for ch in self.str_seq) > > > > def count_consonants(self): > > self.consonants = sum(ch not in "AEIOUaeoiu " for ch in > > self.str_seq) > > > > def count_space(self): > > self.spaces = self.str_seq.count(" ") > > Given that you need to loop over the entire string for each > method why not just have one method that sets all the counters > in one pass of the string? The extra work over just setting one > is minimal. > > > def compute_stats(self): > > self.count_uppercase() > > self.count_lowercase() > > self.count_vowels() > > self.count_consonants() > > self.count_space() > > So this would become: > > def compute_stats(self): > for ch in self.seq: > if ch in string.vowels > self.vowels =+=1 > elif ch.islower(): > self.lower += 1 > elif ch.isupper(): > self.upper == 1 > etc. > > > def show_stats(self): > > print("String is-", self.str_seq) > > print("Count of vowels-", self.vowel) > > print("Count of consonants-", self.consonants) > > print("Count of uppercase letters-", self.uppercase) > > print("Count of lowercase letters", self.lowercase) > > print("Count of spaces", self.spaces) > > And this could become your __str__() method, returning > the string instead of printing it. > > So the modified class creates neater client code: > > > s_seq = "Amar Singh" > > st1 = StringStats() > > st1.set_string(s_seq) > > st1.compute_stats() > > st1.show_stats() > > Becomes > > s_seq = "Amar Singh" > st1 = StringStats(s_seq) > print(st1) > > > Just need your comments on this class against following points : > > > > 1) The way i have initialized all instance variables inside __init__(), > later > > on assigning values to each instance variable in different methods . > > See above. The use of inconsistent types and the deferred assignment of > the string leaves the class in an inconsistent state compared to its > peers. This could cause issues in the processing of collections of > Stringstat objects. At the very least it will require extra checks > to see if None is returned. > > Much better to initialize the string in the init as a parameter. > > > 2) I have made different methods to set each instance variable, can't all > > these instance variables be set in a single method ? > > Yes, it would be better that way. it seems very unlikely that > a user would want to only initialise one statistic. > > > 3) Any further improvement in the class > Does it need to be a class at all? > This could just be a function that takes a string and returns > a dict or tuple of results. > > It's the old rule that a class that only has an init() plus > one method is really a function. (getter/setters and _str_() > don't really count as methods in that context) > > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.amazon.com/author/alan_gauld > Follow my photo-blog on Flickr at: > http://www.flickr.com/photos/alangauldphotos > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From rmlibre at riseup.net Thu Aug 19 11:55:32 2021 From: rmlibre at riseup.net (rmlibre at riseup.net) Date: Thu, 19 Aug 2021 08:55:32 -0700 Subject: [Tutor] Need to discuss about a class that can be used to print number of vowels, consonants, uppercase and lowercase letters and spaces in a string In-Reply-To: References: Message-ID: I'd create sets of each category because they have handy methods for doing inclusion / exclusion, plus they were designed to be very efficient at checking for contained values. This will be faster than checking `char.isupper()` etc. for every letter and for each category. That sounds like O(c*l) complexity, which means the running time would scale with the product of both the number of categories & the length of the string. I agree with the other responder that properties would be helpful here, since you can do the calculations only when they are needed. This also means the calculations will need to be done every time the value is queried. For some problems, you might want to only do calculations when they are needed, but you also only want to do them once if they are needed multiple times. In that case, the built-in functools module has an lru_cache decorator which handles caching calculated method values. I'd also use the Counter class from the built-in collections module. It will take quick care of giving the raw counts of each character (even the one's you haven't prepared for e.g. punctuation, control chars). The built-in string module also seems helpful here. If you can find built-ins which have already solved some of the problem you're needing to solve, it's a good idea to try finding them -- they're there to make your life easier. StringStats seems like a reasonable name. Even though abbreviations, or otherwise shortened words, are generally discouraged by PEP8 and other style guides in names, "stats" is a common enough shortening IMHO. However, you may also consider StringAnalysis. The method names could also be more clear. If the name of the class is changed to StringAnalysis, then it becomes more evident that each method should indicate which kind of value is going to be returned, if any. In your case, a character count. Names are important for many reasons. A name like CharacterCounter could also work, and may obviate the role of each method. It's up to you, that's why they're called style guides. Resources for Naming: (https://www.youtube.com/watch?v=5cafjDPPtJ0) (https://www.youtube.com/watch?v=n0Ak6xtVXno) Finally, and kind of restating a previous point, your class will struggle to compute over very large strings. As well, saving the string in the instance, while necessary in some cases, should be avoided if isn't for a good reason. Every time a string is moved, passed as an argument or stored in a variable, then the string is copied over to this new location. This can lead to lots of simultaneous copies of the same data during the runtime of a program. If the string is very large, then this issue can become a big problem in memory overhead. Below I'm posting an example of how I might write a class to get this job done. Consider it MIT licensed. Best, rmlibre import string from collections import Counter from functools import lru_cache # See also https://docs.python.org/3/library/functools.html#functools.cached_property class StringAnalysis: """ Creates objects which calculate the number of varying kinds of characters in a given string. Usage example: string = "some string to be analyzed" string_analysis = StringAnalysis(string) print(string_analysis.lowercase_count) 22 print(string_analysis.whitespace_count) 4 """ _LETTERS = frozenset(string.ascii_letters) _LOWERCASE = frozenset(string.ascii_lowercase) _UPPERCASE = frozenset(string.ascii_uppercase) _WHITESPACE = frozenset(string.whitespace) _VOWELS = frozenset("AEIOUaeiou") _CONSONANTS = _LETTERS.difference(_VOWELS) def __init__(self, string: str): """ Saves memory by keeping only the count of each letter within the instance, avoiding the copying of the entire string. Analysis is done within the instance properties. """ self._raw_count = Counter(string) @property @lru_cache() def vowel_count(self): vowels = self._VOWELS.intersection(self._raw_count) return sum(self._raw_count[char] for char in vowels) @property @lru_cache() def consonant_count(self): consonants = self._CONSONANTS.intersection(self._raw_count) return sum(self._raw_count[char] for char in consonants) @property @lru_cache() def uppercase_count(self): uppercase = self._UPPERCASE.intersection(self._raw_count) return sum(self._raw_count[char] for char in uppercase) @property @lru_cache() def lowercase_count(self): lowercase = self._LOWERCASE.intersection(self._raw_count) return sum(self._raw_count[char] for char in lowercase) @property @lru_cache() def whitespace_count(self): whitespace = self._WHITESPACE.intersection(self._raw_count) return sum(self._raw_count[char] for char in whitespace) On 2021-08-18 09:05, tutor-request at python.org wrote: > Send Tutor mailing list submissions to > tutor at python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/tutor > or, via email, send a message with subject or body 'help' to > tutor-request at python.org > > You can reach the person managing the list at > tutor-owner at python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Tutor digest..." > > Today's Topics: > > 1. exec('a=1') in functions (Jan Kenin) > 2. Need to discuss about a class that can be used to print > number of vowels, consonants, uppercase and lowercase letters and > spaces in a string (Manprit Singh) > 3. Re: Need to discuss about a class that can be used to print > number of vowels, consonants, uppercase and lowercase letters and > spaces in a string (dn) > 4. Re: exec('a=1') in functions (Peter Otten) > 5. Re: Need to discuss about a class that can be used to print > number of vowels, consonants, uppercase and lowercase letters and > spaces in a string (Alan Gauld) > > _______________________________________________ > Tutor maillist - Tutor at python.org > https://mail.python.org/mailman/listinfo/tutor From alan.gauld at yahoo.co.uk Thu Aug 19 13:08:15 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Thu, 19 Aug 2021 18:08:15 +0100 Subject: [Tutor] Need to discuss about a class that can be used to print number of vowels, consonants, uppercase and lowercase letters and spaces in a string In-Reply-To: References: Message-ID: On 19/08/2021 15:54, Manprit Singh wrote: > class StringStats: > def __init__(self, seq = ""): > self.str_seq = seq > self.cnt_lowercase = 0 > self.cnt_space= 0 > self.cnt_uppercase = 0 > self.compute_stats() > > def compute_stats(self): > for ch in self.str_seq: > if ch == " ": > self.cnt_space += 1 > elif ch.isupper(): > self.cnt_uppercase += 1 > elif ch.islower(): > self.cnt_lowercase += 1 > !) How can i place self.compute_stats() inside __init__ without > placing all those instance variables that are initialized to 0. You could put them into the compute_stats() method. You need to set them to zero there anyway so you might as well just initialize them there every time. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From PyTutor at DancesWithMice.info Thu Aug 19 17:44:05 2021 From: PyTutor at DancesWithMice.info (dn) Date: Fri, 20 Aug 2021 09:44:05 +1200 Subject: [Tutor] Need to discuss about a class that can be used to print number of vowels, consonants, uppercase and lowercase letters and spaces in a string In-Reply-To: References: Message-ID: <4fd30310-3ca9-766f-c1cd-47723aebf74b@DancesWithMice.info> On 20/08/2021 05.08, Alan Gauld via Tutor wrote: > On 19/08/2021 15:54, Manprit Singh wrote: > >> class StringStats: >> def __init__(self, seq = ""): >> self.str_seq = seq >> self.cnt_lowercase = 0 >> self.cnt_space= 0 >> self.cnt_uppercase = 0 >> self.compute_stats() >> >> def compute_stats(self): >> for ch in self.str_seq: >> if ch == " ": >> self.cnt_space += 1 >> elif ch.isupper(): >> self.cnt_uppercase += 1 >> elif ch.islower(): >> self.cnt_lowercase += 1 > >> !) How can i place self.compute_stats() inside __init__ without >> placing all those instance variables that are initialized to 0. > > You could put them into the compute_stats() method. > You need to set them to zero there anyway so you might > as well just initialize them there every time. +1, keep things 'together' and keep usage(s) 'local' and 'close'. Isn't there a convention/habit (or perhaps has come-across from another language where it is a requirement) that all data-attributes should either be defined in the class or __init__()? (I realise that in Python we (usually) initialise value, which (inherently) defines the attribute's type, rather than considering them two potentially-separate steps) I remember being 'dinged' about this by some 'checker' facility. Regret unable to reference which one. Perhaps (only) relevant to @dataclasses? However, why not combine both suggestions? Conflate __init__() and compute_stats() - thus the consideration 'goes away'! (at which time, the class vs function decision will come into stark focus...) To the OP: should you decide to include any of this advice in any of your articles, please remember, per PSF code, to pass credit. Especially as could also mention @Alan's course materials to further benefit the reader. Conversely, I would like to propose the original code (suitably anonymised) as a group-coding (refactoring/TDD) exercise, for future use at a local (learners) Python Users' Group coding evening. -- Regards, =dn From alan.gauld at yahoo.co.uk Thu Aug 19 20:23:06 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Fri, 20 Aug 2021 01:23:06 +0100 Subject: [Tutor] Need to discuss about a class that can be used to print number of vowels, consonants, uppercase and lowercase letters and spaces in a string In-Reply-To: <4fd30310-3ca9-766f-c1cd-47723aebf74b@DancesWithMice.info> References: <4fd30310-3ca9-766f-c1cd-47723aebf74b@DancesWithMice.info> Message-ID: On 19/08/2021 22:44, dn via Tutor wrote: > +1, keep things 'together' and keep usage(s) 'local' and 'close'. > > Isn't there a convention/habit (or perhaps has come-across from another > language where it is a requirement) that all data-attributes should > either be defined in the class or __init__()? This is the dichotomy with this issue. Normally I'd advocate initializing in the init(). But in this case the variables need to be reinitialized every time compute() is called so you might as well do it once there and call compute() from init(). > However, why not combine both suggestions? Conflate __init__() and > compute_stats() - thus the consideration 'goes away'! If you mean put the compute() code into init that raises the problem of not being able to recompute the stats if the string changes. OTOH strings are immutable in python so for consistency perhaps we should require a new StringStats instance? But if we do that we are even more strongly arguing that Stringstats is really a function rather than a true class... > (at which time, the class vs function decision will come into stark > focus...) Just so... If the class only has an init() and a __str__() is it really a class? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From mats at wichmann.us Thu Aug 19 20:25:23 2021 From: mats at wichmann.us (Mats Wichmann) Date: Thu, 19 Aug 2021 18:25:23 -0600 Subject: [Tutor] Need to discuss about a class that can be used to print number of vowels, consonants, uppercase and lowercase letters and spaces in a string In-Reply-To: References: <4fd30310-3ca9-766f-c1cd-47723aebf74b@DancesWithMice.info> Message-ID: On 8/19/21 6:23 PM, Alan Gauld via Tutor wrote: > Just so... If the class only has an init() and a __str__() > is it really a class? > but it might be... a @dataclass :) From alan.gauld at yahoo.co.uk Fri Aug 20 03:49:39 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Fri, 20 Aug 2021 08:49:39 +0100 Subject: [Tutor] Need to discuss about a class that can be used to print number of vowels, consonants, uppercase and lowercase letters and spaces in a string In-Reply-To: References: <4fd30310-3ca9-766f-c1cd-47723aebf74b@DancesWithMice.info> Message-ID: On 20/08/2021 01:25, Mats Wichmann wrote: > On 8/19/21 6:23 PM, Alan Gauld via Tutor wrote: > >> Just so... If the class only has an init() and a __str__() >> is it really a class? >> > > but it might be... a @dataclass :) True, but I don't consider these to be real classes, it's just substituting for Python's lack of a Record or Structure class. It's one of the big lacks in Python and traditionally filled by named tuples or dicts for one-off instances or a method-free class for collections. But it's all a bit of a kludge in my opinion and encourages abuse of OOP concepts. People claim that they have an OOP program because they have some classes but in fact they only have data structures. OTOH it would be difficult to add a major new keyword and structure into the language at this late date that didn't break loads of existing code. How many programs out there have 'record' or 'struct' or 'data' or similar as variable names?! -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From __peter__ at web.de Fri Aug 20 08:09:15 2021 From: __peter__ at web.de (Peter Otten) Date: Fri, 20 Aug 2021 14:09:15 +0200 Subject: [Tutor] Need to discuss about a class that can be used to print number of vowels, consonants, uppercase and lowercase letters and spaces in a string In-Reply-To: <4fd30310-3ca9-766f-c1cd-47723aebf74b@DancesWithMice.info> References: <4fd30310-3ca9-766f-c1cd-47723aebf74b@DancesWithMice.info> Message-ID: On 19/08/2021 23:44, dn via Tutor wrote: > language where it is a requirement) that all data-attributes should > either be defined in the class or __init__()? > > (I realise that in Python we (usually) initialise value, which > (inherently) defines the attribute's type, rather than considering them > two potentially-separate steps) > > I remember being 'dinged' about this by some 'checker' facility. Regret > unable to reference which one. pylint, probably: PS C:\Users\Peter> type demo.py class Stuff: def __init__(self): self.x = 0 def update(self): self.y = 0 PS C:\Users\Petery> pylint demo.py ************* Module demo ... demo.py:5:8: W0201: Attribute 'y' defined outside __init__ (attribute-defined-outside-init) ... ------------------------------------ Your code has been rated at -4.00/10 But it's hard to take a checker that awards minus four out of ten points seriously:) From PyTutor at DancesWithMice.info Fri Aug 20 19:06:40 2021 From: PyTutor at DancesWithMice.info (dn) Date: Sat, 21 Aug 2021 11:06:40 +1200 Subject: [Tutor] Need to discuss about a class that can be used to print number of vowels, consonants, uppercase and lowercase letters and spaces in a string In-Reply-To: References: <4fd30310-3ca9-766f-c1cd-47723aebf74b@DancesWithMice.info> Message-ID: On 21/08/2021 00.09, Peter Otten wrote: > On 19/08/2021 23:44, dn via Tutor wrote: > >> language where it is a requirement) that all data-attributes should >> either be defined in the class or __init__()? >> >> (I realise that in Python we (usually) initialise value, which >> (inherently) defines the attribute's type, rather than considering them >> two potentially-separate steps) >> >> I remember being 'dinged' about this by some 'checker' facility. Regret >> unable to reference which one. > > pylint, probably: > > PS C:\Users\Peter> type demo.py > class Stuff: > ??? def __init__(self): > ??????? self.x = 0 > ??? def update(self): > ??????? self.y = 0 > PS C:\Users\Petery> pylint demo.py > ************* Module demo > ... > demo.py:5:8: W0201: Attribute 'y' defined outside __init__ > (attribute-defined-outside-init) > ... > ------------------------------------ > Your code has been rated at -4.00/10 Thanks Peter, yes that looks vaguely-familiar! > But it's? hard to take a checker that awards minus four out of ten > points seriously:) In which case, there's a treat in-store for you! Somewhat following in the tradition of Monty Python and 'clever' British comedy, may I introduce you to a TV show called "QI" (Quite Interesting). Seldom does any of the four panellists manage to achieve a positive-score - and even less frequently can anyone explain or justify the scores anyway:- ?Quite Interesting - or 'QI' to its friends - could loosely be described as a comedy panel quiz. However, none of the stellar line-up of comedians is expected to be able to answer any questions, and if anyone ends up with a positive score, they can be very happy with their performance. Points are awarded for being interesting or funny (and, very occasionally, right) but points are deducted for answers which merely repeat common misconceptions and urban myth. (Alan Davies has turned this aspect of the game into somewhat of an artform.) It's okay to be wrong, but don't be obviously, boringly wrong. In this way, QI tries to rid the world of the flotsam of nonsense and old wives' tales that can build up in your mind. QI not only makes us look more closely at things, it encourages us to question all the received wisdom we have carried with us since childhood. Think of the program as a humorous cranial de-scaler. QI isn't really about pointless information, or shoring up vast banks of trivia, It's about finding undiscovered connections and seeing hidden patterns, just like the best comedy. After all, curiosity is hardwired in all of us; we just lose the ability to indulge it. "The lust of the mind", Thomas Hobbes called it, "that exceedeth the short vehemence of any carnal pleasure". There you have it, and from a philosopher not a press release. QI: better than sex. ... ? http://old.qi.com/tv/ Strangely-enough, the current or up-to-date web coverage is decidedly boring and lacking in humor:- ?Our flagship show, QI (Quite Interesting), is a BBC comedy panel quiz with impossible questions and points for being interesting. Each series looks at a different letter of the alphabet. ... ? https://qi.com/shows/qi -- Regards, =dn From learn2program at gmail.com Fri Aug 20 19:59:37 2021 From: learn2program at gmail.com (Alan Gauld) Date: Sat, 21 Aug 2021 00:59:37 +0100 Subject: [Tutor] Need to discuss about a class that can be used to print number of vowels, consonants, uppercase and lowercase letters and spaces in a string In-Reply-To: References: <4fd30310-3ca9-766f-c1cd-47723aebf74b@DancesWithMice.info> Message-ID: On 21/08/2021 00:06, dn via Tutor wrote: > > ?Our flagship show, QI (Quite Interesting), is a BBC comedy panel quiz > with impossible questions and points for being interesting. Each series > looks at a different letter of the alphabet. > ... > ? > https://qi.com/shows/qi +1 for QI ;-) There is also a board game of the show for those rainy days at home with the family... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From manpritsinghece at gmail.com Sun Aug 22 00:40:05 2021 From: manpritsinghece at gmail.com (Manprit Singh) Date: Sun, 22 Aug 2021 10:10:05 +0530 Subject: [Tutor] Concept of multithreading Message-ID: Dear sir, As far as I have gone through the literature available, i got to know that multithreading is used for concurrent subprocesses, Although they are not truly concurrent (appears to be concurrent) - like if i have two subprocesses, i can process them in a concurrent way using multithreading module(An Example -in that way if CPU is idle for one subprocess, second subprocess can take place at that time, This would same total execution time). For Example : import time import threading def sumoflist(seq): sum_no = 0 for num in seq: sum_no += num time.sleep(6) print("Sum of list",sum_no) def sumoftuple(seq): sum_no = 0 for num in seq: sum_no += num time.sleep(3) print("Sum of tuple",sum_no) Calling these two functions in a sequential manner will take a total time approx. 9 seconds, as delay of 6 second and 3 seconds is introduced in the functions. Now using threading module, with following code: t1 = threading.Thread(target=sumoflist, args=([1, 2, 3],)) t2 = threading.Thread(target=sumoftuple, args=((2, 4, 3),)) t1.start() t2.start() t1.join() t2.join() It will take less time, because one of the thread may utilize the CPU idle time of another thread . Is my understanding correct ? This can be treated as a very basic example of multithreading ? Although the threading module is used for IO bound processes. Regards Manprit Singh sumoflist([1, 2, 3]) sumoftuple((2, 4, 3)) From phillor9 at gmail.com Sun Aug 22 03:25:10 2021 From: phillor9 at gmail.com (Phil) Date: Sun, 22 Aug 2021 17:25:10 +1000 Subject: [Tutor] A question about None Message-ID: The distance sensor outputs "None" if the distance is greater than can be measured and "None", of course cannot be compared. So I came up with the following which is not quite correct: while True ??? if distance_sensor.get_distance_cm() is not None: ??????? if distance_sensor.get_distance_cm() > 20: ??????????? motor_pair.set_default_speed(20) ??????????? motor_pair.start() ??????? else: ??????????? continue ??? elif distance_sensor.get_distance_cm() is not None: ??????? if distance_sensor.get_distance_cm() < 20: ??????????? motor_pair.set_default_speed(-20) ??????????? motor_pair.start() ??????? else: ??????????? continue The idea is that if "None" returned from the sensor then nothing will happen and the loop will continue until "None" is not returned. Can anyone see the error of my ways? -- Regards, Phil From alan.gauld at yahoo.co.uk Sun Aug 22 04:22:13 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sun, 22 Aug 2021 09:22:13 +0100 Subject: [Tutor] A question about None In-Reply-To: References: Message-ID: On 22/08/2021 08:25, Phil wrote: > The distance sensor outputs "None" if the distance is greater than can > be measured and "None", of course cannot be compared. So I came up with > the following which is not quite correct: > > while True > ??? if distance_sensor.get_distance_cm() is not None: > ??????? if distance_sensor.get_distance_cm() > 20: > ??????????? motor_pair.set_default_speed(20) > ??????????? motor_pair.start() > ??????? else: > ??????????? continue > > ??? elif distance_sensor.get_distance_cm() is not None: > ??????? if distance_sensor.get_distance_cm() < 20: > ??????????? motor_pair.set_default_speed(-20) > ??????????? motor_pair.start() > ??????? else: > ??????????? continue > > The idea is that if "None" returned from the sensor then nothing will > happen and the loop will continue until "None" is not returned. Can > anyone see the error of my ways? The logic semsoverly complex, why not something like: while True: value = distance_sensor.get_distance_cm() if not value: continue # back to the while if value > 20: ... elif value < 20: ... if value == 20: ???? That seems to express more clearly what you stated in the text. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From PyTutor at DancesWithMice.info Sun Aug 22 04:25:42 2021 From: PyTutor at DancesWithMice.info (dn) Date: Sun, 22 Aug 2021 20:25:42 +1200 Subject: [Tutor] A question about None In-Reply-To: References: Message-ID: On 22/08/2021 19.25, Phil wrote: > The distance sensor outputs "None" if the distance is greater than can > be measured and "None", of course cannot be compared. So I came up with > the following which is not quite correct: > > while True > ??? if distance_sensor.get_distance_cm() is not None: > ??????? if distance_sensor.get_distance_cm() > 20: > ??????????? motor_pair.set_default_speed(20) > ??????????? motor_pair.start() > ??????? else: > ??????????? continue > > ??? elif distance_sensor.get_distance_cm() is not None: > ??????? if distance_sensor.get_distance_cm() < 20: > ??????????? motor_pair.set_default_speed(-20) > ??????????? motor_pair.start() > ??????? else: > ??????????? continue > > The idea is that if "None" returned from the sensor then nothing will > happen and the loop will continue until "None" is not returned. Can > anyone see the error of my ways? What if the distance is 20(cm)? while True measure = distance_sensor.get_distance_cm() if measure is not None: if measure > 20: motor_pair.set_default_speed(20) else: motor_pair.set_default_speed(-20) motor_pair.start() Assuming: - correct understanding of >, <, or = - start() applies in either case and as long as the sensor returned a value - consider adding a sleep() to the loop, depending upon the necessary frequency of sensing -- Regards, =dn From PyTutor at DancesWithMice.info Sun Aug 22 04:29:02 2021 From: PyTutor at DancesWithMice.info (dn) Date: Sun, 22 Aug 2021 20:29:02 +1200 Subject: [Tutor] OT: [was] A question about None In-Reply-To: References: Message-ID: <970f729c-290f-3a00-f6b0-1d6169a09883@DancesWithMice.info> On 22/08/2021 20.22, Alan Gauld via Tutor wrote: > while True: > value = distance_sensor.get_distance_cm() > if not value: continue # back to the while > > if value > 20: > ... > elif value < 20: > ... > if value == 20: > ???? Have cold and fuzzy-head. Do you (more clearly) recall... back in the good?bad, old days, wasn't there (a language with) an if-statement that went something like: if expression is positive, or zero, or negative: goto positive-branch, zero-branch, negative-branch -- Regards, =dn From PyTutor at DancesWithMice.info Sun Aug 22 04:39:48 2021 From: PyTutor at DancesWithMice.info (dn) Date: Sun, 22 Aug 2021 20:39:48 +1200 Subject: [Tutor] A question about None In-Reply-To: References: Message-ID: <2c3e7d2a-485e-bc0d-db4f-965f799bf8bf@DancesWithMice.info> On 22/08/2021 20.25, dn via Tutor wrote: > On 22/08/2021 19.25, Phil wrote: >> The distance sensor outputs "None" if the distance is greater than can >> be measured and "None", of course cannot be compared. So I came up with >> the following which is not quite correct: >> >> while True >> ??? if distance_sensor.get_distance_cm() is not None: >> ??????? if distance_sensor.get_distance_cm() > 20: >> ??????????? motor_pair.set_default_speed(20) >> ??????????? motor_pair.start() >> ??????? else: >> ??????????? continue >> >> ??? elif distance_sensor.get_distance_cm() is not None: >> ??????? if distance_sensor.get_distance_cm() < 20: >> ??????????? motor_pair.set_default_speed(-20) >> ??????????? motor_pair.start() >> ??????? else: >> ??????????? continue >> >> The idea is that if "None" returned from the sensor then nothing will >> happen and the loop will continue until "None" is not returned. Can >> anyone see the error of my ways? > > > What if the distance is 20(cm)? while True measure = distance_sensor.get_distance_cm() if measure is not None: if measure > 20: motor_pair.set_default_speed(20) else: motor_pair.set_default_speed(-20) motor_pair.start() > > while True > measure = distance_sensor.get_distance_cm() > if measure is not None: > if measure > 20: > motor_pair.set_default_speed(20) > else: > motor_pair.set_default_speed(-20) > motor_pair.start() > > Assuming: > - correct understanding of >, <, or = > - start() applies in either case and as long as the sensor returned a value > - consider adding a sleep() to the loop, depending upon the necessary > frequency of sensing Further thoughts:- 1 improve documentation, and 2 remove 'magic values' which are embedded/hidden within the code forwards = 1 backwards = -1 speed = 20 significant_distance = 20 ... while True measure = distance_sensor.get_distance_cm() if measure is not None: if measure > significant_distance: direction = forwards else: direction = backwards motor_pair.set_default_speed( speed * forwards ) motor_pair.start() Am wondering why 'speed' is not an argument to start()? The if-condition needs closer analysis, and could be compressed into a single-line (if meaning remains clear) Also, could replace the outer if-construct with a try-except. Ultimately, this might give you the opportunity to count the number of consecutive zero-responses from the sensor, in order to take appropriate action. -- Regards, =dn From phillor9 at gmail.com Sun Aug 22 05:46:27 2021 From: phillor9 at gmail.com (Phil) Date: Sun, 22 Aug 2021 19:46:27 +1000 Subject: [Tutor] A question about None In-Reply-To: References: Message-ID: <4fb96dbb-47f9-6420-794d-4c3224c5a646@gmail.com> On 22/8/21 6:22 pm, Alan Gauld via Tutor wrote: > The logic semsoverly complex, why not something like: > while True: > value = distance_sensor.get_distance_cm() > if not value: continue # back to the while > > if value > 20: > ... > elif value < 20: > ... > if value == 20: > ???? Thank you Alan, Mark and dn for your replies. I thought having two continue statements was not a good idea but I persisted with that idea. The simplified code offered is a far better approach and I should be able to sort this out now. Internet speed at my current location is only in the order of 100 bits per second so we'll see if? this reply leaves here. -- Regards, Phil From __peter__ at web.de Sun Aug 22 05:54:02 2021 From: __peter__ at web.de (Peter Otten) Date: Sun, 22 Aug 2021 11:54:02 +0200 Subject: [Tutor] A question about None In-Reply-To: References: Message-ID: On 22/08/2021 10:22, Alan Gauld via Tutor wrote: > On 22/08/2021 08:25, Phil wrote: >> The distance sensor outputs "None" if the distance is greater than can >> be measured and "None", of course cannot be compared. So I came up with >> the following which is not quite correct: >> >> while True >> ??? if distance_sensor.get_distance_cm() is not None: >> ??????? if distance_sensor.get_distance_cm() > 20: >> ??????????? motor_pair.set_default_speed(20) >> ??????????? motor_pair.start() >> ??????? else: >> ??????????? continue >> >> ??? elif distance_sensor.get_distance_cm() is not None: >> ??????? if distance_sensor.get_distance_cm() < 20: >> ??????????? motor_pair.set_default_speed(-20) >> ??????????? motor_pair.start() >> ??????? else: >> ??????????? continue >> >> The idea is that if "None" returned from the sensor then nothing will >> happen and the loop will continue until "None" is not returned. Can >> anyone see the error of my ways? > > > The logic semsoverly complex, why not something like: > > while True: > value = distance_sensor.get_distance_cm() > if not value: continue # back to the while That conflates "far enough away" with "already hit the obstacle" ;) > > if value > 20: > ... > elif value < 20: > ... > if value == 20: > ???? > > That seems to express more clearly what you stated in the text. Phil, you should break even this small sequence of statements into functions. Example: while True: if obstacle_ahead(): drive_backwards() else: drive_forward() Here's a spoiler for def obstacle_ahead(): dist = distance_sensor.get_distance_cm() return dist is not None and dist < 20 The function returns True if you are less than 20 cm from an obstacle. Suppose we're at 21 cm from an obstacle. No obstacle ahead, so we're going forward, say two cm until the next measurement. Now there is an obstacle, let's reverse gears and go backwards -- until we're back at 21 or 22 cm when there is no more obstacle and we can go forward again. Effectively the vehicle oscillates around a distance of 20 cm of the first obstacle it encounters. No fun to watch! Therefore I recommend that you devise a more interesting strategy before you resume coding. Giving the steps expressive names like in my example should help you understand what will happen without even implementing those steps. From learn2program at gmail.com Sun Aug 22 07:31:37 2021 From: learn2program at gmail.com (Alan Gauld) Date: Sun, 22 Aug 2021 12:31:37 +0100 Subject: [Tutor] OT: [was] A question about None In-Reply-To: <970f729c-290f-3a00-f6b0-1d6169a09883@DancesWithMice.info> References: <970f729c-290f-3a00-f6b0-1d6169a09883@DancesWithMice.info> Message-ID: <924cf21e-5cfd-e2eb-84e5-fdb3bb99e3a4@yahoo.co.uk> On 22/08/2021 09:29, dn via Tutor wrote: > back in the good?bad, old days, wasn't there (a language with) an > if-statement that went something like: > > if expression is positive, or zero, or negative: > goto positive-branch, zero-branch, negative-branch There are a few I can think of. Perhaps the most obvious example today is SQL with its DECODE function. DECODE(Value, if1, then1,if2,then2,...,ifN,thenN,else) It's somewhat like a case or switch statement wrapped up in a function. > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > http://www.amazon.com/author/alan_gauld > Follow my photo-blog on Flickr at: > http://www.flickr.com/photos/alangauldphotos > From breamoreboy at gmail.com Sun Aug 22 03:59:24 2021 From: breamoreboy at gmail.com (Mark Lawrence) Date: Sun, 22 Aug 2021 08:59:24 +0100 Subject: [Tutor] A question about None In-Reply-To: References: Message-ID: On 22/08/2021 08:25, Phil wrote: > The distance sensor outputs "None" if the distance is greater than can > be measured and "None", of course cannot be compared. So I came up with > the following which is not quite correct: > > while True Tut tut not your actual code, while True: :) > ??? if distance_sensor.get_distance_cm() is not None: > ??????? if distance_sensor.get_distance_cm() > 20: > ??????????? motor_pair.set_default_speed(20) > ??????????? motor_pair.start() > ??????? else: > ??????????? continue > > ??? elif distance_sensor.get_distance_cm() is not None: > ??????? if distance_sensor.get_distance_cm() < 20: > ??????????? motor_pair.set_default_speed(-20) > ??????????? motor_pair.start() > ??????? else: > ??????????? continue > > The idea is that if "None" returned from the sensor then nothing will > happen and the loop will continue until "None" is not returned. Can > anyone see the error of my ways? > I think that you're over complicating it. With your code the subsequent calls to get_distance_cm could return None so the comparison would fail anyway. How about:- while True : distance = distance_sensor.get_distance_cm() if distance is not None: if distance > 20: speed = 20 else: speed = -20 motor_pair.set_default_speed(speed) motor_pair.start() break There are other ways to structure the loop using flags but IMHO it's six of one, half a dozen of the other. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From alan.gauld at yahoo.co.uk Sun Aug 22 07:51:17 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sun, 22 Aug 2021 12:51:17 +0100 Subject: [Tutor] A question about None In-Reply-To: References: Message-ID: On 22/08/2021 08:25, Phil wrote: > The distance sensor outputs "None" if the distance is greater than can > be measured and "None", of course cannot be compared. So I came up with > the following which is not quite correct: > > while True > ??? if distance_sensor.get_distance_cm() is not None: > ??????? if distance_sensor.get_distance_cm() > 20: > ??????????? motor_pair.set_default_speed(20) > ??????????? motor_pair.start() > ??????? else: > ??????????? continue > > ??? elif distance_sensor.get_distance_cm() is not None: > ??????? if distance_sensor.get_distance_cm() < 20: > ??????????? motor_pair.set_default_speed(-20) > ??????????? motor_pair.start() > ??????? else: > ??????????? continue Thee is another problem with the code above that nobody has pointed out yet (although all the alternatives avoid it). When reading from a dynamic data source such as a sensor it is very important to process all possible values at the same time - ie. before reading the sensor again. To illustrate why, consider what happens if the sensor alternates between None and a value >20. In the code above the first if test fails so no action is taken for the None result, that's OK. But then, in the elif clause, we take a second reading which is >20. Then in the if clause we take a third reading which will be None so no action gets taken. But what about the reading >20? It has been ignored.... That's another reason to read the value into a variable then use that fixed value for all tests. It may seem an unlikely or arcane condition but these things happen very often when you start interacting with the real world. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From wlfraed at ix.netcom.com Sun Aug 22 13:31:55 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Sun, 22 Aug 2021 13:31:55 -0400 Subject: [Tutor] Concept of multithreading References: Message-ID: On Sun, 22 Aug 2021 10:10:05 +0530, Manprit Singh declaimed the following: >As far as I have gone through the literature available, i got to know that >multithreading is used for concurrent subprocesses, Although they are not >truly concurrent (appears to be concurrent) - like if i have two >subprocesses, i can process them in a concurrent way using multithreading >module(An Example -in that way if CPU is idle for one subprocess, second >subprocess can take place at that time, This would same total execution >time). That partly depends upon the hardware processor and the OS capabilities... Most common processors these days are multi-core (and possibly multi-threaded [hyper-threads]). My ancient Dell machine has a hyper-threaded quad-core SMP-type processor. To the OS it appears as 8-processors. EACH processor can be running in parallel, and if the code fits into the internal cache of the processor, there is no contention for memory access. If you use the MULTIPROCESSING module you achieve, effectively, true concurrency. The threading module, in contrast, does have serialization (for the common C-language Python) concerns -- that is due to the Global Interpreter Lock (GIL), which only allows one thread at a time to be modifying Python internal structures. Some implementations do not have the GIL (Jython, for example, probably doesn't). The odds are also good, that since the code is in a single "process", it runs on a single processor, and hence has to time-slice regardless of the GIL. Threads do work well when the threads spend most of their time waiting for some sort of signal (I/O completion -- including Queue, semaphores, mutex/locks). The entire Python program could be in WAIT states, and the OS can give the processor to something completely different. But this is, really, getting into concerns that aren't really Python -- Python just implements a form of threading/concurrency (two forms if you include multiprocess). For concurrency, itself, you need to study core theory. Unfortunately, searching Amazon books for "concurrency" just produces pages of "concurrency in ". Things like (yes, they are somewhat old -- most modern stuff treats computers as commodities and skips over hard-learned theory): https://www.amazon.com/Communicating-sequential-processes-Prentice-Hall-International/dp/0131532715/ref=sr_1_2?dchild=1&keywords=communicating+sequential+processes&qid=1629650630&s=books&sr=1-2 https://www.amazon.com/Communicating-Sequential-Processes-First-Years/dp/3540810498/ref=sr_1_5?dchild=1&keywords=communicating+sequential+processes&qid=1629650781&s=books&sr=1-5 Possibly: https://www.amazon.com/Operating-Systems-Three-Easy-Pieces/dp/198508659X/ref=sr_1_13?crid=13M9P659WHVDG&dchild=1&keywords=operating+system+concepts&qid=1629650909&s=books&sprefix=operating+system%2Cstripbooks%2C193&sr=1-13 (since even Python threads, at the bottom, commonly defer to the OS for threading/process control). https://www.amazon.com/Practical-Course-Operating-Systems/dp/0387912193/ref=sr_1_1?dchild=1&keywords=practical+course+on+operating+systems&qid=1629651257&s=books&sr=1-1 (at least no one is inflating the price -- I paid $13 for it new, back in the 80s) https://www.amazon.com/logical-operating-Prentice-Hall-automatic-computation/dp/0135401127/ref=sr_1_3?dchild=1&keywords=logical+design+of+operating+systems&qid=1629651483&s=books&sr=1-3 https://www.amazon.com/Structured-Concurrent-Programming-Applications-Addison-Wesley/dp/0201029375/ref=sr_1_2?dchild=1&keywords=structured+concurrent+programming&qid=1629651540&s=books&sr=1-2 https://www.amazon.com/Principles-Concurrent-Programming-Computer-Science/dp/0137010788/ref=sr_1_2?dchild=1&keywords=principles+of+concurrent+programming&qid=1629651576&s=books&sr=1-2 https://www.amazon.com/Operating-Systems-Advanced-Lecture-Computer/dp/3540087559/ref=sr_1_2?dchild=1&keywords=operating+systems+an+advanced+course+springer-verlag&qid=1629651715&s=books&sr=1-2 (Now that one has a bloated price) Then there are Tanenbaum (MINIX) and Comer (XINU -- "XINU Is Not UNIX") -- but those are more focused on implementing a teaching OS (BSD and ATT UNIX had recently gone "closed source/licensed" so universities lost access to an OS they could study at the source code level), and less on theory. (And yes, I do own copies of all of those, including an older edition of Tanenbaum, and all four XINU volumes). Python specific... BUT FREE! https://greenteapress.com/wp/semaphores/ (along with others that might interest you https://greenteapress.com/wp/ ) [Pity, 2nd Ed of Think Bayes is HTML-only ; Semaphores, DSP, Python, and Stats are available in PDF) -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From wlfraed at ix.netcom.com Sun Aug 22 13:46:33 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Sun, 22 Aug 2021 13:46:33 -0400 Subject: [Tutor] A question about None References: Message-ID: On Sun, 22 Aug 2021 17:25:10 +1000, Phil declaimed the following: >The distance sensor outputs "None" if the distance is greater than can >be measured and "None", of course cannot be compared. So I came up with >the following which is not quite correct: > >while True > ??? if distance_sensor.get_distance_cm() is not None: > ??????? if distance_sensor.get_distance_cm() > 20: Note that you have a race condition... Between the test for None and the test for >20 you have TWO "reads" of distance -- the distance could have changed enough between those reads that the second could return None! > ??????????? motor_pair.set_default_speed(20) > ??????????? motor_pair.start() > ??????? else: > ??????????? continue > > ??? elif distance_sensor.get_distance_cm() is not None: You will only get here is the first read returns None! And the continue statements are unneeded. > ??????? if distance_sensor.get_distance_cm() < 20: > ??????????? motor_pair.set_default_speed(-20) > ??????????? motor_pair.start() > ??????? else: > ??????????? continue > You perform FOUR reads of distance for a loop that should only require ONE read. Also, what happens if distance is exactly "20". >The idea is that if "None" returned from the sensor then nothing will >happen and the loop will continue until "None" is not returned. Can >anyone see the error of my ways? Uhm... "lots" Consider... speed = 0 While True: speed_change = True distance = distance_sensor.get_distance_cm() if distance is not None: #assumes distance of 0 is a valid result #otherwise, just use # if distance: if distance >= 20 and speed <> 20: speed = 20 elif distance < 20 and speed <> -20: speed = -20 else: speed_change = False if speed_change: #no sense setting speed and starting motors if they #are already configured #NOTE: should there be a motor_pair.stop() before #reversing speed value? motor_pair.set_default_speed(speed) motor_pair.start() -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From wlfraed at ix.netcom.com Sun Aug 22 14:01:58 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Sun, 22 Aug 2021 14:01:58 -0400 Subject: [Tutor] OT: [was] A question about None References: <970f729c-290f-3a00-f6b0-1d6169a09883@DancesWithMice.info> Message-ID: <5f35igtv06t5m3250h7r4au8v1smnd6bo7@4ax.com> On Sun, 22 Aug 2021 20:29:02 +1200, dn via Tutor declaimed the following: >back in the good?bad, old days, wasn't there (a language with) an >if-statement that went something like: > >if expression is positive, or zero, or negative: > goto positive-branch, zero-branch, negative-branch FORTRAN Arithmetic IF (as I recall: IF (x) 100, 200, 300 ... 100 continue C negative condition logic here ... 200 continue C zero condition logic here ... 300 continue C positive condition logic here In contrast to a computed GOTO (or BASIC ON...GOTO/GOSUB) which requires positive integer and selects a destination label based on the value GOTO (label1, label2, ..., labelN) x BASIC: ON x GOSUB label1, label2, ..., labelN ON y GOTO label1, label2, ..., labelN -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From PyTutor at DancesWithMice.info Sun Aug 22 16:56:51 2021 From: PyTutor at DancesWithMice.info (dn) Date: Mon, 23 Aug 2021 08:56:51 +1200 Subject: [Tutor] OT: [was] A question about None In-Reply-To: <5f35igtv06t5m3250h7r4au8v1smnd6bo7@4ax.com> References: <970f729c-290f-3a00-f6b0-1d6169a09883@DancesWithMice.info> <5f35igtv06t5m3250h7r4au8v1smnd6bo7@4ax.com> Message-ID: <9bed1a66-8dab-f3fb-d70a-49002eb7ad66@DancesWithMice.info> On 23/08/2021 06.01, Dennis Lee Bieber wrote: > On Sun, 22 Aug 2021 20:29:02 +1200, dn via Tutor > declaimed the following: > >> back in the good?bad, old days, wasn't there (a language with) an >> if-statement that went something like: >> >> if expression is positive, or zero, or negative: >> goto positive-branch, zero-branch, negative-branch > > FORTRAN Arithmetic IF > > (as I recall: > > IF (x) 100, 200, 300 > ... > > 100 continue > C negative condition logic here > ... > 200 continue > C zero condition logic here > ... > 300 continue > C positive condition logic here You got it! That's exactly what my fuzzy-mind was trying to recall. Did we call it an "Arithmetic IF"? Probably, more-or-less completely displaced when "Logical-IFs" arrived, eg IF ( A .LT. B ) THEN stmt; in FORTRAN-77. Relevance to this thread? The OP offered: if distance_sensor.get_distance_cm() > 20: ... if distance_sensor.get_distance_cm() < 20: with the observed omission of the equals-condition. So, my ancient bones were taking me back to multiple, mutually-exclusive conditions, but like the Python language without a case/select command: IF ( distance - 20 ) THEN 100, 200, 300 -- Regards, =dn From manpritsinghece at gmail.com Mon Aug 23 07:44:29 2021 From: manpritsinghece at gmail.com (Manprit Singh) Date: Mon, 23 Aug 2021 17:14:29 +0530 Subject: [Tutor] Fwd: Concept of multithreading In-Reply-To: References: Message-ID: Dear Sir , Just need to know if the below written example can be taken as an introduction level example of multithreading or not ? import time import threading def sumoflist(seq): sum_no = 0 for num in seq: sum_no += num time.sleep(6) print("Sum of list",sum_no) def sumoftuple(seq): sum_no = 0 for num in seq: sum_no += num time.sleep(3) print("Sum of tuple",sum_no) sumoflist([1, 2, 3]) sumoftuple((2, 4, 3)) This is sequential execution , it takes around 9 seconds to complete, due to a delay of 6 second in one function and 3 seconds delay in another function . On the other hand if i take an another approach as given below : t1 = threading.Thread(target=sumoflist, args=([1, 2, 3],)) t2 = threading.Thread(target=sumoftuple, args=((2, 4, 3),)) t1.start() t2.start() t1.join() t2.join() It takes around 6 seconds to complete the task, in this approach the seconds subprocess executes when the CPU is idle during delay in the first thread . This approach, according to me, can be a basic introduction to the idea of multithreading . i need your comments . Regards Manprit Singh ---------- Forwarded message --------- From: Manprit Singh Date: Sun, Aug 22, 2021 at 10:10 AM Subject: Concept of multithreading To: Dear sir, As far as I have gone through the literature available, i got to know that multithreading is used for concurrent subprocesses, Although they are not truly concurrent (appears to be concurrent) - like if i have two subprocesses, i can process them in a concurrent way using multithreading module(An Example -in that way if CPU is idle for one subprocess, second subprocess can take place at that time, This would same total execution time). For Example : import time import threading def sumoflist(seq): sum_no = 0 for num in seq: sum_no += num time.sleep(6) print("Sum of list",sum_no) def sumoftuple(seq): sum_no = 0 for num in seq: sum_no += num time.sleep(3) print("Sum of tuple",sum_no) Calling these two functions in a sequential manner will take a total time approx. 9 seconds, as delay of 6 second and 3 seconds is introduced in the functions. Now using threading module, with following code: t1 = threading.Thread(target=sumoflist, args=([1, 2, 3],)) t2 = threading.Thread(target=sumoftuple, args=((2, 4, 3),)) t1.start() t2.start() t1.join() t2.join() It will take less time, because one of the thread may utilize the CPU idle time of another thread . Is my understanding correct ? This can be treated as a very basic example of multithreading ? Although the threading module is used for IO bound processes. Regards Manprit Singh sumoflist([1, 2, 3]) sumoftuple((2, 4, 3)) From wlfraed at ix.netcom.com Mon Aug 23 16:59:31 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Mon, 23 Aug 2021 16:59:31 -0400 Subject: [Tutor] Fwd: Concept of multithreading References: Message-ID: <2138ighg3ak7406rh20240lukskmfp3b6j@4ax.com> On Mon, 23 Aug 2021 17:14:29 +0530, Manprit Singh declaimed the following: >Dear Sir , > >Just need to know if the below written example can be taken as an >introduction level example of multithreading or not ? >import time >import threading > >def sumoflist(seq): > sum_no = 0 > for num in seq: > sum_no += num > time.sleep(6) > print("Sum of list",sum_no) > >def sumoftuple(seq): > sum_no = 0 > for num in seq: > sum_no += num > time.sleep(3) > print("Sum of tuple",sum_no) > First comment -- other than the sleep duration, and the print message, these are identical. It would be better to just define one function and pass the duration/type to it (actually, I'd want to REMOVE the print messages from the function, and have that returned to the parent -- which, for threads, would likely mean using a Queue structure). It's also inefficient, since Python HAS a sum() function that likely operates at the assembly language level, rather than looping at Python's interpreted level. >t1 = threading.Thread(target=sumoflist, args=([1, 2, 3],)) >t2 = threading.Thread(target=sumoftuple, args=((2, 4, 3),)) >t1.start() >t2.start() >t1.join() >t2.join() > >It takes around 6 seconds to complete the task, in this approach the How do you /really/ know -- there is no timing logic in your example. Granted, one can conclude that by simple analysis of the logic. Note that t1.start() could result in swapping to the t1 thread, and it could execute the entire summation loop, then get to the sleep(), and only then does t2 get started. It could then run the entire summation loop before suspending on the sleep(). Given the values, it will likely complete its sleep() call before t1 does -- so the entire duration is controlled by t1's sleep(). Your sleep() calls overwhelm any processing, so you could just remove the summation loop and print statements and get the same result. For an example which illustrates threads actually doing "work", along with I/O blocking, study... Watch how the first 10 "." are output, then pauses as the queue is full. -=-=-=- import threading import queue import random import time inQueue = queue.Queue(10) #limit input queue, otherwise main thread could put ALL inputs on queue #before any thread even runs outQueue = queue.Queue() DONE = object() NUMWORKERS = 3 NUMVALUES = 250 #with average sleep of 1.0 second, will take 4 minutes to complete def worker(id, inQ, outQ): sumv = 0 numv = 0 startPC = time.perf_counter() startPT = time.process_time() while True: vlu = inQ.get() if vlu == DONE: break sumv += vlu[0] numv += 1 time.sleep(vlu[1]) inQ.put(vlu) #propagate the DONE flag to next outQ.put((id, sumv, numv, time.perf_counter() - startPC)) start = time.perf_counter() threads = [threading.Thread(target=worker, args=(i, inQueue, outQueue)) for i in range(NUMWORKERS)] for t in threads: t.start() for _ in range(NUMVALUES): v = random.randint(1, NUMVALUES // 2) s = random.random() * 2.0 inQueue.put((v, s)) #will block if 10 items are in the queue, letting thread(s) run print(". ", end="", flush=True) #progress indicator print("\n\n") inQueue.put(DONE) for t in threads: t.join() for _ in range(NUMWORKERS): id, sm, nm, dur = outQueue.get() print("Worker: %s\tSum: %s\tNumber Values: %s\tTime: %s" % (id, sm, nm, dur)) print("Total time: %s" % (time.perf_counter() - start)) -=-=-=- C:\Users\Wulfraed\Documents\_Hg-Repositories\Python Progs>Q_Threads.py . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Worker: 2 Sum: 5136 Number Values: 82 Time: 75.9186911 Worker: 1 Sum: 5349 Number Values: 84 Time: 76.3049776 Worker: 0 Sum: 5424 Number Values: 84 Time: 76.3747903 Total time: 76.3832255 C:\Users\Wulfraed\Documents\_Hg-Repositories\Python Progs> Note that the time output is wall-clock durations, and mostly counts the time spent sleeping (or for main thread, blocked trying to queue the next value). -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From steevekerou at yahoo.fr Mon Aug 23 07:16:19 2021 From: steevekerou at yahoo.fr (Steeve Kerou) Date: Mon, 23 Aug 2021 11:16:19 +0000 (UTC) Subject: [Tutor] Fw: Why Pyto package not available yet ??? In-Reply-To: <2124684154.1365011.1629717345151@mail.yahoo.com> References: <2124684154.1365011.1629717345151.ref@mail.yahoo.com> <2124684154.1365011.1629717345151@mail.yahoo.com> Message-ID: <77684613.1362493.1629717379711@mail.yahoo.com> Hi,? Here is Pyto, the first animated character that help you Learn Python like you play video games?Why not implement me on Python PyPI ??? Episode 1: Intro / Who is Pyto ? / The Idle World | Pytorial | | | | | | | | | | | Episode 1: Intro / Who is Pyto ? / The Idle World | Pytorial | | | Alan V. From mayoadams at gmail.com Mon Aug 23 18:12:51 2021 From: mayoadams at gmail.com (Mayo Adams) Date: Mon, 23 Aug 2021 18:12:51 -0400 Subject: [Tutor] Fw: Why Pyto package not available yet ??? In-Reply-To: <77684613.1362493.1629717379711@mail.yahoo.com> References: <2124684154.1365011.1629717345151.ref@mail.yahoo.com> <2124684154.1365011.1629717345151@mail.yahoo.com> <77684613.1362493.1629717379711@mail.yahoo.com> Message-ID: Certainly happy to have this. On Mon, Aug 23, 2021 at 5:41 PM Steeve Kerou via Tutor wrote: > Hi, > > > > > > Here is Pyto, the first animated character that help you Learn Python like > you play video games Why not implement me on Python PyPI ??? > > > > Episode 1: Intro / Who is Pyto ? / The Idle World | Pytorial > > > | > | > | > | | | > > | > > | > | > | | > Episode 1: Intro / Who is Pyto ? / The Idle World | Pytorial > > > | > > | > > | > > > > > Alan V. > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > -- Mayo Adams 287 Erwin Rd. Chapel Hill, NC 27514 (919)-780-3917 mayoadams at gmail.com From wlfraed at ix.netcom.com Tue Aug 24 10:51:37 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Tue, 24 Aug 2021 10:51:37 -0400 Subject: [Tutor] Fwd: Concept of multithreading References: <2138ighg3ak7406rh20240lukskmfp3b6j@4ax.com> Message-ID: On Mon, 23 Aug 2021 16:59:31 -0400, Dennis Lee Bieber declaimed the following: > Note that the time output is wall-clock durations, and mostly counts >the time spent sleeping (or for main thread, blocked trying to queue the >next value). I tried changing the time.perf_counter() into time.thread_time_ns() and time.process_time_ns(), neither of which include the time spent in sleep() (nor, I suspect, any other blocking operation), but so little work is actually done each time the thread(s) wake up that the accumulated times came out as 0ns. This even after I doubled the work, cut the sleep() times to average 0.5s, and moved the generation of random integers and random sleep times /into/ the worker threads. The main thread is just queuing integers to wake up the workers -- the integer is otherwise ignored. -=-=-=- C:\Users\Wulfraed\Documents\_Hg-Repositories\Python Progs>Q_Threads.py . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . Worker: 2 Sum: 22722 Number Values: 181 Time: 83.207052 Worker: 0 Sum: 17799 Number Values: 151 Time: 83.56292210000001 Worker: 1 Sum: 21747 Number Values: 168 Time: 83.8785809 Total time: 83.8851056 C:\Users\Wulfraed\Documents\_Hg-Repositories\Python Progs> -=-=-=- DIFF format rather than full listing of source... @@ -11,5 +11,6 @@ DONE = object() NUMWORKERS = 3 -NUMVALUES = 250 #with average sleep of 1.0 second, will take 4 minutes to complete +NUMVALUES = 500 #with average sleep of 0.5 second, will take 4 minutes to complete + #IF done linearly. with three threads, about 85 seconds each @@ -15,4 +16,7 @@ +#tried using time.thread_time and time.process_time as they do not count +#time spent in sleep() or otherwise blocked, but the degree of work done +#on each cycle is so small the accumulated times were 0.0! def worker(id, inQ, outQ): sumv = 0 numv = 0 @@ -16,7 +20,7 @@ def worker(id, inQ, outQ): sumv = 0 numv = 0 - startPC = time.perf_counter() + start = time.perf_counter() while True: vlu = inQ.get() if vlu == DONE: break @@ -20,5 +24,7 @@ while True: vlu = inQ.get() if vlu == DONE: break - sumv += vlu[0] + #increment count of processed items + #generate random value ("work") + #generate random sleep between 0 and 1 second numv += 1 @@ -24,4 +30,6 @@ numv += 1 - time.sleep(vlu[1]) - inQ.put(vlu) #propagate the DONE flag to next + sumv += random.randint(1, NUMVALUES // 2) + time.sleep(random.random()) + inQ.put(vlu) #propagate the DONE flag to next thread + #return results outQ.put((id, sumv, numv, @@ -27,3 +35,3 @@ outQ.put((id, sumv, numv, - time.perf_counter() - startPC)) + time.perf_counter() - start)) @@ -29,8 +37,8 @@ -startPC = time.perf_counter() +start = time.perf_counter() threads = [threading.Thread(target=worker, args=(i, inQueue, outQueue)) for i in range(NUMWORKERS)] for t in threads: t.start() @@ -31,13 +39,12 @@ threads = [threading.Thread(target=worker, args=(i, inQueue, outQueue)) for i in range(NUMWORKERS)] for t in threads: t.start() -for _ in range(NUMVALUES): - v = random.randint(1, NUMVALUES // 2) - s = random.random() * 2.0 - inQueue.put((v, s)) #will block if 10 items are in the queue, letting thread(s) run +for i in range(NUMVALUES): + #i is put in queue only to release next worker, it is otherwise meaningless + inQueue.put(i) #will block if 10 items are in the queue, letting thread(s) run print(". ", end="", flush=True) #progress indicator print("\n\n") @@ -52,5 +59,5 @@ print("Worker: %s\tSum: %s\tNumber Values: %s\tTime: %s" % (id, sm, nm, dur)) -print("Total timePC: %s" % - (time.perf_counter() - startPC)) +print("Total time: %s" % + (time.perf_counter() - start)) -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From nathan-tech at hotmail.com Tue Aug 24 16:06:42 2021 From: nathan-tech at hotmail.com (nathan Smith) Date: Tue, 24 Aug 2021 21:06:42 +0100 Subject: [Tutor] beautiful soup raw text workarounds? Message-ID: Hi List, I'm using beautiful soup to pass a website which is all going well. I'm having problems though with getting it to include the raw text, that is to say, text not in any tag. I've done some Googling on this and it seems beautiful soup does not support the text outside of tags? Fair enough! I was wondering how I could work around this issue? For instance, is there like, tag.endpos next_tag.startpos so I could do raw-text=text[endpos:nextpos] I've included the web page below for reference so you can see what I mean. the thing I am stuck on is below h2. Nathan ' Website: This is my website

Headings

Paragraphs and such.

Another heading.

This text here doesn't
want to show in bs. From nathan-tech at hotmail.com Tue Aug 24 16:15:22 2021 From: nathan-tech at hotmail.com (nathan Smith) Date: Tue, 24 Aug 2021 21:15:22 +0100 Subject: [Tutor] beautiful soup raw text workarounds? In-Reply-To: References: Message-ID: I actually fixed this myself. It was a perspective issue. I was looking at: tags=soup.find_all() which returns tags only. Raw text are not tags. Sorry for the time waster. On 24/08/2021 21:06, nathan Smith wrote: > Hi List, > > > I'm using beautiful soup to pass a website which is all going well. > > I'm having problems though with getting it to include the raw text, > that is to say, text not in any tag. > > I've done some Googling on this and it seems beautiful soup does not > support the text outside of tags? Fair enough! > > I was wondering how I could work around this issue? > > For instance, is there like, tag.endpos next_tag.startpos so I could > do raw-text=text[endpos:nextpos] > > > I've included the web page below for reference so you can see what I > mean. the thing I am stuck on is below h2. > > > Nathan ' > > > Website: > > > > > > This is my website > > > > > >

Headings

> >

Paragraphs and such.

> >

Another heading.

> > This text here doesn't
> > want to show in bs. > > > > > > _______________________________________________ > Tutor maillist? -? Tutor at python.org > To unsubscribe or change subscription options: > https://emea01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmail.python.org%2Fmailman%2Flistinfo%2Ftutor&data=04%7C01%7C%7C7171480747aa46d96a6908d9673ae7b0%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637654324947862911%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=2wl4lWEZnztcPR6f%2BzUBOH7qIDOEoapygjPTOWEkIRY%3D&reserved=0 > From dghmerch at comcast.net Tue Aug 24 12:48:52 2021 From: dghmerch at comcast.net (David Haan) Date: Tue, 24 Aug 2021 10:48:52 -0600 Subject: [Tutor] terminating driver instance prior to completion Message-ID: I have a driver written in Python by others that is used to move a stepper motor.? The parameters of the driver calls for, among other things, the number of steps the motor needs to be moved.? It can be from 1 to infinity. I also have a hardware interrupt available that will indicate when the device the motor is moving has reached its home position.? I need to stop the driver at that point regardless of the number of steps it has processed. I create an instance of the driver and pass it appropriate values for its variables.? Once the driver instance starts, it will not return to the main program until it has processed all the steps I passed in one of the instance variables. When the hardware interrupt fires, I want to terminate the driver code execution so it will return control immediately to the main program flow. My question is if there is a method to terminate the Python driver code instance.? I really don't want to copy the driver code and modify it specifically for my own program if I don't have to. Any suggestions would be appreciated. From alan.gauld at yahoo.co.uk Wed Aug 25 07:34:59 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Wed, 25 Aug 2021 12:34:59 +0100 Subject: [Tutor] beautiful soup raw text workarounds? In-Reply-To: References: Message-ID: On 24/08/2021 21:15, nathan Smith wrote: > I actually fixed this myself. Good, but it would be useful to share how, for the benefit of future readers... Surely "raw text" is still inside a tag, even if its only the top level tag? > tags=soup.find_all() > which returns tags only. > > Raw text are not tags. So how did you extract it? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From wlfraed at ix.netcom.com Wed Aug 25 12:35:19 2021 From: wlfraed at ix.netcom.com (Dennis Lee Bieber) Date: Wed, 25 Aug 2021 12:35:19 -0400 Subject: [Tutor] terminating driver instance prior to completion References: Message-ID: <71rcigpem13q44pvedeoeqi3tmc363tug5@4ax.com> On Tue, 24 Aug 2021 10:48:52 -0600, David Haan declaimed the following: >I have a driver written in Python by others that is used to move a >stepper motor.? The parameters of the driver calls for, among other >things, the number of steps the motor needs to be moved.? It can be from >1 to infinity. > >I also have a hardware interrupt available that will indicate when the >device the motor is moving has reached its home position.? I need to >stop the driver at that point regardless of the number of steps it has >processed. > >I create an instance of the driver and pass it appropriate values for >its variables.? Once the driver instance starts, it will not return to >the main program until it has processed all the steps I passed in one of >the instance variables. > >When the hardware interrupt fires, I want to terminate the driver code >execution so it will return control immediately to the main program flow. > >My question is if there is a method to terminate the Python driver code >instance.? I really don't want to copy the driver code and modify it >specifically for my own program if I don't have to. Given you are tallking stepper motors, this doesn't sound like a desktop Python -- Circuit Python or microPython, perhaps? For the most part, Python does not have means for true hardware interrupts. One has to use something like asyncio (I have no experience with that -- my mind understands threading, though CircuitPython doesn't) or threads. In the case of threads, one would have a thread that is blocked (or, ugh, polls) on the interrupt signal. When the signal triggers, the thread would set a global flag, which all other threads have to periodically test, and exit if they find the flag set. So -- yes, you likely WILL need to tweak this driver. Somewhere in it you need a loop: while True: if intFlag: break #step the motor by one unit #doing whatever current checks exist for non-infinite movement #maybe end with a sleep(0) to trigger thread swap so monitor #thread can check the interrupt. # #note: if ONLY the motor driver cares about the interrupt, you #don't need a separate thread, and can replace # if intFlag: #with whatever is needed to actually check the interrupt -- Wulfraed Dennis Lee Bieber AF6VN wlfraed at ix.netcom.com http://wlfraed.microdiversity.freeddns.org/ From manpritsinghece at gmail.com Wed Aug 25 13:50:15 2021 From: manpritsinghece at gmail.com (Manprit Singh) Date: Wed, 25 Aug 2021 23:20:15 +0530 Subject: [Tutor] A complex number class, making a class instance of same class inside one of its methods Message-ID: Dear sir, I have written a class Complex_num, from which i can set the values of real and imaginary part of complex number(setvalue) , and compare the magnitude of two instances of this class(__gt__()) . Add real and imaginary parts of two instances of this class(__add__()) and print the values of instance objects with __str__(). class Complex_num: def __init__(self): self.real = 0 self.imag = 0 def setvalue(self, real, imag): self.real = real self.imag = imag def __gt__(self, com_num): c1 = (self.real**2 + self.imag**2)**0.5 c2 = (com_num.real**2 + com_num.imag**2)**0.5 return c1 > c2 def __add__(self, com_num): temp = Complex_num() rpart = self.real + com_num.real ipart = self.imag + com_num.imag temp.setvalue(rpart, ipart) return temp def __str__(self): sign ="+" if self.imag >= 0 else "" com = "("+str(self.real)+sign +str(self.imag)+"j)" return com x = Complex_num() x.setvalue(4, 1) y = Complex_num() y.setvalue(4, -5) print(x) - returns (4+1j) print(y) - returns (4-5j) x > y returns False print(x+y) returns (8-4j) The answers are correct , but i have a question : Can we call the class inside any method of the class ? If I need an instance object of the class inside one of the methods of the same class. In the above given class Complex_num, inside def__add__() , i have made an instance object temp, The idea is after finding the sum of real and imaginary parts of instance objects x & y inside __add__(), I have set the instance variables of the instance object temp with this sum of real and imaginary parts, returning this temp will cause print(x+y) to print the sum of real and imaginary parts of x& y.due to __str__(). Regards Manprit Singh From alan.gauld at yahoo.co.uk Wed Aug 25 14:25:01 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Wed, 25 Aug 2021 19:25:01 +0100 Subject: [Tutor] A complex number class, making a class instance of same class inside one of its methods In-Reply-To: References: Message-ID: On 25/08/2021 18:50, Manprit Singh wrote: > I have written a class Complex_num, Given that Python has a built-in complex number type, that's probably not the best learning example you could have chosen. However... > from which i can set the values of real > and imaginary part of complex number(setvalue) Numbers in Python are immutable so to follow that example you should try to make your objects follow similar patterns. Thus setting the real/imaginary parts after the number as been created is probably a mistake, instead perform an operation that returns the new values and use those to create a new object. > , and compare the magnitude > of two instances of this class(__gt__()) . Add real and imaginary parts of > two instances of this class(__add__()) and print the values of instance > objects with __str__(). Thats all good stuff. The built-in version does a lot more of course! > class Complex_num: > > def __init__(self): > self.real = 0 > self.imag = 0 Why not pass these attributes as parameters with defaults of zero: def __init__(self, real=0, imag=0): self.real = real self.imag = imag And now you initialise as zer_comp = Complex_num() val_comp = Complex_num(3,4) > def setvalue(self, real, imag): > self.real = real > self.imag = imag You then don't need this which probably shouldn't exist anyway. > def __gt__(self, com_num): > c1 = (self.real**2 + self.imag**2)**0.5 > c2 = (com_num.real**2 + com_num.imag**2)**0.5 > return c1 > c2 > > def __add__(self, com_num): > temp = Complex_num() > rpart = self.real + com_num.real > ipart = self.imag + com_num.imag > temp.setvalue(rpart, ipart) > return temp If you put the attributes in the Init(): def __add__(self, com_num): rpart = self.real + com_num.real ipart = self.imag + com_num.imag return Complex_num(rpart,ipart) > def __str__(self): > sign ="+" if self.imag >= 0 else "" > com = "("+str(self.real)+sign +str(self.imag)+"j)" > return com > > x = Complex_num() > x.setvalue(4, 1) > y = Complex_num() > y.setvalue(4, -5) > print(x) - returns (4+1j) > print(y) - returns (4-5j) > x > y returns False > print(x+y) returns (8-4j) > > The answers are correct , but i have a question : > > Can we call the class inside any method of the class ? If I need an > instance object of the class inside one of the methods of the same class. Yes, you can create instances inside methods of the class. That is perfectly normal in OOP. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Wed Aug 25 14:33:43 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Wed, 25 Aug 2021 19:33:43 +0100 Subject: [Tutor] terminating driver instance prior to completion In-Reply-To: References: Message-ID: On 24/08/2021 17:48, David Haan wrote: > I have a driver written in Python by others that is used to move a > stepper motor.? The parameters of the driver calls for, among other > things, the number of steps the motor needs to be moved.? It can be from > 1 to infinity. > > I also have a hardware interrupt available that will indicate when the > device the motor is moving has reached its home position.? I need to > stop the driver at that point regardless of the number of steps it has > processed. > We can't really comment on this without knowing the API. I would expect the API to provide a stop() method as a matter of course - there are many reasons why you would want to stop the motor immediately! As for the hardware interrupt that all depends on the OS and how it handles hardware interrupts. It might require some assembler programming in the worst case! How do these interrupts manifest themselves to your Python code? Is there a library? > I create an instance of the driver and pass it appropriate values for > its variables.? Once the driver instance starts, it will not return to > the main program until it has processed all the steps I passed in one of > the instance variables. One (inefficient) possibility is always pass single steps and put it in a loop (maybe even within a thread). That way control is all within Python. But that may introduce speed issues. Or even interface issues, we simply can't tell without a lot more detail on the API. > When the hardware interrupt fires, I want to terminate the driver code > execution so it will return control immediately to the main program flow. > > My question is if there is a method to terminate the Python driver code > instance.? We don't know. What does your API say? > I really don't want to copy the driver code and modify it > specifically for my own program if I don't have to. If its an OOP interface you can modify behaviour without changing the original code simply by using inheritance to create your own class and override the necessary methods and add any needed new ones. The ability to change an existing class's behaviour without modifying its code is one of the strongest features of OOP. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From nathan-tech at hotmail.com Wed Aug 25 14:39:11 2021 From: nathan-tech at hotmail.com (nathan Smith) Date: Wed, 25 Aug 2021 19:39:11 +0100 Subject: [Tutor] beautiful soup raw text workarounds? In-Reply-To: References: Message-ID: Hi! I sure can :) So as stated previously: tags = soup.find_all() Will get you a list of the tags in some html text, however, raw text, EG that which is not in a tag is something else. The method I used, and will explain below, is probably unnecessary as BeautifulSoup arranges itself in a tree like state, so to access the body tag it's soup.html.body but for my purposes what I did was: 1. Run from the top of the tree downward, collecting children on the way and compile them into a list: def extract_tags(element): ?t=[element] # include the parent object ?if(type(element)==bs4.Comment or type(element)==bs4.Stylesheet or type(element)==bs4.element.NavigableString): ? return t # These do not and? can not have children ?for child in element.children: ? t.extend(extract_tags(child)) ?return t The function above recursively gets all the elements from a parent so to get all the elements (elements being tags and raw strings) you simply do: soup=BeautifulSoup(your_html_code) full_list=extract_tags(soup) Then if you wanted to list only raw strings you could do: for x in full_list: ?if(type(x)==bs4.element.NavigableString): ? print(str(x.string)) You have to use str(x.string) because Beautiful soup has it's own subclass of string (I think that's the correct terminology) and from my experience today, python will throw a fit if you try and combine it with a regular string (for obvious reasons I guess, they're not the same type of object). I hope this helps someone! :) Nathan On 25/08/2021 12:34, Alan Gauld via Tutor wrote: > On 24/08/2021 21:15, nathan Smith wrote: >> I actually fixed this myself. > Good, but it would be useful to share how, for the > benefit of future readers... > > Surely "raw text" is still inside a tag, even if > its only the top level tag? > >> tags=soup.find_all() >> which returns tags only. >> >> Raw text are not tags. > So how did you extract it? > From malikfuller5 at gmail.com Thu Aug 26 00:30:03 2021 From: malikfuller5 at gmail.com (Malik Fuller) Date: Wed, 25 Aug 2021 21:30:03 -0700 Subject: [Tutor] Python Help Message-ID: Hi, I am very new to the coding world and I am in need of a very basic crash course on the format of inserts and how to achieve certain results when coding in python. Any help would be greatly appreciated! From alan.gauld at yahoo.co.uk Thu Aug 26 04:22:02 2021 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Thu, 26 Aug 2021 09:22:02 +0100 Subject: [Tutor] Python Help In-Reply-To: References: Message-ID: On 26/08/2021 05:30, Malik Fuller wrote: > Hi, I am very new to the coding world and I am in need of a very basic > crash course on the format of inserts Can you explain what you mean by that? Give an example perhaps? Also if you have any code and error messages(in full) please paste them into your message using plain text. Finally a note of your OS and Python version helps too. The more specific the information you give us the more specific will be the answer you get. What are you trying to insert and where are you trying to insert it? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos