From asad.hasan2004 at gmail.com Wed Jan 2 10:09:53 2019 From: asad.hasan2004 at gmail.com (Asad) Date: Wed, 2 Jan 2019 20:39:53 +0530 Subject: [Tutor] Log file for Nested if-elif Message-ID: Hi All , Need advice on the following piece of code : with open(r"file1.log", 'r') as f: tail = deque(maxlen=8) # the last eight lines script = None for line in f: tail.append(line) if re.search('\?/patch/\d{8}/\d{8}/admin/load.sql',line,re.IGNORECASE): script = line elif re.search(r'Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8}', line, re.IGNORECASE): script = line elif re.search(r'set_metadata', line ,re.IGNORECASE) is not None: print "Reason of error \n", tail[-1] print "Script:\n", script print "Block of code:\n" for item in tail: print item print " Danger " break Now this is printing the last cached line in the variable line . However I would like to see the following output : 1) if it matches the pattern: \?/patch/\d{8}/\d{8}/admin/load.sql then look for the line "set_metadata" in the file1.log if it finds the pattern then print the line which matches the pattern \?/patch/\d{8}/\d{8}/admin/load.sql print last 4 lines of the tail array and exit 2) if it doesnot match '\?/patch/\d{8}/\d{8}/admin/load.sql' then look of the anothern pattern :Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8} if it find the pattern then look for line "set_metadata" in the file1.log if it finds the pattern then print the line which matches the pattern \?/patch/\d{8}/\d{8}/admin/load.sql print all the lines in tail print a recommendation "Please check the installation" 3 ) if it doesnot match the pattern: \?/patch/\d{8}/\d{8}/admin/load.sql or '\?/patch/\d{8}/\d{8}/admin/load.sql' print "No match found refer to install guide" Can you advice what I can do to change the code . Thanks, -- From steve at pearwood.info Wed Jan 2 23:16:45 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 3 Jan 2019 15:16:45 +1100 Subject: [Tutor] Log file for Nested if-elif In-Reply-To: References: Message-ID: <20190103041645.GO13616@ando.pearwood.info> On Wed, Jan 02, 2019 at 08:39:53PM +0530, Asad wrote: > Hi All , > > Need advice on the following piece of code : Let me write it in a more "Pythonic" style: PATCH = r'\?/patch/\d{8}/\d{8}/admin/load.sql' APPLY = r'Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8}' ERROR = r'set_metadata' tail = deque(maxlen=8) # the last eight lines script = None with open("file1.log", 'r') as f: for line in f: tail.append(line) if (re.search(PATCH, line, re.IGNORECASE) or re.search(APPLY, line, re.IGNORECASE): script = line elif re.search(ERROR, line, re.IGNORECASE): print "Reason for error \n", line print "Script:", script print "Tail:\n", tail print " Danger " # Seriously? This is dangerous? break > Now this is printing the last cached line in the variable line . However > I would like to see the following output : > > 1) if it matches the pattern: \?/patch/\d{8}/\d{8}/admin/load.sql then > look for the line "set_metadata" in the file1.log if it finds the pattern > then print the line which matches the pattern > \?/patch/\d{8}/\d{8}/admin/load.sql Where are you looking? *Anywhere* in the log file? Only in the immediate next line? Anywhere forward of the "patch" line? You are looking for two patterns, but does the order they appear, or the distance apart, matter? blah blah blah set_metadata blah blah ... 100 lines ... blah blah blah /patch/ ... admin/load.sql ... 100 lines ... Is that a match? > print last 4 lines of the tail array and exit print tail[-4:] sys.exit() > 2) if it doesnot match '\?/patch/\d{8}/\d{8}/admin/load.sql' > > then look of the anothern pattern > :Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8} if it find the pattern > > then look for line "set_metadata" in the file1.log if it finds the pattern > then print the line which matches the pattern > \?/patch/\d{8}/\d{8}/admin/load.sql This is a contradiction: you just said that it DOESN'T match the patch...load.sql pattern, but now you want it to print the line that matched. There is no line that matches! > 3 ) if it doesnot match the pattern: \?/patch/\d{8}/\d{8}/admin/load.sql > or '\?/patch/\d{8}/\d{8}/admin/load.sql' > > print "No match found refer to install guide" > > Can you advice what I can do to change the code . You need to tighten the specifications to make it more clear what you are trying to do. -- Steve From __peter__ at web.de Thu Jan 3 06:11:08 2019 From: __peter__ at web.de (Peter Otten) Date: Thu, 03 Jan 2019 12:11:08 +0100 Subject: [Tutor] Log file for Nested if-elif References: Message-ID: Asad wrote: > Hi All , > > Need advice on the following piece of code : > > with open(r"file1.log", 'r') as f: > tail = deque(maxlen=8) # the last eight lines > script = None > for line in f: > tail.append(line) > if > re.search('\?/patch/\d{8}/\d{8}/admin/load.sql',line,re.IGNORECASE): > script = line > elif re.search(r'Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8}', > line, re.IGNORECASE): > script = line > elif re.search(r'set_metadata', line ,re.IGNORECASE) is not None: > print "Reason of error \n", tail[-1] > print "Script:\n", script > print "Block of code:\n" > for item in tail: > print item > print " Danger " > break > Now this is printing the last cached line in the variable line . However > I would like to see the following output : > > 1) if it matches the pattern: \?/patch/\d{8}/\d{8}/admin/load.sql then > > look for the line "set_metadata" in the file1.log if it finds the pattern > then print the line which matches the pattern > \?/patch/\d{8}/\d{8}/admin/load.sql > > print last 4 lines of the tail array and exit > > 2) if it doesnot match '\?/patch/\d{8}/\d{8}/admin/load.sql' > > then look of the anothern pattern > :Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8} if it find the pattern > > then look for line "set_metadata" in the file1.log if it finds the > pattern then print the line which matches the pattern > \?/patch/\d{8}/\d{8}/admin/load.sql > > print all the lines in tail > > print a recommendation "Please check the installation" > > > 3 ) if it doesnot match the pattern: \?/patch/\d{8}/\d{8}/admin/load.sql > or '\?/patch/\d{8}/\d{8}/admin/load.sql' > > print "No match found refer to install guide" > > Can you advice what I can do to change the code . You have "nested" in the subject, do not make an attempt to structure your loop. Why is that? In my experience using lots of small functions makes code easier to understand. Below is a suggestion how you might break up your code. # untested; expect a few bugs! def with_tail(items, maxlen): tail = deque(maxlen=maxlen) def gen_items(): for item in items: tail.append(item) yield item return tail, gen_items() def search_end_of_section(lines): for line in lines: if re.search(r'set_metadata', line, re.IGNORECASE) is not None: break else: raise ValueError("Reached end of file...panic!") def dump(script, tail, advice=None): print "Reason of error \n", tail[-1] print "Script:\n", script print "Block of code:\n" for item in tail[-limit:]: print item print " Danger " if advice is not None: print advice RE_PATCH = re.compile(r'\?/patch/\d{8}/\d{8}/admin/load.sql', re.IGNORECASE) RE_APPLY = re.compile( r'Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8}', re.IGNORECASE ) with open(r"file1.log", 'r') as f: tail, lines = with_tail(f) for line in lines: if RE_PATCH.search(line) is not None: search_end_of_section(lines) dump(script=line, tail=tail[-4:]) break elif RE_APPLY.search(line) is not None: search_end_of_section(lines) dump( script=line, tail=tail, advice="Please check the installation" ) break From davidmlynch0 at gmail.com Sat Jan 5 11:18:04 2019 From: davidmlynch0 at gmail.com (David Lynch) Date: Sat, 5 Jan 2019 11:18:04 -0500 Subject: [Tutor] function question Message-ID: Hello, I'm not sure if this is where I can find this sort of help but I am struggling understanding functions. I have written a simple math code but I feel like it could be improved if I were to use it in a function. >From what I've read about functions I should be able to define a function with 2 variables? And then I can add all of my code into that function by indenting it. And then shouldn't I be able to function(float(input("enter input: "))) function(float(input("enter input 2: "))) return(function) ?? Also, since it'll be in a function, when I type return(function) will it rerun the code from the top? I'd appreciate any advice! David Lynch -------------- next part -------------- print("Passive income from incentivised onboarding") capital = float(input("Amount of capital raised: ")) amount_invested = float(input("Amount invested: ")) revenue = (capital * .75) / .2 total_royalty = (capital / 100) + 3000000 royalty_blocs_owned = amount_invested / 100 percentage_owned = royalty_blocs_owned / total_royalty total_percent_owned = royalty_blocs_owned / total_royalty passive_income = revenue * .1 * total_percent_owned print("Assuming 75% of the capital raised is used for incentivised onboarding and there is no organic growth. Your passive income, by the time the onboarding funds are depleted, will be: ") print(passive_income) print("") print("Passive income with estimated revenue") capital = float(input("Amount of capital raised: ")) total_royalty = (capital / 100) + 3000000 amount_invested = float(input("Amount invested: ")) royalty_blocs_owned = amount_invested / 100 percentage_owned = royalty_blocs_owned / total_royalty total_percent_owned = royalty_blocs_owned / total_royalty revenue = float(input("Companies Revenue: ")) passive_income = revenue * .1 * total_percent_owned print("Your passive income based on the revenue provided will be: ") print(passive_income) From alan.gauld at yahoo.co.uk Sat Jan 5 12:24:19 2019 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sat, 5 Jan 2019 17:24:19 +0000 Subject: [Tutor] function question In-Reply-To: References: Message-ID: On 05/01/2019 16:18, David Lynch wrote: > Hello, > I'm not sure if this is where I can find this sort of help but I am > struggling understanding functions. I have written a simple math code but I > feel like it could be improved if I were to use it in a function. You arein the right place and you have the right idea. You might find my tutorial on functions worth looking at: http://www.alan-g.me.uk/l2p2/tutfunc.htm But for the short version keep reading... > From what I've read about functions I should be able to define a function > with 2 variables? And then I can add all of my code into that function by > indenting it. That's correct, more or less. So a function with two variables will look like def aFunction(var1,var2): your code here return result > And then shouldn't I be able to > function(float(input("enter input: "))) No because you are only passing one variable into the function - the result of the input)() function. input() is just a function like any other, it takes an input parameter and returns a value. > function(float(input("enter input 2: "))) And again you call your function with a single value. The function has no memory of previous calls so it won't know about your previous attempt with input > return(function) And the return statement needs to be inside the function. It returns the resuilt to the caller of the function. So you really want something like result = function(input('Value1'), input('Value 2')) Although we generally consider using input like that a bad idea. It would be better to write: val1 = input('Value 1') val2 = input('value 2') result = function(val1,val2) It's slightly longer but much easier to debug if things go wrong! > Also, since it'll be in a function, when I type return(function) will it > rerun the code from the top? Every time you type function() it will run the function code afresh. To try to clarify things, here is a very simple function that simply adds two numbers and the code that uses it. def add(val1,val2): total = val1 + val2 return total x = int( input('Val1? ') ) # convert input string to integer y = int( input('Val2? ') ) result = add(x,y) print(x, '+', y, '=', result) #or print("%d + %d = %d" % (x,y,result)) See my tutorial paqe for more examples and a more complete explanation. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From steve at pearwood.info Sat Jan 5 18:46:13 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 6 Jan 2019 10:46:13 +1100 Subject: [Tutor] function question In-Reply-To: References: Message-ID: <20190105234612.GV13616@ando.pearwood.info> Hello David, and welcome! On Sat, Jan 05, 2019 at 11:18:04AM -0500, David Lynch wrote: [...] > From what I've read about functions I should be able to define a function > with 2 variables? And then I can add all of my code into that function by > indenting it. So far so good! Here's an example of such a function: def add(x, y): return x + y Of course in real programs we wouldn't bother with such a trivial function, since it is easier to read and write "a + b" rather than "add(a, b)". But it illustrates the concept. > And then shouldn't I be able to > function(float(input("enter input: "))) > function(float(input("enter input 2: "))) > return(function) > ?? No, nothing like this. You are calling the function twice, but each time only supplying a single argument when it needs two. And the "return" keyword is only legal inside functions. I *strongly* recommend that you make use of one of Python's best and most powerful features: the interactive interpreter. There you can try out small snippets of code with ease, see the result of running code live, use Python as a calculator, etc. That's an excellent way of learning to use the language, and you can always ask for help here if you have questions. Do you know how to start the interactive interpreter? That will depend on your operating system (Windows, Mac, Linux) but generally you will open a "terminal" or "console", which gives you a command-line. That will have a % or $ prompt where you type commands. Type "python" and hit the ENTER key. You will then see something similar to this: Python 3.5.2 (default, Oct 12 2016, 10:47:40) [GCC 4.1.2 20080704 (Red Hat 4.1.2-55)] on linux Type "help", "copyright", "credits" or "license" for more information. and then be presented with >>> the default Python prompt, where you type Python code. In my examples below, I use the custom prompt "py>" because I think it looks nicer :-) If you can get to the point of seeing the Python prompt, you're ready to start running live Python code. At the prompt, type: 12*23 and hit ENTER, and you should see the answer 276: py> 12*23 276 Now type in your function. Notice that you need to indent the body of the function, and that the prompt changes to ... inside the body. py> def add(x, y): ... return x + y ... py> (Remember, don't type the "py>" or "..." prompts.) Now you should be ready to try calling the function: py> add(99, 7) 106 If you get to that point, then you're ready to go to the next stage: intentionally doing something wrong so you can see what errors to expect. What happens if you enter: add(99) at the prompt? What error do you get? Once you've understood that, they you can start writing more serious, useful functions. Good luck, and don't hesitate to ask if you have any further questions! -- Steve From nathan-tech at hotmail.com Sun Jan 6 09:14:10 2019 From: nathan-tech at hotmail.com (nathan tech) Date: Sun, 6 Jan 2019 14:14:10 +0000 Subject: [Tutor] running a game server Message-ID: Hello all, Above all I am a game developer and I want to make multi player games. For my first trick I want to make a game that is building related, with a map and such. For the map I was thinking of using a dict and running along those lines. My question is, is python really the way to go for game servers? I remember, long ago, running a script that went along the lines of: ??? import shutil ??? import time ??? while 1: ???? f=open("backup.count_file.txt","r"); ???? num=int(f.read()) ???? f.close() ???? f=open("backup/count_file.txt","w"); ???? f.write(str(num+1)); ???? f.close() ???? shutil.copy("my_file.txt", "my_file.txt.backup."+str(num)) ???? time.sleep(900) After running for a day, this had to be killed because it sucked up more memory than anything I'd ever seen before. This is obviously not ideal. So I wonder, is it something about my code up there? Should I be freeing something or other? The game server would have to, obviously, process messages from clients, perhaps keep track of running tasks through a loop of times such as: ??? timevar=time.time() ??? for x in tasks: ???? if(x[0]>timevar): ????? #task complete, do code I don't mind too much if its using a bit of memory, but, if python is just going to leak all over the place and I'm better off learning c or c sharp or something, then I'd like to know! Thanks a lot for any help. Nate From alan.gauld at yahoo.co.uk Sun Jan 6 13:09:10 2019 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sun, 6 Jan 2019 18:09:10 +0000 Subject: [Tutor] running a game server In-Reply-To: References: Message-ID: On 06/01/2019 14:14, nathan tech wrote: > My question is, is python really the way to go for game servers? Its certainly one possibility. Without a lot more details about the design, expected load, OS etc its impossible to say with any certainty. > I remember, long ago, running a script that went along the lines of: > > ??? import shutil > ??? import time > > ??? while 1: > ???? f=open("backup.count_file.txt","r"); > ???? num=int(f.read()) > ???? f.close() > > ???? f=open("backup/count_file.txt","w"); > ???? f.write(str(num+1)); > ???? f.close() > > ???? shutil.copy("my_file.txt", "my_file.txt.backup."+str(num)) > ???? time.sleep(900) > > > After running for a day, this had to be killed because it sucked up more > memory than anything I'd ever seen before. It's impossible to debug code that may or may not be the same as that shown. But there is certainly nothing in the code above that should eat memory. That having been said opening and closing the file like that is probably not the most efficient way of doing things. But for a test its probably OK. > I don't mind too much if its using a bit of memory, but, if python is > just going to leak all over the place and I'm better off learning c or c > sharp or something, then I'd like to know! Python certainly doesn't "leak all over the place" and is used for many industrial strength servers that run for months at a time. But if you don't clean up properly it can, of course, leak. If you can find the actual code you used and tell us the Python version and OS we might spot the problem. Otherwise I can only reiterate that Python might be suitable depending on your exact requirements. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Sun Jan 6 17:27:34 2019 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sun, 6 Jan 2019 22:27:34 +0000 Subject: [Tutor] running a game server In-Reply-To: References: Message-ID: <58f686e4-cf35-c9cf-84d3-420638038f79@yahoo.co.uk> On 06/01/2019 20:40, nathan tech wrote: > Hi Alan, > > Thanks for the swift response. > > Are there any places in python code I need to free variables? Its more about not hanging on to them by accident. For example putting them into a list and then deleting the original variable. The reference in the list keeps the object alive even though you think it's gone. But mostly in Python if you let things drop out of scope they will be garbage collected. > going into this, is there anything obvious I should know that are best > practices? Only the things that are common to any server implementation. Create single instances of any common data items before entering the infinite loop. If you expect high volumes look at using threads for each service request (or using async or concurrency instead of threading, depending on your work types) Also try to wrap up the services as pure functions with minimal side effects. Don't use global variables inside the functions if at all possible. Beyond that, it all depends what you are trying to do and what the expected volumes of both data and requests entail. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From steve at pearwood.info Sun Jan 6 18:41:19 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 7 Jan 2019 10:41:19 +1100 Subject: [Tutor] running a game server In-Reply-To: References: Message-ID: <20190106234118.GZ13616@ando.pearwood.info> On Sun, Jan 06, 2019 at 02:14:10PM +0000, nathan tech wrote: > My question is, is python really the way to go for game servers? *shrug* Do game servers have unique requirements that are different from (say) mailing list servers and other servers? That's not a rhetorical question. I don't know the answer. My home server runs at least three long-running Python processes (hpssd, whatever that is, fail2ban, and mailman), and I frequently run interactive Python sessions which stay open for a week at a time. It has been up for 62 days now, and the only reason that uptime isn't three times that is because I had a power failure. Now admittedly it is a home server with two users, not a public game server with 100,000 users, but still, this is evidence that Python does not leak memory. You might consider that Stackless is used as the game scripting engine for "Eve Online", and they do have hundreds of thousands of users. (Stackless is a version of Python that avoids the C stack.) > I remember, long ago, running a script that went along the lines of: > > ??? import shutil > ??? import time > ??? while 1: > ???? f=open("backup.count_file.txt","r"); > ???? num=int(f.read()) > ???? f.close() > ???? f=open("backup/count_file.txt","w"); > ???? f.write(str(num+1)); > ???? f.close() > ???? shutil.copy("my_file.txt", "my_file.txt.backup."+str(num)) > ???? time.sleep(900) > > > After running for a day, this had to be killed because it sucked up more > memory than anything I'd ever seen before. As Alan says, it is hard to debug code that may or may not be the actual code you ran, and there's no obvious reason why your code should leak. The first improvement I would make (aside from fixing the obvious typos, removing unnecessary semicolons, etc) is that there's no need to read the count from a file on every backup. You only need to read it once, when the script starts: import shutil import time f = open("backup.count_file.txt", "r") num = int(f.read() or '0') f.close() while True: num += 1 ???f = open("backup.count_file.txt", "w") ??? f.write(str(num)) ??? f.close() ??? shutil.copy("my_file.txt", "my_file.txt.backup." + str(num)) ??? time.sleep(900) I suppose it is possible that shutil.copy has, or had, a memory leak, but without knowing the actual code you used, the version of Python, and the platform, I don't propose trying to investigate any further. [...] > I don't mind too much if its using a bit of memory, but, if python is > just going to leak all over the place and I'm better off learning c or c > sharp or something, then I'd like to know! So you're a game developer, but you don't know Python, C, C# or "something". What language(s) do you know? P.S. please reply to the mailing list, not me personally. -- Steve From mhysnm1964 at gmail.com Sun Jan 6 21:38:35 2019 From: mhysnm1964 at gmail.com (mhysnm1964 at gmail.com) Date: Mon, 7 Jan 2019 13:38:35 +1100 Subject: [Tutor] Data pattern query. Message-ID: <008e01d4a632$183a8150$48af83f0$@gmail.com> All, I am currently writing a Python program to identify common text patterns in a excel spreadsheet which I have imported using openpyxl. All the descriptions of the transactions are in a single column. I am trying to work out the easiest method of identifying the same pattern of text in the fields. End of the day, I want a list of all the vendor transaction names. I don?t care if they are a credit or debit at this stage. Then I am going to group these vendors by categories. All this data has been downloaded from my bank originally. In the field, there is the vendor name, suburb/town, type of transaction, etc. I am only interested in the vendor name. So the vendor could be called ?a?, ?b?, ?c?, etc. As I don?t know all the different vendors. How can I teach the program to learn new vendor names? I was thinking of removing all the duplicate entries as a start. Was thinking of using dictionaries for this. But not sure if this is the best approach. I am really stuck and if there is a library out there which can do this for me. I would be grateful. ? Note, I am new to Python and not an advance programmer. Sean From ramprasad.kolla at gmail.com Sun Jan 6 21:15:27 2019 From: ramprasad.kolla at gmail.com (ram) Date: Sun, 6 Jan 2019 21:15:27 -0500 Subject: [Tutor] Bulk update to a SQL Server table Message-ID: Hi All, I have 7000+ records in a SQL Server table with 3 columns, which uniquely identify each record (I didn't define them as composite primary key). I load these 7000+ records into a Pandas DataFrame and update a subset of these records. What's the optimal way to update this subset of records to SQL table? I'm hesitant to using DataFrame.to_sql(...., if_exists="replace") as it removes unchanged records too and adds them alongwith the updated ones. Please advise. Thank you! Venkat Kolla From alan.gauld at yahoo.co.uk Mon Jan 7 04:19:58 2019 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Mon, 7 Jan 2019 09:19:58 +0000 Subject: [Tutor] Data pattern query. In-Reply-To: <008e01d4a632$183a8150$48af83f0$@gmail.com> References: <008e01d4a632$183a8150$48af83f0$@gmail.com> Message-ID: On 07/01/2019 02:38, mhysnm1964 at gmail.com wrote: > All the descriptions of the transactions are > in a single column. I am trying to work out the > easiest method of identifying the same pattern > of text in the fields. What does a singe column mean? That presumably is how it appears in the spreadsheet? But how is it stored in your Python code? A list? a list of lists? a dictionary? We don't know what your data looks like. Post a sample along with an explanation of how it is structured. In general when looking for patterns in text a regular expression is the tool of choice. But only if you know what the pattern looks like. Identifying patterns as you go is a much more difficult challenge > Then I am going to group these vendors by categories. And how do you categorize them? Is the category also in the data or is it some arbitrary thing that you have devised? > In the field, there is the vendor name, suburb/town, type of transaction, etc. etc is kind of vague! Show us some data and tel;l us which field is which. Without that its difficult to impossible to tell you how to extract anything! The important thing is not how it looked in the spreadsheet but how it looks now you have it in Python. > How can I teach the program to learn new vendor names? Usually you would use a set or dictionary and add new names as you find them. > I was thinking of removing all the duplicate entries Using a set would do that for you automatically > Was thinking of using dictionaries for this. > But not sure if this is the best approach. If you make the vendor name the key of a dictionary then it has the same effect as using a set. But whether a set or dict is best depends on what else you need to store. If its only the vendor names then a set is best. If you want to store associated data then a dict is better. You need to be much more specific about what your data looks like, how you identify the fields you want, and how you will categorize them. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Mon Jan 7 05:37:07 2019 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Mon, 7 Jan 2019 10:37:07 +0000 Subject: [Tutor] Bulk update to a SQL Server table In-Reply-To: References: Message-ID: On 07/01/2019 02:15, ram wrote: > I load these 7000+ records into a Pandas DataFrame and update a subset of > these records. > > What's the optimal way to update this subset of records to SQL table? First 7000 records is not a lot so performance shouldn't be a big issue even if you do update them all. But since its nice not to waste cycles there are (at least) two approaches: 1) tag your records in Pandas when they get updates then loop over the pandas frame updating the tagged records. 2) Use SQL to only update those records which have changes (since its only 3 fields the SQL should not be too difficult. I don't know SQL Server syntax but most major databases allow you to access the existing value from within in a SQL statement. A quick google suggests that it may be using the value attribute in SQL Server. Something like UPDATE.... WHERE field1.value <> Value1 OR field2.value <> Value2 OR field3.value <> value3 Do a bit of digging in your SQL reference you should find it. But it will be non portable since its not part of the SQL standard but a common vendor extension. Of the two approaches Python is easiest to debug but less scalable. For production use doing it on the server will be more efficient. For your volumes either should be fine. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From mousumi.nina.sahu1 at gmail.com Mon Jan 7 05:06:01 2019 From: mousumi.nina.sahu1 at gmail.com (mousumi sahu) Date: Mon, 7 Jan 2019 15:36:01 +0530 Subject: [Tutor] Python installtion Message-ID: Dear Sir, I am trying to install python 2.7.10 on HPC. Python 2.6 has already been install on root. I do not have root authority. Please suggest me how can I do this. From steve at pearwood.info Mon Jan 7 06:20:07 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 7 Jan 2019 22:20:07 +1100 Subject: [Tutor] Python installtion In-Reply-To: References: Message-ID: <20190107112007.GG13616@ando.pearwood.info> On Mon, Jan 07, 2019 at 03:36:01PM +0530, mousumi sahu wrote: > Dear Sir, > I am trying to install python 2.7.10 on HPC. Python 2.6 has already been > install on root. I do not have root authority. Please suggest me how can I > do this. What's HPC? If you don't have root permission, do you have permission to install anything? Are you comfortable with installing from source? You will need a C compiler. If everything works, it is easy, but if there are problems, it be difficult or impossible to solve them. Would it be easier to just ask your system administrator to install it for you? -- Steve From oscar.j.benjamin at gmail.com Mon Jan 7 07:17:41 2019 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 7 Jan 2019 12:17:41 +0000 Subject: [Tutor] Python installtion In-Reply-To: References: Message-ID: On Mon, 7 Jan 2019 at 11:10, mousumi sahu wrote: > > Dear Sir, > I am trying to install python 2.7.10 on HPC. Python 2.6 has already been > install on root. I do not have root authority. Please suggest me how can I > do this. Does HPC stand for High-Performance Computing? Are you trying to set up Python to run your code in a Linux super computer / compute cluster? You can compile Python yourself easily enough. First download Python. If the cluster has internet access you can do that with this: $ wget https://www.python.org/ftp/python/3.7.2/Python-3.7.2.tgz Then compile Python: $ tar -xzf Python-3.7.2.tgz $ cd Python-3.7.2 $ ./configure $ make $ ./python This can all take place in your user directory without needing root. -- Oscar From alan.gauld at yahoo.co.uk Mon Jan 7 10:49:22 2019 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Mon, 07 Jan 2019 15:49:22 +0000 Subject: [Tutor] Fwd: Data pattern query. Message-ID: <1kbnf68oe3g0f3qc3q23kk2f.1546876162464@email.android.com> Sending to list. Please use reply-all when responding to the list. -------- Original Message -------- Subject: RE: [Tutor] Data pattern query. From: mhysnm1964 at gmail.com Sent: Monday, 7 January 2019 11:58 To: 'Alan Gauld' CC: Alan, Thanks for responding. I am storing the spreadsheet into a dictionary using a two dimension list. The key for the dictionary is the sheet name. Possibly in the future I might just have a simple list. 6 columns across with variable rows. Max is over 1000 rows. Below is the code: import openpyxl # module to import spreadsheet. # module found on the net to remove duplicates. from more_itertools import unique_everseen # Loding the workbook and names for the sheets. wb = openpyxl.load_workbook("test.xlsx") # Storing sheet names into a list for the loop. wss = wb.get_sheet_names() # defining variables for data structure transaction = {} # dictionary of bank transactions. description = [] # the descriptions of transactions for ws_name in wss: # sheet being worked upon. ws = wb.get_sheet_by_name(ws_name) data = []# temp variable to store 2 dimension list. for column in ws.columns: c = [] # temp variable for current column for cell in column: c.append(cell.value) # is this correct? # end for cell loop if c[0] == "Description": #Populating the description list description.extend(c) data.append(c) # creating two dimension list # end for for column # creating dictionary with 2 dimension list transaction[ws_name] = data #end for /*** the below code is not working ***/ If I sort the description list. I get untyped errors. When I use the debug. The items (elements) in the description list at the end are 'None" value. Not sure if this is due to the location of defining the empty list or the method of using append with the list c. If the below code could sort, then the remaining lines would perform the duplicate on the whole value of the item. While I want to identify duplicates with the vendor name only. description.sort() for result in unique_everseen (description): print (result) Some example data structure. I have modified for privacy reasons. PAY/SALARY FROM vendor-name EFTPOS vendor-name ####HORNSBY country-code bank-name BANKING FUNDS TFER TRANSFER ###### TO ###### vendor-name suburb state vendor-name #### suburb DEPOSIT vendor-name PAYMENT ######## state-name WITHDRAWAL BY EFTPOS #### vendor-name #### suburb 12/06 DEPOSIT-vendor-name Aust organisation_name ######### -----Original Message----- From: Tutor On Behalf Of Alan Gauld via Tutor Sent: Monday, 7 January 2019 8:20 PM To: tutor at python.org Subject: Re: [Tutor] Data pattern query. On 07/01/2019 02:38, mhysnm1964 at gmail.com wrote: > All the descriptions of the transactions are in a single column. I am > trying to work out the easiest method of identifying the same pattern > of text in the fields. What does a singe column mean? That presumably is how it appears in the spreadsheet? But how is it stored in your Python code? A list? a list of lists? a dictionary? We don't know what your data looks like. Post a sample along with an explanation of how it is structured. In general when looking for patterns in text a regular expression is the tool of choice. But only if you know what the pattern looks like. Identifying patterns as you go is a much more difficult challenge > Then I am going to group these vendors by categories. And how do you categorize them? Is the category also in the data or is it some arbitrary thing that you have devised? > In the field, there is the vendor name, suburb/town, type of transaction, etc. etc is kind of vague! Show us some data and tel;l us which field is which. Without that its difficult to impossible to tell you how to extract anything! The important thing is not how it looked in the spreadsheet but how it looks now you have it in Python. > How can I teach the program to learn new vendor names? Usually you would use a set or dictionary and add new names as you find them. > I was thinking of removing all the duplicate entries Using a set would do that for you automatically > Was thinking of using dictionaries for this. > But not sure if this is the best approach. If you make the vendor name the key of a dictionary then it has the same effect as using a set. But whether a set or dict is best depends on what else you need to store. If its only the vendor names then a set is best. If you want to store associated data then a dict is better. You need to be much more specific about what your data looks like, how you identify the fields you want, and how you will categorize them. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos _______________________________________________ Tutor maillist - Tutor at python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor From amitsyadav1999 at gmail.com Mon Jan 7 11:29:31 2019 From: amitsyadav1999 at gmail.com (Amit Yadav) Date: Mon, 7 Jan 2019 21:59:31 +0530 Subject: [Tutor] Doubt Message-ID: How can simply typing print "hello world" work? Like without including any header file or import statements how can it work. I?m protected online with Avast Free Antivirus. Get it here ? it?s free forever. <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2> From mats at wichmann.us Mon Jan 7 14:08:46 2019 From: mats at wichmann.us (Mats Wichmann) Date: Mon, 7 Jan 2019 12:08:46 -0700 Subject: [Tutor] Doubt In-Reply-To: References: Message-ID: On 1/7/19 9:29 AM, Amit Yadav wrote: > How can simply typing > > print "hello world" > > work? > Like without including any header file or import statements how can it work. because you're using a function (or in your case - which looks like Python 2.x because it is not written as a function call - a statement) which is part of the language, there is no need to import anything. From __peter__ at web.de Mon Jan 7 15:43:51 2019 From: __peter__ at web.de (Peter Otten) Date: Mon, 07 Jan 2019 21:43:51 +0100 Subject: [Tutor] Doubt References: Message-ID: Amit Yadav wrote: > How can simply typing > > print "hello world" > > work? > Like without including any header file or import statements how can it > work. In Python 2 print is part of the syntax that the compiler knows, just like int or for (... ) {} in C. In Python 3 print is just a name, like len, say, in both py2 and py3. A name on the module level is looked up first in the module's namespace, and if it's not found in another special namespace that is built into the interpreter: >>> __builtins__ >>> __builtins__.len is len True The unqualified name "len" is the same object as __builtins__.len because it is looked up there. When you overwrite it with your own name this will take precedence: >>> def len(x): return 42 ... >>> __builtins__.len is len False >>> len("abc") 42 OK that was probably a bad idea, let's remove our len() to see the built-in again: >>> del len >>> len("abc") 3 The built-in namespace is user-writeable: >>> __builtins__.spam = "ham" >>> spam 'ham' From steve at pearwood.info Mon Jan 7 19:26:29 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 8 Jan 2019 11:26:29 +1100 Subject: [Tutor] Doubt In-Reply-To: References: Message-ID: <20190108002628.GL13616@ando.pearwood.info> On Mon, Jan 07, 2019 at 09:59:31PM +0530, Amit Yadav wrote: > How can simply typing > > print "hello world" > > work? > Like without including any header file or import statements how can it work. Why shouldn't it work? Python is not C and doesn't use header files. In Python, the interpreter knows the meaning of print, just as it knows the meaning of strings ("abc") and ints (123) and floats (4.56). Some operations (keywords, operators like + - * etc, a handful of special constants like None) are known to the interpreter. Other functions (len, chr, ord, min, max etc) are in the "builtins" module, which doesn't need to be imported because it is always available. And some functions are in additional modules which need to be imported. -- Steve From arj.python at gmail.com Mon Jan 7 20:47:07 2019 From: arj.python at gmail.com (Abdur-Rahmaan Janhangeer) Date: Tue, 8 Jan 2019 05:47:07 +0400 Subject: [Tutor] Doubt In-Reply-To: References: Message-ID: it's because of how the language works. c is a compiled lang. py an interpreted one ( more of a hybrid i would say ) if you write a language in c like if a == 5: do this then run your c program to execute it, you get pretty much what python is Abdur-Rahmaan Janhangeer http://www.pythonmembers.club | https://github.com/Abdur-rahmaanJ Mauritius From asad.hasan2004 at gmail.com Fri Jan 11 09:12:07 2019 From: asad.hasan2004 at gmail.com (Asad) Date: Fri, 11 Jan 2019 19:42:07 +0530 Subject: [Tutor] Log file for to search in order In-Reply-To: References: Message-ID: Hi All , Let me answer : > Where are you looking? *Anywhere* in the log file? >> Yes anywhere in the log file . > Only in the immediate next line? Anywhere forward of the "patch" line? >> Anywhere forward from the line which matches the search > You are looking for two patterns, but does the order they appear, or the > distance apart, matter? > > Yes the order does matter : 1) first it looks for : \?/patch/\d{8}/\d{8}/admin/load.sql - Condition if there are 10 lines containing the same pattern then it should save and print the last line containing the pattern means the 10th line - if it find the above string it start searching from that line with successful match regex match until it gets to the line containing set_metadata - Once it get the line containing set_metadata print last 4 lines prior to the line containing set_metadata 2) if it doesnot find the above pattern look for second in order :Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8} - Condition if there are 10 lines containing the same pattern then it should save and print the last line containing the pattern means the 10th line - if it find the above string it start searching from that line with successful match regex match until it gets to the line containing set_metadata - Once it get the line containing set_metadata print last 10 lines prior to the line containing set_metadata 3) if it doesnot match the pattern: \?/patch/\d{8}/\d{8}/admin/load.sql or '\?/patch/\d{8}/\d{8}/admin/load.sql' anywhere in the log file . print "No match found refer to install guide" 4) If I have two errors how do I prioritize one above other ? Thanks, > > ---------- Forwarded message ---------- > From: Asad > To: tutor at python.org > Cc: > Bcc: > Date: Wed, 2 Jan 2019 20:39:53 +0530 > Subject: [Tutor] Log file for Nested if-elif > Hi All , > > Need advice on the following piece of code : > > with open(r"file1.log", 'r') as f: > tail = deque(maxlen=8) # the last eight lines > script = None > for line in f: > tail.append(line) > if > re.search('\?/patch/\d{8}/\d{8}/admin/load.sql',line,re.IGNORECASE): > script = line > elif re.search(r'Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8}', > line, re.IGNORECASE): > script = line > elif re.search(r'set_metadata', line ,re.IGNORECASE) is not None: > print "Reason of error \n", tail[-1] > print "Script:\n", script > print "Block of code:\n" > for item in tail: > print item > print " Danger " > break > Now this is printing the last cached line in the variable line . However > I would like to see the following output : > > 1) if it matches the pattern: \?/patch/\d{8}/\d{8}/admin/load.sql then > > look for the line "set_metadata" in the file1.log if it finds the pattern > then print the line which matches the pattern > \?/patch/\d{8}/\d{8}/admin/load.sql > > print last 4 lines of the tail array and exit > > 2) if it doesnot match '\?/patch/\d{8}/\d{8}/admin/load.sql' > > then look of the anothern pattern > :Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8} if it find the pattern > > then look for line "set_metadata" in the file1.log if it finds the pattern > then print the line which matches the pattern > \?/patch/\d{8}/\d{8}/admin/load.sql > > print all the lines in tail > > print a recommendation "Please check the installation" > > > 3 ) if it doesnot match the pattern: \?/patch/\d{8}/\d{8}/admin/load.sql > or '\?/patch/\d{8}/\d{8}/admin/load.sql' > > print "No match found refer to install guide" > > Can you advice what I can do to change the code . > > Thanks, > -- > > > > > ---------- Forwarded message ---------- > From: "Steven D'Aprano" > To: tutor at python.org > Cc: > Bcc: > Date: Thu, 3 Jan 2019 15:16:45 +1100 > Subject: Re: [Tutor] Log file for Nested if-elif > On Wed, Jan 02, 2019 at 08:39:53PM +0530, Asad wrote: > > Hi All , > > > > Need advice on the following piece of code : > > Let me write it in a more "Pythonic" style: > > > > PATCH = r'\?/patch/\d{8}/\d{8}/admin/load.sql' > APPLY = r'Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8}' > ERROR = r'set_metadata' > > tail = deque(maxlen=8) # the last eight lines > script = None > with open("file1.log", 'r') as f: > for line in f: > tail.append(line) > if (re.search(PATCH, line, re.IGNORECASE) > or re.search(APPLY, line, re.IGNORECASE): > script = line > elif re.search(ERROR, line, re.IGNORECASE): > print "Reason for error \n", line > print "Script:", script > print "Tail:\n", tail > print " Danger " # Seriously? This is dangerous? > break > > > > Now this is printing the last cached line in the variable line . > However > > I would like to see the following output : > > > > 1) if it matches the pattern: \?/patch/\d{8}/\d{8}/admin/load.sql then > > look for the line "set_metadata" in the file1.log if it finds the > pattern > > then print the line which matches the pattern > > \?/patch/\d{8}/\d{8}/admin/load.sql > > Where are you looking? *Anywhere* in the log file? Only in the > immediate next line? Anywhere forward of the "patch" line? You are > looking for two patterns, but does the order they appear, or the > distance apart, matter? > > > blah blah blah set_metadata blah blah > ... 100 lines ... > blah blah blah /patch/ ... admin/load.sql > ... 100 lines ... > > Is that a match? > > > print last 4 lines of the tail array and exit > > print tail[-4:] > sys.exit() > > > > > 2) if it doesnot match '\?/patch/\d{8}/\d{8}/admin/load.sql' > > > > then look of the anothern pattern > > :Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8} if it find the pattern > > > > then look for line "set_metadata" in the file1.log if it finds the > pattern > > then print the line which matches the pattern > > \?/patch/\d{8}/\d{8}/admin/load.sql > > This is a contradiction: you just said that it DOESN'T match the > patch...load.sql pattern, but now you want it to print the line that > matched. There is no line that matches! > > > 3 ) if it doesnot match the pattern: \?/patch/\d{8}/\d{8}/admin/load.sql > > or '\?/patch/\d{8}/\d{8}/admin/load.sql' > > > > print "No match found refer to install guide" > > > > Can you advice what I can do to change the code . > > > > You need to tighten the specifications to make it more clear what you > are trying to do. > > > > -- > Steve > > > > > ---------- Forwarded message ---------- > From: Peter Otten <__peter__ at web.de> > To: tutor at python.org > Cc: > Bcc: > Date: Thu, 03 Jan 2019 12:11:08 +0100 > Subject: Re: [Tutor] Log file for Nested if-elif > Asad wrote: > > > Hi All , > > > > Need advice on the following piece of code : > > > > with open(r"file1.log", 'r') as f: > > tail = deque(maxlen=8) # the last eight lines > > script = None > > for line in f: > > tail.append(line) > > if > > re.search('\?/patch/\d{8}/\d{8}/admin/load.sql',line,re.IGNORECASE): > > script = line > > elif re.search(r'Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8}', > > line, re.IGNORECASE): > > script = line > > elif re.search(r'set_metadata', line ,re.IGNORECASE) is not None: > > print "Reason of error \n", tail[-1] > > print "Script:\n", script > > print "Block of code:\n" > > for item in tail: > > print item > > print " Danger " > > break > > Now this is printing the last cached line in the variable line . > However > > I would like to see the following output : > > > > 1) if it matches the pattern: \?/patch/\d{8}/\d{8}/admin/load.sql then > > > > look for the line "set_metadata" in the file1.log if it finds the > pattern > > then print the line which matches the pattern > > \?/patch/\d{8}/\d{8}/admin/load.sql > > > > print last 4 lines of the tail array and exit > > > > 2) if it doesnot match '\?/patch/\d{8}/\d{8}/admin/load.sql' > > > > then look of the anothern pattern > > :Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8} if it find the pattern > > > > then look for line "set_metadata" in the file1.log if it finds the > > pattern then print the line which matches the pattern > > \?/patch/\d{8}/\d{8}/admin/load.sql > > > > print all the lines in tail > > > > print a recommendation "Please check the installation" > > > > > > 3 ) if it doesnot match the pattern: \?/patch/\d{8}/\d{8}/admin/load.sql > > or '\?/patch/\d{8}/\d{8}/admin/load.sql' > > > > print "No match found refer to install guide" > > > > Can you advice what I can do to change the code . > > You have "nested" in the subject, do not make an attempt to structure your > loop. Why is that? > > In my experience using lots of small functions makes code easier to > understand. Below is a suggestion how you might break up your code. > > # untested; expect a few bugs! > def with_tail(items, maxlen): > tail = deque(maxlen=maxlen) > > def gen_items(): > for item in items: > tail.append(item) > yield item > > return tail, gen_items() > > > def search_end_of_section(lines): > for line in lines: > if re.search(r'set_metadata', line, re.IGNORECASE) is not None: > break > else: > raise ValueError("Reached end of file...panic!") > > > def dump(script, tail, advice=None): > print "Reason of error \n", tail[-1] > print "Script:\n", script > print "Block of code:\n" > for item in tail[-limit:]: > print item > print " Danger " > if advice is not None: > print advice > > > RE_PATCH = re.compile(r'\?/patch/\d{8}/\d{8}/admin/load.sql', > re.IGNORECASE) > RE_APPLY = re.compile( > r'Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8}', re.IGNORECASE > ) > > with open(r"file1.log", 'r') as f: > tail, lines = with_tail(f) > for line in lines: > if RE_PATCH.search(line) is not None: > search_end_of_section(lines) > dump(script=line, tail=tail[-4:]) > break > elif RE_APPLY.search(line) is not None: > search_end_of_section(lines) > dump( > script=line, tail=tail, > advice="Please check the installation" > ) > break > > > > _______________________________________________ > Tutor maillist - Tutor at python.org > https://mail.python.org/mailman/listinfo/tutor > -- Asad Hasan +91 9582111698 From alan.gauld at yahoo.co.uk Fri Jan 11 18:58:24 2019 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Fri, 11 Jan 2019 23:58:24 +0000 Subject: [Tutor] Log file for to search in order In-Reply-To: References: Message-ID: On 11/01/2019 14:12, Asad wrote: Let metry and translate your description into pseudo code... > >> Anywhere forward from the line which matches the search > > >> You are looking for two patterns, but does the order they appear, or the >> distance apart, matter? >> >> Yes the order does matter : > > 1) first it looks for : \?/patch/\d{8}/\d{8}/admin/load.sql > - Condition if there are 10 lines containing the same pattern > then it should save and print the last line containing the pattern means > the 10th line > - if it find the above string it start searching from that line > with successful match regex match until it gets to the line containing > set_metadata > - Once it get the line containing set_metadata print last 4 > lines prior to the line containing set_metadata start = False lines = [] with open(fname) as infile: for line in infile: lines.append(line) # or use a circular list size 4 if hasPattern(patt1,line): start = True break if start and hasPattern(patt2, line): print lines[-4:] Is that right so far? > 2) if it doesnot find the above pattern look for second in order > :Starting\s+apply\s+for\s+patch\s+\d{8}/\d{8} > - Condition if there are 10 lines containing the same pattern > then it should save and print the last line containing the pattern means > the 10th line > - if it find the above string it start searching from that line > with successful match regex match until it gets to the line containing > set_metadata > - Once it get the line containing set_metadata print last 10 > lines prior to the line containing set_metadata This sounds like the same thing except with a different start pattern and a different number of lines printed. So may be its a function? The only thing I'm not clear on is the comment "if it does not find the above pattern" Which pattern, there are two... do you mean the first one? And if so do you need to search the entire file for the second pattern? If so you need to repeat the search fuinction twice: found = False with open(fname) as infile: found = searchFile(infile, pattern1, buffsize=4) if not found: infile.seek(0) # go back to beginning of file found = searchFile(infile,pattern2, buffsize=10) > 3) if it doesnot match the pattern: > or anywhere in the log file . > > print "No match found refer to install guide" if not found: print .... Does the above look like what you want? > 4) If I have two errors how do I prioritize one above other ? I have no idea. How do you prioritize them? That's a requirement issue not a solution issue. Once you tell us how you want to prioritize them we can consider possible programming options. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From joseph.gulizia at gmail.com Sat Jan 12 10:44:21 2019 From: joseph.gulizia at gmail.com (Joseph Gulizia) Date: Sat, 12 Jan 2019 09:44:21 -0600 Subject: [Tutor] Best way to write this countdown code Message-ID: Thanks in advance as I've gotten wordy. I want to integrate the following working code into a website: beef_quantity = 28 ## Set before campaign starts. print('Original Beef Quantity: ') ## Displays the wording "Original Beef Quantity: " print (beef_quantity) ## Displays the beef quantity before order beef_choice = 'Quarter_Beef' ## Based on Customer's choice if beef_choice == 'Quarter_Beef': print('Quarter Beef selected') ## Notice of beef quantity ordered print('New Beef Quantity: ') ## Displays the wording "New beef quantity: " new_beef_quantity = beef_quantity - 1 ## Updates the beef quantity after order print(new_beef_quantity) ## Displays the updated beef quantity after order elif beef_choice == 'Half_Beef': print('Half Beef selected') ## Notice of beef quantity ordered print('New Beef Quantity: ') ## Displays the wording "New beef quantity: " new_beef_quantity = beef_quantity - 2 ## Updates the beef quantity after order print(new_beef_quantity) ## Displays the updated beef quantity after order else: print('Whole Beef selected') ## Notice of beef quantity ordered print('New Beef Quantity: ') ## Displays the wording "New beef quantity: " new_beef_quantity = beef_quantity - 4 ## Updates the beef quantity after order print(new_beef_quantity) ## Displays the updated beef quantity after order I also have two other similar code pieces (delivery.py and locker.py) that I want to integrate, but I'll put them in other emails. My end goal (which I still have to figure out) is to have the customer click on three sections of radio buttons and enter two text fields which will then display in an alert box where they can click a button to send them to PayPal for payment. The other part of this is that the number of available beef will countdown as each order is placed until zero s available which will then display "Sold Out!" Not sure if this is ALL python or part python and part javascript. Again, Thank you. Joe From steve at pearwood.info Sat Jan 12 11:48:30 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 13 Jan 2019 03:48:30 +1100 Subject: [Tutor] Best way to write this countdown code In-Reply-To: References: Message-ID: <20190112164829.GZ13616@ando.pearwood.info> On Sat, Jan 12, 2019 at 09:44:21AM -0600, Joseph Gulizia wrote: > Thanks in advance as I've gotten wordy. > > I want to integrate the following working code into a website: [...] Start with a simpler question: how would you integrate *any* Python code into a website? Few (i.e. no) browsers directly support Python. This is why nearly all client-side (running in the brower) web development is done in Javascript. Server-side code can be written in Python, or any other language (Perl and PHP are other common choices) but for code that runs in the browser, it is difficult to use anything but Javascript. But all is not quite lost: http://www.skulpt.org/ https://duckduckgo.com/?q=python+in+the+browser Start by getting a simple "Hello World" type program running in the browser, and then move on to something more ambitious. -- Steve From alan.gauld at yahoo.co.uk Sat Jan 12 12:47:46 2019 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sat, 12 Jan 2019 17:47:46 +0000 Subject: [Tutor] Best way to write this countdown code In-Reply-To: References: Message-ID: On 12/01/2019 15:44, Joseph Gulizia wrote: > I want to integrate the following working code into a website: First thing to note is that web sites speak HTML, so anything you output for display needs to be in HTML not plain text. (Plain text will usually be understood by a browser but will look terrible!) The normal way to build a web app is to create an HTML template file with place holders for the data. Then use Python (or any other server language) to inject the data via some kind of web framework. (Flask and Bottle are two simple Python examples) An alternative technique involves writing the app in JavaScript that executes on the browser. That means creating an HTML file that either contains your app as embedded JavaScript or putting your JavaScript code into a separate file and importing it into the HTML. The latter approach is preferred for industrial sites but the single file approach is OK for small applications. Looking at your code we can remove all the lines that should be in your HTML file: > beef_quantity = 28 ## Set before campaign starts. beef_choice = "" > if beef_choice == 'Quarter_Beef': > new_beef_quantity = beef_quantity - 1 > elif beef_choice == 'Half_Beef': > new_beef_quantity = beef_quantity - 2 > else: > new_beef_quantity = beef_quantity - 4 The setting of the data values into the HTML will depend on your framework. > My end goal (which I still have to figure out) is to have the customer > click on three sections of radio buttons and enter two text fields > which will then display in an alert box where they can click a button That's all HTML function not Python or even JavaScript > to send them to PayPal for payment. Sending to Paypal is a whole new ball game. That requires quite a bit of extra work. > the number of available beef will countdown as each order is placed > until zero s available which will then display "Sold Out!" Not sure > if this is ALL python or part python and part javascript. It could be either or both. But given your need to send to Paypal I'd suggest you find a framework and work in Python. (Especially since you seem to know some Python) You probably should read the section in my tutorial(or any other) that covers web applications. It's in the last section starting with the topic entitled "Working with the Web" and moving on to "Using Web Application Frameworks" HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From mhysnm1964 at gmail.com Sat Jan 12 21:16:10 2019 From: mhysnm1964 at gmail.com (mhysnm1964 at gmail.com) Date: Sun, 13 Jan 2019 13:16:10 +1100 Subject: [Tutor] Debugging a sort error. Message-ID: <002801d4aae5$f5223740$df66a5c0$@gmail.com> Hello everyone. I am hoping someone can help with the below error using Python3.5 in the Windows 10 bash environment. I found the below link which I am not sure if this is related to the issue or not. As I don't fully understand the answer. https://github.com/SethMMorton/natsort/issues/7 Issue, following error is generated after trying to sort a list of strings. description.sort() TypeError: unorderable types: float() < str() Each list items (elements) contain a mixture of alpha chars, numbers, punctuation chars like / and are in a string type. Below is an example extract of the data from the list. ['Description', 'EFTPOS WOOLWORTHS 1294 ", "withdrawal Hudson street 3219"] There is over 2000 such entries. This used to work and now doesn't. Only change to the code was modifying the data variable from a list to a dictionary. Below is the full code: import openpyxl from openpyxl.utils import get_column_letter from more_itertools import unique_everseen # Loding the workbook and names for the sheets. wb = openpyxl.load_workbook("test.xlsx") wss = wb.get_sheet_names() description = [] # all the descriptions data = {} for ws_name in wss: ws = wb.get_sheet_by_name(ws_name) for column in ws.columns: col_data = [] # column list for cell in column: value = cell.value col_data.append(value) # end for cell loop if ws_name == "WestPac": if col_data[0] == 'Credit Amount': num = 1 while num <= (len(col_data) - 1): value = col_data[num] if value: data['Amount'][num] = value else: data['Amount'][num] = data['Amount'][num] * -1 num += 1 # end while num break # end if # end if if col_data[0] in data: data[col_data[0]].extend(col_data[1:-1]) else: data[col_data[0]] = col_data # end for column #end for ws_name description = data['Description'] for i in description: if not str(i): print "not a string") description.sort() I am suspecting it is something to do with the data but cannot track down the cause. Any suggestions on how to debug this? Sean From alan.gauld at yahoo.co.uk Sun Jan 13 03:55:24 2019 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sun, 13 Jan 2019 08:55:24 +0000 Subject: [Tutor] Debugging a sort error. In-Reply-To: <002801d4aae5$f5223740$df66a5c0$@gmail.com> References: <002801d4aae5$f5223740$df66a5c0$@gmail.com> Message-ID: On 13/01/2019 02:16, mhysnm1964 at gmail.com wrote: > Issue, following error is generated after trying to sort a list of strings. > > description.sort() > TypeError: unorderable types: float() < str() Please send the complete error message not just the last line summary. There is a lot of potentially useful information in the missing lines. > ['Description', 'EFTPOS WOOLWORTHS 1294 ", "withdrawal Hudson > street 3219"] That doesn't look right. The quotes are all mixed up and miss-matched. Can you send an actual example? > There is over 2000 such entries. This used to work and now doesn't. Only > change to the code was modifying the data variable from a list to a > dictionary. The data variable appears to only have one entry, namely "Amount". Is that what you intended? Below is the full code: > > > > import openpyxl > > from openpyxl.utils import get_column_letter > > from more_itertools import unique_everseen > > > > # Loding the workbook and names for the sheets. > > wb = openpyxl.load_workbook("test.xlsx") > > wss = wb.get_sheet_names() > > > > description = [] # all the descriptions > > data = {} > > > > for ws_name in wss: > > ws = wb.get_sheet_by_name(ws_name) > > for column in ws.columns: > > col_data = [] # column list > > for cell in column: > > value = cell.value > > col_data.append(value) > > # end for cell loop > > if ws_name == "WestPac": > > if col_data[0] == 'Credit Amount': > > num = 1 > > while num <= > (len(col_data) - 1): > > value = > col_data[num] > > if > value: > > > data['Amount'][num] = value > > else: > > > data['Amount'][num] = data['Amount'][num] * -1 > > num += > 1 > > # end while num > > break > > # end if > > # end if > > if col_data[0] in data: > > > data[col_data[0]].extend(col_data[1:-1]) > > else: > > data[col_data[0]] = col_data > > # end for column > > #end for ws_name > > > > description = data['Description'] You try to assign data["Description"] but you never populated that in the code above. You only ever used data["Amount"]. > for i in description: > if not str(i): > print "not a string") I'm not sure what you think that's doing but the print should give a syntax error if you are using Python 3.5 as you claim. You need parens around it. Can you cut n paste the actual code? Also, it is unlikely to ever print anything because most things can be converted to a string. However, given that description is supposed to contain lists of values I suspect you need another inner loop to convert the list contents. I think you wanted something like: for lst in description: for i in lst: if type(i) is not str: print(i, "is not a string") > description.sort() > > I am suspecting it is something to do with the data but cannot track down > the cause. Any suggestions on how to debug this? Send the full error and some genuine data and try the modified type test loop above. Oh, and using smaller indentation will make it more readable too. 3-4 spaces is the usual size. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From cs at cskk.id.au Sun Jan 13 04:11:48 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Sun, 13 Jan 2019 20:11:48 +1100 Subject: [Tutor] Debugging a sort error. In-Reply-To: <002801d4aae5$f5223740$df66a5c0$@gmail.com> References: <002801d4aae5$f5223740$df66a5c0$@gmail.com> Message-ID: <20190113091148.GA57585@cskk.homeip.net> Discussion inline below. On 13Jan2019 13:16, mhysnm1964 at gmail.com wrote: >I am hoping someone can help with the below error using Python3.5 in >the Windows 10 bash environment. I found the below link which I am not sure if >this is related to the issue or not. As I don't fully understand the answer. > >https://github.com/SethMMorton/natsort/issues/7 I'm not sure that URL is very relevant, except to the extent that it points out that Python 3 issues an error when comparing incomparable types. In Python 2 this problem could go unnoticed, and that just leads to problems later, much farther from the source of the issue. >Issue, following error is generated after trying to sort a list of strings. > >description.sort() >TypeError: unorderable types: float() < str() > >Each list items (elements) contain a mixture of alpha chars, numbers, >punctuation chars like / and are in a string type. Below is an example >extract of the data from the list. > >['Description', 'EFTPOS WOOLWORTHS 1294 ", "withdrawal Hudson >street 3219"] The error message says that some of these values are not strings. One at least is a float. My expectation is that the openpyxl module is reading a floating point value into your description array. This might be openpxyl being too clever, or it might be (more likely IMO) be Excel turning something that looked like a float into a float. Spreadsheets can be ... helpful like that. >There is over 2000 such entries. This used to work and now doesn't. You'll need to examine the values. But I see that you're trying to do this. I've snipped the data loading phase. Here: >description = data['Description'] >for i in description: > if not str(i): > print "not a string") This is not a valid check that "i" is a string. That expression: str(i) tries to convert "i" into a string (via its __str__ method). Most objects have such a method, and str() of a float is the textual representation of the float. So the if statement doesn't test what you want to test. Try this: if not isinstance(i, str): print("not a string:", i, type(i)) >description.sort() >I am suspecting it is something to do with the data but cannot track >down the cause. Any suggestions on how to debug this? Your exception is in here, but as you expect you want to inspect the description types first. If the description column does contain a float in the original data then you could convert it to a string first! Note that this may not match visually what was in the spreadsheet. (BTW, your cited code never fills out the description list, not it cannot be current.) But first, fine out what's wrong. Try the type test I suggest and see how far you get. Cheers, Cameron Simpson From steve at pearwood.info Sun Jan 13 05:34:24 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 13 Jan 2019 21:34:24 +1100 Subject: [Tutor] Debugging a sort error. In-Reply-To: <002801d4aae5$f5223740$df66a5c0$@gmail.com> References: <002801d4aae5$f5223740$df66a5c0$@gmail.com> Message-ID: <20190113103423.GA13616@ando.pearwood.info> On Sun, Jan 13, 2019 at 01:16:10PM +1100, mhysnm1964 at gmail.com wrote: > Issue, following error is generated after trying to sort a list of strings. > > description.sort() > TypeError: unorderable types: float() < str() That tells you that you don't have a list of strings. You have a list of strings with at least one float mixed in. > There is over 2000 such entries. This used to work and now doesn't. Only > change to the code was modifying the data variable from a list to a > dictionary. I doubt that. Anytime people say "nothing changed except..." it invariably turns out to be "oh yeah, these other ten things changed too..." *wink* > Below is the full code: Not it isn't, because the code you supply has at least one syntax error. So it is impossible for it to be the actual code you are running. In any case, we are volunteers, not paid consultants. If you want to pay us $200 an hour to debug your code, you get to dump a big ball of mud in our laps and say "fix it". But as volunteers, we have no obligation to trawl through your code trying to debug it (unless it looks interesting, or easy, or we're bored and looking for something to do...) To maximize your chances of people actually helping, please read this: http://www.sscce.org/ Having said that, I have spotted one obvious bug: > description = data['Description'] > > for i in description: > if not str(i): > print "not a string") (See what I mean about a syntax error?) That does not do what you think it does. My *guess* is that you are trying to check that every item in description is a string. But that's not what you do: you run through the items, make a string copy of each one (regardless of what kind of object it is) and then test whether the string is not the empty string (which it never will be, unless the original was the empty string). Instead, try this: for item in description: if not isinstance(item, str): print(type(item), repr(item), "is not a string") -- Steve From __peter__ at web.de Sun Jan 13 05:59:54 2019 From: __peter__ at web.de (Peter Otten) Date: Sun, 13 Jan 2019 11:59:54 +0100 Subject: [Tutor] Debugging a sort error. References: <002801d4aae5$f5223740$df66a5c0$@gmail.com> Message-ID: mhysnm1964 at gmail.com wrote: > Issue, following error is generated after trying to sort a list of > strings. > > description.sort() > TypeError: unorderable types: float() < str() Consider >>> descriptions = ["foo", "bar", 123, 3.14, 42, 200.1, "0"] >>> sorted(descriptions) Traceback (most recent call last): File "", line 1, in TypeError: unorderable types: int() < str() If there are only numbers and strings in the list you can force the sort to succeed with the following custom key function: >>> def key(item): ... return isinstance(item, str), item ... This will move the numbers to the beginning of the list: >>> sorted(descriptions, key=key) [3.14, 42, 123, 200.1, '0', 'bar', 'foo'] From sanelson at gmail.com Sun Jan 13 09:15:08 2019 From: sanelson at gmail.com (Stephen Nelson-Smith) Date: Sun, 13 Jan 2019 14:15:08 +0000 Subject: [Tutor] Debugging a sort error. In-Reply-To: <002801d4aae5$f5223740$df66a5c0$@gmail.com> References: <002801d4aae5$f5223740$df66a5c0$@gmail.com> Message-ID: Hi, On Sun, Jan 13, 2019 at 8:34 AM wrote: > description.sort() > TypeError: unorderable types: float() < str() So, fairly obviously, we can't test whether a float is less than a string. Any more than we can tell if a grapefruit is faster than a cheetah. So there must be items in description that are strings and floats. With 2000 lines, you're going to struggle to eyeball this, so try something like this: In [69]: irrational_numbers = [3.14159265, 1.606695, "pi", "Pythagoras Constant"] In [70]: from collections import Counter In [71]: dict(Counter([type(e) for e in irrational_numbers])) Out[71]: {float: 2, str: 2} If with your data, this shows only strings, I'll eat my hat. S. From sanelson at gmail.com Sun Jan 13 09:22:15 2019 From: sanelson at gmail.com (Stephen Nelson-Smith) Date: Sun, 13 Jan 2019 14:22:15 +0000 Subject: [Tutor] Python installtion In-Reply-To: References: Message-ID: Hi, On Mon, Jan 7, 2019 at 11:11 AM mousumi sahu wrote: > > Dear Sir, > I am trying to install python 2.7.10 on HPC. Python 2.6 has already been > install on root. I do not have root authority. Please suggest me how can I > do this. Sorry - I replied to you directly, by accident. Take 2, with reply all: You need to do a local installation of Python, and set up your system to use that in preference to the one at the system level. Although it's possible to do this with various manual steps, there's a really handy tool you can use which will make your life easier, and allow you to manage multiple versions of Python, which might be useful, if you wanted, say, to be able to run both Python 2 and Python 3. The tool is called `pyenv`, and as long as you have a bash/zsh shell, and your system has a C compiler and associated tools already installed, you can install and use it. The simplest approach is to clone the tool it from git, modify your shell to use it, and then use it to install Python. Here's a sample way to set it up. This won't necessarily match your exact requirements, but you can try it, and please come back if you have any further questions: 1. Clone the git repo into your home directory git clone https://github.com/pyenv/pyenv.git ~/.pyenv Pyenv is very simple, conceptually. It's just a set of shell scripts to automate the process of fetching, compiling, and installing versions of Python, and then massaging your shell to make sure the versions you have installed are used in preference to anything else. So now you have the tool, you need to configure your shell to use it. I'm going to assume you're using Bash. 2. Make sure the contents of the pyenv tool is available on your path echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bash_profile echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bash_profile Note - this might need to be .bashrc, or something else, depending on your os/distro/setup. However, in principle you're just making the pyenv tool (which itself is just a set of shell scripts) available at all times. 3. Set your shell to initialise the pyenv tool every time you start a new shell echo -e 'if command -v pyenv 1>/dev/null 2>&1; then\n eval "$(pyenv init -)"\nfi' >> ~/.bash_profile Again: this might need to be .bashrc 4. Now open a new shell, and check you have pyenv available: $ pyenv pyenv 1.2.9-2-g6309aaf2 Usage: pyenv [] Some useful pyenv commands are: commands List all available pyenv commands local Set or show the local application-specific Python version global Set or show the global Python version shell Set or show the shell-specific Python version install Install a Python version using python-build uninstall Uninstall a specific Python version rehash Rehash pyenv shims (run this after installing executables) version Show the current Python version and its origin versions List all Python versions available to pyenv which Display the full path to an executable whence List all Python versions that contain the given executable See `pyenv help ' for information on a specific command. For full documentation, see: https://github.com/pyenv/pyenv#readme If you don't have pyenv working at this stage, come back and I'll help you troubleshoot. Assuming you do, continue: 5. Now you can install a version of Python, locally : pyenv install --list This shows you the various options of Pythons you can install. You want the latest 2.7: pyenv install 2.7.15 This will fetch the source code of Python, and compile and install it for you, and place it in your local shell environment, where you can use it. If this step doesn't work, it's probably because your system doesn't have a compiler and associated tools. I can help you troubleshoot that, but ultimately you'll need support from your system administrator at this point. Assuming it's install Python, now you just need to tell your shell that you want to use it: pyenv local 2.7.15 This will make your shell find your 2.7.15 installation ahead of the system python: $ python --version Python 2.7.15 Now you can run and use your Python. Any further questions, sing out. S. From cranky.frankie at gmail.com Sun Jan 13 09:47:52 2019 From: cranky.frankie at gmail.com (Cranky Frankie) Date: Sun, 13 Jan 2019 09:47:52 -0500 Subject: [Tutor] Running Python 3 on Linux Mint Message-ID: I want to start developing an application in Python 3 on my main computer which runs Linux Mint with Python 2.7. What is the best way to work with Python 3 on this system? -- Frank L. "Cranky Frankie" Palmeri, Risible Riding Raconteur & Writer "If you have a garden and a library, you have everything you need." - Cicero From mhysnm1964 at gmail.com Sun Jan 13 04:55:06 2019 From: mhysnm1964 at gmail.com (mhysnm1964 at gmail.com) Date: Sun, 13 Jan 2019 20:55:06 +1100 Subject: [Tutor] Debugging a sort error. In-Reply-To: <20190113091148.GA57585@cskk.homeip.net> References: <002801d4aae5$f5223740$df66a5c0$@gmail.com> <20190113091148.GA57585@cskk.homeip.net> Message-ID: <000e01d4ab26$11dadcd0$35909670$@gmail.com> Everyone, I did find out the issue. When looking at the output in a spreadsheet. I was inserting floats into the description dictionary from the code I was using to extract the data. Thus I have gone back to the drawing board. Rather than extracting by columns which became difficult to achieve what I want to do. I am now extracting by row and the code has greatly simplified. Sorry about the data example. AS I am dealing with personal information. I don't want to share the original data. I will look at my editor to see if it is able to change the indent. Note: I am a Vision Impaired (blind) person learning to code in Python. Thus indents don't really matter to me. ? But I will change the indents to make it easier to read. Thanks for the test, this will help greatly. -----Original Message----- From: Cameron Simpson Sent: Sunday, 13 January 2019 8:12 PM To: mhysnm1964 at gmail.com Cc: Tutor at python.org Subject: Re: [Tutor] Debugging a sort error. Discussion inline below. On 13Jan2019 13:16, mhysnm1964 at gmail.com wrote: >I am hoping someone can help with the below error using Python3.5 in >the Windows 10 bash environment. I found the below link which I am not >sure if this is related to the issue or not. As I don't fully understand the answer. > >https://github.com/SethMMorton/natsort/issues/7 I'm not sure that URL is very relevant, except to the extent that it points out that Python 3 issues an error when comparing incomparable types. In Python 2 this problem could go unnoticed, and that just leads to problems later, much farther from the source of the issue. >Issue, following error is generated after trying to sort a list of strings. > >description.sort() >TypeError: unorderable types: float() < str() > >Each list items (elements) contain a mixture of alpha chars, numbers, >punctuation chars like / and are in a string type. Below is an example >extract of the data from the list. > >['Description', 'EFTPOS WOOLWORTHS 1294 ", "withdrawal Hudson >street 3219"] The error message says that some of these values are not strings. One at least is a float. My expectation is that the openpyxl module is reading a floating point value into your description array. This might be openpxyl being too clever, or it might be (more likely IMO) be Excel turning something that looked like a float into a float. Spreadsheets can be ... helpful like that. >There is over 2000 such entries. This used to work and now doesn't. You'll need to examine the values. But I see that you're trying to do this. I've snipped the data loading phase. Here: >description = data['Description'] >for i in description: > if not str(i): > print "not a string") This is not a valid check that "i" is a string. That expression: str(i) tries to convert "i" into a string (via its __str__ method). Most objects have such a method, and str() of a float is the textual representation of the float. So the if statement doesn't test what you want to test. Try this: if not isinstance(i, str): print("not a string:", i, type(i)) >description.sort() >I am suspecting it is something to do with the data but cannot track >down the cause. Any suggestions on how to debug this? Your exception is in here, but as you expect you want to inspect the description types first. If the description column does contain a float in the original data then you could convert it to a string first! Note that this may not match visually what was in the spreadsheet. (BTW, your cited code never fills out the description list, not it cannot be current.) But first, fine out what's wrong. Try the type test I suggest and see how far you get. Cheers, Cameron Simpson From alan.gauld at yahoo.co.uk Sun Jan 13 14:01:42 2019 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sun, 13 Jan 2019 19:01:42 +0000 Subject: [Tutor] Running Python 3 on Linux Mint In-Reply-To: References: Message-ID: On 13/01/2019 14:47, Cranky Frankie wrote: > I want to start developing an application in Python 3 on my main computer > which runs Linux Mint with Python 2.7. What is the best way to work with > Python 3 on this system? You don't say which Mint version but assuming its 17 or greater then you can just use the software manager (or Synaptic) and install the python3 packages. You need the base plus IDLE plus any other third party libraries you use. (IDLE should pull in Tkinter as a dependency). It will probably only be version 3.5 but unless you are obsessive about having the latest version it should be adequate. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From robertvstepp at gmail.com Sun Jan 13 19:42:53 2019 From: robertvstepp at gmail.com (boB Stepp) Date: Sun, 13 Jan 2019 18:42:53 -0600 Subject: [Tutor] OT: A bit of humor related to my sporadic quest to learn Python Message-ID: My son sent me this link, which I think captures my situation with Python quite nicely: https://cdn-images-1.medium.com/max/720/1*7RZKI-g4K_syDf6XQEGWKw.jpeg -- boB From mhysnm1964 at gmail.com Sun Jan 13 17:29:48 2019 From: mhysnm1964 at gmail.com (mhysnm1964 at gmail.com) Date: Mon, 14 Jan 2019 09:29:48 +1100 Subject: [Tutor] Debugging a sort error. In-Reply-To: References: <002801d4aae5$f5223740$df66a5c0$@gmail.com> Message-ID: <003601d4ab8f$801a5770$804f0650$@gmail.com> All, Once again thanks for all the suggestions. It was the input data after all. As I am importing three sheets into python. One of the sheets had one less column. Thus how I ended up with float points. The testing for the string helped here greatly. Now things are correct again. On and forward to start working on text pattern exercise which I always have struggled with. Last language I did this in was Perl and had all sorts of headaches. ? Python seems cleaner from the reading I have done thus far. Lets see what challenges wait in front of me. -----Original Message----- From: Stephen Nelson-Smith Sent: Monday, 14 January 2019 1:15 AM To: mhysnm1964 at gmail.com Cc: Python Tutor mailing list Subject: Re: [Tutor] Debugging a sort error. Hi, On Sun, Jan 13, 2019 at 8:34 AM wrote: > description.sort() > TypeError: unorderable types: float() < str() So, fairly obviously, we can't test whether a float is less than a string. Any more than we can tell if a grapefruit is faster than a cheetah. So there must be items in description that are strings and floats. With 2000 lines, you're going to struggle to eyeball this, so try something like this: In [69]: irrational_numbers = [3.14159265, 1.606695, "pi", "Pythagoras Constant"] In [70]: from collections import Counter In [71]: dict(Counter([type(e) for e in irrational_numbers])) Out[71]: {float: 2, str: 2} If with your data, this shows only strings, I'll eat my hat. S. From mhysnm1964 at gmail.com Sun Jan 13 17:32:28 2019 From: mhysnm1964 at gmail.com (mhysnm1964 at gmail.com) Date: Mon, 14 Jan 2019 09:32:28 +1100 Subject: [Tutor] Debugging a sort error. In-Reply-To: References: <002801d4aae5$f5223740$df66a5c0$@gmail.com> Message-ID: <003701d4ab8f$df4597a0$9dd0c6e0$@gmail.com> Peter, Thanks for the code for a custom key. That will come in handy later down the track. -----Original Message----- From: Tutor On Behalf Of Peter Otten Sent: Sunday, 13 January 2019 10:00 PM To: tutor at python.org Subject: Re: [Tutor] Debugging a sort error. mhysnm1964 at gmail.com wrote: > Issue, following error is generated after trying to sort a list of > strings. > > description.sort() > TypeError: unorderable types: float() < str() Consider >>> descriptions = ["foo", "bar", 123, 3.14, 42, 200.1, "0"] >>> sorted(descriptions) Traceback (most recent call last): File "", line 1, in TypeError: unorderable types: int() < str() If there are only numbers and strings in the list you can force the sort to succeed with the following custom key function: >>> def key(item): ... return isinstance(item, str), item ... This will move the numbers to the beginning of the list: >>> sorted(descriptions, key=key) [3.14, 42, 123, 200.1, '0', 'bar', 'foo'] _______________________________________________ Tutor maillist - Tutor at python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor From rba2124 at gmail.com Sun Jan 13 23:21:34 2019 From: rba2124 at gmail.com (Roger B. Atkins) Date: Sun, 13 Jan 2019 21:21:34 -0700 Subject: [Tutor] OT: A bit of humor related to my sporadic quest to learn Python In-Reply-To: References: Message-ID: Ditto, but the angle is steeper, I've lost my climbing equipment, and the volcano is erupting. On Sun, Jan 13, 2019 at 5:44 PM boB Stepp wrote: > My son sent me this link, which I think captures my situation with > Python quite nicely: > > https://cdn-images-1.medium.com/max/720/1*7RZKI-g4K_syDf6XQEGWKw.jpeg > > -- > boB > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From cranky.frankie at gmail.com Mon Jan 14 18:08:07 2019 From: cranky.frankie at gmail.com (Cranky Frankie) Date: Mon, 14 Jan 2019 18:08:07 -0500 Subject: [Tutor] Running Python 3 on Linux Mint In-Reply-To: References: Message-ID: On Sun, Jan 13, 2019 at 2:04 PM Alan Gauld via Tutor wrote: > "You don't say which Mint version but assuming its 17 or > greater then you can just use the software manager > (or Synaptic) and install the python3 packages." > > Thanks so much, I'm all set now. -- Frank L. "Cranky Frankie" Palmeri, Risible Riding Raconteur & Writer "If you have a garden and a library, you have everything you need." - Cicero From mats at wichmann.us Mon Jan 14 18:56:23 2019 From: mats at wichmann.us (Mats Wichmann) Date: Mon, 14 Jan 2019 16:56:23 -0700 Subject: [Tutor] Running Python 3 on Linux Mint In-Reply-To: References: Message-ID: <14bc1587-80d3-57c2-b242-d6c71c078d85@wichmann.us> On 1/14/19 4:08 PM, Cranky Frankie wrote: > On Sun, Jan 13, 2019 at 2:04 PM Alan Gauld via Tutor > wrote: >> "You don't say which Mint version but assuming its 17 or >> greater then you can just use the software manager >> (or Synaptic) and install the python3 packages." > Thanks so much, I'm all set now. Given the impending end of support for Python 2 - officially less than a year now - distros have been surprisingly slow in switching to Python 3 being the default (meaning instead of having to specially ask for python3, you would have to specially ask for python2). So don't feel bad that this wasn't instantly obvious... Mint follows Ubuntu, which follows Debian (mostly) so it's not exactly the Mint folks' problem, but even they could have chosen to be proactive if they wanted to. From cs at cskk.id.au Mon Jan 14 19:53:57 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Tue, 15 Jan 2019 11:53:57 +1100 Subject: [Tutor] Debugging a sort error. In-Reply-To: <003601d4ab8f$801a5770$804f0650$@gmail.com> References: <003601d4ab8f$801a5770$804f0650$@gmail.com> Message-ID: <20190115005357.GA64832@cskk.homeip.net> On 14Jan2019 09:29, mhysnm1964 at gmail.com wrote: >Once again thanks for all the suggestions. It was the input data after >all. As I am importing three sheets into python. One of the sheets had >one less column. Semantic nit: "fewer". "less" is for continuous values. I've had to deal with loosely defined spreadsheets in the past year, and ended up with some mapping classes for reading "heading based" CSV and Excel sheets, where the first column contains column headings. It reads the rows and constructs a namedtuple class for the rows with field names based on the headings and then fills in the fields for each subsequent row. The advantage here is that then you can access the column data by name, eg: row.description provided the column heading remains the same. See the xl_import function from the cs.csvutils module: https://pypi.org/project/cs.csvutils/ The code is here: https://bitbucket.org/cameron_simpson/css/src/tip/lib/python/cs/csvutils.py Right at the bottom of that file is an example use of xl_import() for a worksheet where row 1 is a title and the column names are in row 2. >On and forward to start working on text pattern exercise which I always >have struggled with. Last language I did this in was Perl and had all >sorts of headaches. ? Python seems cleaner from the reading I have >done thus far. Lets see what challenges wait in front of me. I used to use Perl extensively. I put off moving to Python for too long. Cheers, Cameron Simpson From kb at kbojens.de Tue Jan 15 08:25:35 2019 From: kb at kbojens.de (Kai Bojens) Date: Tue, 15 Jan 2019 14:25:35 +0100 Subject: [Tutor] Running Python 3 on Linux Mint In-Reply-To: <14bc1587-80d3-57c2-b242-d6c71c078d85@wichmann.us> References: <14bc1587-80d3-57c2-b242-d6c71c078d85@wichmann.us> Message-ID: <20190115132535.GA22923@mail.kbojens.de> On 14/01/2019 ?? 16:56:23PM -0700, Mats Wichmann wrote: > Given the impending end of support for Python 2 - officially less than a > year now - distros have been surprisingly slow in switching to Python 3 > being the default (meaning instead of having to specially ask for > python3, you would have to specially ask for python2). As I learned today: "If the python command is installed, it should invoke the same version of Python as the python2 command" (PEP 394) -> https://www.python.org/dev/peps/pep-0394/ -> https://developers.redhat.com/blog/2018/11/27/what-no-python-in-rhel-8-beta/ From maninathsahoo06 at gmail.com Wed Jan 16 13:26:28 2019 From: maninathsahoo06 at gmail.com (Maninath sahoo) Date: Wed, 16 Jan 2019 23:56:28 +0530 Subject: [Tutor] Doubt in Python Message-ID: Using of relational operator i Python? How to use relational operator in List. Please answer me.. From alan.gauld at yahoo.co.uk Wed Jan 16 19:39:21 2019 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Thu, 17 Jan 2019 00:39:21 +0000 Subject: [Tutor] Doubt in Python In-Reply-To: References: Message-ID: On 16/01/2019 18:26, Maninath sahoo wrote: > Using of relational operator i Python? > How to use relational operator in List. > Please answer me.. Relational operators are more commonly called comparisons (or predicates). You use them with a List much as you use any other object. if list1 == list2... if list2 != list3... if list4 is list1... if listX < listY Your best bet it just to try them at the interactive >>> prompt. For example: >>> x = [1,2,3,4] >>> y = [2,5,8] >>> x == y False >>> x < y True >>> y > x True >>> x <= y True >>> z = [1,2,3] >>> x < z False >>> z < x True and so on... The other important operator with regard to lists is 'in' >>> 3 in x True >>> 3 in y False You can also see which operators are supported by looking at the dir() function output: >>> dir([]) ['__add__', '__class__', '__contains__', '__delattr__', '__delitem__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__gt__', '__hash__', '__iadd__', '__imul__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__reversed__', '__rmul__', '__setattr__', '__setitem__', '__sizeof__', '__str__', '__subclasshook__', 'append', 'clear', 'copy', 'count', 'extend', 'index', 'insert', 'pop', 'remove', 'reverse', 'sort'] Notice __eq__, __ne__, __gt__, __contains__, etc -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Thu Jan 17 04:57:03 2019 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Thu, 17 Jan 2019 09:57:03 +0000 Subject: [Tutor] Fwd: Re: Doubt in Python In-Reply-To: <2b59d7af-c5fa-9355-cf19-640c0d48afde@yahoo.co.uk> References: <2b59d7af-c5fa-9355-cf19-640c0d48afde@yahoo.co.uk> Message-ID: CCing the list. Please use Reply All or Reply List on responses to the list. On 17/01/2019 07:24, Maninath sahoo wrote: >>>> a=[100,50,30] >>>> b=[100,90,3] >>>> a True >>>> a>b > False > > > How it compares between two lists > The algorithm is probably described somewhere in the documentation but my understanding is that it looks something like this(in pdeudo code): def isEqual(X,Y): ??? if len(X) != len(Y): return False ??? if X[0] != Y[0]: return False ??? return X[1:] == Y[1:] def isGreater(X,Y): ?? if len(X) <= len(Y) return False ?? if X[0] < Y[0] return False ?? if X[0] == Y[0] return X[1:] > Y[1:] ?? return True And similarly for the other operators. But that's just based on playing with the interpreter and exploring different values... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From steve at pearwood.info Thu Jan 17 10:28:12 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 18 Jan 2019 02:28:12 +1100 Subject: [Tutor] Fwd: Re: Doubt in Python In-Reply-To: References: <2b59d7af-c5fa-9355-cf19-640c0d48afde@yahoo.co.uk> Message-ID: <20190117152811.GI13616@ando.pearwood.info> On Thu, Jan 17, 2019 at 09:57:03AM +0000, Alan Gauld via Tutor wrote: > The algorithm is probably described somewhere in the documentation > but my understanding is that it looks something like this(in pdeudo code): List, tuple and string comparisons are defined as lexicographical order: http://docs.python.org/tutorial/datastructures.html#comparing-sequences-and-other-types https://en.wikipedia.org/wiki/Lexicographical_order That's a fancy way of saying "dictionary order". That means that lists are ordered in the same way that words are ordered in the dictionary: - match up letters in the word (items in the list) in pairs; - so long as the pairs of letters (items) are equal, keep going; - as soon as you hit a pair that aren't equal, the order of that pair determines the order of the words (lists); - if one word runs out of letters (list runs out of items), then it comes first; - if all the pairs are equal, the words (lists) are equal. Some examples: [0, 1, 2] < [1, 2, 3] because 0 < 1 [0, 1, 2] < [0, 1, 2, 3] because the first three items are equal but the first list is shorter. [0, 1, 2, 3, 4, 5] < [0, 1, 999] because 2 < 999 [0, 999, 999, 999] < [1, 2] because 0 < 1 -- Steve From __peter__ at web.de Thu Jan 17 10:34:27 2019 From: __peter__ at web.de (Peter Otten) Date: Thu, 17 Jan 2019 16:34:27 +0100 Subject: [Tutor] Implementation of list comparison operators, was Re: Doubt in Python References: <2b59d7af-c5fa-9355-cf19-640c0d48afde@yahoo.co.uk> Message-ID: Alan Gauld via Tutor wrote: > On 17/01/2019 07:24, Maninath sahoo wrote: > >>>> a=[100,50,30] > >>>> b=[100,90,3] > >>>> a > True > >>>> a>b > > False > > > > >> How it compares between two lists >> > The algorithm is probably described somewhere in the documentation > but my understanding is that it looks something like this(in pdeudo code): > def isGreater(X,Y): > > if len(X) <= len(Y) return False > > if X[0] < Y[0] return False > > if X[0] == Y[0] return X[1:] > Y[1:] > > return True > > > And similarly for the other operators. > > But that's just based on playing with the interpreter and exploring > different values... >>> [10] > [1, 2, 3] True >>> [1] > [10, 2, 3] False So you cannot decide on length alone. My analogy of list comparison would be words in a dictionary; you compare individual letters until you reach a decision: mad <--> mad m -> equal a -> equal d -> equal (no character on both sides) --> mad == mad madam <--> mad m -> equal a -> equal d -> equal a is greater than (no character) --> madam > mad madam <--> man m -> equal a -> equal d is less than n --> madam < man A tentative implementation of less() (untested): # zip() intentionally omitted def less(x, y): for i in range(min(len(x), len(y))): if not x[i] == y[i]: return x[i] < y[i] return len(x) < len(y) Finally here's a tool that wraps all comparison operations of the list items. $ cat track_compare.py class A: def __init__(self, value): self.value = value def __lt__(self, other): print(self, "?", other) return self.value > other.value def __le__(self, other): print(self, "<=?", other) return self.value <= other.value def __ge__(self, other): print(self, ">=?", other) return self.value >= other.value def __ne__(self, other): print(self, "!=?", other) return self.value != other.value def __repr__(self): return repr(self.value) def noisy(items): return [A(item) for item in items] a = noisy([1, 2, 3]) b = noisy([1, 10, 20]) c = noisy([1]) d = noisy([1]) print("{} < {}".format(a, b)) a < b print("\n{} != {}".format(c, d)) c != d print("\n{} == {}".format(c, d)) c == d $ python3 track_compare.py [1, 2, 3] < [1, 10, 20] 1 ==? 1 2 ==? 10 2 Message-ID: One obscure detail of the implementation of list equality: In Python an object can be unequal to itself: >>> class A: ... def __eq__(self, other): return False ... >>> a = A() >>> a == a False However, the list assumes that (a is a) implies a == a, so >>> [a] == [a] True From david at graniteweb.com Thu Jan 17 14:04:51 2019 From: david at graniteweb.com (David Rock) Date: Thu, 17 Jan 2019 13:04:51 -0600 Subject: [Tutor] Implementation of list comparison operators In-Reply-To: References: <2b59d7af-c5fa-9355-cf19-640c0d48afde@yahoo.co.uk> Message-ID: > On Jan 17, 2019, at 12:39, Peter Otten <__peter__ at web.de> wrote: > > One obscure detail of the implementation of list equality: > > In Python an object can be unequal to itself: > >>>> class A: > ... def __eq__(self, other): return False > ... >>>> a = A() >>>> a == a > False Isn?t this a bit artificial, though? The reason this is False is because you explicitly tell it to return False when using equality. That?s not the same thing as using __eq__ without overriding it?s behavior internally. ? David Rock david at graniteweb.com From __peter__ at web.de Thu Jan 17 14:40:46 2019 From: __peter__ at web.de (Peter Otten) Date: Thu, 17 Jan 2019 20:40:46 +0100 Subject: [Tutor] Implementation of list comparison operators References: <2b59d7af-c5fa-9355-cf19-640c0d48afde@yahoo.co.uk> Message-ID: David Rock wrote: > >> On Jan 17, 2019, at 12:39, Peter Otten <__peter__ at web.de> wrote: >> >> One obscure detail of the implementation of list equality: >> >> In Python an object can be unequal to itself: >> >>>>> class A: >> ... def __eq__(self, other): return False >> ... >>>>> a = A() >>>>> a == a >> False > > Isn?t this a bit artificial, though? The reason this is False is because > you explicitly tell it to return False when using equality. That?s not > the same thing as using __eq__ without overriding it?s behavior > internally. Sorry, I don't understand that argument. My point wasn't whether it's a good idea to write objects that compare unequal to themselves -- such objects already exist: >>> nan = float("nan") >>> nan == nan False I only warned that a list containing such an object does not meet the intuitive expectation that list_a == list_b implies that all items in list_a compare equal to the respective items in list_b. From david at graniteweb.com Thu Jan 17 16:05:17 2019 From: david at graniteweb.com (David Rock) Date: Thu, 17 Jan 2019 15:05:17 -0600 Subject: [Tutor] Implementation of list comparison operators In-Reply-To: References: <2b59d7af-c5fa-9355-cf19-640c0d48afde@yahoo.co.uk> Message-ID: <7AF987E5-E8DE-42E2-B8D9-BB1C0FA35063@graniteweb.com> > On Jan 17, 2019, at 13:40, Peter Otten <__peter__ at web.de> wrote: > > David Rock wrote: > >> >> Isn?t this a bit artificial, though? The reason this is False is because >> you explicitly tell it to return False when using equality. That?s not >> the same thing as using __eq__ without overriding it?s behavior >> internally. > > Sorry, I don't understand that argument. My point wasn't whether it's a good > idea to write objects that compare unequal to themselves -- such objects > already exist: > >>>> nan = float("nan") >>>> nan == nan > False > > I only warned that a list containing such an object does not meet the > intuitive expectation that list_a == list_b implies that all items in list_a > compare equal to the respective items in list_b. It?s certainly a valid warning. My confusion came from you using an arbitrary example of creating a class that breaks the logic in an override rather than one that already exists as a concrete example. To me, your class example looked contrived. What is it about the float(?nan?) example that makes this break? In [5]: nan = float("nan?) In [6]: type(nan) Out[6]: float In [7]: nan == nan Out[7]: False In [8]: a = 1.1 In [9]: a ==a Out[9]: True In [10]: type(a) Out[10]: float both a and nan are floats, so why does a == a work, but nan == nan doesn?t? ? David Rock david at graniteweb.com From steve at pearwood.info Thu Jan 17 17:01:34 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 18 Jan 2019 09:01:34 +1100 Subject: [Tutor] Implementation of list comparison operators In-Reply-To: <7AF987E5-E8DE-42E2-B8D9-BB1C0FA35063@graniteweb.com> References: <2b59d7af-c5fa-9355-cf19-640c0d48afde@yahoo.co.uk> <7AF987E5-E8DE-42E2-B8D9-BB1C0FA35063@graniteweb.com> Message-ID: <20190117220133.GJ13616@ando.pearwood.info> On Thu, Jan 17, 2019 at 03:05:17PM -0600, David Rock wrote: > In [7]: nan == nan > Out[7]: False > > In [8]: a = 1.1 > > In [9]: a ==a > Out[9]: True > both a and nan are floats, so why does a == a work, but nan == nan > doesn?t? They both "work", because they both do what they are designed to do. Equality between two floats works something like this: if either number is a NAN: return False if both numbers are 0.0 or -0.0: return True if both numbers have the same bit-pattern: return True otherwise return False -- Steve From __peter__ at web.de Thu Jan 17 17:13:31 2019 From: __peter__ at web.de (Peter Otten) Date: Thu, 17 Jan 2019 23:13:31 +0100 Subject: [Tutor] Implementation of list comparison operators References: <2b59d7af-c5fa-9355-cf19-640c0d48afde@yahoo.co.uk> <7AF987E5-E8DE-42E2-B8D9-BB1C0FA35063@graniteweb.com> Message-ID: David Rock wrote: > >> On Jan 17, 2019, at 13:40, Peter Otten <__peter__ at web.de> wrote: >> >> David Rock wrote: >> >>> >>> Isn?t this a bit artificial, though? The reason this is False is >>> because >>> you explicitly tell it to return False when using equality. That?s not >>> the same thing as using __eq__ without overriding it?s behavior >>> internally. >> >> Sorry, I don't understand that argument. My point wasn't whether it's a >> good idea to write objects that compare unequal to themselves -- such >> objects already exist: >> >>>>> nan = float("nan") >>>>> nan == nan >> False >> >> I only warned that a list containing such an object does not meet the >> intuitive expectation that list_a == list_b implies that all items in >> list_a compare equal to the respective items in list_b. > > It?s certainly a valid warning. My confusion came from you using an > arbitrary example of creating a class that breaks the logic in an override > rather than one that already exists as a concrete example. To me, your > class example looked contrived. > > What is it about the float(?nan?) example that makes this break? > > In [5]: nan = float("nan?) > > In [6]: type(nan) > Out[6]: float > > In [7]: nan == nan > Out[7]: False > > In [8]: a = 1.1 > > In [9]: a ==a > Out[9]: True > > In [10]: type(a) > Out[10]: float > > both a and nan are floats, so why does a == a work, but nan == nan > doesn?t? It does "work", it's only produces a result you didn't expect ;) Python just follows the standard here https://en.wikipedia.org/wiki/IEEE_754 https://en.wikipedia.org/wiki/NaN#Comparison_with_NaN Also: https://stackoverflow.com/questions/10034149/why-is-nan-not-equal-to-nan From david at graniteweb.com Thu Jan 17 17:41:27 2019 From: david at graniteweb.com (David Rock) Date: Thu, 17 Jan 2019 16:41:27 -0600 Subject: [Tutor] Implementation of list comparison operators In-Reply-To: References: <2b59d7af-c5fa-9355-cf19-640c0d48afde@yahoo.co.uk> <7AF987E5-E8DE-42E2-B8D9-BB1C0FA35063@graniteweb.com> Message-ID: > On Jan 17, 2019, at 16:13, Peter Otten <__peter__ at web.de> wrote: > > David Rock wrote: > >> both a and nan are floats, so why does a == a work, but nan == nan >> doesn?t? > > It does "work", it's only produces a result you didn't expect ;) > Python just follows the standard here > > https://en.wikipedia.org/wiki/IEEE_754 > https://en.wikipedia.org/wiki/NaN#Comparison_with_NaN > > Also: > > https://stackoverflow.com/questions/10034149/why-is-nan-not-equal-to-nan Touch? :-) ? David Rock david at graniteweb.com From solbrilla1 at gmail.com Sun Jan 20 19:14:55 2019 From: solbrilla1 at gmail.com (Ello Solcraft) Date: Mon, 21 Jan 2019 01:14:55 +0100 Subject: [Tutor] Function not using updated variable? Message-ID: # currLocation doesn't use the updated variable. for x in inputList: currCalc.append(x) currCalc2 = ''.join(currCalc) currLocation = mapLocation(mapInput(currCalc2))#mapInput(currCalc2)) # I tried printing currCalc2, it updates like it should. But currLocation doesn't use the updated currCalc2. Hoewer it does work if I input a string without the currLocation variable. From steve at pearwood.info Sun Jan 20 20:17:33 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 21 Jan 2019 12:17:33 +1100 Subject: [Tutor] Function not using updated variable? In-Reply-To: References: Message-ID: <20190121011732.GU13616@ando.pearwood.info> Hi Ello, and welcome! On Mon, Jan 21, 2019 at 01:14:55AM +0100, Ello Solcraft wrote: > # currLocation doesn't use the updated variable. How do you know? I'm sorry, but it is impossible for me to tell what is going on here. There simply isn't enough information to understand your code. We don't know what inputList, currCalc, mapLocation or mapInput do. We don't know how you know that currLocation doesn't "use the updated variable" (currCalc2), or what you mean by "use". When asking for help to solve a problem like this, it is very helpful if you can provide SIMPLE code we can run ourselves, as well as the results you expect and the results you actually get. (We're volunteers, and you are not paying us to troll through 200 or 300 lines of code looking for a bug. Keep it short and simple.) More comments below. > for x in inputList: > currCalc.append(x) > currCalc2 = ''.join(currCalc) > currLocation = mapLocation(mapInput(currCalc2))#mapInput(currCalc2)) > > # I tried printing currCalc2, it updates like it should. But currLocation > doesn't use the updated currCalc2. How do you know? What happens if you print(currLocation) each time through the loop? What do mapInput and mapLocation do? If currLocation never changes its value, perhaps one of mapInput or mapLocation always returns the same thing? -- Steve From alan.gauld at yahoo.co.uk Mon Jan 21 03:13:37 2019 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Mon, 21 Jan 2019 08:13:37 +0000 Subject: [Tutor] Function not using updated variable? In-Reply-To: References: Message-ID: On 21/01/2019 00:14, Ello Solcraft wrote: > # currLocation doesn't use the updated variable. > > for x in inputList: > > currCalc.append(x) > currCalc2 = ''.join(currCalc) > currLocation = mapLocation(mapInput(currCalc2))#mapInput(currCalc2)) You haven't hgiven us any information that would help us diagnose the problem. All we can say from this is that you are passing the correct variable to mapInput(). But what does mapUInput() do? Show us that code too. And what does mapLocation do with the return value from mapInput()? We need to see that code too. > ... Hoewer it does work if I input a string > without the currLocation variable. That probably makes sense to you but makes no sense to us. How can it "work" if you remove the variable that you are saying doesn't work? Maybe you need to explain what you are trying to achieve? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From mhysnm1964 at gmail.com Sun Jan 20 22:17:21 2019 From: mhysnm1964 at gmail.com (mhysnm1964 at gmail.com) Date: Mon, 21 Jan 2019 14:17:21 +1100 Subject: [Tutor] Importing XML files. Message-ID: <013001d4b137$d41b3ee0$7c51bca0$@gmail.com> All, I am trying to import ITunes XML files. I have been doing some reading and I am somewhat confused with some XML terminology. What is: Tag ? I think it is the in the itunes library file. Text = The text between the Attrib -- don?t know. If I am correct in my thinking. If you look up all the tags using the xml module. How do you then isolate all the name text? I do not have any working code at this present time. I have been playing with the xml methods and reading some example tutorials. But I am stumped with the above terminology which is a small road block. Below is an example extract of my xML ITunes to see if this helps. I am doing this in Python 3.7 for Windows 10. Major Version1 Minor Version1 Date2019-01-14T03:56:30Z Application Version12.8.0.150 Features5 Show Content Ratings Music Folderfile:///Volumes/Itunes/iTunes/iTunes%20Media/ Library Persistent IDF2D33B339F0788F0 Tracks 6493 Track ID6493 NameIn Her Sights: A Montgomery Justice Novel, Book 1 (Unabridged) ArtistRobin Perini Album ArtistRobin Perini AlbumIn Her Sights (Unabridged) GenreAudiobook KindAudible file Size206806038 Total Time25574318 Year2012 Date Modified2015-12-12T23:48:18Z Date Added2015-12-12T23:48:20Z Bit Rate64 Sample Rate22050 CommentsJasmine 'Jazz' Parker, Jefferson County SWAT???s only female sniper, can thread the eye of a needle with a bullet. But she carries with her a secret from her past.... Normalization790 Artwork Count1 Persistent ID2C4CC3C31A2B95B5 Track TypeFile Protected Locationfile:///Volumes/Itunes/iTunes/iTunes%20Media/Audiobooks/Robin%20Perini/In%20Her%20Sights_%20A%20Montgomery%20Justice%20Novel,%20Book%201%20(Unabridged).aax File Folder Count4 Library Folder Count1 From alan.gauld at yahoo.co.uk Mon Jan 21 08:57:41 2019 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Mon, 21 Jan 2019 13:57:41 +0000 Subject: [Tutor] Importing XML files. In-Reply-To: <013001d4b137$d41b3ee0$7c51bca0$@gmail.com> References: <013001d4b137$d41b3ee0$7c51bca0$@gmail.com> Message-ID: On 21/01/2019 03:17, mhysnm1964 at gmail.com wrote: > confused with some XML terminology. > > What is: > Tag ? I think it is the in the itunes library file. > > Text = The text between the > > Attrib -- don?t know. You really need to find a tutorial on XML (And maybe even start with one on HTML?) To parse out the information from an XML file, even using a parser like etree, requires that you understand the structure of the file. And that includes really understanding the terminology. In very simple terms an XML file (or more correctly a "document") is a tree structure. It starts with a parent node then that has child nodes. Each child may have children of its own ad infinitum. Each node is identified by a pair of tags which are simply names between angle brackets - for example There is a start and closing tag, the closing one having a '/' at the start. Between the tags is the text (which may include more tags for child nodes). tags can also include attributes which are names inside the opening tags brackets, often with an assigned value. The rules governing what constitutes a valid tag or attribute and which tags can be nested within which others are all defined in a definition file, often called a schema or DTD file. There is a standard header that tells the reader where to find the schema. You might also see comments which are just notes to explain whats going on and contain no actual data... Here is a meaningless pseudocode example: Some text here Child2 text here Text here on same line More child2 text Text for subchild Finishing off child2 text... More text here, could be lorts of it over many lines For more than that google XML tutorial... HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From __peter__ at web.de Mon Jan 21 13:22:09 2019 From: __peter__ at web.de (Peter Otten) Date: Mon, 21 Jan 2019 19:22:09 +0100 Subject: [Tutor] Importing XML files. References: <013001d4b137$d41b3ee0$7c51bca0$@gmail.com> Message-ID: mhysnm1964 at gmail.com wrote: > I am trying to import ITunes XML files. I have been doing some reading and > I am somewhat confused with some XML terminology. > What is: > > > > Tag ? I think it is the in the itunes library file. > > Text = The text between the > > Attrib -- don?t know. > > > > If I am correct in my thinking. If you look up all the tags using the xml > module. How do you then isolate all the name text? I do not have any > working code at this present time. I have been playing with the xml > methods and reading some example tutorials. But I am stumped with the > above terminology which is a small road block. Below is an example extract > of my xML ITunes to see if this helps. I am doing this in Python 3.7 for > Windows 10. > > > > > > "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> > > You may be lucky in that you can avoid the hard way outlined by Alan -- Python's stdandard library includes a module that handles Apple's plist format. I tried your sample, and the module seems to turn it into nested dicts: >>> import plistlib, pprint >>> with open("sample.xml", "rb") as f: ... data = plistlib.load(f) ... >>> pprint.pprint(data, width=70) {'Application Version': '12.8.0.150', 'Date': datetime.datetime(2019, 1, 14, 3, 56, 30), 'Features': 5, 'Library Persistent ID': 'F2D33B339F0788F0', 'Major Version': 1, 'Minor Version': 1, 'Music Folder': 'file:///Volumes/Itunes/iTunes/iTunes%20Media/', 'Show Content Ratings': True, 'Tracks': {'6493': {'Album': 'In Her Sights (Unabridged)', 'Album Artist': 'Robin Perini', 'Artist': 'Robin Perini', 'Artwork Count': 1, 'Bit Rate': 64, 'Comments': "Jasmine 'Jazz' Parker, " 'Jefferson County SWAT???s only ' 'female sniper, can thread the ' 'eye of a needle with a bullet. ' 'But she carries with her a ' 'secret from her past....', [snip] >>> data["Date"].isoformat() '2019-01-14T03:56:30' >>> list(data["Tracks"].values())[0]["Artist"] 'Robin Perini' Looks good, except for the mojibake -- but that was already in your email. From mhysnm1964 at gmail.com Mon Jan 21 18:39:00 2019 From: mhysnm1964 at gmail.com (mhysnm1964 at gmail.com) Date: Tue, 22 Jan 2019 10:39:00 +1100 Subject: [Tutor] Importing XML files. In-Reply-To: References: <013001d4b137$d41b3ee0$7c51bca0$@gmail.com> Message-ID: <008001d4b1e2$7eb91110$7c2b3330$@gmail.com> Peter and Alan, Peter, Thanks for the information. The library did the trick and I can get access to the XML content. Alan, thanks for the explanation of the tree structure. I was aware of this already for HTML and XML. Just didn't understand the terminology used from the XML library. The tutorials I have viewed didn't explain this information. I will continue investigating. The library provided has addressed my concerns for now. -----Original Message----- From: Tutor On Behalf Of Peter Otten Sent: Tuesday, 22 January 2019 5:22 AM To: tutor at python.org Subject: Re: [Tutor] Importing XML files. mhysnm1964 at gmail.com wrote: > I am trying to import ITunes XML files. I have been doing some reading > and I am somewhat confused with some XML terminology. > What is: > > > > Tag ? I think it is the in the itunes library file. > > Text = The text between the > > Attrib -- don?t know. > > > > If I am correct in my thinking. If you look up all the tags using the > xml module. How do you then isolate all the name text? I do not have > any working code at this present time. I have been playing with the > xml methods and reading some example tutorials. But I am stumped with > the above terminology which is a small road block. Below is an example > extract of my xML ITunes to see if this helps. I am doing this in > Python 3.7 for Windows 10. > > > > > > "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> > > You may be lucky in that you can avoid the hard way outlined by Alan -- Python's stdandard library includes a module that handles Apple's plist format. I tried your sample, and the module seems to turn it into nested dicts: >>> import plistlib, pprint >>> with open("sample.xml", "rb") as f: ... data = plistlib.load(f) ... >>> pprint.pprint(data, width=70) {'Application Version': '12.8.0.150', 'Date': datetime.datetime(2019, 1, 14, 3, 56, 30), 'Features': 5, 'Library Persistent ID': 'F2D33B339F0788F0', 'Major Version': 1, 'Minor Version': 1, 'Music Folder': 'file:///Volumes/Itunes/iTunes/iTunes%20Media/', 'Show Content Ratings': True, 'Tracks': {'6493': {'Album': 'In Her Sights (Unabridged)', 'Album Artist': 'Robin Perini', 'Artist': 'Robin Perini', 'Artwork Count': 1, 'Bit Rate': 64, 'Comments': "Jasmine 'Jazz' Parker, " 'Jefferson County SWAT???s only ' 'female sniper, can thread the ' 'eye of a needle with a bullet. ' 'But she carries with her a ' 'secret from her past....', [snip] >>> data["Date"].isoformat() '2019-01-14T03:56:30' >>> list(data["Tracks"].values())[0]["Artist"] 'Robin Perini' Looks good, except for the mojibake -- but that was already in your email. _______________________________________________ Tutor maillist - Tutor at python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor From matthew.polack at htlc.vic.edu.au Tue Jan 22 01:57:54 2019 From: matthew.polack at htlc.vic.edu.au (Matthew Polack) Date: Tue, 22 Jan 2019 17:57:54 +1100 Subject: [Tutor] Recommended Resurce or strategy for beginning students Message-ID: Hi All, In our growing school we're teaching Python programming for the first time as an elective subject with Year 9 and 10 students. (Had a dabble at this last year with 3 students in Year 11) I'm wondering what specific resource or stategy people would recommend for absolute beginners? ie. a course or program, book,...set of activities to follow that strategically introduces and develops key skills. At this age level I don't think we need to be achieving 'rocket science'..but rather giving the students a good solid introduction. Some of the leadership wanted me to use this programming in combination with building robots...I've even wondered whether this is trying to achieve too many things...and we're better off focused on programming itself... but am open to this idea too... I've had a play with using the excellent PySimpleGUI...which is an excellent resource for building a GUI...but I've realised before doing too much of this we might need to get a grip on core fundamentals.... The challenge is trying to find a way to making this 'fun' for students whilst also having them genuinely learn rather than just 'copying pasting' code...achieving something that looks good...but not really understanding what they are doing. So far my strategy will be: 1.) Establish some core basics(utlising some form of 'course',,,which goes through basics of syntax..variables...loops etc. utilising just raw code...(probably a simple 'Adventure Game') 2.) Build some simple programs using PySimple GUi..eg. Some quiz games etc. (there are some great examples on Github by another teacher and also the author Mike of PySimpleGUI. 3.) Possibly explore robotics. Can anyone make any recommendations on either resources or teaching/learning strategy/curriculum. Thank you, Matt Matthew Polack | Teacher [image: Emailbanner3.png] Trinity Drive | PO Box 822 Horsham Victoria 3402 p. 03 5382 2529 m. 0402456854 e. matthew.polack at htlc.vic.edu.au w. www.htlc.vic.edu.au -- **Disclaimer: *Whilst every attempt has been made to ensure that material contained in this email is free from computer viruses or other defects, the attached files are provided, and may only be used, on the basis that the user assumes all responsibility for use of the material transmitted. This email is intended only for the use of the individual or entity named above and may contain information that is confidential and privileged. If you are not the intended recipient, please note that any dissemination, distribution or copying of this email is strictly prohibited. If you have received this email in error, please notify us immediately by return email or telephone +61 3 5382 2529**?and destroy the original message.* From mhysnm1964 at gmail.com Tue Jan 22 04:25:48 2019 From: mhysnm1964 at gmail.com (mhysnm1964 at gmail.com) Date: Tue, 22 Jan 2019 20:25:48 +1100 Subject: [Tutor] exporting lists into CSV issue. Message-ID: <003301d4b234$77caf420$6760dc60$@gmail.com> All, Thanks in advance for any assistance. This is a follow up msg from my iTunes import issue with the xml. I have managed to successfully import the data from the XML with Peter's and Alan's help. I have moved the information into a list and only extracting the data I want. Now I am trying to export to a CSV file. There is no syntax or logical errors I can see. The information is being exported to the CSV. But when I bring it into Excel. I am getting a blank row between each row. The exported data looks like this: Name,Artist,Kind,Total Time,Year,Date Modified,Date Added,Bit Rate,Sample Rate,Comments,Location "In Her Sights: A Montgomery Justice Novel, Book 1 (Unabridged)",Robin Perini,Audible file,25574318,2012,2015-12-12 23:48:18,2015-12-12 23:48:20,64,22050,"Jasmine 'Jazz' Parker, Jefferson County SWAT's only female sniper, can thread the eye of a needle with a bullet. But she carries with her a secret from her past....","file:///Volumes/Itunes/iTunes/iTunes%20Media/Audiobooks/Robin%20P erini/In%20Her%20Sights_%20A%20Montgomery%20Justice%20Novel,%20Book%201%20(U nabridged).aax" "Behind the Lies: A Montgomery Justice Novel, Book 2 (Unabridged)",Robin Perini,Audible file,35142797,2013,2015-12-12 23:53:33,2015-12-12 23:53:33,64,22050,"Of the six Montgomery brothers, Zach has always walked on the wild side. He rocketed to fame playing a hero in a movie, but off screen he's living in the shadows. Zach's dark secret: He leads a double life as a CIA operative....",file:///Volumes/Itunes/iTunes/iTunes%20Media/Audiobooks/Robin %20Perini/Behind%20the%20Lies_%20A%20Montgomery%20Justice%20Novel,%20Book%20 2%20(Unabridged).aax I would have expected a new line between each book, but there is a space. The code looks like this: import plistlib, pprint, csv with open("library.xml", "rb") as f: data = plistlib.load(f) books =[['Name', 'Artist', 'Kind', 'Total Time', 'Year', 'Date Modified', 'Date Added', 'Bit Rate', 'Sample Rate', 'Comments', 'Location']] for book in list(data['Tracks'].values()): tmp = [] if not 'Audible file' in book['Kind']: break # skip book for key in books[0]: if key in book: tmp.append(book[key]) else: tmp.append("") books.append(tmp) with open ('books-list.csv', 'w') as wf: writer = csv.writer(wf) writer.writerows(books) wf.close() When I debug the above. The output of books look fine to me. For example: (Pdb) ['Name', 'Artist', 'Kind', 'Total Time', 'Year', 'Date Modified', 'Date Added', 'Bit Rate', 'Sample Rate', 'Comments', ' Location'] (Pdb) (Pdb) books[0] ] 0 ['In Her Sights: A Montgomery Justice Novel, Book 1 (Unabridged)', 'Robin Perini', 'Audible file', 25574318, 2012, datet ime.datetime(2015, 12, 12, 23, 48, 18), datetime.datetime(2015, 12, 12, 23, 48, 20), 64, 22050, "Jasmine 'Jazz' Parker, Jefferson County SWAT's only female sniper, can thread the eye of a needle with a bullet. But she carries with her a sec ret from her past....", 'file:///Volumes/Itunes/iTunes/iTunes%20Media/Audiobooks/Robin%20Perini/In%2 0Her%20Sights_%20A%2 0Montgomery%20Justice%20Novel,%20Book%201%20(Unabridged).aax'] (Pdb) (Pdb) books[1] ] 1 ['Behind the Lies: A Montgomery Justice Novel, Book 2 (Unabridged)', 'Robin Perini', 'Audible file', 35142797, 2013, dat etime.datetime(2015, 12, 12, 23, 53, 33), datetime.datetime(2015, 12, 12, 23, 53, 33), 64, 22050, 'Of the six Montgomery brothers, Zach has always walked on the wild side. He rocketed to fame playing a hero in a movie, but off screen he's l iving in the shadows. Zach's dark secret: He leads a double life as a CIA operative....', 'file:///Volumes/Itunes/iTunes /iTunes%20Media/Audiobooks/Robin%20Perini/Behind%20the%20Lies_%20A%20Montgom ery%20Justice%20Novel,%20Book%202%20(Unabrid ged).aax'] (Pdb) Any ideas why this could be occurring? Regards Sean From alan.gauld at yahoo.co.uk Tue Jan 22 04:57:10 2019 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Tue, 22 Jan 2019 09:57:10 +0000 Subject: [Tutor] Recommended Resurce or strategy for beginning students In-Reply-To: References: Message-ID: On 22/01/2019 06:57, Matthew Polack wrote: > In our growing school we're teaching Python programming for the first time > as an elective subject with Year 9 and 10 students. (Had a dabble at this > last year with 3 students in Year 11) > > I'm wondering what specific resource or stategy people would recommend for > absolute beginners? There is a beginners page on the Python web site for non programmers. It lists many tutorials that you could consider. Also there used to be an Education Special Interest Group (edu-sig) populated with teachers using Python in schools. I don't know if it is still active but you could try asking there. I do know that at least two schools have used my tutorial(see .sig). One of them for a class of 14 year olds (I have no idea what age range your year 9/10 students are). It is intended to teach good programming theory as well as practice. Another more academic tutorial is Allan Downey's "Think Python" site. It is slightly deeper than mine but covers less breadth. > Some of the leadership wanted me to use this programming in combination > with building robots...I've even wondered whether this is trying to achieve > too many things...and we're better off focused on programming itself... but > am open to this idea too... In the UK many schools use the RaspberryPi project to teach robots to kids as part of their Technology courses. The programming is picked up by osmosis on an as-needed basis. The upside is that it's a lot of fun and gets kids used to the concepts of hardware and software working in unison. The downside is that they learn a lot of bad coding habits and don't understand the theoretical underpinnings of either the hardware or software. But as a way to get them hooked it works well . > I've had a play with using the excellent PySimpleGUI...which is an > excellent resource for building a GUI...but I've realised before doing too > much of this we might need to get a grip on core fundamentals.... The turtle module is good too. Turtle graphics were invented explicitly to teach kids programming (using the LOGO programming language). They are even better when combined with a real robot turtle, but I'm not sure how you would do that using Python. There may be an off the shelf option if you Google it... > 1.) Establish some core basics(utlising some form of 'course',,,which goes > through basics of syntax..variables...loops etc. utilising just raw > code...(probably a simple 'Adventure Game') Don't underestimate the power of CLI when starting out. Classic games like oxo, hangman and Conway's life are challenging for beginners and easy to do on a terminal. (They are also great for demonstrating how good data structure design can radically simplify complex code.) Adventure games like the Colossal Cave are obviously easy on a CLI too. > 2.) Build some simple programs using PySimple GUi..eg. Some quiz games etc. > (there are some great examples on Github by another teacher and also the > author Mike of PySimpleGUI. Consider including basic GUIs too. They are the simplest way to get them into event driven thinking and for some types of programming are easier to write than CLI apps. The only snag with Python is the lack of a GUI builder which forces the kids to write a lots of boring boiler plate code... > 3.) Possibly explore robotics. Or web programming maybe? There are lots of easy Web frameworks for Python and since, for most kids today, computing *is* the web it might be worthwhile. The downside is it introduces a lot of new concepts - HTML, networks etc Overall it sounds good. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From alan.gauld at yahoo.co.uk Tue Jan 22 05:22:30 2019 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Tue, 22 Jan 2019 10:22:30 +0000 Subject: [Tutor] exporting lists into CSV issue. In-Reply-To: <003301d4b234$77caf420$6760dc60$@gmail.com> References: <003301d4b234$77caf420$6760dc60$@gmail.com> Message-ID: On 22/01/2019 09:25, mhysnm1964 at gmail.com wrote: > being exported to the CSV. But when I bring it into Excel. I am getting a > blank row between each row. The exported data looks like this: I can't see anything obvious. > import plistlib, pprint, csv > > with open("library.xml", "rb") as f: > data = plistlib.load(f) > > books =[['Name', 'Artist', 'Kind', 'Total Time', 'Year', 'Date Modified', > 'Date Added', 'Bit Rate', 'Sample Rate', 'Comments', 'Location']] > > for book in list(data['Tracks'].values()): You don't need to convert to list. You can just iterate over the values(). I don't think that should add any newlines but it might be worth removing the list to see. > > tmp = [] > if not 'Audible file' in book['Kind']: > break # skip book > for key in books[0]: > if key in book: > tmp.append(book[key]) > else: > tmp.append("") > books.append(tmp > > with open ('books-list.csv', 'w') as wf: > writer = csv.writer(wf) > writer.writerows(books) You could try writing(and printing) the rows individually: for row in books: # print(row) # debugging only writer.writerow(row) To see if it is writerows() that is adding the spurious lines. If that's the case there may be a parameter to change or suppress the separator. It might also be easier to use the csv.dictwriter since your data appears to be in the form of a dict to start with. Although I'm not sure how dictwriter copes with missing fields. > wf.close() Don't use close() if you are using "with open..." Closing is done automatically. It's one of the advantages of the 'with' style. hth -- -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From __peter__ at web.de Tue Jan 22 06:26:11 2019 From: __peter__ at web.de (Peter Otten) Date: Tue, 22 Jan 2019 12:26:11 +0100 Subject: [Tutor] exporting lists into CSV issue. References: <003301d4b234$77caf420$6760dc60$@gmail.com> Message-ID: mhysnm1964 at gmail.com wrote: > Now I am > trying to export to a CSV file. There is no syntax or logical errors I can > see. The information is being exported to the CSV. But when I bring it > into Excel. I am getting a blank row between each row. > with open ('books-list.csv', 'w') as wf: Try switching off newline conversion: with open('books-list.csv', 'w', newline="") as wf: ... The csv.writer() delimits records with "\r\n", and on Windows open() converts "\n" to "\r\n" by default. Therefore you probably (no Window handy to to confirm) end up with "\r\r\n" between the rows. See also . > writer = csv.writer(wf) > writer.writerows(books) At this point the file is already closed by the context manager, no need for an extra > wf.close() From mhysnm1964 at gmail.com Tue Jan 22 05:16:20 2019 From: mhysnm1964 at gmail.com (mhysnm1964 at gmail.com) Date: Tue, 22 Jan 2019 21:16:20 +1100 Subject: [Tutor] exporting lists into CSV issue. In-Reply-To: <003301d4b234$77caf420$6760dc60$@gmail.com> References: <003301d4b234$77caf420$6760dc60$@gmail.com> Message-ID: <000e01d4b23b$86d33430$94799c90$@gmail.com> All, I have addressed the issue. When I open the file, I am using newline='' as one of the parameters. with open ('books-list.csv', 'w', newline='', encode='utf-8') as wf: I added the UTF-8 as I discovered after sending my original email I was not importing all the books. I discovered I was getting an utf-8 error due to some of the content. Now I have all my books in a spreadsheet. Stage 2, is to cross reference all this information against my audible library which I will have to extract. All this is to learn the different methods of extracting data. From: mhysnm1964 at gmail.com Sent: Tuesday, 22 January 2019 8:26 PM To: tutor at python.org Subject: exporting lists into CSV issue. All, Thanks in advance for any assistance. This is a follow up msg from my iTunes import issue with the xml. I have managed to successfully import the data from the XML with Peter's and Alan's help. I have moved the information into a list and only extracting the data I want. Now I am trying to export to a CSV file. There is no syntax or logical errors I can see. The information is being exported to the CSV. But when I bring it into Excel. I am getting a blank row between each row. The exported data looks like this: Name,Artist,Kind,Total Time,Year,Date Modified,Date Added,Bit Rate,Sample Rate,Comments,Location "In Her Sights: A Montgomery Justice Novel, Book 1 (Unabridged)",Robin Perini,Audible file,25574318,2012,2015-12-12 23:48:18,2015-12-12 23:48:20,64,22050,"Jasmine 'Jazz' Parker, Jefferson County SWAT's only female sniper, can thread the eye of a needle with a bullet. But she carries with her a secret from her past....","file:///Volumes/Itunes/iTunes/iTunes%20Media/Audiobooks/Robin%20P erini/In%20Her%20Sights_%20A%20Montgomery%20Justice%20Novel,%20Book%201%20(U nabridged).aax" "Behind the Lies: A Montgomery Justice Novel, Book 2 (Unabridged)",Robin Perini,Audible file,35142797,2013,2015-12-12 23:53:33,2015-12-12 23:53:33,64,22050,"Of the six Montgomery brothers, Zach has always walked on the wild side. He rocketed to fame playing a hero in a movie, but off screen he's living in the shadows. Zach's dark secret: He leads a double life as a CIA operative....",file:///Volumes/Itunes/iTunes/iTunes%20Media/Audiobooks/Robin %20Perini/Behind%20the%20Lies_%20A%20Montgomery%20Justice%20Novel,%20Book%20 2%20(Unabridged).aax I would have expected a new line between each book, but there is a space. The code looks like this: import plistlib, pprint, csv with open("library.xml", "rb") as f: data = plistlib.load(f) books =[['Name', 'Artist', 'Kind', 'Total Time', 'Year', 'Date Modified', 'Date Added', 'Bit Rate', 'Sample Rate', 'Comments', 'Location']] for book in list(data['Tracks'].values()): tmp = [] if not 'Audible file' in book['Kind']: break # skip book for key in books[0]: if key in book: tmp.append(book[key]) else: tmp.append("") books.append(tmp) with open ('books-list.csv', 'w') as wf: writer = csv.writer(wf) writer.writerows(books) wf.close() When I debug the above. The output of books look fine to me. For example: (Pdb) ['Name', 'Artist', 'Kind', 'Total Time', 'Year', 'Date Modified', 'Date Added', 'Bit Rate', 'Sample Rate', 'Comments', ' Location'] (Pdb) (Pdb) books[0] ] 0 ['In Her Sights: A Montgomery Justice Novel, Book 1 (Unabridged)', 'Robin Perini', 'Audible file', 25574318, 2012, datet ime.datetime(2015, 12, 12, 23, 48, 18), datetime.datetime(2015, 12, 12, 23, 48, 20), 64, 22050, "Jasmine 'Jazz' Parker, Jefferson County SWAT's only female sniper, can thread the eye of a needle with a bullet. But she carries with her a sec ret from her past....", 'file:///Volumes/Itunes/iTunes/iTunes%20Media/Audiobooks/Robin%20Perini/In%2 0Her%20Sights_%20A%2 0Montgomery%20Justice%20Novel,%20Book%201%20(Unabridged).aax'] (Pdb) (Pdb) books[1] ] 1 ['Behind the Lies: A Montgomery Justice Novel, Book 2 (Unabridged)', 'Robin Perini', 'Audible file', 35142797, 2013, dat etime.datetime(2015, 12, 12, 23, 53, 33), datetime.datetime(2015, 12, 12, 23, 53, 33), 64, 22050, 'Of the six Montgomery brothers, Zach has always walked on the wild side. He rocketed to fame playing a hero in a movie, but off screen he's l iving in the shadows. Zach's dark secret: He leads a double life as a CIA operative....', 'file:///Volumes/Itunes/iTunes /iTunes%20Media/Audiobooks/Robin%20Perini/Behind%20the%20Lies_%20A%20Montgom ery%20Justice%20Novel,%20Book%202%20(Unabrid ged).aax'] (Pdb) Any ideas why this could be occurring? Regards Sean From mike_barnett at hotmail.com Tue Jan 22 09:17:40 2019 From: mike_barnett at hotmail.com (Mike Barnett) Date: Tue, 22 Jan 2019 14:17:40 +0000 Subject: [Tutor] Recommended Resurce or strategy for beginning students In-Reply-To: References: Message-ID: I like the idea of starting out right away on a GUI. I know this is completely backwards to what would normally be taught, but hear me out. Kids today are used to GUI interfaces. They're on their phones, their computers, their TV sets. Why not teach kids to output to a window instead of a command line? What if it's just was easy, or easier, to work with a GUI as it is the command line? To output to the command line in standard Python: print('my string', variable1, variable2) To output the same information to a window using PySimpleGUI: Popup('my string', variable1, variable2) Or, you can "print" to a debug window if that's your thing. Print('takes the same parameters as print') If the ultimate goal is to teach kids about how to design a GUI window, how to lay out a GUI using good user interface design principals, then it would be nice to get the GUI coding out of the way and let the focus instead be on the GUI itself. This is when having a drag-and-drop Designer Tool is handy. If not, then the next best thing is a simple programming interface. PySimpleGUI was designed so that the code visually matches the window layout. It's capable of duplicating pretty much any layout and widget combination that you can create coding directly to tkinter's (or Qt's or WxPython's) interfaces. PySimpleGUI simply creates and executes the "boilerplate" code that is often brought up when GUIs are discussed. A goal was to remove all of the boilerplate code and provide a programmer with a simple, friendly and flexible set of APIs. You write a single line of code per row of widgets in your window plus a 1/2 dozen lines to implement the event loop. I don't see the harm in approaching the problem from a different direction. It could be wildly successful. Or... not... The worst that can happen is you screw up a classroom full of future programmers, creating a warped vision that GUIs can be fun and easy. @mike -----Original Message----- From: Matthew Polack Sent: Tuesday, January 22, 2019 1:58 AM To: tutor at python.org Subject: [Tutor] Recommended Resurce or strategy for beginning students Hi All, In our growing school we're teaching Python programming for the first time as an elective subject with Year 9 and 10 students. (Had a dabble at this last year with 3 students in Year 11) I'm wondering what specific resource or stategy people would recommend for absolute beginners? ie. a course or program, book,...set of activities to follow that strategically introduces and develops key skills. At this age level I don't think we need to be achieving 'rocket science'..but rather giving the students a good solid introduction. Some of the leadership wanted me to use this programming in combination with building robots...I've even wondered whether this is trying to achieve too many things...and we're better off focused on programming itself... but am open to this idea too... I've had a play with using the excellent PySimpleGUI...which is an excellent resource for building a GUI...but I've realised before doing too much of this we might need to get a grip on core fundamentals.... The challenge is trying to find a way to making this 'fun' for students whilst also having them genuinely learn rather than just 'copying pasting' code...achieving something that looks good...but not really understanding what they are doing. So far my strategy will be: 1.) Establish some core basics(utlising some form of 'course',,,which goes through basics of syntax..variables...loops etc. utilising just raw code...(probably a simple 'Adventure Game') 2.) Build some simple programs using PySimple GUi..eg. Some quiz games etc. (there are some great examples on Github by another teacher and also the author Mike of PySimpleGUI. 3.) Possibly explore robotics. Can anyone make any recommendations on either resources or teaching/learning strategy/curriculum. Thank you, Matt Matthew Polack | Teacher [image: Emailbanner3.png] Trinity Drive | PO Box 822 Horsham Victoria 3402 p. 03 5382 2529 m. 0402456854 e. matthew.polack at htlc.vic.edu.au w. www.htlc.vic.edu.au -- **Disclaimer: *Whilst every attempt has been made to ensure that material contained in this email is free from computer viruses or other defects, the attached files are provided, and may only be used, on the basis that the user assumes all responsibility for use of the material transmitted. This email is intended only for the use of the individual or entity named above and may contain information that is confidential and privileged. If you are not the intended recipient, please note that any dissemination, distribution or copying of this email is strictly prohibited. If you have received this email in error, please notify us immediately by return email or telephone +61 3 5382 2529**?and destroy the original message.* From Scott at ScottLarsen.com Tue Jan 22 14:15:19 2019 From: Scott at ScottLarsen.com (Scott Larsen) Date: Tue, 22 Jan 2019 11:15:19 -0800 Subject: [Tutor] Recommended Resurce or strategy for beginning students, (Matthew Polack) In-Reply-To: References: Message-ID: <3b237ac0-a196-bbe0-265d-e81c30beb4ca@ScottLarsen.com> > 3. Recommended Resurce or strategy for beginning students > (Matthew Polack) Matt, ??? I don't have any specific recommendations other than to make sure you were aware of a podcast I recently stumbled across called Teaching Python by Sean Tibor and Kelly Paredes.? I've only listened to one episode but it sounds as though she's a teacher and he's a programmer and they're working together to do something similar to what you're taking on. https://itunes.apple.com/us/podcast/teaching-python/id1445806053?mt=2 Best of luck, Scott From mhysnm1964 at gmail.com Tue Jan 22 21:21:28 2019 From: mhysnm1964 at gmail.com (Sean Murphy) Date: Wed, 23 Jan 2019 13:21:28 +1100 Subject: [Tutor] Recommended Resurce or strategy for beginning students In-Reply-To: References: Message-ID: I like this concept. The only additional information I would add in relation to any training or educational information. You must include accessibility. As the laws in many countries require this to be a part of the product. So it is a perfect time to educate students on this important topic. A high level awareness is required only. Introducing the basics. Keyboard navigation, colour contrast, ensuring the GUI works with a screen reader. The platforms used for GUI should do most of the heavy lifting. The other aspect is you need to ensure the course is accessible to possible disable students for now and the future. If you are based in the Usa. Then there could be legal requirements for this. Not sure. Out of my scope of focus in the accessibility world. A bare minimum is to understand the bare basics which are called POUR. Reference W3C for the explaination. Sean My experience is the part > On 23 Jan 2019, at 1:17 am, Mike Barnett wrote: > > I like the idea of starting out right away on a GUI. I know this is completely backwards to what would normally be taught, but hear me out. Kids today are used to GUI interfaces. They're on their phones, their computers, their TV sets. > > Why not teach kids to output to a window instead of a command line? What if it's just was easy, or easier, to work with a GUI as it is the command line? > > To output to the command line in standard Python: > print('my string', variable1, variable2) > > To output the same information to a window using PySimpleGUI: > Popup('my string', variable1, variable2) > > Or, you can "print" to a debug window if that's your thing. > Print('takes the same parameters as print') > > If the ultimate goal is to teach kids about how to design a GUI window, how to lay out a GUI using good user interface design principals, then it would be nice to get the GUI coding out of the way and let the focus instead be on the GUI itself. This is when having a drag-and-drop Designer Tool is handy. If not, then the next best thing is a simple programming interface. > > PySimpleGUI was designed so that the code visually matches the window layout. > > It's capable of duplicating pretty much any layout and widget combination that you can create coding directly to tkinter's (or Qt's or WxPython's) interfaces. PySimpleGUI simply creates and executes the "boilerplate" code that is often brought up when GUIs are discussed. > > A goal was to remove all of the boilerplate code and provide a programmer with a simple, friendly and flexible set of APIs. You write a single line of code per row of widgets in your window plus a 1/2 dozen lines to implement the event loop. > > I don't see the harm in approaching the problem from a different direction. It could be wildly successful. Or... not... The worst that can happen is you screw up a classroom full of future programmers, creating a warped vision that GUIs can be fun and easy. > > > @mike > > -----Original Message----- > From: Matthew Polack > Sent: Tuesday, January 22, 2019 1:58 AM > To: tutor at python.org > Subject: [Tutor] Recommended Resurce or strategy for beginning students > > Hi All, > > In our growing school we're teaching Python programming for the first time as an elective subject with Year 9 and 10 students. (Had a dabble at this last year with 3 students in Year 11) > > I'm wondering what specific resource or stategy people would recommend for absolute beginners? > > ie. a course or program, book,...set of activities to follow that strategically introduces and develops key skills. > > At this age level I don't think we need to be achieving 'rocket science'..but rather giving the students a good solid introduction. > > Some of the leadership wanted me to use this programming in combination with building robots...I've even wondered whether this is trying to achieve too many things...and we're better off focused on programming itself... but am open to this idea too... > > I've had a play with using the excellent PySimpleGUI...which is an excellent resource for building a GUI...but I've realised before doing too much of this we might need to get a grip on core fundamentals.... > > The challenge is trying to find a way to making this 'fun' for students whilst also having them genuinely learn rather than just 'copying pasting' > code...achieving something that looks good...but not really understanding what they are doing. > > So far my strategy will be: > > 1.) Establish some core basics(utlising some form of 'course',,,which goes through basics of syntax..variables...loops etc. utilising just raw code...(probably a simple 'Adventure Game') > 2.) Build some simple programs using PySimple GUi..eg. Some quiz games etc. > (there are some great examples on Github by another teacher and also the author Mike of PySimpleGUI. > 3.) Possibly explore robotics. > > Can anyone make any recommendations on either resources or teaching/learning strategy/curriculum. > > Thank you, > Matt > > > Matthew Polack | Teacher > > > [image: Emailbanner3.png] > > Trinity Drive | PO Box 822 > > Horsham Victoria 3402 > > p. 03 5382 2529 m. 0402456854 > > e. matthew.polack at htlc.vic.edu.au > > w. www.htlc.vic.edu.au > > -- > **Disclaimer: *Whilst every attempt has been made to ensure that material contained in this email is free from computer viruses or other defects, the attached files are provided, and may only be used, on the basis that the user assumes all responsibility for use of the material transmitted. This email is intended only for the use of the individual or entity named above and may contain information that is confidential and privileged. If you are not the intended recipient, please note that any dissemination, distribution or copying of this email is strictly prohibited. If you have received this email in error, please notify us immediately by return email or telephone +61 3 5382 2529** and destroy the original message.* > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor From rileyjharris12 at gmail.com Wed Jan 23 23:03:28 2019 From: rileyjharris12 at gmail.com (Riley Harris) Date: Thu, 24 Jan 2019 15:03:28 +1100 Subject: [Tutor] ssl verification failed error Message-ID: <5c493910.1c69fb81.e64c.a998@mx.google.com> When I try to install packages in python this error occurs. I have found a way around this by using: pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org but this only works when the error is while installing packages. Now I am trying to access a dataset from Quandl and it returns the same error. I don?t know what the core problem is or how to fix it. Or I at least need to know a way to get around it when accessing data sets and such. Sent from Mail for Windows 10 From steve at pearwood.info Thu Jan 24 08:57:27 2019 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 25 Jan 2019 00:57:27 +1100 Subject: [Tutor] ssl verification failed error In-Reply-To: <5c493910.1c69fb81.e64c.a998@mx.google.com> References: <5c493910.1c69fb81.e64c.a998@mx.google.com> Message-ID: <20190124135727.GA26605@ando.pearwood.info> On Thu, Jan 24, 2019 at 03:03:28PM +1100, Riley Harris wrote: > When I try to install packages in python this error occurs. Please copy and paste (don't summarise or re-type from memory) the actual command you are running and the exact error message you receive. -- Steve From mats at wichmann.us Thu Jan 24 10:44:46 2019 From: mats at wichmann.us (Mats Wichmann) Date: Thu, 24 Jan 2019 08:44:46 -0700 Subject: [Tutor] asking questions Message-ID: Just for a bit of amusement, one of the scenarios we see here is famous enough it has its very own website: http://xyproblem.info/ enjoy :) From asad.hasan2004 at gmail.com Sat Jan 26 03:20:52 2019 From: asad.hasan2004 at gmail.com (Asad) Date: Sat, 26 Jan 2019 13:50:52 +0530 Subject: [Tutor] python - files Message-ID: Hi All , I would like to know how do I make and option file as an argument on command propmnt in python . At present I using : if len(sys.argv) == 3: first = sys.argv[1] second = sys.argv[2] else: print "enter the second argument" It works well for the following command : python test.py file1 file2 However I have another case where only file1 may be present so file1 is mandatory for this script to run however file2 is optionnal : if len(sys.argv) == 2: first_log = sys.argv[1] second_log = sys.argv[2] pthon test.py file1 It gives error : second_log = sys.argv[2] IndexError: list index out of range How do I acheive this because if python test.py file1 file2 then I would process both files . If python test.py file1 is given then I would process file1 using the variable first_log . Please advice . Thanks, From alan.gauld at yahoo.co.uk Sat Jan 26 04:27:01 2019 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Sat, 26 Jan 2019 09:27:01 +0000 Subject: [Tutor] python - files In-Reply-To: References: Message-ID: On 26/01/2019 08:20, Asad wrote: > At present I using : > > if len(sys.argv) == 3: > first = sys.argv[1] > second = sys.argv[2] > else: > print "enter the second argument" > It works well for the following command : > python test.py file1 file2 Correct because it tests if there are 2 arguments passed > However I have another case where only file1 may be present so file1 is > mandatory for this script to run however file2 is optionnal : > > if len(sys.argv) == 2: > first_log = sys.argv[1] > second_log = sys.argv[2] But this will always give an error because you test for only one argument but then try to read two! It will always give an error. > It gives error : > > second_log = sys.argv[2] > IndexError: list index out of range > > > How do I acheive this because if python test.py file1 file2 then I would > process both files . You need to check how many arguments are passed. if len(sys.argv) == 2: # read one argument elif len(sys.argv) == 3: # read 2 arguments elif ... etc You could alternatively use exception handling to catch the IndexError but I think the explicit test in this case more clearly shows what you are intending. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From mats at wichmann.us Sat Jan 26 12:20:59 2019 From: mats at wichmann.us (Mats Wichmann) Date: Sat, 26 Jan 2019 10:20:59 -0700 Subject: [Tutor] python - files In-Reply-To: References: Message-ID: <93fb649b-ed87-a16c-00fc-ee051d8517b3@wichmann.us> On 1/26/19 1:20 AM, Asad wrote: > Hi All , > > I would like to know how do I make and option file as an argument on > command propmnt in python I don't know your context for asking this question. Alan has already explained what you need to do for your issue, and whatever your needs it is certainly worthwhile to understand how sys.argv works. Just wanted to add, you find yourself having a need for processing a non-trivial amount of arguments, this is a problem that is well addressed in the Python community... the standard library now has three modules for that - the "original" (UNIX/POSIX like) getopt module, the currently preferred argparse module, and the optparse module it replaced (which is now considered deprecated), as well as several very good modules which are not in the standard library like docopt and click (just naming two as examples, not endorsing anything). So you don't need to reinvent the wheel here if you have heavy duty needs - spend your time on other parts of the problems you are solving. From cs at cskk.id.au Sat Jan 26 17:27:12 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Sun, 27 Jan 2019 09:27:12 +1100 Subject: [Tutor] python - files In-Reply-To: References: Message-ID: <20190126222712.GA41339@cskk.homeip.net> Mats has mentioned the modules getopt and argparse etc. These are primarily aimed at option parsing ("-v", "-o foo"). Your situation occurs _after_ the option parsing (in your case, there are no options). Alan has talked about explicitly checking the length of sys.argv, much as you are doing, or accessing the (missing) argument and catching an exception. There's a middle ground, which is a little more flexible, which is to _consume_ the command line arguments. The argv array is a list, and can be modified. So: def main(argv): cmd = argv.pop(0) # collect the command word badopts = False # mandatory first argument if not argv: print("%s: missing first argument" % cmd, file=sys.stderr) badopts = True else: first = argv.pop(0) # optional second argument if argv: second = argv.pop(0) # explicit argument 2, use it else: second = None # or some otherdefault if argv: print("%s: extra arguments: %r" % (cmd, argv), file=sys.stderr) badopts = true if badopts: print("%s: invalid invocation, aborting" % cmd, file=sys.stderr) return 2 ... work with first and second ... You can see here that we process the arguments from left to right, consuming the valid ones as we find them, and setting a flag for incorrect arguments. At the end we test the flag and abort if it is set. otherwise we process knowing we have valid values. One important aspect of the above code is that you do not wire in an explicit length for sys.argv such as 2 or 3. That way you can easily change your code later if you want more arguments without running around adjusting such fixed numbers. The other important aspect is usability: the above code complains about each issue it encounters, and finally quits with an additional message. In a real programme that addition message would include a "usage" message which describes the expected arguments. Cheers, Cameron Simpson From mhysnm1964 at gmail.com Sun Jan 27 00:14:35 2019 From: mhysnm1964 at gmail.com (mhysnm1964 at gmail.com) Date: Sun, 27 Jan 2019 16:14:35 +1100 Subject: [Tutor] Web scraping using selenium and navigating nested dictionaries / lists. Message-ID: <004401d4b5ff$33a64690$9af2d3b0$@gmail.com> All, Goal of new project. I want to scrape all my books from Audible.com that I have purchased. Eventually I want to export this as a CSV file or maybe Json. I have not got that far yet. The reasoning behind this is to learn selenium for my work and get the list of books I have purchased. Killing two birds with one stone here. The work focus is to see if selenium can automate some of the testing I have to do and collect useful information from the web page for my reports. This part of the goal is in the future. As I need to build my python skills up. Thus far, I have been successful in logging into Audible and showing the library of books. I am able to store the table of books and want to use BeautifulSoup to extract the relevant information. Information I will want from the table is: * Author * Title * Date purchased * Length * Is the book in a series (there is a link for this) * Link to the page storing the publish details. * Download link Hopefully this has given you enough information on what I am trying to achieve at this stage. AS I learn more about what I am doing, I am adding possible extra's tasks. Such as verifying if I have the book already download via itunes. Learning goals: Using the BeautifulSoup structure that I have extracted from the page source for the table. I want to navigate the tree structure. BeautifulSoup provides children, siblings and parents methods. This is where I get stuck with programming logic. BeautifulSoup does provide find_all method plus selectors which I do not want to use for this exercise. As I want to learn how to walk a tree starting at the root and visiting each node of the tree. Then I can look at the attributes for the tag as I go. I believe I have to set up a recursive loop or function call. Not sure on how to do this. Pseudo code: Build table structure Start at the root node. Check to see if there is any children. Pass first child to function. Print attributes for tag at this level In function, check for any sibling nodes. If exist, call function again If no siblings, then start at first sibling and get its child. This is where I get struck. Each sibling can have children and they can have siblings. So how do I ensure I visit each node in the tree? Any tips or tricks for this would be grateful. As I could use this in other situations. Sean From __peter__ at web.de Sun Jan 27 04:30:12 2019 From: __peter__ at web.de (Peter Otten) Date: Sun, 27 Jan 2019 10:30:12 +0100 Subject: [Tutor] python - files References: <20190126222712.GA41339@cskk.homeip.net> Message-ID: Cameron Simpson wrote: > Mats has mentioned the modules getopt and argparse etc. These are > primarily aimed at option parsing ("-v", "-o foo"). Your situation > occurs _after_ the option parsing (in your case, there are no options). Not argparse. The main advantage over optparse is its handling of positional arguments. Your custom logic > def main(argv): > cmd = argv.pop(0) # collect the command word > badopts = False > # mandatory first argument > if not argv: > print("%s: missing first argument" % cmd, file=sys.stderr) > badopts = True > else: > first = argv.pop(0) > # optional second argument > if argv: > second = argv.pop(0) # explicit argument 2, use it > else: > second = None # or some otherdefault > if argv: > print("%s: extra arguments: %r" % (cmd, argv), file=sys.stderr) > badopts = true > if badopts: > print("%s: invalid invocation, aborting" % cmd, file=sys.stderr) > return 2 > ... work with first and second ... can roughly be replicated with the two lines parser.add_argument("first") parser.add_argument("second", nargs="?") A working script: $ cat test.py #!/usr/bin/python3 import argparse def main(): parser = argparse.ArgumentParser() parser.add_argument("first") parser.add_argument("second", nargs="?") args = parser.parse_args() print("first:", args.first) print("second:", args.second) if __name__ == "__main__": main() $ ./test.py usage: test.py [-h] first [second] test.py: error: the following arguments are required: first $ ./test.py -h usage: test.py [-h] first [second] positional arguments: first second optional arguments: -h, --help show this help message and exit $ ./test.py ONE first: ONE second: None $ ./test.py ONE TWO first: ONE second: TWO $ ./test.py ONE TWO THREE usage: test.py [-h] first [second] test.py: error: unrecognized arguments: THREE Argparse makes a usable standard command line interface easy to set up (if you need non-standard behaviour it gets a bit harder). There is also a companion module argcomplete (not in the stdlib) that enables autocompletion. From cs at cskk.id.au Sun Jan 27 04:54:13 2019 From: cs at cskk.id.au (Cameron Simpson) Date: Sun, 27 Jan 2019 20:54:13 +1100 Subject: [Tutor] python - files In-Reply-To: References: Message-ID: <20190127095413.GA79679@cskk.homeip.net> On 27Jan2019 10:30, Peter Otten <__peter__ at web.de> wrote: >Cameron Simpson wrote: >> Mats has mentioned the modules getopt and argparse etc. These are >> primarily aimed at option parsing ("-v", "-o foo"). Your situation >> occurs _after_ the option parsing (in your case, there are no options). > >Not argparse. The main advantage over optparse is its handling of positional >arguments. I stand corrected. >Your custom logic [...] >can roughly be replicated with the two lines > >parser.add_argument("first") >parser.add_argument("second", nargs="?") >[... extended example ...] Thank you! Cheers, Cameron Simpson From mmistroni at gmail.com Sun Jan 27 05:46:06 2019 From: mmistroni at gmail.com (Marco Mistroni) Date: Sun, 27 Jan 2019 10:46:06 +0000 Subject: [Tutor] Web scraping using selenium and navigating nested dictionaries / lists. In-Reply-To: <004401d4b5ff$33a64690$9af2d3b0$@gmail.com> References: <004401d4b5ff$33a64690$9af2d3b0$@gmail.com> Message-ID: Hi my 2 cents. Have a look at scrapy for scraping.selenium is v good tool to learn but is mainly to automate uat of guis Scrapy will scrape for you and u can automate it via cron. It's same stuff I am doing ATM Hth On Sun, Jan 27, 2019, 8:34 AM All, > > > > Goal of new project. > > I want to scrape all my books from Audible.com that I have purchased. > Eventually I want to export this as a CSV file or maybe Json. I have not > got > that far yet. The reasoning behind this is to learn selenium for my work > and get the list of books I have purchased. Killing two birds with one > stone > here. The work focus is to see if selenium can automate some of the > testing I have to do and collect useful information from the web page for > my > reports. This part of the goal is in the future. As I need to build my > python skills up. > > > > Thus far, I have been successful in logging into Audible and showing the > library of books. I am able to store the table of books and want to use > BeautifulSoup to extract the relevant information. Information I will want > from the table is: > > * Author > * Title > * Date purchased > * Length > * Is the book in a series (there is a link for this) > * Link to the page storing the publish details. > * Download link > > Hopefully this has given you enough information on what I am trying to > achieve at this stage. AS I learn more about what I am doing, I am adding > possible extra's tasks. Such as verifying if I have the book already > download via itunes. > > > > Learning goals: > > Using the BeautifulSoup structure that I have extracted from the page > source for the table. I want to navigate the tree structure. BeautifulSoup > provides children, siblings and parents methods. This is where I get stuck > with programming logic. BeautifulSoup does provide find_all method plus > selectors which I do not want to use for this exercise. As I want to learn > how to walk a tree starting at the root and visiting each node of the tree. > Then I can look at the attributes for the tag as I go. I believe I have to > set up a recursive loop or function call. Not sure on how to do this. > Pseudo > code: > > > > Build table structure > > Start at the root node. > > Check to see if there is any children. > > Pass first child to function. > > Print attributes for tag at this level > > In function, check for any sibling nodes. > > If exist, call function again > > If no siblings, then start at first sibling and get its child. > > > > This is where I get struck. Each sibling can have children and they can > have > siblings. So how do I ensure I visit each node in the tree? > > Any tips or tricks for this would be grateful. As I could use this in other > situations. > > > > Sean > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From __peter__ at web.de Sun Jan 27 06:13:10 2019 From: __peter__ at web.de (Peter Otten) Date: Sun, 27 Jan 2019 12:13:10 +0100 Subject: [Tutor] Web scraping using selenium and navigating nested dictionaries / lists. References: <004401d4b5ff$33a64690$9af2d3b0$@gmail.com> Message-ID: mhysnm1964 at gmail.com wrote: > All, > > > > Goal of new project. > > I want to scrape all my books from Audible.com that I have purchased. > Eventually I want to export this as a CSV file or maybe Json. I have not > got > that far yet. The reasoning behind this is to learn selenium for my work > and get the list of books I have purchased. Killing two birds with one > stone > here. The work focus is to see if selenium can automate some of the > testing I have to do and collect useful information from the web page for > my reports. This part of the goal is in the future. As I need to build my > python skills up. > > > > Thus far, I have been successful in logging into Audible and showing the > library of books. I am able to store the table of books and want to use > BeautifulSoup to extract the relevant information. Information I will want > from the table is: > > * Author > * Title > * Date purchased > * Length > * Is the book in a series (there is a link for this) > * Link to the page storing the publish details. > * Download link > > Hopefully this has given you enough information on what I am trying to > achieve at this stage. AS I learn more about what I am doing, I am adding > possible extra's tasks. Such as verifying if I have the book already > download via itunes. > > > > Learning goals: > > Using the BeautifulSoup structure that I have extracted from the page > source for the table. I want to navigate the tree structure. BeautifulSoup > provides children, siblings and parents methods. This is where I get stuck > with programming logic. BeautifulSoup does provide find_all method plus > selectors which I do not want to use for this exercise. As I want to learn > how to walk a tree starting at the root and visiting each node of the > tree. I think you make your life harder than necessary if you avoid the tools provided by the library you are using. > Then I can look at the attributes for the tag as I go. I believe I > have to set up a recursive loop or function call. Not sure on how to do > this. Pseudo code: > > > > Build table structure > > Start at the root node. > > Check to see if there is any children. > > Pass first child to function. > > Print attributes for tag at this level > > In function, check for any sibling nodes. > > If exist, call function again > > If no siblings, then start at first sibling and get its child. > > > > This is where I get struck. Each sibling can have children and they can > have siblings. So how do I ensure I visit each node in the tree? The problem with your description is that siblings do not matter. Just - process root - iterate over its children and call the function recursively with every child as the new root. To make the function more useful you can pass a function instead of hard- coding what you want to do with the elements. Given def process_elements(elem, do_stuff): do_stuff(elem) for child in elem.children: process_elements(child, do_stuff) you can print all elements with soup = BeautifulSoup(...) process_elements(soup, print) and process_elements(soup, lambda elem: print(elem.name)) will print only the names. You need a bit of error checking to make it work, though. But wait -- Python's generators let you rewrite process_elements so that you can use it without a callback: def gen_elements(elem): yield elem for child in elem.children: yield from gen_elements(child) for elem in gen_elements(soup): print(elem.name) Note that 'yield from iterable' is a shortcut for 'for x in iterable: yield x', so there are actually two loops in gen_elements(). > Any tips or tricks for this would be grateful. As I could use this in > other situations. From mhysnm1964 at gmail.com Sun Jan 27 06:03:43 2019 From: mhysnm1964 at gmail.com (mhysnm1964 at gmail.com) Date: Sun, 27 Jan 2019 22:03:43 +1100 Subject: [Tutor] Web scraping using selenium and navigating nested dictionaries / lists. In-Reply-To: References: <004401d4b5ff$33a64690$9af2d3b0$@gmail.com> Message-ID: <004c01d4b62f$f94e6690$ebeb33b0$@gmail.com> Marco, Thanks. The reason for learning selenium is for the automation. As I want to test web sites for keyboard and mouse interaction and record the results. That at least is the long term goal. In the short term, I will have a look at your suggestion. From: Marco Mistroni Sent: Sunday, 27 January 2019 9:46 PM To: mhysnm1964 at gmail.com Cc: tutor at python.org Subject: Re: [Tutor] Web scraping using selenium and navigating nested dictionaries / lists. Hi my 2 cents. Have a look at scrapy for scraping.selenium is v good tool to learn but is mainly to automate uat of guis Scrapy will scrape for you and u can automate it via cron. It's same stuff I am doing ATM Hth On Sun, Jan 27, 2019, 8:34 AM wrote: All, Goal of new project. I want to scrape all my books from Audible.com that I have purchased. Eventually I want to export this as a CSV file or maybe Json. I have not got that far yet. The reasoning behind this is to learn selenium for my work and get the list of books I have purchased. Killing two birds with one stone here. The work focus is to see if selenium can automate some of the testing I have to do and collect useful information from the web page for my reports. This part of the goal is in the future. As I need to build my python skills up. Thus far, I have been successful in logging into Audible and showing the library of books. I am able to store the table of books and want to use BeautifulSoup to extract the relevant information. Information I will want from the table is: * Author * Title * Date purchased * Length * Is the book in a series (there is a link for this) * Link to the page storing the publish details. * Download link Hopefully this has given you enough information on what I am trying to achieve at this stage. AS I learn more about what I am doing, I am adding possible extra's tasks. Such as verifying if I have the book already download via itunes. Learning goals: Using the BeautifulSoup structure that I have extracted from the page source for the table. I want to navigate the tree structure. BeautifulSoup provides children, siblings and parents methods. This is where I get stuck with programming logic. BeautifulSoup does provide find_all method plus selectors which I do not want to use for this exercise. As I want to learn how to walk a tree starting at the root and visiting each node of the tree. Then I can look at the attributes for the tag as I go. I believe I have to set up a recursive loop or function call. Not sure on how to do this. Pseudo code: Build table structure Start at the root node. Check to see if there is any children. Pass first child to function. Print attributes for tag at this level In function, check for any sibling nodes. If exist, call function again If no siblings, then start at first sibling and get its child. This is where I get struck. Each sibling can have children and they can have siblings. So how do I ensure I visit each node in the tree? Any tips or tricks for this would be grateful. As I could use this in other situations. Sean _______________________________________________ Tutor maillist - Tutor at python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor From mhysnm1964 at gmail.com Sun Jan 27 06:24:30 2019 From: mhysnm1964 at gmail.com (mhysnm1964 at gmail.com) Date: Sun, 27 Jan 2019 22:24:30 +1100 Subject: [Tutor] Web scraping using selenium and navigating nested dictionaries / lists. In-Reply-To: References: <004401d4b5ff$33a64690$9af2d3b0$@gmail.com> Message-ID: <006101d4b632$e0d40b80$a27c2280$@gmail.com> Peter, I am aware that I am avoiding functions that can make my life easier. But I want to learn some of this data structure navigation concepts to improve my skills in programming. What you have provided I will review in depth and have a play with. A big thanks. -----Original Message----- From: Tutor On Behalf Of Peter Otten Sent: Sunday, 27 January 2019 10:13 PM To: tutor at python.org Subject: Re: [Tutor] Web scraping using selenium and navigating nested dictionaries / lists. mhysnm1964 at gmail.com wrote: > All, > > > > Goal of new project. > > I want to scrape all my books from Audible.com that I have purchased. > Eventually I want to export this as a CSV file or maybe Json. I have > not got that far yet. The reasoning behind this is to learn selenium > for my work and get the list of books I have purchased. Killing two > birds with one stone > here. The work focus is to see if selenium can automate some of the > testing I have to do and collect useful information from the web page > for my reports. This part of the goal is in the future. As I need to > build my python skills up. > > > > Thus far, I have been successful in logging into Audible and showing > the library of books. I am able to store the table of books and want > to use BeautifulSoup to extract the relevant information. Information > I will want from the table is: > > * Author > * Title > * Date purchased > * Length > * Is the book in a series (there is a link for this) > * Link to the page storing the publish details. > * Download link > > Hopefully this has given you enough information on what I am trying to > achieve at this stage. AS I learn more about what I am doing, I am > adding possible extra's tasks. Such as verifying if I have the book > already download via itunes. > > > > Learning goals: > > Using the BeautifulSoup structure that I have extracted from the page > source for the table. I want to navigate the tree structure. > BeautifulSoup provides children, siblings and parents methods. This is > where I get stuck with programming logic. BeautifulSoup does provide > find_all method plus selectors which I do not want to use for this > exercise. As I want to learn how to walk a tree starting at the root > and visiting each node of the tree. I think you make your life harder than necessary if you avoid the tools provided by the library you are using. > Then I can look at the attributes for the tag as I go. I believe I > have to set up a recursive loop or function call. Not sure on how to > do this. Pseudo code: > > > > Build table structure > > Start at the root node. > > Check to see if there is any children. > > Pass first child to function. > > Print attributes for tag at this level > > In function, check for any sibling nodes. > > If exist, call function again > > If no siblings, then start at first sibling and get its child. > > > > This is where I get struck. Each sibling can have children and they > can have siblings. So how do I ensure I visit each node in the tree? The problem with your description is that siblings do not matter. Just - process root - iterate over its children and call the function recursively with every child as the new root. To make the function more useful you can pass a function instead of hard- coding what you want to do with the elements. Given def process_elements(elem, do_stuff): do_stuff(elem) for child in elem.children: process_elements(child, do_stuff) you can print all elements with soup = BeautifulSoup(...) process_elements(soup, print) and process_elements(soup, lambda elem: print(elem.name)) will print only the names. You need a bit of error checking to make it work, though. But wait -- Python's generators let you rewrite process_elements so that you can use it without a callback: def gen_elements(elem): yield elem for child in elem.children: yield from gen_elements(child) for elem in gen_elements(soup): print(elem.name) Note that 'yield from iterable' is a shortcut for 'for x in iterable: yield x', so there are actually two loops in gen_elements(). > Any tips or tricks for this would be grateful. As I could use this in > other situations. _______________________________________________ Tutor maillist - Tutor at python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor From asad.hasan2004 at gmail.com Sun Jan 27 09:57:16 2019 From: asad.hasan2004 at gmail.com (Asad) Date: Sun, 27 Jan 2019 20:27:16 +0530 Subject: [Tutor] python-files In-Reply-To: References: Message-ID: Hi All , I tried the following code : parser = argparse.ArgumentParser() parser.add_argument("first") parser.add_argument("second", nargs="?") args = parser.parse_args() print("first:", args.first) print("second:", args.second) When I execute the script it gives error : python python_json_20001_oratest_v1.py "file1" ('first:', 'file1') ('second:', None) Traceback (most recent call last): File "test.py", line 211, in with open(args.second, 'r') as f : TypeError: coercing to Unicode: need string or buffer, NoneType found if I see in line number 211 it with open(args.second, 'r') as f : try : with open(args.second, 'r') as f : for line in f: print line except IOError: print "The default error is err-1 because file2 was not provided " Does that mean my try and except block is not working because if args.second is None as in this case then it should print "The default error is err-1 because file2 was not provided " Please advice , Thanks, > ---------- Forwarded message ---------- > From: Peter Otten <__peter__ at web.de> > To: tutor at python.org > Cc: > Bcc: > Date: Sun, 27 Jan 2019 10:30:12 +0100 > Subject: Re: [Tutor] python - files > Cameron Simpson wrote: > > > Mats has mentioned the modules getopt and argparse etc. These are > > primarily aimed at option parsing ("-v", "-o foo"). Your situation > > occurs _after_ the option parsing (in your case, there are no options). > > Not argparse. The main advantage over optparse is its handling of > positional > arguments. Your custom logic > > > def main(argv): > > cmd = argv.pop(0) # collect the command word > > badopts = False > > # mandatory first argument > > if not argv: > > print("%s: missing first argument" % cmd, file=sys.stderr) > > badopts = True > > else: > > first = argv.pop(0) > > # optional second argument > > if argv: > > second = argv.pop(0) # explicit argument 2, use it > > else: > > second = None # or some otherdefault > > if argv: > > print("%s: extra arguments: %r" % (cmd, argv), file=sys.stderr) > > badopts = true > > if badopts: > > print("%s: invalid invocation, aborting" % cmd, file=sys.stderr) > > return 2 > > ... work with first and second ... > > can roughly be replicated with the two lines > > parser.add_argument("first") > parser.add_argument("second", nargs="?") > > A working script: > > $ cat test.py > #!/usr/bin/python3 > import argparse > > def main(): > parser = argparse.ArgumentParser() > > parser.add_argument("first") > parser.add_argument("second", nargs="?") > > args = parser.parse_args() > > print("first:", args.first) > print("second:", args.second) > > if __name__ == "__main__": > main() > > $ ./test.py > usage: test.py [-h] first [second] > test.py: error: the following arguments are required: first > > $ ./test.py -h > usage: test.py [-h] first [second] > > positional arguments: > first > second > > optional arguments: > -h, --help show this help message and exit > > $ ./test.py ONE > first: ONE > second: None > > $ ./test.py ONE TWO > first: ONE > second: TWO > > $ ./test.py ONE TWO THREE > usage: test.py [-h] first [second] > test.py: error: unrecognized arguments: THREE > > Argparse makes a usable standard command line interface easy to set up (if > you need non-standard behaviour it gets a bit harder). > > There is also a companion module argcomplete (not in the stdlib) that > enables autocompletion. > > > > > > > ---------- Forwarded message ---------- > From: Cameron Simpson > To: tutor at python.org > Cc: > Bcc: > Date: Sun, 27 Jan 2019 20:54:13 +1100 > Subject: Re: [Tutor] python - files > On 27Jan2019 10:30, Peter Otten <__peter__ at web.de> wrote: > >Cameron Simpson wrote: > >> Mats has mentioned the modules getopt and argparse etc. These are > >> primarily aimed at option parsing ("-v", "-o foo"). Your situation > >> occurs _after_ the option parsing (in your case, there are no options). > > > >Not argparse. The main advantage over optparse is its handling of > positional > >arguments. > > I stand corrected. > > >Your custom logic > [...] > >can roughly be replicated with the two lines > > > >parser.add_argument("first") > >parser.add_argument("second", nargs="?") > >[... extended example ...] > > Thank you! > > Cheers, > Cameron Simpson > > > > > ---------- Forwarded message ---------- > From: Marco Mistroni > To: mhysnm1964 at gmail.com > Cc: tutor at python.org > Bcc: > Date: Sun, 27 Jan 2019 10:46:06 +0000 > Subject: Re: [Tutor] Web scraping using selenium and navigating nested > dictionaries / lists. > Hi my 2 cents. Have a look at scrapy for scraping.selenium is v good tool > to learn but is mainly to automate uat of guis > Scrapy will scrape for you and u can automate it via cron. It's same stuff > I am doing ATM > Hth > > On Sun, Jan 27, 2019, 8:34 AM > > All, > > > > > > > > Goal of new project. > > > > I want to scrape all my books from Audible.com that I have purchased. > > Eventually I want to export this as a CSV file or maybe Json. I have not > > got > > that far yet. The reasoning behind this is to learn selenium for my > work > > and get the list of books I have purchased. Killing two birds with one > > stone > > here. The work focus is to see if selenium can automate some of the > > testing I have to do and collect useful information from the web page for > > my > > reports. This part of the goal is in the future. As I need to build my > > python skills up. > > > > > > > > Thus far, I have been successful in logging into Audible and showing the > > library of books. I am able to store the table of books and want to use > > BeautifulSoup to extract the relevant information. Information I will > want > > from the table is: > > > > * Author > > * Title > > * Date purchased > > * Length > > * Is the book in a series (there is a link for this) > > * Link to the page storing the publish details. > > * Download link > > > > Hopefully this has given you enough information on what I am trying to > > achieve at this stage. AS I learn more about what I am doing, I am adding > > possible extra's tasks. Such as verifying if I have the book already > > download via itunes. > > > > > > > > Learning goals: > > > > Using the BeautifulSoup structure that I have extracted from the page > > source for the table. I want to navigate the tree structure. > BeautifulSoup > > provides children, siblings and parents methods. This is where I get > stuck > > with programming logic. BeautifulSoup does provide find_all method plus > > selectors which I do not want to use for this exercise. As I want to > learn > > how to walk a tree starting at the root and visiting each node of the > tree. > > Then I can look at the attributes for the tag as I go. I believe I have > to > > set up a recursive loop or function call. Not sure on how to do this. > > Pseudo > > code: > > > > > > > > Build table structure > > > > Start at the root node. > > > > Check to see if there is any children. > > > > Pass first child to function. > > > > Print attributes for tag at this level > > > > In function, check for any sibling nodes. > > > > If exist, call function again > > > > If no siblings, then start at first sibling and get its child. > > > > > > > > This is where I get struck. Each sibling can have children and they can > > have > > siblings. So how do I ensure I visit each node in the tree? > > > > Any tips or tricks for this would be grateful. As I could use this in > other > > situations. > > > > > > > > Sean > > > > _______________________________________________ > > Tutor maillist - Tutor at python.org > > To unsubscribe or change subscription options: > > https://mail.python.org/mailman/listinfo/tutor > > > > _______________________________________________ > Tutor maillist - Tutor at python.org > https://mail.python.org/mailman/listinfo/tutor > -- Asad Hasan +91 9582111698 From __peter__ at web.de Sun Jan 27 13:52:41 2019 From: __peter__ at web.de (Peter Otten) Date: Sun, 27 Jan 2019 19:52:41 +0100 Subject: [Tutor] python-files References: Message-ID: Asad wrote: > Hi All , > > I tried the following code : > > parser = argparse.ArgumentParser() > parser.add_argument("first") > parser.add_argument("second", nargs="?") > args = parser.parse_args() > print("first:", args.first) > > print("second:", args.second) > > When I execute the script it gives error : > > python python_json_20001_oratest_v1.py "file1" > ('first:', 'file1') > ('second:', None) > Traceback (most recent call last): > File "test.py", line 211, in > with open(args.second, 'r') as f : > TypeError: coercing to Unicode: need string or buffer, NoneType found > > > if I see in line number 211 it with open(args.second, 'r') as f : > > try : > with open(args.second, 'r') as f : > for line in f: > print line > except IOError: > print "The default error is err-1 because file2 was not provided > " How do you know that the file was not provided? The name might have been misspelt or the user lacks the permission to read it. > Does that mean my try and except block is not working because if > args.second is None as in this case then it should print "The default > error is err-1 because file2 was not provided " > > Please advice , If you just want to terminate the script with a helpful message you should make the second argument mandatory, too: parser = argparse.ArgumentParser() parser.add_argument("first") parser.add_argument("second") args = parser.parse_args() That way the parse_args() call will terminate the script and the user will see an error message immediately. If for some reason you want to keep the argument optional you can check for None before trying to open the file: if args.second is not None: try: with open(args.second, 'r') as f : for line in f: print line except IOError as err: print err # print actually what happened, not what you guess else: print "File 'second' was not provided on the command line" From alan.gauld at yahoo.co.uk Sun Jan 27 20:01:50 2019 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Mon, 28 Jan 2019 01:01:50 +0000 Subject: [Tutor] python-files In-Reply-To: References: Message-ID: On 27/01/2019 14:57, Asad wrote: > print("first:", args.first) > print("second:", args.second) > > When I execute the script it gives error : > > python python_json_20001_oratest_v1.py "file1" > ('first:', 'file1') > ('second:', None) Note that the second file is None. > Traceback (most recent call last): > File "test.py", line 211, in > with open(args.second, 'r') as f : > TypeError: coercing to Unicode: need string or buffer, NoneType found Note that you get a TypeError because you passed None instead of a valid filename. > try : > with open(args.second, 'r') as f : > for line in f: > print line > except IOError: > print "The default error is err-1 because file2 was not provided " Note that you are trying to catch an IOError but you are getting a TypeError. You need to add another except clause for the TypeError. Or test for a None second file value at the start of your code... > Does that mean my try and except block is not working because if > args.second is None as in this case then it should print "The default > error is err-1 because file2 was not provided " It is working. You just aren't catching the correct error type. It will only type the message you've given if you get an IOError, but the open() code isn't getting that far, it's failing on the parameter type. > Please advice , OK, I advise you to delete the text below where you no longer need it. Some users pay by the byte and all this excess text costs them money. More than 75% of your message is included text that we have already seen. > Thanks, > > >> ---------- Forwarded message ---------- >> From: Peter Otten <__peter__ at web.de> >> To: tutor at python.org ... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From jetspacejt at gmail.com Tue Jan 29 16:07:38 2019 From: jetspacejt at gmail.com (jetspacejt) Date: Tue, 29 Jan 2019 16:07:38 -0500 Subject: [Tutor] Not able to get to Script Mode Message-ID: Using version 3.7.1 Where is File Edit Shell...etc. Not at the top of my screen From breamoreboy at gmail.com Tue Jan 29 18:38:35 2019 From: breamoreboy at gmail.com (Mark Lawrence) Date: Tue, 29 Jan 2019 23:38:35 +0000 Subject: [Tutor] Not able to get to Script Mode In-Reply-To: References: Message-ID: On 29/01/2019 21:07, jetspacejt wrote: > Using version 3.7.1 > Where is File Edit Shell...etc. > Not at the top of my screen Please give more detail as we're not mind readers. What are you running, python from a command line, IDLE, some other IDE? What OS are you on? -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence From mats at wichmann.us Tue Jan 29 18:39:43 2019 From: mats at wichmann.us (Mats Wichmann) Date: Tue, 29 Jan 2019 16:39:43 -0700 Subject: [Tutor] Not able to get to Script Mode In-Reply-To: References: Message-ID: On 1/29/19 2:07 PM, jetspacejt wrote: > Using version 3.7.1 > Where is File Edit Shell...etc. > Not at the top of my screen You're looking (probably) for IDLE, not for Python itself (IDLE is an editor/simple IDE written in Python, that is usually bundled with Python). On the wild guess you're using Windows (it always helps to tell us. Please?), start typing IDLE in the seach box, that should get you there. From alan.gauld at yahoo.co.uk Tue Jan 29 18:47:55 2019 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Tue, 29 Jan 2019 23:47:55 +0000 Subject: [Tutor] Not able to get to Script Mode In-Reply-To: References: Message-ID: On 29/01/2019 21:07, jetspacejt wrote: > Using version 3.7.1 > Where is File Edit Shell...etc. > Not at the top of my screen Vanilla python is just a language interpreter, it has no GUI. If you want a GUI environment you need to use an IDE - there are dozens to choose from. There is a very basic one that comes with most versions of Python called IDLE. It will get you started but you may want to try some of the more advanced options if you get serious. This page should help you choose: https://wiki.python.org/moin/IntegratedDevelopmentEnvironments Most (but not all) of them are free. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From asad.hasan2004 at gmail.com Wed Jan 30 09:06:47 2019 From: asad.hasan2004 at gmail.com (Asad) Date: Wed, 30 Jan 2019 19:36:47 +0530 Subject: [Tutor] Webinterface for python script Message-ID: Hi All , I have created certain python scripts to analyze log files and suggest solution based on logic which I invoke on the command line . I need some information on how to execute these through browser . I am using : python test.py file1 file2 How do I use the browser to upload the files file1 and file2 and it process the files . Please advice , Thanks, -- From joel.goldstick at gmail.com Wed Jan 30 11:42:13 2019 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Wed, 30 Jan 2019 11:42:13 -0500 Subject: [Tutor] Webinterface for python script In-Reply-To: References: Message-ID: On Wed, Jan 30, 2019 at 11:20 AM Asad wrote: > > Hi All , > > I have created certain python scripts to analyze log files and > suggest solution based on logic which I invoke on the command line . I need > some information on how to execute these through browser . I am using : > > python test.py file1 file2 > > How do I use the browser to upload the files file1 and file2 and it process > the files . > > Please advice , > > Thanks, > Django may be overkill, but there is a large support community. There are other smaller web frameworks, flask is one I can recall. There is a learning curve, but once you get past that, you could do this in a couple days -- Joel Goldstick http://joelgoldstick.com/blog http://cc-baseballstats.info/stats/birthdays From alan.gauld at yahoo.co.uk Wed Jan 30 13:53:47 2019 From: alan.gauld at yahoo.co.uk (Alan Gauld) Date: Wed, 30 Jan 2019 18:53:47 +0000 Subject: [Tutor] Webinterface for python script In-Reply-To: References: Message-ID: On 30/01/2019 14:06, Asad wrote: > I have created certain python scripts to analyze log files and > suggest solution based on logic which I invoke on the command line . I need > some information on how to execute these through browser . I am using : > > python test.py file1 file2 > > How do I use the browser to upload the files file1 and file2 and it process > the files . It's not clear how you think this will work. You say 'upload the files through a browser'. Where do you want to upload them too? Do you have a web server somewhere or can you set one up? If so then you need to write a web application to select the local files from the users PC and upload them to the server. Once you have that, you can call your Python script on the server and display the output as HTML back on the users browser. To do that will probably involve changes to your existing code to make it callable as a function and to have the output displayable as HTML. There are many web frameworks you can use and for this type of project any of them will suffice. There is a page summarising the most common here: https://wiki.python.org/moin/WebFrameworks/ I recommend Flask and you can see a short tutorial on its use in my tutor under the topic "Using Web App Frameworks." There are several other, more in-depth, tutorials if you think its for you. For the browser specific bits you can probably find all the code you need via Google... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos From mhysnm1964 at gmail.com Thu Jan 31 05:22:39 2019 From: mhysnm1964 at gmail.com (mhysnm1964 at gmail.com) Date: Thu, 31 Jan 2019 21:22:39 +1100 Subject: [Tutor] How to get Selenium to wait for page load Message-ID: <001601d4b94e$e62ddce0$b28996a0$@gmail.com> Hi all, I have found an excellent article on identifying stale elements. The issue is when I try and use their example code. I get a failure where for_wait is not defined. http://www.obeythetestinggoat.com/how-to-get-selenium-to-wait-for-page-load- after-a-click.html Traceback (most recent call last): File "", line 5, in NameError: name 'wait_for' is not defined >>> When I look through the examples. I only find one reference for the above function. But it doesn't look correct and I am confused. The author indicates the definition of the function is half way up the page. As I cannot see, this reference doesn't help. So can someone help and provide the definition for this method/function? The code which generated the error: rows = [] pageNav = browser.find_element_by_id("center-5") curr_page = pageNav.find_element_by_css_selector('span .pageNumberElement').text prev_page = "" while prev_page in curr_page: # wait_for(link_has_gone_stale): prev_page = curr_page rows.extend(tableNavigation (pageNav)) if wait_for(link_has_gone_stale): pageNav = browser.find_element_by_id("center-5") curr_page = pageNav.find_element_by_css_selector('span .pageNumberElement').text if prev_page == curr_page: print ("Page no has not changed:",curr_page) else: prev_page = curr_page From mmistroni at gmail.com Thu Jan 31 06:14:27 2019 From: mmistroni at gmail.com (Marco Mistroni) Date: Thu, 31 Jan 2019 11:14:27 +0000 Subject: [Tutor] How to get Selenium to wait for page load In-Reply-To: <001601d4b94e$e62ddce0$b28996a0$@gmail.com> References: <001601d4b94e$e62ddce0$b28996a0$@gmail.com> Message-ID: Hi You won't find much help here as this is a python moist Have a look at WebDriverWait from selenium.webdriver.support.ui It allows you to wait for certain conditions Hth On Thu, Jan 31, 2019, 11:09 AM Hi all, > > > > I have found an excellent article on identifying stale elements. The issue > is when I try and use their example code. I get a failure where for_wait is > not defined. > > > http://www.obeythetestinggoat.com/how-to-get-selenium-to-wait-for-page-load- > after-a-click.html > > > > > Traceback (most recent call last): > File "", line 5, in > NameError: name 'wait_for' is not defined > >>> > > > > When I look through the examples. I only find one reference for the above > function. But it doesn't look correct and I am confused. The author > indicates the definition of the function is half way up the page. As I > cannot see, this reference doesn't help. So can someone help and provide > the > definition for this method/function? > > > > The code which generated the error: > > > > rows = [] > > pageNav = browser.find_element_by_id("center-5") > > curr_page = pageNav.find_element_by_css_selector('span > .pageNumberElement').text > > prev_page = "" > > while prev_page in curr_page: > > # wait_for(link_has_gone_stale): > > prev_page = curr_page > > rows.extend(tableNavigation (pageNav)) > > if wait_for(link_has_gone_stale): > > pageNav = browser.find_element_by_id("center-5") > > curr_page = pageNav.find_element_by_css_selector('span > .pageNumberElement').text > > if prev_page == curr_page: > > print ("Page no has not changed:",curr_page) > > else: > > prev_page = curr_page > > > > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > https://mail.python.org/mailman/listinfo/tutor > From breamoreboy at gmail.com Thu Jan 31 06:32:27 2019 From: breamoreboy at gmail.com (Mark Lawrence) Date: Thu, 31 Jan 2019 11:32:27 +0000 Subject: [Tutor] How to get Selenium to wait for page load In-Reply-To: References: <001601d4b94e$e62ddce0$b28996a0$@gmail.com> Message-ID: On 31/01/2019 11:14, Marco Mistroni wrote: > Hi > You won't find much help here as this is a python moist I've no idea what the above is meant to mean. > Have a look at WebDriverWait from selenium.webdriver.support.ui > > It allows you to wait for certain conditions > Hth > Please don't top post here as it's so damn irritating when reading long threads. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence