From md123 at nycap.rr.com Mon Jul 1 03:41:31 2013 From: md123 at nycap.rr.com (Matt D) Date: Sun, 30 Jun 2013 21:41:31 -0400 Subject: [Tutor] Need help appending data to a logfile In-Reply-To: References: <51BF4905.6010801@nycap.rr.com> <51C78BD7.9030502@davea.name> <51C8BC8A.9060008@nycap.rr.com> <51C8C0E4.9040508@davea.name> <51CC4DF4.3020205@nycap.rr.com> <51CC5276.2080807@nycap.rr.com> <51CC6082.4060102@davea.name> <51CC6964.4040704@nycap.rr.com> <51CC6E3A.8020609@davea.name> <51CCAA1F.2020808@nycap.rr.com> <51CCB266.5050206@davea.name> <51CD7328.8070407@nycap.rr.com> <51CD7592.7030902@davea.name> <51CD8D40.4000700@nycap.rr.com> <51CD92C9.6040303@pearwood.info> <51CDEA08.6060407@nycap.rr.com> <51CDFF3E.8030308@nycap.rr.com> <51CE270F.2050302@davea.name> <51CED25A.9070807@nycap.rr.com> <51CEDDF7.60107@davea.name> <51CEF6A4.9060901@nycap.rr.com> <51CF8BFC.2090608@pearwood.info> <51CFA902.4090509@nycap.rr.com> <51D03AE3.1060600@nycap.rr.com> <51D051DA.8020307@nycap.rr.com> Message-ID: <51D0DE4B.3020008@nycap.rr.com> On 06/30/2013 01:43 PM, Alan Gauld wrote: > On 30/06/13 16:42, Matt D wrote: > >> im sorry i don't understand how to pass the file name from the open >> dialog into the update method? I'm sorry i just don't get it. so if >> the user opened file is 'handle' like this?: > > OK, technically you probably donb;t want to pasws it into the metjod but > to store it in an attribute of the object. > >> def openFile(self, evt): >> with wx.FileDialog(self, "Choose a file", os.getcwd(), "", >> "*.txt*", wx.OPEN) as dlg: >> if dlg.ShowModal() == wx.ID_OK: >> self.logpath = dlg.GetPath() > > Now self.logpath holds a record of the full path to the log file. > (assuming dlg.getPath does return the full path, I haven't > used wxPython for a while...) > >> def update(self, field_values): >> with open(self.logpath, 'a') as logfile: >> # format the line and write to file as before > > And that is all you need. > logfile is now your open logfile object referencing the file named by > the user in the dialog. Because we opened it in a 'with' construct it > will automatically be closed at the end of the block ready to be > reopened next time update gets called. > > And that is it. No more should be required. Provided you have permission > to write to the file/folder then you are now logging > to the file specified by the user. > I got it done with this: def openFile(self, evt): with wx.FileDialog(self, "Choose a file", os.getcwd(), "", "*.txt*", wx.OPEN) as dlg: if dlg.ShowModal() == wx.ID_OK: path = dlg.GetPath() in_data = self.logfile.read() user_file = open(path, 'a') user_file.write(in_data) user_file.close() so when the user starts the program strait away its automatically logging. if the user wishes, before closing, he can save the logged data to any file he wishes. if the program is left running unattended, for lets say five days, and the program dies its ok because its automatically sort of 'backed up' in the logfile.txt and can be appended to the appropriate file on restart. no variables or arrays or lists hogging memory. From phil_lor at bigpond.com Mon Jul 1 10:02:57 2013 From: phil_lor at bigpond.com (Phil) Date: Mon, 01 Jul 2013 18:02:57 +1000 Subject: [Tutor] Self and class functions In-Reply-To: <51CFD1F0.3080000@davea.name> References: <51CF616F.8060300@bigpond.com> <51CF8CD2.50509@davea.name> <51CFAF9F.8090603@bigpond.com> <51CFD1F0.3080000@davea.name> Message-ID: <51D137B1.7070101@bigpond.com> On 30/06/13 16:36, Dave Angel wrote: > mywindow = DrawTest() > > But since I don't use QT, I can't tell you if that's reasonable or not. > It is. I woke up during the early hours of the morning with the answer. I don't know why I was so intent on instantiating the main window class when it was an instance of the dialog class that was needed. I remember when I was a university student, many years ago, a lecturer suggesting that even seeking advice from our pet dog can often lead to a solution. Taking a break from from a problem also often works me. -- Regards, Phil From johnsteedman360 at gmail.com Mon Jul 1 11:58:37 2013 From: johnsteedman360 at gmail.com (John Steedman) Date: Mon, 1 Jul 2013 10:58:37 +0100 Subject: [Tutor] Test Question Message-ID: Good morning all, A question that I am unsure about. I THINK I have the basics, but I am not sure and remain curious. 1. What does this mean? >>> if my_object in my_sequence: ... 2. What can go wrong with this? What should a code review pick up on? I believe that "my_sequence" might be a either container class or a sequence type. An effective __hash__ function would be required for each "my_object". I HTINK you'd need to avoid using floating point variables that might round incorrectly. Are there other issues? Many thanks indeed. John -------------- next part -------------- An HTML attachment was scrubbed... URL: From davea at davea.name Mon Jul 1 13:01:11 2013 From: davea at davea.name (Dave Angel) Date: Mon, 01 Jul 2013 07:01:11 -0400 Subject: [Tutor] Test Question In-Reply-To: References: Message-ID: <51D16177.6070409@davea.name> On 07/01/2013 05:58 AM, John Steedman wrote: > Good morning all, > > A question that I am unsure about. I THINK I have the basics, but I am not > sure and remain curious. > > 1. What does this mean? >>>> if my_object in my_sequence: > ... We can be sure what 'if' and 'in' mean, but not the other two items. They're just names, and the behavior of the test will depend on the types of the objects the names are bound to. By calling it my_sequence, you're implying that the object is not only a collection, but an ordered one. So if we trust the names, this will iterate through the sequence, testing each item in the sequence against my_object for "==" and stop when a match is found. If one is found the if clause will execute, and if the sequence is exhausted without finding one, the else clause (or equivalent) will execute. > > 2. What can go wrong with this? What should a code review pick up on? The main thing that can go wrong is that the objects might not match the names used. For example, if the name my_sequence is bound to an int, you'll get a runtime exception. Second, if the items look similar (eg. floating point, but not limited to that), but aren't actually equal, you could get a surprise. For example if my_object is a byte string, and one of the items in my_sequence is a unicode string representing exactly the same thing. Python 2 will frequently consider them the same, and Python 3 will know they're different. Third if my_object is something that doesn't equal anything else, such as a floating point NAN. Two NANs are not equal, and a NAN is not even equal to itself. By a different definition of 'wrong' if the sequence is quite large, and if all its items are hashable and it may have been faster to pass a dict instead of a sequence. And if my_sequence is a dict, it'll probably be faster, but the name is a misleading one. > > I believe that "my_sequence" might be a either container class or a > sequence type. An effective __hash__ function would be required for each > "my_object". "in" doesn't care if there's a __hash__ function. It just cares if the collection has a __contains__() method. If the collection is a dict, the dict will enforce whatever constraints it needs. If the collection is a list, no has is needed, but the __contains__() method will probably be slower. In an arbitrary sequence it won't have a __contains__() method, and I believe 'in' will iterate. > I HTINK you'd need to avoid using floating point variables > that might round incorrectly. > One of the issues already covered. > Are there other issues? > Those are all I can think of off the top of my head. -- DaveA From hugo.yoshi at gmail.com Mon Jul 1 13:21:29 2013 From: hugo.yoshi at gmail.com (Hugo Arts) Date: Mon, 1 Jul 2013 13:21:29 +0200 Subject: [Tutor] Test Question In-Reply-To: <51D16177.6070409@davea.name> References: <51D16177.6070409@davea.name> Message-ID: On Mon, Jul 1, 2013 at 1:01 PM, Dave Angel wrote: > On 07/01/2013 05:58 AM, John Steedman wrote: > >> >> I believe that "my_sequence" might be a either container class or a >> sequence type. An effective __hash__ function would be required for each >> "my_object". >> > > "in" doesn't care if there's a __hash__ function. It just cares if the > collection has a __contains__() method. If the collection is a dict, the > dict will enforce whatever constraints it needs. If the collection is a > list, no has is needed, but the __contains__() method will probably be > slower. In an arbitrary sequence it won't have a __contains__() method, > and I believe 'in' will iterate. > > It will indeed iterate. It's always useful to check the language reference in cases like these to see precise behaviour: http://docs.python.org/2/reference/expressions.html#membership-test-details Hugo -------------- next part -------------- An HTML attachment was scrubbed... URL: From oscar.j.benjamin at gmail.com Mon Jul 1 14:15:39 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 1 Jul 2013 13:15:39 +0100 Subject: [Tutor] Test Question In-Reply-To: <51D16177.6070409@davea.name> References: <51D16177.6070409@davea.name> Message-ID: On 1 July 2013 12:01, Dave Angel wrote: > Third if my_object is something that doesn't equal anything else, such as a > floating point NAN. Two NANs are not equal, and a NAN is not even equal to > itself. Many builtin collection types do an identity 'is' check before an equality '==' check so nan behaviour depends on whether it is the exact same nan: >>> float('nan') in [1, 2, 3, float('nan')] False >>> nan = float('nan') >>> nan in [1, 2, 3, nan] True I don't know to what extent that is a deliberate feature or an optimisation that wasn't fully considered but it at least maintains the following invariant: >>> my_sequence = [1, 2, 3, float('nan')] >>> all(item in my_sequence for item in my_sequence) True Oscar From steve at pearwood.info Mon Jul 1 15:27:19 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 01 Jul 2013 23:27:19 +1000 Subject: [Tutor] Test Question In-Reply-To: References: <51D16177.6070409@davea.name> Message-ID: <51D183B7.3060006@pearwood.info> On 01/07/13 22:15, Oscar Benjamin wrote: > On 1 July 2013 12:01, Dave Angel wrote: >> Third if my_object is something that doesn't equal anything else, such as a >> floating point NAN. Two NANs are not equal, and a NAN is not even equal to >> itself. > > Many builtin collection types do an identity 'is' check before an > equality '==' check so nan behaviour depends on whether it is the > exact same nan: > >>>> float('nan') in [1, 2, 3, float('nan')] > False >>>> nan = float('nan') >>>> nan in [1, 2, 3, nan] > True > > I don't know to what extent that is a deliberate feature or an > optimisation that wasn't fully considered but it at least maintains > the following invariant: It's an optimization, but one which by declaration of the BDFL stands. It simply isn't worth slowing down list processing for something as uncommon as NANs. Lists are general purpose containers, and shouldn't be expected to understand every little quirk of every weird value. I am a strong supporter of the IEEE 754 behaviour of NANs, namely that if x is a NAN, x == x returns False. There are good reasons for this, *in the context of numerical computing*. Nevertheless, I support Guido's ruling on this. Lists are not part of IEEE 754, neither is object identity, and if somebody wants a list that is NAN-aware, they can make one: # Untested. class MyList(list): def __contains__(self, value): if math.isnan(value): return False return super().__contains__(value) -- Steven From steve at pearwood.info Mon Jul 1 15:40:43 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 01 Jul 2013 23:40:43 +1000 Subject: [Tutor] Test Question In-Reply-To: References: Message-ID: <51D186DB.6040301@pearwood.info> On 01/07/13 19:58, John Steedman wrote: > Good morning all, > > A question that I am unsure about. I THINK I have the basics, but I am not > sure and remain curious. > > 1. What does this mean? >>>> if my_object in my_sequence: > ... Others have already answered this, but for completion, it is testing whether "my_object" can be found as an element of the sequence, iterable, or container "my_sequence". > 2. What can go wrong with this? What should a code review pick up on? Depends on context. Without context, nearly anything could go wrong: NameError -- perhaps one or both of the names are undefined; TypeError -- perhaps the names are misleading, and my_sequence is not a sequence at all; Perhaps my_sequence is an infinite iterator and the above will never complete; etc. > I believe that "my_sequence" might be a either container class or a > sequence type. An effective __hash__ function would be required for each > "my_object". Correct, if my_sequence is in fact a dict or other mapping that relies on hashing. But in fact it's not just the presence of a __hash__ method on my_object which is required, but that the __hash__ method can actually return. E.g. tuples have a __hash__ method, but that relies on every element of the tuple being hashable: py> (1, 2, 3) in {} False py> (1, 2, [3]) in {} Traceback (most recent call last): File "", line 1, in TypeError: unhashable type: 'list' > I HTINK you'd need to avoid using floating point variables > that might round incorrectly. No, Python floats are not rounded when doing containment testing. They may have been rounded earlier, but `x in container` will use the full precision of the float. > Are there other issues? Nothing obvious to me. -- Steven From jhamerly at medford.k12.ma.us Mon Jul 1 18:26:55 2013 From: jhamerly at medford.k12.ma.us (jhamerly at medford.k12.ma.us) Date: Mon, 01 Jul 2013 12:26:55 -0400 Subject: [Tutor] How to remove from mailing list when server access blocked? In-Reply-To: <51CDC615.2000303@davea.name> References: <7872a015-f2bb-4e9d-8d26-75bde933e918@email.android.com> <51CDC615.2000303@davea.name> Message-ID: <20130701122655.5d87dw53wgg04s44@portal.medford.k12.ma.us> Hello, Can anyone tell me how to remove myself from this mailing list? When I click on unsubscribe my browser displays a message that I can not access the unsubscribe url or server because my IP is on a blacklist at Spamhaus. I am writing from my work email and don't have permission to change settings, etc. What are my options? Thank you, Jessica H. From ramit.prasad at jpmorgan.com Mon Jul 1 18:57:13 2013 From: ramit.prasad at jpmorgan.com (Prasad, Ramit) Date: Mon, 1 Jul 2013 16:57:13 +0000 Subject: [Tutor] How to remove from mailing list when server access blocked? In-Reply-To: <20130701122655.5d87dw53wgg04s44@portal.medford.k12.ma.us> References: <7872a015-f2bb-4e9d-8d26-75bde933e918@email.android.com> <51CDC615.2000303@davea.name> <20130701122655.5d87dw53wgg04s44@portal.medford.k12.ma.us> Message-ID: <5B80DD153D7D744689F57F4FB69AF47418512BB8@SCACMX008.exchad.jpmchase.net> jhamerly at medford.k12.ma.us wrote: > Hello, > > Can anyone tell me how to remove myself from this mailing list? When I > click on unsubscribe my browser displays a message that I can not > access the unsubscribe url or server because my IP is on a blacklist > at Spamhaus. I am writing from my work email and don't have permission > to change settings, etc. > > What are my options? > > Thank you, > Jessica H. > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor Use the above link that gets appended to every message. ~Ramit P This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email. From davea at davea.name Mon Jul 1 20:34:10 2013 From: davea at davea.name (Dave Angel) Date: Mon, 01 Jul 2013 14:34:10 -0400 Subject: [Tutor] How to remove from mailing list when server access blocked? In-Reply-To: <20130701122655.5d87dw53wgg04s44@portal.medford.k12.ma.us> References: <7872a015-f2bb-4e9d-8d26-75bde933e918@email.android.com> <51CDC615.2000303@davea.name> <20130701122655.5d87dw53wgg04s44@portal.medford.k12.ma.us> Message-ID: <51D1CBA2.3080501@davea.name> On 07/01/2013 12:26 PM, jhamerly at medford.k12.ma.us wrote: > Hello, > > Can anyone tell me how to remove myself from this mailing list? When I > click on unsubscribe my browser displays a message that I can not access > the unsubscribe url or server because my IP is on a blacklist at > Spamhaus. I am writing from my work email and don't have permission to > change settings, etc. > > What are my options? > > Sorry that Ramit didn't read your email carefully enough. I'd suggest you tell your company's IT person about the Spamhaus problem, so she can get it cleared up. But you can unsubscribe entirely by email: Send an email to: tutor-request at python.org with a subject of Help and you'll get an email back with instructions. Probably all you'll need then is to reply, changing the subject to Unsubscribe -- DaveA From ramit.prasad at jpmorgan.com Mon Jul 1 21:08:02 2013 From: ramit.prasad at jpmorgan.com (Prasad, Ramit) Date: Mon, 1 Jul 2013 19:08:02 +0000 Subject: [Tutor] How to remove from mailing list when server access blocked? In-Reply-To: <51D1CBA2.3080501@davea.name> References: <7872a015-f2bb-4e9d-8d26-75bde933e918@email.android.com> <51CDC615.2000303@davea.name> <20130701122655.5d87dw53wgg04s44@portal.medford.k12.ma.us> <51D1CBA2.3080501@davea.name> Message-ID: <5B80DD153D7D744689F57F4FB69AF47418513290@SCACMX008.exchad.jpmchase.net> Dave Angel wrote: > On 07/01/2013 12:26 PM, jhamerly at medford.k12.ma.us wrote: > > Hello, > > > > Can anyone tell me how to remove myself from this mailing list? When I > > click on unsubscribe my browser displays a message that I can not access > > the unsubscribe url or server because my IP is on a blacklist at > > Spamhaus. I am writing from my work email and don't have permission to > > change settings, etc. > > > > What are my options? > > > > > > Sorry that Ramit didn't read your email carefully enough. > [snip] Oops! :( Saw another email about unsubscribing and wrongly assumed it was the same. Thanks for the correction Dave. > > -- > DaveA ~Ramit P This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email. From johnsteedman360 at gmail.com Mon Jul 1 21:27:12 2013 From: johnsteedman360 at gmail.com (John Steedman) Date: Mon, 1 Jul 2013 20:27:12 +0100 Subject: [Tutor] Test Question In-Reply-To: <51D186DB.6040301@pearwood.info> References: <51D186DB.6040301@pearwood.info> Message-ID: Many thanks, everyone. Great answers. I decided to read the manual properly. May take some time but well worth it. On Mon, Jul 1, 2013 at 2:40 PM, Steven D'Aprano wrote: > On 01/07/13 19:58, John Steedman wrote: > >> Good morning all, >> >> A question that I am unsure about. I THINK I have the basics, but I am >> not >> sure and remain curious. >> >> 1. What does this mean? >> >>> if my_object in my_sequence: >>>>> >>>> ... >> > > > Others have already answered this, but for completion, it is testing > whether "my_object" can be found as an element of the sequence, iterable, > or container "my_sequence". > > > > 2. What can go wrong with this? What should a code review pick up on? >> > > Depends on context. Without context, nearly anything could go wrong: > > NameError -- perhaps one or both of the names are undefined; > > TypeError -- perhaps the names are misleading, and my_sequence is not a > sequence at all; > > Perhaps my_sequence is an infinite iterator and the above will never > complete; > > etc. > > > I believe that "my_sequence" might be a either container class or a >> sequence type. An effective __hash__ function would be required for each >> "my_object". >> > > Correct, if my_sequence is in fact a dict or other mapping that relies on > hashing. > > But in fact it's not just the presence of a __hash__ method on my_object > which is required, but that the __hash__ method can actually return. E.g. > tuples have a __hash__ method, but that relies on every element of the > tuple being hashable: > > py> (1, 2, 3) in {} > False > py> (1, 2, [3]) in {} > Traceback (most recent call last): > File "", line 1, in > TypeError: unhashable type: 'list' > > > > I HTINK you'd need to avoid using floating point variables >> that might round incorrectly. >> > > No, Python floats are not rounded when doing containment testing. They may > have been rounded earlier, but `x in container` will use the full precision > of the float. > > > Are there other issues? >> > > Nothing obvious to me. > > > > > > -- > Steven > ______________________________**_________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/**mailman/listinfo/tutor > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jacklittlemc at yahoo.com Mon Jul 1 22:41:01 2013 From: jacklittlemc at yahoo.com (Jack Little) Date: Mon, 1 Jul 2013 13:41:01 -0700 (PDT) Subject: [Tutor] Boosts Message-ID: <1372711261.40117.YahooMailNeo@web124504.mail.ne1.yahoo.com> In my concept, when the player buys a boost, their karma (a global) is multiplied by 25% when added to. How would I do this? Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From davea at davea.name Mon Jul 1 22:48:06 2013 From: davea at davea.name (Dave Angel) Date: Mon, 01 Jul 2013 16:48:06 -0400 Subject: [Tutor] Boosts In-Reply-To: <1372711261.40117.YahooMailNeo@web124504.mail.ne1.yahoo.com> References: <1372711261.40117.YahooMailNeo@web124504.mail.ne1.yahoo.com> Message-ID: <51D1EB06.1010000@davea.name> On 07/01/2013 04:41 PM, Jack Little wrote: > In my concept, when the player buys a boost, their karma (a global) is multiplied by 25% when added to. How would I do this? > Making a changeable global is almost certainly a mistake. But to multiply by 25%, you simply divide by 4. karma /= 4 -- DaveA From steve at pearwood.info Mon Jul 1 22:49:38 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 02 Jul 2013 06:49:38 +1000 Subject: [Tutor] Boosts In-Reply-To: <1372711261.40117.YahooMailNeo@web124504.mail.ne1.yahoo.com> References: <1372711261.40117.YahooMailNeo@web124504.mail.ne1.yahoo.com> Message-ID: <51D1EB62.4090304@pearwood.info> On 02/07/13 06:41, Jack Little wrote: > In my concept, when the player buys a boost, their karma (a global) is multiplied by 25% when added to. How would I do this? karma = karma * 0.25 which can be written as: karma *= 0.25 If this is inside a function (as it should be!) you will need to indent the line, and include a declaration at the top of the function: global karma karma *= 0.25 However, are you sure you want to multiply their karma by 25%? That makes it smaller. Perhaps you want to INCREASE it by 25%, which would be calculated like this: global karma karma *= 1.25 or perhaps: karma = karma + 0.25*karma -- Steven From ramit.prasad at jpmorgan.com Mon Jul 1 22:58:38 2013 From: ramit.prasad at jpmorgan.com (Prasad, Ramit) Date: Mon, 1 Jul 2013 20:58:38 +0000 Subject: [Tutor] Boosts In-Reply-To: <51D1EB06.1010000@davea.name> References: <1372711261.40117.YahooMailNeo@web124504.mail.ne1.yahoo.com> <51D1EB06.1010000@davea.name> Message-ID: <5B80DD153D7D744689F57F4FB69AF47418514FB1@SCACMX008.exchad.jpmchase.net> Dave Angel wrote: > On 07/01/2013 04:41 PM, Jack Little wrote: > > In my concept, when the player buys a boost, their karma (a global) is multiplied by 25% when added > to. How would I do this? > > > > > Making a changeable global is almost certainly a mistake. > > But to multiply by 25%, you simply divide by 4. > > karma /= 4 > I get the feeling the OP might have meant "increased by 25%" in which case multiply by 1.25. karma*= 1.25 > > > -- > DaveA ~Ramit P This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email. From wayne at waynewerner.com Mon Jul 1 23:48:31 2013 From: wayne at waynewerner.com (Wayne Werner) Date: Mon, 1 Jul 2013 16:48:31 -0500 (CDT) Subject: [Tutor] Generate word list from specified letters In-Reply-To: References: Message-ID: On Sun, 30 Jun 2013, Scurvy Scott wrote: > Hello all. > > I'm trying to create a program right now that will pull a dictionary from urban dictionary, then use that > dictionary as the basis for generating words with specified letters. > > Maybe I'm not articulating that well enough.. > > I'm not at all sure where to start and any guidance would be helpful. > > I'm running Debian and Python 2.7 Are you trying to generate *real* words? Or just real sounding ones? You might want to look at this: https://github.com/xiongchiamiov/pronounceable-password-generator HTH, Wayne From jacklittlemc at yahoo.com Tue Jul 2 22:22:25 2013 From: jacklittlemc at yahoo.com (Jack Little) Date: Tue, 2 Jul 2013 13:22:25 -0700 (PDT) Subject: [Tutor] Error in Game Message-ID: <1372796545.40865.YahooMailNeo@web124505.mail.ne1.yahoo.com> The player has an option to upgrade or not. I know the code is correct, but it doesn't send the player to the shop. Here is the code: def lvl3_2(): ? ? print "You beat level 3!" ? ? print "Congratulations!" ? ? print "You have liberated the Bristol Channel!" ? ? print "[Y] to go to the shop or [N] to advance." ? ? final1=raw_input(">>") ? ? if final1.lower()=="y": ? ? ? ? shop2() ? ? elif final1.lower()=="n": ? ? ? ? lvl4() Help? -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan.gauld at btinternet.com Tue Jul 2 23:18:52 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Tue, 02 Jul 2013 22:18:52 +0100 Subject: [Tutor] Error in Game In-Reply-To: <1372796545.40865.YahooMailNeo@web124505.mail.ne1.yahoo.com> References: <1372796545.40865.YahooMailNeo@web124505.mail.ne1.yahoo.com> Message-ID: On 02/07/13 21:22, Jack Little wrote: > The player has an option to upgrade or not. I know the code is correct, > but it doesn't send the player to the shop. So what does it do? > def lvl3_2(): > print "You beat level 3!" > print "Congratulations!" > print "You have liberated the Bristol Channel!" > print "[Y] to go to the shop or [N] to advance." > final1=raw_input(">>") > if final1.lower()=="y": > shop2() > elif final1.lower()=="n": > lvl4() Have you tried putting some debug print statement in? For example .... if final1.lower()=="y": print 'going to shop2' shop2() elif final1.lower()=="n": print 'advancing' lvl4() And in the shop2() and lvl4() functions a print to show you got there? Just some ideas. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From joel.goldstick at gmail.com Tue Jul 2 23:20:06 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Tue, 2 Jul 2013 17:20:06 -0400 Subject: [Tutor] Error in Game In-Reply-To: <1372796545.40865.YahooMailNeo@web124505.mail.ne1.yahoo.com> References: <1372796545.40865.YahooMailNeo@web124505.mail.ne1.yahoo.com> Message-ID: On Tue, Jul 2, 2013 at 4:22 PM, Jack Little wrote: > The player has an option to upgrade or not. I know the code is correct, > but it doesn't send the player to the shop. Here is the code: > > def lvl3_2(): > print "You beat level 3!" > print "Congratulations!" > print "You have liberated the Bristol Channel!" > print "[Y] to go to the shop or [N] to advance." > final1=raw_input(">>") > if final1.lower()=="y": > shop2() > elif final1.lower()=="n": > lvl4() > > What does 'upgrade' have to do with this code? when you run this code, what is the traceback ? How do you know the code is correct? Isn't the definition of the code being correct, that it will perform the logic you intended? > > > Help? > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > > -- Joel Goldstick http://joelgoldstick.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From ramit.prasad at jpmorgan.com Tue Jul 2 23:22:41 2013 From: ramit.prasad at jpmorgan.com (Prasad, Ramit) Date: Tue, 2 Jul 2013 21:22:41 +0000 Subject: [Tutor] Error in Game In-Reply-To: <1372796545.40865.YahooMailNeo@web124505.mail.ne1.yahoo.com> References: <1372796545.40865.YahooMailNeo@web124505.mail.ne1.yahoo.com> Message-ID: <5B80DD153D7D744689F57F4FB69AF4741851C680@SCACMX008.exchad.jpmchase.net> Jack Little wrote: > The player has an option to upgrade or not. I know the code is correct, but it doesn't send the player > to the shop. Here is the code: > > def lvl3_2(): > ? ? print "You beat level 3!" > ? ? print "Congratulations!" > ? ? print "You have liberated the Bristol Channel!" > ? ? print "[Y] to go to the shop or [N] to advance." > ? ? final1=raw_input(">>") > ? ? if final1.lower()=="y": > ? ? ? ? shop2() > ? ? elif final1.lower()=="n": > ? ? ? ? lvl4() > > > > Help? Obviously the code must not be correct or it would send the player to the shop. :) What happens when you enter "y" or "n"? What do you expect to happen? Do you see any exceptions being printed? I wonder if some space, new line, or unprintable character is being captured unintentionally. Have you tried to `print repr(final1)`? ~Ramit Prasad This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email. From dfjennings at gmail.com Wed Jul 3 02:47:26 2013 From: dfjennings at gmail.com (Don Jennings) Date: Tue, 2 Jul 2013 20:47:26 -0400 Subject: [Tutor] Saving Progress In-Reply-To: <3FE557DB-0F34-4604-BE6A-82278EDFE031@yahoo.com> References: <3FE557DB-0F34-4604-BE6A-82278EDFE031@yahoo.com> Message-ID: <911E9354-EE6B-4F14-A20A-89DEF633AA41@gmail.com> On Jun 27, 2013, at 11:16 AM, Jack Little wrote: > Is there a way to save a players progress in a game using python without any modules Sure, right it to an open file [1]. Of course, you might just mean without using any *external* modules. In that case, you might consider the included database (import sqlite3) [2]. Take care, Don [1] http://docs.python.org/2/tutorial/inputoutput.html#reading-and-writing-files [2] http://docs.python.org/2/library/sqlite3.html From bgailer at gmail.com Wed Jul 3 15:33:20 2013 From: bgailer at gmail.com (bob gailer) Date: Wed, 03 Jul 2013 09:33:20 -0400 Subject: [Tutor] Error in Game In-Reply-To: <1372796545.40865.YahooMailNeo@web124505.mail.ne1.yahoo.com> References: <1372796545.40865.YahooMailNeo@web124505.mail.ne1.yahoo.com> Message-ID: <51D42820.4060505@gmail.com> On 7/2/2013 4:22 PM, Jack Little wrote: > I know the code is correct As Joel said- how could it be, since you do not get the desired results? When posting questions tell us: - what version of Python? - what operating system? - what you use to edit (write) your code - what you do to run your code copy and paste the execution > , but it doesn't send the player to the shop. Here is the code: > > def lvl3_2(): > print "You beat level 3!" > print "Congratulations!" > print "You have liberated the Bristol Channel!" > print "[Y] to go to the shop or [N] to advance." > final1=raw_input(">>") > if final1.lower()=="y": > shop2() > elif final1.lower()=="n": > lvl4() It is a good idea to add an else clause to handle the case where the user's entry does not match the if or elif tests. It is not a good idea to use recursion to navigate a game structure. It is better to have each function return to a main program, have the main program determine the next step and invoke it. > > > > Help? Since we are volunteers, the more you tell us the easier it is for us to do that. -- Bob Gailer 919-636-4239 Chapel Hill NC -------------- next part -------------- An HTML attachment was scrubbed... URL: From walksloud at gmail.com Wed Jul 3 20:17:53 2013 From: walksloud at gmail.com (Andre' Walker-Loud) Date: Wed, 3 Jul 2013 11:17:53 -0700 Subject: [Tutor] memory consumption Message-ID: <1682F923-F5EF-4553-8011-53B39CFD3B71@gmail.com> Hi All, I wrote some code that is running out of memory. It involves a set of three nested loops, manipulating a data file (array) of dimension ~ 300 x 256 x 1 x 2. It uses some third party software, but my guess is I am just not aware of how to use proper memory management and it is not the 3rd party software that is the culprit. Memory management is new to me, and so I am looking for some general guidance. I had assumed that reusing a variable name in a loop would automatically flush the memory by just overwriting it. But this is probably wrong. Below is a very generic version of what I am doing. I hope there is something obvious I am doing wrong or not doing which I can to dump the memory in each cycle of the innermost loop. Hopefully, what I have below is meaningful enough, but again, I am new to this, so we shall see. ################################################ # generic code skeleton # import a class I wrote to utilize the 3rd party software import my_class # instantiate the function do_stuff my_func = my_class.do_stuff() # I am manipulating a data array of size ~ 300 x 256 x 1 x 2 data = my_data # my_data is imported just once and has the size above # instantiate a 3d array of size 20 x 10 x 10 and fill it with all zeros my_array = numpy.zeros([20,10,10]) # loop over parameters and fill array with desired output for i in range(loop_1): for j in range(loop_2): for k in range(loop_3): # create tmp_data that has a shape which is the same as data except the first dimension can range from 1 - 1024 instead of being fixed at 300 ''' Is the next line where I am causing memory problems? ''' tmp_data = my_class.chop_data(data,i,j,k) my_func(tmp_data) my_func.third_party_function() my_array([i,j,k]) = my_func.results() # this is just a floating point number ''' should I do something to flush tmp_data? ''' ############################################# Thanks, Andre From steve at pearwood.info Wed Jul 3 20:55:40 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 04 Jul 2013 04:55:40 +1000 Subject: [Tutor] memory consumption In-Reply-To: <1682F923-F5EF-4553-8011-53B39CFD3B71@gmail.com> References: <1682F923-F5EF-4553-8011-53B39CFD3B71@gmail.com> Message-ID: <51D473AC.8000900@pearwood.info> On 04/07/13 04:17, Andre' Walker-Loud wrote: > Hi All, > > I wrote some code that is running out of memory. How do you know? What are the symptoms? Do you get an exception? Computer crashes? Something else? > It involves a set of three nested loops, manipulating a data file (array) of dimension ~ 300 x 256 x 1 x 2. Is it a data file, or an array? They're different things. > It uses some third party software, but my guess is I am just not aware of how to use proper memory management and it is not the 3rd party software that is the culprit. As a general rule, you shouldn't need to worry about such things, at least 99% of the time. > Memory management is new to me, and so I am looking for some general guidance. I had assumed that reusing a variable name in a loop would automatically flush the memory by just overwriting it. But this is probably wrong. Below is a very generic version of what I am doing. I hope there is something obvious I am doing wrong or not doing which I can to dump the memory in each cycle of the innermost loop. Hopefully, what I have below is meaningful enough, but again, I am new to this, so we shall see. Completely non-meaningful. > ################################################ > # generic code skeleton > # import a class I wrote to utilize the 3rd party software > import my_class Looking at the context here, "my_class" is a misleading name, since it's actually a module, not a class. > # instantiate the function do_stuff > my_func = my_class.do_stuff() This is getting confusing. Either you've oversimplified your pseudo-code, or you're using words in ways that do not agree with standard terminology. Or both. You don't instantiate functions, you instantiate a class, which gives you an instance (an object), not a function. So I'm lost here -- I have no idea what my_class is (possibly a module?), or do_stuff (possibly a class?) or my_func (possibly an instance?). > # I am manipulating a data array of size ~ 300 x 256 x 1 x 2 > data = my_data # my_data is imported just once and has the size above Where, and how, is my_data imported from? What is it? You say it is "a data array" (what sort of data array?) of size 300x256x1x2 -- that's a four-dimensional array, with 153600 entries. What sort of entries? Is that 153600 bytes (about 150K) or 153600 x 64-bit floats (about 1.3 MB)? Or 153600 data structures, each one holding 1MB of data (about 153 GB)? > # instantiate a 3d array of size 20 x 10 x 10 and fill it with all zeros > my_array = numpy.zeros([20,10,10]) At last, we finally see something concrete! A numpy array. Is this the same sort of array used above? > # loop over parameters and fill array with desired output > for i in range(loop_1): > for j in range(loop_2): > for k in range(loop_3): How big are loop_1, loop_2, loop_3? You should consider using xrange() rather than range(). If the number is very large, xrange will be more memory efficient. > # create tmp_data that has a shape which is the same as data except the first dimension can range from 1 - 1024 instead of being fixed at 300 > ''' Is the next line where I am causing memory problems? ''' > tmp_data = my_class.chop_data(data,i,j,k) How can we possibly tell if chop_data is causing memory problems when you don't show us what chop_data does? > my_func(tmp_data) > my_func.third_party_function() Again, no idea what they do. > my_array([i,j,k]) = my_func.results() # this is just a floating point number > ''' should I do something to flush tmp_data? ''' No. Python will automatically garbage collect is as needed. Well, that's not quite true. It depends on what my_tmp actually is. So, *probably* no. But without seeing the code for my_tmp, I cannot be sure. -- Steven From davea at davea.name Wed Jul 3 20:55:33 2013 From: davea at davea.name (Dave Angel) Date: Wed, 03 Jul 2013 14:55:33 -0400 Subject: [Tutor] memory consumption In-Reply-To: <1682F923-F5EF-4553-8011-53B39CFD3B71@gmail.com> References: <1682F923-F5EF-4553-8011-53B39CFD3B71@gmail.com> Message-ID: <51D473A5.3090802@davea.name> On 07/03/2013 02:17 PM, Andre' Walker-Loud wrote: > Hi All, > > I wrote some code that is running out of memory. And you know this how? What OS are you using, and specifically how is it telling you that you've run out of memory? And while you're at it, what version of Python? And are the OS and Python 32 and 32 bit, or 64 and 64, or mixed? > It involves a set of three nested loops, manipulating a data file (array) of dimension ~ 300 x 256 x 1 x 2. It uses some third party software, but my guess is I am just not aware of how to use proper memory management and it is not the 3rd party software that is the culprit. In particular you're using numpy, and I don't know its quirks. So I'll just speak of Python in general, and let someone else address numpy. > > Memory management is new to me, and so I am looking for some general guidance. I had assumed that reusing a variable name in a loop would automatically flush the memory by just overwriting it. It could be useful to learn how Python memory is manipulated. To start with, the 'variable' doesn't take a noticeable amount of space. It's the object its bound to that might take up lots of space, directly or indirectly. When you bind a new object to it, you free up the last one, unless something else is also bound to it. By indirectly, I refer to something like a list, which is one object, but which generally is bound to dozens or millions of others, and any of those may be bound to lots of others. Unbinding the list will usually free up all that stuff. The other thing that can happen is an object may indirectly be bound to itself. Trivial example: >>> mylist = [1,2] >>> mylist.append(mylist) >>> mylist [1, 2, [...]] >>> Fortunately for us, the repr() display of mylist doesn't descend infinitely into the guts of the elements, or it would be still printing next week (or until the printing logic ran out of memory). Anyway, once you have such a binding loop, the simple memory freeing logic (refcount) has to defer to the slower and less frequently run gc (garbage collection). > But this is probably wrong. Below is a very generic version of what I am doing. I hope there is something obvious I am doing wrong or not doing which I can to dump the memory in each cycle of the innermost loop. Hopefully, what I have below is meaningful enough, but again, I am new to this, so we shall see. > > ################################################ > # generic code skeleton > # import a class I wrote to utilize the 3rd party software > import my_class > > # instantiate the function do_stuff > my_func = my_class.do_stuff() So this is a class-static method which returns a callable object? One with methods of its own? > > # I am manipulating a data array of size ~ 300 x 256 x 1 x 2 > data = my_data # my_data is imported just once and has the size above > > # instantiate a 3d array of size 20 x 10 x 10 and fill it with all zeros > my_array = numpy.zeros([20,10,10]) > # loop over parameters and fill array with desired output > for i in range(loop_1): > for j in range(loop_2): > for k in range(loop_3): > # create tmp_data that has a shape which is the same as data except the first dimension can range from 1 - 1024 instead of being fixed at 300 > > ''' Is the next line where I am causing memory problems? ''' Hard to tell. is chop-data() a trivial function you could have posted? It's a class method, not an instance method. Is it keeping references to the data it's returning? Perhaps for caching purposes? > tmp_data = my_class.chop_data(data,i,j,k) > my_func(tmp_data) > my_func.third_party_function() > my_array([i,j,k]) = my_func.results() # this is just a floating point number > > ''' should I do something to flush tmp_data? ''' You don't show us any code that would cause me to suspect tmp_data. > ############################################# > You leave out so much that it's hard to know what parts to ask you to post. if data is a numpy array, and my_class.chop_data is a class method, perhaps you could post that class method. Do you have a tool for your OS that lets you examine memory usage dynamically? If you do, sometimes it's instructive to watch while a program is running to see what the dynamics are. Note that Python, like nearly any other program written with the C library, will not necessarily free memory all the way to the OS at any particular moment in time. If you (A C programmer) were to malloc() a megabyte block and immediately free it, you might not see the free externally, but new allocations would instead be carved out of that freed block. Those specifics vary with OS and with C compiler. And it may very well vary with size of block. Thus individual blocks over a certain size may be allocated directly from the OS, and freed immediately when done, while smaller blocks are coalesced in the library and reused over and over. -- DaveA From alan.gauld at btinternet.com Wed Jul 3 20:57:31 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 03 Jul 2013 19:57:31 +0100 Subject: [Tutor] memory consumption In-Reply-To: <1682F923-F5EF-4553-8011-53B39CFD3B71@gmail.com> References: <1682F923-F5EF-4553-8011-53B39CFD3B71@gmail.com> Message-ID: On 03/07/13 19:17, Andre' Walker-Loud wrote: Your terminology is all kixed up and therefore does not make sense. WE definitely need to know more about the my_class module and do_stuff.... > ################################################ > # generic code skeleton > # import a class I wrote to utilize the 3rd party software > import my_class This import a module which may contain some code that you wrote but... > # instantiate the function do_stuff > my_func = my_class.do_stuff() You don;t instantiate functions you call them. You are setting my_func to be the return value of do_stuff(). What is that return value? What does my_func actually refer to? > my_array = numpy.zeros([20,10,10]) > # loop over parameters and fill array with desired output > for i in range(loop_1): > for j in range(loop_2): > for k in range(loop_3): > # create tmp_data that has a shape which is the same as data except the first dimension can range from 1 - 1024 instead of being fixed at 300 > > ''' Is the next line where I am causing memory problems? ''' > tmp_data = my_class.chop_data(data,i,j,k) Again we must guess what the chop_data function is returning. Some sample data would be useful here. > my_func(tmp_data) Here you call a function but do not store any return values. Or are you using global variables somewhere? > my_func.third_party_function() But now you are accessing an attribute of my_func. What is my_func? Is it a function or an object? We cannot begin to guess what is going on without knowing that. > my_array([i,j,k]) = my_func.results() # this is just a floating point number > > ''' should I do something to flush tmp_data? ''' No idea, you haven't begun to give us enough information. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From walksloud at gmail.com Wed Jul 3 21:50:38 2013 From: walksloud at gmail.com (Andre' Walker-Loud) Date: Wed, 3 Jul 2013 12:50:38 -0700 Subject: [Tutor] memory consumption In-Reply-To: <51D473AC.8000900@pearwood.info> References: <1682F923-F5EF-4553-8011-53B39CFD3B71@gmail.com> <51D473AC.8000900@pearwood.info> Message-ID: <54AEACBC-4BB2-45BF-856F-66ACAE54C8F8@gmail.com> Hi Steven, Dave, Allen and All, OK - forgive my poor terminology. That is also something I am learning. The three of you asked very similar questions - so I will just reply this once. >> I wrote some code that is running out of memory. > > How do you know? What are the symptoms? Do you get an exception? Computer crashes? Something else? I am using mac OS, and have the "Activity Monitor" open, and can watch the memory slowly get consumed. Then, a crash when it runs out, about 2/3 of the way through Python(50827,0xa051c540) malloc: *** mmap(size=16777216) failed (error code=12) *** error: can't allocate region *** set a breakpoint in malloc_error_break to debug Bus error I have 8GB of RAM, and the underlying data file I start with is small (see below). >> It involves a set of three nested loops, manipulating a data file (array) of dimension ~ 300 x 256 x 1 x 2. > > Is it a data file, or an array? They're different things. OK. I begin with a data file (hdf5 to be specific). I load a piece of the data file into memory as a numpy array. The size of the array is 300 x 256 x 1 x 2 and each element is a double precision float, so the total array is ~1.3MB My code is not in the boiled-down-to-simplest-failing-state, which is why I attempted to just describe. But I will get more specific below. >> ################################################ >> # generic code skeleton >> # import a class I wrote to utilize the 3rd party software >> import my_class > > Looking at the context here, "my_class" is a misleading name, since it's actually a module, not a class. Yes. In "my_class" I wrote a class. I think the essentials are ########################## # my_class # perhaps a poor name, but I will stick with it for the re-post ''' The third_party software performs numerical minimization. It has a function that constructs the function to be minimized, and then a routing to actually do the minimization. It minimizes a chi^2 with respect to "other_vars" ''' import third_party class do_stuff: # I am aware this doesn't follow the class naming convention, just sticking with my previous post name def __call__(data,other_vars): self.fit = third_party.function_set_up(data,other_vars) def minimize(self): try: self.fit.minimize() self.have_fit = True except third_party.Error: self.have_fit = False ########################## I should say the above construction works as I expect when I do a single fit. > # instantiate the function do_stuff >> my_func = my_class.do_stuff() [snip] >> # I am manipulating a data array of size ~ 300 x 256 x 1 x 2 >> data = my_data # my_data is imported just once and has the size above > > Where, and how, is my_data imported from? What is it? You say it is "a data array" (what sort of data array?) of size 300x256x1x2 -- that's a four-dimensional array, with 153600 entries. What sort of entries? Is that 153600 bytes (about 150K) or 153600 x 64-bit floats (about 1.3 MB)? Or 153600 data structures, each one holding 1MB of data (about 153 GB)? see above - each element is a double precision (64) float, so 1.3 MB. >> # instantiate a 3d array of size 20 x 10 x 10 and fill it with all zeros >> my_array = numpy.zeros([20,10,10]) > > At last, we finally see something concrete! A numpy array. Is this the same sort of array used above? > > >> # loop over parameters and fill array with desired output >> for i in range(loop_1): >> for j in range(loop_2): >> for k in range(loop_3): > > How big are loop_1, loop_2, loop_3? The sizes of the loops are not big len(loop_1) = 20 len(loop_2) = 10 len(loop_3) = 10 > You should consider using xrange() rather than range(). If the number is very large, xrange will be more memory efficient. > > >> # create tmp_data that has a shape which is the same as data except the first dimension can range from 1 - 1024 instead of being fixed at 300 >> ''' Is the next line where I am causing memory problems? ''' >> tmp_data = my_class.chop_data(data,i,j,k) > > How can we possibly tell if chop_data is causing memory problems when you don't show us what chop_data does? OK, below is chop data ################### import numpy as np def chop_data(data,N,M): n_orig = data.shape[0] # data is a numpy array, data.shape returns the size of each dimension data_new = [] for b in range(N): # draw M random samples with repeat from original data and take the mean # np.random.randint(0,n_orig,M) creates a list ranging from 0 to n_orig-1 of size M data_new.append(data[np.random.randint(0,n_orig,M)].mean(axis=0)) data_new = np.array(data_new) # convert to a numpy array return data_new ################### To be clear, I will re-write the loop I had in the original post ################### import my_class import numpy my_func = my_class.do_stuff() other_vars = np.array([''' list of guess initial values for numerical minimization ''']) loop_lst = np.array([1,2,4,8,16,32,64,128,256,512,1024]) n_sample = 20 my_array = numpy.zeros([n_sample,len(loop_lst),len(loop_lst)]) for s in range(n_sample): for n_i,n in enumerate(loop_lst): for m_i,m in enumerate(loop_lst): tmp_data = my_class.chop_data(data,n,m) my_func(tmp_data,other_vars) my_func.minimize() my_array[s,n_i,m_i] = my_func.fit.results() # returns a 64-float, results is a built in third_party function This all runs as expected, except each pass through the loops, the code consumes more and more memory. Two-thirds through the outer loop, it crashes with the above mentioned memory error. Hopefully, this is more clear. Andre From oscar.j.benjamin at gmail.com Wed Jul 3 21:57:29 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 3 Jul 2013 20:57:29 +0100 Subject: [Tutor] memory consumption In-Reply-To: <1682F923-F5EF-4553-8011-53B39CFD3B71@gmail.com> References: <1682F923-F5EF-4553-8011-53B39CFD3B71@gmail.com> Message-ID: On 3 July 2013 19:17, Andre' Walker-Loud wrote: > Hi All, > > I wrote some code that is running out of memory. It involves a set of three nested loops, manipulating a data file (array) of dimension ~ 300 x 256 x 1 x 2. It uses some third party software, but my guess is I am just not aware of how to use proper memory management and it is not the 3rd party software that is the culprit. You mention third-party software but you need to say which software. I use numpy extensively so if that's what you mean I can help. > Memory management is new to me, and so I am looking for some general guidance. I had assumed that reusing a variable name in a loop would automatically flush the memory by just overwriting it. But this is probably wrong. It's not wrong unless you happen to be storing a reference to the variable somewhere else. > Below is a very generic version of what I am doing. I hope there is something obvious I am doing wrong or not doing which I can to dump the memory in each cycle of the innermost loop. Hopefully, what I have below is meaningful enough, but again, I am new to this, so we shall see. This is too generic and also incomplete. Ideally you should try to post an example that someone else can actually run. The best way is to progressively simplify your program while still verifying that the confusing/incorrect behaviour occurs. You can find some handy tips here: http://sscce.org/ > > ################################################ > # generic code skeleton > # import a class I wrote to utilize the 3rd party software > import my_class > > # instantiate the function do_stuff > my_func = my_class.do_stuff() > > # I am manipulating a data array of size ~ 300 x 256 x 1 x 2 > data = my_data # my_data is imported just once and has the size above > > # instantiate a 3d array of size 20 x 10 x 10 and fill it with all zeros > my_array = numpy.zeros([20,10,10]) > # loop over parameters and fill array with desired output > for i in range(loop_1): > for j in range(loop_2): > for k in range(loop_3): If the above is supposed to just say e.g. 'for i in range(20)' then this is not a good simplification. By the way numpy has a better solution for this kind of nested loop problem: >>> import numpy >>> a = numpy.zeros([2, 3, 4], float) # Always specify the type of the array >>> a array([[[ 0., 0., 0., 0.], [ 0., 0., 0., 0.], [ 0., 0., 0., 0.]], [[ 0., 0., 0., 0.], [ 0., 0., 0., 0.], [ 0., 0., 0., 0.]]]) >>> a.shape (2, 3, 4) >>> for i, j, k in numpy.ndindex(*a.shape): ... print(i, j, k) ... 0 0 0 0 0 1 0 0 2 0 0 3 0 1 0 0 1 1 0 1 2 ... and so on ... > # create tmp_data that has a shape which is the same as data except the first dimension can range from 1 - 1024 instead of being fixed at 300 > > ''' Is the next line where I am causing memory problems? ''' > tmp_data = my_class.chop_data(data,i,j,k) I can't say whether any of the lines in the loop are or are not causing memory problems as I don't know what any of them does. > my_func(tmp_data) Given that you're not using the return from my_func I assume that it changes the value of something somewhere else. Are you storing a reference to tmp_data somewhere in that function? > my_func.third_party_function() When making your code generic you should try to use a function name that gives some indication of what it does in so far as it is relevant; if it's not relevant then just remove it from your example. > my_array([i,j,k]) = my_func.results() # this is just a floating point number This would be the only line that I could definitely understand. Unfortunately though it's a syntax error e.g.: >>> f([1,2,3]) = g() File "", line 1 SyntaxError: can't assign to function call The correct way to assign to an element of a numpy array (assuming that's what you meant) is my_array[i, j, k] = my_func.results() Oscar From bjames at Jamesgang.dyndns.org Wed Jul 3 21:51:57 2013 From: bjames at Jamesgang.dyndns.org (bjames at Jamesgang.dyndns.org) Date: Wed, 3 Jul 2013 14:51:57 -0500 Subject: [Tutor] Cleaning up output Message-ID: <74d4d46f5221e21ff2547915a3c7e201.squirrel@jamesgang.dyndns.org> I've written my first program to take a given directory and look in all directories below it for duplicate files (duplicate being defined as having the same MD5 hash, which I know isn't a perfect solution, but for what I'm doing is good enough) My problem now is that my output file is a rather confusing jumble of paths and I'm not sure the best way to make it more user readable. My gut reaction would be to go through and list by first directory, but is there a logical way to do it so that all the groupings that have files in the same two directories would be grouped together? So I'm thinking I'd have: First File Dir /some/directory/ Duplicate directories: some/other/directory/ Original file 1 , dupicate file 1 Original file 2, duplicate file 2 some/third directory/ original file 3, duplicate file 3 and so forth, where the Original file would be the file name in the First files so that all the ones are the same there. I fear I'm not explaining this well but I'm hoping someone can either ask questions to help get out of my head what I'm trying to do or can decipher this enough to help me. Here's a git repo of my code if it helps: https://github.com/CyberCowboy/FindDuplicates From eryksun at gmail.com Wed Jul 3 22:07:01 2013 From: eryksun at gmail.com (eryksun) Date: Wed, 3 Jul 2013 16:07:01 -0400 Subject: [Tutor] memory consumption In-Reply-To: <1682F923-F5EF-4553-8011-53B39CFD3B71@gmail.com> References: <1682F923-F5EF-4553-8011-53B39CFD3B71@gmail.com> Message-ID: On Wed, Jul 3, 2013 at 2:17 PM, Andre' Walker-Loud wrote: > > I had assumed that reusing a variable name in a loop would automatically > flush the memory by just overwriting it. But this is probably wrong. CPython uses reference counting and a garbage collector that breaks cycles (see gc.collect). You can use sys.getrefcount() to check the count (subtract 1 for the call). Also, an extension type may allocate memory for caches and such (e.g. numpy's fft), and you may not have direct control over this. Try a memory profiler: https://pypi.python.org/pypi/memory_profiler From oscar.j.benjamin at gmail.com Wed Jul 3 22:25:10 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 3 Jul 2013 21:25:10 +0100 Subject: [Tutor] memory consumption In-Reply-To: <54AEACBC-4BB2-45BF-856F-66ACAE54C8F8@gmail.com> References: <1682F923-F5EF-4553-8011-53B39CFD3B71@gmail.com> <51D473AC.8000900@pearwood.info> <54AEACBC-4BB2-45BF-856F-66ACAE54C8F8@gmail.com> Message-ID: On 3 July 2013 20:50, Andre' Walker-Loud wrote: > Hi Steven, Dave, Allen and All, > > OK - forgive my poor terminology. That is also something I am learning. > The three of you asked very similar questions - so I will just reply this once. > > >>> I wrote some code that is running out of memory. >> >> How do you know? What are the symptoms? Do you get an exception? Computer crashes? Something else? > > I am using mac OS, and have the "Activity Monitor" open, and can watch the memory slowly get consumed. Then, a crash when it runs out, about 2/3 of the way through > > Python(50827,0xa051c540) malloc: *** mmap(size=16777216) failed (error code=12) > *** error: can't allocate region > *** set a breakpoint in malloc_error_break to debug > Bus error The error is for creating an mmap object. This is not something that numpy does unless you tell it to. > I have 8GB of RAM, and the underlying data file I start with is small (see below). > >>> It involves a set of three nested loops, manipulating a data file (array) of dimension ~ 300 x 256 x 1 x 2. >> >> Is it a data file, or an array? They're different things. > > OK. I begin with a data file (hdf5 to be specific). So you're using pytables or h5py, or something else? It really would help if you would specify this instead of trying to be generic. My guess is that the hdf5 library loads the array as an mmap'ed memory block and you're not actually working with an ordinary numpy array (even if it has a similar interface). > I load a piece of the data file into memory as a numpy array. The size of the array is > > 300 x 256 x 1 x 2 > > and each element is a double precision float, so the total array is ~1.3MB Have you checked the actual memory size of the array? If it's a real numpy array you can use the nbytes attribute: >>> a = numpy.zeros([300, 256, 1, 2], float) >>> a.nbytes 1228800 > > My code is not in the boiled-down-to-simplest-failing-state, which is why I attempted to just describe. I think that you should get into the boiled down state. Your attempt to summarise misses too much relevant information. > But I will get more specific below. > >>> ################################################ >>> # generic code skeleton >>> # import a class I wrote to utilize the 3rd party software >>> import my_class >> >> Looking at the context here, "my_class" is a misleading name, since it's actually a module, not a class. > > Yes. In "my_class" I wrote a class. I think the essentials are > > ########################## > # my_class # perhaps a poor name, but I will stick with it for the re-post > ''' > The third_party software performs numerical minimization. It has a function that constructs the function to be minimized, and then a routing to actually do the minimization. It minimizes a chi^2 with respect to "other_vars" > ''' > import third_party > > class do_stuff: > # I am aware this doesn't follow the class naming convention, just sticking with my previous post name > def __call__(data,other_vars): > self.fit = third_party.function_set_up(data,other_vars) > > def minimize(self): > try: > self.fit.minimize() > self.have_fit = True > except third_party.Error: > self.have_fit = False > ########################## If you write code like the above then you really cannot expect other people to just understand what you mean if you don't show them the code. Specifically the use of __call__ is confusing. Really, though, this class is just a distraction from your problem and should have been simplified away. > > I should say the above construction works as I expect when I do a single fit. > >> # instantiate the function do_stuff >>> my_func = my_class.do_stuff() > [snip] >>> # I am manipulating a data array of size ~ 300 x 256 x 1 x 2 >>> data = my_data # my_data is imported just once and has the size above >> >> Where, and how, is my_data imported from? What is it? You say it is "a data array" (what sort of data array?) of size 300x256x1x2 -- that's a four-dimensional array, with 153600 entries. What sort of entries? Is that 153600 bytes (about 150K) or 153600 x 64-bit floats (about 1.3 MB)? Or 153600 data structures, each one holding 1MB of data (about 153 GB)? > > see above - each element is a double precision (64) float, so 1.3 MB. > >>> # instantiate a 3d array of size 20 x 10 x 10 and fill it with all zeros >>> my_array = numpy.zeros([20,10,10]) >> >> At last, we finally see something concrete! A numpy array. Is this the same sort of array used above? >> >> >>> # loop over parameters and fill array with desired output >>> for i in range(loop_1): >>> for j in range(loop_2): >>> for k in range(loop_3): >> >> How big are loop_1, loop_2, loop_3? > > The sizes of the loops are not big > len(loop_1) = 20 > len(loop_2) = 10 > len(loop_3) = 10 > >> You should consider using xrange() rather than range(). If the number is very large, xrange will be more memory efficient. Or numpy.nditer in this case. >> >> >>> # create tmp_data that has a shape which is the same as data except the first dimension can range from 1 - 1024 instead of being fixed at 300 >>> ''' Is the next line where I am causing memory problems? ''' >>> tmp_data = my_class.chop_data(data,i,j,k) >> >> How can we possibly tell if chop_data is causing memory problems when you don't show us what chop_data does? > > OK, below is chop data > > ################### > import numpy as np > def chop_data(data,N,M): > n_orig = data.shape[0] # data is a numpy array, data.shape returns the size of each dimension > data_new = [] > for b in range(N): > # draw M random samples with repeat from original data and take the mean > # np.random.randint(0,n_orig,M) creates a list ranging from 0 to n_orig-1 of size M > data_new.append(data[np.random.randint(0,n_orig,M)].mean(axis=0)) > data_new = np.array(data_new) # convert to a numpy array The above function can be vectorised to something like: def chop_data(data, N, M): return data[np.random.randint(0, data.shape[0], (M, N))].mean(axis=0) > return data_new > ################### > > To be clear, I will re-write the loop I had in the original post > > ################### > import my_class > import numpy > my_func = my_class.do_stuff() > other_vars = np.array([''' list of guess initial values for numerical minimization ''']) > loop_lst = np.array([1,2,4,8,16,32,64,128,256,512,1024]) > n_sample = 20 > my_array = numpy.zeros([n_sample,len(loop_lst),len(loop_lst)]) > for s in range(n_sample): > for n_i,n in enumerate(loop_lst): > for m_i,m in enumerate(loop_lst): > tmp_data = my_class.chop_data(data,n,m) Where did data come from? Is that the mmap'ed array from the hdf5 library? > my_func(tmp_data,other_vars) > my_func.minimize() I now know that the above two lines call thirdparty.function_setup() and my_func.fit.minimize(). I still have no idea what they do though. > my_array[s,n_i,m_i] = my_func.fit.results() # returns a 64-float, results is a built in third_party function > This all runs as expected, except each pass through the loops, the code consumes more and more memory. Two-thirds through the outer loop, it crashes with the above mentioned memory error. > > > Hopefully, this is more clear. Only slightly. The details that you choose to include are not the ones that are needed to understand your problem. Instead of paraphrasing simplify this into a short but *complete* example and post that. Oscar From ramit.prasad at jpmorgan.com Wed Jul 3 22:30:58 2013 From: ramit.prasad at jpmorgan.com (Prasad, Ramit) Date: Wed, 3 Jul 2013 20:30:58 +0000 Subject: [Tutor] Cleaning up output In-Reply-To: <74d4d46f5221e21ff2547915a3c7e201.squirrel@jamesgang.dyndns.org> References: <74d4d46f5221e21ff2547915a3c7e201.squirrel@jamesgang.dyndns.org> Message-ID: <5B80DD153D7D744689F57F4FB69AF474185234B6@SCACMX008.exchad.jpmchase.net> bjames at Jamesgang.dyndns.org wrote: > I've written my first program to take a given directory and look in all > directories below it for duplicate files (duplicate being defined as > having the same MD5 hash, which I know isn't a perfect solution, but for > what I'm doing is good enough) > > My problem now is that my output file is a rather confusing jumble of > paths and I'm not sure the best way to make it more user readable. My gut > reaction would be to go through and list by first directory, but is there > a logical way to do it so that all the groupings that have files in the > same two directories would be grouped together? > > So I'm thinking I'd have: > First File Dir /some/directory/ > Duplicate directories: > some/other/directory/ > Original file 1 , dupicate file 1 > Original file 2, duplicate file 2 > some/third directory/ > original file 3, duplicate file 3 > > and so forth, where the Original file would be the file name in the First > files so that all the ones are the same there. > > I fear I'm not explaining this well but I'm hoping someone can either ask > questions to help get out of my head what I'm trying to do or can decipher > this enough to help me. > > Here's a git repo of my code if it helps: > https://github.com/CyberCowboy/FindDuplicates > Your file was not too big to just paste in the email. (Pasted below) import os, hashlib hashdict = {} # content signature -> list of filenames dups = [] def dupe(rootdir): """goes through directory tree, compares md5 hash of all files, combines files with same hash value into list in hashmap directory""" for path, dirs, files in os.walk(unicode(rootdir)): #this section goes through the given directory, and all subdirectories/files below #as part of a loop reading them in for filename in files: #steps through each file and starts the process of getting the MD5 hashes for the file #comparing that hash to known hashes that have already been calculated and either merges it #with the known hash (which indicates duplicates) or adds it so that it can be compared to future #files fullname = os.path.join(path, filename) with open(fullname) as f: #does the actual hashing md5 = hashlib.md5() while True: d = f.read(4096) if not d: break md5.update(d) h = md5.hexdigest() filelist = hashdict.setdefault(h, []) filelist.append(fullname) for currenthash in hashdict.itervalues(): #goes through and if has has more than one file listed with it #considers it a duplicate and adds it to the output list if len(currenthash) > 1: dups.append(currenthash) output = open('duplicates.txt','w') for x in dups: output.write(str(x)) output.write("\n") output.close() Why do you only read 4096 bytes at a time? Guarding against large files? I would probably sort on original file and print "original: dup1 dup2". That way it should automatically put original files together by directory. for currenthash in hashdict.itervalues(): #goes through and if has has more than one file listed with it #considers it a duplicate and adds it to the output list if len(currenthash) > 1: dups.append(currenthash) dups.sort() # Should sort by original file then first duplicate, etc # if you want to only sort by original file use # dups.sort(key=lambda x: x[0]) with open('duplicates.txt','w') as output: for x in dups: output.write('Original {0} | {1}\n'.format(x[0], ' '.join(x[1:]))) ~Ramit This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email. From oscar.j.benjamin at gmail.com Wed Jul 3 22:57:55 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 3 Jul 2013 21:57:55 +0100 Subject: [Tutor] memory consumption In-Reply-To: References: <1682F923-F5EF-4553-8011-53B39CFD3B71@gmail.com> <51D473AC.8000900@pearwood.info> <54AEACBC-4BB2-45BF-856F-66ACAE54C8F8@gmail.com> Message-ID: On 3 July 2013 21:25, Oscar Benjamin wrote: > On 3 July 2013 20:50, Andre' Walker-Loud wrote: >> >> My code is not in the boiled-down-to-simplest-failing-state, which is why I attempted to just describe. > > I think that you should get into the boiled down state. On further reflection I have decided to retract this last statement. Oscar From davea at davea.name Wed Jul 3 23:10:52 2013 From: davea at davea.name (Dave Angel) Date: Wed, 03 Jul 2013 17:10:52 -0400 Subject: [Tutor] Cleaning up output In-Reply-To: <74d4d46f5221e21ff2547915a3c7e201.squirrel@jamesgang.dyndns.org> References: <74d4d46f5221e21ff2547915a3c7e201.squirrel@jamesgang.dyndns.org> Message-ID: <51D4935C.8050002@davea.name> On 07/03/2013 03:51 PM, bjames at Jamesgang.dyndns.org wrote: > I've written my first program to take a given directory and look in all > directories below it for duplicate files (duplicate being defined as > having the same MD5 hash, which I know isn't a perfect solution, but for > what I'm doing is good enough) This is a great first project for learning Python. It's a utility which doesn't write any data to the disk (other than the result file), and therefore bugs won't cause much havoc. Trust me, you will have bugs, we all do. One of the things experience teaches you is how to isolate the damage that bugs do before they're discovered. > > My problem now is that my output file is a rather confusing jumble of > paths and I'm not sure the best way to make it more user readable. My gut > reaction would be to go through and list by first directory, but is there > a logical way to do it so that all the groupings that have files in the > same two directories would be grouped together? I've come up with the same "presentation problem" with my own similar utilities. Be assured, there's no one "right answer." First question is have you considered what you want when there are MORE than two copies of one of those files? When you know what you'd like to see if there are four identical files, you might have a better idea what you should do even for two. Additionally, consider that two identical files may be in the same directory, with different names. Anyway, if you can explain why you want a particular grouping, we might better understand how to accomplish it. > > So I'm thinking I'd have: > First File Dir /some/directory/ > Duplicate directories: > some/other/directory/ > Original file 1 , dupicate file 1 > Original file 2, duplicate file 2 > some/third directory/ > original file 3, duplicate file 3 At present, this First File Dir could be any of the directories involved; without some effort, os.walk doesn't promise you any order of processing. But if you want them to appear in sorted order, you can do sorts at key points inside your os.walk code, and they'll at least come out in an order that's recognizable. (Some OS's may also sort things they feed to os.walk, but you'd do better not to count on it) You also could sort each list in itervalues of hashdict, after the dict is fully populated. Even with sorting, you run into the problem that there may be duplicates between some/other/directory and some/third/directory that are not in /some/directory. So in the sample you show above, they won't be listed with the ones that are in /some/directory. > > and so forth, where the Original file would be the file name in the First > files so that all the ones are the same there. > > I fear I'm not explaining this well but I'm hoping someone can either ask > questions to help get out of my head what I'm trying to do or can decipher > this enough to help me. > > Here's a git repo of my code if it helps: > https://github.com/CyberCowboy/FindDuplicates > At 40 lines, you should have just included it. It's usually much better to include the code inline if you want any comments on it. Think of what the archives are going to show in a year, when you're removed that repo, or thoroughly updated it. Somebody at that time will not be able to make sense of comments directed at the current version of the code. BTW, thanks for posting as text, since that'll mean that when you do post code, it shouldn't get mangled. So I'll comment on the code. You never call the dupe() function, so presumably this is a module intended to be used from some place else. But if that's the case, I would have expected it to be factored better, at least to separate the input processing from the output file formatting. That way you could re-use the dups logic and provide a new save formatting without duplicating anything. The first function could return the hashdict, and the second one could analyze it to produce a particular formatted output. The hashdict and dups variables should be initialized within the function, since they are not going to be used outside. Avoid non-const globals. And of course once you factor it, dups will be in the second function only. You do have a if __name__ == "__main__": line, but it's inside the function. Probably you meant it to be at the left margin. And importing inside a conditional is seldom a good idea, though it doesn't matter here since you're not using the import. Normally you want all your imports at the top, so they're easy to spot. You also probably want a call to dupe() inside the conditional. And perhaps some parsing of argv to get rootdir. You don't mention your OS, but many OS's have symbolic links or the equivalent. There's no code here to handle that possibility. Symlinks are a pain to do right. You could just add it in your docs, that no symlink is allowed under the rootdir. Your open() call has no mode switch. If you want the md5 to be 'correct", it has to be opened in binary. So you want something like: with open(fullname, "rb") as f: It doesn't matter this time, since you never export the hashes. But if you ever needed to debug it, it'd be nice if the hashes matched the standard values provided by md5sum. Besides, if you're on Windows, treating a binary file as though it were text could increase the likelihood of getting md5 collisions. -- DaveA From alan.gauld at btinternet.com Thu Jul 4 00:03:37 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 03 Jul 2013 23:03:37 +0100 Subject: [Tutor] memory consumption In-Reply-To: <54AEACBC-4BB2-45BF-856F-66ACAE54C8F8@gmail.com> References: <1682F923-F5EF-4553-8011-53B39CFD3B71@gmail.com> <51D473AC.8000900@pearwood.info> <54AEACBC-4BB2-45BF-856F-66ACAE54C8F8@gmail.com> Message-ID: On 03/07/13 20:50, Andre' Walker-Loud wrote: >>> # loop over parameters and fill array with desired output >>> for i in range(loop_1): >>> for j in range(loop_2): >>> for k in range(loop_3): >> >> How big are loop_1, loop_2, loop_3? > > The sizes of the loops are not big > len(loop_1) = 20 > len(loop_2) = 10 > len(loop_3) = 10 This is confusing. The fact that you are getting values for the len() of these variables suggests they are some kind of collection? But you are using them in range which expects number(s) What kind of things are loop_1 etc? What happens at the >>> prompt if you try to >>> print range(loop_1) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From walksloud at gmail.com Thu Jul 4 00:11:03 2013 From: walksloud at gmail.com (Andre' Walker-Loud) Date: Wed, 3 Jul 2013 15:11:03 -0700 Subject: [Tutor] memory consumption In-Reply-To: References: <1682F923-F5EF-4553-8011-53B39CFD3B71@gmail.com> <51D473AC.8000900@pearwood.info> <54AEACBC-4BB2-45BF-856F-66ACAE54C8F8@gmail.com> Message-ID: <8D91EAD6-5B66-433B-9FA6-B3D1F5D01CA0@gmail.com> Hi Alan, >>>> # loop over parameters and fill array with desired output >>>> for i in range(loop_1): >>>> for j in range(loop_2): >>>> for k in range(loop_3): >>> >>> How big are loop_1, loop_2, loop_3? >> >> The sizes of the loops are not big >> len(loop_1) = 20 >> len(loop_2) = 10 >> len(loop_3) = 10 > > This is confusing. > The fact that you are getting values for the len() of these > variables suggests they are some kind of collection? But you > are using them in range which expects number(s) Yes, I was being sloppy. My later post clarified what I meant. The loops are really lists, and I was really using enumerate() to get both the iter and the element. loop_2 = [1,2,4,8,16,32,64,128,256,512,1024] for i,n in enumerate(loop_2): ... Thanks, Andre From walksloud at gmail.com Thu Jul 4 00:37:24 2013 From: walksloud at gmail.com (Andre' Walker-Loud) Date: Wed, 3 Jul 2013 15:37:24 -0700 Subject: [Tutor] memory consumption In-Reply-To: References: <1682F923-F5EF-4553-8011-53B39CFD3B71@gmail.com> <51D473AC.8000900@pearwood.info> <54AEACBC-4BB2-45BF-856F-66ACAE54C8F8@gmail.com> Message-ID: <6C829390-CBAC-46C0-80EE-C49B8E69AF13@gmail.com> Hi Oscar, [snip] > The error is for creating an mmap object. This is not something that > numpy does unless you tell it to. I have not explicitly told it to. [snip] > So you're using pytables or h5py, or something else? It really would > help if you would specify this instead of trying to be generic. All the third party software I am using: numpy pytables hdf5 pyminuit [python interface to Minuit] Minuit [Minuit - a c++ code based developed in CERN for numerical minimization] I was hoping I was must making some obvious mistake, assuming people are mostly not familiar with much of any of the specific code I am using. But I guess it is either not so obvious, or my explanations are too poor. > My guess is that the hdf5 library loads the array as an mmap'ed memory > block and you're not actually working with an ordinary numpy array > (even if it has a similar interface). I specifically load the data once, as my_file = tables.openFile(my_data_file.h5) my_data = my_file.getNode(path_to_data).read() after this, "my_data" seems to have all the features of a numpy array. for example, > Have you checked the actual memory size of the array? If it's a real > numpy array you can use the nbytes attribute: >>>> a = numpy.zeros([300, 256, 1, 2], float) >>>> a.nbytes > 1228800 this works on my_data. [snip] >> class do_stuff: >> # I am aware this doesn't follow the class naming convention, just sticking with my previous post name >> def __call__(data,other_vars): >> self.fit = third_party.function_set_up(data,other_vars) >> >> def minimize(self): >> try: >> self.fit.minimize() >> self.have_fit = True >> except third_party.Error: >> self.have_fit = False >> ########################## > > If you write code like the above then you really cannot expect other > people to just understand what you mean if you don't show them the > code. Specifically the use of __call__ is confusing. Really, though, > this class is just a distraction from your problem and should have > been simplified away. Off my main topic, but could you explain more? I am also not very experienced writing classes, and was learning from an example. So I am not sure why __call__ is confusing. I thought that was correct. [snip] > The above function can be vectorised to something like: > > def chop_data(data, N, M): > return data[np.random.randint(0, data.shape[0], (M, N))].mean(axis=0) Thanks. [snip] >> tmp_data = my_class.chop_data(data,n,m) > > Where did data come from? Is that the mmap'ed array from the hdf5 library? as above, data is loaded once with pytables, using the .read() function. Subsequently, with my "chop_data" function, I believe that returns a numpy array, and I have not explicitly asked it anything about "mmap" so I am not sure. How would you check? >> my_func(tmp_data,other_vars) >> my_func.minimize() > > I now know that the above two lines call thirdparty.function_setup() > and my_func.fit.minimize(). I still have no idea what they do though. I was trying to avoid that, since I suspect(ed) the problem is with me and not the third party. Without going into specifics, the first function constructs a chi^2 function which looks like chisq = sum_i ( (data[i] - fit_func(i,fit_params)) / data_error[i] )**2 the second function works to numerically minimize chisq with respect to the "fit_params" which are a 1d array. [snip] >> Hopefully, this is more clear. > > Only slightly. The details that you choose to include are not the ones > that are needed to understand your problem. Instead of paraphrasing > simplify this into a short but *complete* example and post that. If you can help me understand the issue of mmap (whether somehow I am creating this unwittingly), that would be great. ie, what tests can I perform to check? Otherwise, it seems perhaps the best thing for me to do now is take eryksun's advice and learn how to use a memory profiler. Thanks, Andre From oscar.j.benjamin at gmail.com Thu Jul 4 01:24:18 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Thu, 4 Jul 2013 00:24:18 +0100 Subject: [Tutor] memory consumption In-Reply-To: <6C829390-CBAC-46C0-80EE-C49B8E69AF13@gmail.com> References: <1682F923-F5EF-4553-8011-53B39CFD3B71@gmail.com> <51D473AC.8000900@pearwood.info> <54AEACBC-4BB2-45BF-856F-66ACAE54C8F8@gmail.com> <6C829390-CBAC-46C0-80EE-C49B8E69AF13@gmail.com> Message-ID: On 3 July 2013 23:37, Andre' Walker-Loud wrote: > Hi Oscar, Hi Andre', (your name shows in my email client with an apostrophe ' after it; I'm not sure if I'm supposed to include that when I write it). >> The error is for creating an mmap object. This is not something that >> numpy does unless you tell it to. > > I have not explicitly told it to. > >> So you're using pytables or h5py, or something else? It really would >> help if you would specify this instead of trying to be generic. > > All the third party software I am using: > numpy > pytables > hdf5 What do you mean above by hdf5? Is that a software library or just the format? I'll assume you're using that format and that you're using the pytables library to access it. > pyminuit [python interface to Minuit] > Minuit [Minuit - a c++ code based developed in CERN for numerical minimization] Okay, this is much better. > I was hoping I was must making some obvious mistake, assuming people are mostly not familiar with much of any of the specific code I am using. But I guess it is either not so obvious, or my explanations are too poor. I wouldn't be surprised if pytables had a memory leak in it. Although I'd be less surprised if you were using it incorrectly and that resulted in a memory leak. >> My guess is that the hdf5 library loads the array as an mmap'ed memory >> block and you're not actually working with an ordinary numpy array >> (even if it has a similar interface). > > I specifically load the data once, as > > my_file = tables.openFile(my_data_file.h5) > my_data = my_file.getNode(path_to_data).read() > > after this, "my_data" seems to have all the features of a numpy array. for example, > >> Have you checked the actual memory size of the array? If it's a real >> numpy array you can use the nbytes attribute: >>>>> a = numpy.zeros([300, 256, 1, 2], float) >>>>> a.nbytes >> 1228800 > > this works on my_data. Okay, so what happens if you don't import pytables, don't load the my_data_file.h5 file and just say data = numpy.zeros([300, 256, 1, 2], float) and run the rest of your code? Do you still get a memory leak? >>> class do_stuff: >>> # I am aware this doesn't follow the class naming convention, just sticking with my previous post name >>> def __call__(data,other_vars): >>> self.fit = third_party.function_set_up(data,other_vars) >>> >>> def minimize(self): >>> try: >>> self.fit.minimize() >>> self.have_fit = True >>> except third_party.Error: >>> self.have_fit = False >>> ########################## >> >> If you write code like the above then you really cannot expect other >> people to just understand what you mean if you don't show them the >> code. Specifically the use of __call__ is confusing. Really, though, >> this class is just a distraction from your problem and should have >> been simplified away. > > Off my main topic, but could you explain more? > I am also not very experienced writing classes, and was learning from an example. So I am not sure why __call__ is confusing. I thought that was correct. You're using __call__ correctly. What I mean is that while it may (or may not) be appropriate for you to use __call__ in your program it seems inappropriate in the simplified example. I expect in a genericised example that `my_func` is a real function. I don't expect it to have a minimize method and I expect it to be imported rather than created. To quote from Dave and Alan earlier in the thread: On 3 July 2013 19:55, Dave Angel wrote: > On 07/03/2013 02:17 PM, Andre' Walker-Loud wrote: >> >> # instantiate the function do_stuff >> my_func = my_class.do_stuff() > > So this is a class-static method which returns a callable object? One with > methods of its own? Also: On 3 July 2013 19:57, Alan Gauld wrote: > On 03/07/13 19:17, Andre' Walker-Loud wrote: > >> # instantiate the function do_stuff >> my_func = my_class.do_stuff() > > You don;t instantiate functions you call them. You are setting my_func to be > the return value of do_stuff(). What is that return value? What does my_func > actually refer to? >>> tmp_data = my_class.chop_data(data,n,m) >> >> Where did data come from? Is that the mmap'ed array from the hdf5 library? > > as above, data is loaded once with pytables, using the .read() function. > Subsequently, with my "chop_data" function, I believe that returns a numpy array, and I have not explicitly asked it anything about "mmap" so I am not sure. How would you check? If you haven't asked for an mmap then you won't get one from any numpy operations. However pytables creates mmap'ed arrays that behave like numpy arrays so I believe that's where the problem comes in. What happens if you do something like: >>> print(type(data)) >>> my_func(tmp_data,other_vars) >>> my_func.minimize() >> >> I now know that the above two lines call thirdparty.function_setup() >> and my_func.fit.minimize(). I still have no idea what they do though. > > I was trying to avoid that, since I suspect(ed) the problem is with me and not the third party. > Without going into specifics, the first function constructs a chi^2 function which looks like > > chisq = sum_i ( (data[i] - fit_func(i,fit_params)) / data_error[i] )**2 > > the second function works to numerically minimize chisq with respect to the "fit_params" which are a 1d array. What happens if you just comment those lines out? Or if you replace them with dummy lines that don't do anything interesting? This is how you'll narrow down the problem. > If you can help me understand the issue of mmap (whether somehow I am creating this unwittingly), that would be great. ie, what tests can I perform to check? > Otherwise, it seems perhaps the best thing for me to do now is take eryksun's advice and learn how to use a memory profiler. I'm not so sure if you'll want to go down that path just yet. I personally expect that I could find the cause of your problem if I was running your code on my computer and I've never used a memory profiler before (alternatively that could mean that I'm not qualified to advise on this point though). Based on the described list of software and your statement that you're not explicitly creating mmaps yourself my suspicion points at pytables (or your usage of pytables). Try using a dummy array and eliminating pytables from your program. If pytables is a necessary part of the memory leak problem then you should probably go to the pytables-users mailing list for more help. If the issue is unrelated to pytables then you're a few steps on your way to simplifying it into a simple but complete example that you can post here. Really though I'm guessing as I still can't see a complete example that demonstrates the problem. Creating a Short, Self-Contained, Correct Example (http://sscce.org/) is a useful thing to do; the process will more often than not enable you to discover the root cause of your problem or at least to fix the problem. If it does not then it gives you something to post to a mailing list that someone else can easily analyse and then tell you what the problem is. Oscar From steve at pearwood.info Thu Jul 4 03:20:48 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 04 Jul 2013 11:20:48 +1000 Subject: [Tutor] memory consumption In-Reply-To: <8D91EAD6-5B66-433B-9FA6-B3D1F5D01CA0@gmail.com> References: <1682F923-F5EF-4553-8011-53B39CFD3B71@gmail.com> <51D473AC.8000900@pearwood.info> <54AEACBC-4BB2-45BF-856F-66ACAE54C8F8@gmail.com> <8D91EAD6-5B66-433B-9FA6-B3D1F5D01CA0@gmail.com> Message-ID: <51D4CDF0.5040603@pearwood.info> On 04/07/13 08:11, Andre' Walker-Loud wrote: > Yes, I was being sloppy. My later post clarified what I meant. > The loops are really lists, and I was really using enumerate() >to get both the iter and the element. > > loop_2 = [1,2,4,8,16,32,64,128,256,512,1024] > for i,n in enumerate(loop_2): > ... Please be careful about portraying yourself as less experienced/more naive than you really are. Otherwise we end up wasting both our time and yours telling you to do things that you're already doing. Have you googled for "Python memory leak os-x"? When I do, I find a link to this numpy bug: https://github.com/numpy/numpy/issues/2969 -- Steven From steve at pearwood.info Thu Jul 4 03:29:45 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 04 Jul 2013 11:29:45 +1000 Subject: [Tutor] memory consumption In-Reply-To: References: <1682F923-F5EF-4553-8011-53B39CFD3B71@gmail.com> <51D473AC.8000900@pearwood.info> <54AEACBC-4BB2-45BF-856F-66ACAE54C8F8@gmail.com> <6C829390-CBAC-46C0-80EE-C49B8E69AF13@gmail.com> Message-ID: <51D4D009.4050405@pearwood.info> On 04/07/13 09:24, Oscar Benjamin wrote: > On 3 July 2013 23:37, Andre' Walker-Loud wrote: >> Hi Oscar, > > Hi Andre', > > (your name shows in my email client with an apostrophe ' after it; I'm > not sure if I'm supposed to include that when I write it). I expect that it's meant to be Andr?, since that is the "correct" spelling even in English (however a lot of non-French Andr?s drop the accent). -- Steven From davea at davea.name Thu Jul 4 09:25:44 2013 From: davea at davea.name (Dave Angel) Date: Thu, 04 Jul 2013 03:25:44 -0400 Subject: [Tutor] Saving Progress In-Reply-To: <3FE557DB-0F34-4604-BE6A-82278EDFE031@yahoo.com> References: <3FE557DB-0F34-4604-BE6A-82278EDFE031@yahoo.com> Message-ID: On 06/27/2013 11:16 AM, Jack Little wrote: > Is there a way to save a players progress in a game using python without any modules > > Jack > If you organize the game in such a way that a relatively few variables holds the state, then you can write those variables to a file, and when restarting the program, load them from the file. Naturally you'd want to pick a file location that represents the player's identity. User name is the obvious, but on many personal (game) computers, people use one user for many people. In this case, you'd want some form of login when the game starts each time. If the variables you need to save are all ints, floats, and strings, writing the state-file is pretty easy. -- DaveA From fomcl at yahoo.com Thu Jul 4 21:40:54 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Thu, 4 Jul 2013 12:40:54 -0700 (PDT) Subject: [Tutor] custom error classes: how many of them does one need? Message-ID: <1372966854.32001.YahooMailNeo@web163805.mail.gq1.yahoo.com> Hello, I created a custom error class like under [A] below (see http://pastebin.com/JLwyPsRL if you want to see highlighted code). Now I am thinking: is this specific enough? "When creating a module that can raise several distinct errors, a common practice is to create a base class for exceptions defined by that module, and subclass that to create specific exception classes for different error conditions" (http://docs.python.org/2/tutorial/errors.html) Given that I have many retcodes, is it a better approach to define a class factory that generates a custom exception for each retcode on-the-fly? (under [B]). Using a class factory seems a little unusual, or isn it? More generally: what criteria should one follow when distinguishing error classes? global retcodes retcodes = {0: "OK", 1: "Error_X", 2: "Error_Y"} ############################# # [A] one custom exception for all purposes, but with a descriptive message ############################# class single_custom_exception(Exception): ??? def __init__(self, retcode, message): ??????? self.retcode = retcode ??????? message += " [%s]" % retcodes.get(retcode) ??????? Exception.__init__(self, message) ############################# # [B] generating custom exceptions with a class factory ############################# class BaseError(Exception): pass def throw(retcode, message): ??? """class factory that generates an error class for ??? each return code label""" ??? class custom_exception(BaseError): ??????? def __init__(self): ??????????? BaseError.__init__(self, message) ??? exception = retcodes.get(retcode) ??? custom_exception.__name__ = exception ??? return custom_exception ####### # demonstrate use ####### def someFunc(scenario): ??? """a nonsensical function""" ??? retcode = 1 ??? if retcode > 0: ??????? message = "error calling %r" % someFunc.__name__ ??????? if scenario == "[A]": ??????????? raise single_custom_exception(retcode, message) ??????? elif scenario == "[B]": ??????????? raise throw(retcode, message) >>> someFunc("[A]") Traceback (most recent call last): ? File "/home/antonia/Desktop/tutor.py", line 44, in ??? someFunc("[A]") ? File "/home/antonia/Desktop/tutor.py", line 40, in someFunc ??? raise single_custom_exception(retcode, message) single_custom_exception: error calling 'someFunc' [Error_X] >>> someFunc("[B]") Traceback (most recent call last): ? File "/home/antonia/Desktop/tutor.py", line 44, in ??? someFunc("[B]") ? File "/home/antonia/Desktop/tutor.py", line 42, in someFunc ??? raise throw(retcode, message) Error_X: error calling 'someFunc' # this may be useful print issubclass(throw(1, "some message"), BaseError)? # returns True ? Regards, Albert-Jan ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health, what have the Romans ever done for us? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~? From marc.tompkins at gmail.com Thu Jul 4 23:01:49 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Thu, 4 Jul 2013 14:01:49 -0700 Subject: [Tutor] Saving Progress In-Reply-To: References: <3FE557DB-0F34-4604-BE6A-82278EDFE031@yahoo.com> Message-ID: On Thu, Jul 4, 2013 at 12:25 AM, Dave Angel wrote: > On 06/27/2013 11:16 AM, Jack Little wrote: > >> Is there a way to save a players progress in a game using python without >> any modules >> >> Jack >> >> > If you organize the game in such a way that a relatively few variables > holds the state, then you can write those variables to a file, and when > restarting the program, load them from the file. > > Naturally you'd want to pick a file location that represents the player's > identity. User name is the obvious, but on many personal (game) computers, > people use one user for many people. In this case, you'd want some form of > login when the game starts each time. > > If the variables you need to save are all ints, floats, and strings, > writing the state-file is pretty easy. > > I'd just like to point out that if the file you write to is human-readable, then it's human-writable too; you probably don't want to have a line that says "NumberOfLivesRemaining=5" or similar. Make it at least a _little_ challenging to cheat. Security-through-obscurity never really works in the end; if your players want to cheat, they'll find a way. But don't make it TOO easy! -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan.gauld at btinternet.com Fri Jul 5 00:55:32 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Thu, 04 Jul 2013 23:55:32 +0100 Subject: [Tutor] custom error classes: how many of them does one need? In-Reply-To: <1372966854.32001.YahooMailNeo@web163805.mail.gq1.yahoo.com> References: <1372966854.32001.YahooMailNeo@web163805.mail.gq1.yahoo.com> Message-ID: On 04/07/13 20:40, Albert-Jan Roskam wrote: > creating a module that can raise several distinct errors, a common practice is > to create a base class for exceptions defined by that module, and subclass that > to create specific exception classes for different error conditions" Yes, although not usually too many... > Given that I have many retcodes, return codes (aka retcodes?) are semantically different from exception or error codes. You should clearly distinguish between valid return codes and error codes. Or do you mean by retcode that you have multiple sub-error codes? That's usually a messy solution... It puts extra work onto the user of your code in disentangling the mess. > is it a better approach to define a class factory that generates > a custom exception for each retcode on-the-fly? That shouldn't really be necessary unless you've gone down the messy multiple levels of error approach. Usually its easier to just define new Error classes. They don;t need to contain much! > retcodes = {0: "OK", 1: "Error_X", 2: "Error_Y"} You don't need the first one. If its OK its OK there should be no exception. The other two should be separate error types I suspect. > ############################# > # [A] one custom exception for all purposes, but with a descriptive message > ############################# > > class single_custom_exception(Exception): > def __init__(self, retcode, message): > self.retcode = retcode > message += " [%s]" % retcodes.get(retcode) > Exception.__init__(self, message) Why not class Error_X(Exception):pass class Error_Y(Exception):pass Just a thought... Or is there some fancy extra functionality required that you're not telling us about? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From steve at pearwood.info Fri Jul 5 04:20:07 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 05 Jul 2013 12:20:07 +1000 Subject: [Tutor] custom error classes: how many of them does one need? In-Reply-To: <1372966854.32001.YahooMailNeo@web163805.mail.gq1.yahoo.com> References: <1372966854.32001.YahooMailNeo@web163805.mail.gq1.yahoo.com> Message-ID: <51D62D57.7060400@pearwood.info> On 05/07/13 05:40, Albert-Jan Roskam wrote: > Hello, > > I created a custom error class like under [A] below (see http://pastebin.com/JLwyPsRL if you want to see highlighted code). Now I am thinking: is this specific enough? "When > creating a module that can raise several distinct errors, a common practice is > to create a base class for exceptions defined by that module, and subclass that > to create specific exception classes for different error conditions" (http://docs.python.org/2/tutorial/errors.html) > Given that I have many retcodes, is it a better approach to define a class factory that generates a custom exception for each retcode on-the-fly? (under [B]). Using a class factory seems a little unusual, or isn it? More generally: what criteria should one follow when distinguishing error classes? Return codes are not exceptions, and generally speaking using return codes for errors is *strongly* discouraged unless you have a language without exceptions. (Although the designers of Go language disagree. I think they are wrong to do so.) Although looking forward, I see that even though you call them "retcodes", you don't actually use them as return codes. > global retcodes > retcodes = {0: "OK", 1: "Error_X", 2: "Error_Y"} You don't need to declare a global variable global in the global part of your file. Although Python allows it, it is redundant and rather silly. Delete the line "global retcodes". > ############################# > # [A] one custom exception for all purposes, but with a descriptive message > ############################# > > class single_custom_exception(Exception): > def __init__(self, retcode, message): > self.retcode = retcode > message += " [%s]" % retcodes.get(retcode) > Exception.__init__(self, message) Exceptions are classes, and like all classes, the convention is to name them in CamelCase. > ############################# > # [B] generating custom exceptions with a class factory > ############################# > > class BaseError(Exception): pass > > def throw(retcode, message): > """class factory that generates an error class for > each return code label""" > class custom_exception(BaseError): > def __init__(self): > BaseError.__init__(self, message) > exception = retcodes.get(retcode) > custom_exception.__name__ = exception > return custom_exception This is overkill. No need to create a custom class *on the fly* for each return code. Doing so is pointless: the caller *cannot* catch them individually, because they are created on the fly. So they can only catch BaseError, then inspect the exception's details. If you have to do that, then having individual classes is a waste of time. You might as well just use a single class and be done with it. What you would like to do, but cannot, since neither Error_X nor Error_Y are defined: try: code_that_might_fail() except Error_X: handle_error_X except Error_Y: handle_error_Y What you are forced to do: try: code_that_might_fail() except BaseError as err: if err.retcode == 0: print ("Error: no error occurred!") elif err.retcode == 1: handle_error_X elif err.retcode == 1: handle_error_Y else: print ("Error: an unexpected error occurred!") which is ugly enough, but it makes the class factory redundant. So, how should you handle this scenario? Use a separate exception class for each high-level *category* of errors. E.g. Python has: TypeError ValueError ZeroDivisionError UnicodeEncodeError etc. Within your app, can you divide the return codes into application-specific groups? If not, then you just need a single exception class: MyApplicationError If you do have a bunch of application-specific groups, then you might develop a hierarchical family of exceptions: WidgetError +-- WrongWidgetTypeError +-- InvalidWidgetError +-- WidgetNotInitialisedError WidgetGroupError +-- MissingWidgetError | +-- MissingCriticalWidgetError +-- TooManyWidgetsError SpoonError +-- TeaspoonError +-- SoupSpoonError +-- DesertSpoonError The parent exceptions, WidgetError, WidgetGroupError and SpoonError do not need to share a common base class. This hierarchy might completely obliterate the need for return codes. Python's build-in exceptions generally do not. However, maybe you do still need them, similar to the way Python's OSError and IOError have error codes. Assuming you do: def someFunc(): perform_calculations() if there are no spoons: raise TeaspoonError("cannot stir my tea") elif some other error: raise InvalidWidgetError(retcode=99, message="Can't use red widget") else: return some value -- Steven From eryksun at gmail.com Fri Jul 5 04:40:03 2013 From: eryksun at gmail.com (eryksun) Date: Thu, 4 Jul 2013 22:40:03 -0400 Subject: [Tutor] custom error classes: how many of them does one need? In-Reply-To: <1372966854.32001.YahooMailNeo@web163805.mail.gq1.yahoo.com> References: <1372966854.32001.YahooMailNeo@web163805.mail.gq1.yahoo.com> Message-ID: On Thu, Jul 4, 2013 at 3:40 PM, Albert-Jan Roskam wrote: > a common practice is to create a base class for exceptions defined by > that module, and subclass that to create specific exception classes > for different error conditions" Look to the reorganization of OSError in 3.3 as a model, specifically the rationale regarding fine-grained exceptions: http://www.python.org/dev/peps/pep-3151 http://docs.python.org/3/library/exceptions#exception-hierarchy > global retcodes > retcodes = {0: "OK", 1: "Error_X", 2: "Error_Y"} FYI, the "global" declaration here is redundant. At module level, locals and globals use the same dict. So there's no point in forcing the compiler to use STORE_GLOBAL instead of the default STORE_LOCAL. It would only be necessary if you were doing something out of the ordinary, such as using exec with globals and locals set to different dicts: >>> nsglobals, nslocals = {}, {} >>> exec 'def f(): pass' in nsglobals, nslocals >>> 'f' in nsglobals False >>> nslocals['f'].__globals__ is nsglobals True Now with a "global f" declaration: >>> nsglobals, nslocals = {}, {} >>> exec 'global f\ndef f(): pass' in nsglobals, nslocals >>> nsglobals['f'].__globals__ is nsglobals True From eryksun at gmail.com Fri Jul 5 04:42:11 2013 From: eryksun at gmail.com (eryksun) Date: Thu, 4 Jul 2013 22:42:11 -0400 Subject: [Tutor] custom error classes: how many of them does one need? In-Reply-To: References: <1372966854.32001.YahooMailNeo@web163805.mail.gq1.yahoo.com> Message-ID: On Thu, Jul 4, 2013 at 10:40 PM, eryksun wrote: > STORE_LOCAL Sorry, that should have been STORE_NAME. From jacklittlemc at yahoo.com Sat Jul 6 05:33:08 2013 From: jacklittlemc at yahoo.com (Jack Little) Date: Fri, 5 Jul 2013 20:33:08 -0700 (PDT) Subject: [Tutor] Exiting a while Message-ID: <1373081588.70134.YahooMailNeo@web124501.mail.ne1.yahoo.com> How would I exit a while statement. The while is to keep the player in combat with an enemy until said enemy is dead. I tried using an if, but it just doesn't work. Jack -------------- next part -------------- An HTML attachment was scrubbed... URL: From amitsaha.in at gmail.com Sat Jul 6 05:38:48 2013 From: amitsaha.in at gmail.com (Amit Saha) Date: Sat, 6 Jul 2013 13:38:48 +1000 Subject: [Tutor] Exiting a while In-Reply-To: <1373081588.70134.YahooMailNeo@web124501.mail.ne1.yahoo.com> References: <1373081588.70134.YahooMailNeo@web124501.mail.ne1.yahoo.com> Message-ID: On Sat, Jul 6, 2013 at 1:33 PM, Jack Little wrote: > How would I exit a while statement. The while is to keep the player in > combat with an enemy until said enemy is dead. I tried using an if, but it > just doesn't work. The 'break' statement can help you. While you are there, also learn about 'continue'. Here is the doc: http://docs.python.org/2/tutorial/controlflow.html#break-and-continue-statements-and-else-clauses-on-loops -- http://echorand.me From alan.gauld at btinternet.com Sat Jul 6 10:25:39 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Sat, 06 Jul 2013 09:25:39 +0100 Subject: [Tutor] Exiting a while In-Reply-To: <1373081588.70134.YahooMailNeo@web124501.mail.ne1.yahoo.com> References: <1373081588.70134.YahooMailNeo@web124501.mail.ne1.yahoo.com> Message-ID: On 06/07/13 04:33, Jack Little wrote: > How would I exit a while statement. The while is to keep the player in > combat with an enemy until said enemy is dead. I tried using an if, but > it just doesn't work. In most cases you make the while condition false. while enemy.state != dead: # your loop code here will automatically break when your enemy state is dead. Another common idiom is to use an infinite loop: while True: #loops forever if enemy.state == dead: break # exits the nearest containing loop # your loop code here The first form is better since it requires less maintenance but had the disadvantage that you often have to initialize the condition prior to the loop which can lead to repeated code - another maintenance headache. You have to choose. hth -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From steve at pearwood.info Sat Jul 6 11:48:45 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 06 Jul 2013 19:48:45 +1000 Subject: [Tutor] Exiting a while In-Reply-To: <1373081588.70134.YahooMailNeo@web124501.mail.ne1.yahoo.com> References: <1373081588.70134.YahooMailNeo@web124501.mail.ne1.yahoo.com> Message-ID: <51D7E7FD.3030602@pearwood.info> On 06/07/13 13:33, Jack Little wrote: > How would I exit a while statement. The while is to keep the player in combat with an enemy until said enemy is dead. I tried using an if, but it just doesn't work. The devil is in the details, which you do not tell us. How precisely did you use an "if"? Should we guess what you did? If you did this: while fighting: do combat if enemy hit points <= 0: print "Hello!" then naturally it would not work. But if you did this: while fighting: do combat if enemy hit points <= 0: break it will. So the real question is, what did you try? -- Steven From cybervigilante at gmail.com Sat Jul 6 23:38:27 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 6 Jul 2013 14:38:27 -0700 Subject: [Tutor] please return flys in ointment Message-ID: I reworked my numbers_to_name program so I can actually follow the logic. Since I think the best way to learn is to follow something through to completion, I'll put it online so I learn how to do that. But naturally, lest I look dumb, I'd appreciate criticism, improvements of bad form, or if anyone can break it, let me know ;') . Seems to work with a bit of random testing, but ya never know. If you did, Msoft wouldn't be putting out those updates all the time. Works with Py 27 or 33, but that brings up a question: I think most of my programs work with both if I don't do anything exotic, except for input(), which is the fly in the ointment. Instead of using the routine below, could I redefine input() to use the routine, or it that a bad idea? I have a feeling the leftpad routine could be simplified, but it isn't coming to me. # Using C:\Python33\python.exe on Win 7 in c:\python33\jimprogs - not Windows-specific # Also works with python 2.7 import sys # Data ones = {'1': 'one', '2': 'two', '3': 'three', '4': 'four', '5': 'five', '6': 'six', '7': 'seven', '8': 'eight', '9': 'nine'} tens = {'2': 'twenty', '3': 'thirty', '4': 'forty', '5': 'fifty', '6': 'sixty', '7': 'seventy', '8': 'eighty', '9': 'ninety'} doubles = {'0': 'ten', '1': 'eleven', '2': 'twelve', '3': 'thirteen', '4': 'fourteen', '5': 'fifteen', '6': 'sixteen', '7': 'seventeen', '8': 'eighteen', '9': 'nineteen'} powers_of_1000 = (' thousand', ' million', ' billion', ' trillion', ' quadrillion', ' quintillion', ' sextillion', ' septillion', ' octillion', ' nonillion', ' decillion') '''add these later, and also option for dollars-and-cents ending. 'vigintillion', 'novemdecillion', 'octodecillion', 'septendecillion', 'sexdecillion', 'quindecillion', 'quattuordecillion', 'tredecillion', 'duodecillion', 'undecillion', 'decillion', 'nonillion' ''' # Functions def make_triplets_from_input(): '''Enter a positive integer. A list of triplets in the same order will be returned. Raise ValueError to loop on non-integer input, or return 'zero' trigger-value of 'xxx' if all zeroes are entered. If input is okay, strip leading zeroes, then zero-pad leftmost triplet to three characters, so all slices are triplets. Adjust input for different Python versions.''' while True: try: if sys.version[0:3] == '2.7': numbers_str = original = raw_input('Enter a positive integer, space \ separated if desired.') elif sys.version[0:3] == '3.3': numbers_str = original = input('Enter a positive integer, space \ separated if desired.') else: print('Python versions 2.7 and 3.3 only supported') sys.exit() numbers_str = ''.join(numbers_str.split()) if numbers_str == '' or numbers_str == ' ': raise ValueError numbers_int = int(numbers_str) if numbers_int == 0: print('Zero') sys.exit() numbers_str = str(numbers_int) if len(numbers_str) > 36: raise ArithmeticError break except KeyboardInterrupt: print('Program cancelled by user') sys.exit() except ValueError as err: print(original, "is not an integer.") continue except ArithmeticError as err: print(original, "is too big.\n999...decillion is max: 10**37-1 or 36 chars \ or 12 groups of 3 chars") leftpad = len(numbers_str) % 3 # zero-pad left, so we get all 3-character triplets if leftpad == 0: leftpad = '' elif leftpad == 2: leftpad = '0' else: leftpad = '00' numbers_str = leftpad + numbers_str triplets = [numbers_str[x:x+3] for x in range(0,len(numbers_str),3)] return (triplets, len(triplets)) def numbers_to_name(triplets, triplen): '''Create a name from each triplet and append the power of ten''' triplen -= 2 number_name = '' for triplet in triplets: triplet_name = '' first, second, third, last_two = (triplet[0], triplet[1], triplet[2], triplet[1:]) if triplet == '000': triplen -= 1 continue if first > '0': if last_two == '00': # special case - snip extra space separator triplet_name += ones.get(first) + ' hundred' else: triplet_name += ones.get(first) + ' hundred ' if second == '0': if third > '0': triplet_name += ones.get(third) elif second == '1': triplet_name += doubles.get(third) elif second > '1': triplet_name += tens.get(second) if third > '0': triplet_name += '-' + ones.get(third) number_name += triplet_name if triplen > -1: number_name += powers_of_1000[triplen] + ', ' triplen -= 1 if number_name[-2] == ',': number_name = number_name[0:-2] # special case - snip extraneous ending comma return number_name # Main Program - turn numeric input into number-names triplets, triplen = make_triplets_from_input() print(numbers_to_name(triplets, triplen)) -- Jim "I asked God for a bike, but I know God doesn't work that way. So I stole a bike and asked for forgiveness." ~ Anonymous -------------- next part -------------- An HTML attachment was scrubbed... URL: From davea at davea.name Sun Jul 7 00:34:57 2013 From: davea at davea.name (Dave Angel) Date: Sat, 06 Jul 2013 18:34:57 -0400 Subject: [Tutor] please return flys in ointment In-Reply-To: References: Message-ID: On 07/06/2013 05:38 PM, Jim Mooney wrote: > I reworked my numbers_to_name program so I can actually follow the logic. > Since I think the best way to learn is to follow something through to > completion, I'll put it online so I learn how to do that. But naturally, > lest I look dumb, I'd appreciate criticism, improvements of bad form, or if > anyone can break it, let me know ;') . Seems to work with a bit of random > testing, but ya never know. If you did, Msoft wouldn't be putting out those > updates all the time. Works with Py 27 or 33, but that brings up a > question: I think most of my programs work with both if I don't do anything > exotic, except for input(), which is the fly in the ointment. Instead of > using the routine below, could I redefine input() to use the routine, or it > that a bad idea? > > I have a feeling the leftpad routine could be simplified, but it isn't > coming to me. > General comments: 1) several lines wrapped in the newreader, so I had to paste things together. Presumably this won't happen whereever you put it online, but you should make sure and check after uploading it to make sure no such problems were introduced. Also, instead of using the backslash line-continuation character, I generally try to use the syntax of the language instead. So if you have a quote which won't fit on one line, but you don't want embedded newlines in it either, you can use the implicit concatenation of adjacent literals. Frequently you have to put parens around the whole thing to make sure the compiler sees it as one line. But I think it's usually more readable that way. First example: the prompt in the input/raw_input lines. > # Using C:\Python33\python.exe on Win 7 in c:\python33\jimprogs - not > Windows-specific > # Also works with python 2.7 > You always take the data in by input/raw_input. Why not permit a value on the command line? And even more important, you're mixing input/output with algorithm here. > import sys > > # Data > ones = {'1': 'one', '2': 'two', '3': 'three', '4': 'four', '5': 'five', > '6': 'six', > '7': 'seven', '8': 'eight', '9': 'nine'} > > tens = {'2': 'twenty', '3': 'thirty', '4': 'forty', '5': 'fifty', '6': > 'sixty', > '7': 'seventy', '8': 'eighty', '9': 'ninety'} > > doubles = {'0': 'ten', '1': 'eleven', '2': 'twelve', '3': 'thirteen', '4': > 'fourteen', > '5': 'fifteen', '6': 'sixteen', '7': 'seventeen', '8': 'eighteen', '9': > 'nineteen'} > > powers_of_1000 = (' thousand', ' million', ' billion', ' trillion', > ' quadrillion', ' quintillion', ' sextillion', ' septillion', ' octillion', > ' nonillion', ' decillion') > > '''add these later, and also option for dollars-and-cents ending. > 'vigintillion', 'novemdecillion', 'octodecillion', 'septendecillion', > 'sexdecillion', 'quindecillion', 'quattuordecillion', 'tredecillion', > 'duodecillion', 'undecillion', > 'decillion', 'nonillion' Those last two you've already implemented > ''' > > # Functions > def make_triplets_from_input(): > '''Enter a positive integer. A list of triplets in the same order will > be returned. > Raise ValueError to loop on non-integer input, or return 'zero' > trigger-value of > 'xxx' if all zeroes are entered. If input is okay, strip leading > zeroes, then > zero-pad leftmost triplet to three characters, so all slices are > triplets. Adjust > input for different Python versions.''' > while True: > try: > if sys.version[0:3] == '2.7': I'd do the version checking in initialization code, not here. And I'd do it by redefining input to do what it does in 3.3 > numbers_str = original = raw_input('Enter a positive > integer, space \ > separated if desired.') > elif sys.version[0:3] == '3.3': > numbers_str = original = input('Enter a positive integer, > space \ > separated if desired.') Note: you explicitly support zero, so this should say non-negative, rather than positive. I'm not sure why you support spaces between the digits, and not commas, but no biggie. BTW, you also support other white space, like tabs. > else: > print('Python versions 2.7 and 3.3 only supported') > sys.exit() > > numbers_str = ''.join(numbers_str.split()) > if numbers_str == '' or numbers_str == ' ': raise ValueError > numbers_int = int(numbers_str) > if numbers_int == 0: > print('Zero') > sys.exit() This is just weird. You're in the middle of an input routine, and you just print something and exit the code?? > numbers_str = str(numbers_int) > if len(numbers_str) > 36: raise ArithmeticError > break > except KeyboardInterrupt: > print('Program cancelled by user') > sys.exit() > except ValueError as err: > print(original, "is not an integer.") > continue > except ArithmeticError as err: > print(original, "is too big.\n999...decillion is max: 10**37-1 > or 36 chars \ > or 12 groups of 3 chars") > > leftpad = len(numbers_str) % 3 # zero-pad left, so we get all > 3-character triplets > if leftpad == 0: leftpad = '' > elif leftpad == 2: leftpad = '0' > else: leftpad = '00' Instead of these four lines, how about: leftpad = "0" * (-i)%3 > numbers_str = leftpad + numbers_str > triplets = [numbers_str[x:x+3] for x in range(0,len(numbers_str),3)] > return (triplets, len(triplets)) Seems a little silly to make this a tuple, when the second value is trivially figured from the first. You could just have the second function take a single list argument and take its len() for itself. And even then, I'd recommend using a slice when calling it instead of passing it a number. Just one more bug lurking in the wings. > > def numbers_to_name(triplets, triplen): > '''Create a name from each triplet and append the power of ten''' > triplen -= 2 > number_name = '' > > for triplet in triplets: > triplet_name = '' > first, second, third, last_two = (triplet[0], triplet[1], > triplet[2], triplet[1:]) > if triplet == '000': > triplen -= 1 > continue > if first > '0': > if last_two == '00': # special case - snip extra space separator > triplet_name += ones.get(first) + ' hundred' > else: > triplet_name += ones.get(first) + ' hundred ' > if second == '0': > if third > '0': > triplet_name += ones.get(third) > elif second == '1': > triplet_name += doubles.get(third) > elif second > '1': > triplet_name += tens.get(second) > if third > '0': > triplet_name += '-' + ones.get(third) > number_name += triplet_name > if triplen > -1: > number_name += powers_of_1000[triplen] + ', ' > triplen -= 1 > if number_name[-2] == ',': > number_name = number_name[0:-2] # special case - snip extraneous > ending comma > return number_name > > # Main Program - turn numeric input into number-names > triplets, triplen = make_triplets_from_input() > print(numbers_to_name(triplets, triplen)) > If you included the usual if __name__ == test around those two lines, you'd be able to use this same file as an importable module. You should have four functions, not two: 1) parse argv 2) get input from user 3) turn string into list ( make_triplets_from_input) 4) turn list into string, ready to print (numbers_to_name) and then the top-level code does something like: if __name__ == "__main__": if len(argv) == 2: raw = parseargv() elif len(argv) == 1: raw = getinputfromuser() else: someerrormessage exit triplets = make_triplets_from_input(raw) print(numbers_to_name(triplets)) -- DaveA From alan.gauld at btinternet.com Sun Jul 7 01:06:29 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Sun, 07 Jul 2013 00:06:29 +0100 Subject: [Tutor] please return flys in ointment In-Reply-To: References: Message-ID: On 06/07/13 22:38, Jim Mooney wrote: > naturally, lest I look dumb, I'd appreciate criticism, OK, here are a few comments, take em or leave em... > import sys > > # Data > # Functions > def make_triplets_from_input(): Why not pass in the prompt string as an argument? > '''Enter a positive integer. A list of triplets in the same order > will be returned. > Raise ValueError to loop on non-integer input, or return 'zero' > trigger-value of > 'xxx' if all zeroes are entered. If input is okay, strip leading > zeroes, then > zero-pad leftmost triplet to three characters, so all slices are > triplets. Adjust > input for different Python versions.''' Not strictly accurate - see comments below. > while True: > try: > if sys.version[0:3] == '2.7': > numbers_str = original = raw_input('Enter a positive > integer, space \ > separated if desired.') I'd probably just use input = raw_input to keep the code simple. > elif sys.version[0:3] == '3.3': > numbers_str = original = input('Enter a positive > integer, space \ > separated if desired.') Then you don't really need this bit unless you really want to exclude other v3.X Python's which seems a tad harsh! > else: > print('Python versions 2.7 and 3.3 only supported') > sys.exit() Probably better to take a chance and say Python versions above 2.7. Backward compatibility is pretty good in Python land. > numbers_str = ''.join(numbers_str.split()) > if numbers_str == '' or numbers_str == ' ': raise ValueError > numbers_int = int(numbers_str) Why not just allow the built in Exception to raise itself? >>> int('') Traceback (most recent call last): File "", line 1, in ValueError: invalid literal for int() with base 10: '' >>> > if numbers_int == 0: > print('Zero') > sys.exit() > numbers_str = str(numbers_int) Not sure why this is needed, you already got the int from the str so why not use the original string? > if len(numbers_str) > 36: raise ArithmeticError > break Its not really Arithmetic? I'd have said a ValueError... > except KeyboardInterrupt: > print('Program cancelled by user') > sys.exit() Again, what's wrong with the standard error message? > except ValueError as err: > print(original, "is not an integer.") > continue > except ArithmeticError as err: > print(original, "is too big.\n999...decillion is max: > 10**37-1 or 36 chars \ > or 12 groups of 3 chars") OK, Now i see why the ArithmeticError. But you cpuild just have printed the message before raising ValueError? Or even passed the message into ValueError? >>> raise ValueError('Value too big, try again') Traceback (most recent call last): File "", line 1, in ValueError: Value too big, try again >>> It feels like you are trying to do much of Python's job yourself. Let the force be with you... > leftpad = len(numbers_str) % 3 # zero-pad left, so we get all > 3-character triplets > if leftpad == 0: leftpad = '' > elif leftpad == 2: leftpad = '0' > else: leftpad = '00' Feels like a job for a dictionary and get() - or even a simple tuple index: leftpad = ('','0','00')[leftpad] Personally I prefer not to change types in a single variable like that (probably my C/Pascal background coming out!) so I'd probably do it like this: leftpad = ('','0','00')[ len(numbers_str) % 3 ] Or introduce an extra vareiable - padlen or somesuch... > numbers_str = leftpad + numbers_str > triplets = [numbers_str[x:x+3] for x in range(0,len(numbers_str),3)] > return (triplets, len(triplets)) Not sure I'd bother returning the len, the user can get it as needed very easily and your function name and docstring give no hint to expect a len result as well as the triplets. No further comments - its getting late! :-) HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From davea at davea.name Sun Jul 7 01:30:04 2013 From: davea at davea.name (Dave Angel) Date: Sat, 06 Jul 2013 19:30:04 -0400 Subject: [Tutor] please return flys in ointment In-Reply-To: References: Message-ID: On 07/06/2013 06:34 PM, Dave Angel wrote: > On 07/06/2013 05:38 PM, Jim Mooney wrote: >> I reworked my numbers_to_name program so I can actually follow the logic. >> Since I think the best way to learn is to follow something through to >> completion, I'll put it online so I learn how to do that. But naturally, >> lest I look dumb, I'd appreciate criticism, improvements of bad form, >> or if >> anyone can break it, let me know ;') . Seems to work with a bit of random >> testing, but ya never know. If you did, Msoft wouldn't be putting out >> those >> updates all the time. Works with Py 27 or 33, but that brings up a >> question: I think most of my programs work with both if I don't do >> anything >> exotic, except for input(), which is the fly in the ointment. Instead of >> using the routine below, could I redefine input() to use the routine, >> or it >> that a bad idea? >> >> I have a feeling the leftpad routine could be simplified, but it isn't >> coming to me. >> > > General comments: > > 1) several lines wrapped in the newreader, so I had to paste things > together. Presumably this won't happen whereever you put it online, but > you should make sure and check after uploading it to make sure no such > problems were introduced. > > Also, instead of using the backslash line-continuation character, I > generally try to use the syntax of the language instead. So if you have > a quote which won't fit on one line, but you don't want embedded > newlines in it either, you can use the implicit concatenation of > adjacent literals. Frequently you have to put parens around the whole > thing to make sure the compiler sees it as one line. But I think it's > usually more readable that way. > > First example: the prompt in the input/raw_input lines. > > >> # Using C:\Python33\python.exe on Win 7 in c:\python33\jimprogs - not >> Windows-specific >> # Also works with python 2.7 >> > > You always take the data in by input/raw_input. Why not permit a value > on the command line? And even more important, you're mixing > input/output with algorithm here. > >> import sys >> >> # Data >> ones = {'1': 'one', '2': 'two', '3': 'three', '4': 'four', '5': 'five', >> '6': 'six', >> '7': 'seven', '8': 'eight', '9': 'nine'} >> >> tens = {'2': 'twenty', '3': 'thirty', '4': 'forty', '5': 'fifty', '6': >> 'sixty', >> '7': 'seventy', '8': 'eighty', '9': 'ninety'} >> >> doubles = {'0': 'ten', '1': 'eleven', '2': 'twelve', '3': 'thirteen', >> '4': >> 'fourteen', >> '5': 'fifteen', '6': 'sixteen', '7': 'seventeen', '8': 'eighteen', '9': >> 'nineteen'} >> >> powers_of_1000 = (' thousand', ' million', ' billion', ' trillion', >> ' quadrillion', ' quintillion', ' sextillion', ' septillion', ' >> octillion', >> ' nonillion', ' decillion') >> >> '''add these later, and also option for dollars-and-cents ending. >> 'vigintillion', 'novemdecillion', 'octodecillion', 'septendecillion', >> 'sexdecillion', 'quindecillion', 'quattuordecillion', 'tredecillion', >> 'duodecillion', 'undecillion', >> 'decillion', 'nonillion' > > Those last two you've already implemented > >> ''' >> >> # Functions >> def make_triplets_from_input(): >> '''Enter a positive integer. A list of triplets in the same order >> will >> be returned. >> Raise ValueError to loop on non-integer input, or return 'zero' >> trigger-value of >> 'xxx' if all zeroes are entered. If input is okay, strip leading >> zeroes, then >> zero-pad leftmost triplet to three characters, so all slices are >> triplets. Adjust >> input for different Python versions.''' >> while True: >> try: >> if sys.version[0:3] == '2.7': > > I'd do the version checking in initialization code, not here. And I'd > do it by redefining input to do what it does in 3.3 > >> numbers_str = original = raw_input('Enter a positive >> integer, space \ >> separated if desired.') >> elif sys.version[0:3] == '3.3': >> numbers_str = original = input('Enter a positive >> integer, >> space \ >> separated if desired.') > > Note: you explicitly support zero, so this should say non-negative, > rather than positive. > > I'm not sure why you support spaces between the digits, and not commas, > but no biggie. BTW, you also support other white space, like tabs. > >> else: >> print('Python versions 2.7 and 3.3 only supported') >> sys.exit() >> >> numbers_str = ''.join(numbers_str.split()) >> if numbers_str == '' or numbers_str == ' ': raise ValueError >> numbers_int = int(numbers_str) >> if numbers_int == 0: >> print('Zero') >> sys.exit() > > This is just weird. You're in the middle of an input routine, and you > just print something and exit the code?? > > >> numbers_str = str(numbers_int) >> if len(numbers_str) > 36: raise ArithmeticError >> break >> except KeyboardInterrupt: >> print('Program cancelled by user') >> sys.exit() >> except ValueError as err: >> print(original, "is not an integer.") >> continue >> except ArithmeticError as err: >> print(original, "is too big.\n999...decillion is max: >> 10**37-1 >> or 36 chars \ >> or 12 groups of 3 chars") >> >> leftpad = len(numbers_str) % 3 # zero-pad left, so we get all >> 3-character triplets >> if leftpad == 0: leftpad = '' >> elif leftpad == 2: leftpad = '0' >> else: leftpad = '00' > > Instead of these four lines, how about: > leftpad = "0" * (-i)%3 > >> numbers_str = leftpad + numbers_str >> triplets = [numbers_str[x:x+3] for x in range(0,len(numbers_str),3)] >> return (triplets, len(triplets)) > > Seems a little silly to make this a tuple, when the second value is > trivially figured from the first. You could just have the second > function take a single list argument and take its len() for itself. > > And even then, I'd recommend using a slice when calling it instead of > passing it a number. Just one more bug lurking in the wings. >> >> def numbers_to_name(triplets, triplen): >> '''Create a name from each triplet and append the power of ten''' >> triplen -= 2 >> number_name = '' >> >> for triplet in triplets: >> triplet_name = '' >> first, second, third, last_two = (triplet[0], triplet[1], >> triplet[2], triplet[1:]) >> if triplet == '000': >> triplen -= 1 >> continue >> if first > '0': >> if last_two == '00': # special case - snip extra space >> separator >> triplet_name += ones.get(first) + ' hundred' >> else: >> triplet_name += ones.get(first) + ' hundred ' >> if second == '0': >> if third > '0': >> triplet_name += ones.get(third) >> elif second == '1': >> triplet_name += doubles.get(third) >> elif second > '1': >> triplet_name += tens.get(second) >> if third > '0': >> triplet_name += '-' + ones.get(third) >> number_name += triplet_name >> if triplen > -1: >> number_name += powers_of_1000[triplen] + ', ' >> triplen -= 1 >> if number_name[-2] == ',': >> number_name = number_name[0:-2] # special case - snip extraneous >> ending comma >> return number_name >> >> # Main Program - turn numeric input into number-names >> triplets, triplen = make_triplets_from_input() >> print(numbers_to_name(triplets, triplen)) >> > > If you included the usual if __name__ == test around those two lines, > you'd be able to use this same file as an importable module. > > You should have four functions, not two: > > 1) parse argv > 2) get input from user > 3) turn string into list ( make_triplets_from_input) > 4) turn list into string, ready to print (numbers_to_name) > > and then the top-level code does something like: > > if __name__ == "__main__": > if len(argv) == 2: > raw = parseargv() > elif len(argv) == 1: > raw = getinputfromuser() > else: > someerrormessage > exit > triplets = make_triplets_from_input(raw) > print(numbers_to_name(triplets)) > > I see that I forgot to include the input/raw_input logic. I'd put in your source, right after imports: #No need to check version, just try to use raw_input. If it's not defined, # then input should already work try: input = raw_input except NameError as e: pass #we must be on version 3.3 Once those lines are executed, just pretend you're on 3.3, and use input. In numbers_to_name(), you can start with: if triplets == ["000"]: return "zero" which will take care of the zero case where it belongs, not in the input function. Incidentally, I changed it to lower case to match the rest of the values. BTW, you use the word billion for 10**9. That's good in the US. But in England at least, a billion is 10**12. So you should qualify that your program is intended for US word-numbers. With the refactoring I suggested, I now can test the code with a simple loop: for i in range(100000): triplets = make_triplets_from_input(str(i)) print(i, numbers_to_name(triplets)) I pipe the output into 'less' and I can examine all those values quickly. I could also write a much more exhaustive test, testing various assertions about the results. And of course I can trivially write/generate a test suite with a list of maybe 10,000 values to check, and use that test after every change to check for regressions. More later. -- DaveA From cybervigilante at gmail.com Sun Jul 7 02:46:36 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 6 Jul 2013 17:46:36 -0700 Subject: [Tutor] please return flys in ointment In-Reply-To: References: Message-ID: On 6 July 2013 15:34, Dave Angel wrote: General comments: etc,,, Thanks very much. Very detailed. I'll print that out and make changes ;') I was worried about linewrap. I'll have to cut my big margins down and pay more attention to it. As a start, where I print Zero and exit, it seemed natural since a user entering Zero is, I assume, asking for what they want, and they get Zero, finissimo. But I guess I should be generous and figure they mistyped, then loop for better input. I think I already asked (or maybe not) if more than one exit is a good idea, though. -- Jim "I asked God for a bike, but I know God doesn't work that way. So I stole a bike and asked for forgiveness." ~ Anonymous -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Sun Jul 7 02:52:17 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 6 Jul 2013 17:52:17 -0700 Subject: [Tutor] please return flys in ointment In-Reply-To: References: Message-ID: On 6 July 2013 16:06, Alan Gauld wrote: Probably better to take a chance and say Python versions above 2.7. > Backward compatibility is pretty good in Python land. Oddly, I was just looking at pyparsing, and it mentioned it's good for Python 2.3 and above. But that left me thinking "Do they mean version 3 and above?" since a lot is not ready for version 3. BTW, I understand the basics of regexes, but had some problems and looked into pyparsing. Has anyone used that and does it seem better than regexes, or easier on the noggin? Jim "I asked God for a bike, but I know God doesn't work that way. So I stole a bike and asked for forgiveness." ~ Anonymous -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan.gauld at btinternet.com Sun Jul 7 03:01:04 2013 From: alan.gauld at btinternet.com (ALAN GAULD) Date: Sun, 7 Jul 2013 02:01:04 +0100 (BST) Subject: [Tutor] please return flys in ointment In-Reply-To: References: Message-ID: <1373158864.1949.YahooMailNeo@web186003.mail.ir2.yahoo.com> >Oddly, I was just looking at pyparsing, and it mentioned it's good for Python 2.3? >and above. But that left me thinking "Do they mean version 3 and above?"? >since a lot is not ready for version 3.?Since we know a lot has changed since 2.3 and especially since 2.3 I'd take it with? a pinch of salt and consider it v2 safe. But if it says 2.7 and above it should be v3 safge too since v2.7 allows pretty good? portability between 2 and 3. But thats just me... BTW, I understand the basics of regexes, but had some problems and looked? >into pyparsing. Has anyone used that and does it seem better than regexes,? >or easier on the noggin?When searching for text use simple string searches where possible. If not possible use regexes where the input and/or search string are unstructured. If its structured use a parser for preference. (HTML, XML or pyparser etc for? proprietary type stuff) There are a few things where regexes are the best tool, but for anything? non-trivial (relatively speaking!) there is usually a better way...?IMHO... Alan g. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Sun Jul 7 03:18:40 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 6 Jul 2013 18:18:40 -0700 Subject: [Tutor] please return flys in ointment In-Reply-To: <1373158864.1949.YahooMailNeo@web186003.mail.ir2.yahoo.com> References: <1373158864.1949.YahooMailNeo@web186003.mail.ir2.yahoo.com> Message-ID: On 6 July 2013 18:01, ALAN GAULD wrote: > > > If its structured use a parser for preference. (HTML, XML or pyparser etc > for > proprietary type stuff > Ah, I see - don't bother with a parser unless you have a lot of structure. BTW, I extracted the main function from my program to a test program and it gets an index error after 6,000 or so tests of random integer strings. in case someone finds that, I'm already workin' on it ;') Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Sun Jul 7 03:45:20 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 6 Jul 2013 18:45:20 -0700 Subject: [Tutor] back to text Message-ID: oops, forgot to switch back to text. Maybe that clobbered my line endings. It would be nice if I could go back to gmail's Old option, which just left your style as default. Now it uses the last-used setting. This isn't the first time I've been gangstered by a "new, improved" option that didn't allow you to go back to the old one. I started shopping at a different food store when their site, which at first allowed you to easily print coupon bargains, tried a new overly-complex option that didn't work as well, but provided way to go back to the working option. There's a lesson in there for programmers, somewhere ;') -- Jim "I asked God for a bike, but I know God doesn't work that way. So I stole a bike and asked for forgiveness." ~ Anonymous From steve at pearwood.info Sun Jul 7 04:07:20 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 07 Jul 2013 12:07:20 +1000 Subject: [Tutor] please return flys in ointment In-Reply-To: References: Message-ID: <51D8CD58.1090608@pearwood.info> On 07/07/13 07:38, Jim Mooney wrote: > I reworked my numbers_to_name program so I can actually follow the logic. > Since I think the best way to learn is to follow something through to > completion, I'll put it online so I learn how to do that. But naturally, > lest I look dumb, I'd appreciate criticism, improvements of bad form, or if > anyone can break it, let me know ;') . Let's look at your overall program design: You should always separate *interface* from *functionality*. As it stands now, your make_triplets_from_input() function does both: it interacts with the user, and performs critical functionality. That means you cannot [easily] either test, or use, that critical functionality *except* interactively. And that brings us to the second issue: with respect, your API (Application Programming Interface) sucks. You have two functions, one to get *a string* of digits directly from the user and convert them to triplets, and one to convert triplets into a string. Nowhere do you have a function to convert *numbers* to names, despite the alleged name of your program. (Programmers who write programs that lie about what they do occupy a special circle of Hell, along with those who talk during movies, people listening to music on their mp3-player so loudly that others can hear it through their headphones, and anyone who wears Lynx/Axe body spray.) So I suggest the following public function: number_to_words: takes an integer and converts to words >>> number_to_words(156) 'one hundred and fifty six' plus some sort of function to handle repeatedly asking the user for a number and converting it. Anything else is just support for the number_to_words function. Thirdly, except for the most throw-away "who cares" scripts, you should try to design your programs so that they have a separate main() function which only runs when you explicitly run the program, not when you import it. That way this number_to_words function can be reused in other programs, just by importing it. Lastly, you're annoyingly cautious about version numbers. I haven't studied your code in sufficient detail to be sure, but nothing stands out that would prevent your code from working in at least Python 2.5-3.3 inclusive. And when 3.4-3.9 come out over the next decade or so, I would expect your code to continue working. There's no need to *prohibit other versions* -- it's enough to say "tested in versions 2.7 and 3.3", and then wait for bug reports to come in: "doesn't work in Python 1.5" "doesn't work in Python 2.6, here's how to fix it" sort of thing. You can then say you won't support anything as old as 1.5 (I don't even consider supporting anything below 2.4) and choose whether or not to accept patches. More to follow... -- Steven From davea at davea.name Sun Jul 7 04:09:43 2013 From: davea at davea.name (Dave Angel) Date: Sat, 06 Jul 2013 22:09:43 -0400 Subject: [Tutor] please return flys in ointment In-Reply-To: References: <1373158864.1949.YahooMailNeo@web186003.mail.ir2.yahoo.com> Message-ID: On 07/06/2013 09:18 PM, Jim Mooney wrote: > On 6 July 2013 18:01, ALAN GAULD wrote: >> >> >> If its structured use a parser for preference. (HTML, XML or pyparser etc >> for >> proprietary type stuff >> > > Ah, I see - don't bother with a parser unless you have a lot of structure. > > BTW, I extracted the main function from my program to a test program and it > gets an index error after 6,000 or so tests of random integer strings. in > case someone finds that, I'm already workin' on it ;') > > Jim > > If the exception was on the line: first, second, third, last_two = (triplet[0], triplet[1], triplet[2], triplet[1:]) then perhaps you want to apply this fix to the leftpad logic: def make_triplets_from_input(numbers_str): leftpad = "0" * ((-len(numbers_str))%3) numbers_str = leftpad + numbers_str triplets = [numbers_str[x:x+3] for x in range(0,len(numbers_str),3)] return triplets Testing with the following, for j in xrange(100000000): i = random.randint(0, 10**12) triplets = make_triplets_from_input(str(i)) print(j, i, numbers_to_name(triplets)) After some 800,000 values, I killed it, and put a conditional print instead, printing every ten-thousandth value. (It still calls numbers_to_name(), obviously.) It has done the full hundred million random values without crashing. If you'd like, I'll post my full version, changed as little as possible from yours, but incorporating the suggestions I've tried to make here. In another message, you had the following: > As a start, where I print Zero and exit, it seemed natural since > a user entering Zero is, I assume, asking for what they want, > and they get Zero, finissimo. But I guess I should be generous > and figure they mistyped, then loop for better input. > I think I already asked (or maybe not) if more than > one exit is a good idea, though. You miss my point. The get_raw_input() function should permit a value of "0" but it should NOT just print a value and abort the program. It breaks all reasonable factoring. Conversely, the numbers_to_name() function SHOULD take a value of zero (which will actually be ["000"] ) and at that point, it's fine to special case. Not by printing "Zero", but by returning "zero". -- DaveA From cybervigilante at gmail.com Sun Jul 7 04:40:15 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 6 Jul 2013 19:40:15 -0700 Subject: [Tutor] please return flys in ointment In-Reply-To: References: <1373158864.1949.YahooMailNeo@web186003.mail.ir2.yahoo.com> Message-ID: On 6 July 2013 19:09, Dave Angel wrote: > > If the exception was on the line: Actually, it was on the line: if number_name[-2] == ',': number_name = number_name[0:-2] # special case - snip extraneous ending comma And only intermittently. It probably occurred in a huge, oddly-formed number, larger than your test, since I'm using randint to give random-numbers of random size for testing. I did that since actually iterating up to the decillions is a bit beyond my modest computer ;') > If you'd like, I'll post my full version, changed as little as possible from That would be helpful. The corrections are already four times the size of the program, and at least three pots of coffee ;') Since I didn't make it a full function yet, just user-input, I abstracted the main func into the test program below, which finds the error between 5,000 and 9,000 tests. The offending line was an easy cheat but didn't work, alas ;') Abstracting it was annoying so I can see where it would be better to make it all into a command line function. And maybe a class later, since I want to add to it a bit. import sys from random import randint, seed seed() # Data ones = {'1': 'one', '2': 'two', '3': 'three', '4': 'four', '5': 'five', '6': 'six', '7': 'seven', '8': 'eight', '9': 'nine'} tens = {'2': 'twenty', '3': 'thirty', '4': 'forty', '5': 'fifty', '6': 'sixty', '7': 'seventy', '8': 'eighty', '9': 'ninety'} doubles = {'0': 'ten', '1': 'eleven', '2': 'twelve', '3': 'thirteen', '4': 'fourteen', '5': 'fifteen', '6': 'sixteen', '7': 'seventeen', '8': 'eighteen', '9': 'nineteen'} powers_of_1000 = (' thousand', ' million', ' billion', ' trillion', ' quadrillion', ' quintillion', ' sextillion', ' septillion', ' octillion', ' nonillion', ' decillion') def numbers_to_name(triplets, triplen): '''Create a name from each triplet and append the power of ten''' triplen -= 2 number_name = '' for triplet in triplets: triplet_name = '' first, second, third, last_two = (triplet[0], triplet[1], triplet[2], triplet[1:]) if triplet == '000': triplen -= 1 continue if first > '0': if last_two == '00': # special case - snip extra space separator triplet_name += ones.get(first) + ' hundred' else: triplet_name += ones.get(first) + ' hundred ' if second == '0': if third > '0': triplet_name += ones.get(third) elif second == '1': triplet_name += doubles.get(third) elif second > '1': triplet_name += tens.get(second) if third > '0': triplet_name += '-' + ones.get(third) number_name += triplet_name if triplen > -1: number_name += powers_of_1000[triplen] + ', ' triplen -= 1 if number_name[-2] == ',': number_name = number_name[0:-2] # special case - snip extraneous ending comma return number_name def testfunc(): triplets = [] length_of_triplets = randint(1,11) for cnt in range(0, length_of_triplets): triplet = '' triplet += str(randint(0,9)) triplet += str(randint(0,9)) triplet += str(randint(0,9)) triplets.append(triplet) return triplets, length_of_triplets with open('triptest.txt', 'w') as triptest: for x in range(9000): triplets, triplen = testfunc() number_name = numbers_to_name(triplets, triplen) printout = str(triplets) + '\n' + number_name + '\n\n' triptest.write(printout) -- Jim "I asked God for a bike, but I know God doesn't work that way. So I stole a bike and asked for forgiveness." ~ Anonymous From davea at davea.name Sun Jul 7 07:30:39 2013 From: davea at davea.name (Dave Angel) Date: Sun, 07 Jul 2013 01:30:39 -0400 Subject: [Tutor] please return flys in ointment In-Reply-To: References: <1373158864.1949.YahooMailNeo@web186003.mail.ir2.yahoo.com> Message-ID: On 07/06/2013 10:40 PM, Jim Mooney wrote: > On 6 July 2013 19:09, Dave Angel wrote: >> > >> If you'd like, I'll post my full version, changed as little as possible from > > That would be helpful. The corrections are already four times the size > of the program, and at least three pots of coffee ;') > I hope this is useful. There are lots more places where I might clean things up, but I wanted to make it as close to yours as possible, so you could reasonably diff the two. # Using C:\Python33\python.exe on Win 7 in c:\python33\jimprogs - not #Windows-specific # Also works with python 2.7 import sys import random #No need to check version, just try to use raw_input. If it's not defined, # then input should already work try: input = raw_input except NameError as e: pass #we must be on version 3.0 or larger try: range = xrange except NameError as e: pass #we must be on version 3.0 or larger # Data ones = {'1': 'one', '2': 'two', '3': 'three', '4': 'four', '5': 'five', '6': 'six', '7': 'seven', '8': 'eight', '9': 'nine'} tens = {'2': 'twenty', '3': 'thirty', '4': 'forty', '5': 'fifty', '6': 'sixty', '7': 'seventy', '8': 'eighty', '9': 'ninety'} doubles = {'0': 'ten', '1': 'eleven', '2': 'twelve', '3': 'thirteen', '4': 'fourteen', '5': 'fifteen', '6': 'sixteen', '7': 'seventeen', '8': 'eighteen', '9': 'nineteen'} powers_of_1000 = (' thousand', ' million', ' billion', ' trillion', ' quadrillion', ' quintillion', ' sextillion', ' septillion', ' octillion', ' nonillion', ' decillion') '''add these later, and also option for dollars-and-cents ending. 'vigintillion', 'novemdecillion', 'octodecillion', 'septendecillion', 'sexdecillion', 'quindecillion', 'quattuordecillion', 'tredecillion', 'duodecillion', 'undecillion', 'decillion', 'nonillion' ''' # Functions def get_raw_input(): '''Enter a positive integer. A list of triplets in the same order will be returned. Raise ValueError to loop on non-integer input, or return 'zero' trigger-value of 'xxx' if all zeroes are entered. If input is okay, strip leading zeroes, then zero-pad leftmost triplet to three characters, so all slices are triplets. Adjust input for different Python versions.''' while True: try: if sys.version[0:3] == '2.7': numbers_str = original = raw_input('Enter a positive integer, space \ separated if desired.') elif sys.version[0:3] == '3.3': numbers_str = original = input('Enter a positive integer, space \ separated if desired.') else: print('Python versions 2.7 and 3.3 only supported') sys.exit() numbers_str = ''.join(numbers_str.split()) if numbers_str == '' or numbers_str == ' ': raise ValueError #NO POINT in special-casing zero. It's a perfectly valid value. #numbers_int = int(numbers_str) #if numbers_int == 0: # print('Zero') # sys.exit() numbers_str = str(int(numbers_str)) #normalize it if len(numbers_str) > 36: raise ArithmeticError break except KeyboardInterrupt: print('Program cancelled by user') sys.exit() except ValueError as err: print(original, "is not an integer.") continue except ArithmeticError as err: print(original, "is too big.\n999...decillion is max: 10**37-1 or 36 chars \ or 12 groups of 3 chars") return numbers_str def make_triplets_from_input(numbers_str): leftpad = "0" * ((-len(numbers_str))%3) numbers_str = leftpad + numbers_str #print "numbers_str:", numbers_str triplets = [numbers_str[x:x+3] for x in range(0,len(numbers_str),3)] #print "triplets are", triplets return triplets def numbers_to_name(triplets): '''Create a name from each triplet and append the power of ten''' if triplets == ["000"]: return "zero" triplen = len(triplets) - 2 number_name = '' #print "triplets are", triplets for triplet in triplets: #print "triplet is", triplet triplet_name = '' first, second, third, last_two = (triplet[0], triplet[1], triplet[2], triplet[1:]) if triplet == '000': triplen -= 1 continue if first > '0': if last_two == '00': # special case - snip extra space separator triplet_name += ones.get(first) + ' hundred' else: triplet_name += ones.get(first) + ' hundred ' if second == '0': if third > '0': triplet_name += ones.get(third) elif second == '1': triplet_name += doubles.get(third) elif second > '1': triplet_name += tens.get(second) if third > '0': triplet_name += '-' + ones.get(third) number_name += triplet_name if triplen > -1: number_name += powers_of_1000[triplen] + ', ' triplen -= 1 if number_name[-2] == ',': number_name = number_name[0:-2] # special case - snip extraneous ending comma return number_name def int_name(i): triplets = make_triplets_from_input(str(i)) return numbers_to_name(triplets) # Main Program - turn numeric input into number-names if __name__ == '__main__': if False: raw = get_raw_input() triplets = make_triplets_from_input(raw) print(numbers_to_name(triplets)) from vegaseat import int2word def compare_em(i): a = int_name(i).replace("-", " ").replace(",", "") b = int2word(i).strip() if b == "": b = "zero" if a !=b: print(i, a, b) input(" press Enter to continue ") for i in range(200001): compare_em(i) #The following shows a place where the vegaseat code # starts getting buggy. In particular, it doesn't handle # numbers bigger than 999octillion or so. #Note, if we make a range starting at 10**30, # we get OverflowError, so we fake it start = 10**30 - 2 for cntr in range(5): i = start + cntr compare_em(i) print "now starting random tests" for j in range(1000000): exponent = random.randint(4, 30) #It appears that vegaseat is buggy beyond 999octillion i = random.randint(0, 10**exponent-1) compare_em(i) -- DaveA From davea at davea.name Sun Jul 7 07:52:58 2013 From: davea at davea.name (Dave Angel) Date: Sun, 07 Jul 2013 01:52:58 -0400 Subject: [Tutor] please return flys in ointment In-Reply-To: References: <1373158864.1949.YahooMailNeo@web186003.mail.ir2.yahoo.com> Message-ID: On 07/07/2013 01:30 AM, Dave Angel wrote: > On 07/06/2013 10:40 PM, Jim Mooney wrote: >> On 6 July 2013 19:09, Dave Angel wrote: >>> > >> >>> If you'd like, I'll post my full version, changed as little as >>> possible from >> >> That would be helpful. The corrections are already four times the size >> of the program, and at least three pots of coffee ;') >> > > I hope this is useful. There are lots more places where I might clean > things up, but I wanted to make it as close to yours as possible, so you > could reasonably diff the two. > Sorry to do this in two parts, but there are a few things I know I added to that code since the last discussion online. (There are undoubtedly others, which is why I suggest you do a diff with your own code, and study the reasons for each difference) As Steve suggested, I added a function int_name() which does what Steven suggested. It encapsulates your entire algorithm, without confusing with input or output schemes. It also has the same interface as the int2word() function I found elsewhere. Main thing is that I found another implementation which I can compare yours to. I didn't copy that code here, because I don't want to contaminate yours with scraps of his. And I didn't read it either, except enough to tell that he has a single useful function that takes an int (long) and returns a string representing the word version of the number. If you look at compare_em(), you'll see that I had to distort both your function and the the int2word() function to make them mostly match up. The other author does not use hyphens or commas in the words. He ends each string with a blank. And he neglected to handle zero. He also is buggy beyond a nonillion-1. So I used this to test your code within those parameters. It won't catch a problem with a very high number, nor one with misplaced or extra commas or dashes. But otherwise, it looks like you two have come very close. I suspect with a little more searching, I could find a different implementation that's closer to yours. BTW, this approach of having two implementations and comparing them is very useful, even if there isn't one out on the net somewhere. When I was microcoding add, subtract, multiply, ... trig, log, ... I also wrote a very crude high precision math package. The slow one was done in a high level language, and very brute force. Then I systematically pitted the two sets of functions against each other, looking for edge cases and bugs. Some things, like the trig functions, I couldn't expect to be precisely the same. (I used things like taylor series for the brute force ones, but with intermediate results carried out to many more places than the accuracy I was actually comparing.) That was in about 1975. -- DaveA From cybervigilante at gmail.com Sun Jul 7 08:47:03 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 6 Jul 2013 23:47:03 -0700 Subject: [Tutor] please return flys in ointment In-Reply-To: References: <1373158864.1949.YahooMailNeo@web186003.mail.ir2.yahoo.com> Message-ID: On 6 July 2013 22:52, Dave Angel wrote: The > other author does not use hyphens or commas in the words. He ends each > string with a blank. And he neglected to handle zero. I did that at first, but I figured I should do it up brown and come close to human notation, although commas and hyphens made it harder, and a comma introduced my first bug. I think I'm learning more from this "trivial" program than from the book I'm studying. Getting it totally right is more difficult that I thought it would be. And of course, Zero is the exception to end all exceptions, so it should always be kept in mind ;') But this and other advice will keep me busy for awhile. Especially since Arizona is going through a heat wave my crummy AC can't handle, even at night, so my computer time is cut way down, especially if my temp monitor screams. I can stand the heat but I have learned from sad experience that a laptop CPU, with their rotten air circulation, cannot. And those overpriced little "laptop fans" are a joke. The big tip about laptops in hot weather is - Don't Watch Videos! I can watch the temp monitor climb the minute I go to YouTube. I'm well along with this, but a thought that occurred to me after I realized our number-naming is far from rational - we're just used to it - was to name the numbers rationally, then at the end translate them with a regex. i.e. the teens are: oneteen, twoteen, threeteen, the tens are onety, twoty, threety, the hundreds are onedred, twodred, thousands are onesand, twosand, etc. illion is all the same ending, so use the prefix - onemill, onebill, onetrill, onequad. No hyphens - so fifty-six is fiftysix. This is how numbers oughtta be, anyway - and Zero is Forbidden ;') That would make it all very simple - then translate it to human speech with a monstrous regex. Probably the same amount of work in the end, though. > He also is buggy beyond a nonillion-1. Interesting. I did a linear test first, but never got to high numbers. It was only with a test that was random, so it got a much higher number now and then, that I got a bug. The downside is sometimes it succeeded with the same number of runs that it failed on, since it's random, so multiple runs were required. Useful to remember, though, since I think it's natural to grab for a linear test and figure random tests won't catch everything. But a shotgun sometimes hits more than a rifle. -- Jim There is evidence the North Polar icecap will disappear sooner than expected, due to unforeseen chaotic events and multiple positive feedback loops. Theorists on both sides can debate it, but Mother Nature will have the last word. From davea at davea.name Sun Jul 7 09:05:51 2013 From: davea at davea.name (Dave Angel) Date: Sun, 07 Jul 2013 03:05:51 -0400 Subject: [Tutor] please return flys in ointment In-Reply-To: References: <1373158864.1949.YahooMailNeo@web186003.mail.ir2.yahoo.com> Message-ID: On 07/07/2013 02:47 AM, Jim Mooney wrote: > On 6 July 2013 22:52, Dave Angel wrote: > The >> other author does not use hyphens or commas in the words. He ends each >> string with a blank. And he neglected to handle zero. > >> He also is buggy beyond a nonillion-1. I found yet another implementation that does work up to at least a decillion, and it agrees with yours throughout (actually, the testing was done with the same million random probes as what I posted earlier.) However, he also omits the commas and the hyphens. An interesting site is: http://rosettacode.org/wiki/Names_to_numbers which has a bunch of different languages (and programmers) implementing the same concept. Note that the spec didn't say how the words should be laid out, and there are many variations in result between the various implementations. These include not only the hyphens, commas, but also the use of the word 'and' between some parts of the numbers. -- DaveA From cybervigilante at gmail.com Sun Jul 7 09:44:41 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sun, 7 Jul 2013 00:44:41 -0700 Subject: [Tutor] please return flys in ointment In-Reply-To: References: <1373158864.1949.YahooMailNeo@web186003.mail.ir2.yahoo.com> Message-ID: On 7 July 2013 00:05, Dave Angel wrote: These include not only the hyphens, commas, but also the > use of the word 'and' between some parts of the numbers. http://rosettacode.org/wiki/Names_to_numbers Interesting link - but I draw the line at 'and' ;') Actually, I thought about it, but it seemed to me that when most people are verbalizing big numbers, they tend not to use 'and.' They'll say "one hundred twenty-two thousand, four hundred sixty-four," not "one hundred and twenty-two thousand, four hundred and sixty-four." But that might be an Americanism. Perhaps in Eton they always include 'and'. The only use for this, besides doing it of course, is for speakers. Trying to frighten voters with the national debt if you're reading a long numeral from a teleprompter could end in disaster. It's much easier to read the words. And of course, if you want to Carl Sagan an audience by reciting the miles to the Andromeda Galaxy, it would be easier on the eyes. Oh, text-to-speech programs. I seem to recall Adobe reading a long number as just the numerals, but that was some time ago. Maybe they're doing better now. Jim "We sit perched on the knife edge of a change of ages. On the one hand it is an amazing opportunity. On the other it is scary as hell. Despite everything man has known in all his time since we parted from our brothers the apes, we have never experienced anything like this." --Espen Olsen From alan.gauld at btinternet.com Sun Jul 7 10:22:21 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Sun, 07 Jul 2013 09:22:21 +0100 Subject: [Tutor] please return flys in ointment In-Reply-To: References: <1373158864.1949.YahooMailNeo@web186003.mail.ir2.yahoo.com> Message-ID: On 07/07/13 08:44, Jim Mooney wrote: > thought about it, but it seemed to me that when most people are > verbalizing big numbers, they tend not to use 'and.' True in the USA but not the UK (and derivatives?). Here most folks seem to put the 'and' in. eg: three thousand, four hundred and fifty six... Incidentally, if you get any Indian users they'll be looking for support for 'laks' - 100,000's :-) HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From cybervigilante at gmail.com Sun Jul 7 18:35:38 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sun, 7 Jul 2013 09:35:38 -0700 Subject: [Tutor] please return flys in ointment In-Reply-To: References: <1373158864.1949.YahooMailNeo@web186003.mail.ir2.yahoo.com> Message-ID: On 7 July 2013 01:22, Alan Gauld wrote: > > Incidentally, if you get any Indian users they'll be looking > for support for 'laks' - 100,000's :-) > I think I'll be Jingoistic for now, or let Subrahim fork it. I haven't even gotten though the Lutz book yet because I stop to fool around like this when I get bored with short book problems. With more corrections than program, and computer time limited by an AZ heatwave, this is enough for now. But frankly, it's a better learning experience than the book. Well, different, anyway - the book is good, too, but book problems aren't as big a headache. I don't wake up thinking, "I'll get back to that book problem!" I wake up thinking, "Got to fix that damn numbers to name program!" -- Jim "We sit perched on the knife edge of a change of ages. On the one hand it is an amazing opportunity. On the other it is scary as hell. Despite everything man has known in all his time since we parted from our brothers the apes, we have never experienced anything like this." --Espen Olsen From fomcl at yahoo.com Sun Jul 7 20:37:13 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Sun, 7 Jul 2013 11:37:13 -0700 (PDT) Subject: [Tutor] custom error classes: how many of them does one need? In-Reply-To: <51D62D57.7060400@pearwood.info> References: <1372966854.32001.YahooMailNeo@web163805.mail.gq1.yahoo.com> <51D62D57.7060400@pearwood.info> Message-ID: <1373222233.14707.YahooMailNeo@web163805.mail.gq1.yahoo.com> ----- Original Message ----- > From: Steven D'Aprano > To: tutor at python.org > Cc: > Sent: Friday, July 5, 2013 4:20 AM > Subject: Re: [Tutor] custom error classes: how many of them does one need? > > On 05/07/13 05:40, Albert-Jan Roskam wrote: >> Hello, >> >> I created a custom error class like under [A] below (see > http://pastebin.com/JLwyPsRL if you want to see highlighted code). Now I am > thinking: is this specific enough? "When >> creating a module that can raise several distinct errors, a common practice > is >> to create a base class for exceptions defined by that module, and subclass > that >> to create specific exception classes for different error conditions" > (http://docs.python.org/2/tutorial/errors.html) >> Given that I have many retcodes, is it a better approach to define a class > factory that generates a custom exception for each retcode on-the-fly? (under > [B]). Using a class factory seems a little unusual, or isn it? More generally: > what criteria should one follow when distinguishing error classes? > > Return codes are not exceptions, and generally speaking using return codes for > errors is *strongly* discouraged unless you have a language without exceptions. > (Although the designers of Go language disagree. I think they are wrong to do > so.) Although looking forward, I see that even though you call them > "retcodes", you don't actually use them as return codes. Hi Steven, Alan, Eryksun, In my app, the return codes are from C functions. < 0 means warning, 0 means OK, > 0 means error. You're right that I was sloppy as I used the terms retcodes and exceptions interchangably. > >> global retcodes >> retcodes = {0: "OK", 1: "Error_X", 2: > "Error_Y"} > > You don't need to declare a global variable global in the global part of > your file. Although Python allows it, it is redundant and rather silly. Delete > the line "global retcodes". Oops, despite of an earlier thread I still did this wrong. I used to think that "global" made The Unforgiveable Thing Of All Times (using a global variable) less bad because at least it was explicitly "declared". Silly me. ? >> ############################# >> # [A] one custom exception for all purposes, but with a descriptive message >> ############################# >> >> class single_custom_exception(Exception): >> ? ? ? def __init__(self, retcode, message): >> ? ? ? ? ? self.retcode = retcode >> ? ? ? ? ? message += " [%s]" % retcodes.get(retcode) >> ? ? ? ? ? Exception.__init__(self, message) > > Exceptions are classes, and like all classes, the convention is to name them in > CamelCase. > > >> ############################# >> # [B] generating custom exceptions with a class factory >> ############################# >> >> class BaseError(Exception): pass >> >> def throw(retcode, message): >> ? ? ? """class factory that generates an error class for >> ? ? ? each return code label""" >> ? ? ? class custom_exception(BaseError): >> ? ? ? ? ? def __init__(self): >> ? ? ? ? ? ? ? BaseError.__init__(self, message) >> ? ? ? exception = retcodes.get(retcode) >> ? ? ? custom_exception.__name__ = exception >> ? ? ? return custom_exception > > This is overkill. No need to create a custom class *on the fly* for each return > code. Doing so is pointless: the caller *cannot* catch them individually, > because they are created on the fly. So they can only catch BaseError, then > inspect the exception's details. If you have to do that, then having > individual classes is a waste of time. You might as well just use a single class > and be done with it. > > What you would like to do, but cannot, since neither Error_X nor Error_Y are > defined: > > try: > ? ? code_that_might_fail() > except Error_X: > ? ? handle_error_X > except Error_Y: > ? ? handle_error_Y > > > > What you are forced to do: > > try: > ? ? code_that_might_fail() > except BaseError as err: > ? ? if err.retcode == 0: > ? ? ? ? print ("Error: no error occurred!") > ? ? elif err.retcode == 1: > ? ? ? ? handle_error_X > ? ? elif err.retcode == 1: > ? ? ? ? handle_error_Y > ? ? else: > ? ? ? ? print ("Error: an unexpected error occurred!") > > > which is ugly enough, but it makes the class factory redundant. It might be ugly, but I think I can live with it. For now, the single custom error works. I was just trying to be critical towards my own code. And it was fun to write a factory class, of course. ;-) > So, how should you handle this scenario? > > Use a separate exception class for each high-level *category* of errors. E.g. > Python has: > > TypeError > ValueError > ZeroDivisionError > UnicodeEncodeError > > etc. Within your app, can you divide the return codes into application-specific > groups? If not, then you just need a single exception class: I tried doing that. Semantically (I mean just by the looks of the messages), I can categorize them. Then I end up with about 15 exceptions. After having done that, I started to be convinced not to follow this route. It might be nice to be able to choose whether warnings (retcodes < 0) should be treated as errors or not (I believe in options() of CRAN R this is called warningsAsErrors). Would the code below be a good implementation (I also experimented with cmp)? # this is a global variable (perhaps in __init__.py) # if warningsAsErrors > 0, even warnings will? raise an error. import operator warningsAsErrors = bool(os.getenv("warningsAsErrors")) warningsAsErrors = operator.ne if warningsAsErrors else operator.gt # this comes after each C function call retcode = someCfunc(... blah...) if retcode and oper(retcode, 0): ??? raise MyCustomError > MyApplicationError > > > If you do have a bunch of application-specific groups, then you might develop a > hierarchical family of exceptions: > > > WidgetError > +-- WrongWidgetTypeError > +-- InvalidWidgetError > +-- WidgetNotInitialisedError > > > WidgetGroupError > +-- MissingWidgetError > |? +-- MissingCriticalWidgetError > +-- TooManyWidgetsError > > > SpoonError > +-- TeaspoonError > +-- SoupSpoonError > +-- DesertSpoonError > DesertSpoonError? LOL! ;-) > The parent exceptions, WidgetError, WidgetGroupError and SpoonError do not need > to share a common base class. > > This hierarchy might completely obliterate the need for return codes. > Python's build-in exceptions generally do not. However, maybe you do still > need them, similar to the way Python's OSError and IOError have error codes. > Assuming you do: > > > def someFunc(): > ? ? perform_calculations() > ? ? if there are no spoons: > ? ? ? ? raise TeaspoonError("cannot stir my tea") > ? ? elif some other error: > ? ? ? ? raise InvalidWidgetError(retcode=99, message="Can't use red > widget") > ? ? else: > ? ? ? ? return some value > > > > -- > Steven > _______________________________________________ > Tutor maillist? -? Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > From jacklittlemc at yahoo.com Mon Jul 8 00:40:48 2013 From: jacklittlemc at yahoo.com (Jack Little) Date: Sun, 7 Jul 2013 15:40:48 -0700 (PDT) Subject: [Tutor] Combat Error Message-ID: <1373236848.96584.YahooMailNeo@web124502.mail.ne1.yahoo.com> When the player of my game fire his/her cannon, it is supposed to get rid of some of the enemy's health. It does not. I have no clue what the error is, so I need help. Here is the Python Shell output: >>fire cannon 1 Fired! Enemy health= 75 They fired at you! Your health is now 55 >>fire cannon 2 Fired! Enemy health= 75 They fired at you! Your health is now 55 >>fire cannon 1 Fired! Enemy health= 75 They fired at you! Your health is now 55 >> Here is the code: def lvl4combat(): ? ? print "The barracks are constructed of wood and cobblestone." ? ? print "A large, single cannon sits on the top, searching the sky for enemy airships." ? ? print "The cannon is manned by a single team of three men." ? ? print "The cannon is large and industrial." ? ? print "Remember your training." ? ? cannonhealth=75 ? ? cannondamage=random.choice([10,17,13]) ? ? currhealth=health ? ? while cannonhealth > 0: ? ? ? ? print "They fired at you!" ? ? ? ? currhealth-cannonhealth ? ? ? ? print "Your health is now",currhealth ? ? ? ? combat=raw_input(">>") ? ? ? ? if combat.lower()=="fire cannon 1": ? ? ? ? ? ? print "Fired!" ? ? ? ? ? ? cannonhealth-attack ? ? ? ? ? ? print "Enemy health=",cannonhealth ? ? ? ? elif combat.lower()=="fire cannon 2": ? ? ? ? ? ? print "Fired!" ? ? ? ? ? ? cannonhealth-attack ? ? ? ? ? ? print "Enemy health=",cannonhealth ? ? ? ? if currhealth <= 0: ? ? ? ? ? ? break ? ? ? ? ? ? defeat1() ? ? ? ? if cannonhealth <= 0: ? ? ? ? ? ? break ? ? ? ? ? ? victory1() ? ? ? ? ? ? if kboost==True: ? ? ? ? ? ? ? ? karma+25*.25 ? ? ? ? ? ? elif kboost==False: ? ? ? ? ? ? ? ? karma+25 -------------- next part -------------- An HTML attachment was scrubbed... URL: From davea at davea.name Mon Jul 8 01:03:33 2013 From: davea at davea.name (Dave Angel) Date: Sun, 07 Jul 2013 19:03:33 -0400 Subject: [Tutor] Combat Error In-Reply-To: <1373236848.96584.YahooMailNeo@web124502.mail.ne1.yahoo.com> References: <1373236848.96584.YahooMailNeo@web124502.mail.ne1.yahoo.com> Message-ID: On 07/07/2013 06:40 PM, Jack Little wrote: > When the player of my game fire his/her cannon, it is supposed to get rid of some of the enemy's health. It does not. I have no clue what the error is, so I need help. Here is the Python Shell output: > >>> fire cannon 1 > Fired! > Enemy health= 75 > They fired at you! > Your health is now 55 >>> fire cannon 2 > Fired! > Enemy health= 75 > They fired at you! > Your health is now 55 >>> fire cannon 1 > Fired! > Enemy health= 75 > They fired at you! > Your health is now 55 >>> > > > > > Here is the code: > > def lvl4combat(): > print "The barracks are constructed of wood and cobblestone." > print "A large, single cannon sits on the top, searching the sky for enemy airships." > print "The cannon is manned by a single team of three men." > print "The cannon is large and industrial." > print "Remember your training." > cannonhealth=75 > cannondamage=random.choice([10,17,13]) > currhealth=health > while cannonhealth > 0: > print "They fired at you!" > currhealth-cannonhealth > print "Your health is now",currhealth > combat=raw_input(">>") > if combat.lower()=="fire cannon 1": > print "Fired!" > cannonhealth-attack This expression has no useful effect. You probably wanted cannonhealth -= attack > print "Enemy health=",cannonhealth > elif combat.lower()=="fire cannon 2": > print "Fired!" > cannonhealth-attack ditto > print "Enemy health=",cannonhealth > if currhealth <= 0: > break > defeat1() > if cannonhealth <= 0: > break > victory1() > if kboost==True: > karma+25*.25 ditto > elif kboost==False: > karma+25 ditto You also have code following break statements that will never execute. In particular, you'll never call defeat1() or victory1() -- DaveA From steve at pearwood.info Mon Jul 8 01:20:50 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 08 Jul 2013 09:20:50 +1000 Subject: [Tutor] Combat Error In-Reply-To: <1373236848.96584.YahooMailNeo@web124502.mail.ne1.yahoo.com> References: <1373236848.96584.YahooMailNeo@web124502.mail.ne1.yahoo.com> Message-ID: <51D9F7D2.2070308@pearwood.info> On 08/07/13 08:40, Jack Little wrote: > def lvl4combat(): > print "The barracks are constructed of wood and cobblestone." > print "A large, single cannon sits on the top, searching the sky for enemy airships." > print "The cannon is manned by a single team of three men." > print "The cannon is large and industrial." > print "Remember your training." > cannonhealth=75 Here you start with cannonhealth set to 75. But you never lower the score, so it never changes and the loop doesn't end. > cannondamage=random.choice([10,17,13]) > currhealth=health > while cannonhealth > 0: > print "They fired at you!" > currhealth-cannonhealth Here you calculate the number (currhealth - cannonhealth), which is the difference between your health and the cannon's health, but do nothing with the result. Which is probably a good thing, since the calculation doesn't mean anything. Perhaps you meant this? currhealth = currenthealth - cannondamage or even this, which is another way of doing the same calculation: currhealth -= cannondamage > print "Your health is now",currhealth > combat=raw_input(">>") > if combat.lower()=="fire cannon 1": > print "Fired!" > cannonhealth-attack Here you calculate the number (cannonhealth - attack), which is the difference between the cannon's health and the damage done, but do nothing with the result. As I suggested the last time you made this same error, you might find it easier to see the problem if you put spaces around operators, so that they are a little easier to notice and read: cannonhealth - attack cannonhealth-attack Like the above, you probably want: cannonhealth = cannonhealth - attack And you make a similar mistake in at least three other places. I will leave you to find them. -- Steven From paradox at pobox.com Mon Jul 8 01:21:11 2013 From: paradox at pobox.com (Paradox) Date: Mon, 08 Jul 2013 07:21:11 +0800 Subject: [Tutor] Combat Error :p: In-Reply-To: <1373236848.96584.YahooMailNeo@web124502.mail.ne1.yahoo.com> References: <1373236848.96584.YahooMailNeo@web124502.mail.ne1.yahoo.com> Message-ID: <51D9F7E7.4020301@pobox.com> On 07/08/2013 06:40 AM, Jack Little wrote: > When the player of my game fire his/her cannon, it is supposed to get > rid of some of the enemy's health. It does not. I have no clue what > the error is, so I need help. Here is the Python Shell output: > > >>fire cannon 1 > Fired! > Enemy health= 75 > They fired at you! > Your health is now 55 Sounds like a fun game (I am a sucker for cannon fire). > > Here is the code: > > > cannonhealth=75 > cannondamage=random.choice([10,17,13]) > currhealth=health > while cannonhealth > 0: > print "They fired at you!" > currhealth-cannonhealth > print "Your health is now",currhealth > combat=raw_input(">>") > if combat.lower()=="fire cannon 1": > print "Fired!" > cannonhealth-attack > print "Enemy health=",cannonhealth Where is "attack" defined? It isn't in the loop you provided that I can see. It almost seems like you want to make a random choice of damage from a list and then subtract that damage from the health of the cannon (don't understand that, why would the cannon do damage to itself?). Maybe you would want to think about renaming some of your variables and decide if "cannondamage" and "attack" are supposed to be the same thing. An easy way to debug these sorts of problems yourself is to add print statements. For instance printing out the value of "attack" just before the line where you print the "Enemy health" may be enlightening. thomas From alan.gauld at btinternet.com Mon Jul 8 01:42:10 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Mon, 08 Jul 2013 00:42:10 +0100 Subject: [Tutor] Combat Error In-Reply-To: <1373236848.96584.YahooMailNeo@web124502.mail.ne1.yahoo.com> References: <1373236848.96584.YahooMailNeo@web124502.mail.ne1.yahoo.com> Message-ID: On 07/07/13 23:40, Jack Little wrote: > When the player of my game fire his/her cannon, it is supposed to get > rid of some of the enemy's health. It does not. I have no clue what the > error is, Its not entirely clear to me who is 'you' and who is 'the enemy' Why does the enemy firing a cannon at me affect the enemy's health? I see how it might affect my health but not the enemies (it might affect their ammunition supplies but that's another story!) > while cannonhealth > 0: > if combat.lower()=="fire cannon 1": > print "Fired!" > cannonhealth-attack I assume these lines are supposed to modify cannonhealth? In fact they do nothing since you don't assign the result. You probably want > cannonhealth -= attack > print "Enemy health=",cannonhealth > elif combat.lower()=="fire cannon 2": > print "Fired!" > cannonhealth-attack > print "Enemy health=",cannonhealth > if currhealth <= 0: > break > defeat1() You do realize that code after a break will never be executed? The break immediately leaves the loop, it does not execute any further statements. > if cannonhealth <= 0: > break same here. All following code is ignored. > victory1() > if kboost==True: > karma+25*.25 I doubt this does what you think it does. You should really get used to using the >>> prompt to try out things... >>> 10+25*.25 16.25 >>> (10+25)*.25 8.75 >>> 10+(25*.25) 16.25 >>> But since you don't assign the result to anything it's pretty harmless in that it has no impact on the code whatsoever except to waste some time. > elif kboost==False: > karma+25 Same here. No assignment means no change. To be honest, it kind of feels like you don't have any design for this thing. You are just making it up as you go along. While evolution is an amazing mechanism for random change it has the downside of taking a very long time to produce useful results. If you plan to progress quickly, design wins every time! Paper and pencils are still effective tools. HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From dfjennings at gmail.com Mon Jul 8 01:15:42 2013 From: dfjennings at gmail.com (Don Jennings) Date: Sun, 7 Jul 2013 19:15:42 -0400 Subject: [Tutor] Combat Error In-Reply-To: <1373236848.96584.YahooMailNeo@web124502.mail.ne1.yahoo.com> References: <1373236848.96584.YahooMailNeo@web124502.mail.ne1.yahoo.com> Message-ID: <5E80FD4B-1E74-427F-883B-04D6D22B1D97@gmail.com> On Jul 7, 2013, at 6:40 PM, Jack Little wrote: > When the player of my game fire his/her cannon, it is supposed to get rid of some of the enemy's health. It does not. I have no clue what the error is, so I need help. > if combat.lower()=="fire cannon 1": > print "Fired!" > cannonhealth-attack > print "Enemy health=",cannonhealth Let's see if I can help you ask a better question. As you can see from your output, the name "cannonhealth" refers to an object which is different than what you expect. So, your question might be, "How do I get a name to refer to a different object? I thought I could just use subtraction." At that point, you might ask, "Well, how did I get that name to refer to an object (75 in this case) in the first place?" Therein, lies your answer. (Hint: assignment.) Take care, Don From eryksun at gmail.com Mon Jul 8 08:28:39 2013 From: eryksun at gmail.com (eryksun) Date: Mon, 8 Jul 2013 02:28:39 -0400 Subject: [Tutor] custom error classes: how many of them does one need? In-Reply-To: <1373222233.14707.YahooMailNeo@web163805.mail.gq1.yahoo.com> References: <1372966854.32001.YahooMailNeo@web163805.mail.gq1.yahoo.com> <51D62D57.7060400@pearwood.info> <1373222233.14707.YahooMailNeo@web163805.mail.gq1.yahoo.com> Message-ID: On Sun, Jul 7, 2013 at 2:37 PM, Albert-Jan Roskam wrote: > # this is a global variable (perhaps in __init__.py) > # if warningsAsErrors > 0, even warnings will raise an error. > import operator > warningsAsErrors = bool(os.getenv("warningsAsErrors")) I'd use uppercase for the environment variable, with the application name as a prefix, and only require that it's defined (i.e. check if it's in os.environ). For a library you can leave it for the user to decide how to handle warnings: http://docs.python.org/2/library/warnings >>> class MyWarning(UserWarning): pass ... >>> warnings.warn('the sky is falling', MyWarning) __main__:1: MyWarning: the sky is falling >>> warnings.simplefilter('error', MyWarning) >>> warnings.warn('the sky is REALLY falling', MyWarning) Traceback (most recent call last): File "", line 1, in __main__.MyWarning: the sky is REALLY falling >>> warnings.simplefilter('ignore', MyWarning) >>> warnings.warn('Listen! The sky is falling!', MyWarning) >>> # crickets >> SpoonError >> +-- TeaspoonError >> +-- SoupSpoonError >> +-- DesertSpoonError > > DesertSpoonError? LOL! ;-) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "", line 2, in SpellingError: No dessert for you! From steve at pearwood.info Mon Jul 8 09:12:15 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Mon, 8 Jul 2013 17:12:15 +1000 Subject: [Tutor] please return flys in ointment In-Reply-To: References: Message-ID: <20130708071215.GA32148@ando> Comments on your code inline below. On Sat, Jul 06, 2013 at 02:38:27PM -0700, Jim Mooney wrote: > import sys > > # Data > ones = {'1': 'one', '2': 'two', '3': 'three', '4': 'four', '5': 'five', > '6': 'six', '7': 'seven', '8': 'eight', '9': 'nine'} > > tens = {'2': 'twenty', '3': 'thirty', '4': 'forty', '5': 'fifty', '6': > 'sixty', '7': 'seventy', '8': 'eighty', '9': 'ninety'} > > doubles = {'0': 'ten', '1': 'eleven', '2': 'twelve', '3': 'thirteen', '4': > 'fourteen', '5': 'fifteen', '6': 'sixteen', '7': 'seventeen', '8': > 'eighteen', '9': 'nineteen'} > > powers_of_1000 = (' thousand', ' million', ' billion', ' trillion', > ' quadrillion', ' quintillion', ' sextillion', ' septillion', ' octillion', > ' nonillion', ' decillion') > > '''add these later, and also option for dollars-and-cents ending. > 'vigintillion', 'novemdecillion', 'octodecillion', 'septendecillion', > 'sexdecillion', 'quindecillion', 'quattuordecillion', 'tredecillion', > 'duodecillion', 'undecillion', > 'decillion', 'nonillion' > ''' > > # Functions > def make_triplets_from_input(): > '''Enter a positive integer. A list of triplets in the same order will > be returned. > Raise ValueError to loop on non-integer input, or return 'zero' > trigger-value of > 'xxx' if all zeroes are entered. If input is okay, strip leading > zeroes, then > zero-pad leftmost triplet to three characters, so all slices are > triplets. Adjust > input for different Python versions.''' > while True: > try: > if sys.version[0:3] == '2.7': > numbers_str = original = raw_input('Enter a positive > integer, space \ > separated if desired.') > elif sys.version[0:3] == '3.3': > numbers_str = original = input('Enter a positive integer, > space \ > separated if desired.') A better way to handle this is to perform a check once, outside the function at the top of the file: try: raw_input except NameError: # Probably Python 3. raw_input = input and then just use raw_input inside the function. Or the other way around, if you prefer: try: raw_input except NameError: pass else: input = raw_input Then, inside the function, just unconditionally call: result = input("Prompt, or 'quit' to exit: ") result = result.lower().strip() if result == 'quit': break > else: > print('Python versions 2.7 and 3.3 only supported') > sys.exit() > > numbers_str = ''.join(numbers_str.split()) > if numbers_str == '' or numbers_str == ' ': raise ValueError Since you're splitting on whitespace, and joining with the empty string, you cannot get numbers_str == ' '. Besides, no need to explicitly check for '' here, since the very next line will do so for you: > numbers_int = int(numbers_str) > if numbers_int == 0: > print('Zero') > sys.exit() One should not treat "0" to mean "I want to quit". > numbers_str = str(numbers_int) > if len(numbers_str) > 36: raise ArithmeticError ArithmeticError??? You're not doing arithmetic, how can this be an arithmetic error? You should be able to handle implausibly/arbitrarily long numbers, just by chaining names (e.g. "billion billion billion billion..."). If you're going to force some arbitrary limit on the number, and then use an exception for flow-control (as you do here, catching ArithmeticError further on), please use a custom exception so as to not confuse the person reading your code. > break > except KeyboardInterrupt: > print('Program cancelled by user') > sys.exit() /me lip curls I'm not sure I like that... > except ValueError as err: > print(original, "is not an integer.") > continue > except ArithmeticError as err: > print(original, "is too big.\n999...decillion is max: 10**37-1 or 36 chars \ > or 12 groups of 3 chars") > > leftpad = len(numbers_str) % 3 # zero-pad left, so we get all 3-character triplets > if leftpad == 0: leftpad = '' > elif leftpad == 2: leftpad = '0' > else: leftpad = '00' > numbers_str = leftpad + numbers_str leftpad = len(numbers_str) % 3 if leftpad != 0: leftpad = 3-leftpad numbers_str = '0'*leftpad + numbers_str > triplets = [numbers_str[x:x+3] for x in range(0,len(numbers_str),3)] > return (triplets, len(triplets)) See also the recipe for "grouper" in the documentation for itertools. -- Steven From cybervigilante at gmail.com Mon Jul 8 19:57:53 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 8 Jul 2013 10:57:53 -0700 Subject: [Tutor] please return flys in ointment In-Reply-To: <20130708071215.GA32148@ando> References: <20130708071215.GA32148@ando> Message-ID: On 8 July 2013 00:12, Steven D'Apranocatching ArithmeticError > further on), please use a custom exception > Well, an Arithmetic Error was better than bubbling up to a totally general one. I'm not sure how do custom error code. Short example, please ;') Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From davea at davea.name Mon Jul 8 21:47:33 2013 From: davea at davea.name (Dave Angel) Date: Mon, 08 Jul 2013 15:47:33 -0400 Subject: [Tutor] please return flys in ointment In-Reply-To: References: <20130708071215.GA32148@ando> Message-ID: On 07/08/2013 01:57 PM, Jim Mooney wrote: > On 8 July 2013 00:12, Steven D'Apranocatching ArithmeticError > >> further on), please use a custom exception >> > > Well, an Arithmetic Error was better than bubbling up to a totally general > one. I'm not sure how do custom error code. Short example, please ;') > > Jim > I think a ValueError might be best. See the description: http://docs.python.org/2/library/exceptions.html#exceptions.ValueError exception ValueError Raised when a built-in operation or function receives an argument that has the right type but an inappropriate value, and the situation is not described by a more precise exception such as IndexError. To make a custom error, first pick another error that's a superset of what you're doing. Then simply derive your error class from it. As a minimum: class TooHugeError (ValueError): pass -- DaveA From cybervigilante at gmail.com Mon Jul 8 21:59:30 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 8 Jul 2013 12:59:30 -0700 Subject: [Tutor] please return flys in ointment In-Reply-To: References: <20130708071215.GA32148@ando> Message-ID: > To make a custom error, first pick another error that's a superset of what > you're doing. Then simply derive your error class from it. As a minimum: > > class TooHugeError (ValueError): > pass > > Actually, I didn't get to classes yet since I wanted to nail procedural - I'm in no rush - but that looks simple enough. Damn, computer is overheating again. I'll be glad when the AZ heatwave is over so I can get back to more hours on the computer, but Reality intervenes ;') Jim > > > > > -- > DaveA > > > ______________________________**_________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/**mailman/listinfo/tutor > -- Jim Most American jurors are not told, by power-mad judges, that they can disregard the law, the evidence, and the instructions of the judge, if they feel a law is unfair, and render a verdict of "innocent". They are kept in the dark about this right, called Jury Nullification. -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan.gauld at btinternet.com Mon Jul 8 22:03:22 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Mon, 08 Jul 2013 21:03:22 +0100 Subject: [Tutor] AMQP listening and user-facing daemon In-Reply-To: <51D9E463.7030403@gmail.com> References: <51D9E463.7030403@gmail.com> Message-ID: On 07/07/13 22:57, Justin Chiu wrote: > What is the best approach to writing a concurrent daemon that can > execute callbacks for different types of events (AMQP messages, parsed > output of a subprocess, HTTP requests)? Given nobody replied to your previous request I suggest you try another forum. The tutor list is for people learning Python and its standard library, more general questions are best addressed to the general python list or newsgroup. I see you have cross posted there too so hopefully somebody will pick up on it soon. HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From cybervigilante at gmail.com Mon Jul 8 22:18:23 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 8 Jul 2013 13:18:23 -0700 Subject: [Tutor] please return flys in ointment In-Reply-To: References: <20130708071215.GA32148@ando> Message-ID: On 8 July 2013 12:47, Dave Angel wrote: > To make a custom error, first pick another error that's a superset of what > you're doing. Then simply derive your error class from it. As a minimum: > > class TooHugeError (ValueError): > pass Hmm, I tried that, but instead of going to the exception for too huge: except TooHugeError as err: , which prints a "too big" statement, it went to a valid exception I already had for a Value Error: except ValueError as err:, and printed the error as a simple Value Error rather than my own statement. i.e. try: blah, blah, blah... if len(numbers_str) > 36: raise TooHugeError break except KeyboardInterrupt: print('Program cancelled by user') sys.exit() except ValueError as err: # I go here print(original, "is not an integer.") continue except TooHugeError as err: # when I want to go here print(original, "is too big.\n999...decillion is max:" " 10**37-1 or 36 chars or 12 groups of 3 chars") BTW, to fit in their new, enormous sidebar ad, gmail is shrinking my text box so small things are gapping and overlapping, so if something looks funny, it was okay in my IDE, which I've cut to about 70 width for mail readers ;') Jim From cybervigilante at gmail.com Mon Jul 8 22:33:42 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 8 Jul 2013 13:33:42 -0700 Subject: [Tutor] please return flys in ointment In-Reply-To: References: <20130708071215.GA32148@ando> Message-ID: On 8 July 2013 12:47, Dave Angel Never mind about the wrong error tripping. I just moved TooHugeError above ValueError and it tripped properly. Finally, a simple solution ;') Jim From fomcl at yahoo.com Mon Jul 8 22:51:10 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Mon, 8 Jul 2013 13:51:10 -0700 (PDT) Subject: [Tutor] custom error classes: how many of them does one need? In-Reply-To: References: <1372966854.32001.YahooMailNeo@web163805.mail.gq1.yahoo.com> <51D62D57.7060400@pearwood.info> <1373222233.14707.YahooMailNeo@web163805.mail.gq1.yahoo.com> Message-ID: <1373316670.103.YahooMailNeo@web163803.mail.gq1.yahoo.com> ----- Original Message ----- > From: eryksun > To: Albert-Jan Roskam > Cc: "tutor at python.org" > Sent: Monday, July 8, 2013 8:28 AM > Subject: Re: [Tutor] custom error classes: how many of them does one need? > > On Sun, Jul 7, 2013 at 2:37 PM, Albert-Jan Roskam wrote: >> # this is a global variable (perhaps in __init__.py) >> # if warningsAsErrors > 0, even warnings will? raise an error. >> import operator >> warningsAsErrors = bool(os.getenv("warningsAsErrors")) > > I'd use uppercase for the environment variable, with the application > name as a prefix, and only require that it's defined (i.e. check if > it's in os.environ). > > For a library you can leave it for the user to decide how to handle warnings: > > http://docs.python.org/2/library/warnings > > ? ? >>> class MyWarning(UserWarning): pass > ? ? ... > ? ? >>> warnings.warn('the sky is falling', MyWarning) > ? ? __main__:1: MyWarning: the sky is falling > > ? ? >>> warnings.simplefilter('error', MyWarning) > ? ? >>> warnings.warn('the sky is REALLY falling', MyWarning) > ? ? Traceback (most recent call last): > ? ? ? File "", line 1, in > ? ? __main__.MyWarning: the sky is REALLY falling > > ? ? >>> warnings.simplefilter('ignore', MyWarning) > ? ? >>> warnings.warn('Listen! The sky is falling!', MyWarning) > ? ? >>> # crickets Awesome. Exactly what I need! I really should try each and every module of the standard library as it contains so much cool stuff. import warnings import os import ctypes # __init__.py retcodes = {i: "%s %s" % ("warning" if i < 0 else "error", i) ??????????? for i in range(-10, 10)} MYAPP_WARNINGS_AS_ERRORS = bool(os.environ.get("MYAPP_WARNINGS_AS_ERRORS")) # error.py (should actually be named exception.py now) class CustomWarning(UserWarning): pass action = "error" if MYAPP_WARNINGS_AS_ERRORS else "default" warnings.simplefilter(action, CustomWarning) class CustomError(Exception): ??? def __init__(self, retcode, msg): ??????? self.retcode = retcode ??????? Exception.__init__(self, msg) def possiblyThrow(retcode, msg): ??? """Throws a warning if retcode < 0 (and warnings are not ignored), ??? and throws an error if retcode > 0. Returns None if retcode == 0""" ??? msg += " [%s]" % retcodes.get(retcode, retcode) ??? if retcode > 0: ??????? raise CustomError(retcode, msg) ??? elif retcode < 0: ??????? warnings.warn(msg, CustomWarning, stacklevel=2) # myapp.py # bogus example, works only on windows. libc = ctypes.cdll.msvcrt retcode = libc.printf("Ooooh %s", "lala!") # retcode > 0 is not even an error here #retcode = -1 possiblyThrow(retcode, "Problem with libc.printf") ? >>> SpoonError >>> +-- TeaspoonError >>> +-- SoupSpoonError >>> +-- DesertSpoonError >> >> DesertSpoonError? LOL! ;-) > > During handling of the above exception, another exception occurred: > > Traceback (most recent call last): > ? File "", line 2, in > SpellingError: No dessert for you! haha, I hadn't even noticed the 'speling eror'. I was just thinking 'what the heck is a desert spoon'? ;-) From davea at davea.name Mon Jul 8 23:07:36 2013 From: davea at davea.name (Dave Angel) Date: Mon, 08 Jul 2013 17:07:36 -0400 Subject: [Tutor] custom error classes: how many of them does one need? In-Reply-To: <1373316670.103.YahooMailNeo@web163803.mail.gq1.yahoo.com> References: <1372966854.32001.YahooMailNeo@web163805.mail.gq1.yahoo.com> <51D62D57.7060400@pearwood.info> <1373222233.14707.YahooMailNeo@web163805.mail.gq1.yahoo.com> <1373316670.103.YahooMailNeo@web163803.mail.gq1.yahoo.com> Message-ID: On 07/08/2013 04:51 PM, Albert-Jan Roskam wrote: > > > > haha, I hadn't even noticed the 'speling eror'. I was just thinking 'what the heck is a desert spoon'? ;-) "A dessert spoon is a spoon designed specifically for eating dessert and sometimes used for soup or cereals." On the other hand, a desert spoon might be one that's extremely light, so that it may be taken along by desert rats out foraging. -- DaveA From marc.tompkins at gmail.com Mon Jul 8 23:03:17 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Mon, 8 Jul 2013 14:03:17 -0700 Subject: [Tutor] custom error classes: how many of them does one need? In-Reply-To: <1373316670.103.YahooMailNeo@web163803.mail.gq1.yahoo.com> References: <1372966854.32001.YahooMailNeo@web163805.mail.gq1.yahoo.com> <51D62D57.7060400@pearwood.info> <1373222233.14707.YahooMailNeo@web163805.mail.gq1.yahoo.com> <1373316670.103.YahooMailNeo@web163803.mail.gq1.yahoo.com> Message-ID: On Mon, Jul 8, 2013 at 1:51 PM, Albert-Jan Roskam wrote: > haha, I hadn't even noticed the 'speling eror'. I was just thinking 'what > the heck is a desert spoon'? ;-) > Many years ago, I saw this painted on the side of a pie shop: "My favorite people are the people of the dessert", said Lawrence as he picked up his fork. -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan.gauld at btinternet.com Mon Jul 8 23:09:13 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Mon, 08 Jul 2013 22:09:13 +0100 Subject: [Tutor] custom error classes: how many of them does one need? In-Reply-To: <1373316670.103.YahooMailNeo@web163803.mail.gq1.yahoo.com> References: <1372966854.32001.YahooMailNeo@web163805.mail.gq1.yahoo.com> <51D62D57.7060400@pearwood.info> <1373222233.14707.YahooMailNeo@web163805.mail.gq1.yahoo.com> <1373316670.103.YahooMailNeo@web163803.mail.gq1.yahoo.com> Message-ID: On 08/07/13 21:51, Albert-Jan Roskam wrote: >I was just thinking 'what the heck is a desert spoon'? ;-) It's the one half-way between a teaspoon and a tablespoon :-) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From steve at pearwood.info Tue Jul 9 03:16:17 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 09 Jul 2013 11:16:17 +1000 Subject: [Tutor] please return flys in ointment In-Reply-To: References: <20130708071215.GA32148@ando> Message-ID: <51DB6461.5020204@pearwood.info> On 09/07/13 05:47, Dave Angel wrote: > On 07/08/2013 01:57 PM, Jim Mooney wrote: >> On 8 July 2013 00:12, Steven D'Apranocatching ArithmeticError >> >>> further on), please use a custom exception >>> >> >> Well, an Arithmetic Error was better than bubbling up to a totally general >> one. I'm not sure how do custom error code. Short example, please ;') >> >> Jim >> > > I think a ValueError might be best. [...] > class TooHugeError (ValueError): > pass I would normally agree, but in Jim's code the exception is purely being used for internal flow control, it is not being exposed to the caller. So I wouldn't inherit from ValueError, I'd keep this a completely independent exception. I wouldn't even call it an Error, since it's actually a limitation of Jim's code, not an actual error. Also, since Jim's code already catches ValueError, he would need ensure that he catches this *first*, otherwise it will be caught by the ValueError handler: try: raise TooHugeError except TooHugeError: # this must come first ... except ValueError: # otherwise this will catch it ... Whereas if you keep the custom exception independent, the order of except clauses doesn't matter. class TooHuge(Exception): pass try: raise TooHuge except ValueError: print("bad value") except TooHuge: print("caught it") -- Steven From steve at pearwood.info Tue Jul 9 04:22:47 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 09 Jul 2013 12:22:47 +1000 Subject: [Tutor] please return flys in ointment In-Reply-To: References: <20130708071215.GA32148@ando> Message-ID: <51DB73F7.3090508@pearwood.info> On 09/07/13 05:59, Jim Mooney wrote: >> To make a custom error, first pick another error that's a superset of what >> you're doing. Then simply derive your error class from it. As a minimum: >> >> class TooHugeError (ValueError): >> pass >> >> Actually, I didn't get to classes yet since I wanted to nail procedural - > I'm in no rush - but that looks simple enough. > > Damn, computer is overheating again. I'll be glad when the AZ heatwave is > over so I can get back to more hours on the computer, but Reality > intervenes ;') That's what I keep telling my AGW-Denialist friend, but he prefers living in a fantasy world where human beings can inject four Hiroshima A-bombs worth of heat into the oceans and atmosphere every second without consequence. (The actual excess heat accumulated over the last decade is 8000 million million megajoules per year.) Global temperatures were increasing even over the last few years while the solar cycle was cooling, now that the solar cycle is returning to its heating phase, so get used to it, you're experiencing a small glimpse of the future. -- Steven From cybervigilante at gmail.com Tue Jul 9 07:26:46 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 8 Jul 2013 22:26:46 -0700 Subject: [Tutor] please return flys in ointment In-Reply-To: <51DB73F7.3090508@pearwood.info> References: <20130708071215.GA32148@ando> <51DB73F7.3090508@pearwood.info> Message-ID: On 8 July 2013 19:22, Steven D'Aprano wrote: > That's what I keep telling my AGW-Denialist friend, but he prefers living in > a fantasy world where human beings can inject four Hiroshima A-bombs worth > of heat into the oceans and atmosphere every second without consequence. > (The actual excess heat accumulated over the last decade is 8000 million > million megajoules per year.) Global temperatures were increasing even over > the last few years while the solar cycle was cooling, now that the solar > cycle is returning to its heating phase, so get used to it, you're > experiencing a small glimpse of the future. There is a recent well-documented doctoral thesis - not the usual scarification - that the North Polar icecap may disappear this year. The masses don't grasp that multiple positive feedback loops, from albedo-change to methane-release, and some not understood, are moving things along Much faster than even pessimistic scientists predicted. Get ready for food wars consequent on climate-disruption. The thesis: http://www.sierraclub.ca/en/AdultDiscussionPlease And some factual supporting information from the US Navy: http://www.fairfaxclimatewatch.com/blog/2013/06/Unprecedented-hole-is-growing-in-Arctic-sea-ice.html Navy specs not indicated at that link, but I backfollowed it to its source, so they're easy enough to find. I tracked this pretty far. It's not vague environmentalist heavings. The N Polar cap shows unprecedented stress-cracking. A strangely persistent cyclone is breaking the ice up even more and scattering the ice. Very oddly, the thickest ice, at the center of the pole, is becoming thinner than outlying regions, for the fist time in known history. But my bet is the short-memoried public will forget the major denialists, who will probably keep doing quite well. Of course, this isn't mentioned in the media. But then, when I warned the repeal of Glass-Steagall would lead directly to vast economic disruption, over ten years ago - but that wasn't mentioned in the media, either. The media tends to predict the past, and Still gets the real causes wrong ;'( It may still be possible, even this late, to reverse some of the disaster with massive terraforming - costly, but still less than the cost of even one war - but of course, we aren't going to do that. And terraforming will follow the law of unintended consequences since we haven't done it before - although we are doing it now, only unconsciously Even if this guy's prediction if off for this year, the regular decrease of North polar ice is unsustainable within five years at most. And I haven't even factored in the South Pole since the North Pole has a much greater effect on global weather patterns in the Northern Hemisphere, where most of us live. Okay, end of wildly off-topic. You just happened to mention something I had only just researched for veracity, so I still had a few links ;') Jim > > > -- > Steven > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor -- Jim Most American jurors are not told, by power-mad judges, that they can disregard the law, the evidence, and the instructions of the judge, if they feel a law is unfair, and render a verdict of "innocent". They are kept in the dark about this right, called Jury Nullification. From cybervigilante at gmail.com Tue Jul 9 20:05:08 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Tue, 9 Jul 2013 11:05:08 -0700 Subject: [Tutor] please return flys in ointment In-Reply-To: <51DB6461.5020204@pearwood.info> References: <20130708071215.GA32148@ando> <51DB6461.5020204@pearwood.info> Message-ID: On 8 July 2013 18:16, Steven D'Aprano wrote: I wouldn't even call it an Error, since it's actually a > limitation of Jim's code, not an actual error. Good point. But since I'm using try-except on input anyway, largely for int-checking, i figured why not use it? Putting in warn or something like that would just complicate matters. I'm a chicken about input since webmasters have some demented people putting all sorts of odd javascript, PHP, and SQL into input to see if they can wreak some damage ;') BTW, everything has an upside. With my computer time limited by a heat wave and another complication, I'm reading the data model I already had printed, and cleared up my misapprehension that complex numbers were kind-of integers, since 3 + 2j looks integral. I see all sorts of things I missed on a first reading ;') Jim Most American jurors are not told, by power-mad judges, that they can disregard the law, the evidence, and the instructions of the judge, if they feel a law is unfair, and render a verdict of "innocent". They are kept in the dark about this right, called Jury Nullification. From steve at pearwood.info Wed Jul 10 03:16:48 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 10 Jul 2013 11:16:48 +1000 Subject: [Tutor] please return flys in ointment In-Reply-To: References: <20130708071215.GA32148@ando> <51DB6461.5020204@pearwood.info> Message-ID: <51DCB600.4070808@pearwood.info> On 10/07/13 04:05, Jim Mooney wrote: > On 8 July 2013 18:16, Steven D'Aprano wrote: >> I wouldn't even call it an Error, since it's actually a >> limitation of Jim's code, not an actual error. > > Good point. But since I'm using try-except on input anyway, largely > for int-checking, i figured why not use it? Putting in warn or > something like that would just complicate matters. Oh, I'm not saying you shouldn't use an exception. I'm just suggesting that you don't call it a Something-or-other-Error, since *internally* it is not a "bad data" error, but a "my function has a limitation" problem. Since the exception is only seen internally, it doesn't really matter though. On the other hand, once you catch the Whatever-the-hell-it-gets-called exception, you treat it as if it were an error, by telling the user to try again. So I'm drawing a *really* fine line between errors and non-errors, and you may freely decide which side of the line to cut. The real point I was hoping you would take from this is that exceptions are not necessarily *errors*. E.g.: TypeError, ValueError, ZeroDivisionError -- all errors. SystemExit, KeyboardInterrupt, StopIteration -- not errors. Correct naming of exceptions are just as important as correct naming of functions and variables. > I'm a chicken about input since webmasters have some demented people > putting all sorts of odd javascript, PHP, and SQL into input to see if > they can wreak some damage ;') :-) My advice is to split your code into two (or more) parts. The "engine", so to speak, should try it's best to handle anything you throw at it, although it may raise an exception or run for a very long time. E.g. if you type in 10**(10**100) (a googolplex), Python will try to calculate the answer, and probably fail with a MemoryError after an hour or so, if you are lucky. The second part faces the user, and acts as interface from the (trusted) engine to the (untrusted) user's data, and that's where you put in any additional restrictions validating user data. That may seem like more work up front, but it pays off in the long run. [...] > Most American jurors are not told, by power-mad judges, that they can > disregard the law, the evidence, and the instructions of the judge, if > they feel a law is unfair, and render a verdict of "innocent". They > are kept in the dark about this right, called Jury Nullification. Completely off-topic, but I think it's unfair to describe them as "power-mad" because they have respect for the law, and incorrect to describe Jury Nullification as a *right*. A jury has *practical power* to ignore the law, since what goes on behind the closed doors of the jury room is sacrosanct and under American law judges and prosecutors have very limited power to inquire into why a jury decides to convict or not. But that's not a legal right, any more than I have the legal right to slander you provided you don't find out about it. It's more like civil disobedience than an explicit right granted to jurors. But consider that Jury Nullification is a dangerous power to wield. Jury nullification has been used to protect escaped slaves from being returned to their slave owners, and to allow racists to commit racially-motivated murder with impunity. It's a double-edged sword, and there's no reason to assume that justice will always be served by encouraging jurors to ignore the law. -- Steven From cybervigilante at gmail.com Wed Jul 10 05:22:19 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Tue, 9 Jul 2013 20:22:19 -0700 Subject: [Tutor] please return flys in ointment In-Reply-To: <51DCB600.4070808@pearwood.info> References: <20130708071215.GA32148@ando> <51DB6461.5020204@pearwood.info> <51DCB600.4070808@pearwood.info> Message-ID: On 9 July 2013 18:16, Steven D'Aprano wrote: > The real point I was hoping you would take from this is that exceptions are > not necessarily *errors*. E.g.: > > TypeError, ValueError, ZeroDivisionError -- all errors. > > SystemExit, KeyboardInterrupt, StopIteration -- not errors. Oh, I'm quite fond of try...except. Although I see it postponed way to the back of some books as an "advanced topic," as if it has the plague ;') I can see it's not just for errors, but the best way to do something on many occasions. It follows the idea of '..better to ask forgiveness than permission." Break something if it's the best way to do things, but catch the break. -- Jim When the Rabbi Kaufman went to visit the Baal Shem Tove, the spiritual light of the age, his students asked him what deep questions he would pose the Great Master. The Rabbi Kaufman replied, "I want to see how he ties his shoes." From lseibold at seiboldsystems.com Wed Jul 3 03:32:36 2013 From: lseibold at seiboldsystems.com (larry seibold) Date: Tue, 02 Jul 2013 18:32:36 -0700 Subject: [Tutor] install on windows Message-ID: <51D37F34.9020609@seiboldsystems.com> I am stuck at step one, installing "python-2.7.5.msi" on windows XP. I downloaded it (~16MB), but when I select it (double click), I get a windows installer pop up with an OK button at the bottom, which when selected seems to terminate the install vs. install the program. I do not see any advice on a different procedure. Help. Do I need to run the msi from a command window with options? From robert.winkler at bioprocess.org Thu Jul 4 21:47:00 2013 From: robert.winkler at bioprocess.org (Robert Winkler) Date: Thu, 04 Jul 2013 14:47:00 -0500 Subject: [Tutor] Convert SOAP response (ArrayOfInt) to Python list Message-ID: <51D5D134.6020506@bioprocess.org> Thanks to the OSA library, which works for SOAP requests with Python 3.x, I can now use SOAP services at http://www.chemspider.com. The result is a list of accession numbers (which correspond to chemical compounds) and I get them in the following format: |(ArrayOfInt){ int[] = [ 5744, 69182, 292, 68027, 3404131, 82616, 18280, 11200, 704646, 543430 ... ] }| How could I transform this to a simple python list? |[5744, 69182, 292,68027, 3404131, 82616, 18280, 11200, 704646, 543430 ...]| Conversion to a numpy array (and subsequent |list()|, or similar) does not solve the problem, since the structure is maintained; the numpy.shape returns |()|. Suggestions? -------------- next part -------------- An HTML attachment was scrubbed... URL: From amandeep.qa at gmail.com Fri Jul 5 20:27:46 2013 From: amandeep.qa at gmail.com (Amandeep Behl) Date: Fri, 5 Jul 2013 11:27:46 -0700 Subject: [Tutor] Difference between max(one, two) and max((one, two)) Message-ID: What is the difference between max(one, two) and max((one, two)) ? Thanks Aman -------------- next part -------------- An HTML attachment was scrubbed... URL: From amandeep.qa at gmail.com Fri Jul 5 20:37:26 2013 From: amandeep.qa at gmail.com (Amandeep Behl) Date: Fri, 5 Jul 2013 11:37:26 -0700 Subject: [Tutor] Difference between max(one, two) and max((one, two)) In-Reply-To: References: Message-ID: and why with sum(one, two) we get an error whereas not with sum((one, two)) ? On Fri, Jul 5, 2013 at 11:27 AM, Amandeep Behl wrote: > What is the difference between max(one, two) and max((one, two)) ? > > Thanks > Aman > -------------- next part -------------- An HTML attachment was scrubbed... URL: From afowler2334 at yahoo.com Fri Jul 5 23:07:56 2013 From: afowler2334 at yahoo.com (Ashley Fowler) Date: Fri, 5 Jul 2013 14:07:56 -0700 (PDT) Subject: [Tutor] I need help on this program please Message-ID: <1373058476.24771.YahooMailNeo@web161804.mail.bf1.yahoo.com> This is what I have to do : On a standard telephone, the alphabetic letters are mapped to numbers in the following fashion: ? A, B, AND C=2 D, E, AND F=3 G, H, AND I=4 J, K, AND L=5 M, N, AND O=6 P, Q, R, AND S=7 T, U, AND V=8 W, X, Y AND Z=9 ? Write a program that asks the user to enter a 10-character telephone numbers in the format ?XXX-XXX-XXXX. The application should display the telephone number with any alphabetic characters that appeared? in the original translated to their numeric equivalent. For example, if the user enters 555-GET-FOOD the application should display 555-438-3663. def main(): ? ? digits = ["2", "3", "4", "5", "6", "7", "8", "9"] ? ?? ? ? numeric_phone=" " ? ? phone_number = input('Enter 10-Digit phone number: ') ? ? ch= digits[index] ? ? for ch in phone_number: ? ? ? ? if ch.isalpha(): ? ? ? ? ? ? if ch == 'A' or ch == 'B' or ch == 'C': ? ? ? ? ? ? ? index = 0 ? ? ? ? ? ? if ch == 'D' or ch == 'E' or ch == 'F': ? ? ? ? ? ? ? ? index = 1 ? ? ? ? ? ? if ch == 'G' or ch == 'H' or ch == 'I': ? ? ? ? ? ? ? ? index = 2 ? ? ? ? ? ? if ch == 'J' or ch == 'K' or ch == 'L': ? ? ? ? ? ? ? ? index = 3 ? ? ? ? ? ? if ch == 'M' or ch == 'N' or ch == 'O': ? ? ? ? ? ? ? ? index = 4 ? ? ? ? ? ? if ch == 'P' or ch == 'Q' or ch == 'R' or ch == 'S': ? ? ? ? ? ? ? ? index = 5 ? ? ? ? ? ? if ch == 'T' or ch == 'U' or ch == 'V': ? ? ? ? ? ? ? ? index = 6 ? ? ? ? ? ? if ch == 'W' or ch == 'X' or ch == 'Y' or ch == 'Z': ? ? ? ? ? ? ? ? index = 7 ? ? ? ?? ? ? ? ? numeric_phone= numberic_phone+ch ? ? ? ? ? ?? ? ? print(numeric_phone) ? ? ? ? ? ? ? ? This is what I have so far. Can anyone make suggestions or tell me what I need to correct? -------------- next part -------------- An HTML attachment was scrubbed... URL: From afowler2334 at yahoo.com Fri Jul 5 23:10:39 2013 From: afowler2334 at yahoo.com (Ashley Fowler) Date: Fri, 5 Jul 2013 14:10:39 -0700 (PDT) Subject: [Tutor] Help Please Message-ID: <1373058639.29054.YahooMailNeo@web161801.mail.bf1.yahoo.com> This is what I have to do : On a standard telephone, the alphabetic letters are mapped to numbers in the following fashion: ? A, B, AND C=2 D, E, AND F=3 G, H, AND I=4 J, K, AND L=5 M, N, AND O=6 P, Q, R, AND S=7 T, U, AND V=8 W, X, Y AND Z=9 ? Write a program that asks the user to enter a 10-character telephone numbers in the format ?XXX-XXX-XXXX. The application should display the telephone number with any alphabetic characters that appeared? in the original translated to their numeric equivalent. For example, if the user enters 555-GET-FOOD the application should display 555-438-3663. def main(): ? ? digits = ["2", "3", "4", "5", "6", "7", "8", "9"] ? ?? ? ? numeric_phone=" " ? ? phone_number = input('Enter 10-Digit phone number: ') ? ? ch= digits[index] ? ? for ch in phone_number: ? ? ? ? if ch.isalpha(): ? ? ? ? ? ? if ch == 'A' or ch == 'B' or ch == 'C': ? ? ? ? ? ? ? index = 0 ? ? ? ? ? ? if ch == 'D' or ch == 'E' or ch == 'F': ? ? ? ? ? ? ? ? index = 1 ? ? ? ? ? ? if ch == 'G' or ch == 'H' or ch == 'I': ? ? ? ? ? ? ? ? index = 2 ? ? ? ? ? ? if ch == 'J' or ch == 'K' or ch == 'L': ? ? ? ? ? ? ? ? index = 3 ? ? ? ? ? ? if ch == 'M' or ch == 'N' or ch == 'O': ? ? ? ? ? ? ? ? index = 4 ? ? ? ? ? ? if ch == 'P' or ch == 'Q' or ch == 'R' or ch == 'S': ? ? ? ? ? ? ? ? index = 5 ? ? ? ? ? ? if ch == 'T' or ch == 'U' or ch == 'V': ? ? ? ? ? ? ? ? index = 6 ? ? ? ? ? ? if ch == 'W' or ch == 'X' or ch == 'Y' or ch == 'Z': ? ? ? ? ? ? ? ? index = 7 ? ? ? ?? ? ? ? ? numeric_phone= numberic_phone+ch ? ? ? ? ? ?? ? ? print(numeric_phone) ? ? ? ? ? ? ? ? This is what I have so far. Can anyone make suggestions or tell me what I need to correct? * * -------------- next part -------------- An HTML attachment was scrubbed... URL: From c.justin88 at gmail.com Fri Jul 5 23:11:24 2013 From: c.justin88 at gmail.com (Justin Chiu) Date: Fri, 05 Jul 2013 14:11:24 -0700 Subject: [Tutor] AMQP listening and user-facing daemon Message-ID: <51D7367C.8060409@gmail.com> Hi all, What is the best approach to writing a concurrent daemon that can execute callbacks for different types of events (AMQP messages, parsed output of a subprocess, HTTP requests)? I am considering [twisted][1], the built-in [threading][2] module, and [greenlet][3]. I must admit that I am very unfamiliar with concurrent programming and Python programming in general (formerly a data analysis driven procedural programmer). Any resources on threaded/concurrent programming (specifically daemons...not just multi-threading a single task) would be much appreciated. Thank you. Details: 1) Listens into AMQP messaging queues and executes callbacks when messages arrive. Example: Immediately after startup, the daemon continuously listens to the [Openstack Notifications messaging queue][4]. When a virtual machine is launched, a notification is generated by Openstack with the hostname, IP address, etc. The daemon should read this message and write some info to a log (or POST the info to a server, or notify the user...something simple). 2) Parse the output of a subprocess and execute callbacks based on the output. Example: Every 30 seconds, a system command "[qstat][5]" is run to query a job resource manager (e.g. TORQUE). Similar callbacks to 1). 3) Receive requests from a user and process them. I think this will be via WSGI HTTP. Example: User submits an XML template with virtual machine templates. The daemon does some simple XML parsing and writes a job script for the job resource manager. The job is submitted to the resource manager and the daemon continually checks for the status of the job with "qstat" and for messages from AMQP. It should return "live" feedback to the user and write to a log. [1]: https://twistedmatrix.com/trac/wiki/Documentation [2]: http://docs.python.org/2/library/threading.html [3]: http://greenlet.readthedocs.org/en/latest/ [4]: https://wiki.openstack.org/wiki/NotificationEventExamples#Immediate_Notifications: [5]: http://www.clusterresources.com/torquedocs21/commands/qstat.shtml Justin Chiu TRIUMF From bluepresley at gastonia.com Sun Jul 7 05:18:35 2013 From: bluepresley at gastonia.com (bluepresley) Date: Sun, 7 Jul 2013 12:18:35 +0900 Subject: [Tutor] What's going on in this Python code from Programming Collective Intelligence? Message-ID: I'm reading the book Programming Collective Intelligence by Toby Segaran. I'm having a lot of difficulty understanding the some of the code from chapter four (the code for this chapter is available online at https://github.com/cataska/programming-collective-intelligence-code/blob/master/chapter4/searchengine.pyand starts at line172), specifically staring with this function: def getscoredlist(self,rows,wordids): totalscores=dict([(row[0],0) for row in rows])# This is where you'll later put the scoring functions weights=[] for (weight,scores) in weights: for url in totalscores: totalscores[url]+=weight*scores[url] return totalscores What does this mean? totalscores=dict([(row[0],0) for row in rows]) I have a lot of experience with PHP and Objective C. If anyone is familiar with PHP or another language could you please provide the equivalent? I think that would really help me understand better. The function just before that, getmatchingrows, provides the arguments for getscoredlist. "rows" is rows from a database query; "wordids" is a list of word ids searched for that generated rows result set. For that function (getmatchingrows) it returns 2 variables simultaneously. I'm unfamiliar with this. What's going on there? Also, as far as I can tell from the getmatchingrows code, it returns a multidimensional array of database results with row[0] being the urlid (NOT the url), and other indices correspond to the word id location. In getscoredlist, totalscores[url] doesn't make sense. Where is [url] coming from? could they have meant to say urlid here? This chapter is also available online for free from O'reilly. Here is the page that talks specifically about this part. Any help understanding what this code in this part of this book is doing would be greatly appreciated. Thanks, Blue http://my.safaribooksonline.com/book/web-development/9780596529321/4dot-searching-and-ranking/querying#X2ludGVybmFsX0h0bWxWaWV3P3htbGlkPTk3ODA1OTY1MjkzMjElMkZjb250ZW50YmFzZWRfcmFua2luZyZxdWVyeT0= -------------- next part -------------- An HTML attachment was scrubbed... URL: From shireenrao at gmail.com Sun Jul 7 22:16:48 2013 From: shireenrao at gmail.com (Srinivas Nyayapati) Date: Sun, 7 Jul 2013 16:16:48 -0400 Subject: [Tutor] Best way to setup Unit Testing? Message-ID: I am tryng to figure out the best way to do Unit Testing for all my projects going forward. I am using the unittest framework. For execution I am executing the test cases by either directly using python or by using the nose test runner. I found the following 3 types of how this can be setup. Type 1 ====== The unit tests are located in the same package as the module being tested. This is the easiest to implement and run. I don't have to do anything special to run the tests. The directory structure is like this Projects/ MyApp/ __init__.py myapp.py test_myapp.py Simply run the test from within the MyApp folder as % python test_myapp.py or use the nose test runner % nosetests Type 2 ====== Here the unit tests are located in its own tests folder in the package you want to test. Running the unit tests from inside the tests folder won't work as it worked in Type 1. That's because the myapp module can not be resolved. Here is how the directory structure looks like Projects/ MyApp/ __init__.py myapp.py tests/ __init__.py test_myapp.py It will work if you ran it from inside the package folder and gave the complete path to the test file as % python tests/test_myapp.py It will also work if you used the nose test runner from inside MyApp. % nosetests It will also work from inside the tests folder if you added MyApp to your PYTHONPATH, but that is an option I dont like. I don't want to keep changing my PYTHONPATH. Type 3 ====== This is where the tests folder is on the same level as the package folder. You can not run the test scripts from within the tests folder as again it will not be able to resolve the myapp package. Here is how the directory structure looks like Projects/ MyApp/ __init__.py myapp.py tests/ __init__.py test_myapp.py You can run the tests from within the MyApp package by referencing the tests folder as % python ../tests/test_myapp.py As you are running the test from the package, the myapp module is automatically in your PYTHONPATH. The other way of doing this would be by putting your package in your PYTHONPATH, which again as I mentioned earlier I don't like to do. What is the best way of doing this? I would really appreciate to know how the experts are doing this. Is there anything I need to consider before I start using test suites. Thank you Srini -------------- next part -------------- An HTML attachment was scrubbed... URL: From c.justin88 at gmail.com Sun Jul 7 23:57:55 2013 From: c.justin88 at gmail.com (Justin Chiu) Date: Sun, 07 Jul 2013 14:57:55 -0700 Subject: [Tutor] AMQP listening and user-facing daemon Message-ID: <51D9E463.7030403@gmail.com> Hi all, What is the best approach to writing a concurrent daemon that can execute callbacks for different types of events (AMQP messages, parsed output of a subprocess, HTTP requests)? I am considering [twisted][1], the built-in [threading][2] module, and [greenlet][3]. I must admit that I am very unfamiliar with concurrent programming and Python programming in general (formerly a data analysis driven procedural programmer). Any resources on threaded/concurrent programming (specifically daemons...not just multi-threading a single task) would be much appreciated. Thank you. Details: 1) Listens into AMQP messaging queues and executes callbacks when messages arrive. Example: Immediately after startup, the daemon continuously listens to the [Openstack Notifications messaging queue][4]. When a virtual machine is launched, a notification is generated by Openstack with the hostname, IP address, etc. The daemon should read this message and write some info to a log (or POST the info to a server, or notify the user...something simple). 2) Parse the output of a subprocess and execute callbacks based on the output. Example: Every 30 seconds, a system command "[qstat][5]" is run to query a job resource manager (e.g. TORQUE). Similar callbacks to 1). 3) Receive requests from a user and process them. I think this will be via WSGI HTTP. Example: User submits an XML template with virtual machine templates. The daemon does some simple XML parsing and writes a job script for the job resource manager. The job is submitted to the resource manager and the daemon continually checks for the status of the job with "qstat" and for messages from AMQP. It should return "live" feedback to the user and write to a log. [1]: https://twistedmatrix.com/trac/wiki/Documentation [2]: http://docs.python.org/2/library/threading.html [3]: http://greenlet.readthedocs.org/en/latest/ [4]: https://wiki.openstack.org/wiki/NotificationEventExamples#Immediate_Notifications: [5]: http://www.clusterresources.com/torquedocs21/commands/qstat.shtml Justin Chiu TRIUMF From typentalk at gmail.com Mon Jul 8 07:15:06 2013 From: typentalk at gmail.com (slip slidEnaway) Date: Sun, 7 Jul 2013 22:15:06 -0700 Subject: [Tutor] after installed modules with softwarecenter -Ubuntu 10.04 LTS - the interpreter gives error message " ImportError: No module named pyglet" Message-ID: so do i include a path? or some other global directive to make the module linked or what? using idle here is a copy of the screen i will read docs if i get links............... thanx for any guidance i found this but i have not used sudo " sudo python setup.py install Once installed you should be able to import pyglet from any terminal without setting the PYTHONPATH. " Python 3.1.2 (r312:79147, Oct 23 2012, 20:07:58) [GCC 4.4.3] on linux2 Type "copyright", "credits" or "license()" for more information. ==== No Subprocess ==== >>> Traceback (most recent call last): File "/home/bob/import pyglet.py", line 1, in import pyglet ImportError: No module named pyglet >>> -------------- next part -------------- An HTML attachment was scrubbed... URL: From nathanschlaffer at gmail.com Mon Jul 8 07:43:25 2013 From: nathanschlaffer at gmail.com (Nathan Schlaffer) Date: Sun, 7 Jul 2013 22:43:25 -0700 Subject: [Tutor] New Python 3.3.2 Install Fails to Start-Up Message-ID: Dear Tutor, I have a Windows 7 Laptop. I installed Python Version 3.3.2 this morning and it worked fine. Then I tried to run a program someone else wrote (see below), and Python closed. Then I couldn't get Python to restart even after rebooting several times. I then uninstalled Python, rebooted, cleaned the Windows registry, and reinstalled Python from scratch. But I stilll couldn't start Python, so I tried uninstalling Python and using a restore point several times with no success. I still can't make Python run from the IDLE (Python GUI) on the start menu. Can anybody please help me? Second, I tried to run my Python programs today via IDLE but when I checked the Edit Menu I couldn't find a Run Menu to execute or run my program. How do I run a program once I have written the code in IDLE? Sincerely, Nathan Schlaffer E-mail: nathanschlaffer at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From bluepresley at gastonia.com Sun Jul 7 03:43:13 2013 From: bluepresley at gastonia.com (bluepresley) Date: Sun, 7 Jul 2013 10:43:13 +0900 Subject: [Tutor] Learning Python with Programming Collective Intelligence Message-ID: Hello, I discovered this list while doing research about the book Programming Collective Intelligence by Toby Segaran. In the archives, a few of the members wanted to create a group to discuss the code and algorithms further. Those threads were a few years old, and unfortunately I'm just now discovering the book. I'm learning Python (as quickly as possible) so I can better understand the algorithms in this book. If that group is still out there, the I'd like some more information about it. If not, I'd like to start digging in with some questions, but wanted to make sure this was the right list before I started. Thanks Blue -------------- next part -------------- An HTML attachment was scrubbed... URL: From tjhanson at yahoo.com Mon Jul 8 08:26:25 2013 From: tjhanson at yahoo.com (Tim Hanson) Date: Sun, 7 Jul 2013 23:26:25 -0700 Subject: [Tutor] learning nested functions Message-ID: <201307072326.25818.tjhanson@yahoo.com> In the first Lutz book, I am learning about nested functions. Here's the book's example demonstrating global scope: >>> def f1(): x=88 def f2(): print(x) f2() >>> f1() 88 No problem so far. I made a change and ran it again: >>> def f1(): x=88 def f2(): print(x) x=99 print(x) f2() >>> f1() Traceback (most recent call last): File "", line 1, in f1() File "", line 7, in f1 f2() File "", line 4, in f2 print(x) UnboundLocalError: local variable 'x' referenced before assignment This doesn't work. To my mind,in f2() I first print(x) then assign a variable with the same name in the local scope, then print the changed x. Why doesn't this work? From sandile.mnukwa at gmail.com Mon Jul 8 11:44:12 2013 From: sandile.mnukwa at gmail.com (Sandile Mnukwa) Date: Mon, 8 Jul 2013 11:44:12 +0200 Subject: [Tutor] Newbie. Please help Message-ID: I just started using Python recently, and i need help with the following: Please assist. 1. Create another function that generates a random number (You will have to import the relevant library to do this) 2. Create a function that is called from the main function, that accepts a number as a parameter and determines if the number is even or odd 3. Now enhance your script to generate 10 random even numbers and write them to a file So far what i have done is: import random def main(): pass if __name__ == '__main__': main() for evenNumber in range (0, 20, 2): print random.randrange(0, 101, 2) f = open("C:\Users\Documents\Myproj.txt", "w"); print f f = open("C:\Users\Documents\Myproj.txt", "a"); print f value = random.randrange(0, 101, 2, ) value1 = random.randrange(0, 201, 2, ) value2 = random.randrange(0, 301, 2, ) myString = str(value) myString1 = str(value1) myString2 = str(value2) print f.write(myString) print f.write(myString1) print f.write(myString2) f.close() Sent from my iPhone From paulrsmith7777 at gmail.com Tue Jul 9 22:00:47 2013 From: paulrsmith7777 at gmail.com (Paul Smith) Date: Tue, 9 Jul 2013 16:00:47 -0400 Subject: [Tutor] syntax error when attempting simple urllib.request.urlopen item Message-ID: Tutor- Ok newbie to coding here attempting to build controlled web scraper and have followed several books-tutorials and am failing at step one. This is the 3.3.1 code I am trying to run.. === import urllib.request htmltext = urllib.request.urlopen("http://google.com").read print htmltext === Other version... === #try py url open 7 import urllib.request res = urllib.request.urlopen('http://python.org/') html = res.read() print = html close res input = ("Press enter to exit") === so when I run what I think is proper 3.3.1 python code it hangs up with syntax error with the idle shell red highlighting the last print reference i.e. "htmltext" or "html". What is this humble newbie not getting? Best regards, Paul S -------------- next part -------------- An HTML attachment was scrubbed... URL: From rahn.no.kamikakushi at gmail.com Tue Jul 9 22:04:00 2013 From: rahn.no.kamikakushi at gmail.com (HRK) Date: Wed, 10 Jul 2013 08:04:00 +1200 Subject: [Tutor] Finding more efficient ways Message-ID: Hello, I'm pretty new to Python so while I can make some basic stuff work, it doesn't look particularly pretty. I wanted to find better ways to solve the following two problems (the current code works, but is clunky.) Any tips or help appreciated. Thanks! PROBLEM #1 heights = {"Mount Everest": 8848, "Mount Cook": 3754, "K2": 8611, "Ben Nevis": 1344, "Hekla": 1488} def mountain_height(heights, mountain): '''Return the height of the mountain. If the mountain isn't in the dictionary, the height returned should be -1. The search should ignore any spaces before or after the name. >>> mountain_height({'K2':8611, 'Hekla':1488}, "K2") 8611 >>> mountain_height({'K2':8611, 'Hekla':1488}, "Hekla ") 1488 >>> mountain_height({'K2':8611, 'Hekla':1488}, "Port Hills") -1 ''' if mountain.strip() in heights: return heights[mountain.strip()] else: return -1 print(mountain_height({'K2':8611, 'Hekla':1488}, "K2")) #8611 print(mountain_height({'K2':8611, 'Hekla':1488}, "Hekla ")) #1488 print(mountain_height({'K2':8611, 'Hekla':1488}, "Port Hills")) #-1 PROBLEM #2 def is_up_down(values): '''Return True if values is an up-down-sequence, otherwise return False. A sequence [x[0], x[1], x[2], ..., x[n]] is an up-down-sequence if there is an index k such that x[0] <= x[1] <= x[2] <= ... <= x[k-1] <= x[k] and x[k] >= x[k+1] >= x[k+2] >= ... >= x[n-1] >= x[n] hold. That is, the first part of the sequence is increasing and the second part is decreasing. Either part may be empty, so any increasing sequence and any decreasing sequence is an up-down-sequence. ''' flip = 0 value = 0 for item in values: if flip == 2: if item > value: return False elif flip == 1: if item < value: flip += 1 value = item elif flip == 0: if item > value: flip += 1 value = item return True print(is_up_down([2,5,5,7,9,9,8])) #True print(is_up_down([2,5,5,7,9,8,9])) #False print(is_up_down([9,8])) #True print(is_up_down([13])) #True From kwpolska at gmail.com Wed Jul 10 10:07:05 2013 From: kwpolska at gmail.com (=?UTF-8?B?Q2hyaXMg4oCcS3dwb2xza2HigJ0gV2Fycmljaw==?=) Date: Wed, 10 Jul 2013 10:07:05 +0200 Subject: [Tutor] Difference between max(one, two) and max((one, two)) In-Reply-To: References: Message-ID: On Fri, Jul 5, 2013 at 8:37 PM, Amandeep Behl wrote: > and why with sum(one, two) we get an error whereas not with sum((one, two)) > ? > > > On Fri, Jul 5, 2013 at 11:27 AM, Amandeep Behl > wrote: >> >> What is the difference between max(one, two) and max((one, two)) ? >> >> Thanks >> Aman > > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > First off, please do not top-post. The answer to your questions is in the documentation: [0] and [1]. The difference between max(1, 2) and max((1, 2)) is none, pretty much. max() takes either multiple arguments or an iterable, while sum() takes only an iterable (and both take an optional argument for some specific use). [0]: http://docs.python.org/2/library/functions.html#max [1]: http://docs.python.org/2/library/functions.html#sum -- Kwpolska | GPG KEY: 5EAAEA16 stop html mail | always bottom-post http://asciiribbon.org | http://caliburn.nl/topposting.html From hugo.yoshi at gmail.com Wed Jul 10 10:18:08 2013 From: hugo.yoshi at gmail.com (Hugo Arts) Date: Wed, 10 Jul 2013 10:18:08 +0200 Subject: [Tutor] learning nested functions In-Reply-To: <201307072326.25818.tjhanson@yahoo.com> References: <201307072326.25818.tjhanson@yahoo.com> Message-ID: On Mon, Jul 8, 2013 at 8:26 AM, Tim Hanson wrote: > In the first Lutz book, I am learning about nested functions. > > Here's the book's example demonstrating global scope: > >>> def f1(): > x=88 > def f2(): > print(x) > f2() > > > >>> f1() > 88 > > No problem so far. I made a change and ran it again: > > >>> def f1(): > x=88 > def f2(): > print(x) > x=99 > print(x) > f2() > > > >>> f1() > Traceback (most recent call last): > File "", line 1, in > f1() > File "", line 7, in f1 > f2() > File "", line 4, in f2 > print(x) > UnboundLocalError: local variable 'x' referenced before assignment > > This doesn't work. To my mind,in f2() I first print(x) then assign a > variable > with the same name in the local scope, then print the changed x. Why > doesn't > this work? > This doesn't work because, if you assign a local variable anywhere in a function, python assumes that all variables with that name in the entire function refer to the one in local scope. In other words, you can't have two variables with the same name from different scopes exist in one function. The reasons behind this are partly technical, and partly philosophical. Technically, the frame that contains all of a function's local names is created when you enter that function, which means that even when the local 'x' has not yet been assigned it exists in the frame, masking the global variable. Philosophically, using two identical names from different scopes leads to difficult to understand code that is very easy to introduce subtle bugs into. You may either use a 'global x' statement to indicate to python that you're using the name from the global scope, or pass the value of the global in through a function argument and use the local name. HTH, Hugo -------------- next part -------------- An HTML attachment was scrubbed... URL: From fomcl at yahoo.com Wed Jul 10 10:32:14 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Wed, 10 Jul 2013 01:32:14 -0700 (PDT) Subject: [Tutor] Best way to setup Unit Testing? In-Reply-To: References: Message-ID: <1373445134.47653.YahooMailNeo@web163801.mail.gq1.yahoo.com> _______________________________ >From: Srinivas Nyayapati >To: tutor at python.org >Sent: Sunday, July 7, 2013 10:16 PM >Subject: [Tutor] Best way to setup Unit Testing? > > > >I am tryng to figure out the best way to do Unit Testing for all my projects going forward. I am using the unittest framework. For execution I am executing the test cases by either directly using python or by using the nose test runner. I found the following 3 types of how this can be setup. > > >Type 1 >====== > > >The unit tests are located in the same package as the module being tested. This is the easiest to implement and run. I don't have to do anything special to run the tests. The directory structure is like this > > >Projects/ >? ? MyApp/ >? ? ? ? __init__.py >? ? ? ? myapp.py >? ? ? ? test_myapp.py > > >Simply run the test from within the MyApp folder as > > >% python test_myapp.py >or use the nose test runner > > >% nosetests > > >Type 2 >====== > > >Here the unit tests are located in its own tests folder in the package you want to test. Running the unit tests from inside the tests folder won't work as it worked in Type 1. That's because the myapp module can not be resolved. Here is how the directory structure looks like > > >Projects/ >? ? MyApp/ >? ? ? ? __init__.py >? ? ? ? myapp.py >? ? ? ? tests/ >? ? ? ? ? ? __init__.py >? ? ? ? ? ? test_myapp.py >It will work if you ran it from inside the package folder and gave the complete path to the test file as > > >% python tests/test_myapp.py >It will also work if you used the nose test runner from inside MyApp. > > >% nosetests >It will also work from inside the tests folder if you added MyApp to your PYTHONPATH, but that is an option I dont like. I don't want to keep changing my PYTHONPATH. > > >Type 3 >====== > > >This is where the tests folder is on the same level as the package folder. You can not run the test scripts from within the tests folder as again it will not be able to resolve the myapp package. Here is how the directory structure looks like > > >Projects/ >? ? MyApp/ >? ? ? ? __init__.py >? ? ? ? myapp.py >? ? tests/ >? ? ? ? __init__.py >? ? ? ? test_myapp.py > > >You can run the tests from within the MyApp package by referencing the tests folder as > > >% python ../tests/test_myapp.py > > >As you are running the test from the package, the myapp module is automatically in your PYTHONPATH. The other way of doing this would be by putting your package in your PYTHONPATH, which again as I mentioned earlier I don't like to do. > > >What is the best way of doing this? I would really appreciate to know how the experts are doing this. Is there anything I need to consider before I start using test suites.? Hi, sorry, I am not an expert but I can tell you how I did it. I would certainly put the tests in a separate folder. I used a? [nosetests] section in setup.cfg to tell nose (IIRC?using where=) where the tests live (with a relative path). From phil_lor at bigpond.com Wed Jul 10 10:41:22 2013 From: phil_lor at bigpond.com (Phil) Date: Wed, 10 Jul 2013 18:41:22 +1000 Subject: [Tutor] Reading numbers from a text file Message-ID: <51DD1E32.7040508@bigpond.com> Thank you for reading this. Kububtu 13.04 Python 3 I'm attempting to read a configuration file that will restore my program to the state that it was in when it was closed. Ideally the config file would be human readable. For this example I want to save and then read back the variables x and y. The variables are written to the file correctly. with open("config_file", "w") as file: x = 12 y = 67 file.write("%d %d\n" % (x, y)) This is my best attempt at reading back the variables. As it stands, myvars = "12". If I change "[0]" to "[1]" then, of course I have the second variable. How do I over come this seemingly simple problem? with open("config_file", "r") as file: myvars = file.read().splitlines() myvars = [i.split(" ")[0] for i in myvars] Maybe I would be better off continuing with pickle? This works perfectly except the file is in a binary format and, although I haven't spent a lot of time on it, I haven't managed to get it working in a text format. import pickle numbers = [12,98] pickle.dump(numbers, open("save_num", "wb")) nums = pickle.load(open("save_num", "rb")) print(nums[0]) print(nums[1]) -- Regards, Phil From hugo.yoshi at gmail.com Wed Jul 10 10:54:21 2013 From: hugo.yoshi at gmail.com (Hugo Arts) Date: Wed, 10 Jul 2013 10:54:21 +0200 Subject: [Tutor] syntax error when attempting simple urllib.request.urlopen item In-Reply-To: References: Message-ID: On Tue, Jul 9, 2013 at 10:00 PM, Paul Smith wrote: > Tutor- > > Ok newbie to coding here attempting to build controlled web scraper and > have followed several books-tutorials and am failing at step one. > > This is the 3.3.1 code I am trying to run.. > === > import urllib.request > > htmltext = urllib.request.urlopen("http://google.com").read > > print htmltext > === > Other version... > === > #try py url open 7 > > import urllib.request > res = urllib.request.urlopen('http://python.org/') > html = res.read() > print = html > close res > input = ("Press enter to exit") > === > > so when I run what I think is proper 3.3.1 python code it hangs up with > syntax error with the idle shell red highlighting the last print reference > i.e. "htmltext" or "html". > > What is this humble newbie not getting? > > > I'm fairly certain that what you're missing is that in python 3.x, print is no longer a statement but a function. The correct line would be "print(htmltext)". HTH, Hugo -------------- next part -------------- An HTML attachment was scrubbed... URL: From oscar.j.benjamin at gmail.com Wed Jul 10 12:07:18 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 10 Jul 2013 11:07:18 +0100 Subject: [Tutor] I need help on this program please In-Reply-To: <1373058476.24771.YahooMailNeo@web161804.mail.bf1.yahoo.com> References: <1373058476.24771.YahooMailNeo@web161804.mail.bf1.yahoo.com> Message-ID: On 5 July 2013 22:07, Ashley Fowler wrote: > This is what I have to do : It would be good to explain what you want help with. Does your program work? Are you just looking to improve it? [snip] > > def main(): > > digits = ["2", "3", "4", "5", "6", "7", "8", "9"] > > numeric_phone=" " > phone_number = input('Enter 10-Digit phone number: ') > ch= digits[index] I guess this line gives you an error. It would be good to show the error if you get one. You probably want to move this line into the loop after you set the value of index. > for ch in phone_number: > if ch.isalpha(): > if ch == 'A' or ch == 'B' or ch == 'C': It's a bit easier to do: if ch in 'A', 'B', 'C': Also what about lower case letters e.g. 'a'? > index = 0 > > if ch == 'D' or ch == 'E' or ch == 'F': > index = 1 > > if ch == 'G' or ch == 'H' or ch == 'I': > index = 2 > > if ch == 'J' or ch == 'K' or ch == 'L': > index = 3 > > if ch == 'M' or ch == 'N' or ch == 'O': > index = 4 > > if ch == 'P' or ch == 'Q' or ch == 'R' or ch == 'S': > index = 5 > > if ch == 'T' or ch == 'U' or ch == 'V': > index = 6 > > if ch == 'W' or ch == 'X' or ch == 'Y' or ch == 'Z': > index = 7 > > numeric_phone= numberic_phone+ch > > print(numeric_phone) > > This is what I have so far. Can anyone make suggestions or tell me what I > need to correct? My suggestion is to make a dict that maps characters to digits e.g.: digits = { '1': '1', '2': '2', ..., 'A': '2', 'B': '2', ... } Then you can find the correct character with digits[ch] instead of checking against each letter bunch separately. If you think that something needs correcting then you should say what happens when you run the code and why it isn't what you want to happen. Oscar From oscar.j.benjamin at gmail.com Wed Jul 10 12:17:15 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 10 Jul 2013 11:17:15 +0100 Subject: [Tutor] Newbie. Please help In-Reply-To: References: Message-ID: On 8 July 2013 10:44, Sandile Mnukwa wrote: > I just started using Python recently, and i need help with the following: Please assist. > > 1. Create another function that generates a random number (You will have to import the relevant library to do this) You should do these steps one at a time. You have code for step 3 but you haven't finished step 1 yet. If it says create a function then you need to write something like def my_function_name(): random_number = return random_number Since you haven't done that you haven't done step 1. > 2. Create a function that is called from the main function, that accepts a number as a parameter and determines if the number is even or odd Your main function is empty. You're not calling anything from it. > 3. Now enhance your script to generate 10 random even numbers and write them to a file Don't attempt this step until you've got the others sorted. > > So far what i have done is: > import random > def main(): > pass The main function above is empty (it doesn't do anything). You need to replace the line that says 'pass' with other lines of code that do whatever your program is supposed to do. > > if __name__ == '__main__': > main() Here you call the main function but it doesn't do anything since the function is empty. The code below is not part of your main function. You probably want to move it in there (in place of the line that says 'pass'). > for evenNumber in range (0, 20, 2): > print random.randrange(0, 101, 2) > > f = open("C:\Users\Documents\Myproj.txt", "w"); > print f > f = open("C:\Users\Documents\Myproj.txt", "a"); > print f You only need to open the file once in 'w' mode. > > value = random.randrange(0, 101, 2, ) > value1 = random.randrange(0, 201, 2, ) > value2 = random.randrange(0, 301, 2, ) > > myString = str(value) > myString1 = str(value1) > myString2 = str(value2) > > print f.write(myString) > print f.write(myString1) > print f.write(myString2) You don't need to use 'print' on the lines above since you're writing to a file and print is for showing things in the terminal. Although you could use print to write to the file e.g.: print >> f, myString If you do that then there's no need to convert the values into strings with 'str' above. > f.close() Oscar From oscar.j.benjamin at gmail.com Wed Jul 10 12:25:15 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 10 Jul 2013 11:25:15 +0100 Subject: [Tutor] Reading numbers from a text file In-Reply-To: <51DD1E32.7040508@bigpond.com> References: <51DD1E32.7040508@bigpond.com> Message-ID: On 10 July 2013 09:41, Phil wrote: > Thank you for reading this. > > Kububtu 13.04 Python 3 > > I'm attempting to read a configuration file that will restore my program to > the state that it was in when it was closed. Ideally the config file would > be human readable. > > For this example I want to save and then read back the variables x and y. > The variables are written to the file correctly. > > with open("config_file", "w") as file: > x = 12 > y = 67 > file.write("%d %d\n" % (x, y)) > > > This is my best attempt at reading back the variables. As it stands, myvars > = "12". If I change "[0]" to "[1]" then, of course I have the second > variable. How do I over come this seemingly simple problem? > > with open("config_file", "r") as file: > myvars = file.read().splitlines() > > myvars = [i.split(" ")[0] for i in myvars] If you just remove the "[0]" from the line above then i.split(" ") will return a list of both x and y. Then you can do e.g.: x, y = myvars[0] > Maybe I would be better off continuing with pickle? That depends how complicated your data really is. Another possibility is json e.g.: >>> x = 1 >>> y = 2 >>> variables = {'x': x, 'y': y} >>> variables {'y': 2, 'x': 1} >>> import json >>> json.dumps(variables, indent=4) '{\n "y": 2, \n "x": 1\n}' >>> print(json.dumps(variables, indent=4)) { "y": 2, "x": 1 } >>> stringdata = json.dumps(variables, indent=4) >>> json.loads(stringdata) {u'y': 2, u'x': 1} That's relatively readable and works for simple data. > > This works perfectly except the file is in a binary format and, although I > haven't spent a lot of time on it, I haven't managed to get it working in a > text format. > > import pickle > numbers = [12,98] > pickle.dump(numbers, open("save_num", "wb")) Instead of the above you should use: with open('save_num', 'wb') as savefile: pickle.dump(numbers, savefile) This ensures that the file is closed properly and that all the data gets written. Oscar From davea at davea.name Wed Jul 10 12:45:15 2013 From: davea at davea.name (Dave Angel) Date: Wed, 10 Jul 2013 06:45:15 -0400 Subject: [Tutor] converting string to text In-Reply-To: <20130628230847.WFUBY.26433.root@cdptpa-web20-z02> References: <20130628230847.WFUBY.26433.root@cdptpa-web20-z02> Message-ID: On 06/28/2013 07:08 PM, grubber at roadrunner.com wrote: > I don't get why this doesn't work. Its a very small program to read in a group of numbers. I've attached the the python file and my test file. The test file just contains one line. The error I get is that it can't convert the string to a float, but its a valid number. > Good job reducing the problem to a pretty small sample. It'd also be nice if you had included the code and data inline, since not everyone can get attachments, or is willing to open them if they do. Please specify Python version and give the complete error message (including traceback), not a paraphrased error. Get rid of the BOM from the data file, and it'll work fine. You don't specify what version of Python you're using, so I have to guess. But there's a utf-8 BOM conversion of a BOM at the beginning of that file, and that's not numeric. Best would be to change the way you generate that file, and don't put in a BOM for utf-8. BOM's are markers that are put at the beginning of certain encodings of files to distinguish between BE and LE encodings. But since your file is utf-8, a BOM is unnecessary and confusing. It may even be illegal, but I'm not sure about that. When in doubt about this sort of thing, use a hex dumper to examine the file. But in this case, all you had to do with to print repr(line) and you would have seen: '\xef\xbb\xbf1.0000000 0.0000000 0.0000000\n' The BOM is by definition a single Unicode code point. But once encoded to utf-8, it's 3 bytes long. You appear to be using Python 2.7, since the string module doesn't contain the split function in 3.3. Therefore you're reading it into a byte string, and the BOM is exactly 3 bytes. So remove the first 3 bytes of lines[0]. You probably want to specifically use a startswith() method to make sure those 3 bytes ARE a BOM, in case some of your files have it and some do not. BTW, there are other comments I could make about the source. But the main concern is that you're mixing tabs and spaces. Nasty bugs can occur that way. -- DaveA From oscar.j.benjamin at gmail.com Wed Jul 10 12:47:52 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 10 Jul 2013 11:47:52 +0100 Subject: [Tutor] Best way to setup Unit Testing? In-Reply-To: References: Message-ID: On 7 July 2013 21:16, Srinivas Nyayapati wrote: > I am tryng to figure out the best way to do Unit Testing for all my projects > going forward. Are you the only one using these applications or are you intending that other people would install them? Do you want other people to run the tests on their machines when they install? Will other people also work on the development of these projects and will they also need to run the tests? > I am using the unittest framework. For execution I am > executing the test cases by either directly using python or by using the > nose test runner. I found the following 3 types of how this can be setup. > > Type 1 > ====== > > The unit tests are located in the same package as the module being tested. > This is the easiest to implement and run. I don't have to do anything > special to run the tests. The directory structure is like this > > Projects/ > MyApp/ > __init__.py > myapp.py > test_myapp.py The (possible) advantage of this setup is that a user can run tests on the application while it is installed using: $ python -m MyApp.test_myapp The above could get messy if the number of submodules increases though. > > Simply run the test from within the MyApp folder as > > % python test_myapp.py > or use the nose test runner > > % nosetests > > Type 2 > ====== > > Here the unit tests are located in its own tests folder in the package you > want to test. Running the unit tests from inside the tests folder won't work > as it worked in Type 1. That's because the myapp module can not be resolved. > Here is how the directory structure looks like > > Projects/ > MyApp/ > __init__.py > myapp.py > tests/ > __init__.py > test_myapp.py This gives a nice clean distinction between your main code and your tests. It is also useful if you don't want to install the tests on the target machines since you can just exclude the tests package in your setup.py when creating an sdist. If you do include the tests and add a __main__.py in the tests folder then users can still run tests with python -m MyApp.tests > It will work if you ran it from inside the package folder and gave the > complete path to the test file as > > % python tests/test_myapp.py > It will also work if you used the nose test runner from inside MyApp. > > % nosetests > It will also work from inside the tests folder if you added MyApp to your > PYTHONPATH, but that is an option I dont like. I don't want to keep changing > my PYTHONPATH. You can always just do that in a script. I would normally have a Makefile and invoke tests with $ make test Inside the Makefile you can do test: PYTHONPATH=src nosetest Then you don't need to fiddle around with PYTHONPATH manually. My preference is that 'make test' should be run from the top-level directory of the project (if you're using version control then that's the main folder that gets created in a checkout/clone). Someone with the code should be able to do $ cd SomeProject $ make test ........ 121 out of 121 test passed. > > Type 3 > ====== > > This is where the tests folder is on the same level as the package folder. > You can not run the test scripts from within the tests folder as again it > will not be able to resolve the myapp package. Here is how the directory > structure looks like > > Projects/ > MyApp/ > __init__.py > myapp.py > tests/ > __init__.py > test_myapp.py In this setup you are not intending for the tests to be installed on the target machines and you have a clear separation between your main code and your tests. > > You can run the tests from within the MyApp package by referencing the tests > folder as > > % python ../tests/test_myapp.py > > As you are running the test from the package, the myapp module is > automatically in your PYTHONPATH. The other way of doing this would be by > putting your package in your PYTHONPATH, which again as I mentioned earlier > I don't like to do. > > What is the best way of doing this? I would really appreciate to know how > the experts are doing this. Is there anything I need to consider before I > start using test suites. Whatever you do just use a script or a Makefile or something so that invoking the tests is a simple one-liner e.g. $ make test $ python runtests.py $ ./test.sh or whatever suits you. If you need to fiddle with PYTHONPATH do it in that script, rather than manually. Oscar From davea at davea.name Wed Jul 10 12:53:44 2013 From: davea at davea.name (Dave Angel) Date: Wed, 10 Jul 2013 06:53:44 -0400 Subject: [Tutor] Difference between max(one, two) and max((one, two)) In-Reply-To: References: Message-ID: On 07/05/2013 02:37 PM, Amandeep Behl wrote: > and why with sum(one, two) we get an error whereas not with sum((one, two)) > ? > When replying to a message, your new text should go AFTER the quote. Putting it first is called top-posting, and it's a Microsoft-introduced abomination that's not compatible with newsgroups. > > On Fri, Jul 5, 2013 at 11:27 AM, Amandeep Behl wrote: > >> What is the difference between max(one, two) and max((one, two)) ? >> When you call max(one, two) you are passing two arguments to max(). The second way you are calling it with one argument, and that argument is a tuple. It happens that max() is happy either way. But for most functions, such as sum(), it makes a big difference. And even with max(), if you had included other arguments, such as keyword argument key=, you would have seen a difference. -- DaveA From davea at davea.name Wed Jul 10 13:39:07 2013 From: davea at davea.name (Dave Angel) Date: Wed, 10 Jul 2013 07:39:07 -0400 Subject: [Tutor] Help Please In-Reply-To: <1373058639.29054.YahooMailNeo@web161801.mail.bf1.yahoo.com> References: <1373058639.29054.YahooMailNeo@web161801.mail.bf1.yahoo.com> Message-ID: On 07/05/2013 05:10 PM, Ashley Fowler wrote: > > > > This is what I have so far. Can anyone make suggestions or tell me what I need to correct? > * > * First thing to correct is the notion that you're due an instant answer. You get frustrated after 3 minutes, and post a new message in a new thread, with slightly different (but still useless) subject line, and the new message is slightly different than the first. People here are volunteers, and you should wait a minimum of 12 hours before reposting. And then it should be a bump, on the same thread, not a new message. And the new message may have corrections to the first, but do them as corrections, not a whole new text to read. So next time: Post a single message, send it as TEXT, not html, pick a subject line that has something to do with the content (90% of the threads here are asking for help, so the word help contributes nothing). Specify your python version, specify what happened when you ran it: 1) I expected this... 2) Instead the program did this... or I ran it with these arguments, and got this exception, including traceback. Thanks. -- DaveA From davea at davea.name Wed Jul 10 13:52:48 2013 From: davea at davea.name (Dave Angel) Date: Wed, 10 Jul 2013 07:52:48 -0400 Subject: [Tutor] learning nested functions In-Reply-To: <201307072326.25818.tjhanson@yahoo.com> References: <201307072326.25818.tjhanson@yahoo.com> Message-ID: On 07/08/2013 02:26 AM, Tim Hanson wrote: > In the first Lutz book, I am learning about nested functions. > > Here's the book's example demonstrating global scope: >>>> def f1(): > x=88 > def f2(): > print(x) > f2() > > >>>> f1() > 88 > But that's not global scope, it's non-local scope. Global would be if you were referencing a symbol defined at top-level. > No problem so far. I made a change and ran it again: > >>>> def f1(): > x=88 > def f2(): > print(x) > x=99 > print(x) > f2() > > >>>> f1() > Traceback (most recent call last): > File "", line 1, in > f1() > File "", line 7, in f1 > f2() > File "", line 4, in f2 > print(x) > UnboundLocalError: local variable 'x' referenced before assignment > > This doesn't work. To my mind,in f2() I first print(x) then assign a variable > with the same name in the local scope, then print the changed x. Why doesn't > this work? As Hugo says, a function is compiled into a single entity, and within that entity, a given name is either local or not. It's decided by the presence (anywhere in the function) of an assignment, or a with statement, or maybe other name-binding statements. -- DaveA From davea at davea.name Wed Jul 10 14:05:37 2013 From: davea at davea.name (Dave Angel) Date: Wed, 10 Jul 2013 08:05:37 -0400 Subject: [Tutor] What's going on in this Python code from Programming Collective Intelligence? In-Reply-To: References: Message-ID: On 07/06/2013 11:18 PM, bluepresley wrote: > I'm reading the book Programming Collective Intelligence by Toby Segaran. > I'm having a lot of difficulty understanding the some of the code from > chapter four (the code for this chapter is available online at > https://github.com/cataska/programming-collective-intelligence-code/blob/master/chapter4/searchengine.pyand > starts at line172), specifically staring with this function: > > def getscoredlist(self,rows,wordids): > totalscores=dict([(row[0],0) for row in rows])# This is where you'll > later put the scoring functions > weights=[] > > for (weight,scores) in weights: > for url in totalscores: > totalscores[url]+=weight*scores[url] > > return totalscores > > What does this mean? > totalscores=dict([(row[0],0) for row in rows]) Two different things might be confusing you. It'd help to refactor it, and explain which line isn't clear. mylist = [(row[0],0) for row in rows] totalscores = dict(mylist) del mylist I'll guess it's the list comprehension that confuses you. That line is roughly equivalent to: mylist = [] for row in rows: mylist.append( (row[0], row) ) So you end up with a list of tuples, each consisting of row[0] and row The call to dict simply processes that list and makes a dict out of it. The first item in each tuple is the key, and the second item is the value. > > I have a lot of experience with PHP and Objective C. If anyone is familiar > with PHP or another language could you please provide the equivalent? I > think that would really help me understand better. > > > The function just before that, getmatchingrows, provides the arguments for > getscoredlist. "rows" is rows from a database query; "wordids" is a list of > word ids searched for that generated rows result set. For that function > (getmatchingrows) it returns 2 variables simultaneously. I'm unfamiliar > with this. What's going on there? I didn't download the link, so I'll just make up a function as an example: def myfunc(): return 12, 42 That comma between the two values creates a tuple out of them. The tuple is normally written (12, 42), and is a immutable version of the list [12, 42] If somebody calls the function like: a = myfunc() then a will be that same tuple. But if somebody calls the function like: c, d = myfunc() then a symmetric trick called "tuple unpacking" goes into effect. When you have a bunch of variables on the left side of an assignment, it'll unpack whatever iterable is being passed. So in this case, c will be 12, and d will be 42. This same syntax works in any other context, so you can do: a, b = b, a to swap two values. > > Also, as far as I can tell from the getmatchingrows code, it returns a > multidimensional array of database results with row[0] being the urlid (NOT > the url), and other indices correspond to the word id location. > > In getscoredlist, totalscores[url] doesn't make sense. Where is [url] > coming from? could they have meant to say urlid here? > > This chapter is also available online for free from O'reilly. Here is the > page that talks specifically about this part. > > Any help understanding what this code in this part of this book is doing > would be greatly appreciated. > > Thanks, > Blue > > http://my.safaribooksonline.com/book/web-development/9780596529321/4dot-searching-and-ranking/querying#X2ludGVybmFsX0h0bWxWaWV3P3htbGlkPTk3ODA1OTY1MjkzMjElMkZjb250ZW50YmFzZWRfcmFua2luZyZxdWVyeT0= > > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > -- DaveA From davea at davea.name Wed Jul 10 14:11:46 2013 From: davea at davea.name (Dave Angel) Date: Wed, 10 Jul 2013 08:11:46 -0400 Subject: [Tutor] New Python 3.3.2 Install Fails to Start-Up In-Reply-To: References: Message-ID: On 07/08/2013 01:43 AM, Nathan Schlaffer wrote: > Dear Tutor, > > I have a Windows 7 Laptop. I installed Python Version 3.3.2 this morning > and it worked fine. Then I tried to run a program someone else wrote (see > below), and Python closed. What's that mean? HOW did you try to run the other program? If you're doing it from within IDLE, then I cannot help, as IDLE makes no sense to me. Please distinguish "Python closed" from "IDLE did ????" > Then I couldn't get Python to restart Again, what did you do to try to start Python? c:\mydir > python3.3 or smething else? > even after > rebooting several times. I then uninstalled Python, rebooted, cleaned the > Windows registry, At that point, nothing runs and you have to reinstall Windows. > and reinstalled Python from scratch. But I stilll > couldn't start Python, so I tried uninstalling Python and using a restore > point several times with no success. I still can't make Python run from the > IDLE (Python GUI) on the start menu. Can anybody please help me? Get Python running, and only then consider using IDLE. > > Second, I tried to run my Python programs today via IDLE but when I checked > the Edit Menu I couldn't find a Run Menu to execute or run my program. How > do I run a program once I have written the code in IDLE? > -- DaveA From davea at davea.name Wed Jul 10 14:52:20 2013 From: davea at davea.name (Dave Angel) Date: Wed, 10 Jul 2013 08:52:20 -0400 Subject: [Tutor] Finding more efficient ways In-Reply-To: References: Message-ID: On 07/09/2013 04:04 PM, HRK wrote: > Hello, > I'm pretty new to Python so while I can make some basic stuff work, it doesn't look particularly pretty. I wanted to find better ways to solve the following two problems (the current code works, but is clunky.) Any tips or help appreciated. Thanks! > In your subject line you say "efficient" but here you say "(less) clunky". Which is it? For the first assignment, it's almost as simple as it's going to get, though you could eliminate the if test by using the dict.get() method. http://docs.python.org/2/library/stdtypes.html?highlight=dict.get#dict.get > PROBLEM #1 > > heights = {"Mount Everest": 8848, "Mount Cook": 3754, "K2": 8611, > "Ben Nevis": 1344, "Hekla": 1488} > > def mountain_height(heights, mountain): > '''Return the height of the mountain. > If the mountain isn't in the dictionary, the > height returned should be -1. > The search should ignore any spaces before or after the name. > >>> mountain_height({'K2':8611, 'Hekla':1488}, "K2") > 8611 > >>> mountain_height({'K2':8611, 'Hekla':1488}, "Hekla ") > 1488 > >>> mountain_height({'K2':8611, 'Hekla':1488}, "Port Hills") > -1 > ''' > if mountain.strip() in heights: > return heights[mountain.strip()] > else: > return -1 > > print(mountain_height({'K2':8611, 'Hekla':1488}, "K2")) > #8611 > print(mountain_height({'K2':8611, 'Hekla':1488}, "Hekla ")) > #1488 > print(mountain_height({'K2':8611, 'Hekla':1488}, "Port Hills")) > #-1 > > > PROBLEM #2 > > def is_up_down(values): > '''Return True if values is an up-down-sequence, otherwise > return False. > A sequence [x[0], x[1], x[2], ..., x[n]] is an up-down-sequence > if there is an index k such that > x[0] <= x[1] <= x[2] <= ... <= x[k-1] <= x[k] and > x[k] >= x[k+1] >= x[k+2] >= ... >= x[n-1] >= x[n] hold. > That is, the first part of the sequence is increasing and the > second part is decreasing. > Either part may be empty, so any increasing sequence and any > decreasing sequence is an up-down-sequence. > ''' > flip = 0 > value = 0 > for item in values: > if flip == 2: > if item > value: > return False > elif flip == 1: > if item < value: > flip += 1 > value = item > elif flip == 0: Usually the lst elif should be spelled "else". Or you should add another else afterwards. Or add an assert() to double-check your logic. > if item > value: > flip += 1 > value = item > return True > > print(is_up_down([2,5,5,7,9,9,8])) > #True > print(is_up_down([2,5,5,7,9,8,9])) > #False > print(is_up_down([9,8])) > #True > print(is_up_down([13])) > #True Here, your code may be about as fast as it's going to get. But it's very hard to follow, which may be what you mean by 'clunky.' If it were my problem, I'd make the code much slower by defining a second function that checks whether a sequence is monotonic, and if so which way. Then the main function will simply loop through the sequence, checking the head and tail with the second function. As soon as it reaches a point where the head and tail are both monotonic, and with the opposite direction, you're done. -- DaveA From davea at davea.name Wed Jul 10 14:58:13 2013 From: davea at davea.name (Dave Angel) Date: Wed, 10 Jul 2013 08:58:13 -0400 Subject: [Tutor] syntax error when attempting simple urllib.request.urlopen item In-Reply-To: References: Message-ID: On 07/09/2013 04:00 PM, Paul Smith wrote: > Tutor- > > Ok newbie to coding here attempting to build controlled web scraper and > have followed several books-tutorials and am failing at step one. > > This is the 3.3.1 code I am trying to run.. > === > import urllib.request > > htmltext = urllib.request.urlopen("http://google.com").read > > print htmltext > === > Other version... > === > #try py url open 7 > > import urllib.request > res = urllib.request.urlopen('http://python.org/') > html = res.read() > print = html > close res > input = ("Press enter to exit") > === > > so when I run what I think is proper 3.3.1 python code it hangs up with > syntax error with the idle shell red highlighting the last print reference > i.e. "htmltext" or "html". > > What is this humble newbie not getting? > The first thing you're missing is that print() is a function in Python 3.3 So you need print(htmltext) Once you're past that, you'll find that you need parentheses if you want to call the read() method. -- DaveA From bgailer at gmail.com Wed Jul 10 15:05:04 2013 From: bgailer at gmail.com (bob gailer) Date: Wed, 10 Jul 2013 09:05:04 -0400 Subject: [Tutor] Convert SOAP response (ArrayOfInt) to Python list In-Reply-To: <51D5D134.6020506@bioprocess.org> References: <51D5D134.6020506@bioprocess.org> Message-ID: <51DD5C00.7050506@gmail.com> On 7/4/2013 3:47 PM, Robert Winkler wrote: > > Thanks to the OSA library, which works for SOAP requests with Python > 3.x, I can now use SOAP services at http://www.chemspider.com. > > The result is a list of accession numbers (which correspond to > chemical compounds) and I get them in the following format: > > (ArrayOfInt){int[]=[5744,69182,292,68027,3404131,82616,18280,11200,704646,543430...]} > > How could I transform this to a simple python list? > > [5744, 69182, 292,68027, 3404131, 82616, 18280, 11200, 704646, 543430 ...] > > Conversion to a numpy array How did you do that? > [snip] since the structure is maintained; the numpy.shape returns (). Huh? Makes no sense to me. Please explain. I assume "the following format" is a character string. How do you convert characters to integers - hint - int() So try applying int() to each accession number and collecting the results in a list. -- Bob Gailer 919-636-4239 Chapel Hill NC -------------- next part -------------- An HTML attachment was scrubbed... URL: From wprins at gmail.com Wed Jul 10 15:27:27 2013 From: wprins at gmail.com (Walter Prins) Date: Wed, 10 Jul 2013 14:27:27 +0100 Subject: [Tutor] using python for parsing In-Reply-To: References: Message-ID: Hello, On 29 June 2013 19:00, Makarand Datar wrote: > Hi, > > So I am able to read in a file, and write out a file from python. I have > some question about the text manipulation that I need to do between reading > in a file and spitting out another. Couple of things I would like to do, > are: if the first character of a line is a comma, I would like to delete > the newline character at the end of the previous line and make a longer > line (one line followed by another without a newline in between); if the > first character of the line is an exclamation mark, I would like to delete > that line altogether. So for instance, if the input file text looks like > the following, > > one, two, three > , four, five, six > ! This is a comment > seven, eight, nine > > I would like to output a file the text in which looks like > > one, two, three, four, five, six > seven, eight, nine > Try this: import re input_str = """ one, two, three , four, five, six ! This is a comment seven, eight, nine """ # Join lines ending in trailing commas: input_str = re.sub('\n,', '', input_str) # Remove comment lines starting with ! input_str = re.sub('!.*\n', '', input_str) print input_str Walter -------------- next part -------------- An HTML attachment was scrubbed... URL: From fomcl at yahoo.com Wed Jul 10 16:37:07 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Wed, 10 Jul 2013 07:37:07 -0700 (PDT) Subject: [Tutor] Best way to setup Unit Testing? In-Reply-To: References: Message-ID: <1373467027.45667.YahooMailNeo@web163805.mail.gq1.yahoo.com> ----- Original Message ----- > From: Oscar Benjamin > To: Srinivas Nyayapati > Cc: tutor at python.org > Sent: Wednesday, July 10, 2013 12:47 PM > Subject: Re: [Tutor] Best way to setup Unit Testing? > > On 7 July 2013 21:16, Srinivas Nyayapati wrote: >> I am tryng to figure out the best way to do Unit Testing for all my > projects >> going forward. > > Are you the only one using these applications or are you intending > that other people would install them? Do you want other people to run > the tests on their machines when they install? ? Shouldn't this always be possible? ? >Will other people also > work on the development of these projects and will they also need to > run the tests? > >> I am using the unittest framework. For execution I am >> executing the test cases by either directly using python or by using the >> nose test runner. I found the following 3 types of how this can be setup. >> >> Type 1 >> ====== >> >> The unit tests are located in the same package as the module being tested. >> This is the easiest to implement and run. I don't have to do anything >> special to run the tests. The directory structure is like this >> >> Projects/ >> ? ? MyApp/ >> ? ? ? ? __init__.py >> ? ? ? ? myapp.py >> ? ? ? ? test_myapp.py > > The (possible) advantage of this setup is that a user can run tests on > the application while it is installed using: > > $ python -m MyApp.test_myapp > > The above could get messy if the number of submodules increases though. > >> >> Simply run the test from within the MyApp folder as >> >> % python test_myapp.py >> or use the nose test runner >> >> % nosetests >> >> Type 2 >> ====== >> >> Here the unit tests are located in its own tests folder in the package you >> want to test. Running the unit tests from inside the tests folder won't > work >> as it worked in Type 1. That's because the myapp module can not be > resolved. >> Here is how the directory structure looks like >> >> Projects/ >> ? ? MyApp/ >> ? ? ? ? __init__.py >> ? ? ? ? myapp.py >> ? ? ? ? tests/ >> ? ? ? ? ? ? __init__.py >> ? ? ? ? ? ? test_myapp.py > > This gives a nice clean distinction between your main code and your > tests. It is also useful if you don't want to install the tests on the > target machines since you can just exclude the tests package in your > setup.py when creating an sdist. > > If you do include the tests and add a __main__.py in the tests folder > then users can still run tests with Nice, I didn't know this one! I found this useful: http://stackoverflow.com/questions/4042905/what-is-main-py ? > python -m MyApp.tests > >> It will work if you ran it from inside the package folder and gave the >> complete path to the test file as >> >> % python tests/test_myapp.py >> It will also work if you used the nose test runner from inside MyApp. >> >> % nosetests >> It will also work from inside the tests folder if you added MyApp to your >> PYTHONPATH, but that is an option I dont like. I don't want to keep > changing >> my PYTHONPATH. > > You can always just do that in a script. I would normally have a > Makefile and invoke tests with > > $ make test ? Isn't this specific for unix-like systems? ? ? From bgailer at gmail.com Wed Jul 10 16:57:11 2013 From: bgailer at gmail.com (bob gailer) Date: Wed, 10 Jul 2013 10:57:11 -0400 Subject: [Tutor] using python for parsing In-Reply-To: References: Message-ID: <51DD7647.9090206@gmail.com> with open(fileName2, 'w') as w: [ w.write(line+'\n') for line in open(fileName).read().replace('\n,', ',').split('\n') if not line.startswith('!')] -- Bob Gailer 919-636-4239 Chapel Hill NC From onyxtic at gmail.com Wed Jul 10 17:01:26 2013 From: onyxtic at gmail.com (Evans Anyokwu) Date: Wed, 10 Jul 2013 16:01:26 +0100 Subject: [Tutor] Convert SOAP response (ArrayOfInt) to Python list In-Reply-To: <51D5D134.6020506@bioprocess.org> References: <51D5D134.6020506@bioprocess.org> Message-ID: On Thu, Jul 4, 2013 at 8:47 PM, Robert Winkler < robert.winkler at bioprocess.org> wrote: > Thanks to the OSA library, which works for SOAP requests with Python > 3.x, I can now use SOAP services at http://www.chemspider.com. > > The result is a list of accession numbers (which correspond to chemical > compounds) and I get them in the following format: > > (ArrayOfInt){ > int[] = [ > 5744, > 69182, > 292, > 68027, > 3404131, > 82616, > 18280, > 11200, > 704646, > 543430 > ... > ]} > > How could I transform this to a simple python list? > > [5744, 69182, 292,68027, 3404131, 82616, 18280, 11200, 704646, 543430 ...] > > Conversion to a numpy array (and subsequent list(), or similar) does not > solve the problem, since the structure is maintained; the numpy.shape > returns (). > > Suggestions? > > Hi Robert, You have not provided enough information. But going by the little we have here, you could turn the returned result into a string. Then parse the string with a regex. I have included a sample here. There are more elegant ways to do this but here's a quick and dirty solution to give you a heads up. import sys import re retVal= '''(ArrayOfInt){ int[] = [ 5744, 69182, 292, 68027, 3404131, 82616, 18280, 11200, 704646, 543430, ] }''' myList = re.findall(r'\d+', retVal) aList = list(myList) print('[' + ', '.join(aList) + ']') That should give you the desired list. Unless of course I'm missing the point. Good luck, Evans -------------- next part -------------- An HTML attachment was scrubbed... URL: From kwpolska at gmail.com Wed Jul 10 17:16:39 2013 From: kwpolska at gmail.com (=?UTF-8?B?Q2hyaXMg4oCcS3dwb2xza2HigJ0gV2Fycmljaw==?=) Date: Wed, 10 Jul 2013 17:16:39 +0200 Subject: [Tutor] install on windows In-Reply-To: <51D37F34.9020609@seiboldsystems.com> References: <51D37F34.9020609@seiboldsystems.com> Message-ID: On Wed, Jul 3, 2013 at 3:32 AM, larry seibold wrote: > I am stuck at step one, installing "python-2.7.5.msi" on windows XP. I > downloaded it (~16MB), but when I select it (double click), I get a windows > installer pop up with an OK button at the bottom, which when selected seems > to terminate the install vs. install the program. I do not see any advice > on a different procedure. Help. Do I need to run the msi from a command > window with options? > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor What does the pop-up say? Are you using a limited account? -- Kwpolska | GPG KEY: 5EAAEA16 stop html mail | always bottom-post http://asciiribbon.org | http://caliburn.nl/topposting.html From kwpolska at gmail.com Wed Jul 10 17:20:48 2013 From: kwpolska at gmail.com (=?UTF-8?B?Q2hyaXMg4oCcS3dwb2xza2HigJ0gV2Fycmljaw==?=) Date: Wed, 10 Jul 2013 17:20:48 +0200 Subject: [Tutor] Best way to setup Unit Testing? In-Reply-To: References: Message-ID: On Sun, Jul 7, 2013 at 10:16 PM, Srinivas Nyayapati wrote: > I am tryng to figure out the best way to do Unit Testing for all my projects > going forward. I am using the unittest framework. For execution I am > executing the test cases by either directly using python or by using the > nose test runner. I found the following 3 types of how this can be setup. > > Type 1 > ====== > > The unit tests are located in the same package as the module being tested. > This is the easiest to implement and run. I don't have to do anything > special to run the tests. The directory structure is like this > > Projects/ > MyApp/ > __init__.py > myapp.py > test_myapp.py > > Simply run the test from within the MyApp folder as > > % python test_myapp.py > or use the nose test runner > > % nosetests It makes a mess and puts your code and tests together, which you should not do. > Type 2 > ====== > > Here the unit tests are located in its own tests folder in the package you > want to test. Running the unit tests from inside the tests folder won't work > as it worked in Type 1. That's because the myapp module can not be resolved. > Here is how the directory structure looks like > > Projects/ > MyApp/ > __init__.py > myapp.py > tests/ > __init__.py > test_myapp.py > It will work if you ran it from inside the package folder and gave the > complete path to the test file as > > % python tests/test_myapp.py > It will also work if you used the nose test runner from inside MyApp. > > % nosetests > It will also work from inside the tests folder if you added MyApp to your > PYTHONPATH, but that is an option I dont like. I don't want to keep changing > my PYTHONPATH. So don?t use this ugly thing and use: > Type 3 > ====== > > This is where the tests folder is on the same level as the package folder. > You can not run the test scripts from within the tests folder as again it > will not be able to resolve the myapp package. Here is how the directory > structure looks like > > Projects/ > MyApp/ > __init__.py > myapp.py > tests/ > __init__.py > test_myapp.py > > You can run the tests from within the MyApp package by referencing the tests > folder as > > % python ../tests/test_myapp.py Are you sure? Won?t that require % cd .. % python tests/test_myapp.py or other magic? Packages *may* not work otherwise. -- Kwpolska | GPG KEY: 5EAAEA16 stop html mail | always bottom-post http://asciiribbon.org | http://caliburn.nl/topposting.html From bodsda at googlemail.com Wed Jul 10 17:42:19 2013 From: bodsda at googlemail.com (Bod Soutar) Date: Wed, 10 Jul 2013 16:42:19 +0100 Subject: [Tutor] install on windows In-Reply-To: <51D37F34.9020609@seiboldsystems.com> References: <51D37F34.9020609@seiboldsystems.com> Message-ID: Right click the msi > run as admin On Wednesday, 3 July 2013, larry seibold wrote: > I am stuck at step one, installing "python-2.7.5.msi" on windows XP. I > downloaded it (~16MB), but when I select it (double click), I get a windows > installer pop up with an OK button at the bottom, which when selected seems > to terminate the install vs. install the program. I do not see any advice > on a different procedure. Help. Do I need to run the msi from a command > window with options? > > ______________________________**_________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/**mailman/listinfo/tutor > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan.gauld at btinternet.com Wed Jul 10 19:16:34 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 10 Jul 2013 18:16:34 +0100 Subject: [Tutor] install on windows In-Reply-To: <51D37F34.9020609@seiboldsystems.com> References: <51D37F34.9020609@seiboldsystems.com> Message-ID: On 03/07/13 02:32, larry seibold wrote: > I am stuck at step one, installing "python-2.7.5.msi" on windows XP. Are you sure you have the right version - 32bit v 64bit? Just a thought. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From marc.tompkins at gmail.com Wed Jul 10 19:19:02 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Wed, 10 Jul 2013 10:19:02 -0700 Subject: [Tutor] converting string to text In-Reply-To: References: <20130628230847.WFUBY.26433.root@cdptpa-web20-z02> Message-ID: On Wed, Jul 10, 2013 at 3:45 AM, Dave Angel wrote: > > Get rid of the BOM from the data file, and it'll work fine. You don't > specify what version of Python you're using, so I have to guess. But > there's a utf-8 BOM conversion of a BOM at the beginning of that file, and > that's not numeric. Best would be to change the way you generate that > file, and don't put in a BOM for utf-8. > > BOM's are markers that are put at the beginning of certain encodings of > files to distinguish between BE and LE encodings. But since your file is > utf-8, a BOM is unnecessary and confusing. Just jumping in to translate a bit of jargon... BOM stands for Byte Order Mark. ( http://www.opentag.com/xfaq_enc.htm#enc_bom) BE stands for "big-endian", and LE stands for "little-endian". Since the first digital computers were built, there have been two schools of thought as to how numbers should be stored: with the "most significant" digits first, or the "least significant" digits first. The two schools are called "big-endian" and "little-endian", after a famous controversy in "Gulliver's Travels". The BOM is a sequence of bytes at the beginning of a Unicode string that tells the reader whether the rest of the string will be big-endian or little-endian. UTF-8 was designed to be endian-agnostic, so a BOM is not actually needed. > It may even be illegal, but I'm not sure about that. > No, it's not illegal; when utf-8 was first introduced it was actually required. It's no longer required - so now even utf-8 comes in two flavors (with and without BOM)! -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan.gauld at btinternet.com Wed Jul 10 19:20:33 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 10 Jul 2013 18:20:33 +0100 Subject: [Tutor] What's going on in this Python code from Programming Collective Intelligence? In-Reply-To: References: Message-ID: On 10/07/13 13:05, Dave Angel wrote: > mylist = [(row[0],0) for row in rows] > > I'll guess it's the list comprehension that confuses you. That line is > roughly equivalent to: > mylist = [] > for row in rows: > mylist.append( (row[0], row) ) Except the second element in the tuple to append should be zero. > So you end up with a list of tuples, each consisting of row[0] and a zero Presumably this will be the starting score for each item. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From marc.tompkins at gmail.com Wed Jul 10 19:25:02 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Wed, 10 Jul 2013 10:25:02 -0700 Subject: [Tutor] Reading numbers from a text file In-Reply-To: <51DD1E32.7040508@bigpond.com> References: <51DD1E32.7040508@bigpond.com> Message-ID: On Wed, Jul 10, 2013 at 1:41 AM, Phil wrote: > Thank you for reading this. > > Kububtu 13.04 Python 3 > > I'm attempting to read a configuration file that will restore my program > to the state that it was in when it was closed. Ideally the config file > would be human readable. > > Python actually comes with a module for this - it's called "configparser": http://docs.python.org/2/library/configparser.html I personally use a module called "configobj" that I find slightly more powerful / easy to use than configparser: https://pypi.python.org/pypi/configobj/ Either way, you don't have to re-invent the wheel... -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan.gauld at btinternet.com Wed Jul 10 19:27:58 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 10 Jul 2013 18:27:58 +0100 Subject: [Tutor] New Python 3.3.2 Install Fails to Start-Up In-Reply-To: References: Message-ID: On 08/07/13 06:43, Nathan Schlaffer wrote: > I have a Windows 7 Laptop. I installed Python Version 3.3.2 this morning > and it worked fine. Then I tried to run a program someone else wrote > (see below) I didn't see anything related below. What program did you try to run? How did you try to run it? > , and Python closed. Then I couldn't get Python to restart You need to be clear whether you mean Python or IDLE. IDLE is a development tool written in Python. To run IDLE you need Python but to run Python you do not need IDLE. > stilll couldn't start Python, Python or IDLE? Can you be more specific about what you are doing? How exactly are you trying to 'start python'? > Second, I tried to run my Python programs today via IDLE but when I > checked the Edit Menu I couldn't find a Run Menu Which implies that you started IDLE. But IDLE requires Python so you must have solved the first question? > How do I run a program once I have written the code in IDLE? If you open a new Edit window and type your code (or open an existing file) then there should be a Run menu item. However that should only be used for testing, IDLE is only a development tool. To run a completed program you should only need to double click it in Windows Explorer (assuming all associations/permissions etc are OK) Of course what you see as a result will depend on the program - does it run in a console? Does it have a GUI? A Web front end? Is it interactive or does it just run and terminate? It all depends... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From alan.gauld at btinternet.com Wed Jul 10 19:39:28 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 10 Jul 2013 18:39:28 +0100 Subject: [Tutor] learning nested functions In-Reply-To: <201307072326.25818.tjhanson@yahoo.com> References: <201307072326.25818.tjhanson@yahoo.com> Message-ID: On 08/07/13 07:26, Tim Hanson wrote: > In the first Lutz book, I am learning about nested functions. >>>> def f1(): > x=88 > def f2(): > print(x) > x=99 > print(x) > f2() > > >>>> f1() > Traceback (most recent call last): > print(x) > UnboundLocalError: local variable 'x' referenced before assignment > > This doesn't work. To my mind,in f2() I first print(x) then assign a variable > with the same name in the local scope, then print the changed x. Why doesn't > this work? Hugo and Dave have given a technical answer. Here's a less technical one (or maybe just a different one!). Python does not execute your program line by line. It reads your program and builds up an internal structure of executable code and data. In your case it sees that the function f2 has a local variable x in it so it assigns that x to that function object in its internal structure. When you then execute f2 it uses the local x throughout, but because you try to print x before assigning it you get an error. If you want to use the "global" x you would need to declare it global (although your x is not strictly global - its just not local!). but that then prevents your assignment from creating a local x. In other words Python doesn't want you creating both local and global references in the same scope. Use a different name for your local variable. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From eryksun at gmail.com Wed Jul 10 19:42:27 2013 From: eryksun at gmail.com (eryksun) Date: Wed, 10 Jul 2013 13:42:27 -0400 Subject: [Tutor] Difference between max(one, two) and max((one, two)) In-Reply-To: References: Message-ID: On Wed, Jul 10, 2013 at 6:53 AM, Dave Angel wrote: > And even with max(), if you had included other arguments, such as keyword > argument key=, you would have seen a difference. CPython max() is declared METH_VARARGS | METH_KEYWORDS, so it's called with both an args tuple and a kwds dict. Using a keyword argument other than "key" will raise a TypeError. As far as unpacking the args tuple, if it's length 1 then the first argument is assumed to be an iterable. Otherwise the args tuple itself is used as the iterable. From alan.gauld at btinternet.com Wed Jul 10 19:50:46 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 10 Jul 2013 18:50:46 +0100 Subject: [Tutor] Finding more efficient ways In-Reply-To: References: Message-ID: On 09/07/13 21:04, HRK wrote: > PROBLEM #1 > > heights = {"Mount Everest": 8848, "Mount Cook": 3754, "K2": 8611, > "Ben Nevis": 1344, "Hekla": 1488} Since you already have a dict of heights why do you pass one in to the function each time? Why not just pass this one in? Or use it as a global? > def mountain_height(heights, mountain): > '''Return the height of the mountain. > If the mountain isn't in the dictionary, the > height returned should be -1. > The search should ignore any spaces before or after the name. > >>> mountain_height({'K2':8611, 'Hekla':1488}, "K2") > 8611 > >>> mountain_height({'K2':8611, 'Hekla':1488}, "Hekla ") > 1488 > >>> mountain_height({'K2':8611, 'Hekla':1488}, "Port Hills") > -1 > ''' > if mountain.strip() in heights: > return heights[mountain.strip()] > else: > return -1 The last 4 lines can be replaced by heights.get(.....) Check the dict documentation. > print(mountain_height({'K2':8611, 'Hekla':1488}, "K2")) > #8611 print( mountain_height(heights, 'K2') ) > PROBLEM #2 > > def is_up_down(values): > '''Return True if values is an up-down-sequence, otherwise > return False. This is more complex and I'm about to get busy, I might post a reply if I get time later and nobody else has... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From cybervigilante at gmail.com Wed Jul 10 21:02:16 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Wed, 10 Jul 2013 12:02:16 -0700 Subject: [Tutor] importing into a function Message-ID: Boy, is the list busy. Python must have been mentioned on USA today, or something ;') Anyway, I was reading the datamodel, and it mentioned imported modules going out of scope, which seemed odd, since I thought they were global and never went out, or am I thinking wrong? Anyway, I figured an import would go out of scope it if was in a function: #Using C:\Python33\python.exe on Win 7 in c:\python33\jimprogs def testimp(inp): import math return math.sqrt(inp) print(testimp(25)) # result - 5.0 print(math.sqrt(25)) # result error traceback, math undefined Sure enough it did. The question is - does importing into a function ever make sense? It occurred to me that if you use from module import * , putting it in a function protects you from invisible namespace pollution by a big module with a lot of names. But importing has to be a big overhead. So does importing into a function make sense at any time, or is it a dead end? -- Jim Although the darling of health faddists, there is no such thing as canola oil, since there is no canola plant - it's genetically modified rapeseed oil given a nicer name ;') From marc.tompkins at gmail.com Wed Jul 10 21:10:49 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Wed, 10 Jul 2013 12:10:49 -0700 Subject: [Tutor] New Python 3.3.2 Install Fails to Start-Up In-Reply-To: References: Message-ID: On Wed, Jul 10, 2013 at 10:27 AM, Alan Gauld wrote: > On 08/07/13 06:43, Nathan Schlaffer wrote: > > Second, I tried to run my Python programs today via IDLE but when I >> checked the Edit Menu I couldn't find a Run Menu >> > > IDLE has two modes: Shell and Edit. Shell mode is an interactive Python session; you can type commands (or blocks of commands) and they will be interpreted directly. Edit mode is a text editor with a few Python-specific features, such as different colors for statements/strings/etc. If you're in Shell mode and want to open a new Edit window, it's "File/New window" or Ctrl-N. If you're in Edit mode and want to run what you've written, it's "Run/Run Module" or F5. You'll be prompted to save first. Also: IDLE is an IDE (Integrated Development Environment), meaning that it's intended as a convenient way for you to write a little/run it/write a little more/run that/etc. (In my opinion, it's not even a very good IDE - but that's a whole 'nother conversation. It's just the free IDE that comes with Python.) IDLE is NOT, however, intended as the way you'd normally run your programs once they're ready for prime time. It uses a GUI module known as Tk, which can interfere big-time with GUI aspects of your own programs, and it introduces other funkiness into your environment. If you plan on distributing your programs for other people to use, you should definitely get used to running them in the bare Python interpreter. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Wed Jul 10 21:39:30 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Wed, 10 Jul 2013 12:39:30 -0700 Subject: [Tutor] Difference between max(one, two) and max((one, two)) In-Reply-To: References: Message-ID: On 5 July 2013 11:27, Amandeep Behl wrote: > What is the difference between max(one, two) and max((one, two)) ? Another clarification would be: >>> max((1,2),(1,3)) (1, 3) # a tuple, not an integer -- Jim From cybervigilante at gmail.com Wed Jul 10 21:47:07 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Wed, 10 Jul 2013 12:47:07 -0700 Subject: [Tutor] New Python 3.3.2 Install Fails to Start-Up In-Reply-To: References: Message-ID: On 7 July 2013 22:43, Nathan Schlaffer wrote: > Dear Tutor, > > I have a Windows 7 Laptop. I installed Python Version 3.3.2 this morning and > it worked fine. Then I tried to run a program someone else wrote (see > below), and Python closed. I don't see the program. From a book, or from a friend? Post the offending program in case it did some sort of damage. Python can do anything, including writing to the registry. Jim From davea at davea.name Wed Jul 10 21:57:37 2013 From: davea at davea.name (Dave Angel) Date: Wed, 10 Jul 2013 15:57:37 -0400 Subject: [Tutor] importing into a function In-Reply-To: References: Message-ID: On 07/10/2013 03:02 PM, Jim Mooney wrote: > Boy, is the list busy. Python must have been mentioned on USA today, > or something ;') > > Anyway, I was reading the datamodel, and it mentioned imported modules > going out of scope, which seemed odd, since I thought they were global > and never went out, or am I thinking wrong? You're confusing two separate things. Once something is imported, it stays imported. sys.modules has a reference to it, and normally, it's never forgotten. However, the name you assigned locally may go out of scope quite readily. And next time the function does the import, it'll basically turn into a simple assignment, something like: def testimp(imp): math = sys.modules["math"] > > Anyway, I figured an import would go out of scope it if was in a function: > > #Using C:\Python33\python.exe on Win 7 in c:\python33\jimprogs > > def testimp(inp): > import math > return math.sqrt(inp) > > print(testimp(25)) # result - 5.0 > > print(math.sqrt(25)) # result error traceback, math undefined > > Sure enough it did. The question is - does importing into a function > ever make sense? It occurred to me that if you use from module > import * , putting it in a function Can't be done. Locals of a function are all known at compile time. > protects you from invisible > namespace pollution by a big module with a lot of names. But importing > has to be a big overhead. So does importing into a function make sense > at any time, or is it a dead end? > Importing is a lot quicker than you realize. Over a hundred modules are imported for you before your script even starts. But more importantly, they are cached. However, a large module that is NOT preloaded by the interpreter, and that may not be actually needed, is a good candidate for loading inside a function, or even conditionally at top-level. The other, less important, reason might be just to spread out the startup overhead by deferring a tiny bit of it till later. -- DaveA From cybervigilante at gmail.com Wed Jul 10 22:20:47 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Wed, 10 Jul 2013 13:20:47 -0700 Subject: [Tutor] New Python 3.3.2 Install Fails to Start-Up In-Reply-To: References: Message-ID: Another thought from my experience with windows. Run chkdsk. If you keep installing the same good program, and it keeps failing, you could be installing over a disk error, over and over, that may have been caused if you wrote a huge data file that didn't close, likely to the python disk space area if your disk isn't cluttered. If chkdks fixes the problem, install again. If that doesn't work, rename the python directory to oldpython, then install, and windows will be forced to install to a different disk area. You can deal with the bad disk area later, using something like spinrite (if that's still around - I'm dating myself ;') Jim On 7 July 2013 22:43, Nathan Schlaffer wrote: > Dear Tutor, > > I have a Windows 7 Laptop. I installed Python Version 3.3.2 this morning and > it worked fine. Then I tried to run a program someone else wrote (see > below), and Python closed. Then I couldn't get Python to restart even after > rebooting several times. I then uninstalled Python, rebooted, cleaned the > Windows registry, and reinstalled Python from scratch. But I stilll couldn't > start Python, so I tried uninstalling Python and using a restore point > several times with no success. I still can't make Python run from the IDLE > (Python GUI) on the start menu. Can anybody please help me? > > Second, I tried to run my Python programs today via IDLE but when I checked > the Edit Menu I couldn't find a Run Menu to execute or run my program. How > do I run a program once I have written the code in IDLE? > > Sincerely, > Nathan Schlaffer > E-mail: nathanschlaffer at gmail.com > > From eryksun at gmail.com Wed Jul 10 22:37:25 2013 From: eryksun at gmail.com (eryksun) Date: Wed, 10 Jul 2013 16:37:25 -0400 Subject: [Tutor] New Python 3.3.2 Install Fails to Start-Up In-Reply-To: References: Message-ID: On Mon, Jul 8, 2013 at 1:43 AM, Nathan Schlaffer wrote: > > I have a Windows 7 Laptop. I installed Python Version 3.3.2 this morning and > it worked fine. Then I tried to run a program someone else wrote (see > below), and Python closed. Then I couldn't get Python to restart even after > rebooting several times. I then uninstalled Python, rebooted, cleaned the > Windows registry, and reinstalled Python from scratch. But I stilll couldn't > start Python, so I tried uninstalling Python and using a restore point > several times with no success. I still can't make Python run from the IDLE > (Python GUI) on the start menu. Can anybody please help me? I doubt the problem is your Python installation. More than likely it's an IDLE bug. Reinstalling doesn't touch the .idlerc config directory in your profile. Try deleting it if it exists. If that doesn't work, run IDLE from the console so you can see if it prints an error. Open a 'command prompt' and run the following: PYTHONHOME\Lib\idlelib\idle.py Replace "PYTHONHOME" with the directory where Python is installed. > Second, I tried to run my Python programs today via IDLE but when I checked > the Edit Menu I couldn't find a Run Menu to execute or run my program. How > do I run a program once I have written the code in IDLE? Run -> Run Module That's just for testing in IDLE, which isn't the normal way to run Python programs. Run your .py scripts (console user interface) and .pyw scripts (graphical user interface) directly from the desktop shell (explorer.exe) or the console shell (cmd.exe). For console applications, running via cmd is better because you inherit the existing console that stays open when your program exits. ---- If you aren't familiar with the Windows command-line interface, here are some basic commands; use "help" to learn more: Built-in Commands exit - quit cmd cls - clear screen echo - display a message type - display a text file set - set/display environment variable start - run a program in a new window dir - list files/dirs cd (chdir) - change/show dir pushd - push current dir; then cd popd - cd to last pushed dir md (mkdir) - create dir rd (rmdir) - remove dir ren (rename) - rename files copy - copy files move - move files del - delete files External Commands chcp - change codepage (437, 1252, 65001) doskey - manage aliases and history more - page a text file findstr - search using regular expressions runas - run as a different user tasklist - list processes taskkill - terminate processes Environment Variables PATH - search path PATHEXT - command extensions APPDATA - AppData\Roaming LOCALAPPDATA - AppData\Local TEMP From eryksun at gmail.com Wed Jul 10 22:54:53 2013 From: eryksun at gmail.com (eryksun) Date: Wed, 10 Jul 2013 16:54:53 -0400 Subject: [Tutor] learning nested functions In-Reply-To: <201307072326.25818.tjhanson@yahoo.com> References: <201307072326.25818.tjhanson@yahoo.com> Message-ID: On Mon, Jul 8, 2013 at 2:26 AM, Tim Hanson wrote: > > No problem so far. I made a change and ran it again: > >>>> def f1(): > x=88 > def f2(): > print(x) > x=99 > print(x) > f2() You need to use a "nonlocal" declaration (3.x only). Using nonlocal explicitly creates a closure. In your first example this was done implicitly. But since you're assigning to x, the compiler assumes you want a local unless you tell it otherwise. def f1(): x = 88 def f2(): nonlocal x print(x) x = 99 print(x) return f2 >>> f2 = f1() >>> f2() 88 99 To pull this off Python creates a cell object that's shared by both frames. The cell lives as long as it's referenced: >>> f2.__closure__[0].cell_contents 99 >>> sys.getrefcount(f2.__closure__[0]) - 1 1 I subtracted 1 for the getrefcount call. The reference is only 1 because the frame for f1 is already garbage collected. If you call f1 again, it creates a new cell for 'x' and a new f2. As you can imagine the bytecode for working with cells is different, and this affects f1 and f2 equally. Instead of LOAD_FAST and STORE_FAST, they use LOAD_DEREF and STORE_DEREF. From oscar.j.benjamin at gmail.com Wed Jul 10 23:13:56 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Wed, 10 Jul 2013 22:13:56 +0100 Subject: [Tutor] Best way to setup Unit Testing? In-Reply-To: <1373467027.45667.YahooMailNeo@web163805.mail.gq1.yahoo.com> References: <1373467027.45667.YahooMailNeo@web163805.mail.gq1.yahoo.com> Message-ID: On 10 July 2013 15:37, Albert-Jan Roskam wrote: >> >> On 7 July 2013 21:16, Srinivas Nyayapati wrote: >>> I am tryng to figure out the best way to do Unit Testing for all my >> projects >>> going forward. >> >> Are you the only one using these applications or are you intending >> that other people would install them? Do you want other people to run >> the tests on their machines when they install? > > Shouldn't this always be possible? That depends if the tests are installed along with the software. It's not uncommon for the development code base to contain tests that are not shipped to end users. In a typical PyPI package you would use the setup.py file to specify which files from the development code-base are included in the source distribution (e.g. 'python setup.py sdist'). The question about how best to organise the tests is affected by this; if you want them in the target users site-packages directory as a subpackage of your MyApp package then that's probably where you should put them in the development code-base. [snip] >> >> You can always just do that in a script. I would normally have a >> Makefile and invoke tests with >> >> $ make test > > Isn't this specific for unix-like systems? You can install make on non-unix systems. I use make on Windows. It's not typically installed on Windows but neither is Python. Again if you anticipate that lots of different people are going to be working with this code using different operating systems and that requiring everyone to have make installed is unreasonable then you may feel that e.g. a Python script is better than a Makefile. On the other hand if the code is only intended for a particular set of computers or you're the only person who will ever run those tests then just use whatever you want. I have a number of projects that are not only very specific to my own work (and not really useful to anyone else) but specific to the particular machines on which they run (so not useful anywhere else). Since I'm the only one who will ever run those tests I don't need to worry about people who don't have make. In any case if you don't have make you can still run the tests since if you look in the Makefile you can find the exact commands that are invoked by 'make test' anyway. Make is a very useful tool for code-base housekeeping. For example, if I have an extension module that needs to be built from C code then my Makefile can have something like: test: myextension.pyd python -m MyApp.tests.run_all myextension.pyd: myextension.c myextension.h python setup.py build_ext --inplace Then 'make test' will rebuild the extension module if necessary (because I modified the C code) and not otherwise. This means I don't need to constantly keep track of whether I remembered to recompile after editing the C code. Otherwise when I'm not sure I have to recompile and rerun the tests (which obviously wastes time). Oscar From ska at luo.to Wed Jul 10 23:05:47 2013 From: ska at luo.to (ska at luo.to) Date: Thu, 11 Jul 2013 00:05:47 +0300 (EEST) Subject: [Tutor] Problems understanding code output Message-ID: <24816197.4240951373490347622.JavaMail.ska@luo.to> def printMax(a, b): if a > b: print(a, 'is maximum') elif a == b: print(a, 'is equal to', b) else: print(b, 'is maximum') printMax(3, 4) # directly give literal values x = 5 y = 7 printMax(x, y) # give variables as arguments How the code above values to: 4 is maximum 7 is maximum and not to: 5 is maximum 7 is maximum This is going a little over my head, please advice, what am I missing in here? -------------- next part -------------- An HTML attachment was scrubbed... URL: From eryksun at gmail.com Wed Jul 10 23:16:44 2013 From: eryksun at gmail.com (eryksun) Date: Wed, 10 Jul 2013 17:16:44 -0400 Subject: [Tutor] importing into a function In-Reply-To: References: Message-ID: On Wed, Jul 10, 2013 at 3:57 PM, Dave Angel wrote: >> >> Sure enough it did. The question is - does importing into a function >> ever make sense? It occurred to me that if you use from module >> import * , putting it in a function > > Can't be done. Locals of a function are all known at compile time. That depends. 2.x issues a warning and disables optimization, so the function's locals use a dict: >>> from inspect import CO_OPTIMIZED >>> def f(): from Tkinter import * ... :1: SyntaxWarning: import * only allowed at module level >>> f.__code__.co_flags & CO_OPTIMIZED 0 I'm not saying this is a good idea. It isn't compatible with 3.x, you lose the ability to create a closure (i.e. a nested function with free variables), you lose the efficiency of the fast locals array, and every call has to create and update a locals dict. From phil_lor at bigpond.com Wed Jul 10 23:26:26 2013 From: phil_lor at bigpond.com (Phil) Date: Thu, 11 Jul 2013 07:26:26 +1000 Subject: [Tutor] Reading numbers from a text file In-Reply-To: References: <51DD1E32.7040508@bigpond.com> Message-ID: <51DDD182.10302@bigpond.com> On 11/07/13 03:25, Marc Tompkins wrote: > > I'm attempting to read a configuration file that will restore my > program to the state that it was in when it was closed. Ideally the > config file would be human readable. > > > Python actually comes with a module for this - it's called "configparser": > http://docs.python.org/2/library/configparser.html > > I personally use a module called "configobj" that I find slightly more > powerful / easy to use than configparser: > https://pypi.python.org/pypi/configobj/ > > Either way, you don't have to re-invent the wheel... > Thanks Marc, there seems to be a module for almost everything imaginable. -- Regards, Phil From davea at davea.name Thu Jul 11 00:46:56 2013 From: davea at davea.name (Dave Angel) Date: Wed, 10 Jul 2013 18:46:56 -0400 Subject: [Tutor] Problems understanding code output In-Reply-To: <24816197.4240951373490347622.JavaMail.ska@luo.to> References: <24816197.4240951373490347622.JavaMail.ska@luo.to> Message-ID: On 07/10/2013 05:05 PM, ska at luo.to wrote: > def printMax(a, b): > if a > b: > print(a, 'is maximum') > elif a == b: > print(a, 'is equal to', b) > else: > print(b, 'is maximum') > printMax(3, 4) # directly give literal values > x = 5 > y = 7 > printMax(x, y) # give variables as arguments > > > > How the code above values to: > > 4 is maximum > 7 is maximum > > and not to: > > 5 is maximum > 7 is maximum > > This is going a little over my head, please advice, what am I missing in > here? > You're calling the function twice, and two lines are printed out. You're questioning why the first time it prints "5 is maximum." The arguments to the function the first time are 3 and 4. The larger of those is 4. How would it manage to come up with a 5 for that call? Perhaps you'd see it easier if you temporarily added another print at the beginning of printMax(): def printMax(a, b): print("Comparing %d to %d" % a, b) if a > b: ....... -- DaveA From davea at davea.name Thu Jul 11 00:50:50 2013 From: davea at davea.name (Dave Angel) Date: Wed, 10 Jul 2013 18:50:50 -0400 Subject: [Tutor] importing into a function In-Reply-To: References: Message-ID: On 07/10/2013 05:16 PM, eryksun wrote: > On Wed, Jul 10, 2013 at 3:57 PM, Dave Angel wrote: >>> >>> Sure enough it did. The question is - does importing into a function >>> ever make sense? It occurred to me that if you use from module >>> import * , putting it in a function >> >> Can't be done. Locals of a function are all known at compile time. > > That depends. 2.x issues a warning and disables optimization, so the > function's locals use a dict: > > >>> from inspect import CO_OPTIMIZED > >>> def f(): from Tkinter import * > ... > :1: SyntaxWarning: import * only allowed at module level > >>> f.__code__.co_flags & CO_OPTIMIZED > 0 > > I'm not saying this is a good idea. It isn't compatible with 3.x, you > lose the ability to create a closure (i.e. a nested function with free > variables), you lose the efficiency of the fast locals array, and > every call has to create and update a locals dict. All good points. But Jim Mooney is running Python 3.3, as evidenced by the location he gives for the interpreter. -- DaveA From eryksun at gmail.com Thu Jul 11 01:34:53 2013 From: eryksun at gmail.com (eryksun) Date: Wed, 10 Jul 2013 19:34:53 -0400 Subject: [Tutor] importing into a function In-Reply-To: References: Message-ID: On Wed, Jul 10, 2013 at 6:50 PM, Dave Angel wrote: > > All good points. But Jim Mooney is running Python 3.3, as evidenced by the > location he gives for the interpreter. Over the past few weeks I've seen Jim bounce between 2.x and 3.x, in both 32-bit and 64-bit, from the official distributions to ActivePython, and 2 or 3 IDEs. So maybe my comment will be relevant next week. ;) From phil_lor at bigpond.com Wed Jul 10 23:21:36 2013 From: phil_lor at bigpond.com (Phil) Date: Thu, 11 Jul 2013 07:21:36 +1000 Subject: [Tutor] Reading numbers from a text file In-Reply-To: References: <51DD1E32.7040508@bigpond.com> Message-ID: <51DDD060.7090709@bigpond.com> On 10/07/13 20:25, Oscar Benjamin wrote: >> with open("config_file", "r") as file: >> myvars = file.read().splitlines() >> >> myvars = [i.split(" ")[0] for i in myvars] > > If you just remove the "[0]" from the line above then i.split(" ") > will return a list of both x and y. Then you can do e.g.: > > x, y = myvars[0] > Thanks Oscar, I keep overlooking this Python feature. I'm so used to x = var[0] any y = var[1]. -- Regards, Phil From kbailey at howlermonkey.net Thu Jul 11 02:37:00 2013 From: kbailey at howlermonkey.net (Kirk Bailey) Date: Wed, 10 Jul 2013 20:37:00 -0400 Subject: [Tutor] Robot Radio in a linux Raspberry Pi In-Reply-To: <51CECB2A.4020206@davea.name> References: <51CE6340.5010704@howlermonkey.net> <51CECB2A.4020206@davea.name> Message-ID: <51DDFE2C.3010303@howlermonkey.net> Anyone here know how to play an mp3 file in python without appealing to an external program? On 6/29/2013 7:55 AM, Dave Angel wrote: > On 06/29/2013 12:32 AM, Kirk Bailey wrote: >> ok, some months back I wrote about the program I wrote to perform the >> office of being a robot radio station in my windows desktop PC. well, I >> got it done and posted the program FOR WINDOWS on this list, and shut >> up. >> >> Then I got interested in the Raspberry Pi, which has adopted python as >> it's language of preference. And I soon not to thinking that this little >> wonder with a st of amplified speakers and a 16Gb SD card in an old >> cabinet would be hell on small dogs as a old time radio recreation. >> >> So now i am starting to meddle with what already works, and convert it >> to Raspbian Linux for installation in a Raspberry Pi. there is a nice >> command line program called mpg123 that plays mp3 files easy pasy, but >> mayhaps someone knows of another way that is a touch simpler to invoke >> from inside a python script? natch, I could do it from an sh script.. >> but that's not python, is it? >> >> Anyone who wants to stick an oar in this water, wade in to the thread. >> > > I've not used Raspberry Pi, but look into the following links. > > http://pymedia.org/tut/ > http://www.filetransit.com/freeware.php?name=Python_Library > -- -Shaboom. Kirk Bailey CEO, Freehold Marketing LLC http://www.OneBuckHosting.com/ Fnord! From drobinow at gmail.com Thu Jul 11 03:55:20 2013 From: drobinow at gmail.com (David Robinow) Date: Wed, 10 Jul 2013 21:55:20 -0400 Subject: [Tutor] install on windows In-Reply-To: References: <51D37F34.9020609@seiboldsystems.com> Message-ID: On Wed, Jul 10, 2013 at 1:16 PM, Alan Gauld wrote: > On 03/07/13 02:32, larry seibold wrote: >> >> I am stuck at step one, installing "python-2.7.5.msi" on windows XP. > > > Are you sure you have the right version - 32bit v 64bit? > > Just a thought. "python-2.7.5.msi" is the 32-bit version. That's always a "right version". Administrator rights should not be required to install "just for me". The problem is elsewhere. From steve at pearwood.info Thu Jul 11 04:34:06 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Thu, 11 Jul 2013 12:34:06 +1000 Subject: [Tutor] importing into a function In-Reply-To: References: Message-ID: <51DE199E.5070802@pearwood.info> On 11/07/13 05:02, Jim Mooney wrote: > The question is - does importing into a function > ever make sense? It occurred to me that if you use from module > import * , putting it in a function protects you from invisible > namespace pollution by a big module with a lot of names. But importing > has to be a big overhead. So does importing into a function make sense > at any time, or is it a dead end? It's not encouraged, but it's not prohibited either. Imports are very expensive the first time you do them, but after that they are cached and are quite fast. There's a long-term cache on disk (the .pyc file) and a short-term cache in memory (inside sys.modules). "import module" looks something like this pseudo-code: if "module" in sys.modules: module = sys.modules["module"] return for directory in sys.path: if there are both "module.py" and "module.pyc" files in the directory, and "module.pyc" is newer than "module.py", or there is only "module.pyc": module = load("module.pyc") break elif there is only a "module.py" file: module = compile("module.py") try: write file "module.pyc" # on-disk cache except: pass break if no module found: raise ImportError sys.modules["modules"] = module # memory cache cache return only the real code is much, much more complex. But the end result is that importing is only slow the first time. Why might you put an import inside a function? The main reason is to delay a circular import. Suppose you have a module X that relies on Y, and Y relies on X. That's tricky to do: X can't import until Y is imported, and Y can't import until X is imported. But if X can delay importing Y until X's functions are called, it will work out fine. -- Steven From cybervigilante at gmail.com Thu Jul 11 05:20:16 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Wed, 10 Jul 2013 20:20:16 -0700 Subject: [Tutor] importing into a function In-Reply-To: References: Message-ID: > Over the past few weeks I've seen Jim bounce between 2.x and 3.x, in > both 32-bit and 64-bit, from the official distributions to > ActivePython, and 2 or 3 IDEs. So maybe my comment will be relevant > next week. ;) That's the fault of Python, not me ;') I prefer learning on 3.3 , but then I'll see something interesting that only works in 2.7, or a good book fragment or web page that is only for 2.7. I'm sure this will all be sorted out in a few years and I can put 2.7 to bed, with fond memories of utter confusion. So far, I think the best Python IDE is Wing Pro, but I have to save for it. Until then Pyscripter is good at the price, there being nothing cheaper than free, but it doesn't have as clear a debugger as Wing 101. So I'll sometimes use Wing, which is set for 2.7 in the free version, if I need a clearer debugger. But 3.3 is preferred and my IDE prints out the version automatically. And of course, it's nice to check and see if my program works in both. However, there is no worry I'm going to adopt one of those dinosaur IDEs that was originally written for Java. I tried one and it took me fifteen minutes to get "Hello, World!" working. Jim Although the darling of health faddists, there is no such thing as canola oil, since there is no canola plant - it's genetically modified rapeseed oil given a nicer name ;') From nsivaram.net at gmail.com Thu Jul 11 05:24:14 2013 From: nsivaram.net at gmail.com (Sivaram Neelakantan) Date: Thu, 11 Jul 2013 08:54:14 +0530 Subject: [Tutor] multiple assignments when reading a file Message-ID: <8761whdbj5.fsf@gmail.com> I'm aware of var1, var2 = lines.split() kind of assignments How do I get to do x1..xn = lines.split() instead of typing out n var names? Is there some pythonic way to map all the data fields automagically given a starter var? I have dataset that's got 100s of fields and I really don't want to type them out. sivaram -- From amitsaha.in at gmail.com Thu Jul 11 05:28:54 2013 From: amitsaha.in at gmail.com (Amit Saha) Date: Thu, 11 Jul 2013 13:28:54 +1000 Subject: [Tutor] multiple assignments when reading a file In-Reply-To: <8761whdbj5.fsf@gmail.com> References: <8761whdbj5.fsf@gmail.com> Message-ID: On Thu, Jul 11, 2013 at 1:24 PM, Sivaram Neelakantan wrote: > > I'm aware of > > var1, var2 = lines.split() > > kind of assignments > > How do I get to do > > x1..xn = lines.split() > > instead of typing out n var names? Is there some pythonic way to map > all the data fields automagically given a starter var? > > I have dataset that's got 100s of fields and I really don't want to > type them out. The split() method returns you a list, so you could do something like: >>> line="abra ca dabra" >>> words = line.split() The words will now be in the list, words. You could then enumerate the list to access the individual words. > > sivaram > -- > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor -- http://echorand.me From davea at davea.name Thu Jul 11 07:15:42 2013 From: davea at davea.name (Dave Angel) Date: Thu, 11 Jul 2013 01:15:42 -0400 Subject: [Tutor] multiple assignments when reading a file In-Reply-To: <8761whdbj5.fsf@gmail.com> References: <8761whdbj5.fsf@gmail.com> Message-ID: On 07/10/2013 11:24 PM, Sivaram Neelakantan wrote: > > I'm aware of > > var1, var2 = lines.split() > > kind of assignments > > How do I get to do > > x1..xn = lines.split() > > instead of typing out n var names? Is there some pythonic way to map > all the data fields automagically given a starter var? > > I have dataset that's got 100s of fields and I really don't want to > type them out. > That's a pretty common question here. And the right answer is "you don't want to, not really." The real question is what you mean by "var names." If you mean you want the particular function you're in to have 217 extra locals, with names something like field000, field001, ... field216 then you're just out of luck. The names have to be known at the function's compile time, so somewhere you have to type them all in, in some form of name-binding assignment. The easiest one of those is what you already described. field000, field001, field002 field216 = lines.split() There might even be a limit of how many locals a function may have, but I don't know what it is. Probably at least 256. Next choice? Make them all global? that indeed can be faked, by manipulating the globals() dict. The idea is abhorrent, however, since mutable globals are a wart on the landscape. third choice? Put them in a list. Whaddya know, they already are. So just assign a name to that list. Referencing them is now by subscript. And you can actually do slices and loops readily, which you can't easily do on "var names." Next choice? Put them in a dict. This works just like a list, except the subscript doesn't have to be continguous ints. Final choice? Put them in a namespace. Something like: class NewSpace: pass data = NewSpace() for index, item in enumerate(lines.split()): data.__dict__["x" + str(index)] = item Actually, this doesn't get the leading zero padding, but you can enhance it from there. To reference these, use data.x42 and data.x0 I give the details on the last one because it's the most acceptable one, to me, with the list being a close second. But if this were really my problem, I'd probably be giving names to those, not numbers, and somewhere I'd have those names all typed out, in order. If it were in the code, it might as well be on the assignment that gives them values. And if it comes from outside (say in the header line of a csv file), I'd use something like: for name, item in zip(names, lines.split()): data.__dict__[name] = item Now you can get at the "addr1" field by: data.addr1 -- DaveA From alan.gauld at btinternet.com Thu Jul 11 10:10:53 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Thu, 11 Jul 2013 09:10:53 +0100 Subject: [Tutor] multiple assignments when reading a file In-Reply-To: <8761whdbj5.fsf@gmail.com> References: <8761whdbj5.fsf@gmail.com> Message-ID: On 11/07/13 04:24, Sivaram Neelakantan wrote: > > I'm aware of > > var1, var2 = lines.split() > > kind of assignments > > How do I get to do > > x1..xn = lines.split() x = lines.split() then you access the fields with x[0], x[1]...x[n] It's the default behaviour. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From kwpolska at gmail.com Thu Jul 11 10:21:32 2013 From: kwpolska at gmail.com (Chris =?utf-8?B?4oCcS3dwb2xza2HigJ0=?= Warrick) Date: Thu, 11 Jul 2013 10:21:32 +0200 Subject: [Tutor] multiple assignments when reading a file In-Reply-To: References: <8761whdbj5.fsf@gmail.com> Message-ID: <20130711082129.GA3016@kwpolska-lin> On Thu, Jul 11, 2013 at 01:15:42AM -0400, Dave Angel wrote: > Final choice? Put them in a namespace. Something like: > > class NewSpace: > pass > > data = NewSpace() > for index, item in enumerate(lines.split()): > data.__dict__["x" + str(index)] = item Please don?t do this. This is a very ugly way to do it. Instead, do: for index, item in enumerate(lines.split()): setattr(data, 'x' + str(index), item) -- Kwpolska | GPG KEY: 5EAAEA16 stop html mail | always bottom-post http://asciiribbon.org | http://caliburn.nl/topposting.html From drobinow at gmail.com Thu Jul 11 13:13:40 2013 From: drobinow at gmail.com (David Robinow) Date: Thu, 11 Jul 2013 07:13:40 -0400 Subject: [Tutor] importing into a function In-Reply-To: References: Message-ID: On Wed, Jul 10, 2013 at 11:20 PM, Jim Mooney wrote:> > So far, I think the best Python IDE is Wing Pro, but I have to save > for it. Until then Pyscripter is good at the price, there being > nothing cheaper than free, but it doesn't have as clear a debugger as > Wing 101. So I'll sometimes use Wing, which is set for 2.7 in the free > version, if I need a clearer debugger. But 3.3 is preferred and my IDE > prints out the version automatically. Wing is not "set for 2.7". Wing defaults to the default python on your system, which is apparently 2.7 You can change that by selecting "Configure Python ..." under the "Edit" menu item. Select the "Custom" radio button in the "Python Executable" section and Browse to the desired executable (probably C:\Python33\python.exe) From nsivaram.net at gmail.com Thu Jul 11 14:18:03 2013 From: nsivaram.net at gmail.com (Sivaram Neelakantan) Date: Thu, 11 Jul 2013 17:48:03 +0530 Subject: [Tutor] multiple assignments when reading a file References: <8761whdbj5.fsf@gmail.com> Message-ID: <871u75cmtg.fsf@gmail.com> Thanks for the detailed explanation below. I've replied below. On Thu, Jul 11 2013,Dave Angel wrote: [snipped 19 lines] > That's a pretty common question here. And the right answer is "you > don't want to, not really." > > The real question is what you mean by "var names." If you mean you > want the particular function you're in to have 217 extra locals, with > names something like > field000, field001, ... field216 > > then you're just out of luck. The names have to be known at the Drat! So nothing to promote laziness in me! > function's compile time, so somewhere you have to type them all in, in > some form of name-binding assignment. The easiest one of those is > what you already described. > field000, field001, field002 field216 = lines.split() > > There might even be a limit of how many locals a function may have, > but I don't know what it is. Probably at least 256. Right, I'll keep that in mind. > > Next choice? Make them all global? that indeed can be faked, by > manipulating the globals() dict. The idea is abhorrent, however, > since mutable globals are a wart on the landscape. > > third choice? Put them in a list. Whaddya know, they already are. > So just assign a name to that list. Referencing them is now by > subscript. And you can actually do slices and loops readily, which you > can't easily do on "var names." This seems the easiest but I already foresee hardcoded subscripts all over the code which I will promptly forget the very next day of what it stands for. > > Next choice? Put them in a dict. This works just like a list, except > the subscript doesn't have to be continguous ints. alright. > > Final choice? Put them in a namespace. Something like: > > class NewSpace: > pass > > data = NewSpace() > for index, item in enumerate(lines.split()): > data.__dict__["x" + str(index)] = item > OO, I'd stay away till I get the basics right. [snipped 19 lines] sivaram -- From oscar.j.benjamin at gmail.com Thu Jul 11 14:25:00 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Thu, 11 Jul 2013 13:25:00 +0100 Subject: [Tutor] Finding more efficient ways In-Reply-To: References: Message-ID: On 9 July 2013 21:04, HRK wrote: > > PROBLEM #2 > > def is_up_down(values): > '''Return True if values is an up-down-sequence, otherwise > return False. > A sequence [x[0], x[1], x[2], ..., x[n]] is an up-down-sequence > if there is an index k such that > x[0] <= x[1] <= x[2] <= ... <= x[k-1] <= x[k] and > x[k] >= x[k+1] >= x[k+2] >= ... >= x[n-1] >= x[n] hold. > That is, the first part of the sequence is increasing and the > second part is decreasing. > Either part may be empty, so any increasing sequence and any > decreasing sequence is an up-down-sequence. > ''' > flip = 0 > value = 0 The line above assumes non-negative inputs. You could use float('-inf') here assuming that this only needs to work for numbers. > for item in values: > if flip == 2: > if item > value: > return False > elif flip == 1: > if item < value: > flip += 1 > value = item > elif flip == 0: > if item > value: > flip += 1 > value = item It would be cleaner to just have a single 'value = item' line at the end of the loop: for item in values: if flip == 2: if item > value: return False elif flip == 1: if item < value: flip += 1 elif flip == 0: if item > value: flip += 1 value = item Some would consider this an abuse of Python's bool type but I might write that as: for item in values: if flip == 2: if item > value: return False elif flip == 1: flip += item < value elif flip == 0: flip += item > value: value = item Perhaps I'd even do: for item in values: if flip == 2: if item > value: return False else: flip += item < value if flip else item > value value = item If you use the pairwise recipe from itertools: from itertools import tee def pairwise(iterable): "s -> (s0,s1), (s1,s2), (s2, s3), ..." a, b = tee(iterable) next(b, None) return zip(a, b) # Assumes Python 3 Then you can just do: pairs = pairwise(values) for item1, item2 in pairs: if item2 < item1: break for item1, item2 in pairs: # Assumes Python 3 if item2 > item1: return False return True or perhaps pairs = pairwise(values) direction = 1 for item1, item2 in pairs: if (item2 - item1) * direction < 0: if direction == 1: direction == -1 else: return False return True but this last version assumes that the items are subtractable (rather than simply orderable). You may prefer any of these to your original but I wouldn't say that any of them is substantially better. I think your first approach was pretty good (apart from not handling negative inputs). To modify the original so that it works for any orderable types you can do def is_up_down(values): '''Return True if values is an up-down-sequence, otherwise return False. A sequence [x[0], x[1], x[2], ..., x[n]] is an up-down-sequence if there is an index k such that x[0] <= x[1] <= x[2] <= ... <= x[k-1] <= x[k] and x[k] >= x[k+1] >= x[k+2] >= ... >= x[n-1] >= x[n] hold. That is, the first part of the sequence is increasing and the second part is decreasing. Either part may be empty, so any increasing sequence and any decreasing sequence is an up-down-sequence. >>> is_up_down([2,5,5,7,9,9,8]) True >>> is_up_down([2,5,5,7,9,8,9]) False >>> is_up_down([9,8]) True >>> is_up_down([13]) True ''' flip = 0 lastitem = sentinel = object() for item in values: if lastitem is not sentinel: if flip == 0: if item > lastitem: flip += 1 elif flip == 1: if item < lastitem: flip += 1 elif flip == 2: if item > lastitem: return False lastitem = item return True import doctest; doctest.testmod() I find it easier to read if the order is the other way up so that flip == 0 is at the top. Also note that by moving your examples into the docstring you now have automatic testing. Oscar From davea at davea.name Thu Jul 11 14:58:35 2013 From: davea at davea.name (Dave Angel) Date: Thu, 11 Jul 2013 08:58:35 -0400 Subject: [Tutor] multiple assignments when reading a file In-Reply-To: <871u75cmtg.fsf@gmail.com> References: <8761whdbj5.fsf@gmail.com> <871u75cmtg.fsf@gmail.com> Message-ID: On 07/11/2013 08:18 AM, Sivaram Neelakantan wrote: > > Thanks for the detailed explanation below. I've replied below. > > On Thu, Jul 11 2013,Dave Angel wrote: > >> >> third choice? Put them in a list. Whaddya know, they already are. >> So just assign a name to that list. Referencing them is now by >> subscript. And you can actually do slices and loops readily, which you >> can't easily do on "var names." > > This seems the easiest but I already foresee hardcoded subscripts all > over the code which I will promptly forget the very next day of what > it stands for. > Good. Then you're changing the requirements spec from unreasonable to something attainable. You're the one who had all these numeric variables on the left hand side of the assignment. If you want them to be real names, then you're going to make progress. Next question is where do these names come from? In the text file, there seem to be just a bunch of values, separated by whitespace. So the only characteristic they have at that moment is sequence. You can refer to the 19th item in the text file, or the 4th. One example of a name correlation would be the traditional comma-separated file (csv). The first line lists the names, separated by commas, and subsequent line(s) contain values for those names. it's one way of import/export from a spreadsheet, database, etc. So if you can put the names in the text file, then you can easily make a dict (see below), or a namespace with those names (see below). >> >> Next choice? Put them in a dict. This works just like a list, except >> the subscript doesn't have to be continguous ints. > > alright. > >> >> Final choice? Put them in a namespace. Something like: >> >> class NewSpace: >> pass >> >> data = NewSpace() >> for index, item in enumerate(lines.split()): >> data.__dict__["x" + str(index)] = item >> > > OO, I'd stay away till I get the basics right. > Chris Warrick corrected me on this. I knew about setattr(), but forgot it could be used in this way. So if you now have a list of names (each a str, and one that's a legal name in Python), you could do: data = NewSpace() for name, item in zip(names, lines.split()): setattr(data, name, item) And now you can have say print(data.address1) to print out the address1 string from that file. Try it with some simple data, it's not very hard. -- DaveA From alan.gauld at btinternet.com Thu Jul 11 15:06:19 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Thu, 11 Jul 2013 14:06:19 +0100 Subject: [Tutor] multiple assignments when reading a file In-Reply-To: <871u75cmtg.fsf@gmail.com> References: <8761whdbj5.fsf@gmail.com> <871u75cmtg.fsf@gmail.com> Message-ID: On 11/07/13 13:18, Sivaram Neelakantan wrote: >>> How do I get to do >>> >>> x1..xn = lines.split() >> third choice? Put them in a list. > > This seems the easiest but I already foresee hardcoded subscripts all > over the code which I will promptly forget the very next day of what > it stands for. So you would remember what x1, x7 and x23 were for, but wouldn't remember x[1], x[7] and x[23]? The whole premise here was that you wanted Python to create the numeric suffixes for you. The reason that's a bad idea is exactly as you said: you don't have a clue what the numbers mean. But if you must use numbers, list indices are no worse than suffixed names. >> Next choice? Put them in a dict. This works just like a list, except >> the subscript doesn't have to be continguous ints. I don't know how you would do this because you still need to create n unique keys... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From eryksun at gmail.com Thu Jul 11 15:53:14 2013 From: eryksun at gmail.com (eryksun) Date: Thu, 11 Jul 2013 09:53:14 -0400 Subject: [Tutor] multiple assignments when reading a file In-Reply-To: References: <8761whdbj5.fsf@gmail.com> Message-ID: On Thu, Jul 11, 2013 at 1:15 AM, Dave Angel wrote: > There might even be a limit of how many locals a function may have, but I > don't know what it is. Probably at least 256. Going by just the stack and opcode limits, one could potentially unpack into over 2 billion local variables in CPython. In practice, that's ridiculous and you'd hit other limits trying to compile it. A quick check unpacking into a million local variables did work, so for all practical purposes running out of space for locals is nothing to worry about. f_template = r''' def f(): %s = xrange(%d)''' def make_f(n): a = ','.join('a%d' % i for i in xrange(n)) return f_template % (a, n) >>> exec make_f(1000000) >>> f.__code__.co_stacksize 1000000 Whatever you do, don't dis() that function. It has a million STORE_FAST ops. I did unpacking here to show that UNPACK_SEQUENCE(count) can be extended to a double-word by a preceding EXTENDED_ARG op. In other words the count can be larger than 65535 in CPython. This was added in 2.0 beta 1: The limits on the size of expressions and file in Python source code have been raised from 2**16 to 2**32. Previous versions of Python were limited because the maximum argument size the Python VM accepted was 2**16. This limited the size of object constructor expressions, e.g. [1,2,3] or {'a':1, 'b':2}, and the size of source files. This limit was raised thanks to a patch by Charles Waldman that effectively fixes the problem. It is now much more likely that you will be limited by available memory than by an arbitrary limit in Python. From oscar.j.benjamin at gmail.com Thu Jul 11 16:20:39 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Thu, 11 Jul 2013 15:20:39 +0100 Subject: [Tutor] Best way to setup Unit Testing? In-Reply-To: <1431526120876682177@unknownmsgid> References: <1431526120876682177@unknownmsgid> Message-ID: On 11 July 2013 13:37, Srinivas Nyayapati wrote: >>> Projects/ >>> MyApp/ >>> __init__.py >>> myapp.py >>> tests/ >>> __init__.py >>> test_myapp.py >> >> This gives a nice clean distinction between your main code and your >> tests. It is also useful if you don't want to install the tests on the >> target machines since you can just exclude the tests package in your >> setup.py when creating an sdist. >> >> If you do include the tests and add a __main__.py in the tests folder >> then users can still run tests with >> >> python -m MyApp.tests > > Love this idea! Would you know how I would run the test_*.py scripts > from __main__.py There are many different ways but for example let's say you're using unittest and you have Projects/ Makefile MyApp/ __init__.py myapp1.py myapp2.py tests/ __init__.py __main__.py test_myapp1.py test_myapp2.py Then e.g. test_myapp1.py looks like: import unittest from MyApp.myapp1 import stuff class TestMyApp(unittest.TestCase): def test_stuff(): self.assertTrue(False) # blah blah if __name__ == "__main__": unittest.main() Then __main__.py looks like: import unittest from MyApp.tests import test_myapp1, test_myapp2 tests = unittest.TestLoader() suite1 = loadTestsFromModule(test_myapp1) suite2 = loadTestsFromModule(test_myapp2) alltests = unittest.TestSuite([suite1, suite2]) if __name__ == "__main__": unittest.TextTestRunner().run(alltests) The above is mainly off the top of my head and may not actually work but hopefully you get the idea. Now you can run just a few tests with: $ python -m MyApp.tests.test_myapp1 or run all the tests with $ python -m MyApp.tests I would still put the above line into a Makefile and use 'make test' while working on the code. That way I don't need to remember what to type. One last thing is that I wouldn't use CamelCase names for packages/modules. My preference is for all package and module names to be lower-case. When I see MyApp in the code it looks to me like a class rather than a module. Oscar From cybervigilante at gmail.com Thu Jul 11 18:28:25 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Thu, 11 Jul 2013 09:28:25 -0700 Subject: [Tutor] importing into a function In-Reply-To: References: Message-ID: > You can change that by selecting "Configure Python ..." under the > "Edit" menu item. Select the "Custom" radio button in the "Python > Executable" section and Browse to the desired executable (probably > C:\Python33\python.exe) Oh, I'm aware of that - but you have to reset it every time you switch Pythons, which is annoying. Wing Pro has projects, so you can just create two main projects and specify which Py version is running. So I keep Wing at 2.7. Pyscripter's debugger is awful with stack data, whereas Wing is very clear. And every time I've found some small awkwardness (or big one) in other IDEs, Wing didn't have it. This tells me it is Very well thought out, even in the 101 version. Commenting out, which I use as a poor man's debugger, takes one click in Wing - it takes two in Pyscripter. That may not seem like a lot (although I use commenting out a lot, so it is), but I judge a restaurant by its coffee. When even small things are well thought out, it's a good sign you're going to buy something worth buying. I believe in free software, but some things really are worth buying and Wing strikes me as one of them. I could never do without Clipmate, which saves tons of work and simplifies much. I'd pay for it again. As for doing raw animation coding in HTML5 Canvas, instead of using Flash, ugh. Sure, you can draw the Mona Lisa with Turtle Graphics, but who has the time ;') Jim From cbc at unc.edu Thu Jul 11 20:27:54 2013 From: cbc at unc.edu (Chris Calloway) Date: Thu, 11 Jul 2013 14:27:54 -0400 Subject: [Tutor] importing into a function In-Reply-To: References: Message-ID: <51DEF92A.2040005@unc.edu> On 7/11/2013 12:28 PM, Jim Mooney wrote: > I believe in free software, but some things really are worth buying > and Wing strikes me as one of them. Ditto this. Wing is the awesome. The support is unbelievably great. -- Sincerely, Chris Calloway http://nccoos.org/Members/cbc office: 3313 Venable Hall phone: (919) 599-3530 mail: Campus Box #3300, UNC-CH, Chapel Hill, NC 27599 From cybervigilante at gmail.com Thu Jul 11 20:47:40 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Thu, 11 Jul 2013 11:47:40 -0700 Subject: [Tutor] importing into a function In-Reply-To: <51DEF92A.2040005@unc.edu> References: <51DEF92A.2040005@unc.edu> Message-ID: On 11 July 2013 11:27, Chris Calloway wrote: > Ditto this. Wing is the awesome. The support is unbelievably great. I tried a ton of them. The java-based ones are horrid. Others are always missing little things. I just tried another, for instance, but it doesn't put itself on the All Programs start menu except, rather oddly, for the uninstall. It's almost like they are encouraging you to get rid of it. So you have to go find it right off. Missing little things like that is annoying, and a sign of bad things to come, IMHO ;'). If you are installing to Windows, you of course put yourself on the All Programs menu, which is the first place you look for a program you haven't pinned or cluttered on to the desktop. A lot of the IDE stuff may not be Necessary, and final test should be in the DOS box, but there's not much sense in using an IDE at all if it doesn't have everything to make life easy, very sensibly arranged, when you Do need something like a debugger or some snippets (an essential since I have a truly rotten memory ;') -- Jim Although the darling of health faddists, there is no such thing as canola oil, since there is no canola plant - it's genetically modified rapeseed oil given a nicer name ;') From alan.gauld at btinternet.com Thu Jul 11 23:49:59 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Thu, 11 Jul 2013 22:49:59 +0100 Subject: [Tutor] importing into a function In-Reply-To: References: <51DEF92A.2040005@unc.edu> Message-ID: On 11/07/13 19:47, Jim Mooney wrote: > in the DOS box, but there's not much sense in using an IDE at all if > it doesn't have everything to make life easy, very sensibly arranged, > when you Do need something like a debugger or some snippets (an > essential since I have a truly rotten memory ;') You can always use a standalone debugger like winpdb. I find it better than any of the IDEs I've tried. But then I don't much like IDEs of any kind (unless you count Unix or emacs as an IDE). -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From cybervigilante at gmail.com Fri Jul 12 02:44:32 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Thu, 11 Jul 2013 17:44:32 -0700 Subject: [Tutor] renaming input works intermittently Message-ID: When I tried a simple rename of input, it worked - in python 2.7 and python 3.3 import sys if int(sys.version[0]) < 3: input = raw_input x = input('type something ') print(x) # this works in both Py versions But when I tried that in my numbers program, I got an error: UnboundLocalError: local variable 'input' referenced before assignment for the below: try: if int(sys.version[0]) < 3: input = raw_input numbers_str = original = input('Enter a positive' 'integer, space separated if desired.') # error occurs here Oddly, the error only occurs in Python 3.3 - the above works in Python 2.7 Here it is, where I run it from the dos box. Fails in Py3 on the first run, works in Py2 on the second run Microsoft Windows [Version 6.1.7601] Copyright (c) 2009 Microsoft Corporation. All rights reserved. Python 3.3 run doesn't work C:\Python33\Jimprogs>py3 numrev1.py Traceback (most recent call last): File "numrev1.py", line 115, in main() File "numrev1.py", line 111, in main name_of_numbers = numbers_to_name() File "numrev1.py", line 75, in numbers_to_name triplets, triplen = create_triplets() File "numrev1.py", line 64, in create_triplets numbers_str = check_input() File "numrev1.py", line 40, in check_input numbers_str = original = input('Enter a positive' UnboundLocalError: local variable 'input' referenced before assignment Python 2.7 run works C:\Python33\Jimprogs>py2 numrev1.py Enter a positiveinteger, space separated if desired.123456 one hundred twenty-three thousand four hundred fifty-six -- Jim Although the darling of health faddists, there is no such thing as canola oil, since there is no canola plant - it's genetically modified rapeseed oil given a nicer name ;') From cybervigilante at gmail.com Fri Jul 12 02:48:54 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Thu, 11 Jul 2013 17:48:54 -0700 Subject: [Tutor] importing into a function In-Reply-To: References: <51DEF92A.2040005@unc.edu> Message-ID: On 11 July 2013 14:49, Alan Gauld wrote: > You can always use a standalone debugger like winpdb. > I find it better than any of the IDEs I've tried. > But then I don't much like IDEs of any kind (unless you > count Unix or emacs as an IDE). You can do so much with Unix that you have an IDE and more, so I really don't think you're only using a Notepad equivalent ;') I'll take a look at winpdb. Might be the best answer for an IDE that is otherwise good but has a poor debugger. Jim From steve at pearwood.info Fri Jul 12 03:01:33 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 12 Jul 2013 11:01:33 +1000 Subject: [Tutor] renaming input works intermittently In-Reply-To: References: Message-ID: <51DF556D.10504@pearwood.info> On 12/07/13 10:44, Jim Mooney wrote: > When I tried a simple rename of input, it worked - in python 2.7 and python 3.3 > > import sys > if int(sys.version[0]) < 3: > input = raw_input > > x = input('type something ') > print(x) # this works in both Py versions > > But when I tried that in my numbers program, I got an error: > UnboundLocalError: local variable 'input' referenced before assignment > for the below: When you assign to a name *anywhere* inside a function, Python treats that name as a local variable regardless of whether it is before, or after, the assignment. So unlike the language Lua, you can't read a global variable, then create a local variable of the same name in the same function. In your case, the fix is to pull the renaming of input out of the function, so it is performed once only, when the module first begins to run, instead of every time you call the function. Put the renaming code at the top of the module, after the "import sys", then just unconditionally use "input" inside the function. > try: > if int(sys.version[0]) < 3: > input = raw_input > numbers_str = original = input('Enter a positive' > 'integer, space separated if desired.') # error occurs here > > Oddly, the error only occurs in Python 3.3 - the above works in Python 2.7 The rules for local variables are rather more complicated in Python 2 and it may be that you're somehow slipping through the cracks. -- Steven From eryksun at gmail.com Fri Jul 12 03:07:27 2013 From: eryksun at gmail.com (eryksun) Date: Thu, 11 Jul 2013 21:07:27 -0400 Subject: [Tutor] renaming input works intermittently In-Reply-To: <51DF556D.10504@pearwood.info> References: <51DF556D.10504@pearwood.info> Message-ID: On Thu, Jul 11, 2013 at 9:01 PM, Steven D'Aprano wrote: >> try: >> if int(sys.version[0]) < 3: >> input = raw_input >> numbers_str = original = input('Enter a positive' >> 'integer, space separated if desired.') # error occurs here >> >> Oddly, the error only occurs in Python 3.3 - the above works in Python 2.7 > > The rules for local variables are rather more complicated in Python 2 and it > may be that you're somehow slipping through the cracks. In 2.x the if statement executes, so the local variable "input" gets assigned. From steve at pearwood.info Fri Jul 12 05:21:34 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 12 Jul 2013 13:21:34 +1000 Subject: [Tutor] renaming input works intermittently In-Reply-To: References: <51DF556D.10504@pearwood.info> Message-ID: <51DF763E.4010700@pearwood.info> On 12/07/13 11:07, eryksun wrote: > On Thu, Jul 11, 2013 at 9:01 PM, Steven D'Aprano wrote: >>> try: >>> if int(sys.version[0]) < 3: >>> input = raw_input >>> numbers_str = original = input('Enter a positive' >>> 'integer, space separated if desired.') # error occurs here >>> >>> Oddly, the error only occurs in Python 3.3 - the above works in Python 2.7 >> >> The rules for local variables are rather more complicated in Python 2 and it >> may be that you're somehow slipping through the cracks. > > In 2.x the if statement executes, so the local variable "input" gets assigned. /facepalm Ah, I mean, I knew that! -- Steven From __peter__ at web.de Fri Jul 12 08:34:11 2013 From: __peter__ at web.de (Peter Otten) Date: Fri, 12 Jul 2013 08:34:11 +0200 Subject: [Tutor] renaming input works intermittently References: Message-ID: Jim Mooney wrote: > When I tried a simple rename of input, it worked - in python 2.7 and > python 3.3 > > import sys > if int(sys.version[0]) < 3: > input = raw_input > > x = input('type something ') > print(x) # this works in both Py versions > > But when I tried that in my numbers program, I got an error: > UnboundLocalError: local variable 'input' referenced before assignment > for the below: > > try: > if int(sys.version[0]) < 3: > input = raw_input > numbers_str = original = input('Enter a positive' > 'integer, space separated if desired.') # error occurs here > > Oddly, the error only occurs in Python 3.3 - the above works in Python 2.7 > > Here it is, where I run it from the dos box. Fails in Py3 on the first > run, works in Py2 on the second run If you want to assign inside a function you can just pick an unused name: def foo(): if sys.version_info.major < 3: # sys.version_info[0] to support # versions below 2.7 safe_input = raw_input else: safe_input = input numstr = safe_input("Enter a positive integer: ") ... Personally I prefer to test for a feature rather than the version: def bar(): try: safe_input = raw_input except NameError: safe_input = input numstr = safe_input("Enter a positive integer: ") ... From elmar at net4werling.de Fri Jul 12 18:52:43 2013 From: elmar at net4werling.de (elmar werling) Date: Fri, 12 Jul 2013 18:52:43 +0200 Subject: [Tutor] datetime: inverse of datetime.date.isocalendar() Message-ID: Hello, how to convert the tuble (iso_year, iso_week, iso_day) to a date object? I have tried the following scipt, but the results are buggy import datetime date_i = datetime.date(2013, 07, 12) date_iso = date_i.isocalendar() year = str(date_iso[0])[2:] week = str(date_iso[1]) day = str(date_iso[2]) date_string = year + week + day date_ii = datetime.datetime.strptime(date_string,'%y%W%w').date() print date_i, date_iso, date_string, date_ii Any help is wellcome. Elmar From alan.gauld at btinternet.com Fri Jul 12 21:00:46 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Fri, 12 Jul 2013 20:00:46 +0100 Subject: [Tutor] datetime: inverse of datetime.date.isocalendar() In-Reply-To: References: Message-ID: On 12/07/13 17:52, elmar werling wrote: > how to convert the tuble (iso_year, iso_week, iso_day) to a date object? > > I have tried the following scipt, but the results are buggy Thanks for sending the code but can you define buggy? What did you expect? What did you get? Were there any error messages if so what (full text please)? > import datetime > > date_i = datetime.date(2013, 07, 12) > date_iso = date_i.isocalendar() > > year = str(date_iso[0])[2:] > week = str(date_iso[1]) > day = str(date_iso[2]) > > date_string = year + week + day But guess is there is a better way of doing the lines above but I don't know the datetime module well enough to say what... > date_ii = datetime.datetime.strptime(date_string,'%y%W%w').date() > > print date_i, date_iso, date_string, date_ii I'm not sure what you expect to see or what differs from your expectation? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From zachary.ware+pytut at gmail.com Fri Jul 12 21:29:07 2013 From: zachary.ware+pytut at gmail.com (Zachary Ware) Date: Fri, 12 Jul 2013 14:29:07 -0500 Subject: [Tutor] datetime: inverse of datetime.date.isocalendar() In-Reply-To: References: Message-ID: On Fri, Jul 12, 2013 at 11:52 AM, elmar werling wrote: > > Hello, > > how to convert the tuble (iso_year, iso_week, iso_day) to a date object? > > I have tried the following scipt, but the results are buggy > > > import datetime > > date_i = datetime.date(2013, 07, 12) > date_iso = date_i.isocalendar() > > year = str(date_iso[0])[2:] > week = str(date_iso[1]) > day = str(date_iso[2]) > > date_string = year + week + day > date_ii = datetime.datetime.strptime(date_string,'%y%W%w').date() > > print date_i, date_iso, date_string, date_ii > > > Any help is wellcome. > > Elmar It looks like you're seeing a conflict between the way ISO8601 and the %W format code count weeks. See here: >>> today = datetime.date.today() >>> today datetime.date(2013, 7, 12) >>> today.isocalendar() (2013, 28, 5) >>> today.strftime('(%Y, %W, %w)') '(2013, 27, 5)' If you change your above 'week = ...' line to 'week = str(date_iso[1] - 1)', you'll get your expected result...this year. It'll also work next year, but 2015 will be a different issue entirely with day-of-week problems. Then, in 2016, your current code would work. The datetime module documentation[1] has a lot of good information and a couple of very informative links. HTH -- Zach [1] http://docs.python.org/2.7/library/datetime From malaclypse2 at gmail.com Fri Jul 12 21:34:50 2013 From: malaclypse2 at gmail.com (Jerry Hill) Date: Fri, 12 Jul 2013 15:34:50 -0400 Subject: [Tutor] datetime: inverse of datetime.date.isocalendar() In-Reply-To: References: Message-ID: On Fri, Jul 12, 2013 at 12:52 PM, elmar werling wrote: > > Hello, > > how to convert the tuble (iso_year, iso_week, iso_day) to a date object? > > I have tried the following scipt, but the results are buggy > > > import datetime > > date_i = datetime.date(2013, 07, 12) > date_iso = date_i.isocalendar() > > year = str(date_iso[0])[2:] > week = str(date_iso[1]) > day = str(date_iso[2]) > > date_string = year + week + day > date_ii = datetime.datetime.strptime(date_string,'%y%W%w').date() > > print date_i, date_iso, date_string, date_ii You can't feed the ISO week number and ISO day-of-week into strptime as-is. They just don't represent the same values. There's more of an explanation in the docs, here http://docs.python.org/2/library/datetime.html#datetime.date.isocalendar . I'm having a hard time summarizing, but the basic difference between the ISO calendar and the C strptime() definitions seems to be how they define the week of the year. In the ISO Calendar, the first week of the year is the first week where a thursday occurs in the week. In the C-style '%W' week-of-year format specifier, I believe the first week of the year is based on where the first monday falls, instead. Let's see if I can come up with an example or two. >>> import datetime >>> jan_1_2013 = datetime.date(2013, 1, 1) >>> jan_1_2013.isocalendar() (2013, 1, 2) So, in the ISO calendar, this is 2013, week 1, day 2. >>> datetime.datetime.strftime(jan_1_2013, "%Y %W %w") '2013 00 2' In the C-style strptime() world, this same date is 2013, week 0, day 2. Does that help explain why your original attempt was exactly one week (7 days) off what you thought it should be? -- Jerry From __peter__ at web.de Fri Jul 12 21:36:59 2013 From: __peter__ at web.de (Peter Otten) Date: Fri, 12 Jul 2013 21:36:59 +0200 Subject: [Tutor] datetime: inverse of datetime.date.isocalendar() References: Message-ID: elmar werling wrote: > how to convert the tuble (iso_year, iso_week, iso_day) to a date object? > > I have tried the following scipt, but the results are buggy > > > import datetime > > date_i = datetime.date(2013, 07, 12) > date_iso = date_i.isocalendar() > > year = str(date_iso[0])[2:] > week = str(date_iso[1]) > day = str(date_iso[2]) > > date_string = year + week + day > date_ii = datetime.datetime.strptime(date_string,'%y%W%w').date() > > print date_i, date_iso, date_string, date_ii Here's what I came up with: import datetime def ywd_to_date(year, week, weekday): """Convert (year, week, isoweekday) tuple to a datetime.date(). >>> datetime.date(2013, 7, 12).isocalendar() (2013, 28, 5) >>> ywd_to_date(2013, 28, 5) datetime.date(2013, 7, 12) """ first = datetime.date(year, 1, 1) first_year, _first_week, first_weekday = first.isocalendar() if first_year == year: week -= 1 return first + datetime.timedelta(days=week*7+weekday-first_weekday) if __name__ == "__main__": # perform an exhaustive adhoc test def iterdays(start=datetime.date(datetime.MINYEAR, 1, 1)): """Generate dates till the end of time aka MAXYEAR.""" d = start one_day = datetime.timedelta(days=1) try: while True: yield d d += one_day except OverflowError: pass for d in iterdays(): # does it round-trip? c = ywd_to_date(*d.isocalendar()) assert d == c, d I don't really understand what I'm doing here, but the test code makes me confident that it's correct ;) From fomcl at yahoo.com Fri Jul 12 21:58:16 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Fri, 12 Jul 2013 12:58:16 -0700 (PDT) Subject: [Tutor] datetime: inverse of datetime.date.isocalendar() In-Reply-To: References: Message-ID: <1373659096.2972.YahooMailNeo@web163805.mail.gq1.yahoo.com> _______________________________ > From: Alan Gauld >To: tutor at python.org >Sent: Friday, July 12, 2013 9:00 PM >Subject: Re: [Tutor] datetime: inverse of datetime.date.isocalendar() > > >On 12/07/13 17:52, elmar werling wrote: > >> how to convert the tuble (iso_year, iso_week, iso_day) to a date object? >> >> I have tried the following scipt, but the results are buggy > >Thanks for sending the code but can you define buggy? >What did you expect? What did you get? >Were there any error messages if so what (full text please)? > > >> import datetime >> >> date_i = datetime.date(2013, 07, 12) >> date_iso = date_i.isocalendar() >> >> year = str(date_iso[0])[2:] >> week = str(date_iso[1]) >> day = str(date_iso[2]) >> >> date_string = year + week + day > >But guess is there is a better way of doing the lines above >but I don't know the datetime module well enough to say what... Maybe this: >>> import datetime>>> "%d%02d%02d" % datetime.date(2013, 07, 12).isocalendar() '20132805' From elmar at net4werling.de Fri Jul 12 22:32:39 2013 From: elmar at net4werling.de (elmar werling) Date: Fri, 12 Jul 2013 22:32:39 +0200 Subject: [Tutor] datetime: inverse of datetime.date.isocalendar() In-Reply-To: References: Message-ID: Am 12.07.2013 21:36, schrieb Peter Otten: > elmar werling wrote: > >> how to convert the tuble (iso_year, iso_week, iso_day) to a date object? >> >> I have tried the following scipt, but the results are buggy >> >> >> import datetime >> >> date_i = datetime.date(2013, 07, 12) >> date_iso = date_i.isocalendar() >> >> year = str(date_iso[0])[2:] >> week = str(date_iso[1]) >> day = str(date_iso[2]) >> >> date_string = year + week + day >> date_ii = datetime.datetime.strptime(date_string,'%y%W%w').date() >> >> print date_i, date_iso, date_string, date_ii > > Here's what I came up with: > > import datetime > > def ywd_to_date(year, week, weekday): > """Convert (year, week, isoweekday) tuple to a datetime.date(). > > >>> datetime.date(2013, 7, 12).isocalendar() > (2013, 28, 5) > >>> ywd_to_date(2013, 28, 5) > datetime.date(2013, 7, 12) > """ > first = datetime.date(year, 1, 1) > first_year, _first_week, first_weekday = first.isocalendar() > > if first_year == year: > week -= 1 > > return first + datetime.timedelta(days=week*7+weekday-first_weekday) > > if __name__ == "__main__": > # perform an exhaustive adhoc test > > def iterdays(start=datetime.date(datetime.MINYEAR, 1, 1)): > """Generate dates till the end of time aka MAXYEAR.""" > d = start > one_day = datetime.timedelta(days=1) > try: > while True: > yield d > d += one_day > except OverflowError: > pass > > for d in iterdays(): > # does it round-trip? > c = ywd_to_date(*d.isocalendar()) > assert d == c, d > > I don't really understand what I'm doing here, but the test code makes me > confident that it's correct ;) > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > Hi Peter, your function is realy doing its job. Should be added to the datetime library !! Cheers Elmar From elmar at net4werling.de Fri Jul 12 22:42:38 2013 From: elmar at net4werling.de (elmar werling) Date: Fri, 12 Jul 2013 22:42:38 +0200 Subject: [Tutor] datetime: inverse of datetime.date.isocalendar() In-Reply-To: References: Message-ID: Am 12.07.2013 21:00, schrieb Alan Gauld: > On 12/07/13 17:52, elmar werling wrote: > >> how to convert the tuble (iso_year, iso_week, iso_day) to a date object? >> >> I have tried the following scipt, but the results are buggy > ,,,,,,,, >> >> print date_i, date_iso, date_string, date_ii > > I'm not sure what you expect to see or what differs > from your expectation? > > according to my expactation date_i should equal date_ii but instead print date_i, date_iso, date_string, date_ii 2013-07-12 (2013, 28, 5) 13285 2013-07-19 From jacklittlemc at yahoo.com Fri Jul 12 21:05:11 2013 From: jacklittlemc at yahoo.com (Jack Little) Date: Fri, 12 Jul 2013 21:05:11 +0200 Subject: [Tutor] Matrix Multiplication and its Inverse Message-ID: <96E62B87-DF20-4B9A-900A-38B35E1CBBF0@yahoo.com> Is there a way in python to do matrix multiplication and its inverse? No external modules is preferred, but it is ok. Sent from my iPod From cybervigilante at gmail.com Sat Jul 13 08:14:02 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Fri, 12 Jul 2013 23:14:02 -0700 Subject: [Tutor] A slight bug in IDLE Message-ID: Here's an odd one. If I run IDLE, which I thought was pretty reliable, and do this: from __future__ import division Then save the file and run the shell from IDLE, __future__ division fails. But if I then import __future__ division from the shell, it works. Also, if I print(5/3) from IDLE or assign 5/3 to a variable name, __future__ division then works okay I tried this with two other editors and they all failed like that, so they must be based on IDLE. This run is After I run the import from IDLE, which fails to register in the shell unless I assign a division to a variable or print it. Python 2.7.3 | 32-bit | (default, Jun 14 2013, 18:15:12) [MSC v.1500 32 bit (Intel)] on win32 Type "copyright", "credits" or "license()" for more information. >>> ================================ RESTART ================================ >>> >>> 5/3 1 >>> from __future__ import division >>> 5/3 1.6666666666666667 >>> By the way, why can't I just import __future__ ? That doesn't work. It always has to be from __future__ -- Jim From __peter__ at web.de Sat Jul 13 09:45:34 2013 From: __peter__ at web.de (Peter Otten) Date: Sat, 13 Jul 2013 09:45:34 +0200 Subject: [Tutor] Matrix Multiplication and its Inverse References: <96E62B87-DF20-4B9A-900A-38B35E1CBBF0@yahoo.com> Message-ID: Jack Little wrote: > Is there a way in python to do matrix multiplication and its inverse? No > external modules is preferred, but it is ok. Use numpy: >>> import numpy, numpy.linalg >>> a = numpy.array([[1, 2], [3, 4]]) >>> b = numpy.linalg.inv(a) >>> b array([[-2. , 1. ], [ 1.5, -0.5]]) >>> numpy.dot(a, b) array([[ 1., 0.], [ 0., 1.]]) From eryksun at gmail.com Sat Jul 13 09:54:19 2013 From: eryksun at gmail.com (eryksun) Date: Sat, 13 Jul 2013 03:54:19 -0400 Subject: [Tutor] A slight bug in IDLE In-Reply-To: References: Message-ID: On Sat, Jul 13, 2013 at 2:14 AM, Jim Mooney wrote: > > If I run IDLE, which I thought was pretty reliable, and do this: > > from __future__ import division > > Then save the file and run the shell from IDLE, __future__ division fails. > But if I then import __future__ division from the shell, it works. A __future__ import modifies compilation of the current module. The "division" directive tells the compiler to use BINARY_TRUE_DIVIDE instead of BINARY_DIVIDE. This works by setting a flag in the code object: >>> __future__.CO_FUTURE_DIVISION # flag 8192 >>> __future__.division _Feature((2, 2, 0, 'alpha', 2), (3, 0, 0, 'alpha', 0), 8192) The flag is inherited in interactive mode, so you only have to set it once: >>> sys._getframe().f_code.co_flags & CO_FUTURE_DIVISION 0 >>> from __future__ import division >>> sys._getframe().f_code.co_flags & CO_FUTURE_DIVISION 8192 Just having the division feature defined in a namespace doesn't affect the code flags. The compiler looks explicitly for "from __future__ ..." statements at the beginning and only the beginning[1] of the current compilation unit. >>> x = 5; y = 3; from __future__ import division; x / y File "", line 1 SyntaxError: from __future__ imports must occur at the beginning of the file [1] An initial string/docstring is allowed, as are multiple __future__ imports. Given this, can you figure out what's going on with IDLE? After running the file you see that "division" is defined, but the CO_FUTURE_DIVISION flag isn't set. > By the way, why can't I just import __future__ ? That doesn't work. It > always has to be from __future__ The compiler looks for an "ImportFrom" node in the abstract syntax tree: >>> print ast.dump(ast.parse('from __future__ import division').body[0]) ImportFrom(module='__future__', names=[alias(name='division', asname=None)], level=0) It looks specifically for the names as defined in compile.h, so you can't do a star import either. From cybervigilante at gmail.com Sat Jul 13 10:39:37 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 13 Jul 2013 01:39:37 -0700 Subject: [Tutor] A slight bug in IDLE In-Reply-To: References: Message-ID: On 13 July 2013 00:54, eryksun wrote: > A __future__ import modifies compilation of the current module. > Hmm, so if I import a module that uses truncated division, that's what I get, even though I imported __future__ division. OTOH, a non-future import will be used by a module imported after it. That's a gotcha to avoid ;') -- Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Sat Jul 13 12:11:48 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 13 Jul 2013 20:11:48 +1000 Subject: [Tutor] A slight bug in IDLE In-Reply-To: References: Message-ID: <51E127E4.6000109@pearwood.info> On 13/07/13 18:39, Jim Mooney wrote: > On 13 July 2013 00:54, eryksun wrote: > > >> A __future__ import modifies compilation of the current module. >> > > Hmm, so if I import a module that uses truncated division, that's what I > get, even though I imported __future__ division. OTOH, a non-future import > will be used by a module imported after it. That's a gotcha to avoid ;') No, actually, it's the opposite of a gotcha. If a module expects to use truncated division, and *fails* to "from __future__ import division", that's what it needs to get. If your import would change what the other module sees, then you could change the behaviour of the other module (and probably break it) just by importing something from __future__. By the way, you can import __future__, but when you do, it is just an ordinary module with no superpowers. Only the "from __future__ import ..." in the first or second line of code has superpowers. Try this: import __future__ dir(__future__) -- Steven From steve at pearwood.info Sat Jul 13 13:54:46 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 13 Jul 2013 21:54:46 +1000 Subject: [Tutor] Matrix Multiplication and its Inverse In-Reply-To: <96E62B87-DF20-4B9A-900A-38B35E1CBBF0@yahoo.com> References: <96E62B87-DF20-4B9A-900A-38B35E1CBBF0@yahoo.com> Message-ID: <51E14006.5050402@pearwood.info> On 13/07/13 05:05, Jack Little wrote: > Is there a way in python to do matrix multiplication and its inverse? No external modules is preferred, but it is ok. If you have numpy, you should use that. If you want a pure Python version, here's a quick and dirty matrix multiplier that works only for 2x2 matrices. def is_matrix(obj): if len(obj) == 2: return len(obj[0]) == 2 and len(obj[1]) == 2 return False def matrix_mult(A, B): """Return matrix A x B.""" if not (is_matrix(A) and is_matrix(B)): raise ValueError('not matrices') [a, b], [c, d] = A [w, x], [y, z] = B return [[a*w + b*y, a*x + b*z], [c*w + d*y, c*x + d*z]] I leave the inverse as an exercise :-) -- Steven From eryksun at gmail.com Sat Jul 13 14:43:45 2013 From: eryksun at gmail.com (eryksun) Date: Sat, 13 Jul 2013 08:43:45 -0400 Subject: [Tutor] A slight bug in IDLE In-Reply-To: References: Message-ID: On Sat, Jul 13, 2013 at 4:39 AM, Jim Mooney wrote: > On 13 July 2013 00:54, eryksun wrote: > >> >> A __future__ import modifies compilation of the current module. > > > Hmm, so if I import a module that uses truncated division, that's what I > get, even though I imported __future__ division. OTOH, a non-future import > will be used by a module imported after it. That's a gotcha to avoid ;') You wouldn't want to force true division on a module written for classic division -- or any other __future__ feature for that matter. FWIW, here's a simple overview of what's happening with IDLE. It's using a subclass of code.InteractiveInterpreter: import code interp = code.InteractiveInterpreter() interp.runsource('from __future__ import division') This saves the flags corresponding to __future__ imports in a codeop.Compiler instance: >>> ok = interp.runsource('print 5 / 3') 1.66666666667 >>> interp.compile.compiler.flags & CO_FUTURE_DIVISION 8192 However, "Run Module" just uses a vanilla compile() call. See the checksyntax() and run_module_event() methods in ScriptBinding.py [1]: run_module_event could update the flags before returning, like so: from codeop import _features compiler = interp.compile.compiler for feature in _features: if code.co_flags & feature.compiler_flag: compiler.flags |= feature.compiler_flag This would approximate running a script with -i (inspect) from the command line, which drops into the interactive loop using the current compiler flags (cf) [2]. [1] http://hg.python.org/cpython/file/2.7/Lib/idlelib/ScriptBinding.py [2] http://hg.python.org/cpython/file/ab05e7dd2788/Modules/main.c#l648 From eryksun at gmail.com Sat Jul 13 14:47:44 2013 From: eryksun at gmail.com (eryksun) Date: Sat, 13 Jul 2013 08:47:44 -0400 Subject: [Tutor] A slight bug in IDLE In-Reply-To: <51E127E4.6000109@pearwood.info> References: <51E127E4.6000109@pearwood.info> Message-ID: On Sat, Jul 13, 2013 at 6:11 AM, Steven D'Aprano wrote: > By the way, you can import __future__, but when you do, it is just an > ordinary module with no superpowers. Only the "from __future__ import ..." > in the first or second line of code has superpowers. Try this: > > import __future__ > dir(__future__) I thought this was clear in my description: >>> __future__.CO_FUTURE_DIVISION # flag 8192 >>> __future__.division _Feature((2, 2, 0, 'alpha', 2), (3, 0, 0, 'alpha', 0), 8192) Note to self: next time show "import __future__". From fomcl at yahoo.com Sat Jul 13 16:14:16 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Sat, 13 Jul 2013 07:14:16 -0700 (PDT) Subject: [Tutor] IOError when importing nose Message-ID: <1373724856.8043.YahooMailNeo@web163805.mail.gq1.yahoo.com> Hi, I am using nose on my laptop (installed it using sudo pip install nose), but I get the following error, unless I run python as sudo. Why is this happening, and what can I do to solve this problem? Maybe use chown? I uninstalled and reinstalled the package already, but it did not solve the problem. Python 2.7.3 (default, Sep 26 2012, 21:53:58) [GCC 4.7.2] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> import nose Traceback (most recent call last): ? File "", line 1, in ? File "/usr/local/lib/python2.7/dist-packages/nose/__init__.py", line 1, in ??? from nose.core import collector, main, run, run_exit, runmodule ? File "/usr/local/lib/python2.7/dist-packages/nose/core.py", line 11, in ??? from nose.config import Config, all_config_files ? File "/usr/local/lib/python2.7/dist-packages/nose/config.py", line 9, in ??? from nose.plugins.manager import NoPlugins ? File "/usr/local/lib/python2.7/dist-packages/nose/plugins/__init__.py", line 185, in ??? from nose.plugins.manager import * ? File "/usr/local/lib/python2.7/dist-packages/nose/plugins/manager.py", line 418, in ??? import pkg_resources ? File "/usr/local/lib/python2.7/dist-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 2823, in ??? add_activation_listener(lambda dist: dist.activate()) ? File "/usr/local/lib/python2.7/dist-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 710, in subscribe ??? callback(dist) ? File "/usr/local/lib/python2.7/dist-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 2823, in ??? add_activation_listener(lambda dist: dist.activate()) ? File "/usr/local/lib/python2.7/dist-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 2255, in activate ??? self.insert_on(path) ? File "/usr/local/lib/python2.7/dist-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 2362, in insert_on ??? self.check_version_conflict() ? File "/usr/local/lib/python2.7/dist-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 2401, in check_version_conflict ??? for modname in self._get_metadata('top_level.txt'): ? File "/usr/local/lib/python2.7/dist-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 2249, in _get_metadata ??? for line in self.get_metadata_lines(name): ? File "/usr/local/lib/python2.7/dist-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 1219, in get_metadata_lines ??? return yield_lines(self.get_metadata(name)) ? File "/usr/local/lib/python2.7/dist-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 1211, in get_metadata ??? return self._get(self._fn(self.egg_info,name)) ? File "/usr/local/lib/python2.7/dist-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 1326, in _get ??? stream = open(path, 'rb') IOError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/pypi_classifiers-0.1-py2.7.egg/EGG-INFO/top_level.txt' >>> quit() antonia at antonia-HP-2133 /usr/local/lib/python2.7/dist-packages/pypi_classifiers-0.1-py2.7.egg/EGG-INFO $ ls -l total 24 -rw------- 1 root staff??? 1 Jan 26 14:36 dependency_links.txt -rw------- 1 root staff 1311 Jan 26 14:36 PKG-INFO drwxr-sr-x 2 root staff 4096 Jan 26 14:36 scripts -rw------- 1 root staff? 269 Jan 26 14:36 SOURCES.txt -rw------- 1 root staff?? 16 Jan 26 14:36 top_level.txt -rw-r--r-- 1 root staff??? 1 Jan 26 14:36 zip-safe ?0.1-py2.7.egg/EGG-INFO $ antonia at antonia-HP-2133 /usr/local/lib/python2.7/dist-packages/pypi_classifiers-0.1-py2.7.egg/EGG-INFO $ cd scripts antonia at antonia-HP-2133 /usr/local/lib/python2.7/dist-packages/pypi_classifiers-0.1-py2.7.egg/EGG-INFO/scripts $ ls -l total 4 -rwxr-xr-x 1 root staff 1386 Jan 26 14:36 pypi-classifiers Thank you in advance! Regards, Albert-Jan ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health, what have the Romans ever done for us? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~? From wprins at gmail.com Sat Jul 13 16:54:25 2013 From: wprins at gmail.com (Walter Prins) Date: Sat, 13 Jul 2013 15:54:25 +0100 Subject: [Tutor] IOError when importing nose In-Reply-To: <1373724856.8043.YahooMailNeo@web163805.mail.gq1.yahoo.com> References: <1373724856.8043.YahooMailNeo@web163805.mail.gq1.yahoo.com> Message-ID: Hi, On 13 July 2013 15:14, Albert-Jan Roskam wrote: > IOError: [Errno 13] Permission denied: > '/usr/local/lib/python2.7/dist-packages/pypi_classifiers-0.1-py2.7.egg/EGG-INFO/top_level.txt' > >>> quit() > antonia at antonia-HP-2133/usr/local/lib/python2.7/dist-packages/pypi_classifiers-0.1-py2.7.egg/EGG-INFO > $ ls -l > total 24 > -rw------- 1 root staff 1 Jan 26 14:36 dependency_links.txt > -rw------- 1 root staff 1311 Jan 26 14:36 PKG-INFO > drwxr-sr-x 2 root staff 4096 Jan 26 14:36 scripts > -rw------- 1 root staff 269 Jan 26 14:36 SOURCES.txt > -rw------- 1 root staff 16 Jan 26 14:36 top_level.txt > -rw-r--r-- 1 root staff 1 Jan 26 14:36 zip-safe > > It appears to me this is not directly a problem with your nose installation. As you can see it's apparently some other package (pypi_classifiers) which has previously been installed and has incorrect file permissions on some files that's causing trouble. (As an aside re your nose installation, note that on Linux it's slightly preferable to use the package manager if available to install even Python packages, instead of pip if possible, e.g on Debian and variants (Ubuntu, Mint etc) you'd use: sudo apt-get install python-nose) That said, if you want to try and blanket fix your current permissions to give group and world (other) readability on all the files in dist-packages (probably a reasonable fix to do) you can do: sudo chmod -R go+r /usr/loca/lib/python2.7/dist-packages Some clarification of options: -R = recursive go = apply to "group" and "other" + = add permissions r = read permission You can also try uninstalling via pip again (sudo pip uninstall nose) and then try installing via the package manager again to see if this helps (but as above, I don't think it will, the problem appears to me not to be directly related to your nose installation.) Walter -------------- next part -------------- An HTML attachment was scrubbed... URL: From brettlea at mac.com Sat Jul 13 17:07:46 2013 From: brettlea at mac.com (Brett Wunderlich) Date: Sat, 13 Jul 2013 10:07:46 -0500 Subject: [Tutor] passing values to a callback In-Reply-To: References: Message-ID: Greetings pythonistas, I have been exploring the use of Tkinter trying to write a fairly simple program which will accept data from a GUI with 3 Entry fields that is commanded by a Button. Once the button is pressed the values in the fields should result in a calculation - the values should all be numeric (I'll work on catching errors later) but I can not figure out how to pass those values to the callback function for the button. Here is my code so far: from Tkinter import * class App: def __init__( self, master ): frame = Frame( master ) frame.grid() entry1 = Entry( frame, bg = "white", width = 3 ).grid(row = 0, column = 0 ) Label( frame, text = "X^2 +" ).grid( row = 0, column = 1 ) entry2 = Entry( frame, bg = "white", width = 3 ).grid(row = 0, column = 2 ) Label( frame, text = "X +" ).grid( row = 0, column = 3 ) entry3 = Entry( frame, bg = "white", width = 3 ).grid(row = 0, column = 4 ) Label( frame, text = "= 0" ).grid( row = 0, column = 5 ) calcbutton = Button( text = "How many Real roots?", command = self.calculate ) calcbutton.grid( row = 1, column = 0 ) def calculate( self ): print 1 root = Tk() root.geometry( '250x100+450+70' ) app = App( root ) root.mainloop() Now the above code runs, but I need to figure out how to get the contents of the fields entry1, entry2 and entry3 to the calculate method. Or maybe I am going about this all wrong and there is a better way to do it. Any help would be appreciated. Brett On Jul 13, 2013, at 9:54 AM, tutor-request at python.org wrote: > Send Tutor mailing list submissions to > tutor at python.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://mail.python.org/mailman/listinfo/tutor > or, via email, send a message with subject or body 'help' to > tutor-request at python.org > > You can reach the person managing the list at > tutor-owner at python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of Tutor digest..." > > > Today's Topics: > > 1. Re: A slight bug in IDLE (Steven D'Aprano) > 2. Re: Matrix Multiplication and its Inverse (Steven D'Aprano) > 3. Re: A slight bug in IDLE (eryksun) > 4. Re: A slight bug in IDLE (eryksun) > 5. IOError when importing nose (Albert-Jan Roskam) > 6. Re: IOError when importing nose (Walter Prins) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sat, 13 Jul 2013 20:11:48 +1000 > From: Steven D'Aprano > To: tutor at python.org > Subject: Re: [Tutor] A slight bug in IDLE > Message-ID: <51E127E4.6000109 at pearwood.info> > Content-Type: text/plain; charset=UTF-8; format=flowed > > On 13/07/13 18:39, Jim Mooney wrote: >> On 13 July 2013 00:54, eryksun wrote: >> >> >>> A __future__ import modifies compilation of the current module. >>> >> >> Hmm, so if I import a module that uses truncated division, that's what I >> get, even though I imported __future__ division. OTOH, a non-future import >> will be used by a module imported after it. That's a gotcha to avoid ;') > > > No, actually, it's the opposite of a gotcha. If a module expects to use truncated division, and *fails* to "from __future__ import division", that's what it needs to get. If your import would change what the other module sees, then you could change the behaviour of the other module (and probably break it) just by importing something from __future__. > > By the way, you can import __future__, but when you do, it is just an ordinary module with no superpowers. Only the "from __future__ import ..." in the first or second line of code has superpowers. Try this: > > import __future__ > dir(__future__) > > > -- > Steven > > > ------------------------------ > > Message: 2 > Date: Sat, 13 Jul 2013 21:54:46 +1000 > From: Steven D'Aprano > To: tutor at python.org > Subject: Re: [Tutor] Matrix Multiplication and its Inverse > Message-ID: <51E14006.5050402 at pearwood.info> > Content-Type: text/plain; charset=UTF-8; format=flowed > > On 13/07/13 05:05, Jack Little wrote: >> Is there a way in python to do matrix multiplication and its inverse? No external modules is preferred, but it is ok. > > > If you have numpy, you should use that. > > If you want a pure Python version, here's a quick and dirty matrix multiplier that works only for 2x2 matrices. > > > def is_matrix(obj): > if len(obj) == 2: > return len(obj[0]) == 2 and len(obj[1]) == 2 > return False > > > def matrix_mult(A, B): > """Return matrix A x B.""" > if not (is_matrix(A) and is_matrix(B)): > raise ValueError('not matrices') > [a, b], [c, d] = A > [w, x], [y, z] = B > return [[a*w + b*y, a*x + b*z], > [c*w + d*y, c*x + d*z]] > > > I leave the inverse as an exercise :-) > > > > -- > Steven > > > ------------------------------ > > Message: 3 > Date: Sat, 13 Jul 2013 08:43:45 -0400 > From: eryksun > To: Jim Mooney > Cc: tutor at python.org > Subject: Re: [Tutor] A slight bug in IDLE > Message-ID: > > Content-Type: text/plain; charset=UTF-8 > > On Sat, Jul 13, 2013 at 4:39 AM, Jim Mooney wrote: >> On 13 July 2013 00:54, eryksun wrote: >> >>> >>> A __future__ import modifies compilation of the current module. >> >> >> Hmm, so if I import a module that uses truncated division, that's what I >> get, even though I imported __future__ division. OTOH, a non-future import >> will be used by a module imported after it. That's a gotcha to avoid ;') > > You wouldn't want to force true division on a module written for > classic division -- or any other __future__ feature for that matter. > > FWIW, here's a simple overview of what's happening with IDLE. It's > using a subclass of code.InteractiveInterpreter: > > import code > > interp = code.InteractiveInterpreter() > interp.runsource('from __future__ import division') > > This saves the flags corresponding to __future__ imports in a > codeop.Compiler instance: > >>>> ok = interp.runsource('print 5 / 3') > 1.66666666667 >>>> interp.compile.compiler.flags & CO_FUTURE_DIVISION > 8192 > > However, "Run Module" just uses a vanilla compile() call. See the > checksyntax() and run_module_event() methods in ScriptBinding.py [1]: > > run_module_event could update the flags before returning, like so: > > from codeop import _features > compiler = interp.compile.compiler > for feature in _features: > if code.co_flags & feature.compiler_flag: > compiler.flags |= feature.compiler_flag > > This would approximate running a script with -i (inspect) from the > command line, which drops into the interactive loop using the current > compiler flags (cf) [2]. > > [1] http://hg.python.org/cpython/file/2.7/Lib/idlelib/ScriptBinding.py > [2] http://hg.python.org/cpython/file/ab05e7dd2788/Modules/main.c#l648 > > > ------------------------------ > > Message: 4 > Date: Sat, 13 Jul 2013 08:47:44 -0400 > From: eryksun > To: "Steven D'Aprano" > Cc: tutor at python.org > Subject: Re: [Tutor] A slight bug in IDLE > Message-ID: > > Content-Type: text/plain; charset=UTF-8 > > On Sat, Jul 13, 2013 at 6:11 AM, Steven D'Aprano wrote: >> By the way, you can import __future__, but when you do, it is just an >> ordinary module with no superpowers. Only the "from __future__ import ..." >> in the first or second line of code has superpowers. Try this: >> >> import __future__ >> dir(__future__) > > I thought this was clear in my description: > >>>> __future__.CO_FUTURE_DIVISION # flag > 8192 >>>> __future__.division > _Feature((2, 2, 0, 'alpha', 2), (3, 0, 0, 'alpha', 0), 8192) > > Note to self: next time show "import __future__". > > > ------------------------------ > > Message: 5 > Date: Sat, 13 Jul 2013 07:14:16 -0700 (PDT) > From: Albert-Jan Roskam > To: Python Mailing List > Subject: [Tutor] IOError when importing nose > Message-ID: > <1373724856.8043.YahooMailNeo at web163805.mail.gq1.yahoo.com> > Content-Type: text/plain; charset=iso-8859-1 > > Hi, > > I am using nose on my laptop (installed it using sudo pip install nose), but I get the following error, unless I run python as sudo. Why is this happening, and what can I do to solve this problem? Maybe use chown? I uninstalled and reinstalled the package already, but it did not solve the problem. > > > Python 2.7.3 (default, Sep 26 2012, 21:53:58) > [GCC 4.7.2] on linux2 > Type "help", "copyright", "credits" or "license" for more information. >>>> import nose > Traceback (most recent call last): > ? File "", line 1, in > ? File "/usr/local/lib/python2.7/dist-packages/nose/__init__.py", line 1, in > ??? from nose.core import collector, main, run, run_exit, runmodule > ? File "/usr/local/lib/python2.7/dist-packages/nose/core.py", line 11, in > ??? from nose.config import Config, all_config_files > ? File "/usr/local/lib/python2.7/dist-packages/nose/config.py", line 9, in > ??? from nose.plugins.manager import NoPlugins > ? File "/usr/local/lib/python2.7/dist-packages/nose/plugins/__init__.py", line 185, in > ??? from nose.plugins.manager import * > ? File "/usr/local/lib/python2.7/dist-packages/nose/plugins/manager.py", line 418, in > ??? import pkg_resources > ? File "/usr/local/lib/python2.7/dist-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 2823, in > ??? add_activation_listener(lambda dist: dist.activate()) > ? File "/usr/local/lib/python2.7/dist-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 710, in subscribe > ??? callback(dist) > ? File "/usr/local/lib/python2.7/dist-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 2823, in > ??? add_activation_listener(lambda dist: dist.activate()) > ? File "/usr/local/lib/python2.7/dist-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 2255, in activate > ??? self.insert_on(path) > ? File "/usr/local/lib/python2.7/dist-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 2362, in insert_on > ??? self.check_version_conflict() > ? File "/usr/local/lib/python2.7/dist-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 2401, in check_version_conflict > ??? for modname in self._get_metadata('top_level.txt'): > ? File "/usr/local/lib/python2.7/dist-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 2249, in _get_metadata > ??? for line in self.get_metadata_lines(name): > ? File "/usr/local/lib/python2.7/dist-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 1219, in get_metadata_lines > ??? return yield_lines(self.get_metadata(name)) > ? File "/usr/local/lib/python2.7/dist-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 1211, in get_metadata > ??? return self._get(self._fn(self.egg_info,name)) > ? File "/usr/local/lib/python2.7/dist-packages/distribute-0.6.34-py2.7.egg/pkg_resources.py", line 1326, in _get > ??? stream = open(path, 'rb') > IOError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/pypi_classifiers-0.1-py2.7.egg/EGG-INFO/top_level.txt' >>>> quit() > antonia at antonia-HP-2133 /usr/local/lib/python2.7/dist-packages/pypi_classifiers-0.1-py2.7.egg/EGG-INFO $ ls -l > total 24 > -rw------- 1 root staff??? 1 Jan 26 14:36 dependency_links.txt > -rw------- 1 root staff 1311 Jan 26 14:36 PKG-INFO > drwxr-sr-x 2 root staff 4096 Jan 26 14:36 scripts > -rw------- 1 root staff? 269 Jan 26 14:36 SOURCES.txt > -rw------- 1 root staff?? 16 Jan 26 14:36 top_level.txt > -rw-r--r-- 1 root staff??? 1 Jan 26 14:36 zip-safe > > ?0.1-py2.7.egg/EGG-INFO $ > antonia at antonia-HP-2133 /usr/local/lib/python2.7/dist-packages/pypi_classifiers-0.1-py2.7.egg/EGG-INFO $ cd scripts > antonia at antonia-HP-2133 /usr/local/lib/python2.7/dist-packages/pypi_classifiers-0.1-py2.7.egg/EGG-INFO/scripts $ ls -l > total 4 > -rwxr-xr-x 1 root staff 1386 Jan 26 14:36 pypi-classifiers > > Thank you in advance! > > > Regards, > Albert-Jan > > > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ > All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a > fresh water system, and public health, what have the Romans ever done for us? > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~? > > > ------------------------------ > > Message: 6 > Date: Sat, 13 Jul 2013 15:54:25 +0100 > From: Walter Prins > Cc: Python Mailing List > Subject: Re: [Tutor] IOError when importing nose > Message-ID: > > Content-Type: text/plain; charset="iso-8859-1" > > Hi, > > > On 13 July 2013 15:14, Albert-Jan Roskam wrote: > >> IOError: [Errno 13] Permission denied: >> '/usr/local/lib/python2.7/dist-packages/pypi_classifiers-0.1-py2.7.egg/EGG-INFO/top_level.txt' >>>>> quit() >> antonia at antonia-HP-2133/usr/local/lib/python2.7/dist-packages/pypi_classifiers-0.1-py2.7.egg/EGG-INFO >> $ ls -l >> total 24 >> -rw------- 1 root staff 1 Jan 26 14:36 dependency_links.txt >> -rw------- 1 root staff 1311 Jan 26 14:36 PKG-INFO >> drwxr-sr-x 2 root staff 4096 Jan 26 14:36 scripts >> -rw------- 1 root staff 269 Jan 26 14:36 SOURCES.txt >> -rw------- 1 root staff 16 Jan 26 14:36 top_level.txt >> -rw-r--r-- 1 root staff 1 Jan 26 14:36 zip-safe >> >> > It appears to me this is not directly a problem with your nose > installation. As you can see it's apparently some other package > (pypi_classifiers) which has previously been installed and has incorrect > file permissions on some files that's causing trouble. > > (As an aside re your nose installation, note that on Linux it's slightly > preferable to use the package manager if available to install even Python > packages, instead of pip if possible, e.g on Debian and variants (Ubuntu, > Mint etc) you'd use: sudo apt-get install python-nose) > > That said, if you want to try and blanket fix your current permissions to > give group and world (other) readability on all the files in dist-packages > (probably a reasonable fix to do) you can do: > > sudo chmod -R go+r /usr/loca/lib/python2.7/dist-packages > > Some clarification of options: > -R = recursive > go = apply to "group" and "other" > + = add permissions > r = read permission > > You can also try uninstalling via pip again (sudo pip uninstall nose) and > then try installing via the package manager again to see if this helps (but > as above, I don't think it will, the problem appears to me not to be > directly related to your nose installation.) > > > Walter > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > Tutor maillist - Tutor at python.org > http://mail.python.org/mailman/listinfo/tutor > > > ------------------------------ > > End of Tutor Digest, Vol 113, Issue 46 > ************************************** -------------- next part -------------- An HTML attachment was scrubbed... URL: From brettlea at mac.com Sat Jul 13 17:08:37 2013 From: brettlea at mac.com (Brett Wunderlich) Date: Sat, 13 Jul 2013 10:08:37 -0500 Subject: [Tutor] passing values to a callback Message-ID: <2F922F58-C601-458E-9AE0-39D973810F41@mac.com> Greetings pythonistas, I have been exploring the use of Tkinter trying to write a fairly simple program which will accept data from a GUI with 3 Entry fields that is commanded by a Button. Once the button is pressed the values in the fields should result in a calculation - the values should all be numeric (I'll work on catching errors later) but I can not figure out how to pass those values to the callback function for the button. Here is my code so far: from Tkinter import * class App: def __init__( self, master ): frame = Frame( master ) frame.grid() entry1 = Entry( frame, bg = "white", width = 3 ).grid(row = 0, column = 0 ) Label( frame, text = "X^2 +" ).grid( row = 0, column = 1 ) entry2 = Entry( frame, bg = "white", width = 3 ).grid(row = 0, column = 2 ) Label( frame, text = "X +" ).grid( row = 0, column = 3 ) entry3 = Entry( frame, bg = "white", width = 3 ).grid(row = 0, column = 4 ) Label( frame, text = "= 0" ).grid( row = 0, column = 5 ) calcbutton = Button( text = "How many Real roots?", command = self.calculate ) calcbutton.grid( row = 1, column = 0 ) def calculate( self ): print 1 root = Tk() root.geometry( '250x100+450+70' ) app = App(root) root.mainloop() Now the above code runs, but I need to figure out how to get the contents of the fields entry1, entry2 and entry3 to the calculate method. Or maybe I am going about this all wrong and there is a better way to do it. Any help would be appreciated. Brett -------------- next part -------------- An HTML attachment was scrubbed... URL: From fomcl at yahoo.com Sat Jul 13 18:33:41 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Sat, 13 Jul 2013 09:33:41 -0700 (PDT) Subject: [Tutor] IOError when importing nose In-Reply-To: References: <1373724856.8043.YahooMailNeo@web163805.mail.gq1.yahoo.com> Message-ID: <1373733221.99460.YahooMailNeo@web163801.mail.gq1.yahoo.com> ______________________________ > From: Walter Prins >To: >Cc: Python Mailing List >Sent: Saturday, July 13, 2013 4:54 PM >Subject: Re: [Tutor] IOError when importing nose > >On 13 July 2013 15:14, Albert-Jan Roskam wrote: > >IOError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/pypi_classifiers-0.1-py2.7.egg/EGG-INFO/top_level.txt' >>>>> quit() >>antonia at antonia-HP-2133 /usr/local/lib/python2.7/dist-packages/pypi_classifiers-0.1-py2.7.egg/EGG-INFO $ ls -l >>total 24 >>-rw------- 1 root staff??? 1 Jan 26 14:36 dependency_links.txt >>-rw------- 1 root staff 1311 Jan 26 14:36 PKG-INFO >>drwxr-sr-x 2 root staff 4096 Jan 26 14:36 scripts >>-rw------- 1 root staff? 269 Jan 26 14:36 SOURCES.txt >>-rw------- 1 root staff?? 16 Jan 26 14:36 top_level.txt >>-rw-r--r-- 1 root staff??? 1 Jan 26 14:36 zip-safe >It appears to me this is not directly a problem with your nose installation. ?As you can see it's apparently some other package (pypi_classifiers) which has previously been installed and has incorrect file permissions on some files that's causing trouble. ? > >(As an aside re your nose installation, note that on Linux it's slightly preferable to use the package manager if available to install even Python packages, instead of pip if possible, e.g on Debian and variants (Ubuntu, Mint etc) you'd use: sudo apt-get install python-nose) > >That said, if you want to try and blanket fix your current permissions to give group and world (other) readability on all the files in dist-packages (probably a reasonable fix to do) you can do: > >sudo chmod -R go+r /usr/loca/lib/python2.7/dist-packages > >Some clarification of options: >-R = recursive >go = apply to "group" and "other" >+ = add permissions >r = read permission > >You can also try uninstalling via pip again (sudo pip uninstall nose) and then try installing via the package manager again to see if this helps (but as above, I don't think it will, the problem appears to me not to be directly related to your nose installation.) Hi Walter, Fantastic, that worked! Thank you! I was hesitant to use chmod because I hadn't (at least not that I was aware of) been messing with the file permissions. Any idea what this may have caused? Before I had problems with this after I temporarily copied a directory (a repository) to a Windows NTFS partition. But that was on a different computer. Even though one learns a thing or two when this happens, it *always* happens on a wrong moment. ;-) Best wishes, Albert-Jan From alan.gauld at btinternet.com Sat Jul 13 19:48:41 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Sat, 13 Jul 2013 18:48:41 +0100 Subject: [Tutor] passing values to a callback In-Reply-To: <2F922F58-C601-458E-9AE0-39D973810F41@mac.com> References: <2F922F58-C601-458E-9AE0-39D973810F41@mac.com> Message-ID: On 13/07/13 16:08, Brett Wunderlich wrote: > commanded by a Button. Once the button is pressed the values in the > fields should result in a calculation - the values should all be numeric You'll need to convert between strings and numbers, the GUI displays strings (with one exception, see below) > class App: > > def __init__( self, master ): > frame = Frame( master ) > frame.grid() > entry1 = Entry( frame, bg = "white", width = 3 ).grid(row = 0, > column = 0 ) You want to store a reference to the entrey in your app so this should read: self.entry1 = Entry( frame, ....) > def calculate( self ): > print 1 Now you can access then entry objects using self print self.entry1.get() Or convert to an int (or float) as required. Finally you can write back to the entry widget the result string self.entry.insert(0,resultString) This is a common requirement so there is a bit of magic that Tk allows namely you can specify a StringVar or IntVar variable in your GUI definition and link it to an entry. The Tk core will then keep the content ogf the variable and widget in synch (and do the int conversion if appropriate). In your init method you can set up a StringVar using self.Entry1Var = StringVar() self.entry1['textvariable'] = self.Entry1Var or an int var like: self.Entry2Var = IntVar() self.entry1['textvariable'] = self.Entry2Var And you can read/change the value by get() or set() actions on the variables. There are also DoubleVar and BooleanVar available too. You can use them on some other types of widgets, but not all. Personally I don't find them that big a help and often just access the widget directly. HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From cybervigilante at gmail.com Sat Jul 13 20:29:08 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 13 Jul 2013 11:29:08 -0700 Subject: [Tutor] A slight bug in IDLE In-Reply-To: <51E127E4.6000109@pearwood.info> References: <51E127E4.6000109@pearwood.info> Message-ID: Steven D'Aprano No, actually, it's the opposite of a gotcha. If a module expects to use truncated division, and *fails* to "from __future__ import division", that's what it needs to get. If your import would change what the other module sees, then you could change the behaviour of the other module (and probably break it) just by importing something from __future__. ==== > \ Or worse, break it in a way that wasn't apparent until the plane fell out of the sky ;') Actually, I meant the reverse gotcha. Expecting float division when your imported module stuck to truncated. But either way it's good to know __future__ is not your usual sort of import. Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From davea at davea.name Sat Jul 13 20:34:29 2013 From: davea at davea.name (Dave Angel) Date: Sat, 13 Jul 2013 14:34:29 -0400 Subject: [Tutor] passing values to a callback In-Reply-To: References: <2F922F58-C601-458E-9AE0-39D973810F41@mac.com> Message-ID: On 07/13/2013 01:48 PM, Alan Gauld wrote: > On 13/07/13 16:08, Brett Wunderlich wrote: > >> commanded by a Button. Once the button is pressed the values in the >> fields should result in a calculation - the values should all be numeric > > You'll need to convert between strings and numbers, the GUI displays > strings (with one exception, see below) > >> class App: >> >> def __init__( self, master ): >> frame = Frame( master ) >> frame.grid() >> entry1 = Entry( frame, bg = "white", width = 3 ).grid(row = 0, >> column = 0 ) > > You want to store a reference to the entrey in your app so this should > read: > > self.entry1 = Entry( frame, ....) Whoops. Brett had chained the reference with his call to grid(), so this would store a None. The line needs to be split up: self.entry1 = Entry( frame, bg = "white", width = 3 ) self.entry1.grid(row = 0, column = 0 ) Otherwise, Alan's answer is right-on. If you want to give an initial string value, then add: self.entry1.insert(0, "42") > >> def calculate( self ): >> print 1 > > Now you can access then entry objects using self > > print self.entry1.get() > > Or convert to an int (or float) as required. > > Finally you can write back to the entry widget the result string > self.entry.insert(0,resultString) > > This is a common requirement so there is a bit of magic that Tk > allows namely you can specify a StringVar or IntVar variable in > your GUI definition and link it to an entry. The Tk core will > then keep the content ogf the variable and widget in synch > (and do the int conversion if appropriate). > > In your init method you can set up a StringVar using > > self.Entry1Var = StringVar() > self.entry1['textvariable'] = self.Entry1Var > > or an int var like: > > self.Entry2Var = IntVar() > self.entry1['textvariable'] = self.Entry2Var > > > And you can read/change the value by get() or set() > actions on the variables. > > There are also DoubleVar and BooleanVar available too. > You can use them on some other types of widgets, but not all. > > Personally I don't find them that big a help and often just access the > widget directly. > > HTH -- DaveA From davea at davea.name Sat Jul 13 21:12:13 2013 From: davea at davea.name (Dave Angel) Date: Sat, 13 Jul 2013 15:12:13 -0400 Subject: [Tutor] A slight bug in IDLE In-Reply-To: References: <51E127E4.6000109@pearwood.info> Message-ID: On 07/13/2013 02:29 PM, Jim Mooney wrote: > Steven D'Aprano > > No, actually, it's the opposite of a gotcha. If a module expects to use > truncated division, and *fails* to "from __future__ import division", > that's what it needs to get. If your import would change what the other > module sees, then you could change the behaviour of the other module (and > probably break it) just by importing something from __future__. > ==== > >> \ > > Or worse, break it in a way that wasn't apparent until the plane fell out > of the sky ;') > > Actually, I meant the reverse gotcha. Expecting float division when your > imported module stuck to truncated. But either way it's good to know > __future__ is not your usual sort of import. > You still don't understand. If you write a module and I import it, then other imports BY ME don't affect your module. Whether it's a __future__ one or not. My import affects my global namespace, not yours. My compile-time switches affect my divide, not yours. Same thing. *Unless of course, we're in a case of recursive imports, or doing monkey-patching, or code-generation. Same thing with the encoding line at the beginning of my module. It affects how string literals and symbol names are to be decoded in my module, not in yours. It's perfectly reasonable and safe to have some source files encoded in utf-8 and some in latin-1. Pain in the neck for the text editor, but that's why the encoding string is flexible, so that editors like emacs can honor the same one. -- DaveA From cybervigilante at gmail.com Sun Jul 14 01:29:35 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 13 Jul 2013 16:29:35 -0700 Subject: [Tutor] A slight bug in IDLE In-Reply-To: References: <51E127E4.6000109@pearwood.info> Message-ID: Dave Angel You still don't understand. If you write a module and I import it, then > other imports BY ME don't affect your module. Whether it's a __future__ one > or not. My import affects my global namespace, not yours. My compile-time > switches affect my divide, not yours. Same thing. > Ah, clarity. This namespace stuff is cool, but it takes getting used to. However, it makes perfect sense that I never want to alter the original intent of the programmer of the module I'm importing, by clobbering his names. Where I got confused is I'm importing a module with a manual input, then renaming the input function to a generator function in my program , so I can bypass manual input and feed the module tons of data. That works fine, but I didn't follow how the imported module code could see that it should go to my generator function instead of the manual input function, unless I was changing the imported module in some way. But it's now clear I'm not. Which brings up a question. I finally settled on Python 2.7 for various reasons, but find some 3.3 things useful. Generators are one-off and input is one-off, so they match well for testing, for instance. I checked the docs and I don't see many __future__ imports. Are these all there are, or are there other useful ones that are better hidden? - nested_scopes, generators, division, absolute_import, with_statement print_function unicode_literals I'm not importing them all, just what seems useful to me at this point, and renaming raw_input, so my 2.7 is kind of 3.3ish. But is there anything I missed or any problems I'm unaware of in what will be my standard header, below? #Using Python 2.7 on Win 7 from __future__ import generators, division, with_statement, print_function import sys if int(sys.version[0]) < 3: input = raw_input Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From davea at davea.name Sun Jul 14 01:45:57 2013 From: davea at davea.name (Dave Angel) Date: Sat, 13 Jul 2013 19:45:57 -0400 Subject: [Tutor] A slight bug in IDLE In-Reply-To: References: <51E127E4.6000109@pearwood.info> Message-ID: On 07/13/2013 07:29 PM, Jim Mooney wrote: > Dave Angel > > You still don't understand. If you write a module and I import it, then >> other imports BY ME don't affect your module. Whether it's a __future__ one >> or not. My import affects my global namespace, not yours. My compile-time >> switches affect my divide, not yours. Same thing. >> > > Ah, clarity. This namespace stuff is cool, but it takes getting used to. > However, it makes perfect sense that I never want to alter the original > intent of the programmer of the module I'm importing, by clobbering his > names. > > Where I got confused is I'm importing a module with a manual input, then > renaming the input function to a generator function in my program , so I > can bypass manual input and feed the module tons of data. That works fine, > but I didn't follow how the imported module code could see that it should > go to my generator function instead of the manual input function, unless I > was changing the imported module in some way. But it's now clear I'm not. > I really couldn't follow that paragraph. What's a manual input? What do you mean renaming the input function (you mean the input statement?) to a generator function? Perhaps this would better be a new thread you should start. > Which brings up a question. I finally settled on Python 2.7 for various > reasons, but find some 3.3 things useful. Generators are one-off and input > is one-off, so they match well for testing, for instance. I checked the > docs and I don't see many __future__ imports. Are these all there are, or > are there other useful ones that are better hidden? - nested_scopes, > generators, division, absolute_import, with_statement print_function > unicode_literals > > I'm not importing them all, just what seems useful to me at this point, and > renaming raw_input, so my 2.7 is kind of 3.3ish. But is there anything I > missed or any problems I'm unaware of in what will be my standard header, > below? > > #Using Python 2.7 on Win 7 > from __future__ import generators, division, with_statement, print_function > import sys > if int(sys.version[0]) < 3: input = raw_input > > Jim > > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > -- DaveA From davea at davea.name Sun Jul 14 01:56:56 2013 From: davea at davea.name (Dave Angel) Date: Sat, 13 Jul 2013 19:56:56 -0400 Subject: [Tutor] A slight bug in IDLE In-Reply-To: References: <51E127E4.6000109@pearwood.info> Message-ID: On 07/13/2013 07:45 PM, Dave Angel wrote: > On 07/13/2013 07:29 PM, Jim Mooney wrote: >> Dave Angel >> >> >> Where I got confused is I'm importing a module with a manual input, then >> renaming the input function to a generator function in my program , so I >> can bypass manual input and feed the module tons of data. That works >> fine, >> but I didn't follow how the imported module code could see that it should >> go to my generator function instead of the manual input function, >> unless I >> was changing the imported module in some way. But it's now clear I'm not. >> > I really couldn't follow that paragraph. What's a manual input? What > do you mean renaming the input function (you mean the input statement?) > to a generator function? Perhaps this would better be a new thread you > should start. Oops. I meant input()/raw_input() function, not statement. The one in builtins. But these as opposed to any old function that happens to do input. -- DaveA From jalespring at gmail.com Sun Jul 14 02:32:53 2013 From: jalespring at gmail.com (Arnel Legaspi) Date: Sun, 14 Jul 2013 08:32:53 +0800 Subject: [Tutor] Writing unit tests that involve email In-Reply-To: <51E14B6B.8060609@zoho.com> References: <51E14B6B.8060609@zoho.com> Message-ID: <51E1F1B5.3060302@gmail.com> Hello, I have written a small Python script that allows the user to post to their blogs via email, converting the draft written in Markdown & sending it to a blog's email address (like what they had in Posterous back then). https://bitbucket.org/acl79/downpost At this point, I've been trying to write unit tests for it to get some practice writing unit tests for other possibly larger projects in the future. I've been using the dsmtpd script available at: https://github.com/matrixise/dsmtpd to run a SMTP server in the background and route all the emails being sent by my script to localhost. The script allows saving the messages to an optional Maildir directory, and I wanted to use that to be able to check various fields in the email for correctness in the unit tests. What I'm having difficulty is getting the unit tests to properly run the script being tested and send the email, get the proper Maildir message, and be able to read it via the unit test script. Right now, even when the SMTP server is not running, all the tests for checking the sent email pass, which is not what I intended. I know the SMTP server works, because if I send some email using another Python or Powershell script, the script logs on the screen that it did receive the email. How should I approach this? Here is the unit test script I have (some lines may get wrapped): #!/usr/bin/env python import os import sys import argparse import ConfigParser import unittest import mailbox MKDOWN_TEXT = """ **Lorem ipsum dolor sit amet**, consectetur adipiscing elit. Sed dapibus dui nibh, ut fermentum dolor tristique fringilla. Pellentesque tristique sagittis dapibus. Nunc pellentesque nisi vitae arcu lacinia, nec ultricies libero rutrum. *Vivamus* enim augue, malesuada in dui ac, laoreet faucibus dolor. * Aliquam eu nisi dictum, tristique eros ut, eleifend elit. * Pellentesque lacinia sodales erat non rhoncus. * Aliquam mattis ullamcorper purus sit amet sodales. Ut mollis ligula id sapien vehicula fringilla eu auctor diam. Praesent consequat tristique arcu, nec iaculis metus porta at. Nam mauris orci, congue vel consectetur vitae, viverra adipiscing urna. > Donec tristique leo erat, sit amet feugiat velit semper sit amet. > Nunc sagittis libero risus, at mollis mauris pharetra quis. Vivamus > adipiscing porttitor velit, sit amet sodales risus mollis quis. Donec sed risus orci. Cras fringilla at lorem ut venenatis. Pellentesque varius lorem neque, euplace rat eros dictum nec. """ HTML_TXT = """

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed dapibus dui nibh, ut fermentum dolor tristique fringilla. Pellentesque tristique sagittis dapibus. Nunc pellentesque nisi vitae arcu lacinia, nec ultricies libero rutrum. Vivamus enim augue, malesuada in dui ac, laoreet faucibus dolor.

  • Aliquam eu nisi dictum, tristique eros ut, eleifend elit.
  • Pellentesque lacinia sodales erat non rhoncus.
  • Aliquam mattis ullamcorper purus sit amet sodales.

Ut mollis ligula id sapien vehicula fringilla eu auctor diam. Praesent consequat tristique arcu, nec iaculis metus porta at. Nam mauris orci, congue vel consectetur vitae, viverra adipiscing urna.

Donec tristique leo erat, sit amet feugiat velit semper sit amet. Nunc sagittis libero risus, at mollis mauris pharetra quis. Vivamus adipiscing porttitor velit, sit amet sodales risus mollis quis.

Donec sed risus orci. Cras fringilla at lorem ut venenatis. Pellentesque varius lorem neque, euplace rat eros dictum nec.

""" TESTARGS = [] def clear_args(): """ Clear the arguments & the internal list (TESTARGS) used for test arguments. This is needed to allow the script to be tested with different arguments. """ del sys.argv[1:] TESTARGS[:] = [] return def add_testfiles(contentfile='post.md', configfile='blog.conf'): """ Clear the arguments & the internal list used. Add the regular test files into the argument list instead. """ clear_args() TESTARGS.append(contentfile) TESTARGS.append(configfile) TESTARGS.append('--debug') sys.argv.append(TESTARGS) return def create_configfile(): """ Create a dummy config file for testing purposes. """ parser = ConfigParser.SafeConfigParser(allow_no_value=True) parser.add_section('blog') parser.set('blog', 'email', 'test at email.net') parser.add_section('smtp') parser.set('smtp', 'server', '127.0.0.1') parser.set('smtp', 'port', '8000') parser.set('smtp', 'username', 'user at wherever.com') parser.set('smtp', 'password', 'DeJpkZa3WL') with open('blog.conf', 'wb') as configfile: parser.write(configfile) class TestReadingSendingPostFile(unittest.TestCase): def setUp(self): """ Prepare a post file for testing. """ create_configfile() with open('post.md', 'wb') as postfile: postfile.write(MKDOWN_TEXT) self.mbox = mailbox.Maildir('mail_dir') def test_wrongconfigfile(self): clear_args() add_testfiles('post.md', 'bloginfo.ini') self.assertRaises(IOError) def test_wrongpostfile(self): clear_args() add_testfiles('some_nonexistent_file.md', 'blog.conf') self.assertRaises(IOError) def test_sendpost(self): clear_args() add_testfiles() for message in self.mbox: self.assertEqual(message['subject'], 'post') self.assertEqual(message.get_payload(), HTML_TXT) def test_sendrawpost(self): clear_args() TESTARGS.append('--raw') add_testfiles() for message in self.mbox: self.assertEqual(message['subject'], 'post') self.assertEqual(message.get_payload(), MKDOWN_TEXT) def test_sendpost_withunderscores(self): clear_args() os.rename('post.md', 'This_Should_Be_Posted_Without_Underscores.md') add_testfiles('This_Should_Be_Posted_Without_Underscores.md', 'blog.conf') for message in self.mbox: self.assertEqual(message['subject'], 'This Should Be Posted Without Underscores') self.assertEqual(message.get_payload(), HTML_TXT) def test_sendpost_withspaces(self): clear_args() os.rename('post.md', 'This Should Be Posted As Is.md') add_testfiles('This Should Be Posted As Is.md', 'blog.conf') for message in self.mbox: self.assertEqual(message['subject'], 'This Should Be Posted As Is') self.assertEqual(message.get_payload(), HTML_TXT) def test_sendpost_withsettitle(self): clear_args() TESTARGS.append('-t "This will be the new title"') add_testfiles() for message in self.mbox: self.assertEqual(message['subject'], 'This will be the new title') self.assertEqual(message.get_payload(), HTML_TXT) def tearDown(self): if os.path.exists('post.md'): os.remove('post.md') if os.path.exists('This_Should_Be_Posted_Without_Underscores.md'): os.remove('This_Should_Be_Posted_Without_Underscores.md') if os.path.exists('This Should Be Posted As Is.md'): os.remove('This Should Be Posted As Is.md') os.remove('blog.conf') if __name__ == '__main__': argp = argparse.ArgumentParser() argp.add_argument('unittest_args', nargs='*') args = argp.parse_args() unit_args = [sys.argv[0]] + args.unittest_args unittest.main(argv=unit_args) Any comments on this & the script I'm writing tests for would be appreciated. Thanks, Arnel From eryksun at gmail.com Sun Jul 14 04:18:18 2013 From: eryksun at gmail.com (eryksun) Date: Sat, 13 Jul 2013 22:18:18 -0400 Subject: [Tutor] IOError when importing nose In-Reply-To: <1373733221.99460.YahooMailNeo@web163801.mail.gq1.yahoo.com> References: <1373724856.8043.YahooMailNeo@web163805.mail.gq1.yahoo.com> <1373733221.99460.YahooMailNeo@web163801.mail.gq1.yahoo.com> Message-ID: On Sat, Jul 13, 2013 at 12:33 PM, Albert-Jan Roskam wrote: > > Fantastic, that worked! Thank you! I was hesitant to use chmod because I > hadn't (at least not that I was aware of) been messing with the file > permissions. Any idea what this may have caused? How did you install pypi_classifiers? I pip installed it in Debian (Jessie) without a hitch, so the problem isn't specific to the pypi_classifiers setup script or the version of pip that I used (1.3.1). From steve at pearwood.info Sun Jul 14 05:03:05 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 14 Jul 2013 13:03:05 +1000 Subject: [Tutor] A slight bug in IDLE In-Reply-To: References: <51E127E4.6000109@pearwood.info> Message-ID: <51E214E9.20708@pearwood.info> On 14/07/13 09:29, Jim Mooney wrote: > Which brings up a question. I finally settled on Python 2.7 for various > reasons, but find some 3.3 things useful. Generators are one-off and input > is one-off, so they match well for testing, for instance. I don't understand that last sentence. > I checked the > docs and I don't see many __future__ imports. Are these all there are, or > are there other useful ones that are better hidden? Try this one: from __future__ import braces > - nested_scopes, > generators, division, absolute_import, with_statement print_function > unicode_literals As of Python 3.3, the full list of __future__ imports are: py> import __future__ py> __future__.all_feature_names ['nested_scopes', 'generators', 'division', 'absolute_import', 'with_statement', 'print_function', 'unicode_literals', 'barry_as_FLUFL'] __future__ features will never be removed, even when they no longer have an effect. For example, nested scopes have become standard since Python 2.2, but "from __future__ import nested_scopes" will be legal (and a no-op) so long as Python exists. There may be new __future__ features in the future. For example, I wouldn't be surprised if Python 3.5 or 3.6 introduces "from __future__ import decimal_literals" or some such thing. > I'm not importing them all, just what seems useful to me at this point, and > renaming raw_input, so my 2.7 is kind of 3.3ish. But is there anything I > missed or any problems I'm unaware of in what will be my standard header, > below? > > #Using Python 2.7 on Win 7 > from __future__ import generators, division, with_statement, print_function > import sys > if int(sys.version[0]) < 3: input = raw_input You don't need generators in 2.7, they have been standard since Python 2.4. -- Steven From cybervigilante at gmail.com Sun Jul 14 06:37:14 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 13 Jul 2013 21:37:14 -0700 Subject: [Tutor] A slight bug in IDLE In-Reply-To: <51E214E9.20708@pearwood.info> References: <51E127E4.6000109@pearwood.info> <51E214E9.20708@pearwood.info> Message-ID: On 13 July 2013 20:03, Steven D'Aprano wrote: > > I don't understand that last sentence. Ah, I can be marvelously unclear. I keep forgetting telepathy only works on my home planet ;') I wiped everything out to start fresh with Py27, so this is just a very simple example of what worked in sending a few million test integers to my numbers-to-name program, which normally took typed input. The real test program used randoms to go up to the decillions and a straight count to go into the quadrillions, for about 40 tests (before my CPU overheated, this being Arizona ;') No doubt there is a standard way to do this, but I wanted to test the numbers program quickly, without mooching all over the net, so I threw something like this together: # The module I import, inputter.py: #Using Python 2.7 on Win 7 def inp(): x = raw_input('type a number: ') return x def intest(): if int(inp()) < 10: print('smallnum') else: print('bignum') def main(): intest() if __name__ == '__main__': main() # The Test Program, test.py that imports the module above: # Using Python 2.7 on Win 7 import inputter x = (x for x in xrange(8,13)) # much bigger, and sometimes random, in the real test program def testfunc(): # This is where my typed input is replaced return next(x) inputter.inp = testfunc for cnt in range(0,5): inputter.intest() '''result: smallnum smallnum bignum bignum bignum ''' -------------- next part -------------- An HTML attachment was scrubbed... URL: From eryksun at gmail.com Sun Jul 14 06:41:40 2013 From: eryksun at gmail.com (eryksun) Date: Sun, 14 Jul 2013 00:41:40 -0400 Subject: [Tutor] A slight bug in IDLE In-Reply-To: <51E214E9.20708@pearwood.info> References: <51E127E4.6000109@pearwood.info> <51E214E9.20708@pearwood.info> Message-ID: On Sat, Jul 13, 2013 at 11:03 PM, Steven D'Aprano wrote: > > from __future__ import braces That joke is hard-coded, going back to version 2.1 when __future__ imports were added. It's also implemented in Jython and PyPy. http://hg.python.org/cpython/file/2.1/Python/future.c http://hg.python.org/cpython/file/3.3/Python/future.c (extremely off topic: look how simple the future_parse function is using an abstract syntax tree compared to working with the low-level tree in 2.1.) >> from __future__ import generators, division, with_statement, >> print_function >> import sys >> if int(sys.version[0]) < 3: input = raw_input > > > You don't need generators in 2.7, they have been standard since Python 2.4. A lot of new packages support versions 2.6+, which have the "with" statement enabled. @Jim: Don't parse sys.version like that. Use sys.version_info[0]. From cybervigilante at gmail.com Sun Jul 14 07:02:32 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 13 Jul 2013 22:02:32 -0700 Subject: [Tutor] A slight bug in IDLE In-Reply-To: References: <51E127E4.6000109@pearwood.info> <51E214E9.20708@pearwood.info> Message-ID: On 13 July 2013 21:41, eryksun wrote: > > A lot of new packages support versions 2.6+, which have the "with" > statement enabled. > > So, since I'm using 2.7 I don't need generators or with? Then all I'm using to be 3.3ish, would be: from __future__ import division, print_function import sys if int(sys.version_info[0]) < 3: input = raw_input range = xrange for my header macro. I'll try to remember about Tkinter. Anything else missing of great import ;') I don't want that to get too big - just the minimum. -- Jim Our next land war will be a "humanitarian" war in the former Eastern bloc, which will Really be over oil, gas, and minerals. The Army is already preparing. You heard it here first. -------------- next part -------------- An HTML attachment was scrubbed... URL: From davea at davea.name Sun Jul 14 07:16:54 2013 From: davea at davea.name (Dave Angel) Date: Sun, 14 Jul 2013 01:16:54 -0400 Subject: [Tutor] A slight bug in IDLE In-Reply-To: References: <51E127E4.6000109@pearwood.info> <51E214E9.20708@pearwood.info> Message-ID: On 07/14/2013 12:37 AM, Jim Mooney wrote: > On 13 July 2013 20:03, Steven D'Aprano wrote: > >> >> I don't understand that last sentence. > > > Ah, I can be marvelously unclear. I keep forgetting telepathy only works > on my home planet ;') You still have done nothing to explain the "last sentence." You obscure that fact by not even keeping the context as to what the sentence was. Repeating it for your edification, and our mystification: """Generators are one-off and input is one-off, so they match well for testing, for instance. """ > I wiped everything out to start fresh with Py27, so this is just a very > simple example of what worked in sending a few million test integers to my > numbers-to-name program, which normally took typed input. The real test > program used randoms to go up to the decillions and a straight count to go > into the quadrillions, for about 40 tests (before my CPU overheated, this > being Arizona ;') > > No doubt there is a standard way to do this, but I wanted to test the > numbers program quickly, without mooching all over the net, so I threw > something like this together: > > # The module I import, inputter.py: > #Using Python 2.7 on Win 7 > > def inp(): > x = raw_input('type a number: ') > return x > > def intest(): > if int(inp()) < 10: > print('smallnum') > else: > print('bignum') > > def main(): > intest() > > if __name__ == '__main__': > main() > > > # The Test Program, test.py that imports the module above: > # Using Python 2.7 on Win 7 > > import inputter > > x = (x for x in xrange(8,13)) # much bigger, and sometimes random, in the > real test program > > def testfunc(): # This is where my typed input is replaced > return next(x) > > inputter.inp = testfunc That's monkey-patching. You're reaching inside the module and modifying it. That's fine since you wrote both of them, but it makes things quite hard to debug in more complex cases, and especially if you didn't write the target module. > > for cnt in range(0,5): > inputter.intest() > > '''result: > smallnum > smallnum > bignum > bignum > bignum > ''' One of the techniques the inputter module should have used is a callback. You pass a function as a parameter, and it calls that function. You can make a default value, of course, to make it easy to test. Anyway, in inputter.py, you'd have def intest(func=inp): if int(func()) < 10: print('smallnum') else: print('bignum') and in your calling module, you would remove the monkeypatch, and just call for cnt in range(0,5): inputter.intest(testfunc) The real problem I have is with your original code. It mixed input, and calculations inside the same function, so you had to patch things up to test it from outside. If you had refactored it as I suggested way back then, and Steven suggested even better, this all would have been a non-problem. -- DaveA From davea at davea.name Sun Jul 14 07:25:38 2013 From: davea at davea.name (Dave Angel) Date: Sun, 14 Jul 2013 01:25:38 -0400 Subject: [Tutor] A slight bug in IDLE In-Reply-To: References: <51E127E4.6000109@pearwood.info> <51E214E9.20708@pearwood.info> Message-ID: On 07/14/2013 01:02 AM, Jim Mooney wrote: > On 13 July 2013 21:41, eryksun wrote: > >> >> A lot of new packages support versions 2.6+, which have the "with" >> statement enabled. >> >> So, since I'm using 2.7 I don't need generators or with? > > Then all I'm using to be 3.3ish, would be: > > from __future__ import division, print_function > import sys > if int(sys.version_info[0]) < 3: > input = raw_input > range = xrange Any time you can avoid making specific version # checks, you're better off. When I was dealing with Microsoft version changes, I frequently ran into problems where one piece of code would lie about versions just to fool certain libraries into compensating, and the resulting mess was a morass of bugs waiting to pop. Remember when Windows 4.0 had an internal version of 3.99, so that some obscure package would still be willing to work? try: input = raw_input except NameError as e: pass try: range = xrange except NameError as e: pass > > for my header macro. I'll try to remember about Tkinter. > > Anything else missing of great import ;') I don't want that to get too big > - just the minimum. > > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > -- DaveA From cybervigilante at gmail.com Sun Jul 14 10:27:21 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sun, 14 Jul 2013 01:27:21 -0700 Subject: [Tutor] A slight bug in IDLE In-Reply-To: References: <51E127E4.6000109@pearwood.info> <51E214E9.20708@pearwood.info> Message-ID: On 13 July 2013 22:16, Dave Angel wrote: That's monkey-patching. You're reaching inside the module and modifying > it. That's interesting. I saw that phrase and wondered what it was. Now I know ;') Actually, I did refactor the numbers program into disparate functions. That's why I started using main(), so I could import it and just use the functions. That test would have been impossible if everything was still mashed up. In fact, once I refactored, I saw how easy it would be to test like that. But callbacks look interesting. I'll have to try them. Except using a callback would mean editing the inputter module, which I am pretending is some module code I'm importing but don't want to mess with. So I just import it and do the so-called monkey patch without actually editing the module. My thought was I could do quick tests and dummy inputs without changing the module itself. Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From davea at davea.name Sun Jul 14 14:19:58 2013 From: davea at davea.name (Dave Angel) Date: Sun, 14 Jul 2013 08:19:58 -0400 Subject: [Tutor] A slight bug in IDLE In-Reply-To: References: <51E127E4.6000109@pearwood.info> <51E214E9.20708@pearwood.info> Message-ID: On 07/14/2013 04:27 AM, Jim Mooney wrote: > On 13 July 2013 22:16, Dave Angel wrote: > > That's monkey-patching. You're reaching inside the module and modifying >> it. > > > That's interesting. I saw that phrase and wondered what it was. Now I know > ;') Actually, I did refactor the numbers program into disparate functions. > That's why I started using main(), so I could import it and just use the > functions. That test would have been impossible if everything was still > mashed up. In fact, once I refactored, I saw how easy it would be to test > like that. But callbacks look interesting. I'll have to try them. Note that callbacks is just one approach to writing flexible modules. Another approach is writing the behavior into a class which may be subclassed. The user then writes a subclass in which he/she has written some overriding methods. If you want to see a sample of how these are done, study any GUI library. > > Except using a callback would mean editing the inputter module, which I am > pretending is some module code I'm importing but don't want to mess with. Exactly. So you're simulating what it's like to use a module whose author didn't plan ahead for your particular type of customization. > So I just import it and do the so-called monkey patch without actually > editing the module. My thought was I could do quick tests and dummy inputs > without changing the module itself. > -- DaveA From cybervigilante at gmail.com Sun Jul 14 21:28:30 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sun, 14 Jul 2013 12:28:30 -0700 Subject: [Tutor] A slight bug in IDLE In-Reply-To: References: <51E127E4.6000109@pearwood.info> <51E214E9.20708@pearwood.info> Message-ID: On 13 July 2013 22:25, Dave Angel wrote: try: > input = raw_input > except NameError as e: > pass > > try: > range = xrange > except NameError as e: > pass > > ====== Couldn't I just shorten that to: if int(sys.version[0]) < 3: try: input = raw_input range = xrange except NameError as err: pass Since it's the same error in both cases, and I only pass in either case, so the logic looks the same to me? Or am I missing something? --- Jim Wash your fruits and veggies, folks. Contrary to the popular delusion of yuppies, the giant farm corporations that hire fruit pickers do not provide outhouses, washing facilities, or the time to use them, anyway. The apple you bit into may be one step from a bumwipe. And no, the markets don't wash things either. They only water-spray them now and then to make them look fresher. -------------- next part -------------- An HTML attachment was scrubbed... URL: From davea at davea.name Mon Jul 15 01:00:57 2013 From: davea at davea.name (Dave Angel) Date: Sun, 14 Jul 2013 19:00:57 -0400 Subject: [Tutor] A slight bug in IDLE In-Reply-To: References: <51E127E4.6000109@pearwood.info> <51E214E9.20708@pearwood.info> Message-ID: On 07/14/2013 03:28 PM, Jim Mooney wrote: > On 13 July 2013 22:25, Dave Angel wrote: > > try: >> input = raw_input >> except NameError as e: >> pass >> >> try: >> range = xrange >> except NameError as e: >> pass >> >> ====== > Couldn't I just shorten that to: > > if int(sys.version[0]) < 3: > try: > input = raw_input > range = xrange > except NameError as err: > pass > > Since it's the same error in both cases, and I only pass in either case, so > the logic looks the same to me? Or am I missing something? > With enough research, you can justify any of those three approaches. But there's a principle here. Instead of explicitly coding when a particular change happened, and hardwiring the version number, one should test the actual property whenever possible. I'm doing that check with the try/catch. And by having two separate try/catch phrases, we won't be making any assumptions about whether the two changes happened at the same moment in revision history. It's analogous to duck-typing. Instead of thinking you know which types are acceptable, just check whether the methods you need are all available. If they are, then the type is probably just fine. And in this case, we don't usually need to check it all up front, but can just use the methods as needed. -- DaveA From cybervigilante at gmail.com Mon Jul 15 01:18:30 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sun, 14 Jul 2013 16:18:30 -0700 Subject: [Tutor] A slight bug in IDLE In-Reply-To: <51E214E9.20708@pearwood.info> References: <51E127E4.6000109@pearwood.info> <51E214E9.20708@pearwood.info> Message-ID: As of Python 3.3, the full list of __future__ imports are: > py> import __future__ > py> __future__.all_feature_names > ['nested_scopes', 'generators', 'division', 'absolute_import', > 'with_statement', 'print_function', 'unicode_literals', 'barry_as_FLUFL'] > I prefer import antigravity , but then I lose the rest of the day ;') -- Jim Cleaning Windows 7: Start Menu > Contol Panel > Programs and Features > Java > Uninstall It's that easy ;') -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc.tompkins at gmail.com Mon Jul 15 03:03:15 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Sun, 14 Jul 2013 18:03:15 -0700 Subject: [Tutor] A slight bug in IDLE In-Reply-To: References: <51E127E4.6000109@pearwood.info> <51E214E9.20708@pearwood.info> Message-ID: On Sun, Jul 14, 2013 at 4:18 PM, Jim Mooney wrote: > I prefer import antigravity , but then I lose the rest of the day ;') > Pretty sure that comes from sampling everything in the medicine cabinet for comparison... -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Mon Jul 15 04:38:00 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sun, 14 Jul 2013 19:38:00 -0700 Subject: [Tutor] A slight bug in IDLE In-Reply-To: References: <51E127E4.6000109@pearwood.info> <51E214E9.20708@pearwood.info> Message-ID: On 14 July 2013 18:03, Marc Tompkins wrote: > On Sun, Jul 14, 2013 at 4:18 PM, Jim Mooney wrote: > > >> I prefer import antigravity , but then I lose the rest of the day ;') >> > > Pretty sure that comes from sampling everything in the medicine cabinet > for comparison... > I am seriously considering the subway tour of North America: http://xkcd.com/1196/large/ -- Jim Cleaning Windows 7: Start Menu > Contol Panel > Programs and Features > Java > Uninstall It's that easy ;') -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc.tompkins at gmail.com Mon Jul 15 09:46:30 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Mon, 15 Jul 2013 00:46:30 -0700 Subject: [Tutor] A slight bug in IDLE In-Reply-To: References: <51E127E4.6000109@pearwood.info> <51E214E9.20708@pearwood.info> Message-ID: On Sun, Jul 14, 2013 at 7:38 PM, Jim Mooney wrote: > I am seriously considering the subway tour of North America: > http://xkcd.com/1196/large/ > I rather like the Anagram Tube Map: http://www.anagramtubemap.pwp.blueyonder.co.uk/ -------------- next part -------------- An HTML attachment was scrubbed... URL: From fomcl at yahoo.com Mon Jul 15 11:55:31 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Mon, 15 Jul 2013 02:55:31 -0700 (PDT) Subject: [Tutor] IOError when importing nose In-Reply-To: References: <1373724856.8043.YahooMailNeo@web163805.mail.gq1.yahoo.com> <1373733221.99460.YahooMailNeo@web163801.mail.gq1.yahoo.com> Message-ID: <1373882131.82725.YahooMailNeo@web163802.mail.gq1.yahoo.com> ----- Original Message ----- > From: eryksun > To: Albert-Jan Roskam > Cc: Python Mailing List > Sent: Sunday, July 14, 2013 4:18 AM > Subject: Re: [Tutor] IOError when importing nose > > On Sat, Jul 13, 2013 at 12:33 PM, Albert-Jan Roskam > wrote: >> >> Fantastic, that worked! Thank you! I was hesitant to use chmod because I >> hadn't (at least not that I was aware of) been messing with the file >> permissions. Any idea what this may have caused? > > How did you install pypi_classifiers? I pip installed it in Debian > (Jessie) without a hitch, so the problem isn't specific to the > pypi_classifiers setup script or the version of pip that I used > (1.3.1). I believe I downloaded the .tar.gz, untarred it and then ran python setup.py install. The .tar.gz is still in my downloads folder, but it is too long ago to check my terminal history what exactly I did (it only remembers the 500 most recent items). So might this mean this is a slight bug in the setup.py script of pypi_classifiers? If so, I could contact the author so he could improve this handy little program. From eryksun at gmail.com Mon Jul 15 13:05:27 2013 From: eryksun at gmail.com (eryksun) Date: Mon, 15 Jul 2013 07:05:27 -0400 Subject: [Tutor] IOError when importing nose In-Reply-To: <1373882131.82725.YahooMailNeo@web163802.mail.gq1.yahoo.com> References: <1373724856.8043.YahooMailNeo@web163805.mail.gq1.yahoo.com> <1373733221.99460.YahooMailNeo@web163801.mail.gq1.yahoo.com> <1373882131.82725.YahooMailNeo@web163802.mail.gq1.yahoo.com> Message-ID: On Mon, Jul 15, 2013 at 5:55 AM, Albert-Jan Roskam wrote: > > I believe I downloaded the .tar.gz, untarred it and then ran python > setup.py install. The .tar.gz is still in my downloads folder, but it > is too long ago to check my terminal history what exactly I did (it > only remembers the 500 most recent items). So might this mean this > is a slight bug in the setup.py script of pypi_classifiers? If so, I > could contact the author so he could improve this handy little program. The files in the tar have no permissions set for group and other. It appears pip silently fixed this, while directly using setup() doesn't. After reinstalling manually, I got the same IOError as you did when trying to import setuptools. From fomcl at yahoo.com Mon Jul 15 13:08:24 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Mon, 15 Jul 2013 04:08:24 -0700 (PDT) Subject: [Tutor] IOError when importing nose In-Reply-To: References: <1373724856.8043.YahooMailNeo@web163805.mail.gq1.yahoo.com> Message-ID: <1373886504.54097.YahooMailNeo@web163805.mail.gq1.yahoo.com> _______________________________ > From: Japhy Bartlett >To: Albert-Jan Roskam >Sent: Saturday, July 13, 2013 7:23 PM >Subject: Re: [Tutor] IOError when importing nose > > >The key line is: > > >>IOError: [Errno 13] Permission denied:? >> >which is pretty common, and usually means your python script doesn't have file permissions for whatever.? > > >I have to disagree with Walter about using system packages instead of pip, though his advice is spot on for fixing this issue and he obviously knows what he's doing. ?Very much my experience that pip is overwhelmingly more popular and reliable.? > > >Something something virtualenvs. Hi, (you sent me a private mail but I am ccing the list now) I was also somewhat surprised by that remark. Pip is better (newer)? than easy_install, and I always assumed that pip was at least as good as e.g. apt-get. But that was just an assumption of mine, based on nothing really. Albert-Jan From wprins at gmail.com Mon Jul 15 17:41:29 2013 From: wprins at gmail.com (Walter Prins) Date: Mon, 15 Jul 2013 16:41:29 +0100 Subject: [Tutor] IOError when importing nose In-Reply-To: <1373886504.54097.YahooMailNeo@web163805.mail.gq1.yahoo.com> References: <1373724856.8043.YahooMailNeo@web163805.mail.gq1.yahoo.com> <1373886504.54097.YahooMailNeo@web163805.mail.gq1.yahoo.com> Message-ID: Hi, On 15 July 2013 12:08, Albert-Jan Roskam wrote: > > From: Japhy Bartlett > >I have to disagree with Walter about using system packages instead of > pip, though his advice is spot on for fixing this issue and he obviously > knows what he's doing. Very much my experience that pip is overwhelmingly > more popular and reliable. > > > >Something something virtualenvs. > > I was also somewhat surprised by that remark. Pip is better (newer) than > easy_install, and I always assumed that pip was at least as good as e.g. > apt-get. But that was just an assumption of mine, based on nothing really. > Well I did say "slightly", so YMMV. But, to elaborate and perhaps belabor the point: If a module is being installed system wide (as it was in this case), then I lean towards preferring wherever possible the OS package manager, so as to help maintain system integrity and avoid potentially breaking for example future automatic upgrades via the system package management system. This helps ensure that the package manager will continue to take care of actually updating all system wide packages on system including Python modules (think security patches and "rest of OS compatibility"), as well as to ensure that the system wide versions installed remains suitable/tested/compatible with other software installed on the operating system by the package manager or used by the OS, some of which may be relying on *particular* versions of the modules/packages. Imagine for example the kinds of trouble that might happen if you, say, install version 1.3 of some imaginary Python module "abc" via pip, and then later you innocently (try to) install what to you is an unrelated package manager software package "XYZ" which happens to also depend on version *1.1* of the Python module "abc"... Probably, the package manager will abort due to finding files already present, or it may just blindly overwrite your pip installed version. Neither option is obviously really a desirable outcome. Or imagine that your system happens to already have Python module "def" version 1.5 installed via package manager as a dependency of, say, some of the OS utilities on your system, and that then you go and upgrade or force-install say version 2.0 of module "def" system wide via pip, which then happens to change a bunch of class definitions/interfaces in the module, thereby breaking all the OS utilities that depend on this module. That would also probably be a painful mess to sort out... In short: If one makes changes/additions/deletions to system files outside of the package management system you in principle always run the risk of accidentally breaking or causing issues either immediately or down the line with future automatic package upgrades, as well as potentially causing problems if in future you try to install other software via the package management system that in turn depends on and then tries to automatically pull in the package manager version of the module that already installed "manually" via pip, etc. That's more or less the rationale. I should probably add for completeness sake (in lieu of the "Something something virtualenvs" comment), that if you're using virtualenv or any type of local/user based python setup/install then pip is obviously your only option. ;) ) Walter -------------- next part -------------- An HTML attachment was scrubbed... URL: From iafleischer at gmail.com Mon Jul 15 20:53:53 2013 From: iafleischer at gmail.com (I. Alejandro Fleischer) Date: Mon, 15 Jul 2013 14:53:53 -0400 Subject: [Tutor] How to install package (I. Alejandro Fleischer) Message-ID: Hi, Im trying to install ipython for using with my already installed python 2.7, on windows xp 32 bit. It says I have to have a package named " distribute". I' ve downladed it. Dont know how to implement the package, wich seems not to be an executable archive. any help please? -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Mon Jul 15 22:40:48 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 15 Jul 2013 13:40:48 -0700 Subject: [Tutor] How to install package (I. Alejandro Fleischer) In-Reply-To: References: Message-ID: I think the easiest thing is to install the Enthought Canopy distro, free - It has Numpy , Scipy , IPython, Matplotlib , and just about every other common dependency you'll need in the future: And it's nice they're all there. I recently installed something that, only afterward the install failed, told me it needed wxPython. I checked their web page - no mention at all of the dependency. That's just not nice ;') https://www.enthought.com/products/epd/free/ This is for Py 2.7 Jim Enthought Python Distribution Free Enthought Python Distribution (EPD) Free has been reloaded, as Canopy Express . Like EPD Free, Canopy Express is a lightweight distribution of Python essentials for scientific and analytic computing. It includes Python packages Numpy, Scipy , IPython , Matplotlib, Pandas , Traits, Enaml , and more. But Canopy Express also comes with a new graphical analysis environment built on top of Enthought's proven Python distribution. On 15 July 2013 11:53, I. Alejandro Fleischer wrote: > > Hi, > Im trying to install ipython for using with my already installed python > 2.7, on windows xp 32 bit. > > It says I have to have a package named " distribute". I' ve downladed it. > > Dont know how to implement the package, wich seems not to be an executable > archive. > > any help please? > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > > -- Jim If you find yourself running into a wall, stop running into a wall! --Mark Lutz If you can't go through a wall, go around it. --Jim Mooney -------------- next part -------------- An HTML attachment was scrubbed... URL: From fomcl at yahoo.com Mon Jul 15 23:04:10 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Mon, 15 Jul 2013 14:04:10 -0700 (PDT) Subject: [Tutor] How to install package (I. Alejandro Fleischer) In-Reply-To: References: Message-ID: <1373922250.40775.YahooMailNeo@web163801.mail.gq1.yahoo.com> >________________________________ > From: Jim Mooney >To: I. Alejandro Fleischer >Cc: tutor at python.org >Sent: Monday, July 15, 2013 10:40 PM >Subject: Re: [Tutor] How to install package (I. Alejandro Fleischer) > > > >I think the easiest thing is to install the Enthought Canopy distro, free - It has Numpy, Scipy, IPython, Matplotlib, and just about every other common dependency you'll need in the future: And it's nice they're all there. I recently installed something that, only afterward the install failed, told me it needed wxPython. I checked their web page - no mention at all of the dependency. That's just not nice ;') > >https://www.enthought.com/products/epd/free/ > Perhaps even better: Python(x, y): https://code.google.com/p/pythonxy/wiki/Downloads From oscar.j.benjamin at gmail.com Mon Jul 15 23:25:45 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 15 Jul 2013 22:25:45 +0100 Subject: [Tutor] How to install package (I. Alejandro Fleischer) In-Reply-To: References: Message-ID: On 15 July 2013 19:53, I. Alejandro Fleischer wrote: > > Hi, > Im trying to install ipython for using with my already installed python 2.7, > on windows xp 32 bit. > > It says I have to have a package named " distribute". I' ve downladed it. > > Dont know how to implement the package, wich seems not to be an executable > archive. I think that you should actually install setuptools instead of distribute (the situation with these two packages is a little confusing at the moment). You can do that with the ez_setup.py script here: https://bitbucket.org/pypa/setuptools/downloads/ez_setup.py If you already know how to run a Python script then just download and run that script and it should download and install setuptools for you. Then you should be able to install ipython. Oscar From ramit.prasad at jpmorgan.com Mon Jul 15 23:06:49 2013 From: ramit.prasad at jpmorgan.com (Prasad, Ramit) Date: Mon, 15 Jul 2013 21:06:49 +0000 Subject: [Tutor] How to install package (I. Alejandro Fleischer) In-Reply-To: References: Message-ID: <5B80DD153D7D744689F57F4FB69AF4741858077D@SCACMX008.exchad.jpmchase.net> I. Alejandro Fleischer wrote: > > Hi, > Im trying to install ipython for using with my already installed python 2.7, on windows xp 32 bit. > > It says I have to have a package named " distribute". I' ve downladed it. > > Dont know how to implement the package, wich seems not to be an executable archive. > > any help please? You can get the installer for iPython here https://pypi.python.org/pypi/ipython. That may not require you to install distribute. If you need to install distribute I think the graphic at the bottom of the page http://pythonhosted.org/distribute/ explains how to install pip and distribute. ~Ramit This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email. From bfishbein79 at gmail.com Mon Jul 15 23:38:14 2013 From: bfishbein79 at gmail.com (Ben Fishbein) Date: Mon, 15 Jul 2013 16:38:14 -0500 Subject: [Tutor] Best way to learn Tkinter Message-ID: <9D8E4E9E-78ED-4D71-B432-378D0412C19C@gmail.com> Hello. I wrote a python program for my small business. It works fine but it is all from command prompt and makes sense to no one but me. I want to create a new version with a GUI that an employee could use. I'm using python 2.7 (though there's nothing in the program that won't work on 2.5) I have been looking through tkinter tutorials online andaop Sent from my iPhone From cybervigilante at gmail.com Tue Jul 16 00:08:13 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 15 Jul 2013 15:08:13 -0700 Subject: [Tutor] How to install package (I. Alejandro Fleischer) In-Reply-To: <1373922250.40775.YahooMailNeo@web163801.mail.gq1.yahoo.com> References: <1373922250.40775.YahooMailNeo@web163801.mail.gq1.yahoo.com> Message-ID: On 15 July 2013 14:04, Albert-Jan Roskam wrote: Perhaps even better: Python(x, y): > https://code.google.com/p/pythonxy/wiki/Downloads > Good point. You're not tied to a commercial distro. And in case it isn't noticed you should uninstall your current python first (saving your standard program file, of course - I've put mine in a different directory since I've wiped out python a few times now ;') (x,y) includes pip and its dependencies, so then you can just do "pip install ..." to install most other things, afterwards (sometimes - it doesn't always find what you want on the net so you have to download it.) A good feature of pip is pip freeze, which shows you what you've already got installed. Figuring out the Python install mess is harder than figuring out Python, IMHO ;') -- Jim If you find yourself running into a wall, stop running into a wall! --Mark Lutz If you can't go through a wall, go around it. --Jim Mooney -------------- next part -------------- An HTML attachment was scrubbed... URL: From oscar.j.benjamin at gmail.com Tue Jul 16 00:33:23 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 15 Jul 2013 23:33:23 +0100 Subject: [Tutor] How to install package (I. Alejandro Fleischer) In-Reply-To: References: <1373922250.40775.YahooMailNeo@web163801.mail.gq1.yahoo.com> Message-ID: On 15 July 2013 23:08, Jim Mooney wrote: > > Figuring out the Python install mess is harder than figuring out Python, > IMHO ;') It's true that Python package installation is harder than it could be. There's a lot of progress being made on this front at the moment though. There are some deep problems in the way that Python has been doing it so far. The people who are working on this right now are spending most of their time laying the basic foundations for a better packaging system and so their work is yet to really manifest itself in visible improvements. The situation should be noticeably better by the time Python 3.4 gets released though. Oscar From cybervigilante at gmail.com Tue Jul 16 00:34:02 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 15 Jul 2013 15:34:02 -0700 Subject: [Tutor] Best way to learn Tkinter In-Reply-To: <9D8E4E9E-78ED-4D71-B432-378D0412C19C@gmail.com> References: <9D8E4E9E-78ED-4D71-B432-378D0412C19C@gmail.com> Message-ID: Ben Fishbein Here's a good source - and has a link to a recent book (ten bucks on kindle) http://www.tkdocs.com/ Jim On 15 July 2013 14:38, Ben Fishbein wrote: > Hello. I wrote a python program for my small business. It works fine but > it is all from command prompt and makes sense to no one but me. I want to > create a new version with a GUI that an employee could use. I'm using > python 2.7 (though there's nothing in the program that won't work on 2.5) > I have been looking through tkinter tutorials online andaop > > Sent from my iPhone > -------------- next part -------------- An HTML attachment was scrubbed... URL: From iafleischer at gmail.com Tue Jul 16 00:37:37 2013 From: iafleischer at gmail.com (I. Alejandro Fleischer) Date: Mon, 15 Jul 2013 18:37:37 -0400 Subject: [Tutor] How to install package (I. Alejandro Fleischer) In-Reply-To: References: <1373922250.40775.YahooMailNeo@web163801.mail.gq1.yahoo.com> Message-ID: Don't know if it's the protocol here, but I thank you very very much. I already have an idea for solving this issue. I'll let you know. Igor El 15/07/2013 18:33, "Oscar Benjamin" escribi?: > On 15 July 2013 23:08, Jim Mooney wrote: > > > > Figuring out the Python install mess is harder than figuring out Python, > > IMHO ;') > > It's true that Python package installation is harder than it could be. > There's a lot of progress being made on this front at the moment > though. There are some deep problems in the way that Python has been > doing it so far. The people who are working on this right now are > spending most of their time laying the basic foundations for a better > packaging system and so their work is yet to really manifest itself in > visible improvements. The situation should be noticeably better by the > time Python 3.4 gets released though. > > > Oscar > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan.gauld at btinternet.com Tue Jul 16 01:11:14 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Tue, 16 Jul 2013 00:11:14 +0100 Subject: [Tutor] Best way to learn Tkinter In-Reply-To: <9D8E4E9E-78ED-4D71-B432-378D0412C19C@gmail.com> References: <9D8E4E9E-78ED-4D71-B432-378D0412C19C@gmail.com> Message-ID: On 15/07/13 22:38, Ben Fishbein wrote: > Hello. I wrote a python program for my small business. >It works fine but it is all from command prompt and makes sense > to no one but me. I want to create a new version with a GUI > that an employee could use. I still like the Pythonware one: http://effbot.org/tkinterbook/ But as an aside have you thought about making it a web app instead? That is much easier to maintain - no need to distribute updates etc. There are many fine Python web frameworks including Django and Pylons. If you can find a server to host it and your users are online it might be a better option. Just a thought. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From jalespring at gmail.com Tue Jul 16 01:16:18 2013 From: jalespring at gmail.com (Arnel Legaspi) Date: Tue, 16 Jul 2013 07:16:18 +0800 Subject: [Tutor] Writing unit tests that involve email In-Reply-To: <51E1F1B5.3060302@gmail.com> References: <51E14B6B.8060609@zoho.com> <51E1F1B5.3060302@gmail.com> Message-ID: On Sun, Jul 14, 2013 at 8:32 AM, Arnel Legaspi wrote: > Hello, > > I have written a small Python script that allows the user to post to > their blogs via email, converting the draft written in Markdown & > sending it to a blog's email address (like what they had in Posterous > back then). > > https://bitbucket.org/acl79/**downpost > > At this point, I've been trying to write unit tests for it to get some > practice writing unit tests for other possibly larger projects in the > future. I've been using the dsmtpd script available at: > > https://github.com/matrixise/**dsmtpd > > to run a SMTP server in the background and route all the emails being > sent by my script to localhost. The script allows saving the messages to > an optional Maildir directory, and I wanted to use that to be able to > check various fields in the email for correctness in the unit tests. > > What I'm having difficulty is getting the unit tests to properly run the > script being tested and send the email, get the proper Maildir message, > and be able to read it via the unit test script. Right now, even when > the SMTP server is not running, all the tests for checking the sent > email pass, which is not what I intended. > > I know the SMTP server works, because if I send some email using another > Python or Powershell script, the script logs on the screen that it did > receive the email. > > How should I approach this? > > Here is the unit test script I have (some lines may get wrapped): > > #!/usr/bin/env python > > import os > import sys > import argparse > import ConfigParser > import unittest > import mailbox > > MKDOWN_TEXT = """ > **Lorem ipsum dolor sit amet**, consectetur adipiscing elit. Sed > dapibus dui nibh, ut fermentum dolor tristique fringilla. Pellentesque > tristique sagittis dapibus. Nunc pellentesque nisi vitae arcu lacinia, > nec ultricies libero rutrum. *Vivamus* enim augue, malesuada in dui ac, > laoreet faucibus dolor. > > * Aliquam eu nisi dictum, tristique eros ut, eleifend elit. > * Pellentesque lacinia sodales erat non rhoncus. > * Aliquam mattis ullamcorper purus sit amet sodales. > > Ut mollis ligula id sapien vehicula fringilla eu auctor diam. > Praesent consequat tristique arcu, nec iaculis metus porta at. > Nam mauris orci, congue vel consectetur vitae, viverra adipiscing > urna. > > > Donec tristique leo erat, sit amet feugiat velit semper sit amet. > > Nunc sagittis libero risus, at mollis mauris pharetra quis. Vivamus > > adipiscing porttitor velit, sit amet sodales risus mollis quis. > > Donec sed risus orci. Cras fringilla at lorem ut venenatis. > Pellentesque varius lorem neque, euplace rat eros dictum nec. > """ > > HTML_TXT = """ >

Lorem ipsum dolor sit amet, consectetur adipiscing > elit. Sed > dapibus dui nibh, ut fermentum dolor tristique fringilla. Pellentesque > tristique sagittis dapibus. Nunc pellentesque nisi vitae arcu lacinia, > nec ultricies libero rutrum. Vivamus enim augue, malesuada in > dui ac, > laoreet faucibus dolor.

>
    >
  • Aliquam eu nisi dictum, tristique eros ut, eleifend elit.
    • >
  • Pellentesque lacinia sodales erat non rhoncus.
    • >
  • Aliquam mattis ullamcorper purus sit amet sodales.
    • >
>

Ut mollis ligula id sapien vehicula fringilla eu auctor diam. > Praesent consequat tristique arcu, nec iaculis metus porta at. > Nam mauris orci, congue vel consectetur vitae, viverra adipiscing > urna.

>
>

Donec tristique leo erat, sit amet feugiat velit semper sit amet. > Nunc sagittis libero risus, at mollis mauris pharetra quis. Vivamus > adipiscing porttitor velit, sit amet sodales risus mollis quis.

>
>

Donec sed risus orci. Cras fringilla at lorem ut venenatis. > Pellentesque varius lorem neque, euplace rat eros dictum nec.

> """ > > TESTARGS = [] > > > def clear_args(): > """ > Clear the arguments & the internal list (TESTARGS) used > for test arguments. This is needed to allow the script to > be tested with different arguments. > """ > del sys.argv[1:] > TESTARGS[:] = [] > return > > > def add_testfiles(contentfile='pos**t.md ', > configfile='blog.conf'): > """ > Clear the arguments & the internal list used. Add the > regular test files into the argument list instead. > """ > clear_args() > TESTARGS.append(contentfile) > TESTARGS.append(configfile) > TESTARGS.append('--debug') > sys.argv.append(TESTARGS) > return > > > def create_configfile(): > """ > Create a dummy config file for testing purposes. > """ > parser = ConfigParser.SafeConfigParser(**allow_no_value=True) > parser.add_section('blog') > parser.set('blog', 'email', 'test at email.net') > parser.add_section('smtp') > parser.set('smtp', 'server', '127.0.0.1') > parser.set('smtp', 'port', '8000') > parser.set('smtp', 'username', 'user at wherever.com') > parser.set('smtp', 'password', 'DeJpkZa3WL') > > with open('blog.conf', 'wb') as configfile: > parser.write(configfile) > > > class TestReadingSendingPostFile(**unittest.TestCase): > > def setUp(self): > """ Prepare a post file for testing. """ > create_configfile() > with open('post.md', 'wb') as postfile: > postfile.write(MKDOWN_TEXT) > self.mbox = mailbox.Maildir('mail_dir') > > def test_wrongconfigfile(self): > clear_args() > add_testfiles('post.md', 'bloginfo.ini') > self.assertRaises(IOError) > > def test_wrongpostfile(self): > clear_args() > add_testfiles('some_**nonexistent_file.md', > 'blog.conf') > self.assertRaises(IOError) > > def test_sendpost(self): > clear_args() > add_testfiles() > for message in self.mbox: > self.assertEqual(message['**subject'], 'post') > self.assertEqual(message.get_**payload(), HTML_TXT) > > def test_sendrawpost(self): > clear_args() > TESTARGS.append('--raw') > add_testfiles() > for message in self.mbox: > self.assertEqual(message['**subject'], 'post') > self.assertEqual(message.get_**payload(), MKDOWN_TEXT) > > def test_sendpost_withunderscores(**self): > clear_args() > os.rename('post.md', > 'This_Should_Be_Posted_**Without_Underscores.md') > add_testfiles('This_Should_Be_**Posted_Without_Underscores.md'**, > 'blog.conf') > for message in self.mbox: > self.assertEqual(message['**subject'], > 'This Should Be Posted Without Underscores') > self.assertEqual(message.get_**payload(), HTML_TXT) > > def test_sendpost_withspaces(self)**: > clear_args() > os.rename('post.md', 'This Should Be Posted As Is.md') > add_testfiles('This Should Be Posted As Is.md', 'blog.conf') > for message in self.mbox: > self.assertEqual(message['**subject'], > 'This Should Be Posted As Is') > self.assertEqual(message.get_**payload(), HTML_TXT) > > def test_sendpost_withsettitle(**self): > clear_args() > TESTARGS.append('-t "This will be the new title"') > add_testfiles() > for message in self.mbox: > self.assertEqual(message['**subject'], > 'This will be the new title') > self.assertEqual(message.get_**payload(), HTML_TXT) > > def tearDown(self): > if os.path.exists('post.md'): > os.remove('post.md') > if os.path.exists('This_Should_**Be_Posted_Without_Underscores.** > md'): > os.remove('This_Should_Be_**Posted_Without_Underscores.md'**) > if os.path.exists('This Should Be Posted As Is.md'): > os.remove('This Should Be Posted As Is.md') > > os.remove('blog.conf') > > > if __name__ == '__main__': > argp = argparse.ArgumentParser() > argp.add_argument('unittest_**args', nargs='*') > args = argp.parse_args() > unit_args = [sys.argv[0]] + args.unittest_args > unittest.main(argv=unit_args) > > > Any comments on this & the script I'm writing tests for would be > appreciated. > Hello, Is there any other information needed here so I can get help re: my unit tests? Perhaps I did something wrong with my post? I'm using Python 2.7.5 on Windows 7 x64, but the script I'm writing tests for has branches for both Python 2 and 3. I'm pretty sure I've made a number of mistakes here, as I have not written any unit tests before this. Thank you, Arnel -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Tue Jul 16 03:14:08 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 16 Jul 2013 11:14:08 +1000 Subject: [Tutor] Writing unit tests that involve email In-Reply-To: References: <51E14B6B.8060609@zoho.com> <51E1F1B5.3060302@gmail.com> Message-ID: <51E49E60.5060403@pearwood.info> On 16/07/13 09:16, Arnel Legaspi wrote: > Hello, > > Is there any other information needed here so I can get help re: my unit > tests? > Perhaps I did something wrong with my post? Hi Arnel, Sorry, your post was big and had a lot of detail, and I put it aside to read later, but real life has intervened and I have not been able to give it the time it requires. If nobody else gives a satisfactory answer, I will try to reply by this time tomorrow. -- Steven From wolfrage8765 at gmail.com Tue Jul 16 05:21:53 2013 From: wolfrage8765 at gmail.com (Jordan) Date: Mon, 15 Jul 2013 23:21:53 -0400 Subject: [Tutor] Bug? multiprocessing.JoinableQueue Message-ID: <51E4BC51.2080200@gmail.com> Hello Tutors. I think I found a bug in multiprocessing.JoinableQueue(). My OS is Linux Mint 15 64-bit Python 3.3.1 (default, Apr 17 2013, 22:30:32) [GCC 4.7.3] on linux Expected Results are achieved with queue.Queue() Like So: >>> import queue >>> test = queue.Queue >>> test = queue.Queue() >>> test.get(block=True, timeout=1) Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3.3/queue.py", line 175, in get raise Empty queue.Empty The interactive session paused/waited for about 1 second. The Bug or Unexpected Results with multiprocessing.JoinableQueue() >>> import multiprocessing >>> test = multiprocessing.JoinableQueue() >>> test.get(block=True, timeout=1) Traceback (most recent call last): File "", line 1, in File "/usr/lib/python3.3/multiprocessing/queues.py", line 109, in get raise Empty queue.Empty The interactive session returns immediately, no pause or wait. If I use timeout=1.01 then the interactive session pauses as expected, or any number other than 1 for the matter. Can others please confirm? If it is confirmed can you provide guidance about how to properly report the bug? Thank you. -- Jordan From eryksun at gmail.com Tue Jul 16 09:35:19 2013 From: eryksun at gmail.com (eryksun) Date: Tue, 16 Jul 2013 03:35:19 -0400 Subject: [Tutor] Bug? multiprocessing.JoinableQueue In-Reply-To: <51E4BC51.2080200@gmail.com> References: <51E4BC51.2080200@gmail.com> Message-ID: On Mon, Jul 15, 2013 at 11:21 PM, Jordan wrote: > > The interactive session returns immediately, no pause or wait. > If I use timeout=1.01 then the interactive session pauses as expected, or > any number other than 1 for the matter. The bug you found is fixed in 3.3.2. Depending on the speed of your machine, even a timeout slightly larger than 1 (e.g. 1.000001) would trigger the rounding problem. See issue 17707: http://bugs.python.org/issue17707 The poll timeout in milliseconds was being calculated incorrectly: # 3.3.1 timeout = int(timeout) * 1000 # timeout is in milliseconds http://hg.python.org/cpython/file/d9893d13c628/Lib/multiprocessing/connection.py#l865 # 3.3.2 timeout = int(timeout * 1000) # timeout is in milliseconds http://hg.python.org/cpython/file/d047928ae3f6/Lib/multiprocessing/connection.py#l865 From wolfrage8765 at gmail.com Tue Jul 16 12:24:58 2013 From: wolfrage8765 at gmail.com (Jordan) Date: Tue, 16 Jul 2013 06:24:58 -0400 Subject: [Tutor] Bug? multiprocessing.JoinableQueue In-Reply-To: References: <51E4BC51.2080200@gmail.com> Message-ID: <51E51F7A.8040901@gmail.com> On 07/16/2013 03:35 AM, eryksun wrote: > On Mon, Jul 15, 2013 at 11:21 PM, Jordan wrote: >> The interactive session returns immediately, no pause or wait. >> If I use timeout=1.01 then the interactive session pauses as expected, or >> any number other than 1 for the matter. > The bug you found is fixed in 3.3.2. Depending on the speed of your > machine, even a timeout slightly larger than 1 (e.g. 1.000001) would > trigger the rounding problem. Thanks, guess I should had Googled harder, but I had not found it when I looked. Also really good to know it was not just me or something I did. > > See issue 17707: > http://bugs.python.org/issue17707 > > The poll timeout in milliseconds was being calculated incorrectly: > > # 3.3.1 > timeout = int(timeout) * 1000 # timeout is in milliseconds > > http://hg.python.org/cpython/file/d9893d13c628/Lib/multiprocessing/connection.py#l865 > > # 3.3.2 > timeout = int(timeout * 1000) # timeout is in milliseconds > > http://hg.python.org/cpython/file/d047928ae3f6/Lib/multiprocessing/connection.py#l865 From karanmatic at gmail.com Tue Jul 16 19:09:21 2013 From: karanmatic at gmail.com (Karan Goel) Date: Tue, 16 Jul 2013 22:39:21 +0530 Subject: [Tutor] A list of 100+ projects to complete to master Python. Message-ID: Hey guys and gals Just a plug here. My repo: https://github.com/thekarangoel/Projects was one of the trending repos on Gh this week and I thought folks on this list might be interested in knowing about it. In a nutshell, I'm trying to complete over a 100 practical language- agnostic projects in Python only. I haven't read the project details yet, and I'm not filtering things out. If you would like to do the same, join me. Fork or star the repo, and start coding (in any language, really). https://github.com/thekarangoel/Projects Let me know if you have any questions or suggestions. - Karan Goel Goel.im | Resume | Github -------------- next part -------------- An HTML attachment was scrubbed... URL: From jalespring at gmail.com Wed Jul 17 03:14:49 2013 From: jalespring at gmail.com (Arnel Legaspi) Date: Wed, 17 Jul 2013 09:14:49 +0800 Subject: [Tutor] Writing unit tests that involve email In-Reply-To: References: Message-ID: <51E5F009.3060106@gmail.com> On 7/16/2013 6:00 PM, tutor-request at python.org wrote: > Message: 1 > Date: Tue, 16 Jul 2013 11:14:08 +1000 > From: Steven D'Aprano > To: tutor at python.org > Subject: Re: [Tutor] Writing unit tests that involve email > Message-ID: <51E49E60.5060403 at pearwood.info> > Content-Type: text/plain; charset=UTF-8; format=flowed > > On 16/07/13 09:16, Arnel Legaspi wrote: >> Hello, >> >> Is there any other information needed here so I can get help re: my unit >> tests? >> Perhaps I did something wrong with my post? > > Hi Arnel, > > > Sorry, your post was big and had a lot of detail, and I put it aside to read later, but real life has intervened and I have not been able to give it the time it requires. > > If nobody else gives a satisfactory answer, I will try to reply by this time tomorrow. Hi Steven, Thanks for the response. I apologize if I sounded demanding. I did not realize my post had that much detail to sift through - I've been reading (and I still am) most of the replies to requests last month, and they're quite speedy. I thought I may have done something wrong. I'm looking forward to your response, and others as well. Thanks, Arnel From cybervigilante at gmail.com Wed Jul 17 03:48:51 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Tue, 16 Jul 2013 18:48:51 -0700 Subject: [Tutor] Deleted dictionary length not reporting Message-ID: This is puzzling me. I have a folder with about 125 movie sound clips, like "I'll be back," blues brothers theme, oompa-loompas, etc. I play seven random ones on windows startup, which works fine. But just so I don't repeat a clip, I delete it from the dictionary on each go-round. However, to make sure that happened, I printed the key-length on each go-round. It should be shorter by one every time but I always get a length of 125. (Yes, I know there are about five different ways to do this, but I thought I'd try a dictionary ;') Also, the printout doesn't occur until after all the sounds play, so maybe that has something to do with it. But I also tried saving the lengths in a list while the program ran, and got the same result. A list of all '125' #Using Python 2.7 on Win 7 import winsound import random, os random.seed() lenlist = [] file_list = os.listdir('WAV') for clip in range(0,7): file_dict = dict(enumerate(file_list)) sound_key = random.choice(range(0,len(file_list))) winsound.PlaySound('WAV/' + file_dict[sound_key], winsound.SND_FILENAME) del file_dict[sound_key] print(len(file_dict.keys())) ''' result 125 125 125 125 125 125 125 ''' -- Jim I sure love that Guv-a-Mint Cheese, Guv-a-Mint Cheese, Guv-a-Mint Cheese Oh please Guv-a-Mint, Guv-a-Mint please, Please bring back that Guv-a-Mint Cheeeeeeeeese! From cybervigilante at gmail.com Wed Jul 17 04:04:47 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Tue, 16 Jul 2013 19:04:47 -0700 Subject: [Tutor] ooops Message-ID: Forget the question on the random sound prog. I had enumerate in the loop when it should have been outside the loop. One dumb-smack for me ;') -- Jim I sure love that Guv-a-Mint Cheese, Guv-a-Mint Cheese, Guv-a-Mint Cheese Oh please Guv-a-Mint, Guv-a-Mint please, Please bring back that Guv-a-Mint Cheeeeeeeeese! From steve at pearwood.info Wed Jul 17 04:04:56 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 17 Jul 2013 12:04:56 +1000 Subject: [Tutor] Deleted dictionary length not reporting In-Reply-To: References: Message-ID: <51E5FBC8.7050801@pearwood.info> On 17/07/13 11:48, Jim Mooney wrote: > This is puzzling me. I have a folder with about 125 movie sound clips, > like "I'll be back," blues brothers theme, oompa-loompas, etc. I play > seven random ones on windows startup, which works fine. But just so I > don't repeat a clip, I delete it from the dictionary on each go-round. > However, to make sure that happened, I printed the key-length on each > go-round. It should be shorter by one every time but I always get a > length of 125. (Yes, I know there are about five different ways to do > this, but I thought I'd try a dictionary ;') That's because your loop recreates the dictionary at the start of each loop. I bet you have 126 WAV files in the directory. You play one, delete it from the dict, print the number of keys (which will be 126-1 = 125), and then start the loop again, which recreates the dict good as new. > Also, the printout doesn't occur until after all the sounds play, so > maybe that has something to do with it. But I also tried saving the > lengths in a list while the program ran, and got the same result. A > list of all '125' That's very strange. print is not supposed to be buffered, it should always print immediately. I have no ideas about that. > #Using Python 2.7 on Win 7 > > import winsound > import random, os > random.seed() > > lenlist = [] > file_list = os.listdir('WAV') > > for clip in range(0,7): > file_dict = dict(enumerate(file_list)) > sound_key = random.choice(range(0,len(file_list))) Here's a better way to do it, which guarantees that there will be no duplicates (unless you have duplicate files): file_list = os.listdir('WAV') random.shuffle(file_list) for name in file_list[:7]: winsound.PlaySound('WAV/' + name, winsound.SND_FILENAME) -- Steven From davea at davea.name Wed Jul 17 04:12:39 2013 From: davea at davea.name (Dave Angel) Date: Tue, 16 Jul 2013 22:12:39 -0400 Subject: [Tutor] Deleted dictionary length not reporting In-Reply-To: References: Message-ID: On 07/16/2013 09:48 PM, Jim Mooney wrote: > This is puzzling me. I have a folder with about 125 movie sound clips, > like "I'll be back," blues brothers theme, oompa-loompas, etc. I play > seven random ones on windows startup, which works fine. But just so I > don't repeat a clip, I delete it from the dictionary on each go-round. > However, to make sure that happened, I printed the key-length on each > go-round. It should be shorter by one every time but I always get a > length of 125. (Yes, I know there are about five different ways to do > this, but I thought I'd try a dictionary ;') > > Also, the printout doesn't occur until after all the sounds play, so > maybe that has something to do with it. But I also tried saving the > lengths in a list while the program ran, and got the same result. A > list of all '125' > > #Using Python 2.7 on Win 7 > > import winsound > import random, os > random.seed() > > lenlist = [] > file_list = os.listdir('WAV') > > for clip in range(0,7): > file_dict = dict(enumerate(file_list)) > sound_key = random.choice(range(0,len(file_list))) > winsound.PlaySound('WAV/' + file_dict[sound_key], winsound.SND_FILENAME) > del file_dict[sound_key] You're deleting something from a dictionary that you're going to recreate next time through the loop. If you really want to delete it, remove it from the file_list. Your dictionary is accomplishing precisely nothing anyway, since file_dict[key] will give you the same value as file_list[key]. But if you like the dictionary, then create it outside the loop. Now deleting things from it will actually mean something. Of course, then your random.choice() will have to be over the dictionary, not over the list. Or you'll have to handle the case where it happens to choose the same item again, with a loop inside the loop to handle those retries... > print(len(file_dict.keys())) Seems to me the obvious choice is just to use random.choice of the enumerated file list, then delete the item from the list. Way too much indirection in the code above. -- DaveA From steve at pearwood.info Wed Jul 17 04:43:34 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 17 Jul 2013 12:43:34 +1000 Subject: [Tutor] Writing unit tests that involve email In-Reply-To: <51E1F1B5.3060302@gmail.com> References: <51E14B6B.8060609@zoho.com> <51E1F1B5.3060302@gmail.com> Message-ID: <51E604D6.7040609@pearwood.info> On 14/07/13 10:32, Arnel Legaspi wrote: [...] > What I'm having difficulty is getting the unit tests to properly run the > script being tested and send the email, get the proper Maildir message, > and be able to read it via the unit test script. Right now, even when > the SMTP server is not running, all the tests for checking the sent > email pass, which is not what I intended. > > I know the SMTP server works, because if I send some email using another > Python or Powershell script, the script logs on the screen that it did > receive the email. > > How should I approach this? > > Here is the unit test script I have (some lines may get wrapped): Firstly, start with the obvious: are you sure that the unittests are being run? If I have counted correctly, you only have seven test cases at the moment, so your unittest output should show seven dots and then print something like: Ran 7 tests in 0.01s OK Can you confirm that this is the case? If not, then your problem is that the tests aren't running at all. [...] > parser.set('smtp', 'password', ***redacted***) I hope this isn't a real password to a real mail server visible from the internet, because you've now made it visible to everyone in the world. > class TestReadingSendingPostFile(unittest.TestCase): > > def setUp(self): > """ Prepare a post file for testing. """ > create_configfile() > with open('post.md', 'wb') as postfile: > postfile.write(MKDOWN_TEXT) > self.mbox = mailbox.Maildir('mail_dir') Personally, I don't use the setUp and testDown methods, but most of my code avoids storing state so they are rarely relevant to me. But in any case, the setUp method is called before *every* test method. If you want it to be called only once, you should put it inside the class __init__ method. (Don't forget to call the TestCase __init__ as well.) I see all of your test methods include clear_args() at the start, so you should put that inside the setUp method. It would help if you tell us which test method you are expecting to fail. My guess is that it is this one: > def test_sendpost(self): > clear_args() > add_testfiles() > for message in self.mbox: > self.assertEqual(message['subject'], 'post') > self.assertEqual(message.get_payload(), HTML_TXT) I don't believe this actually tries to send any email, and as far as I can see you never actually clear the mailbox, so all you're doing is looking at old emails that happened to exist. If the mbox happens to be empty, then neither of the assertEqual calls will happen, and the test will just pass. Add this to the test: if not self.mbox: self.fail("No messages were sent") or better still, add a separate test to check that email is sent, without caring about the content of the email. That way, if you get a failure, you can immediately see whether the failure is "email wasn't sent", or "email was sent, but contains the wrong stuff". I suggest you factor out any tests of sending mail into a separate class. Give this class a setUp() that creates a mailbox, and a tearDown that moves everything from that mailbox into another one. That means you have a nice clean mailbox at the start of each test, but you can still inspect the emails by eye if you want. It also means that you can separate tests with an external dependency (the mail server) from those without. -- Steven From cybervigilante at gmail.com Wed Jul 17 04:55:04 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Tue, 16 Jul 2013 19:55:04 -0700 Subject: [Tutor] Deleted dictionary length not reporting In-Reply-To: <51E5FBC8.7050801@pearwood.info> References: <51E5FBC8.7050801@pearwood.info> Message-ID: > That's very strange. print is not supposed to be buffered, it should always > print immediately. I have no ideas about that. Something winsound must do, since I printed out the time at the end of each loop and the loop delays for the length of each sound clip, which are a few seconds long. Tue Jul 16 19:48:01 2013 Tue Jul 16 19:48:12 2013 Tue Jul 16 19:48:16 2013 Tue Jul 16 19:48:22 2013 Tue Jul 16 19:48:25 2013 but the print that's in the loop doesn't appear until the loop is done, eve after I put the enumerate where it should go. anyway, it worked fine until I got a key error now and then, which I don't want to figure out, so I'm going back to just popping used items from a list, which was dirt-simple. Jim I sure love that Guv-a-Mint Cheese, Guv-a-Mint Cheese, Guv-a-Mint Cheese Oh please Guv-a-Mint, Guv-a-Mint please, Please bring back that Guv-a-Mint Cheeeeeeeeese! From eryksun at gmail.com Wed Jul 17 05:11:25 2013 From: eryksun at gmail.com (eryksun) Date: Tue, 16 Jul 2013 23:11:25 -0400 Subject: [Tutor] Deleted dictionary length not reporting In-Reply-To: References: Message-ID: On Tue, Jul 16, 2013 at 9:48 PM, Jim Mooney wrote: > > Also, the printout doesn't occur until after all the sounds play, so > maybe that has something to do with it. But I also tried saving the > lengths in a list while the program ran, and got the same result. A > list of all '125' The print occurs after each sound is played, not after "all the sounds play". winsound.PlaySound defaults to blocking mode (synchronous). To play without blocking (asynchronous), use the flag winsound.SND_ASYNC. But you don't want that for queuing up multiple tracks since they'd all run simultaneously. PlaySound http://msdn.microsoft.com/en-us/library/dd743680 http://docs.python.org/2/library/winsound > import winsound > import random, os > random.seed() The system random is an instance of random.Random, which already calls self.seed(): def __init__(self, x=None): """Initialize an instance. Optional argument x controls seeding, as for Random.seed(). """ self.seed(x) self.gauss_next = None > lenlist = [] > file_list = os.listdir('WAV') > > for clip in range(0,7): > file_dict = dict(enumerate(file_list)) > sound_key = random.choice(range(0,len(file_list))) > winsound.PlaySound('WAV/' + file_dict[sound_key], winsound.SND_FILENAME) > del file_dict[sound_key] > print(len(file_dict.keys())) I'd use random.sample: NUM_CLIPS = 7 CLIP_PATH = 'WAV' # I'd use an absolute path here clip_list = random.sample(os.listdir(CLIP_PATH), NUM_CLIPS) for clip in clip_list: clip = os.path.join(CLIP_PATH, clip) winsound.PlaySound(clip, winsound.SND_FILENAME) From eryksun at gmail.com Wed Jul 17 05:16:15 2013 From: eryksun at gmail.com (eryksun) Date: Tue, 16 Jul 2013 23:16:15 -0400 Subject: [Tutor] Deleted dictionary length not reporting In-Reply-To: References: <51E5FBC8.7050801@pearwood.info> Message-ID: On Tue, Jul 16, 2013 at 10:55 PM, Jim Mooney wrote: >> That's very strange. print is not supposed to be buffered, it should always >> print immediately. I have no ideas about that. > > Something winsound must do, since I printed out the time at the end of > each loop and the loop delays for the length of each sound clip, which > are a few seconds long. > > Tue Jul 16 19:48:01 2013 > Tue Jul 16 19:48:12 2013 > Tue Jul 16 19:48:16 2013 > Tue Jul 16 19:48:22 2013 > Tue Jul 16 19:48:25 2013 > > but the print that's in the loop doesn't appear until the loop is > done, eve after I put the enumerate where it should go. I cannot reproduce this in the console. Are you using an IDE? In that case it could be an issue with a stdout proxy or a buffered pipe. From cybervigilante at gmail.com Wed Jul 17 05:25:23 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Tue, 16 Jul 2013 20:25:23 -0700 Subject: [Tutor] Deleted dictionary length not reporting In-Reply-To: References: Message-ID: On 16 July 2013 20:11, eryksun wrote: > The system random is an instance of random.Random, which already calls > self.seed(): > Ah, good, saving steps is always good. > I'd use random.sample: Another saving. Can I assume random.sample is designed to Not pick duplicates? And I found the index error (in the original prog, that uses a list, not a dict). I was thinking of range and such, that cuts off the last value, but randint doesn't cut off the end value. -- Jim I sure love that Guv-a-Mint Cheese, Guv-a-Mint Cheese, Guv-a-Mint Cheese Oh please Guv-a-Mint, Guv-a-Mint please, Please bring back that Guv-a-Mint Cheeeeeeeeese! From jalespring at gmail.com Wed Jul 17 05:34:04 2013 From: jalespring at gmail.com (Arnel Legaspi) Date: Wed, 17 Jul 2013 03:34:04 +0000 (UTC) Subject: [Tutor] Writing unit tests that involve email References: <51E14B6B.8060609@zoho.com> <51E1F1B5.3060302@gmail.com> <51E604D6.7040609@pearwood.info> Message-ID: Steven D'Aprano pearwood.info> writes: > > On 14/07/13 10:32, Arnel Legaspi wrote: > [...] > > What I'm having difficulty is getting the unit tests to properly run the > > script being tested and send the email, get the proper Maildir message, > > and be able to read it via the unit test script. Right now, even when > > the SMTP server is not running, all the tests for checking the sent > > email pass, which is not what I intended. > > > > I know the SMTP server works, because if I send some email using another > > Python or Powershell script, the script logs on the screen that it did > > receive the email. > > > > How should I approach this? > > > > Here is the unit test script I have (some lines may get wrapped): > > Firstly, start with the obvious: are you sure that the unittests are being > run? If I have counted correctly, you only have seven test cases at the > moment, so your unittest output should show seven dots and then print > something like: > > Ran 7 tests in 0.01s > > OK > > Can you confirm that this is the case? If not, then your problem is that the > tests aren't running at all. Yes, they do. All the tests pass, but when I look inside the Maildir directories (and the dsmtpd log on the console), no email appears to get sent. I was expecting something along the lines of (see the 3rd output line): $ dsmtpd -d mail_dir -p 8000 2013-07-17 11:07:53,029 INFO: Starting dsmtpd 0.2.2 at 127.0.0.1:8000 2013-07-17 11:07:53,053 INFO: Store the incoming emails into mail_dir 2013-07-17 11:13:18,618 INFO: 127.0.0.1:38231: user at wherever.com -> test at email.net [Email Subject] If I run this test script, I should be seeing 7 lines similar to it. > [...] > > parser.set('smtp', 'password', ***redacted***) > > I hope this isn't a real password to a real mail server visible from the > internet, because you've now made it visible to everyone in the world. Nope, it's not. It's really just for the unit test. (I also hope no one uses "user at wherever.com" but it shouldn't be likely.) > > class TestReadingSendingPostFile(unittest.TestCase): > > > > def setUp(self): > > """ Prepare a post file for testing. """ > > create_configfile() > > with open('post.md', 'wb') as postfile: > > postfile.write(MKDOWN_TEXT) > > self.mbox = mailbox.Maildir('mail_dir') > > Personally, I don't use the setUp and testDown methods, but most of my code > avoids storing state so they are rarely relevant to me. But in any case, the > setUp method is called before *every* test method. If you want it to be > called only once, you should put it inside the class __init__ method. > (Don't forget to call the TestCase __init__ as well.) I needed the setUp / tearDown methods because the script I wrote (on the Bitbucket repo) reads email sending/receiving information from a config file. Putting something in class __init__ method is new to me. > I see all of your test methods include clear_args() at the start, so you > should put that inside the setUp method. It would help if you tell us > which test method you are expecting to fail. My guess is that it is this one: > > > def test_sendpost(self): > > clear_args() > > add_testfiles() > > for message in self.mbox: > > self.assertEqual(message['subject'], 'post') > > self.assertEqual(message.get_payload(), HTML_TXT) > > I don't believe this actually tries to send any email, and as far as I can > see you never actually clear the mailbox, so all you're doing is looking at > old emails that happened to exist. > > If the mbox happens to be empty, then neither of the assertEqual calls will > happen, and the test will just pass. Add this to the test: > > if not self.mbox: > self.fail("No messages were sent") > > or better still, add a separate test to check that email is sent, without > caring about the content of the email. That way, if you get a failure, you > can immediately see whether the failure is "email wasn't sent", or "email was > sent, but contains the wrong stuff". I did put in something like that before, which is how I confirmed the test was not sending the emails like I wanted it to do. > I suggest you factor out any tests of sending mail into a separate class. > Give this class a setUp() that creates a mailbox, and a tearDown that moves > everything from that mailbox into another one. That means you have a nice > clean mailbox at the start of each test, but you can still inspect the emails > by eye if you want. It also means that you can separate tests with an > external dependency (the mail server) from those without. All right, but the trouble I have is on making the unit tests run such that it will force the script I'm testing to send the email. If I just use the script on my own, it does send the emails, no problem. With the unit tests I've written, it's not doing so. Sorry if I come off a little dense, but I don't think I misread what you were trying to say here. Thanks, Arnel From steve at pearwood.info Wed Jul 17 05:42:40 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 17 Jul 2013 13:42:40 +1000 Subject: [Tutor] Deleted dictionary length not reporting In-Reply-To: References: <51E5FBC8.7050801@pearwood.info> Message-ID: <51E612B0.6080706@pearwood.info> On 17/07/13 12:55, Jim Mooney wrote: > so I'm going back to just popping used items > from a list, which was dirt-simple. And also dirt-simple to get wrong, and inefficient as well. list.pop has its uses, but I strongly recommend that you learn more "Pythonic" techniques that don't rely on modifying the list unnecessarily. For example, instead of something like this: while some_list: process(some_list[0]) # work on the first item some_list.pop(0) # and then drop it this will be *much* more efficient, as well as easier to understand: for item in some_list: process(item) If you really need to clear the list, it's more efficient to clear it all at once at the end than to clear it item-by-item: some_list[:] = [] Here's a question for you, to test your Python knowledge: Assuming that some_list is already a list, what's the difference between these two lines, and under what circumstances why would you want to use the second one? some_list = [] some_list[:] = [] (Tutors, please leave this one for the beginners to answer.) -- Steven From steve at pearwood.info Wed Jul 17 06:04:15 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 17 Jul 2013 14:04:15 +1000 Subject: [Tutor] Writing unit tests that involve email In-Reply-To: References: <51E14B6B.8060609@zoho.com> <51E1F1B5.3060302@gmail.com> <51E604D6.7040609@pearwood.info> Message-ID: <51E617BF.2090108@pearwood.info> On 17/07/13 13:34, Arnel Legaspi wrote: > the trouble I have is on making the unit tests run such that > it > will force the script I'm testing to send the email. If I just use the > script > on my own, it does send the emails, no problem. With the unit tests I've > written, it's not doing so. How does your script send emails? At some place, there will be something that effectively does the following steps: * build email * send email I recommend that your script has a function that does each: somewhere you have something that says def build_email_body(arg1, arg2, arg3): ... def send_email(body): ... Then you can test each separately. In your unit tests, you just call build_email_body and sees if it returns the correct data, no sending required; and then you call send_email and see that it actually sends an email. Oh, all of this assumes that the unit test imports your script! This makes it critical that your script includes something like this at the end: def run(): # or "main", if you prefer # script logic goes in here if __name__ == '__main__': run() Then your tests can import the script without it automatically running. -- Steven From jalespring at gmail.com Wed Jul 17 07:14:28 2013 From: jalespring at gmail.com (Arnel Legaspi) Date: Wed, 17 Jul 2013 05:14:28 +0000 (UTC) Subject: [Tutor] Writing unit tests that involve email References: <51E14B6B.8060609@zoho.com> <51E1F1B5.3060302@gmail.com> <51E604D6.7040609@pearwood.info> <51E617BF.2090108@pearwood.info> Message-ID: Steven D'Aprano pearwood.info> writes: > > On 17/07/13 13:34, Arnel Legaspi wrote: > > > the trouble I have is on making the unit tests run such that > > it > > will force the script I'm testing to send the email. If I just use the > > script > > on my own, it does send the emails, no problem. With the unit tests I've > > written, it's not doing so. > > How does your script send emails? At some place, there will be something > that effectively does the following steps: > > * build email > * send email > > I recommend that your script has a function that does each: somewhere you > have something that says > > def build_email_body(arg1, arg2, arg3): > ... > > def send_email(body): > ... > > Then you can test each separately. In your unit tests, you just call > build_email_body and sees if it returns the correct data, no sending > required; and then you call send_email and see that it actually sends an > email. The script does have a send_post() function that essentially builds the email message itself, along with the necessary headers, and runs the smtplib.SMTP_SSL() method to send the whole thing off to the SMTP server. Should I factor each component out of the send_post() function to test for them separately, or just leave it as is? Is it better to do so? > Oh, all of this assumes that the unit test imports your script! This makes > it critical that your script includes something like this at the end: > > def run(): # or "main", if you prefer > # script logic goes in here > > if __name__ == '__main__': > run() > > Then your tests can import the script without it automatically running. So it does come down to importing the script methods *inside* the unit tests. I thought it was done otherwise. In the unit tests, I have methods that clears the arguments given to the script (clear_args()) and replaces them with the ones created by the setUp() method (add_testfiles()). I do have a run() function which gets the arguments, and runs the send_post() function. I guess I'll have to try and see whether it's enough for the script to run with the unit-test-supplied arguments and probably just add something like: import downpost ... class TestReadingSendingPostFile(unittest.TestCase): def setUp(): ... def test_sendpost(self): clear_args() add_testfiles() downpost.run() All right, Steven. Thanks for the nudge in the right direction! --Arnel From karanmatic at gmail.com Wed Jul 17 07:26:23 2013 From: karanmatic at gmail.com (Karan Goel) Date: Wed, 17 Jul 2013 10:56:23 +0530 Subject: [Tutor] A list of 100+ projects to complete to master Python. In-Reply-To: References: Message-ID: I'm not entirely sure if your reply was directed towards my message. - Karan Goel Goel.im | Resume | Github On Wed, Jul 17, 2013 at 5:09 AM, David Hutto wrote: > First thing you should learn is offsite backups, I've lost several > projects in the works because of a hd mishap. Secondarily, I would > recommend using existing primarily used python projects to 'reinvent the > wheel' so to speak. Thirdly, make sure the code is properly documented, and > serves a purpose. And lastly, utilize other programs with python api's, > such as blender, which can make your apps pop, or diversify into game > development kits. > > > On Tue, Jul 16, 2013 at 1:09 PM, Karan Goel wrote: > >> Hey guys and gals >> >> Just a plug here. My repo: https://github.com/thekarangoel/Projects >> was one of the trending repos on Gh this week and I thought folks >> on this list might be interested in knowing about it. >> >> In a nutshell, I'm trying to complete over a 100 practical language- >> agnostic projects in Python only. I haven't read the project details yet, >> and I'm not filtering things out. >> >> If you would like to do the same, join me. Fork or star the repo, >> and start coding (in any language, really). >> >> https://github.com/thekarangoel/Projects >> >> Let me know if you have any questions or suggestions. >> >> - Karan Goel >> Goel.im | Resume >> | Github >> >> _______________________________________________ >> Tutor maillist - Tutor at python.org >> To unsubscribe or change subscription options: >> http://mail.python.org/mailman/listinfo/tutor >> >> > > > -- > Best Regards, > David Hutto > *CEO:* *http://www.hitwebdevelopment.com* > -------------- next part -------------- An HTML attachment was scrubbed... URL: From __peter__ at web.de Wed Jul 17 08:36:55 2013 From: __peter__ at web.de (Peter Otten) Date: Wed, 17 Jul 2013 08:36:55 +0200 Subject: [Tutor] Writing unit tests that involve email References: <51E14B6B.8060609@zoho.com> <51E1F1B5.3060302@gmail.com> Message-ID: Arnel Legaspi wrote: > class TestReadingSendingPostFile(unittest.TestCase): > def test_wrongconfigfile(self): > clear_args() > add_testfiles('post.md', 'bloginfo.ini') > self.assertRaises(IOError) Where did you catch up that idiom? The assertRaises() call above does nothing. You must either invoke it with a callable self.assertRaises(IOError, function_supposed_to_raise) or use it as a context manager with self.assertRaises(IOError): ... # code supposed to raise an IOError From cybervigilante at gmail.com Wed Jul 17 08:48:44 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Tue, 16 Jul 2013 23:48:44 -0700 Subject: [Tutor] Deleted dictionary length not reporting In-Reply-To: <51E612B0.6080706@pearwood.info> References: <51E5FBC8.7050801@pearwood.info> <51E612B0.6080706@pearwood.info> Message-ID: On 16 July 2013 20:42, Steven D'Aprano wrote: > Here's a question for you, to test your Python knowledge: What knowledge? I've been fooling around in this awful AZ heatwave and only got to chap 5 in the Lutz book. ;') > > Assuming that some_list is already a list, what's the difference between > these two lines, and under what circumstances why would you want to use the > second one? > > some_list = [] > some_list[:] = [] > However, fooling around helps, so I'd say: >>> p = [1,2,3] >>> id(p) 34586456 >>> p[:] = [] >>> id(p) 34586456 >>> p = [] >>> id(p) 34586752 The slice will pass an "is" test, the equal will not. No fair making me think.This is as bad as being back in school. -- Jim I sure love that Guv-a-Mint Cheese, Guv-a-Mint Cheese, Guv-a-Mint Cheese Oh please Guv-a-Mint, Guv-a-Mint please, Please bring back that Guv-a-Mint Cheeeeeeeeese! From jalespring at gmail.com Wed Jul 17 09:09:20 2013 From: jalespring at gmail.com (Arnel Legaspi) Date: Wed, 17 Jul 2013 15:09:20 +0800 Subject: [Tutor] Writing unit tests that involve email In-Reply-To: References: Message-ID: <51E64320.4040609@gmail.com> Peter Otten wrote: > Arnel Legaspi wrote: > > class TestReadingSendingPostFile(unittest.TestCase): > > def test_wrongconfigfile(self): > > clear_args() > > add_testfiles('post.md', 'bloginfo.ini') > > self.assertRaises(IOError) > > Where did you catch up that idiom? The assertRaises() call above does > nothing. You must either invoke it with a callable > > or use it as a context manager > > with self.assertRaises(IOError): > ... # code supposed to raise an IOError Ah, so that's how it was supposed to be used. That idiom was a product of my misunderstanding how assertRaises() was described in the manual. Thanks, Peter. (This really should have been the answer when I asked about it on StackOverflow.) --Arnel From eryksun at gmail.com Wed Jul 17 12:01:53 2013 From: eryksun at gmail.com (eryksun) Date: Wed, 17 Jul 2013 06:01:53 -0400 Subject: [Tutor] Deleted dictionary length not reporting In-Reply-To: References: <51E5FBC8.7050801@pearwood.info> <51E612B0.6080706@pearwood.info> Message-ID: > On 16 July 2013 20:42, Steven D'Aprano wrote: > >> Assuming that some_list is already a list, what's the difference between >> these two lines, and under what circumstances why would you want to use the >> second one? >> >> some_list = [] >> some_list[:] = [] 3.3 adds clear() to list and bytearray (as per dict/set), so now there are 3 simple ways to clear a list: del some_list[:] # DELETE_SUBSCR some_list[:] = [] # STORE_SUBSCR some_list.clear() # CALL_FUNCTION From karanmatic at gmail.com Wed Jul 17 13:12:20 2013 From: karanmatic at gmail.com (Karan Goel) Date: Wed, 17 Jul 2013 16:42:20 +0530 Subject: [Tutor] How to write on Planet Python Message-ID: How do I publish a post on Planet Python (http://planet.python.org/). I cannot find any information for contributors. - Karan Goel Goel.im | Resume | Github -------------- next part -------------- An HTML attachment was scrubbed... URL: From karanmatic at gmail.com Wed Jul 17 13:13:49 2013 From: karanmatic at gmail.com (Karan Goel) Date: Wed, 17 Jul 2013 16:43:49 +0530 Subject: [Tutor] What all technologies should a python developer know? Message-ID: So I want to be good at python (web) development. What all technologies do I need to know? Git. Linux. Django. webapp2. Heroku? What else? How proficient should I be in each? (x-post from the reddit thread) - Karan Goel Goel.im | Resume | Github -------------- next part -------------- An HTML attachment was scrubbed... URL: From kwpolska at gmail.com Wed Jul 17 13:24:57 2013 From: kwpolska at gmail.com (=?UTF-8?B?Q2hyaXMg4oCcS3dwb2xza2HigJ0gV2Fycmljaw==?=) Date: Wed, 17 Jul 2013 13:24:57 +0200 Subject: [Tutor] Fwd: How to write on Planet Python In-Reply-To: References: Message-ID: On Wed, Jul 17, 2013 at 1:12 PM, Karan Goel wrote: > How do I publish a post on Planet Python (http://planet.python.org/). I > cannot find any information for contributors. > > - Karan Goel > Goel.im | Resume | Github > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > You need a blog with a RSS feed. When you get one, send mail to planet at python.org requesting inclusion of your site in Planet Python, which just happens to be a RSS aggregator. https://en.wikipedia.org/wiki/Planet_(software) -- Kwpolska | GPG KEY: 5EAAEA16 stop html mail | always bottom-post http://asciiribbon.org | http://caliburn.nl/topposting.html From wayne at waynewerner.com Wed Jul 17 13:35:27 2013 From: wayne at waynewerner.com (Wayne Werner) Date: Wed, 17 Jul 2013 06:35:27 -0500 (CDT) Subject: [Tutor] Best way to learn Tkinter In-Reply-To: References: <9D8E4E9E-78ED-4D71-B432-378D0412C19C@gmail.com> Message-ID: On Tue, 16 Jul 2013, Alan Gauld wrote: > On 15/07/13 22:38, Ben Fishbein wrote: >> Hello. I wrote a python program for my small business. >> It works fine but it is all from command prompt and makes sense >> to no one but me. I want to create a new version with a GUI >> that an employee could use. > > I still like the Pythonware one: > > http://effbot.org/tkinterbook/ > > But as an aside have you thought about making it a web app instead? > That is much easier to maintain - no need to distribute updates etc. > There are many fine Python web frameworks including Django and > Pylons. If you can find a server to host it and your users are > online it might be a better option. Flask is another great choice, and one I find to be quite simple. If you have say, one computer at work that everybody uses, or a computer at work that is available on the network, any of these web frameworks would work just swell. Here's a sample Flask app: from flask import Flask, request app = Flask(__name__) @app.route('/') def main(): name = request.args.get('name') if name: return "Hello, {}!".format(name) else: return "Hello, werld!" if __name__ == "__main__": app.run('0.0.0.0', #listens to the whole network debug=True, #Useful when developing. Auto restart on changes port=5000) #Default port. Can be changed to any available num Flask (as well as the others, I'm sure) also offer plugins that help with forms, keeping users logged in, authenticating with OAuth... I find it quite fantastic, personally, and use it on almost all my projects anymore. I even ported it to run on Python for Android and can run Flask apps on my phone. How awesome is that??? HTH, Wayne From francois.dion at gmail.com Wed Jul 17 13:40:00 2013 From: francois.dion at gmail.com (Francois Dion) Date: Wed, 17 Jul 2013 07:40:00 -0400 Subject: [Tutor] How to write on Planet Python In-Reply-To: References: Message-ID: On Jul 17, 2013, at 7:12 AM, Karan Goel wrote: > How do I publish a post on Planet Python (http://planet.python.org/). I cannot find any information for contributors. At the bottom of the list of blogs is this: To request addition or removal: e-mail planet at python.org (note, responses can take up to a few days) Now, make sure you subscribe a URL that feeds python articles, not your vacation pictures or a restaurant review... You can tag your python posts and provide the url for that tag. Francois -- Www.pyptug.org. - raspberry-python.blogspot.com. - @f_dion -------------- next part -------------- An HTML attachment was scrubbed... URL: From francois.dion at gmail.com Wed Jul 17 13:55:11 2013 From: francois.dion at gmail.com (Francois Dion) Date: Wed, 17 Jul 2013 07:55:11 -0400 Subject: [Tutor] What all technologies should a python developer know? In-Reply-To: References: Message-ID: On Jul 17, 2013, at 7:13 AM, Karan Goel wrote: > So I want to be good at python (web) development. What all technologies do I need to know? What is your goal? You dont need to know any technology but those you need to accomplish your goal. Perhaps you are looking for employment? Maybe even a specific employer? What technology do they use? Start with those, then. As far as being good at python (web) development, first and foremost you have to be good at python itself, and that just requires time and effort. And lots of practice. Learn how to write idiomatic python. Then learn a microframework (aka antiframework). That will occupy your next 12 months (i did say time and effort). Francois -- www.pyptug.org. - raspberry-python.blogspot.com. - @f_dion From mail at timgolden.me.uk Wed Jul 17 14:00:34 2013 From: mail at timgolden.me.uk (Tim Golden) Date: Wed, 17 Jul 2013 13:00:34 +0100 Subject: [Tutor] What all technologies should a python developer know? In-Reply-To: References: Message-ID: <51E68762.2000809@timgolden.me.uk> On 17/07/2013 12:13, Karan Goel wrote: > So I want to be good at python (web) development. What all technologies > do I need to know? Git. Linux. Django. webapp2. Heroku? > What else? How proficient should I be in each? You don't *need* any of those specifically. It would make sense to be competent with: * Some version control system (possibly git) * The O/S on which your apps are developed and/or deployed (possibly Linux) * Some web framework that fits your need (possibly Django) (Very optionally): * Some PaaS for deployment (possibly Heroku) But the main points are to understand how the web works, both the older (CGI-derived server-refresh) world and newer (Single-Page App / Javascript-heavy) world, and how Python works. That sounds suspiciously like a tautology: To be good at Python Web Development, you need to be good at Python and at Web Development! But I'm constantly amazed at people who launch out without either, hoping for a drag-and-drop experience and frustrated when they don't get one. :) TJG From steve at pearwood.info Wed Jul 17 14:08:43 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 17 Jul 2013 22:08:43 +1000 Subject: [Tutor] Deleted dictionary length not reporting In-Reply-To: References: <51E5FBC8.7050801@pearwood.info> <51E612B0.6080706@pearwood.info> Message-ID: <51E6894B.7030609@pearwood.info> On 17/07/13 16:48, Jim Mooney wrote: > On 16 July 2013 20:42, Steven D'Aprano wrote: > >> Here's a question for you, to test your Python knowledge: > > What knowledge? I've been fooling around in this awful AZ heatwave and > only got to chap 5 in the Lutz book. ;') >> >> Assuming that some_list is already a list, what's the difference between >> these two lines, and under what circumstances why would you want to use the >> second one? >> >> some_list = [] >> some_list[:] = [] >> > However, fooling around helps, so I'd say: > >>>> p = [1,2,3] >>>> id(p) > 34586456 >>>> p[:] = [] >>>> id(p) > 34586456 >>>> p = [] >>>> id(p) > 34586752 > > The slice will pass an "is" test, the equal will not. Correct, but you may not have worked out the consequences that follow from that. [Aside: ID numbers may be reused. So it is possible, although unlikely, that id(p) will remain the same by chance after p = []. That can't happen with Jython or IronPython, as they never re-use IDs, and I don't think it can happen for CPython, but it would be permitted behaviour for Python implementations. So if you wish to test for "is", you should actually use "is", and not just eyeball the ID.] p = [1, 2, 3] q = p # confirm that q is not a copy of p, but they are # two names for the same object assert p is q p[:] = [] assert p is q # p and q remain the same object assert q == [] p = q = [1, 2, 3] p = [] assert p is not q # p is a new list, q is still the old one This is not just of theoretical interest. Sometimes you will have a function that accepts a list argument, and you might wish to clear the list. Depending on how you do so, the change may or may not propagate outwards to the caller: mylist = [1, 2, 3, 4] process(mylist) print mylist What gets printed depends on what process() does. If process() clears the list using the slice version, then it will be cleared on the outside too. But if instead it assigns a new empty list to it, then the old list object remains untouched and mylist will not change. > No fair making me think.This is as bad as being back in school. Heh :-) -- Steven From darinlh at gmail.com Wed Jul 17 14:30:08 2013 From: darinlh at gmail.com (Darin Lawson Hosking) Date: Wed, 17 Jul 2013 07:30:08 -0500 Subject: [Tutor] What all technologies should a python developer know? In-Reply-To: <51E68762.2000809@timgolden.me.uk> References: <51E68762.2000809@timgolden.me.uk> Message-ID: Speaking as a self taught old school tech instructor to a beginning tech wanting to start right. Learn the basic technology first Basic HTML formating gets vs posts reading writing files Parsing strings / regex inserting, searching and deleting via "SQL" communication standards RSS XML JSON etc.. NON standard data mining scraping web pages Basics of version control and deployment standards and master python core and main libraries building some of your own libraries and tools. Just these will place you far ahead of the curve. I don't know how many times I have asked a "new hire" to build xyz and they have asked me "you want me to create a report and publish as an rss?? whats rss and why? or "we get that report as a daily email in a spreadsheet why to you need to add it to a database?" Web development is only part of the upcoming skills needed "big data" and data mining are where many jobs will be created. Good luck Darin On Wed, Jul 17, 2013 at 7:00 AM, Tim Golden wrote: > On 17/07/2013 12:13, Karan Goel wrote: > > So I want to be good at python (web) development. What all technologies > > do I need to know? Git. Linux. Django. webapp2. Heroku? > > What else? How proficient should I be in each? > > > You don't *need* any of those specifically. It would make sense to be > competent with: > > * Some version control system (possibly git) > > * The O/S on which your apps are developed and/or deployed (possibly Linux) > > * Some web framework that fits your need (possibly Django) > > (Very optionally): > > * Some PaaS for deployment (possibly Heroku) > > > But the main points are to understand how the web works, both the older > (CGI-derived server-refresh) world and newer (Single-Page App / > Javascript-heavy) world, and how Python works. > > That sounds suspiciously like a tautology: To be good at Python Web > Development, you need to be good at Python and at Web Development! But > I'm constantly amazed at people who launch out without either, hoping > for a drag-and-drop experience and frustrated when they don't get one. :) > > TJG > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > -------------- next part -------------- An HTML attachment was scrubbed... URL: From davea at davea.name Wed Jul 17 14:42:39 2013 From: davea at davea.name (Dave Angel) Date: Wed, 17 Jul 2013 08:42:39 -0400 Subject: [Tutor] What all technologies should a python developer know? In-Reply-To: <51E68762.2000809@timgolden.me.uk> References: <51E68762.2000809@timgolden.me.uk> Message-ID: On 07/17/2013 08:00 AM, Tim Golden wrote: > On 17/07/2013 12:13, Karan Goel wrote: >> So I want to be good at python (web) development. What all technologies >> do I need to know? Git. Linux. Django. webapp2. Heroku? >> What else? How proficient should I be in each? > > > You don't *need* any of those specifically. It would make sense to be > competent with: > > * Some version control system (possibly git) > > * The O/S on which your apps are developed and/or deployed (possibly Linux) > > * Some web framework that fits your need (possibly Django) > > (Very optionally): > > * Some PaaS for deployment (possibly Heroku) > > > But the main points are to understand how the web works, both the older > (CGI-derived server-refresh) world and newer (Single-Page App / > Javascript-heavy) world, and how Python works. > > That sounds suspiciously like a tautology: To be good at Python Web > Development, you need to be good at Python and at Web Development! But > I'm constantly amazed at people who launch out without either, hoping > for a drag-and-drop experience and frustrated when they don't get one. :) > I'm also amazed at how many people launching into programming (usually with degrees in Computer Science) still don't have a good knowledge of bits, bytes, concurrency, protocols, namespaces, recursion, caching, memory management, algorithms, flow control, indirection, virtual functions, dispatch tables, syntax trees, ...) Understanding many of these things and other similar, regardless of which programming language, makes it much easier to learn a new language, a new environment, and to avoid traps that every environment has for you. If you (Karan) just want to do web development, and using a particular set of tools, then learn those. If you're trying to get a first job, learn the things that the job requires. But if you're also trying to build a career, dig. Dig deep. And that will require either a very good teacher(s), a very good student, or a large variety of "environments," preferably all three. Another, seldom-mentioned, useful tool is a thesaurus. Get used to finding meaningful names for things. Several times I've joined a new company or new department, and my first task was to learn the programming language I was to be using. Something like 35 during my career, plus many more for recreation and study. Learn at least one language in each of several categories, and your mind will be much more nimble, whichever one you actually need to use: assembler, C++, lisp, Forth, Python, ... When you're learning Forth, don't just write an application. Extend the compiler, learn what a code field really is, and think about how it relates to the vptr in C++. When you're learning lisp, think about what it means that "functions" are no different than other data, and can be manipulated as readily. You do similar things in Python and C++ too, but the meaning is entirely different. When you're learning Python, write a decorator of your own, don't just use the ones in the standard library. When you're learning java, study the byte code file format, and think about what it means to have a const static variable in a class. (Maybe they fixed the file format and semantics since I last looked, but this was a fundamental flaw in the original format) When you're learning C++, step through a constructor of a derived class that calls virtual functions that have been overridden. Watch the vptr change while the object is being constructed, and see what that means to your code. Naturally, this is only possible since you learned assembler first. I could continue, randomly attacking other aspects of a development environment. The point is that there are traps everywhere, and beautiful symmetries everywhere. Learn how to recognize the former, and learn to appreciate the latter. Learn what to do when the docs fall short (eg. drop to assembler to see what the language construct is really doing, reverse engineer a file format, write some code to "do it by hand," etc.). -- DaveA From kwpolska at gmail.com Wed Jul 17 15:09:45 2013 From: kwpolska at gmail.com (=?UTF-8?B?Q2hyaXMg4oCcS3dwb2xza2HigJ0gV2Fycmljaw==?=) Date: Wed, 17 Jul 2013 15:09:45 +0200 Subject: [Tutor] Fwd: How to write on Planet Python In-Reply-To: References: Message-ID: Make sure you include tutor at python.org in the To: or CC: field. ---------- Forwarded message ---------- From: Karan Goel Date: Wed, Jul 17, 2013 at 1:30 PM Subject: Re: [Tutor] How to write on Planet Python To: Chris ?Kwpolska? Warrick But I just want to publish one single post. Should I create a blog just for that? - Karan Goel Sent from Nexus 4. Please ignore typos and abbreviations. On Jul 17, 2013 4:54 PM, "Chris ?Kwpolska? Warrick" wrote: > > On Wed, Jul 17, 2013 at 1:12 PM, Karan Goel wrote: > > How do I publish a post on Planet Python (http://planet.python.org/). I > > cannot find any information for contributors. > > > > - Karan Goel > > Goel.im | Resume | Github > > > > _______________________________________________ > > Tutor maillist - Tutor at python.org > > To unsubscribe or change subscription options: > > http://mail.python.org/mailman/listinfo/tutor > > > > You need a blog with a RSS feed. When you get one, send mail to > planet at python.org requesting inclusion of your site in Planet Python, > which just happens to be a RSS aggregator. > https://en.wikipedia.org/wiki/Planet_(software) > > -- > Kwpolska | GPG KEY: 5EAAEA16 > stop html mail | always bottom-post > http://asciiribbon.org | http://caliburn.nl/topposting.html -- Kwpolska | GPG KEY: 5EAAEA16 stop html mail | always bottom-post http://asciiribbon.org | http://caliburn.nl/topposting.html From wolfrage8765 at gmail.com Wed Jul 17 17:09:33 2013 From: wolfrage8765 at gmail.com (wolfrage8765 at gmail.com) Date: Wed, 17 Jul 2013 11:09:33 -0400 Subject: [Tutor] What all technologies should a python developer know? In-Reply-To: References: <51E68762.2000809@timgolden.me.uk> Message-ID: On Wed, Jul 17, 2013 at 8:42 AM, Dave Angel wrote: > On 07/17/2013 08:00 AM, Tim Golden wrote: >> >> On 17/07/2013 12:13, Karan Goel wrote: >>> >>> So I want to be good at python (web) development. What all technologies >>> do I need to know? Git. Linux. Django. webapp2. Heroku? >>> What else? How proficient should I be in each? >> >> >> >> You don't *need* any of those specifically. It would make sense to be >> competent with: >> >> * Some version control system (possibly git) >> >> * The O/S on which your apps are developed and/or deployed (possibly >> Linux) >> >> * Some web framework that fits your need (possibly Django) >> >> (Very optionally): >> >> * Some PaaS for deployment (possibly Heroku) >> >> >> But the main points are to understand how the web works, both the older >> (CGI-derived server-refresh) world and newer (Single-Page App / >> Javascript-heavy) world, and how Python works. >> >> That sounds suspiciously like a tautology: To be good at Python Web >> Development, you need to be good at Python and at Web Development! But >> I'm constantly amazed at people who launch out without either, hoping >> for a drag-and-drop experience and frustrated when they don't get one. :) >> > > I'm also amazed at how many people launching into programming (usually with > degrees in Computer Science) still don't have a good knowledge of bits, > bytes, concurrency, protocols, namespaces, recursion, caching, memory > management, algorithms, flow control, indirection, virtual functions, > dispatch tables, syntax trees, ...) I am not so suprised as I used to be. Since I have seen that schools do not teach like they should, instead they skip the fundamentals and head straight to teaching a language but usually with out any reason as to why or how. Many of my peers simply do not have the knowledge needed to actually program with out a book directing them through the process. But the book still leaves out the fundamentals. > > Understanding many of these things and other similar, regardless of which > programming language, makes it much easier to learn a new language, a new > environment, and to avoid traps that every environment has for you. > > If you (Karan) just want to do web development, and using a particular set > of tools, then learn those. If you're trying to get a first job, learn the > things that the job requires. But if you're also trying to build a career, > dig. Dig deep. And that will require either a very good teacher(s), a very > good student, or a large variety of "environments," preferably all three. But you can be your own teacher. I say that because I have found good teachers hard to come by. But I have successfully taught myself a lot of what is being stated here, although I have a long ways to go. Since it does take a lot of time and effort to truely be a good programmer. I have learned there is no magic trick or moment, or language.... just time and effort. > > Another, seldom-mentioned, useful tool is a thesaurus. Get used to finding > meaningful names for things. > > Several times I've joined a new company or new department, and my first task > was to learn the programming language I was to be using. Something like 35 > during my career, plus many more for recreation and study. Learn at least > one language in each of several categories, and your mind will be much more > nimble, whichever one you actually need to use: assembler, C++, lisp, Forth, > Python, ... Wow 35, I hope to accel to such a level one day. > > When you're learning Forth, don't just write an application. Extend the > compiler, learn what a code field really is, and think about how it relates > to the vptr in C++. > > When you're learning lisp, think about what it means that "functions" are no > different than other data, and can be manipulated as readily. You do similar > things in Python and C++ too, but the meaning is entirely different. Lisp or Erlang is probably my next language to learn, as I wish to better understand Functional Programming. > > When you're learning Python, write a decorator of your own, don't just use > the ones in the standard library. > > When you're learning java, study the byte code file format, and think about > what it means to have a const static variable in a class. (Maybe they fixed > the file format and semantics since I last looked, but this was a > fundamental flaw in the original format) > > When you're learning C++, step through a constructor of a derived class that > calls virtual functions that have been overridden. Watch the vptr change > while the object is being constructed, and see what that means to your code. > Naturally, this is only possible since you learned assembler first. > > I could continue, randomly attacking other aspects of a development > environment. The point is that there are traps everywhere, and beautiful > symmetries everywhere. Learn how to recognize the former, and learn to > appreciate the latter. Learn what to do when the docs fall short (eg. drop > to assembler to see what the language construct is really doing, reverse > engineer a file format, write some code to "do it by hand," etc.). Dave, how do I "drop down to assembler" with Python? Can you expand on this? Also do you think it is still of value to learn Assembly Language, and if so which variant, as I understand it, there is a unique set for each architecture, although they are usually similiar. > > > -- > DaveA > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor From davea at davea.name Wed Jul 17 19:34:33 2013 From: davea at davea.name (Dave Angel) Date: Wed, 17 Jul 2013 13:34:33 -0400 Subject: [Tutor] What all technologies should a python developer know? In-Reply-To: References: <51E68762.2000809@timgolden.me.uk> Message-ID: On 07/17/2013 11:09 AM, wolfrage8765 at gmail.com wrote: > On Wed, Jul 17, 2013 at 8:42 AM, Dave Angel wrote: >> I could continue, randomly attacking other aspects of a development >> environment. The point is that there are traps everywhere, and beautiful >> symmetries everywhere. Learn how to recognize the former, and learn to >> appreciate the latter. Learn what to do when the docs fall short (eg. drop >> to assembler to see what the language construct is really doing, reverse >> engineer a file format, write some code to "do it by hand," etc.). > Dave, how do I "drop down to assembler" with Python? Can you expand > on this? Also do you think it is still of value to learn Assembly > Language, and if so which variant, as I understand it, there is a > unique set for each architecture, although they are usually similiar. There aren't many places where it's worth your time to explicitly code in assembler these days. But that's very different from whether it's worth studying and understanding. That is VERY useful, and IMHO will always be. Having said that, I've never bothered with CPython, as CPython is two levels removed from assembler. I did with Java, so CPython probably wouldn't be any worse. CPython is compiled into byte code, and that byte code is its assembly language, for most purposes. You should get familiar with that language, either by using dis.dis, or by reverse engineering the byte code files. It can be very informative to see how a particular expression or statement gets transformed into the VM. I'd love to find a debugger that let you step into the byte-code instructions the way a good C debugger lets you switch to assembly mode. So far, I've only statically looked at (disassembled and examined) selected parts of the code to understand why certain constructs work more efficiently or just differently than others. As for "real" machine language, like the Pentium 32 bit architecture, that's useful for understanding a traditional compiled language like C. Notice that you usually have to turn off the compiler optimizations, as the optimized code gets totally convoluted, and very hard to understand. Again, you can use the shortcut provided by most compilers where you ask it for a listing of mixed C and assembly code. That's analogous to the dis.dis approach in CPython. Nowhere am I recommending that everyone should WRITE in assembler. Gain a reading knowledge of it, and it'll stand you in good stead. Step into such code with a good debugger, and see where registers are pointing, and try to figure out why. For example, in Microsoft's MSC compiler in C++ mode (32bit), the 'this' pointer of C++ is almost always in EBX register. And a call to a virtual function is done with a strange addressing mode on that register where the machine takes the register value, adds a constant offset to it, dereferences 32bits from ram at that location, and does a call to that final location. And if the class involved is virtually derived from another, there will be another indirection or two. -- DaveA From cybervigilante at gmail.com Wed Jul 17 20:11:08 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Wed, 17 Jul 2013 11:11:08 -0700 Subject: [Tutor] Best way to learn Tkinter In-Reply-To: References: <9D8E4E9E-78ED-4D71-B432-378D0412C19C@gmail.com> Message-ID: On 17 July 2013 04:35, Wayne Werner wrote: > I find it quite fantastic, personally, and use it on almost all my projects > anymore. I even ported it to run on Python for Android and can run Flask > apps on my phone. How awesome is that??? And it's an easy pip install which, ahem, doesn't fail, unlike some I've tried: pip install Flask I like easy ;') Except where do I get the time to learn all these kewl packages? Sleep, I have to stop sleeping. And of course, if you eagerly run the demo without reading and start cursing, since it appears to hang, it's really in your web browser at the Flask port: http://127.0.0.1:5000/ (So sue me, I thought it would load the browser ;') Jim "Yea, I say unto you, listen not to fat gasbags, for the poor, too, are deserving of a good life. When they bring unto you their Government Cheese, do thou render unto them a trade in beer and cigarettes." --Jesus the Christ From cybervigilante at gmail.com Wed Jul 17 20:21:30 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Wed, 17 Jul 2013 11:21:30 -0700 Subject: [Tutor] How to write on Planet Python In-Reply-To: References: Message-ID: On 17 July 2013 04:12, Karan Goel wrote: > How do I publish a post on Planet Python (http://planet.python.org/). I > cannot find any information for contributors. Excellent resource. I was wondering how to get started with python(x,y) but the front page has a huge list of relevant tutorials. As for posting, I would guess you must subscribe: To request addition or removal: e-mail planet at python.org (note, responses can take up to a few days) -- Jim "Yea, I say unto you, listen not to fat gasbags, for the poor, too, are deserving of a good life. When they bring unto you their Government Cheese, do thou render unto them a trade in beer and cigarettes." --Jesus the Christ From cybervigilante at gmail.com Wed Jul 17 20:52:13 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Wed, 17 Jul 2013 11:52:13 -0700 Subject: [Tutor] Deleted dictionary length not reporting In-Reply-To: References: <51E5FBC8.7050801@pearwood.info> <51E612B0.6080706@pearwood.info> Message-ID: On 17 July 2013 03:01, eryksun wrote: > del some_list[:] # DELETE_SUBSCR > some_list[:] = [] # STORE_SUBSCR > some_list.clear() # CALL_FUNCTION And the obvious gotcha for del, since I am great at tripping over them, is: >>> l = [1,2,3] >>> del l[:] >>> l [] >>> del l >>> l Traceback (most recent call last): -- Jim "Yea, I say unto you, listen not to fat gasbags, for the poor, too, are deserving of a good life. When they bring unto you their Government Cheese, do thou render unto them a trade in beer and cigarettes." --Jesus the Christ From cybervigilante at gmail.com Wed Jul 17 21:19:38 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Wed, 17 Jul 2013 12:19:38 -0700 Subject: [Tutor] What all technologies should a python developer know? In-Reply-To: References: <51E68762.2000809@timgolden.me.uk> Message-ID: On 17 July 2013 05:42, Dave Angel wrote: > I'm also amazed at how many people launching into programming (usually with > degrees in Computer Science) still don't have a good knowledge of bits, > bytes, concurrency, protocols, namespaces, recursion, caching, memory > management, algorithms, flow control, indirection, virtual functions, > dispatch tables, syntax trees, ...) I'd have gone nuts already if I hadn't used A86 assembler thirty years ago on Fidonet, so I know all that arcane, exotic, and scary-sounding stuff is just addresses in memory and ways to access them ;') Jim From alan.gauld at btinternet.com Wed Jul 17 22:00:59 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 17 Jul 2013 21:00:59 +0100 Subject: [Tutor] What all technologies should a python developer know? In-Reply-To: References: <51E68762.2000809@timgolden.me.uk> Message-ID: On 17/07/13 16:09, wolfrage8765 at gmail.com wrote: > I am not so suprised as I used to be. Since I have seen that schools > do not teach like they should, instead they skip the fundamentals and > head straight to teaching a language It gets worse. We used to have a local university that taught a very good software engineering course. They started teaching the students how to boot their old DEC mini computer using the hardware switches on front then progressed to machine code and assembler then C and Pascal before heading into SQL, Lisp and Prolog. Some of our best hires came from there. Nowadays they start with Scratch and move to Smalltalk before introducing Java and then C++. It's the reverse of the old approach and much less detailed. Are the students any better? No, the opposite. But apparently the old course was too difficult and students were going elsewhere - market forces, and the student is king... > Lisp or Erlang is probably my next language to learn, as I wish to > better understand Functional Programming. To be honest i wouldn't use Lisp to learn FP. Erlanfg or Haskell would be my recommendation. Lisp has too many other features and its not always clear which bits are FP related and which aren't. Lisp is a great FP language if you undertands FP and know how to apply Lisp to it. But not so great for learning the new paradigm. Just my opinion of course others may disagree. > Dave, how do I "drop down to assembler" with Python? Can you expand > on this? Its really the disassembly listing Dave is talking about I think. Look at several of Eryksun's recent post for examples of how the disassembly can show what the compiler is doing. > Also do you think it is still of value to learn Assembly > Language, and if so which variant, as I understand it, there is a > unique set for each architecture, although they are usually similiar. Yes to both points. Assembler is interesting in its concepts rather than its immediate use - unless you plan on writing device drivers. There are some emulators on the net you can use(*) - I'd recommend a simple architecture like the Z80 or 6502. Try to ensure the assembler has macro support because using macros is another useful technique that originated in assemblers but can be used (or faked) in higher level languages too. (*)At least for writing. I still occasionally drop into the assembler listing when using C/C++ or Pascal(Delphi). But strictly in read-only mode. I haven't written any assembler in over 10 years... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From cybervigilante at gmail.com Wed Jul 17 23:09:55 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Wed, 17 Jul 2013 14:09:55 -0700 Subject: [Tutor] What all technologies should a python developer know? In-Reply-To: References: <51E68762.2000809@timgolden.me.uk> Message-ID: On 17 July 2013 13:00, Alan Gauld wrote: >> Also do you think it is still of value to learn Assembly >> Language, and if so which variant, as I understand it, there is a >> unique set for each architecture, although they are usually similiar. > A86 assembler, for Intel x86, has a wonderful macro language. You can practically write a higher-level language with it, but I don't know if it supports windows beyond XP, since it was only updated that far. Also, the D86 dynamic disassembler lets you run through active code. But you'd have to write the author to see if it works on later stuff than XP-based machines. I haven't used it in thirty years. www.eji.com -- Jim "Yea, I say unto you, listen not to fat gasbags, for the poor, too, are deserving of a good life. When they bring unto you their Government Cheese, do thou render unto them a trade in beer and cigarettes." --Jesus the Christ From alanwilter at gmail.com Wed Jul 10 13:04:59 2013 From: alanwilter at gmail.com (Alan) Date: Wed, 10 Jul 2013 12:04:59 +0100 Subject: [Tutor] Help installing Python3 in a particular folder Message-ID: Hi there, I am doing this: cd /temp wget -c http://www.python.org/ftp/python/3.3.2/Python-3.3.2.tar.xz tar xvf Python-3.3.2.tar.xz cd Python-3.3.2 ./configure --prefix=/sw/arch make make test make install Then, when I try: /sw/arch/bin/python3 Traceback (most recent call last): File "/sw/arch/lib/python3.3/site.py", line 69, in import os File "/sw/arch/lib/python3.3/os.py", line 659, in from collections.abc import MutableMapping File "/sw/arch/lib/python3.3/collections/__init__.py", line 12, in from keyword import iskeyword as _iskeyword ImportError: No module named 'keyword' But if I do: export PYTHONPATH=/temp/Python-3.3.2/Lib then python3 will work. However, this does seems correct for me. I shouldn't depend on the installation folder (which I want to delete btw). I compared /sw/arch/lib/python3.3/ X /temp/Python-3.3.2/Lib and found this files missing: Only in /temp/Python-3.3.2/Lib: base64.py Only in /temp/Python-3.3.2/Lib: cgi.py Only in /temp/Python-3.3.2/Lib: cProfile.py Only in /temp/Python-3.3.2/Lib: keyword.py Only in /temp/Python-3.3.2/Lib: msilib Only in /temp/Python-3.3.2/Lib: pdb.py Only in /temp/Python-3.3.2/Lib: plat-aix4 Only in /temp/Python-3.3.2/Lib: plat-darwin Only in /temp/Python-3.3.2/Lib: platform.py Only in /temp/Python-3.3.2/Lib: plat-freebsd4 Only in /temp/Python-3.3.2/Lib: plat-freebsd5 Only in /temp/Python-3.3.2/Lib: plat-freebsd6 Only in /temp/Python-3.3.2/Lib: plat-freebsd7 Only in /temp/Python-3.3.2/Lib: plat-freebsd8 Only in /temp/Python-3.3.2/Lib: plat-generic Only in /temp/Python-3.3.2/Lib: plat-netbsd1 Only in /temp/Python-3.3.2/Lib: plat-next3 Only in /temp/Python-3.3.2/Lib: plat-os2emx Only in /temp/Python-3.3.2/Lib: plat-sunos5 Only in /temp/Python-3.3.2/Lib: plat-unixware7 Only in /temp/Python-3.3.2/Lib: profile.py Only in /temp/Python-3.3.2/Lib: pydoc.py Only in /temp/Python-3.3.2/Lib: quopri.py Only in /temp/Python-3.3.2/Lib: smtpd.py Only in /temp/Python-3.3.2/Lib: symbol.py Only in /temp/Python-3.3.2/Lib: tabnanny.py Only in /temp/Python-3.3.2/Lib: token.py Only in /temp/Python-3.3.2/Lib: uu.py During "make install" I noticed this: /sw/arch/lib/python3.3 already exist, returning without doing anything arguments where -c ./Lib/keyword.py /sw/arch/lib/python3.3 /sw/arch/bin/install -c ./Lib/keyword.py /sw/arch/lib/python3.3 So, apparently keyword.py should have been copied into /sw/arch/lib/python3.3 but it's really not happening. What am I doing wrong here? Thanks, Alan -- Alan Wilter SOUSA da SILVA, DSc Bioinformatician, UniProt - PANDA, EMBL-EBI CB10 1SD, Hinxton, Cambridge, UK +44 1223 49 4588 -------------- next part -------------- An HTML attachment was scrubbed... URL: From lu.nemec at gmail.com Wed Jul 10 15:35:38 2013 From: lu.nemec at gmail.com (Lukas Nemec) Date: Wed, 10 Jul 2013 15:35:38 +0200 Subject: [Tutor] syntax error when attempting simple urllib.request.urlopen item In-Reply-To: References: Message-ID: <51DD632A.6080206@gmail.com> > On 07/09/2013 04:00 PM, Paul Smith wrote: >> Tutor- >> >> Ok newbie to coding here attempting to build controlled web scraper and >> have followed several books-tutorials and am failing at step one. >> >> This is the 3.3.1 code I am trying to run.. >> === >> import urllib.request >> >> htmltext = urllib.request.urlopen("http://google.com").read >> >> print htmltext >> === >> Other version... >> === >> #try py url open 7 >> >> import urllib.request >> res = urllib.request.urlopen('http://python.org/') >> html = res.read() >> print = html >> close res >> input = ("Press enter to exit") >> === >> >> so when I run what I think is proper 3.3.1 python code it hangs up with >> syntax error with the idle shell red highlighting the last print >> reference >> i.e. "htmltext" or "html". >> >> What is this humble newbie not getting? > In the second example you have print = html .. you can't do that, print is a build in function, so try print(html) you should consider using some IDE that will alert you to these simple mistakes you can try NinjaIDE, Eclipse (PyDev), or Vim with pylint plugin Lukas From andrew.vanvalkenburg at gmail.com Thu Jul 11 00:08:23 2013 From: andrew.vanvalkenburg at gmail.com (Andrew Van Valkenburg) Date: Wed, 10 Jul 2013 18:08:23 -0400 Subject: [Tutor] Problems understanding code output In-Reply-To: <24816197.4240951373490347622.JavaMail.ska@luo.to> References: <24816197.4240951373490347622.JavaMail.ska@luo.to> Message-ID: I'm not really sure what you are asking, but the formatting for your code is pretty screwy. I reformatted it and changed the print statements slightly to make it more readable and it works fine from what I can see. def printMax(a, b): if a > b: print a, 'is maximum' elif a == b: print a, 'is equal to', b else: print b, 'is maximum' printMax(3,4) On Wed, Jul 10, 2013 at 5:05 PM, wrote: > def printMax(a, b): if a > b: print(a, 'is maximum') elif a == b: print(a, > 'is equal to', b) else: print(b, 'is maximum') printMax(3, 4) # directly > give literal values x = 5 y = 7 printMax(x, y) # give variables as > arguments How the code above values to: 4 is maximum 7 is maximum and not > to: 5 is maximum 7 is maximum This is going a little over my head, please > advice, what am I missing in here? > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From datar at wisc.edu Thu Jul 11 00:12:16 2013 From: datar at wisc.edu (Makarand Datar) Date: Wed, 10 Jul 2013 17:12:16 -0500 Subject: [Tutor] Interpret the contents of a line Message-ID: Hi, I am working on a parser and my input file is a description about bunch of things written line by line. So for instance, consider the following two lines and what the corresponding variables in python should look like. example Line1: position = 1, 1, rotation = 90, 0, 0, mass = 120; Here I want the variables to be position = [1,1,0], rotation = [90,0,0], mass = [120] example Line2: position = 1, 1, 2, mass = 120, rotation = 90, 0; Here I want the variables to be position = [1,1,2], rotation = [90,0,0], mass = [120] example Line3: mass = 120, rotation = 90, 0; Here I want the variables to be position = [0,0,0], rotation = [90,0,0], mass = [120] I know the maximum number of arguments possible for each variable. For example, in the first line above, only two numbers for position ares specified; that means that the third entry in position list is zero. So I need to handle these cases while reading in the file as well. Additionally, the text might not always be in the same order; like shown in the second line above. And finally, sometimes numbers for one variable might not exist at all; like shown in line 3. Here, the position is then read as position = [0,0,0]. How do I implement such stuff? Is there some smart way of doing this? All I have in mind is this: read a line as comma or a space delimited list of words variable and then some how use if else + len(list) to figure out whats going on. But that seems way too tedious and I feel that there might be an easier way to read such things. Any help is highly appreciated. Thank you -------------- next part -------------- An HTML attachment was scrubbed... URL: From shireenrao at gmail.com Thu Jul 11 14:37:33 2013 From: shireenrao at gmail.com (Srinivas Nyayapati) Date: Thu, 11 Jul 2013 08:37:33 -0400 Subject: [Tutor] Best way to setup Unit Testing? In-Reply-To: References: Message-ID: <1431526120876682177@unknownmsgid> On Jul 10, 2013, at 6:48 AM, Oscar Benjamin wrote: > On 7 July 2013 21:16, Srinivas Nyayapati wrote: >> I am tryng to figure out the best way to do Unit Testing for all my projects >> going forward. > > Are you the only one using these applications or are you intending > that other people would install them? Do you want other people to run > the tests on their machines when they install? Will other people also > work on the development of these projects and will they also need to > run the tests? So far I am the only one. But in case I wanted to share my projects (I hope!) I wanted to make sure the other party can run tests easily. >> I am using the unittest framework. For execution I am >> executing the test cases by either directly using python or by using the >> nose test runner. I found the following 3 types of how this can be setup. >> >> Type 1 >> ====== >> >> The unit tests are located in the same package as the module being tested. >> This is the easiest to implement and run. I don't have to do anything >> special to run the tests. The directory structure is like this >> >> Projects/ >> MyApp/ >> __init__.py >> myapp.py >> test_myapp.py > > The (possible) advantage of this setup is that a user can run tests on > the application while it is installed using: > > $ python -m MyApp.test_myapp > > The above could get messy if the number of submodules increases though. I agree and will not be doing this. >> Simply run the test from within the MyApp folder as >> >> % python test_myapp.py >> or use the nose test runner >> >> % nosetests >> >> Type 2 >> ====== >> >> Here the unit tests are located in its own tests folder in the package you >> want to test. Running the unit tests from inside the tests folder won't work >> as it worked in Type 1. That's because the myapp module can not be resolved. >> Here is how the directory structure looks like >> >> Projects/ >> MyApp/ >> __init__.py >> myapp.py >> tests/ >> __init__.py >> test_myapp.py > > This gives a nice clean distinction between your main code and your > tests. It is also useful if you don't want to install the tests on the > target machines since you can just exclude the tests package in your > setup.py when creating an sdist. > > If you do include the tests and add a __main__.py in the tests folder > then users can still run tests with > > python -m MyApp.tests Love this idea! Would you know how I would run the test_*.py scripts from __main__.py >> It will work if you ran it from inside the package folder and gave the >> complete path to the test file as >> >> % python tests/test_myapp.py >> It will also work if you used the nose test runner from inside MyApp. >> >> % nosetests >> It will also work from inside the tests folder if you added MyApp to your >> PYTHONPATH, but that is an option I dont like. I don't want to keep changing >> my PYTHONPATH. > > You can always just do that in a script. I would normally have a > Makefile and invoke tests with > > $ make test > > Inside the Makefile you can do > > test: > PYTHONPATH=src nosetest > > Then you don't need to fiddle around with PYTHONPATH manually. My > preference is that 'make test' should be run from the top-level > directory of the project (if you're using version control then that's > the main folder that gets created in a checkout/clone). Someone with > the code should be able to do > > $ cd SomeProject > $ make test > ........ 121 out of 121 test passed. Another really neat idea! I am going to implement make files to do my tests.. Came across an issue while implementing this. The problem was my test target would not run as it said it was upto date. Adding the following took care of that. .PHONY: tests >> >> Type 3 >> ====== >> >> This is where the tests folder is on the same level as the package folder. >> You can not run the test scripts from within the tests folder as again it >> will not be able to resolve the myapp package. Here is how the directory >> structure looks like >> >> Projects/ >> MyApp/ >> __init__.py >> myapp.py >> tests/ >> __init__.py >> test_myapp.py > > In this setup you are not intending for the tests to be installed on > the target machines and you have a clear separation between your main > code and your tests. I get the difference between the different styles now. >> >> You can run the tests from within the MyApp package by referencing the tests >> folder as >> >> % python ../tests/test_myapp.py >> >> As you are running the test from the package, the myapp module is >> automatically in your PYTHONPATH. The other way of doing this would be by >> putting your package in your PYTHONPATH, which again as I mentioned earlier >> I don't like to do. >> >> What is the best way of doing this? I would really appreciate to know how >> the experts are doing this. Is there anything I need to consider before I >> start using test suites. > > Whatever you do just use a script or a Makefile or something so that > invoking the tests is a simple one-liner e.g. > > $ make test > $ python runtests.py > $ ./test.sh > > or whatever suits you. If you need to fiddle with PYTHONPATH do it in > that script, rather than manually. Thank you for your detailed answer. Learnt something new today. Srini From luka.radosh at gmail.com Tue Jul 16 14:31:00 2013 From: luka.radosh at gmail.com (=?UTF-8?Q?Luka_Rado=C5=A1?=) Date: Tue, 16 Jul 2013 14:31:00 +0200 Subject: [Tutor] Python noob In-Reply-To: References: Message-ID: Hi :) I am a programming noob, but i dont consider myself as complete noob cause i did some C programming and i do have basic knowledge how programming works. Problem is that i did C programming 4 years ago when i was in high school. Now i study computer science and i was thinking to get some programming experience before i leave college. I live in screwed up country, named Croatia and i cant get any job/experience/quick start unless i am guru at it. And i am not a guru. I do have interest in learning Python and i got myself a book "programming in python 4th edition" and i plan to learn from it during summer holidays. My question is, can you recommend me some learning materials? Or any kind of advice for me? -------------- next part -------------- An HTML attachment was scrubbed... URL: From dwightdhutto at gmail.com Wed Jul 17 01:39:41 2013 From: dwightdhutto at gmail.com (David Hutto) Date: Tue, 16 Jul 2013 19:39:41 -0400 Subject: [Tutor] A list of 100+ projects to complete to master Python. In-Reply-To: References: Message-ID: First thing you should learn is offsite backups, I've lost several projects in the works because of a hd mishap. Secondarily, I would recommend using existing primarily used python projects to 'reinvent the wheel' so to speak. Thirdly, make sure the code is properly documented, and serves a purpose. And lastly, utilize other programs with python api's, such as blender, which can make your apps pop, or diversify into game development kits. On Tue, Jul 16, 2013 at 1:09 PM, Karan Goel wrote: > Hey guys and gals > > Just a plug here. My repo: https://github.com/thekarangoel/Projects > was one of the trending repos on Gh this week and I thought folks > on this list might be interested in knowing about it. > > In a nutshell, I'm trying to complete over a 100 practical language- > agnostic projects in Python only. I haven't read the project details yet, > and I'm not filtering things out. > > If you would like to do the same, join me. Fork or star the repo, > and start coding (in any language, really). > > https://github.com/thekarangoel/Projects > > Let me know if you have any questions or suggestions. > > - Karan Goel > Goel.im | Resume > | Github > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > > -- Best Regards, David Hutto *CEO:* *http://www.hitwebdevelopment.com* -------------- next part -------------- An HTML attachment was scrubbed... URL: From JATINSHR001 at e.ntu.edu.sg Mon Jul 15 10:53:27 2013 From: JATINSHR001 at e.ntu.edu.sg (#PATHANGI JANARDHANAN JATINSHRAVAN#) Date: Mon, 15 Jul 2013 08:53:27 +0000 Subject: [Tutor] Cannot understand what this means Message-ID: <52B2907AE37EB94B8690A0B45EE20918C93A8C@HKNPRD0111MB386.apcprd01.prod.exchangelabs.com> Hello All, I have just started out with python and was feeling pretty comfortable and confident to the point of solving problems from websites. But now, there is a program from Google's Python Exercises whose main() part is already given to you. def main(): # This command-line parsing code is provided. # Make a list of command line arguments, omitting the [0] element # which is the script itself. args = sys.argv[1:] if not args: print 'usage: [--summaryfile] file [file ...]' sys.exit(1) # Notice the summary flag and remove it from args if it is present. summary = False if args[0] == '--summaryfile': summary = True del args[0] The problem is one where you have to define a function to read a html file containing the 1000 most popular baby names of a particular year(separate files for different years) and print each name in alphabetical order with their corresponding rank for that year. My problem is that I cannot understand anything in this main() module. Nothing at all. So can somebody please explain what this means and what is its significance? Thank You -------------- next part -------------- An HTML attachment was scrubbed... URL: From lseibold at seiboldsystems.com Wed Jul 10 18:10:53 2013 From: lseibold at seiboldsystems.com (larry seibold) Date: Wed, 10 Jul 2013 09:10:53 -0700 Subject: [Tutor] install on windows In-Reply-To: References: <51D37F34.9020609@seiboldsystems.com> Message-ID: <51DD878D.7080902@seiboldsystems.com> On 7/10/2013 8:16 AM, Chris ?Kwpolska? Warrick wrote: > On Wed, Jul 3, 2013 at 3:32 AM, larry seibold > wrote: > >> I am stuck at step one, installing "python-2.7.5.msi" on windows XP. I >> downloaded it (~16MB), but when I select it (double click), I get a windows >> installer pop up with an OK button at the bottom, which when selected seems >> to terminate the install vs. install the program. I do not see any advice >> on a different procedure. Help. Do I need to run the msi from a command >> window with options? >> > What does the pop-up say? Are you using a limited account? > > Windows Installer window contents, followed by OK button but no install takes place: ? Windows ? Installer. V 3.01.4001.5512 msiexec /Option [Optional Parameter] Install Options Installs or configures a product /a Administrative install - Installs a product on the network /j [/t ] [/g ] Advertises a product - m to all users, u to current user Uninstalls the product Display Options /quiet Quiet mode, no user interaction /passive Unattended mode - progress bar only /q[n|b|r|f] Sets user interface level n - No UI b - Basic UI r - Reduced UI f - Full UI (default) /help Help information Restart Options /norestart Do not restart after the installation is complete /promptrestart Prompts the user for restart if necessary /forcerestart Always restart the computer after installation Logging Options /l[i|w|e|a|r|u|c|m|o|p|v|x|+|!|*] i - Status messages w - Nonfatal warnings e - All error messages a - Start up of actions r - Action-specific records u - User requests c - Initial UI parameters m - Out-of-memory or fatal exit information o - Out-of-disk-space messages p - Terminal properties v - Verbose output x - Extra debugging information + - Append to existing log file ! - Flush each line to the log * - Log all information, except for v and x options /log Equivalent of /l* Update Options /update [;Update2.msp] Applies update(s) /uninstall [;Update2.msp] /package Remove update(s) for a product Repair Options /f[p|e|c|m|s|o|d|a|u|v] Repairs a product p - only if file is missing o - if file is missing or an older version is installed (default) e - if file is missing or an equal or older version is installed d - if file is missing or a different version is installed c - if file is missing or checksum does not match the calculated value a - forces all files to be reinstalled u - all required user-specific registry entries (default) m - all required computer-specific registry entries (default) s - all existing shortcuts (default) v - runs from source and recaches local package Setting Public Properties [PROPERTY=PropertyValue] Consult the Windows ? Installer SDK for additional documentation on the command line syntax. Copyright ? Microsoft Corporation. All rights reserved. Portions of this software are based in part on the work of the Independent JPEG Group. ? I continued to work this issue from last week. I solved the issue with a bit of help from a colleague. I am surprised that neither the install instructions in the Python site nor in the "Hard Way" book adresses this, as it must be the case with almost everyone, with similar issues with later windows releases. Here is what I found out. Even though I had administrator rights (my user was in the administrator group) on the windows XP machine, I was not the "Administrator" user. This causes some program installs to have problems (I wonder what would happen if an attempt was made as the Administrator user that was not in the administrators group?). Microsoft addresses this by using the "runas" feature, normally a right mouse selection when selecting the install program. In the case of an MSI (**.msi), this is not an option. What I chose to try that worked, was to start a command shell with the "runas administrator" feature. From there I changed to the directory where the python msi was downloaded, and then ran the install program. from Start --> Search programs and files: runas /user:domain/adminusername "msiexec /i msiname.msi" it will prompt for admin password -or- Right click on Command Prompt, Run as administrator msiexec /i "YouMsi.msi" I hope this helps others. I hope someone adds a note to the windows install instructions. -Larry -------------- next part -------------- An HTML attachment was scrubbed... URL: From paulrsmith7777 at gmail.com Mon Jul 15 22:27:41 2013 From: paulrsmith7777 at gmail.com (Paul Smith) Date: Mon, 15 Jul 2013 16:27:41 -0400 Subject: [Tutor] Timestamp issues when importing from FireFox sqlite field Message-ID: All- Newb here so apologies upfront. Attempting timestamp translation via python importing info from sqlite datetime item in firefox ,'1369751000393000'. Typical unix date time translation fails. I noted micro second addition but even '1369751000393000' / 1e6 does not get a chew-able range, at least in python datetime module. Any suggestions? Best regards, Paul Failing code examples: >>> import datetime >>> print datetime.datetime.fromtimestamp(int("1369751000393000")).strftime('%Y-%m-%d %H:%M:%S') Traceback (most recent call last): File "", line 1, in print datetime.datetime.fromtimestamp(int("1369751000393000")).strftime('%Y-%m-%d %H:%M:%S') ValueError: timestamp out of range for platform localtime()/gmtime() function microseconds /1e6? >>> import datetime >>> print datetime.datetime.fromtimestamp(int("1369751000393000/1e6")).strftime('%Y-%m-%d %H:%M:%S') Traceback (most recent call last): File "", line 1, in print datetime.datetime.fromtimestamp(int("1369751000393000/1e6")).strftime('%Y-%m-%d %H:%M:%S') ValueError: invalid literal for int() with base 10: '1369751000393000/1e6' -------------- next part -------------- An HTML attachment was scrubbed... URL: From robertvstepp at gmail.com Sun Jul 14 03:12:20 2013 From: robertvstepp at gmail.com (boB Stepp) Date: Sat, 13 Jul 2013 20:12:20 -0500 Subject: [Tutor] What every computer science major should know Message-ID: This may be of interest to some of us: http://matt.might.net/articles/what-cs-majors-should-know/ Perhaps it will generate some interesting discussion? boB From sakroger at gmail.com Tue Jul 16 16:15:44 2013 From: sakroger at gmail.com (Samuel Kroger) Date: Tue, 16 Jul 2013 09:15:44 -0500 Subject: [Tutor] New to programing and the tutoring system Message-ID: Hello I am very very new to programing entirely, I have this book I am following, and I made a word jumbling game, it chooses a word from random from a turple I made, then scrambles the word and asks you to guess it, it is very simple but for some reason it only works when it's in idle. Here is the script. import random WORDS=("python","jumble","easy","difficult","answer","xylophone") word=random.choice(WORDS) correct=word jumble="" while word: position=random.randrange(len(word)) jumble+=word[position] word=word[:position]+word[(position+1):] print(""" Welcome to Word Jumble! Unscramble the letters to make a word. (Press the enter key at the prompt to quit.) """ ) print("The jumble is:",jumble) # Working as intened at this point guess=input("\nYour guess: ") while guess != correct and guess !="": print("Sorry, that's not it.") guess=input("Your guess: ") if guess==correct: print("That's it! You guessed it!\n") print("Thanks for playing.") input("\n\nPress the enter key to exit.") Also I am using the most recently updated version of python33. Thank you -------------- next part -------------- An HTML attachment was scrubbed... URL: From vick1975 at orange.mu Fri Jul 12 12:08:54 2013 From: vick1975 at orange.mu (Vick) Date: Fri, 12 Jul 2013 14:08:54 +0400 Subject: [Tutor] hi Message-ID: <000001ce7ee7$d2233170$76699450$@orange.mu> Hi, I'm using Windows 7 and Python 2.7.3 I have written a code to perform a numerical solution to 1st order Ordinary Differential Equations (ODEs). I have written codes for the famous Runge Kutta 4th order and the DOPRI 8(7)13. However the codes are for 1 variable only; viz. the "y" variable. I have written another code in Excel VBA for DOPRI 8 for a multi-variable capability. However I have not been able to reproduce it in Python. I'm having trouble in making arrays or lists, I don't know which is supposed to work. I have attached my Excel VBA code for a multi-variable numerical integrator in PDF format. This does work in Excel. I have also attached my python code. Can anyone help me out please? Thanks Vick -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: multi-varia dopri.pdf Type: application/pdf Size: 110780 bytes Desc: not available URL: -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: dopri multi var.py URL: From lseibold at seiboldsystems.com Thu Jul 11 07:46:26 2013 From: lseibold at seiboldsystems.com (larry seibold) Date: Wed, 10 Jul 2013 22:46:26 -0700 Subject: [Tutor] install on windows In-Reply-To: References: Message-ID: <51DE46B2.40207@seiboldsystems.com> On 7/10/2013 7:34 PM, tutor-request at python.org wrote: > From: David Robinow Subject: Re: [Tutor] install on windows On Wed, > Jul 10, 2013 at 1:16 PM, Alan Gauld wrote: >> > On 03/07/13 02:32, larry seibold wrote: >> >>> >> >>> >> I am stuck at step one, installing "python-2.7.5.msi" on windows XP. >>> >> > >> > >> > Are you sure you have the right version - 32bit v 64bit? >> > >> > Just a thought. >> > "python-2.7.5.msi" is the 32-bit version. That's always a "right version". > > Administrator rights should not be required to install "just for me". > The problem is elsewhere. > I am not sure what the problem is, but I responded to this thread in detail more than 12 hours ago, but have yet to see that response in the forum digest. Administrator rights and Adminstrator user were both required to install on windows XP. -Larry From alan.gauld at btinternet.com Thu Jul 18 19:17:19 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Thu, 18 Jul 2013 18:17:19 +0100 Subject: [Tutor] install on windows In-Reply-To: <51DD878D.7080902@seiboldsystems.com> References: <51D37F34.9020609@seiboldsystems.com> <51DD878D.7080902@seiboldsystems.com> Message-ID: On 10/07/13 17:10, larry seibold wrote: > this, as it must be the case with almost everyone, with similar issues > with later windows releases. Here is what I found out. > > Even though I had administrator rights (my user was in the administrator > group) on the windows XP machine, I was not the "Administrator" user. I've installed Python on many Windows boxes and never had to be Adminisatrator nor even in the Admin group. (I am usually a PowerUser though.) > What I chose to try that worked, was to start a command shell with the > "runas administrator" feature. From there I changed to the directory > where the python msi was downloaded, and then ran the install program. > > from Start --> Search programs and files: > > runas /user:domain/adminusername "msiexec /i msiname.msi" > And I've never had to do anything but double click the msi file and the installer has run just fine. All very odd. Glad you got it installed though. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From alan.gauld at btinternet.com Thu Jul 18 19:24:59 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Thu, 18 Jul 2013 18:24:59 +0100 Subject: [Tutor] Cannot understand what this means In-Reply-To: <52B2907AE37EB94B8690A0B45EE20918C93A8C@HKNPRD0111MB386.apcprd01.prod.exchangelabs.com> References: <52B2907AE37EB94B8690A0B45EE20918C93A8C@HKNPRD0111MB386.apcprd01.prod.exchangelabs.com> Message-ID: On 15/07/13 09:53, #PATHANGI JANARDHANAN JATINSHRAVAN# wrote: > websites. But now, there is a program from Google's Python Exercises > whose main() part is already given to you. > > > def main(): > # This command-line parsing code is provided. > # Make a list of command line arguments, omitting the [0] element > # which is the script itself. > args = sys.argv[1:] > > if not args: > print 'usage: [--summaryfile] file [file ...]' > sys.exit(1) > > # Notice the summary flag and remove it from args if it is present. > summary = False > if args[0] == '--summaryfile': > summary = True > del args[0] > My problem is that I cannot understand anything in this main() module. > Nothing at all. It sounds like you need a more basic tutorial then. Do you really not know what def main(): means? Hint: its a function not a module... And do you know what "command line arguments" are? Or what sys.argv represents? Do you understand sys.exit()? I assume 'print' is self explanatory? Does the 'if' test mean anything? If you really don't understand any of those things you need to find a more basic tutorial and work through it. (You might try the first part of mine, see below, as an example) If you do understand those constructs, then you understand quite a lot of main and need to ask a more specific question. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From alan.gauld at btinternet.com Thu Jul 18 19:27:09 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Thu, 18 Jul 2013 18:27:09 +0100 Subject: [Tutor] install on windows In-Reply-To: <51DE46B2.40207@seiboldsystems.com> References: <51DE46B2.40207@seiboldsystems.com> Message-ID: On 11/07/13 06:46, larry seibold wrote: > I am not sure what the problem is, but I responded to this thread in > detail more than 12 hours ago, but have yet to see that response in the > forum digest. > > Administrator rights and Adminstrator user were both required to install > on windows XP. The problem of the delay was that it got caught in the moderation queue which I only clear out once or twice a week at most (its normally low volume apart from spam). Possibly you posted from a different address to the one you registered? -- Alan G Moderator. From ilhs_hs at yahoo.com Thu Jul 18 19:27:45 2013 From: ilhs_hs at yahoo.com (Hs Hs) Date: Thu, 18 Jul 2013 10:27:45 -0700 (PDT) Subject: [Tutor] Selecting from list In-Reply-To: References: Message-ID: <1374168465.55644.YahooMailNeo@web163503.mail.gq1.yahoo.com> hi list: In the following list, is there a simply way to find element less than 200 sandwiched between two numbers greater than 1000. a = [3389, 178, 2674, 2586, 13731, 3189, 785, 1038, 25956, 33551] in a, 178 is between 3389 and 2674. ?How this particular list can be selected for further processing.? (sorry this is not homework question. I want to avoid looping, because I have 300K lines to parse through) Thanks Hs. -------------- next part -------------- An HTML attachment was scrubbed... URL: From joel.goldstick at gmail.com Thu Jul 18 19:38:01 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Thu, 18 Jul 2013 13:38:01 -0400 Subject: [Tutor] hi In-Reply-To: <000001ce7ee7$d2233170$76699450$@orange.mu> References: <000001ce7ee7$d2233170$76699450$@orange.mu> Message-ID: On Fri, Jul 12, 2013 at 6:08 AM, Vick wrote: > Hi,**** > > ** ** > > I?m using Windows 7 and Python 2.7.3**** > > ** ** > > I have written a code to perform a numerical solution to 1st order > Ordinary Differential Equations (ODEs). I have written codes for the famous > Runge Kutta 4th order and the DOPRI 8(7)13. However the codes are for 1 > variable only; viz. the ?y? variable.**** > > ** ** > > I have written another code in Excel VBA for DOPRI 8 for a multi-variable > capability. However I have not been able to reproduce it in Python. I?m > having trouble in making arrays or lists, I don?t know which is supposed to > work.**** > > ** ** > > I have attached my Excel VBA code for a multi-variable numerical > integrator in PDF format. This does work in Excel. I have also attached my > python code.**** > > ** ** > > Can anyone help me out please? > So are you saying your python code doesn't run, or it runs but doesn't give the same answers as your vb code? If your python code throws an exception you should list the traceback for better help. Also, try using a better subject line next time. > **** > > Thanks**** > > Vick**** > > ** ** > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > > -- Joel Goldstick http://joelgoldstick.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan.gauld at btinternet.com Thu Jul 18 19:44:12 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Thu, 18 Jul 2013 18:44:12 +0100 Subject: [Tutor] Python noob In-Reply-To: References: Message-ID: On 16/07/13 13:31, Luka Rado? wrote: > My question is, can you recommend me some learning materials? Or any > kind of advice for me? Visit the Python.org website. It has lists of online tutorials you can use. Find one you like and make a start. If you get stuck ask questions here. Let us know what OS you use, which Python version. And provide full printout of any error messages not just a summary. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From alan.gauld at btinternet.com Thu Jul 18 19:46:39 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Thu, 18 Jul 2013 18:46:39 +0100 Subject: [Tutor] New to programing and the tutoring system In-Reply-To: References: Message-ID: On 16/07/13 15:15, Samuel Kroger wrote: > guess it, it is very simple but for some reason it only works when it's > in idle. What doesn't work? Do you get any output? An error message? How are you running it outside IDLE? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From wolfrage8765 at gmail.com Thu Jul 18 19:55:41 2013 From: wolfrage8765 at gmail.com (wolfrage8765 at gmail.com) Date: Thu, 18 Jul 2013 13:55:41 -0400 Subject: [Tutor] Python noob In-Reply-To: References: Message-ID: On Thu, Jul 18, 2013 at 1:44 PM, Alan Gauld wrote: > On 16/07/13 13:31, Luka Rado? wrote: > >> My question is, can you recommend me some learning materials? Or any >> kind of advice for me? The recommendations also strongly depend on your initial field of study. There are at least two very distinct major fields of study: Applications (GUI, CLI, Other) or Web Development (Flask, Django)... even then both can overlap, but I suggest starting in one or the other and then narrowing further. Perhaps create a project, and determine it's requirements, then figure out what you will need to learn to achieve those requirements and set out to learn them. > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ ^ ^ ^ ^ ^ ^ ^ ^ A very informative tutorial created by Alan. From __peter__ at web.de Thu Jul 18 20:00:31 2013 From: __peter__ at web.de (Peter Otten) Date: Thu, 18 Jul 2013 20:00:31 +0200 Subject: [Tutor] New to programing and the tutoring system References: Message-ID: Samuel Kroger wrote: > Hello I am very very new to programing entirely, I have this book I am > following, and I made a word jumbling game, it chooses a word from random > from a turple I made, then scrambles the word and asks you to guess it, it > is very simple but for some reason it only works when it's in idle. Here > is the script. > import random [...] > input("\n\nPress the enter key to exit.") > Also I am using the most recently updated version of python33. Thank you You don't tell us the error message. If you get a NameError after entering your guess you may be a Windows user and have Python2.7 or similar installed in parallel. Try adding #! python3 as the first line of your script to inform the launcher about the Python version you wish to use. See also http://docs.python.org/3.3/using/windows.html#python-launcher-for-windows From __peter__ at web.de Thu Jul 18 20:23:47 2013 From: __peter__ at web.de (Peter Otten) Date: Thu, 18 Jul 2013 20:23:47 +0200 Subject: [Tutor] Timestamp issues when importing from FireFox sqlite field References: Message-ID: Paul Smith wrote: > Attempting timestamp translation via python importing info from sqlite > datetime item in firefox ,'1369751000393000'. Typical unix date time > translation fails. I noted micro second addition but even > '1369751000393000' / 1e6 does not get a chew-able range, at least in > python datetime module. Any suggestions? > datetime.datetime.fromtimestamp(int("1369751000393000/1e6")).strftime('%Y-%m-%d > %H:%M:%S') > ValueError: invalid literal for int() with base 10: '1369751000393000/1e6' The problem is not with the timestamp or the datetime module. You are attempting the division inside the string. Wrong: >>> int("1369751000393000/1e6") Traceback (most recent call last): File "", line 1, in ValueError: invalid literal for int() with base 10: '1369751000393000/1e6' Correct: >>> int("1369751000393000")/1e6 1369751000.393 >>> import datetime >>> datetime.datetime.fromtimestamp(int("1369751000393000")/1e6) datetime.datetime(2013, 5, 28, 16, 23, 20, 393000) >>> datetime.datetime.fromtimestamp(int("1369751000393000")/1e6).strftime("%c") 'Tue May 28 16:23:20 2013' From __peter__ at web.de Thu Jul 18 21:10:56 2013 From: __peter__ at web.de (Peter Otten) Date: Thu, 18 Jul 2013 21:10:56 +0200 Subject: [Tutor] Selecting from list References: <1374168465.55644.YahooMailNeo@web163503.mail.gq1.yahoo.com> Message-ID: Hs Hs wrote: > In the following list, is there a simply way to find element less than 200 > sandwiched between two numbers greater than 1000. > > a = [3389, 178, 2674, 2586, 13731, 3189, 785, 1038, 25956, 33551] > > > in a, 178 is between 3389 and 2674. How this particular list can be > selected for further processing. (sorry this is not homework question. I > want to avoid looping, because I have 300K lines to parse through) I'm afraid you have to use a loop. Give us your solution, and we'll see if we can improve it. As a bonus here's a numpy solution without explicit loops: >>> a = numpy.array([3389, 178, 2674, 2586, 13731, 3189, 785, 1038, 25956, 33551]) >>> g = a > 1000 >>> l = a < 200 >>> x = g[:-2] & g[2:] & l[1:-1] >>> numpy.where(x)[0]+1 array([1]) I'll withhold remarks regarding clarity and predictions regarding performance ;) From eryksun at gmail.com Thu Jul 18 21:37:06 2013 From: eryksun at gmail.com (eryksun) Date: Thu, 18 Jul 2013 15:37:06 -0400 Subject: [Tutor] install on windows In-Reply-To: <51DD878D.7080902@seiboldsystems.com> References: <51D37F34.9020609@seiboldsystems.com> <51DD878D.7080902@seiboldsystems.com> Message-ID: On Wed, Jul 10, 2013 at 12:10 PM, larry seibold wrote: > It appears your post was flagged as spam. > Windows Installer window contents, followed by OK button but no install > takes place: > > Windows ? Installer. V 3.01.4001.5512 > > msiexec /Option [Optional Parameter] It's odd that it would pop up the help message box, as if it wasn't run with the correct /i option. Maybe the file association is misconfigured. Check assoc and ftype: C:\>assoc .msi .msi=Msi.Package C:\>ftype Msi.Package Msi.Package="%SystemRoot%\System32\msiexec.exe" /i "%1" %* > Even though I had administrator rights (my user was in the administrator > group) on the windows XP machine, I was not the "Administrator" user. In XP (NT 5.1), a user in the local Administrators group has full control, including installing device drivers and installing files to %SystemRoot%\System32 such as python27.dll. http://ss64.com/nt/syntax-security_groups.html From eryksun at gmail.com Thu Jul 18 22:37:06 2013 From: eryksun at gmail.com (eryksun) Date: Thu, 18 Jul 2013 16:37:06 -0400 Subject: [Tutor] Help installing Python3 in a particular folder In-Reply-To: References: Message-ID: On Wed, Jul 10, 2013 at 7:04 AM, Alan wrote: > and found this files missing: > Only in /temp/Python-3.3.2/Lib: plat-aix4 > Only in /temp/Python-3.3.2/Lib: msilib Don't worry about the plat directories; I assume plat-linux was copied. And msilib is for Windows. > Only in /temp/Python-3.3.2/Lib: base64.py > Only in /temp/Python-3.3.2/Lib: cgi.py > Only in /temp/Python-3.3.2/Lib: cProfile.py > Only in /temp/Python-3.3.2/Lib: keyword.py > Only in /temp/Python-3.3.2/Lib: pdb.py > Only in /temp/Python-3.3.2/Lib: platform.py > Only in /temp/Python-3.3.2/Lib: profile.py > Only in /temp/Python-3.3.2/Lib: pydoc.py > Only in /temp/Python-3.3.2/Lib: quopri.py > Only in /temp/Python-3.3.2/Lib: smtpd.py > Only in /temp/Python-3.3.2/Lib: symbol.py > Only in /temp/Python-3.3.2/Lib: tabnanny.py > Only in /temp/Python-3.3.2/Lib: token.py > Only in /temp/Python-3.3.2/Lib: uu.py These are executable scripts, so check the Makefile setting for INSTALL_SCRIPT. On my system it's just "${INSTALL}", where the latter is "/usr/bin/install -c". From davea at davea.name Thu Jul 18 22:51:12 2013 From: davea at davea.name (Dave Angel) Date: Thu, 18 Jul 2013 16:51:12 -0400 Subject: [Tutor] Problems understanding code output In-Reply-To: <24816197.4240951373490347622.JavaMail.ska@luo.to> References: <24816197.4240951373490347622.JavaMail.ska@luo.to> Message-ID: On 07/10/2013 05:05 PM, ska at luo.to wrote: > This is going a little over my head, please advice, what am I missing in > here? While you're watching, we could also point out that you posted a message in html format. There were actually two messages inside there, and many people will see the garbled one. This newsgroup is a text newsgroup. This mailing list is a text mailing list. Please post your messages in text format. Otherwise, some people will have problems seeing your code. -- DaveA From cybervigilante at gmail.com Fri Jul 19 00:39:52 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Thu, 18 Jul 2013 15:39:52 -0700 Subject: [Tutor] What every computer science major should know In-Reply-To: References: Message-ID: On 13 July 2013 18:12, boB Stepp wrote: > This may be of interest to some of us: > > http://matt.might.net/articles/what-cs-majors-should-know/ -- Here is how every computer on earth works, even those using Java. It's also how DNA works. I just saved you 200 grand in ruinous student loans ;') Computer Science in six lines: Read serial input device: Until: if serial input device says Stop, stop do something or go somewhere else on serial input, as instructed advance serial input one frame and read serial input device repeat: The 'do something' part can involve complications. Jim From alan.gauld at btinternet.com Fri Jul 19 00:44:12 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Thu, 18 Jul 2013 23:44:12 +0100 Subject: [Tutor] What every computer science major should know In-Reply-To: References: Message-ID: On 14/07/13 02:12, boB Stepp wrote: > This may be of interest to some of us: > > http://matt.might.net/articles/what-cs-majors-should-know/ Interesting post Bob, thanks. I think I've covered about 50-60% of the recommended reading and about 75% of the topics. But... I think its a tad old fashioned and neglects the biggest areas of weakness and growth in my experience of modern software engineering - large scale systems. More and more systems don't run on single hosts. They don't even run on single CPU architectures within a single box(the mention of GPU processing and CUDA hints at that). But learning parallelism on a single computer is only the tip of the iceberg. The vast majority of corporate programming today builds systems running across dozens and sometimes hundreds of physical servers. (One CRM implementation I did had over 200 servers across 3 sites all integrated and self managed as part of the project design.) There is one disparaging comment about UML but UML is about the only tool we have for documenting large scale systems effectively. It's far from perfect but a critical skill for grads IMHO. (Ironically it's usually taught as a tool for designing OOP programs with a few classes and that is its weakest area of application.) The other problem in all of this is that it is effectively impossible to teach all of that in a standard 3 or 4 year Bachelors course. Even with 2 years extra for a Masters degree. I did my degree in Electrical/Electronic engineering with specialism in software. It took me about 5 more years to read up on all the CS bits I wanted to cover that weren't in our course. I discovered these areas by discussion with colleagues who had attended other colleges or done other degrees. None of us had done all that was needed. And that's the problem. Computing has become too widely encompassing for anyone to learn the breadth of material needed in the time available in an undergraduate and/or masters course. Other engineering/science disciplines have addressed this by specializing and that is already happening with Games programming degrees becoming increasingly common and many courses in Business computing. I think we will see more of this in the future. It's simply unrealistic to ever expect college grads to have the breadth suggested in the article - they may aspire to it and maybe after 10 years or more attain it. But by then there will be another array of paradigms to absorb... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From alan.gauld at btinternet.com Fri Jul 19 01:00:28 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Fri, 19 Jul 2013 00:00:28 +0100 Subject: [Tutor] Selecting from list In-Reply-To: <1374168465.55644.YahooMailNeo@web163503.mail.gq1.yahoo.com> References: <1374168465.55644.YahooMailNeo@web163503.mail.gq1.yahoo.com> Message-ID: On 18/07/13 18:27, Hs Hs wrote: > hi list: Hi, please don't use an old thread to start a new one. This message appeared under a thread dated back in January. I nearly gave up looking for the unread message that my reader said was there. Start a new thread - you'll get better responses if you do. > In the following list, is there a simply way to find element less than > 200 sandwiched between two numbers greater than 1000. > > a = [3389, 178, 2674, 2586, 13731, 3189, 785, 1038, 25956, 33551] You example is ambiguous. What should happen in this example: a = [3389, 178, 66, 2674, 2586, 13731, 3189, 785, 1038, 25956, 33551] Should it list 178 and 66, one of them(which?) or neither? Or what about: a = [389, 178, 2674, 2586, 13731, 3189, 785, 1038, 25956, 33551] Should 178 now be ignored because 389 is lower than 1000? > (sorry this is not homework question. I want to avoid looping, because I > have 300K lines to parse through) If you have to parse 300K lines you will need a loop. It may not be explicit but it will be there. Get used to that idea and worry about how you process the data. And maybe making sure you only loop over everything once! -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From cybervigilante at gmail.com Fri Jul 19 01:13:35 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Thu, 18 Jul 2013 16:13:35 -0700 Subject: [Tutor] What every computer science major should know In-Reply-To: <51E870EF.2080201@btinternet.com> References: <51E870EF.2080201@btinternet.com> Message-ID: Alan Gauld > Except not all input devices are serial and not all systems take input, some > only generate output... But as a generalization its not bad :-) Except I stole it from Alan Turing ;') Jim From alan.gauld at btinternet.com Fri Jul 19 00:49:19 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Thu, 18 Jul 2013 23:49:19 +0100 Subject: [Tutor] What every computer science major should know In-Reply-To: References: Message-ID: <51E870EF.2080201@btinternet.com> On 18/07/13 23:39, Jim Mooney wrote: > Here is how every computer on earth works, even those using Java. > It's also how DNA works. I just saved you 200 grand in ruinous student > loans ;') > > Computer Science in six lines: > > Read serial input device: > Until: > if serial input device says Stop, stop > do something or go somewhere else on serial input, as instructed > advance serial input one frame and read serial input device > repeat: > > The 'do something' part can involve complications. Except not all input devices are serial and not all systems take input, some only generate output... But as a generalization its not bad :-) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From eryksun at gmail.com Fri Jul 19 01:19:11 2013 From: eryksun at gmail.com (eryksun) Date: Thu, 18 Jul 2013 19:19:11 -0400 Subject: [Tutor] install on windows In-Reply-To: <51E85E43.80500@seiboldsystems.com> References: <51D37F34.9020609@seiboldsystems.com> <51DD878D.7080902@seiboldsystems.com> <51E85E43.80500@seiboldsystems.com> Message-ID: On Thu, Jul 18, 2013 at 5:29 PM, larry seibold wrote: > This is what I get as me and in an Administrator shell: > > C:\>assoc .msi > .msi=msiexec.Document > > C:\>ftype Msi.Package > Msi.Package="%SystemRoot%\System32\msiexec.exe" /i "%1" %* > > C:\>ftype Msi.Document > File type 'Msi.Document' not found or no open command associated with it. > > I am not sure why running in an administrator shell worked, but the > implication of the above responce is that I need to change the association > of .msi to Msi.Package. The following will update the association: assoc .msi=Msi.Package As to why it worked in the console shell, there you specified the /i option and filename manually, i.e. "msiexec /i python-2.7.5.msi". Since your user account is a member of Administrators, you shouldn't have to install as *the* Administrator -- at least not in my experience. From joel.goldstick at gmail.com Fri Jul 19 02:08:30 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Thu, 18 Jul 2013 20:08:30 -0400 Subject: [Tutor] What every computer science major should know In-Reply-To: References: <51E870EF.2080201@btinternet.com> Message-ID: On Thu, Jul 18, 2013 at 7:13 PM, Jim Mooney wrote: > Alan Gauld > > > Except not all input devices are serial and not all systems take input, > some > > only generate output... But as a generalization its not bad :-) > > Except I stole it from Alan Turing ;') > > points for that! > Jim > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > -- Joel Goldstick http://joelgoldstick.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Fri Jul 19 02:18:14 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Thu, 18 Jul 2013 17:18:14 -0700 Subject: [Tutor] Selecting from list In-Reply-To: <1374168465.55644.YahooMailNeo@web163503.mail.gq1.yahoo.com> References: <1374168465.55644.YahooMailNeo@web163503.mail.gq1.yahoo.com> Message-ID: On 18 July 2013 10:27, Hs Hs wrote: > hi list: > > In the following list, is there a simply way to find element less than 200 > sandwiched between two numbers greater than 1000. > > a = [3389, 178, 2674, 2586, 13731, 3189, 785, 1038, 25956, 33551] > > in a, 178 is between 3389 and 2674. How this particular list can be > selected for further processing. > (sorry this is not homework question. I want to avoid looping, because I > have 300K lines to parse through) > > Thanks > Hs. Not sure what you want to do. If you only want to fulfill the test once, here is a way without a loop, using a list comprehension. #Using Python 2.7 on win 7 from __future__ import print_function a = [3389, 3666, 13, 2586, 13731, 2, 785, 1038, 25956, 33551, 199, 1500] under_200 = [x for x in xrange(0,len(a)) if a[x] - 200 < 0] #result: [2, 5, 10] - the indexes of list a for items < 200 # At this point I'm not sure what you want to do - if the triplet of over-1000 # sandwiching under-200 occurs more than once matters to you or not to you. # Either way you have a list of indexes of anything under 200. # If you're only interested in one triplet fulfilling, you're done: # a[index-1] > 1000 and a[index+1] > 1000 says the list passes the test and you can gather # the numbers into a 3-tuple using the indexes if you want to print them out. # If you have to test more than once, you should probably loop through the under_200 list. And of # course, you have to test for invalid index when you add or subtract to check for numbers over # 1000. Actually, you could probably replace the second loop with a comprehension, but # I think that's getting too complicated. >Jim From steve at pearwood.info Fri Jul 19 07:09:09 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 19 Jul 2013 15:09:09 +1000 Subject: [Tutor] Selecting from list In-Reply-To: References: <1374168465.55644.YahooMailNeo@web163503.mail.gq1.yahoo.com> Message-ID: <51E8C9F5.1010308@pearwood.info> On 19/07/13 10:18, Jim Mooney wrote: > On 18 July 2013 10:27, Hs Hs wrote: [...] >> (sorry this is not homework question. I want to avoid looping, because I >> have 300K lines to parse through) >> >> Thanks >> Hs. > > Not sure what you want to do. If you only want to fulfill the test once, > here is a way without a loop, using a list comprehension. A list comprehension *is* a loop. It even includes a "for" inside it. I really don't understand why people so often say things like "I have a bunch of stuff to do repeatedly, but I want to do it without a loop". To put it another way, "I want to repeat something without repeating it". WTF??? The only way to avoid a loop *somewhere* is to have a parallel-processing computer with at least as many parallel processes as you have things to repeat. So if Hs has 300K lines to process, he would need 300K processors, one per line. Since that's impractical unless you're Google or the NSA[1] you're going to need a loop, the only question is whether it is an *explicit* loop or an *implicit* loop. For example, a standard for-loop: for line in many_lines: process(line) or a list-comprehension: [process(line) for line in many_lines] are explicit loops. The map built-in is implicit: map(process, many_lines) So are many of numpy's array functions. But regardless of whether *you* write the loop, or Python does it for you, there is still a loop. [1] Hi guys! -- Steven From cybervigilante at gmail.com Fri Jul 19 19:30:50 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Fri, 19 Jul 2013 10:30:50 -0700 Subject: [Tutor] Selecting from list In-Reply-To: <51E8C9F5.1010308@pearwood.info> References: <1374168465.55644.YahooMailNeo@web163503.mail.gq1.yahoo.com> <51E8C9F5.1010308@pearwood.info> Message-ID: On 18 July 2013 22:09, Steven D'Aprano wrote: > A list comprehension *is* a loop. It even includes a "for" inside it. > True, but correct me if I'm wrong, I thought comprehensions proceeded at close to C speed, faster than a loop you could write normally, and speed was the major requirement with 200K elements. Actually, the second part, after excising the the 0 and len-1 indexes that would cause an index error when checking for >1000, could be a comprehension, too. But it was unclear what was desired. Can the condition only occur once or multiple times, and do you want to just test True/False, or get a list of the number-triplets that fulfill the condition? Anyway, I'll do it that way when I feel like it since I really like list comprehensions. For some reason to do with my obfuscated brain, I actually find them clearer than a fully-written-out loop ;') Jim From emile at fenx.com Fri Jul 19 19:34:18 2013 From: emile at fenx.com (Emile van Sebille) Date: Fri, 19 Jul 2013 10:34:18 -0700 Subject: [Tutor] Network PC additional fields Message-ID: Hi Ethan, Can you add some descriptive fields by ip address? I'm thinking things like what this machine is used for; who accesses it; what other systems does it depend on; where is it backed up to; etc. General information that really doesn't have another home. On that note, a thought comes to mind. In VSDS (the system I started writing in the 70s that TRU is running) I created a sidekick-ish interface that was invoked with the F1 key part of which was a memo pad of 40 pages that I used in a variety of ways to keep notes on various topics. The initial display was two columns of 20 rows of page 'Titles' that expanded to full pages of text when selected. I'm thinking we may want something similar available within openERP. We can enter free form notes on the various Note tabs we have now, and extending that such that we can pre-configure 'required' page titles by application for use as Note tabs would allow us to enter any additional memo content. Give it some thought and we can expand on the idea over time. Emile From emile at fenx.com Fri Jul 19 20:15:34 2013 From: emile at fenx.com (Emile van Sebille) Date: Fri, 19 Jul 2013 11:15:34 -0700 Subject: [Tutor] Network PC additional fields In-Reply-To: References: Message-ID: Oops. On 7/19/2013 10:34 AM, Emile van Sebille wrote: From brookenatorij at gmail.com Thu Jul 18 21:20:51 2013 From: brookenatorij at gmail.com (Charles McAndrews) Date: Thu, 18 Jul 2013 11:20:51 -0800 Subject: [Tutor] AttributeError: Message-ID: Whenever I try to type 'python' into the cmd prompt, it always gives me this error: Traceback (most recent call last): File "C:\Python27\lib\site.py", line 62, in import os File "C:\Python27\lib\os.py", line 398, in import UserDict File "C:\Python27\lib\UserDict.py", line 83, in import _abcoll File "C:\Python27\lib\_abcoll.py", line 70, in Iterable.register(str) File "C:\Python27\lib\abc.py", line 107, in register if not isinstance(subclass, (type, types.ClassType)): AttributeError: 'module' object has no attribute 'ClassType' -------------- next part -------------- An HTML attachment was scrubbed... URL: From nsivaram.net at gmail.com Fri Jul 19 21:24:17 2013 From: nsivaram.net at gmail.com (Sivaram Neelakantan) Date: Sat, 20 Jul 2013 00:54:17 +0530 Subject: [Tutor] suggestions for splitting file based on date Message-ID: <87a9litkta.fsf@gmail.com> I've got some stock indices data that I plan to plot using matplotlib. The data is simply date, idx_close_value and my plan is to plot the last 30 day, 90, 180 day & all time graphs of the indices. a) I can do the date computations using the python date libs b) plotting with matplotlib, I can get that done what is the best way to split the file into the last 30 day recs, 90 day recs when the data is in increasing time order? My initial thinking is to first reverse the file, append to various 30/90/180 day lists for every rec > computed date for the corresponding date windows. Is that the way to go or is there a better way? sivaram -- From __peter__ at web.de Fri Jul 19 21:39:59 2013 From: __peter__ at web.de (Peter Otten) Date: Fri, 19 Jul 2013 21:39:59 +0200 Subject: [Tutor] AttributeError: References: Message-ID: Charles McAndrews wrote: > Whenever I try to type 'python' into the cmd prompt, it always gives me > this error: > > Traceback (most recent call last): > File "C:\Python27\lib\site.py", line 62, in > import os > File "C:\Python27\lib\os.py", line 398, in > import UserDict > File "C:\Python27\lib\UserDict.py", line 83, in > import _abcoll > File "C:\Python27\lib\_abcoll.py", line 70, in > Iterable.register(str) > File "C:\Python27\lib\abc.py", line 107, in register > if not isinstance(subclass, (type, types.ClassType)): > AttributeError: 'module' object has no attribute 'ClassType' You have a file types.py, perhaps written by yourself, which Python imports instead of the types module in the standard library. Try invoking Python with the -S option: python -S If that gets you into the interactive interpreter do >>> import types >>> types.__file__ 'yadda/types.pyc' Instead of yadda/types.pyc you will see the path of the actual file that hides the types module of the stdlib on your system. Remove that types.pyc, rename the corresponding types.py, and you should be able to start python in the usual way again. From __peter__ at web.de Fri Jul 19 22:00:44 2013 From: __peter__ at web.de (Peter Otten) Date: Fri, 19 Jul 2013 22:00:44 +0200 Subject: [Tutor] suggestions for splitting file based on date References: <87a9litkta.fsf@gmail.com> Message-ID: Sivaram Neelakantan wrote: > I've got some stock indices data that I plan to plot using matplotlib. > The data is simply date, idx_close_value and my plan is to plot the > last 30 day, 90, 180 day & all time graphs of the indices. > > a) I can do the date computations using the python date libs > b) plotting with matplotlib, I can get that done > > what is the best way to split the file into the last 30 day recs, 90 > day recs when the data is in increasing time order? My initial > thinking is to first reverse the file, append to various 30/90/180 day > lists for every rec > computed date for the corresponding date > windows. > > Is that the way to go or is there a better way? I'd start with a single list for the complete data, reverse that using the aptly named method and then create the three smaller lists using slicing. For example: >>> stock_data = range(10) >>> stock_data.reverse() >>> stock_data [9, 8, 7, 6, 5, 4, 3, 2, 1, 0] >>> stock_data[:3] # the last three days [9, 8, 7] On second thought I don't see why you want to reverse the data. If you omit that step you need to modify the slicing: >>> stock_data = range(10) >>> stock_data[-3:] # the last three days [7, 8, 9] From alan.gauld at btinternet.com Fri Jul 19 22:04:04 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Fri, 19 Jul 2013 21:04:04 +0100 Subject: [Tutor] messages held in moderation queue Message-ID: There have been several recent issues with people finding their messages being trapped in the moderation queue. On investigation I discovered that about 5% of the list addresses had been turned onto full moderation. I have no idea why or how or when this happened, it's certainly not normal moderator practice since it just creates extra work for us! I've gone through the entire list and manually reset the flags. This is not exactly a slick process and I may have missed a few so if you think your messages are not getting through and may still be getting trapped let me know and I'll investigate. Apologies to those affected (which included at least 3 old time list regulars!) and hopefully normal service will now resume. -- Alan G List Moderator From davea at davea.name Fri Jul 19 22:30:23 2013 From: davea at davea.name (Dave Angel) Date: Fri, 19 Jul 2013 16:30:23 -0400 Subject: [Tutor] suggestions for splitting file based on date In-Reply-To: References: <87a9litkta.fsf@gmail.com> Message-ID: On 07/19/2013 04:00 PM, Peter Otten wrote: > Sivaram Neelakantan wrote: > >> I've got some stock indices data that I plan to plot using matplotlib. >> The data is simply date, idx_close_value and my plan is to plot the >> last 30 day, 90, 180 day & all time graphs of the indices. >> >> a) I can do the date computations using the python date libs >> b) plotting with matplotlib, I can get that done >> >> what is the best way to split the file into the last 30 day recs, 90 >> day recs when the data is in increasing time order? My initial >> thinking is to first reverse the file, append to various 30/90/180 day >> lists for every rec > computed date for the corresponding date >> windows. >> >> Is that the way to go or is there a better way? > > I'd start with a single list for the complete data, reverse that using the > aptly named method and then create the three smaller lists using slicing. > > For example: > >>>> stock_data = range(10) >>>> stock_data.reverse() >>>> stock_data > [9, 8, 7, 6, 5, 4, 3, 2, 1, 0] >>>> stock_data[:3] # the last three days > [9, 8, 7] > > On second thought I don't see why you want to reverse the data. If you omit > that step you need to modify the slicing: > >>>> stock_data = range(10) >>>> stock_data[-3:] # the last three days > [7, 8, 9] > > I see Alan has assumed that the data is already divided into day-size hunks, so that subscripting those hunks is possible. He also assumed all the data will fit in memory at one time. But in my envisioning of your description, I pictured a variable number of records per day, with each record being a variable length stream of bytes starting with a length field. I pictured needing to handle a month with either zero entries or one with 3 billion entries. And even if a month is reasonable, I pictured the file as having 10 years of spurious data before you get to the 180 day point. Are you looking for an optimal solution, or just one that works? What order do you want the final data to be in. How is the data organized on disk? Is each record a fixed size? If so, you can efficiently do a binary search in the file to find the 30, 90, and 180 day points. Once you determine the offset in the file for those 180, 90, and 30 day points, it's a simple matter to just seek to one such spot and process all the records following. Most records need never be read from disk at all. If the records are not fixed length, you can still do the same thing, but you will need one complete pass through the file to find those same 3 offsets. -- DaveA From wprins at gmail.com Fri Jul 19 22:32:22 2013 From: wprins at gmail.com (Walter Prins) Date: Fri, 19 Jul 2013 21:32:22 +0100 Subject: [Tutor] suggestions for splitting file based on date In-Reply-To: <87a9litkta.fsf@gmail.com> References: <87a9litkta.fsf@gmail.com> Message-ID: Hi, On 19 July 2013 20:24, Sivaram Neelakantan wrote: > > I've got some stock indices data that I plan to plot using matplotlib. > The data is simply date, idx_close_value and my plan is to plot the > last 30 day, 90, 180 day & all time graphs of the indices. > > a) I can do the date computations using the python date libs > b) plotting with matplotlib, I can get that done > > what is the best way to split the file into the last 30 day recs, 90 > day recs when the data is in increasing time order? My initial > thinking is to first reverse the file, append to various 30/90/180 day > lists for every rec > computed date for the corresponding date > windows. > > Is that the way to go or is there a better way? You can do things manually as Peter's suggested, there's some value to be had from learning to do things "by hand". However, if you're going to continue to work with matplotlib and stock data I highly recommend you eventually pick up and learn Pandas: http://pandas.pydata.org/ Another useful link: https://bitbucket.org/hrojas/learn-pandas Walter -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Sat Jul 20 00:21:10 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Fri, 19 Jul 2013 15:21:10 -0700 Subject: [Tutor] Selecting from list In-Reply-To: <1374168465.55644.YahooMailNeo@web163503.mail.gq1.yahoo.com> References: <1374168465.55644.YahooMailNeo@web163503.mail.gq1.yahoo.com> Message-ID: On 18 July 2013 10:27, Hs Hs wrote: > hi list: > > In the following list, is there a simply way to find element less than 200 > sandwiched between two numbers greater than 1000. > > a = [3389, 178, 2674, 2586, 13731, 3189, 785, 1038, 25956, 33551] > > in a, 178 is between 3389 and 2674. How this particular list can be > selected for further processing. > (sorry this is not homework question. I want to avoid looping, because I > have 300K lines to parse through) > > Thanks > Hs. Looks like my last message didn't get delivered, so I'll post this as a late response: Here's the whole thing in a list comprehension, which as mentioned, is still an unseen loop, but which is faster than one written out. Although as also mentioned, numpy is even faster for a huge amount of data. Although I think that means huge lists, not necessarily a huge record base, since accessing records would dwarf any style of processing a short list. Since I'm not sure what was required, I took the 'greedy' option of listing all 3-tuples that fulfilled the requirement as I understood it. The comprehension also takes care of eliminating endpoints that would cause an index-error crash. I changed the data to add a bunch of possible gotchas ;') a = [0,3389, 178, 2674, 199, 2586, 13731, 12, 13, 14, 3189, 15, 785, 10, 1038, 25956, 6, 8, 2, 3551, 2] meets_requirements = [(a[idx-1],a[idx],a[idx+1]) for idx in range(1,len(a)-1) \ if a[idx] < 200 and a[idx-1] > 1000 and a[idx+1] > 1000] # result [(3389, 178, 2674), (2674, 199, 2586)] -- Jim The universe is made up of patches that are only woven together as they are accessed. Sometimes a stitch is dropped and something really weird happens. Only nobody will ever believe you From cybervigilante at gmail.com Sat Jul 20 03:40:26 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Fri, 19 Jul 2013 18:40:26 -0700 Subject: [Tutor] Selecting from list In-Reply-To: References: <1374168465.55644.YahooMailNeo@web163503.mail.gq1.yahoo.com> Message-ID: > a = [0,3389, 178, 2674, 199, 2586, 13731, 12, 13, 14, 3189, 15, 785, 10, 1038, > 25956, 6, 8, 2, 3551, 2] > > meets_requirements = [(a[idx-1],a[idx],a[idx+1]) for idx in range(1,len(a)-1) \ > if a[idx] < 200 and a[idx-1] > 1000 and a[idx+1] > 1000] > > # result [(3389, 178, 2674), (2674, 199, 2586)] Although on third thought, a real loop is on average fastest if the requirement is you only need find the first triplet, or there is only one, since you can then break out of the loop. I don't think you can break out of a comprehension. If you can it would be an ugly business, IMHO ;') > -- > Jim > > The universe is made up of patches that are only woven together as > they are accessed. Sometimes a stitch is dropped and something really > weird happens. Only nobody will ever believe you -- Jim The universe is made up of patches that are only woven together as they are accessed. Sometimes a stitch is dropped and something really weird happens. Only nobody will ever believe you From nsivaram.net at gmail.com Sat Jul 20 06:53:43 2013 From: nsivaram.net at gmail.com (Sivaram Neelakantan) Date: Sat, 20 Jul 2013 10:23:43 +0530 Subject: [Tutor] suggestions for splitting file based on date References: <87a9litkta.fsf@gmail.com> Message-ID: <87siz9rfvs.fsf@gmail.com> On Sat, Jul 20 2013,Peter Otten wrote: > Sivaram Neelakantan wrote: [snipped 16 lines] > I'd start with a single list for the complete data, reverse that using the > aptly named method and then create the three smaller lists using slicing. > > For example: > >>>> stock_data = range(10) >>>> stock_data.reverse() >>>> stock_data > [9, 8, 7, 6, 5, 4, 3, 2, 1, 0] >>>> stock_data[:3] # the last three days > [9, 8, 7] > > On second thought I don't see why you want to reverse the data. If you omit > that step you need to modify the slicing: > >>>> stock_data = range(10) >>>> stock_data[-3:] # the last three days > [7, 8, 9] > Thank you, this is as simple as it gets. The only wiggle is the trading day versus the calendar day. If I use the trading day, the above slicing will work(a simple numerical count, to represent 30 *trading* day window) but the calendar day is a lot more hops accounting for weekends or days of no trading. I believe I'd have to end up doing a date compare. [snipped 7 lines] sivaram -- From nsivaram.net at gmail.com Sat Jul 20 07:00:04 2013 From: nsivaram.net at gmail.com (Sivaram Neelakantan) Date: Sat, 20 Jul 2013 10:30:04 +0530 Subject: [Tutor] suggestions for splitting file based on date References: <87a9litkta.fsf@gmail.com> Message-ID: <87ob9xrfl7.fsf@gmail.com> On Sat, Jul 20 2013,Dave Angel wrote: > On 07/19/2013 04:00 PM, Peter Otten wrote: >> Sivaram Neelakantan wrote: >> [snipped 35 lines] > > I see Alan has assumed that the data is already divided into day-size > hunks, so that subscripting those hunks is possible. He also assumed > all the data will fit in memory at one time. Yes, I'm just taking the day end index close, so the fixed line format is small and the number of records is ~2K. > > But in my envisioning of your description, I pictured a variable > number of records per day, with each record being a variable length > stream of bytes starting with a length field. I pictured needing to > handle a month with either zero entries or one with 3 billion entries. > And even if a month is reasonable, I pictured the file as having 10 > years of spurious data before you get to the 180 day point. I'll get there eventually.....once you see my name splashed on Wall Street, raking in the millions. :-) > > Are you looking for an optimal solution, or just one that works? What > order do you want the final data to be in. How is the data organized > on disk? Is each record a fixed size? If so, you can efficiently do > a binary search in the file to find the 30, 90, and 180 day points. These are small,fixed line extracts. > > Once you determine the offset in the file for those 180, 90, and 30 > day points, it's a simple matter to just seek to one such spot and > process all the records following. Most records need never be read > from disk at all. Will this work when the trading days and calendar days are not the same? My 30 days is the calendar days while the 30 trading days could mean an extra 1-2 calendar weeks. sivaram -- From nsivaram.net at gmail.com Sat Jul 20 07:01:49 2013 From: nsivaram.net at gmail.com (Sivaram Neelakantan) Date: Sat, 20 Jul 2013 10:31:49 +0530 Subject: [Tutor] suggestions for splitting file based on date References: <87a9litkta.fsf@gmail.com> Message-ID: <87k3klrfia.fsf@gmail.com> On Sat, Jul 20 2013,Walter Prins wrote: [snipped 18 lines] >> Is that the way to go or is there a better way? > > > You can do things manually as Peter's suggested, there's some value to be > had from learning to do things "by hand". However, if you're going to > continue to work with matplotlib and stock data I highly recommend you > eventually pick up and learn Pandas: http://pandas.pydata.org/ > > Another useful link: https://bitbucket.org/hrojas/learn-pandas > Thanks, will learn this too. [snipped 5 lines] sivaram -- From __peter__ at web.de Sat Jul 20 11:11:17 2013 From: __peter__ at web.de (Peter Otten) Date: Sat, 20 Jul 2013 11:11:17 +0200 Subject: [Tutor] suggestions for splitting file based on date References: <87a9litkta.fsf@gmail.com> <87siz9rfvs.fsf@gmail.com> Message-ID: Sivaram Neelakantan wrote: > On Sat, Jul 20 2013,Peter Otten wrote: > >> Sivaram Neelakantan wrote: > > [snipped 16 lines] > >> I'd start with a single list for the complete data, reverse that using >> the aptly named method and then create the three smaller lists using >> slicing. >> >> For example: >> >>>>> stock_data = range(10) >>>>> stock_data.reverse() >>>>> stock_data >> [9, 8, 7, 6, 5, 4, 3, 2, 1, 0] >>>>> stock_data[:3] # the last three days >> [9, 8, 7] >> >> On second thought I don't see why you want to reverse the data. If you >> omit that step you need to modify the slicing: >> >>>>> stock_data = range(10) >>>>> stock_data[-3:] # the last three days >> [7, 8, 9] >> > > Thank you, this is as simple as it gets. The only wiggle is the > trading day versus the calendar day. If I use the trading day, the > above slicing will work(a simple numerical count, to represent 30 > *trading* day window) but the calendar day is a lot more hops > accounting for weekends or days of no trading. I believe I'd have to > end up doing a date compare. You could insert dummy items into the list for days with no trading. Alternatively, use bisect() to determine the list index, see http://docs.python.org/2.7/library/bisect.html From davea at davea.name Sat Jul 20 11:24:12 2013 From: davea at davea.name (Dave Angel) Date: Sat, 20 Jul 2013 05:24:12 -0400 Subject: [Tutor] suggestions for splitting file based on date In-Reply-To: <87ob9xrfl7.fsf@gmail.com> References: <87a9litkta.fsf@gmail.com> <87ob9xrfl7.fsf@gmail.com> Message-ID: On 07/20/2013 01:00 AM, Sivaram Neelakantan wrote: > On Sat, Jul 20 2013,Dave Angel wrote: > > > These are small,fixed line extracts. > >> >> Once you determine the offset in the file for those 180, 90, and 30 >> day points, it's a simple matter to just seek to one such spot and >> process all the records following. Most records need never be read >> from disk at all. > > Will this work when the trading days and calendar days are not the > same? My 30 days is the calendar days while the 30 trading days could > mean an extra 1-2 calendar weeks. > Certainly it'll work. Once you've done your binary search to find one of the 3 starting places. you can process the data sequentially. If I can describe your file spec, you have a file of fixed-length records, each with a date stamp. You have a variable number of records per day (zero or one, the way you describe it, but that doesn't affect our algorithm). You want to make a list of all of the records since todays_date-N, where N is 30, 90, 180, or whatever. Since you don't have tons of data, you could load all of it into a list ahead of time. Manipulating that list will be easier than manipulating the file. But because the records are fixed size, the two are isomorphic. So: read all the records into a list. (Each item in the list is a tuple of date and data) For a particular value of "age", create a sublist of those records newer than age: target is today-age. Do a binary search in the list for target. Using that index as a starting point, return a slice in the list. Now just call that function 3 times, for your three different values of age. Since you've got an in-memory list, it's straightforward to use bisect.bisect_left() to do the binary search.i But since your data is small, a simple linear search isn't too much slower either. See the following link: http://docs.python.org/2/library/bisect.html#searching-sorted-lists http://docs.python.org/3.3/library/bisect.html#searching-sorted-lists -- DaveA From sunil.techspk at gmail.com Sat Jul 20 12:17:02 2013 From: sunil.techspk at gmail.com (Sunil Tech) Date: Sat, 20 Jul 2013 15:47:02 +0530 Subject: [Tutor] 3 Dimensional Dictionaries Message-ID: Hi Everyone, I have a list of dictionaries like world = [{'continent':'Asia','continent_code':1,'ocean':'Pacific','country':'India','country_code':1,'state':'Kerala', 'state_pin':500001}, {'continent':'Asia','continent_code':1,'ocean':'Pacific','country':'India','country_code':1,'state':'Karnataka', 'state_pin':500002}, {'continent':'Africa','continent_code':2,'ocean':'Atlantic','country':'Egypt','country_code':2,'state':'East Egypt', 'state_pin':700001}, {'continent':'Africa','continent_code':2,'ocean':'Atlantic','country':'Egypt','country_code':2,'state':'West Egypt', 'state_pin':700002}, {'continent':'Africa','continent_code':2,'ocean':'Atlantic','country':'Egypt','country_code':2,'state':'North Egypt', 'state_pin':700003}] i am trying to to make it in this format world = [{'continent':'Asia', 'ocean':'Pacific', 'countries':[{'country':'India', 'states':[{'state':'Kerala', 'state_pin':500001}, {'state':'Karnataka', 'state_pin':500002}] }] }, {'continent':'Africa', 'ocean':'Atlantic', 'countries':[{'country':'Egypt', 'states':[{'state':'East Egypt', 'state_pin':700001}, {'state':'West Egypt', 'state_pin':700002}, {'state':'North Egypt', 'state_pin':700003}] }] } ] Please help me in this regard. -------------- next part -------------- An HTML attachment was scrubbed... URL: From nik at naturalnet.de Sat Jul 20 12:28:12 2013 From: nik at naturalnet.de (Dominik George) Date: Sat, 20 Jul 2013 12:28:12 +0200 Subject: [Tutor] 3 Dimensional Dictionaries In-Reply-To: References: Message-ID: <20130720102812.GC30129@keks.naturalnet.de> Hi, > world = > [{'continent':'Asia','continent_code':1,'ocean':'Pacific','country':'India','country_code':1,'state':'Kerala', > 'state_pin':500001}, > [...] > > i am trying to to make it in this format > to clarify the task at hand: this is comparible to SQL's GROUP BY clause, right? -nik -- Auf welchem Server liegt das denn jetzt?? Wenn es nicht ?bers Netz kommt bei Hetzner, wenn es nicht gelesen wird bei STRATO, wenn es klappt bei manitu. PGP-Fingerprint: 3C9D 54A4 7575 C026 FB17 FD26 B79A 3C16 A0C4 F296 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 905 bytes Desc: Digital signature URL: From sunil.techspk at gmail.com Sat Jul 20 12:37:30 2013 From: sunil.techspk at gmail.com (Sunil Tech) Date: Sat, 20 Jul 2013 16:07:30 +0530 Subject: [Tutor] 3 Dimensional Dictionaries In-Reply-To: <20130720102812.GC30129@keks.naturalnet.de> References: <20130720102812.GC30129@keks.naturalnet.de> Message-ID: Hi, yes Dominik & the result should be in that format as stated. On Sat, Jul 20, 2013 at 3:58 PM, Dominik George wrote: > Hi, > > > world = > > > [{'continent':'Asia','continent_code':1,'ocean':'Pacific','country':'India','country_code':1,'state':'Kerala', > > 'state_pin':500001}, > > [...] > > > > i am trying to to make it in this format > > > > to clarify the task at hand: this is comparible to SQL's GROUP BY > clause, right? > > -nik > > -- > Auf welchem Server liegt das denn jetzt?? > Wenn es nicht ?bers Netz kommt bei Hetzner, wenn es nicht > gelesen wird bei STRATO, wenn es klappt bei manitu. > > PGP-Fingerprint: 3C9D 54A4 7575 C026 FB17 FD26 B79A 3C16 A0C4 F296 > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From davea at davea.name Sat Jul 20 13:14:29 2013 From: davea at davea.name (Dave Angel) Date: Sat, 20 Jul 2013 07:14:29 -0400 Subject: [Tutor] 3 Dimensional Dictionaries In-Reply-To: References: Message-ID: On 07/20/2013 06:17 AM, Sunil Tech wrote: > Hi Everyone, > > I have a list of dictionaries like > > world = > [{'continent':'Asia','continent_code':1,'ocean':'Pacific','country':'India','country_code':1,'state':'Kerala', > 'state_pin':500001}, > {'continent':'Asia','continent_code':1,'ocean':'Pacific','country':'India','country_code':1,'state':'Karnataka', > 'state_pin':500002}, > {'continent':'Africa','continent_code':2,'ocean':'Atlantic','country':'Egypt','country_code':2,'state':'East > Egypt', 'state_pin':700001}, > {'continent':'Africa','continent_code':2,'ocean':'Atlantic','country':'Egypt','country_code':2,'state':'West > Egypt', 'state_pin':700002}, > {'continent':'Africa','continent_code':2,'ocean':'Atlantic','country':'Egypt','country_code':2,'state':'North > Egypt', 'state_pin':700003}] > > > i am trying to to make it in this format > > world = [{'continent':'Asia', 'ocean':'Pacific', > 'countries':[{'country':'India', > 'states':[{'state':'Kerala', 'state_pin':500001}, > {'state':'Karnataka', 'state_pin':500002}] > }] > }, > {'continent':'Africa', 'ocean':'Atlantic', > 'countries':[{'country':'Egypt', > 'states':[{'state':'East Egypt', 'state_pin':700001}, > {'state':'West Egypt', 'state_pin':700002}, > {'state':'North Egypt', 'state_pin':700003}] > }] > } > ] > Is this an assignment? If so, it'd be best to tell us the exact assignment statement. Is it to fit existing code that's designed to work with the second format? Or is this a starting place, and subject to change? In particular, I'd immediately define a namedtuple, and redefine the first list as a list of such named-tuples. Then I'd make an index to such tuples via your two-dimensional dict entries. If you stick with your present plan, you should realize that there are assumptions built into it. For example, the assumption that a continent has only one ocean -- that all countries within the continent have the same ocean. Given all that, the simplest approach is to create defaultdict with the default being list. Then each time you add a record to a dict, you ust append to the list; if it's a new key, an empty list is created for you automatically. both namedtuple and defaultdict are in the collections module. http://docs.python.org/2/library/collections.html -- DaveA From steve at pearwood.info Sat Jul 20 18:17:05 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 21 Jul 2013 02:17:05 +1000 Subject: [Tutor] 3 Dimensional Dictionaries In-Reply-To: References: Message-ID: <51EAB801.3010405@pearwood.info> On 20/07/13 20:17, Sunil Tech wrote: > Hi Everyone, > > I have a list of dictionaries like Hi Sunil, A couple of style issues here. Please put spaces between parts of your code, for two reasons. Firstly, it makes it a lot easier to read, or perhaps I should say, notusingspacesmakesitmuchhardertoread. Secondly, when you don't use spaces, it's much harder for the email program to word-wrap lines, and your readers have to do a lot more work to read your email in detail. Since we're volunteers, and not being paid to answer your questions, you should make it as simple as possible for us. If you make it hard to understand your question, we simply might not bother. As far as your actual question, I'm not sure how we can answer it. You already know how to turn the list of dictionaries into a list of nested dictionaries and lists, because you did it yourself. Write down the steps you did, in English (or the human language of your choice), then turn each step into Python code. As your question stands right now, I'm not even sure I understand what process you used to merge the list-of-dicts into a different list-of-dicts. But I can see one thing: the continent code is unused, so I have deleted it from your examples to make it easier to read. Your data currently looks like this: old_world = [ {'continent': 'Asia', 'ocean': 'Pacific', 'country': 'India', 'country_code': 1, 'state': 'Kerala', 'state_pin': 500001}, {'continent': 'Asia', 'ocean': 'Pacific', 'country': 'India', 'country_code': 1, 'state': 'Karnataka', 'state_pin': 500002}, {'continent': 'Africa', 'ocean': 'Atlantic', 'country': 'Egypt', 'country_code': 2, 'state': 'East Egypt', 'state_pin': 700001}, {'continent': 'Africa', 'ocean': 'Atlantic', 'country': 'Egypt', 'country_code': 2, 'state': 'West Egypt', 'state_pin': 700002}, {'continent': 'Africa', 'ocean': 'Atlantic', 'country': 'Egypt', 'country_code': 2, 'state': 'North Egypt', 'state_pin': 700003}, ] (By the way, Egypt is nowhere near the Atlantic.) And you want to collect it in the form: world = [ {'continent': 'Asia', 'ocean': 'Pacific', 'countries': [{'country': 'India', 'states': [{'state': 'Kerala', 'state_pin': 500001}, {'state': 'Karnataka', 'state_pin': 500002} ] } ] }, {'continent': 'Africa', 'ocean': 'Atlantic', 'countries': [{'country': 'Egypt', 'states': [{'state': 'East Egypt', 'state_pin': 700001}, {'state': 'West Egypt', 'state_pin': 700002}, {'state': 'North Egypt', 'state_pin': 700003} ] } ] } ] In my opinion, that is a horrible design for a data structure. XML has got a *lot* to answer for. (Actually, XML is not the problem. XML is fine for what it is designed for, which is transferring data from one application to another. What is the problem is languages and programmers who try to use XML, and XML-inspired designs, for their internal data structures.) Anyway, I think we can improve it by using a few helpful data structures, and smarter use of dictionaries: from collections import namedtuple Continent = namedtuple("Continent", "ocean countries") State = namedtuple("State", "state pin") world = {} # hold the new data for d in old_world: continent = d['continent'] if continent not in world: world[continent] = Continent(d['ocean'], {}) country = d['country'] if country not in world[continent].countries: world[continent].countries[country] = [] state = State(d['state'], d['state_pin']) world[continent].countries[country].append(state) print(world) which gives this data structure: {'Asia': Continent(ocean='Pacific', countries={ 'India': [State(state='Kerala', pin=500001), State(state='Karnataka', pin=500002) ] } ), 'Africa': Continent(ocean='Atlantic', countries={ 'Egypt': [State(state='East Egypt', pin=700001), State(state='West Egypt', pin=700002), State(state='North Egypt', pin=700003) ] } ) } (Disclaimer: I manually pretty-printed the above. When Python prints it, it doesn't look anywhere nearly so good.) I realise that this is not exactly what you asked for, but in my opinion this is a better data structure than what you suggested. If you disagree, you should be able to modify my code to do what you prefer, the basic style will be the same. Good luck! -- Steven From cybervigilante at gmail.com Sat Jul 20 20:24:48 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 20 Jul 2013 11:24:48 -0700 Subject: [Tutor] slashes in paths Message-ID: I was looking at os.path.join, which is supposed to join paths intelligently. I've used it but this time I actually looked at the path for this: #Using Python 2.7 on Win 7 from __future__ import division, print_function import os soundpath = 'c:/python27/jimprogs/wav' soundlist = os.listdir(soundpath) soundfile13 = os.path.join(soundpath, soundlist[13]) The result is: >>> soundfile13 'c:/python27/jimprogs/wav\\bicycle_bell.wav' >>> with single forward slashes mixed with a double backslash it comes out even worse if I print it c:/python27/jimprogs/wav\bicycle_bell.wav - no double backslash, which could create a mess if someone copied that and tried it. Oddly, the mixed path does work for playing sounds, >>> import winsound >>> winsound.PlaySound('c:/python27/jimprogs/wav\\bicycle_bell.wav',winsound.SND_FILENAME) >>> That rings a bell. But it just doesn't look right. If forward slash is a Python convention instead of double backslash, it should follow through. I can see where you could get in trouble with it. It does work right if I end the path with a / - i.e. - soundpath = 'c:/python27/jimprogs/wav/' but that's not required. Ending with a forward slash or not seems to work either way, although I'm not sure about Linux. -- Jim The universe is made up of patches that are only woven together as they are accessed. Sometimes a stitch is dropped and something really weird happens. Only nobody will ever believe you From nik at naturalnet.de Sat Jul 20 22:06:59 2013 From: nik at naturalnet.de (Dominik George) Date: Sat, 20 Jul 2013 22:06:59 +0200 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: <20130720200658.GD30129@keks.naturalnet.de> Hi, > >>> soundfile13 > 'c:/python27/jimprogs/wav\\bicycle_bell.wav' > >>> > > with single forward slashes mixed with a double backslash > > it comes out even worse if I print it > > c:/python27/jimprogs/wav\bicycle_bell.wav - no double backslash, > which could create a mess if someone copied that and tried it. That's because the interactive python interpreter has bad behaviour when you just throw a name at it. It tries to find a good trade-off between exact representation of the internal data and human readibility. The result you see is printed in such a way that you can copy and paste it as a Python string, with escaped backslashes. When you use print, the human-readable string data is written to stdout, with the literal backslashes that the string really contains. > Oddly, the mixed path does work for playing sounds, > > >>> import winsound > >>> winsound.PlaySound('c:/python27/jimprogs/wav\\bicycle_bell.wav',winsound.SND_FILENAME) > >>> > That rings a bell. This will work in any case. First of all, in this example code, the \\ creates a literal backslash in the string, so what is used in the end is a single backslash. If that weren't the case, this would still work because empty directory names are ignored in all reasonable operating systems. > But it just doesn't look right. If forward slash is a Python > convention instead of double backslash, it should follow through. I > can see where you could get in trouble with it. Well, who says it is a Python convention? It is a POSIX convention, and in your code above you jsut used that. Because Python is such a good friend, it doesn't blow up but take what you gave to it. As you can see by using the os.path.join() function is that on Windows, when you ask Python to do the work for you, it really uses *single* backslashes. -nik -- Ein Jabber-Account, sie alle zu finden; ins Dunkel zu treiben und ewig zu binden; im NaturalNet, wo die Schatten droh'n ;)! PGP-Fingerprint: 3C9D 54A4 7575 C026 FB17 FD26 B79A 3C16 A0C4 F296 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 905 bytes Desc: Digital signature URL: From cybervigilante at gmail.com Sat Jul 20 22:16:03 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 20 Jul 2013 13:16:03 -0700 Subject: [Tutor] slashes in paths In-Reply-To: <20130720200658.GD30129@keks.naturalnet.de> References: <20130720200658.GD30129@keks.naturalnet.de> Message-ID: Dominik George > Well, who says it is a Python convention? It is a POSIX convention, and > in your code above you jsut used that. Because Python is such a good > friend, it doesn't blow up but take what you gave to it. As you can see > by using the os.path.join() function is that on Windows, when you ask > Python to do the work for you, it really uses *single* backslashes. But oddly, it makes all slashes forward if I end the path with a forward slash, so it's not consistent with itself. BTW, is ending a path with a slash a Linux requirement, or just a convention, or a who-cares-either-way? I forget since I've been on windows a long time, and it accepts either method. I guess the best thing is to always end all paths with a forward slash, catching both OS's and being consistent. Jim From alan.gauld at btinternet.com Sat Jul 20 22:41:05 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Sat, 20 Jul 2013 21:41:05 +0100 Subject: [Tutor] 3 Dimensional Dictionaries In-Reply-To: References: Message-ID: On 20/07/13 11:17, Sunil Tech wrote: > Hi Everyone, > > I have a list of dictionaries like > > world = > [{'continent':'Asia','continent_code':1,'ocean':'Pacific','country':'India','country_code':1,'state':'Kerala', > 'state_pin':500001}, > > i am trying to to make it in this format > > world = [{'continent':'Asia', 'ocean':'Pacific', > 'countries':[{'country':'India', > 'states':[{'state':'Kerala', 'state_pin':500001}, > {'state':'Karnataka', 'state_pin':500002}] > }] > }, > Please help me in this regard. In what regard? Where do you need the help? You seem to know the data format you want? The only thing I'd suggest is to consider using classes if you are familiar with them. world => list of continents continent => class containing countries country => class containing states state => class containing data It then becomes easier to build helper methods to extract/manipulate the data you are interested in. Alternatively, if you have a large amount of data a database may be another option. Swap table for class above and use SQL to manage the data. But other than those suggestions I don't know what kind of help you want? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From alan.gauld at btinternet.com Sat Jul 20 22:46:00 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Sat, 20 Jul 2013 21:46:00 +0100 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: On 20/07/13 19:24, Jim Mooney wrote: > I was looking at os.path.join, which is supposed to join paths > intelligently. It does including taking account of OS specific separators. Which in the case of Windows is notionally the backslash. The fact that you gave it a prefix containing forward slashes is confusing things. It works best with a list of individual folder names and let it provide the os.sep. Try splitting your partial path before feeding it to join. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From nik at naturalnet.de Sat Jul 20 22:37:30 2013 From: nik at naturalnet.de (Dominik George) Date: Sat, 20 Jul 2013 22:37:30 +0200 Subject: [Tutor] slashes in paths In-Reply-To: References: <20130720200658.GD30129@keks.naturalnet.de> Message-ID: <20130720203730.GE30129@keks.naturalnet.de> Hi Jim, > But oddly, it makes all slashes forward if I end the path with a > forward slash, so it's not consistent with itself. It is, in the sense that it preserves any form of seperator that is already there. It just doesn't throw away what you want to be there: >>> import ntpath >>> ntpath.join("a", "b") 'a\\b' >>> ntpath.join("a/", "b") 'a/b' >>> ntpath.join("a/", "b", "c") 'a/b\\c' >>> "Explicit is better than implicit", so if you explicitly require it to use a certain seperator it uses it. If you don't, it implie the OS-native seperator. Remember that Windows NT is alright with forward slashes, even outside Python, so it is ok to preserve it if it is already there. (On a side note, I imported ntpath to force the NT pathname behaviour on my Linux Python. os.path always matches the *path module for the current OS, so there is no difference there.) > BTW, is ending a path with a slash a Linux requirement, or just a > convention, or a who-cares-either-way? I forget since I've been on > windows a long time, and it accepts either method. I guess the best > thing is to always end all paths with a forward slash, catching both > OS's and being consistent. It is a POSIX convention. I think a POSIX OS is allowed to rais ENOENT if you leave the last slash out when referencing a directory, but most shells and programs imply it. -nik -- Auf welchem Server liegt das denn jetzt?? Wenn es nicht ?bers Netz kommt bei Hetzner, wenn es nicht gelesen wird bei STRATO, wenn es klappt bei manitu. PGP-Fingerprint: 3C9D 54A4 7575 C026 FB17 FD26 B79A 3C16 A0C4 F296 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 905 bytes Desc: Digital signature URL: From eryksun at gmail.com Sat Jul 20 23:16:01 2013 From: eryksun at gmail.com (eryksun) Date: Sat, 20 Jul 2013 17:16:01 -0400 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: > soundpath = 'c:/python27/jimprogs/wav' > soundlist = os.listdir(soundpath) > soundfile13 = os.path.join(soundpath, soundlist[13]) > >>>> soundfile13 > 'c:/python27/jimprogs/wav\\bicycle_bell.wav' You can use os.path.normpath() to make it consistent: >>> soundpath = 'c:/python27/jimprogs/wav\\bicycle_bell.wav' >>> print os.path.normpath(soundpath) c:\python27\jimprogs\wav\bicycle_bell.wav > Oddly, the mixed path does work for playing sounds The NT runtime library takes care of translating the path, such as converting slash to backslash, ignoring repeated slashes, handling "..", and so on. NT itself uses backslashes. For exmaple, if you need to use the \\?\ prefix to get around the 260-character path limit, you have to use an absolute path that only uses backslash. This bypasses the normal translation and lets you access paths that are over 30,000 characters long. Here's an example that directly calls the library function used to translate a DOS-style path: RtlDosPathNameToNtPathName_U. It's exported from ntdll.dll (lovely name that is): Some ctypes defintions: from ctypes import * dos_to_nt = windll.ntdll.RtlDosPathNameToNtPathName_U # we need a buffer for the result class UNICODE_STRING(Structure): _fields_ = [ ('Length', c_ushort), ('MaximumLength', c_ushort), ('Buffer', POINTER(c_wchar)), ] ntpath = UNICODE_STRING() >>> soundpath = u'c:/python27/jimprogs/wav\\bicycle_bell.wav' >>> dos_to_nt(soundpath, byref(ntpath), None, None) 1 >>> ntpath.Length // 2 # length in bytes, not wchar_t 45 >>> print ntpath.Buffer[:45] \??\c:\python27\jimprogs\wav\bicycle_bell.wav The \?? prefix is the directory in NT's object namespace where it keeps symbolic links for DOS device names. If you have WinObj from sysinterals, take a look in "\GLOBAL??". You'll see that C: is a symbolic link, usually to the NT native path \Device\HarddiskVolume1. http://technet.microsoft.com/en-us/sysinternals/bb896657.aspx > It does work right if I end the path with a / - i.e. - soundpath = > 'c:/python27/jimprogs/wav/' but that's not required. Ending with a > forward slash or not seems to work either way, although I'm not sure > about Linux. If the path already ends in a '/' or '\', then ntpath.join (os.path.join) just appends to the existing separator: http://hg.python.org/cpython/file/ab05e7dd2788/Lib/ntpath.py#l94 From cybervigilante at gmail.com Sun Jul 21 01:37:39 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 20 Jul 2013 16:37:39 -0700 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: On 20 July 2013 13:46, Alan Gauld wrote: > The fact that you gave it a prefix containing forward > slashes is confusing things. It works best with a list > of individual folder names and let it provide the os.sep. > Try splitting your partial path before feeding it to join. Except I can't do that with 'C:' since the docs stay that on windows, only 'c:/' is accepted as the base path, not 'c:'. If only Bill Gates hadn't chosen '\', which is awkward to type and hard to make compatible - but I think he figured his wonderful DOS would be a Unix-killer, reign supreme, and there would be no compatibility problem. All I can say to that is, "thank God for competition." ;') Jim At one time if someone had a lame idea, you'd say, "That and ten cents will get you a cup of coffee." Now you have to say, "That and five dollars and forty-eight cents will get you a double creme-de-menthe Grande frappe, with a touch of juniper, nonfat creme, and low on the sugar." It just ain't the same. From nik at naturalnet.de Sun Jul 21 01:43:16 2013 From: nik at naturalnet.de (Dominik George) Date: Sun, 21 Jul 2013 01:43:16 +0200 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: Hi, the base path is \, and one exists for every drive. C:\foo is foo in C:'s root, C:foo is foo in C:'s current working directory. -nik Jim Mooney schrieb: >On 20 July 2013 13:46, Alan Gauld wrote: > >> The fact that you gave it a prefix containing forward >> slashes is confusing things. It works best with a list >> of individual folder names and let it provide the os.sep. >> Try splitting your partial path before feeding it to join. > >Except I can't do that with 'C:' since the docs stay that on windows, >only 'c:/' is accepted as the base path, not 'c:'. > >If only Bill Gates hadn't chosen '\', which is awkward to type and >hard to make compatible - but I think he figured his wonderful DOS >would be a Unix-killer, reign supreme, and there would be no >compatibility problem. All I can say to that is, "thank God for >competition." ;') > >Jim > >At one time if someone had a lame idea, you'd say, "That and ten >cents will get you a cup of coffee." Now you have to say, "That and >five dollars and forty-eight cents will get you a double >creme-de-menthe Grande frappe, with a touch of juniper, nonfat creme, >and low on the sugar." > >It just ain't the same. >_______________________________________________ >Tutor maillist - Tutor at python.org >To unsubscribe or change subscription options: >http://mail.python.org/mailman/listinfo/tutor -- Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at alchemy.com Sun Jul 21 02:24:12 2013 From: steve at alchemy.com (Steve Willoughby) Date: Sat, 20 Jul 2013 17:24:12 -0700 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: <859ACA34-B9AE-4078-982A-9D6BF87B3E67@alchemy.com> On 20-Jul-2013, at 16:37, Jim Mooney wrote: > If only Bill Gates hadn't chosen '\', which is awkward to type and > hard to make compatible - but I think he figured his wonderful DOS > would be a Unix-killer, reign supreme, and there would be no > compatibility problem. All I can say to that is, "thank God for > competition." ;') If I recall correctly, the earliest versions of DOS (which supported directories) allowed you to configure your environment to use / as a directory separator so you could really use A:/path/to/directory for things. That wasn't the default since, if I were to guess, they had already adopted the use of "/" to introduce command-line switches which was the existing syntax on systems like CP/M and various DEC operating systems of the same era, so it would be ambiguous as to whether "foo/bar" referred to a file "bar" in the directory "foo" or the file "foo" with a "/bar" switch applied to its usage. Having already burned that bridge, flipping the slash the other way to \ for directory separators probably seemed like a good compromise at the time. In hindsight, I think it would have been better to just make a clean break at some early point and just change to "/" for directories, but that's never quite as easy to see as the right move at the time. I'm just glad they didn't start out using the VMS-style directory syntax or our paths would look like $DISK:[foo.bar.baz]file.ext;2 From davea at davea.name Sun Jul 21 03:20:27 2013 From: davea at davea.name (Dave Angel) Date: Sat, 20 Jul 2013 21:20:27 -0400 Subject: [Tutor] slashes in paths In-Reply-To: <859ACA34-B9AE-4078-982A-9D6BF87B3E67@alchemy.com> References: <859ACA34-B9AE-4078-982A-9D6BF87B3E67@alchemy.com> Message-ID: On 07/20/2013 08:24 PM, Steve Willoughby wrote: > > On 20-Jul-2013, at 16:37, Jim Mooney wrote: >> If only Bill Gates hadn't chosen '\', which is awkward to type and >> hard to make compatible - but I think he figured his wonderful DOS >> would be a Unix-killer, reign supreme, and there would be no >> compatibility problem. All I can say to that is, "thank God for >> competition." ;') > > If I recall correctly, the earliest versions of DOS (which supported directories) allowed you to configure your environment to use / as a directory separator so you could really use A:/path/to/directory for things. That wasn't the default since, if I were to guess, they had already adopted the use of "/" to introduce command-line switches which was the existing syntax on systems like CP/M and various DEC operating systems of the same era, so it would be ambiguous as to whether "foo/bar" referred to a file "bar" in the directory "foo" or the file "foo" with a "/bar" switch applied to its usage. > A few more details, from my memory. DOS 1.x had various kludges to emulate CP/M, including the use of / for a switch character. When DOS 2 was introduced, the first to explicitly support a hard disk, and the new fat16, subdirectories were added. The default syntax for that was the backslash. When Wang's MSDOS was introduced (2.01), a pair of calls were added to set and test switchar, which was supposed to be used for command interpreting. The idea was to use '-' as the switch character, leaving / to be used to separate directories. Unfortunately, most commands in COMMAND.COM ignored the new function, and very few third party programs paid any attention at all. The OS paid no attention to the switch, and just treated the slash and backslash as interchangeable. Wang's switch character defaulted to "-", but they had little influence on the DOS application world. > Having already burned that bridge, flipping the slash the other way to \ for directory separators probably seemed like a good compromise at the time. In hindsight, I think it would have been better to just make a clean break at some early point and just change to "/" for directories, but that's never quite as easy to see as the right move at the time. > > I'm just glad they didn't start out using the VMS-style directory syntax or our paths would look like $DISK:[foo.bar.baz]file.ext;2 > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > -- DaveA From eryksun at gmail.com Sun Jul 21 04:42:50 2013 From: eryksun at gmail.com (eryksun) Date: Sat, 20 Jul 2013 22:42:50 -0400 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: On Sat, Jul 20, 2013 at 7:43 PM, Dominik George wrote: > the base path is \, and one exists for every drive. C:\foo is foo in C:'s > root, C:foo is foo in C:'s current working directory. NT process parameters only track a single working directory, while DOS maintained a current directory for each drive. This is emulated in NT by setting special environment variables such as "=C:" for drive C: (which is really \Device\HarddiskVolume1 in NT's native namespace). Using environment variables is a simple way to have this inherited in child processes. Since Microsoft's C runtime ignores these variables when copying the process environment, you won't find them in Python's os.environ. They're supposed to be 'hidden'. Here's an example of using kernel32 GetEnvironmentVariableW to read the value of "=C:": >>> os.chdir('C:\Python27') >>> from ctypes import * >>> cwd_c = (c_wchar * 260)() >>> windll.kernel32.GetEnvironmentVariableW(u'=C:', cwd_c, 260) 11 >>> print cwd_c.value C:\Python27 The cmd shell gives you read-only access: >>> os.system('echo %=C:%') C:\Python27 0 In practice, ntdll's RtlGetFullPathName_U uses these variables, but RtlSetCurrentDirectory_U doesn't *set* them. NT leaves it up to the application to maintain this state. cmd.exe is an obvious example of a Win32 application that uses this. Also, if you use Microsoft's C runtime, it sets these for you when you _wchdir(). Since CPython rolls its own nt.chdir, it has to manage these "magic" variables: http://hg.python.org/cpython/file/ab05e7dd2788/Modules/posixmodule.c#l890 This trick isn't perfect. You can get silly results if you manually modify the values (use SetEnvironmentVariableW via ctypes). I'll leave that as an exercise. From nsivaram.net at gmail.com Sun Jul 21 07:31:18 2013 From: nsivaram.net at gmail.com (Sivaram Neelakantan) Date: Sun, 21 Jul 2013 11:01:18 +0530 Subject: [Tutor] suggestions for splitting file based on date References: <87a9litkta.fsf@gmail.com> Message-ID: <8761w44gyh.fsf@gmail.com> On Sat, Jul 20 2013,Walter Prins wrote: [snipped 21 lines] > You can do things manually as Peter's suggested, there's some value to be > had from learning to do things "by hand". However, if you're going to > continue to work with matplotlib and stock data I highly recommend you > eventually pick up and learn Pandas: http://pandas.pydata.org/ > > Another useful link: https://bitbucket.org/hrojas/learn-pandas > Pandas seems quite good, I was able to get a basic graph out with just the following lines colnames =['a','b','c','d','e','f','g'] df = pd.read_csv("cnx_100.dat", names=colnames, index_col=0,parse_dates=True) plt.plot(df.index,df['e']) plt.show() The above plotted the index_day_close against the date. I had to do the colnames part as I was getting the following errors. Some rows have 6 cols, some 7. CParserError: Error tokenizing data. C error: Expected 6 fields in line 1350, saw 7 [snipped 6 lines] sivaram -- From marc.tompkins at gmail.com Mon Jul 22 03:18:48 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Sun, 21 Jul 2013 18:18:48 -0700 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: On Sat, Jul 20, 2013 at 4:37 PM, Jim Mooney wrote: > If only Bill Gates hadn't chosen '\', which is awkward to type and > hard to make compatible - but I think he figured his wonderful DOS > would be a Unix-killer, reign supreme, and there would be no > compatibility problem. All I can say to that is, "thank God for > competition." ;') > I've run across this many times before - the idea that Gates saw UNIX off in the distance as his great competitor. It just isn't true. Digital Research's CP/M (the first OS I had hands-on experience with) was the 1,000-pound gorilla in the nascent PC market; DOS was consciously designed to be an easy transition from CP/M both for users and programmers. (This was not done out of the goodness of Microsoft's heart, of course - it was a way to poach the CP/M customer base, and it worked like a charm. WordStar - the most popular word processor for CP/M, and up to that point the top-selling application in the world - was famously ported over by changing a single byte, though - equally famously - nobody remembers what that byte was. http://www.joelonsoftware.com/articles/fog0000000054.html) UNIX was in an entirely separate universe, and furthermore UNIX didn't need Bill Gates' help to kill itself - AT&T's ridiculous licensing arrangements were doing that job quite nicely. No way in Hell was UNIX ever going to be a successful, popular desktop operating system for the masses, as DOS ended up being. If it weren't for a brilliant, profane Finno-Swede (or is that Swedo-Finn?) named Linus Torvalds, UNIX would probably be an academic footnote by now. Yes, I'm aware that plenty of big iron still runs various flavors of actual UNIX, and I'm equally aware that the Internet predates Linux... but if it weren't for the commercial Internet, which is mostly powered by Linux, very few of us would be able to access that big iron and the Internet would be reserved for academics, the military, and the filthy rich. But back in the late 1970s, no way in Hell did Gates see Linux on the horizon. He saw CP/M, and the choices that he (and MS in general) made at that time were intended to be compatible with CP/M, not incompatible with UNIX. You wanna blame somebody, blame Gary Kildall. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Mon Jul 22 04:41:27 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sun, 21 Jul 2013 19:41:27 -0700 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: On 21 July 2013 18:18, Marc Tompkins wrote: > But back in the late 1970s, no way in Hell did Gates see Linux on the > horizon. He saw CP/M, and the choices that he (and MS in general) made at > that time were intended to be compatible with CP/M, not incompatible with > UNIX. You wanna blame somebody, blame Gary Kildall. Good link - to think that I was going to buy a Timex Sinclair 1000. Looked like a hot item at the time. Must have had good ads ;') And I remember the Kildall business, of him flying about in a plane while Gates stole a march with IBM. DOS sure was crippled, though, even compared to drdos. And IE after it. I remember having resumeable downloads on Fidonet with zmodem, well over thirty years ago, before the Internet, yet IE couldn't do that for decades. Every time a download failed I'd think "Why could ancient Zmodem resume and IE couldn't?" Go figure. Then I became a webmaster and discovered IE was not compatible with standards, and its own versions weren't even compatible with each other - so you wrote one website for all the other browsers, then three more for IE. Ugh. -- Jim At one time if someone had a lame idea, you'd say, "That and ten cents will get you a cup of coffee." Now you have to say, "That and five dollars and forty-eight cents will get you a double creme-de-menthe Grande frappe, with a touch of juniper, nonfat creme, and low on the sugar." It just ain't the same. From marc.tompkins at gmail.com Mon Jul 22 06:57:38 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Sun, 21 Jul 2013 21:57:38 -0700 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: On Sun, Jul 21, 2013 at 7:41 PM, Jim Mooney wrote: > On 21 July 2013 18:18, Marc Tompkins wrote: > > > But back in the late 1970s, no way in Hell did Gates see Linux on the > > horizon. He saw CP/M, and the choices that he (and MS in general) made > at > > that time were intended to be compatible with CP/M, not incompatible with > > UNIX. You wanna blame somebody, blame Gary Kildall. > > Good link - to think that I was going to buy a Timex Sinclair 1000. > Looked like a hot item at the time. Must have had good ads ;') And I > remember the Kildall business, of him flying about in a plane while > Gates stole a march with IBM. DOS sure was crippled, though, even > compared to drdos. And IE after it. I remember having resumeable > downloads on Fidonet with zmodem, well over thirty years ago, before > the Internet, yet IE couldn't do that for decades. Every time a > download failed I'd think "Why could ancient Zmodem resume and IE > couldn't?" Go figure. Then I became a webmaster and discovered IE was > not compatible with standards, and its own versions weren't even > compatible with each other - so you wrote one website for all the > other browsers, then three more for IE. Ugh. > > Zmodem was (and remains) a proprietary algorithm; the licensing to include it in IE would have been prohibitive*. AND incompatible with Internet standards; if we're going to rag on IE for lousy compliance with standards - and we should - we really can't complain about failing to introduce a proprietary file transfer protocol... Resumable downloads require the ability to tell the sender to (re)start sending from an arbitrary file location; you can't do that unless the sender (the Web host) understands what you're asking for - which means that, even if IE had supported resumable downloading from the get-go, it would only have worked when downloading from Microsoft IIS (the first version of which came out in 1996 or so), and working with IIS is already enough of a nightmare for the rest of us... To this day, resumable downloads don't work _evverywhere_ with ANY browser that I'm aware of; they _mostly_ work, but not always. This is not really a Thing we can blame Microsoft for, I'm afraid. * "Prohibitive" in this case is a relative term, since in 1995 Microsoft had more money than God. However, they were giving IE away for free - and paying per-copy royalties on something you give away for free is... counterintuitive to most business types. -- > Jim > > At one time if someone had a lame idea, you'd say, "That and ten > cents will get you a cup of coffee." Now you have to say, "That and > five dollars and forty-eight cents will get you a double > creme-de-menthe Grande frappe, with a touch of juniper, nonfat creme, > and low on the sugar." > > It just ain't the same. > Dunno about Arizona, but here in L.A. a 20-oz. (you don't have to call it "venti" if you don't want to) drip coffee costs $1.75 at Starbucks, same as at McDonalds. (You can still get a 10-cent cup of coffee* at Philippe's Original downtown - a place I highly recommend, for just about everything EXCEPT the coffee - but it tastes like industrial runoff. Freakin' Farmer Brothers.) http://www.philippes.com/ * On the on-line menu, it's 45 cents, but in person it's 10. Go figure. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eryksun at gmail.com Mon Jul 22 12:51:33 2013 From: eryksun at gmail.com (eryksun) Date: Mon, 22 Jul 2013 06:51:33 -0400 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: On Sun, Jul 21, 2013 at 9:18 PM, Marc Tompkins wrote: > > But back in the late 1970s, no way in Hell did Gates see Linux on the > horizon. He saw CP/M, and the choices that he (and MS in general) made at > that time were intended to be compatible with CP/M, not incompatible with > UNIX. You wanna blame somebody, blame Gary Kildall. Using '/' switches primarily comes from DEC operating systems, such as TOPS-10 on the PDP-10. TOPS-10 had commands with dozens of switches (not just 2 or 3 like DOS commands). Of course in DEC systems the notation for paths is completely different (e.g. DSKA:DOC.TXT[14,5,DIRA,DIRB]), so using '/' for switches was a non-issue. CP/M itself didn't use '/' switches in its internal CCP commands, even if some 3rd party programs did. Neither did COMMAND.COM in Tim Paterson's 86-DOS. Microsoft added the switches (but Paterson was there in 81-82). By the time PC-DOS 2.0 shipped there were switches for FORMAT, COPY, DIR, DISKCOMP, DISKCOPY, BACKUP, CHKDSK, PRINT, RESTORE, and TREE (is that ironic?). From oscar.j.benjamin at gmail.com Mon Jul 22 13:59:07 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 22 Jul 2013 12:59:07 +0100 Subject: [Tutor] hi In-Reply-To: <000001ce7ee7$d2233170$76699450$@orange.mu> References: <000001ce7ee7$d2233170$76699450$@orange.mu> Message-ID: On 12 July 2013 11:08, Vick wrote: > Hi, Hi Vick, Sorry for the delayed response I didn't see this email until a while after you sent it. > I have written a code to perform a numerical solution to 1st order Ordinary > Differential Equations (ODEs). I have written codes for the famous Runge > Kutta 4th order and the DOPRI 8(7)13. However the codes are for 1 variable > only; viz. the ?y? variable. It would be good to show us the code for this in your email (not in an attachment). Also the code should be complete: the code uses a function rungkdp8 which is not shown. Where did that come from? Until you know what you're doing you should test your integrator on a simpler set of equations. Also when posting to a mailing list you should simplify the problem as much as possible which would mean using simpler ODEs. In short read this: http://sscce.org/ The code attached is incorrect as it applies each stage of the Runge-Kutta method to all time-steps sequentially before moving on to the next stage. It should apply each stage for one time step and then use the output of that for the next time-step. In other words you need to change the order of the loops. Here is an example of how to implement an Euler stepper that should hopefully get you started on the right track. I've deliberately avoided using numpy in this example even though it makes most of this easier. I did this to focus on the basics of Python which you should learn first. # integrator.py nan = float('nan') # indices for the two variables in the system POS = 0 VEL = 1 VARS = ['pos', 'vel'] # system has one parameter OMEGA OMEGA = 10 # Frequency in Hz is OMEGA / 2pi # function for the derivative of the system def shm(t, x, dxdt): '''Equation of motion for a mass in a spring with freq OMEGA''' dxdt[POS] = x[VEL] # Python uses square brackets for indexing dxdt[VEL] = - OMEGA**2 * x[POS] # Initial conditions for the problem x0 = [nan] * 2 x0[POS] = 1 x0[VEL] = 0 # This is the function that you should replace with e.g. rk4 or dopri8 def euler(t1, x1, t2): '''Take 1 Euler step from x1 at t1 to x2 at t2''' # Create empty lists for answer x2 = [nan] * len(x1) dxdt = [nan] * len(x1) # Call derivative function to fill in dxdt shm(t1, x1, dxdt) # Compute x2 and return dt = t2 - t1 for n in range(len(x1)): x2[n] = x1[n] + dt * dxdt[n] return x2 def solve(ts, x0): '''Compute states corresponding to the times in ts x0 is the state at ts[0] (the initial condition). ''' # Create an empty list of lists for the solution Xt = [x0[:]] # The initial conditions # Loop through timesteps computing the next value for n in range(len(ts) - 1): Xt.append(euler(ts[n], Xt[n], ts[n+1])) return Xt def print_line(*items): print(', '.join(str(i) for i in items)) def print_solution(ts, Xt, vars): print_line('t', *vars) for t, x in zip(times, Xt): print_line(t, *x) # Numerical parameters for the solution DT = 0.001 t0 = 0 T = 1 times = [t0] while times[-1] < T: times.append(times[-1] + DT) # Solve and print Xt = solve(times, x0) print_solution(times, Xt, VARS) > I have written another code in Excel VBA for DOPRI 8 for a multi-variable > capability. However I have not been able to reproduce it in Python. I?m > having trouble in making arrays or lists, I don?t know which is supposed to > work. > > I have attached my Excel VBA code for a multi-variable numerical integrator > in PDF format. This does work in Excel. I have also attached my python code. I'm not familiar with Excel VBA but I think that this code suffers from the same problem that the loop over stages takes place outside the loop over time-steps so I believe that it is incorrect. > Can anyone help me out please? I happen to know quite a lot about this subject so feel free to ask more questions. Oscar From cybervigilante at gmail.com Mon Jul 22 19:47:02 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 22 Jul 2013 10:47:02 -0700 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: On 22 July 2013 03:51, eryksun wrote: > On Sun, Jul 21, 2013 at 9:18 PM, Marc Tompkins > wrote: > > CP/M itself didn't use '/' switches in its internal CCP commands, even > if some 3rd party programs did. Neither did COMMAND.COM in Tim > Paterson's 86-DOS. Microsoft added the switches (but Paterson was > there in 81-82). By the time PC-DOS 2.0 shipped there were switches > for FORMAT, COPY, DIR, DISKCOMP, DISKCOPY, BACKUP, CHKDSK, PRINT, > RESTORE, and TREE (is that ironic?). > I forgot about TREE. But figured piping C:\Python27>tree /f > pytree.txt might be illuminating. I piped since it took forever to print because I have python(x,y). Unfortunately, I got tiny numbers and A with umlauts instead of the nice path outlines in the dos box: ? ? ? ? ????tests Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Mon Jul 22 20:14:08 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 22 Jul 2013 11:14:08 -0700 Subject: [Tutor] unichr not working as expected Message-ID: I tried translating the odd chars I found in my dos tree /f listing to symbols, but I'm getting this error. The chars certainly aren't over 10000, The ord is only 13 - so what's wrong here? def main(): zark = '' for x in "????": zark += unichr(ord(x)-45) print(zark) unichr() arg not in range(0x10000) (narrow Python build) -- Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From eryksun at gmail.com Mon Jul 22 20:23:34 2013 From: eryksun at gmail.com (eryksun) Date: Mon, 22 Jul 2013 14:23:34 -0400 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: On Mon, Jul 22, 2013 at 1:47 PM, Jim Mooney wrote: > I forgot about TREE. But figured piping C:\Python27>tree /f > pytree.txt > might be illuminating. I piped since it took forever to print because I have > python(x,y). Unfortunately, I got tiny numbers and A with umlauts instead of > the nice path outlines in the dos box: > > ? ? ? ? ????tests I assume you're using U.S. code pages. In the console run more pytree.txt Then run chcp 1252 more pytree.txt The OEM codepage is 437. The lower 7-bit range is ASCII, but what do with the upper half of the 8-bit range wasn't standardized in the 80s. IBM used it for an assortment of letters, symbols, and drawing characters: http://en.wikipedia.org/wiki/Code_page_437#Standard_code_page http://en.wikipedia.org/wiki/Box-drawing_characters For Windows, Microsoft no longer needed the drawing characters, so they switched to code pages that extended ISO character sets, such as Window 1252 (extends Latin-1) http://en.wikipedia.org/wiki/Code_page_1252 That's the default encoding if you open a text file with notepad. From marc.tompkins at gmail.com Mon Jul 22 20:26:44 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Mon, 22 Jul 2013 11:26:44 -0700 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: On Mon, Jul 22, 2013 at 10:47 AM, Jim Mooney wrote: I forgot about TREE. But figured piping C:\Python27>tree /f > pytree.txt > might be illuminating. I piped since it took forever to print because I > have python(x,y). Unfortunately, I got tiny numbers and A with umlauts > instead of the nice path outlines in the dos box: > > ? ? ? ? ????tests > > That's an encoding problem; TREE is apparently not Unicode-aware, and uses the old ASCII-US code page for values above 127. I suspect that bringing ancient command-line utilities into the Notepad++ is my default text editor; I was able to see the line-drawing characters properly after I selected Encoding/Character sets/Western European/OEM-US. No idea what you'd need to do in other text editors... TREE /? displays the following: Graphically displays the folder structure of a drive or path. TREE [drive:][path] [/F] [/A] /F Display the names of the files in each folder. /A Use ASCII instead of extended characters. Using /f /a will give you a readable file, no matter which text editor you use. I suspect that it hasn't been updated for Unicode for two reasons: 1) updating TREE to use Unicode for line-drawing would break compatibility for people who pipe its output into other CLI programs in the *nix style 2) although Microsoft could get around that by adding another command-line switch, there probably isn't an awful lot of demand - who uses TREE anymore, except in the context of discussions like this? If you haven't already read it, may I suggest Joel's intro to Unicode? http://www.joelonsoftware.com/articles/Unicode.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Mon Jul 22 20:30:16 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 22 Jul 2013 11:30:16 -0700 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: On 22 July 2013 11:26, Marc Tompkins wrote: > > > > If you haven't already read it, may I suggest Joel's intro to Unicode? > http://www.joelonsoftware.com/articles/Unicode.html > I had a bad feeling I'd end up learning Unicode ;') -- Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc.tompkins at gmail.com Mon Jul 22 20:33:45 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Mon, 22 Jul 2013 11:33:45 -0700 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: On Mon, Jul 22, 2013 at 11:30 AM, Jim Mooney wrote: > On 22 July 2013 11:26, Marc Tompkins wrote: > >> >> >> >> If you haven't already read it, may I suggest Joel's intro to Unicode? >> http://www.joelonsoftware.com/articles/Unicode.html >> > > I had a bad feeling I'd end up learning Unicode ;') > It's not as painful as you might think! Try it - you'll like it! Actually, once you start getting used to working in Unicode by default, having to deal with programs that are non-Unicode-aware feels extremely irritating. -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan.gauld at btinternet.com Mon Jul 22 20:52:50 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Mon, 22 Jul 2013 19:52:50 +0100 Subject: [Tutor] unichr not working as expected In-Reply-To: References: Message-ID: On 22/07/13 19:14, Jim Mooney wrote: > zark += unichr(ord(x)-45) > > unichr() arg not in range(0x10000) (narrow Python build) What if ord() is returning a value less than 45? What does unichr() do with negative vales?> Just a guess... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From eryksun at gmail.com Mon Jul 22 20:55:12 2013 From: eryksun at gmail.com (eryksun) Date: Mon, 22 Jul 2013 14:55:12 -0400 Subject: [Tutor] unichr not working as expected In-Reply-To: References: Message-ID: On Mon, Jul 22, 2013 at 2:14 PM, Jim Mooney wrote: > I tried translating the odd chars I found in my dos tree /f listing to > symbols, but I'm getting this error. The chars certainly aren't over 10000, > The ord is only 13 - so what's wrong here? > > def main(): > zark = '' > for x in "????": > zark += unichr(ord(x)-45) > > print(zark) > > unichr() arg not in range(0x10000) (narrow Python build) 13 is a carriage return (i.e. '\r'). That's giving you a negative result when you subtract 45. By the way, why are you doing that? The characters shown in your string are 195 (0xC3) and 196 (0xC4) in cp1252. Refer to the following chart for their value as cp437 box drawing characters: http://en.wikipedia.org/wiki/Box-drawing_character#DOS Also, the upper limit for Unicode ordinals on a narrow build is 0x10000, not 10000. In a narrow build, you get the 16-bit basic multilingual plane (BMP). \U literals that are above that produce UTF-16 surrogate pairs. From cybervigilante at gmail.com Mon Jul 22 20:57:37 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 22 Jul 2013 11:57:37 -0700 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: I had a bad feeling I'd end up learning Unicode ;') > > > It's not as painful as you might think! Try it - you'll like it! > Actually, once you start getting used to working in Unicode by default, > having to deal with programs that are non-Unicode-aware feels extremely > irritating. > I'll have to, to write a python program to translate the tree output. I tried More, but since I have Python(x,y) I gave up after holding down the enter key for five minutes. I tried piping More but that didn't work right. I probably forgot DOS. but since DOS isn't commonly used, a python program would be better anyway. -- Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Mon Jul 22 20:27:35 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 22 Jul 2013 11:27:35 -0700 Subject: [Tutor] close, but no cigar Message-ID: Okay, I'm getting there, but this should be translating A umlaut to an old DOS box character, according to my ASCII table, but instead it's print small 'u': def main(): zark = '' for x in "????": print(unichr(ord(u'x')-3), end=' ') result: u u u u -- Jim What the Nations of the World Fail to Realize, is That we are Living in a Closed System: In March 2013, researchers from the Stanford University Hopkins Marine Station issued a report on Bluefin tuna caught off the California coast and tested for radioactive cesium. The report found that Bluefin tuna were 100 per cent contaminated, that not one was cesium-free. The report did not address such questions as whether cesium would continue to accumulate in tuna or whether it was appearing in other fish species. -------------- next part -------------- An HTML attachment was scrubbed... URL: From sunil.techspk at gmail.com Mon Jul 22 20:59:03 2013 From: sunil.techspk at gmail.com (Sunil Tech) Date: Tue, 23 Jul 2013 00:29:03 +0530 Subject: [Tutor] 3 Dimensional Dictionaries In-Reply-To: References: Message-ID: THANK YOU ALL for your time. The first format which I pasted was from the DB The second format(exactly the same), is to be sent to the view. If the logic can be fitted in One or two methods it'll help me to easily understand & to apply. so I request you to help... On Sunday, July 21, 2013, Alan Gauld wrote: > On 20/07/13 11:17, Sunil Tech wrote: >> >> Hi Everyone, >> >> I have a list of dictionaries like >> >> world = >> [{'continent':'Asia','continent_code':1,'ocean':'Pacific','country':'India','country_code':1,'state':'Kerala', >> 'state_pin':500001}, > >> >> i am trying to to make it in this format >> >> world = [{'continent':'Asia', 'ocean':'Pacific', >> 'countries':[{'country':'India', >> 'states':[{'state':'Kerala', 'state_pin':500001}, >> {'state':'Karnataka', 'state_pin':500002}] >> }] >> }, > >> Please help me in this regard. > > In what regard? Where do you need the help? > You seem to know the data format you want? > > The only thing I'd suggest is to consider using classes if you are familiar with them. > > world => list of continents > continent => class containing countries > country => class containing states > state => class containing data > > It then becomes easier to build helper methods to extract/manipulate the data you are interested in. > > Alternatively, if you have a large amount of data a database may be another option. Swap table for class above and use SQL to manage the data. > > But other than those suggestions I don't know what kind of > help you want? > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan.gauld at btinternet.com Mon Jul 22 21:05:01 2013 From: alan.gauld at btinternet.com (ALAN GAULD) Date: Mon, 22 Jul 2013 20:05:01 +0100 (BST) Subject: [Tutor] Cannot understand what this means In-Reply-To: <52B2907AE37EB94B8690A0B45EE20918C943CC@HKNPRD0111MB386.apcprd01.prod.exchangelabs.com> References: <52B2907AE37EB94B8690A0B45EE20918C93A8C@HKNPRD0111MB386.apcprd01.prod.exchangelabs.com>, <52B2907AE37EB94B8690A0B45EE20918C943CC@HKNPRD0111MB386.apcprd01.prod.exchangelabs.com> Message-ID: <1374519901.84291.YahooMailNeo@web186001.mail.ir2.yahoo.com> CCing the list. Please always use ReplyAll to include the list. Alan Gauld Author of the Learn To Program website http://www.alan-g.me.uk/ >________________________________ > From: #PATHANGI JANARDHANAN JATINSHRAVAN# >To: Alan Gauld >Sent: Monday, 22 July 2013, 17:14 >Subject: RE: [Tutor] Cannot understand what this means > > >Hi >I have doubts only on the arguments part.? > >OK, But what about them? Do you understand what command line arguments are? > def main(): > >>? ? # This command-line parsing code is provided. >>? ? # Make a list of command line arguments, omitting the [0] element >>? ? # which is the script itself. >>? ? args = sys.argv[1:] This fetches the command line arguments missing the first which is the program name. Do you understand slicing? Do you understand how sys.argv works - its similar to the argc, argv mechanism of ?C++ which you've maybe seen before. >> ? ?if not args: >>? ? ? print 'usage: [--summaryfile] file [file ...]' >>? ? ? sys.exit(1) The logic here says that if args is empty then print the message and exit >> ? ?# Notice the summary flag and remove it from args if it is present. >> ? ?summary = False >>? ? if args[0] == '--summaryfile': >>? ? ? summary = True >>? ? ? del args[0] I have no idea why this is here but for whatever reason he is setting? the summary flag to True if the first argument was --summaryfile Now, is there anything else you don't understand?? The more specific you make your questions the better we can? answer them.? Alan G. From cybervigilante at gmail.com Mon Jul 22 21:22:35 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 22 Jul 2013 12:22:35 -0700 Subject: [Tutor] unichr not working as expected In-Reply-To: References: Message-ID: On 22 July 2013 11:52, Alan Gauld wrote: > On 22/07/13 19:14, Jim Mooney wrote: > > zark += unichr(ord(x)-45) >> >> >> unichr() arg not in range(0x10000) (narrow Python build) >> > > > What if ord() is returning a value less than 45? > What does unichr() do with negative vales?> > > Just a guess... > I already changed to u for the char, so I got a bigger number, and only subtracted 3 from umlaut, which should have given me the dos line-drawing dash, but now my problem is I can't seem to set encoding for that: import sys sys.setdefaultencoding('cp437') gives me the error: AttributeError: 'module' object has no attribute 'setdefaultencoding' Jim > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > > ______________________________**_________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/**mailman/listinfo/tutor > -- Jim What the Nations of the World Fail to Realize, is That we are Living in a Closed System: In March 2013, researchers from the Stanford University Hopkins Marine Station issued a report on Bluefin tuna caught off the California coast and tested for radioactive cesium. The report found that Bluefin tuna were 100 per cent contaminated, that not one was cesium-free. The report did not address such questions as whether cesium would continue to accumulate in tuna or whether it was appearing in other fish species. -------------- next part -------------- An HTML attachment was scrubbed... URL: From oscar.j.benjamin at gmail.com Mon Jul 22 21:25:13 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Mon, 22 Jul 2013 20:25:13 +0100 Subject: [Tutor] hi In-Reply-To: <000901ce8703$4e599b30$eb0cd190$@orange.mu> References: <000001ce7ee7$d2233170$76699450$@orange.mu> <000901ce8703$4e599b30$eb0cd190$@orange.mu> Message-ID: On 22 July 2013 18:45, Vick wrote: > > The DOPRI 8(7)13 is the Dormand Prince 8th order with 13 stages method for > solving ordinary differential equations. It is a member of the Runge-Kutta > family of ODE solvers. I have been able to reproduce it in Python with a > multi-variable capability. On a particular test equation the error of the > RK4 is about 1e-4 whereas the DOPRI 8(7)13 is about 1e-13. At this error level you should consider using Kahan-summation for the increment step since rounding error is likely to be significant: http://en.wikipedia.org/wiki/Kahan_summation_algorithm Have you verified that the result is of order 8? i.e. you should compute the local or global error as a function of dt (or h as it's often known) and verify that log(error) vs log(dt) has the appropriate slope. Out of interest why do you choose to use this particular method? It isn't commonly used so I wonder if there a reason that it is particularly suited to your problem or if you would be better with something else. > I have then used the multi-variable DOPRI to compute for the n-body problem > using 1st order ODEs which requires you to have 4 equations for 1 planet. I > have already made a 10-body problem through an n-body solver in python and > using DOPRI as my integrator. Do you have a way of testing accuracy in this situation (conservation of energy perhaps)? Oscar From marc.tompkins at gmail.com Mon Jul 22 21:48:05 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Mon, 22 Jul 2013 12:48:05 -0700 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: On Mon, Jul 22, 2013 at 11:57 AM, Jim Mooney wrote: > I had a bad feeling I'd end up learning Unicode ;') > >> >> >> It's not as painful as you might think! Try it - you'll like it! >> Actually, once you start getting used to working in Unicode by default, >> having to deal with programs that are non-Unicode-aware feels extremely >> irritating. >> > > I'll have to, to write a python program to translate the tree output. I > tried More, but since I have Python(x,y) I gave up after holding down the > enter key for five minutes. I tried piping More but that didn't work right. > I probably forgot DOS. but since DOS isn't commonly used, a python program > would be better anyway. > You'd be better off skipping TREE entirely and going pure-Python. TREE - being Unicode-naive - can't deal with any foreign-alphabet characters beyond the few baked in alongside the box-drawing characters; they all get turned into question marks. I'm guessing that's not an issue on your own computer, but if you ever want to deal with files from other people... -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc.tompkins at gmail.com Mon Jul 22 22:45:26 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Mon, 22 Jul 2013 13:45:26 -0700 Subject: [Tutor] close, but no cigar In-Reply-To: References: Message-ID: On Mon, Jul 22, 2013 at 11:27 AM, Jim Mooney wrote: > Okay, I'm getting there, but this should be translating A umlaut to an old > DOS box character, according to my ASCII table, but instead it's print > small 'u': > > def main(): > zark = '' > for x in "????": > print(unichr(ord(u'x')-3), end=' ') > > result: u u u u > When you type "?" in a Python string (without specifying which encoding you're trying to represent), it doesn't necessarily have the same ordinal value as the line-drawing character that gets mistakenly displayed as "?" in your text editor. Depending on which Python version you happen to be using at the moment (and therefor depending on the default encoding), "?" might be a Unicode Latin Capital Letter A With Diaeresis (U+00C4), or it might be character code 0x8E, or it might be 0xC4... For a quick visualization of what I'm talking about, just fire up the Character Map program and find "?" in the following fonts: Arial, Terminal, and Roman. Float your mouse cursor over it each time to see the character code associated with it. If you insist on parsing the output of TREE (instead of letter Python do things in a modern, Unicode-aware way), here's how I would do it: inFileName = "/Users/Marc/Desktop/rsp/tree.txt" with open(inFileName, 'r') as inFile: inString = inFile.read().decode('cp437') print inString This printed out the line-drawing characters just fine; my test Cyrillic filename remained a string of question marks, because TREE itself had trashed that filename and there wasn't anything for .decode() to decode. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Mon Jul 22 22:55:21 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 22 Jul 2013 13:55:21 -0700 Subject: [Tutor] close, but no cigar In-Reply-To: References: Message-ID: On 22 July 2013 13:45, Marc Tompkins wrote: > > inFileName = "/Users/Marc/Desktop/rsp/tree.txt" > with open(inFileName, 'r') as inFile: > inString = inFile.read().decode('cp437') > print inString > > I already tried something similar and got an error: with open('../pytree.txt') as pytree, open('newpytree.txt','w') as pyout: for line in pytree: for char in line: newchar = char.decode('cp437') pyout.write(newchar) UnicodeEncodeError: 'ascii' codec can't encode character u'\u2502' in position 0: ordinal not in range(128) What's my error here, since I want to write to a file. I have python(x,y) so printing that is kind of time-taking and not too illuminating ;') Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Mon Jul 22 23:00:20 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 22 Jul 2013 14:00:20 -0700 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: On 22 July 2013 12:48, Marc Tompkins wrote: > > > You'd be better off skipping TREE entirely and going pure-Python. TREE - > being Unicode-naive - can't deal with any foreign-alphabet characters > beyond the few baked in alongside the box-drawing characters; they all get > turned into question marks. I'm guessing that's not an issue on your own > computer, but if you ever want to deal with files from other people... > so I should just walk the python27 tree and write my own box drawing chars? Or is there a more global alternative to DOS box-drawing chars to illustrate a tree structure, other than graphic processing? -- Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc.tompkins at gmail.com Mon Jul 22 23:11:07 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Mon, 22 Jul 2013 14:11:07 -0700 Subject: [Tutor] close, but no cigar In-Reply-To: References: Message-ID: On Mon, Jul 22, 2013 at 1:55 PM, Jim Mooney wrote: > On 22 July 2013 13:45, Marc Tompkins wrote: > >> >> inFileName = "/Users/Marc/Desktop/rsp/tree.txt" >> with open(inFileName, 'r') as inFile: >> inString = inFile.read().decode('cp437') >> print inString >> >> I already tried something similar and got an error: > > with open('../pytree.txt') as pytree, open('newpytree.txt','w') as pyout: > for line in pytree: > for char in line: > newchar = char.decode('cp437') > pyout.write(newchar) > > UnicodeEncodeError: 'ascii' codec can't encode character u'\u2502' in > position 0: ordinal not in range(128) > > What's my error here, since I want to write to a file. I have python(x,y) > so printing that is kind of time-taking and not too illuminating ;') > The error's in your error message: Python has decoded the string properly, but (since you haven't specified an encoding) is trying to encode to the default, which in Python < 3 is 'ascii'... which has a great big blank space where all characters over 128 should be. One way to deal with this is to specify an encoding: newchar = char.decode('cp437').encode('utf-8') which works just fine for me. Another way is to tell Python what it should do in case of encoding errors: - ignore (just drop the offending characters from the output): newchar = char.decode('cp437').encode('ascii', 'ignore') - replace (all offending characters get replaced with something non-offending - like TREE did with my Cyrillic filename): newchar = char.decode('cp437').encode('ascii', 'replace') - xmlcharrefreplace (replace offending characters with their XML encodings): newchar = char.decode('cp437').encode('ascii', 'xmlcharrefreplace') This page is your friend: http://docs.python.org/2/howto/unicode.html and so is this one: http://docs.python.org/2/library/codecs.html#standard-encodings -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc.tompkins at gmail.com Mon Jul 22 23:15:49 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Mon, 22 Jul 2013 14:15:49 -0700 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: On Mon, Jul 22, 2013 at 2:00 PM, Jim Mooney wrote: > so I should just walk the python27 tree and write my own box drawing > chars? Or is there a more global alternative to DOS box-drawing chars to > illustrate a tree structure, other than graphic processing? > You could do worse... Again, my issue with TREE is that it willfully throws away information (non-ASCII characters in filenames) before passing it on to you. As a result, the tree you print out may not correspond to the actual filesystem structure, and there's nothing you can do about it. If that's not a problem for you, go for it. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Mon Jul 22 23:22:12 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 22 Jul 2013 14:22:12 -0700 Subject: [Tutor] close, but no cigar In-Reply-To: References: Message-ID: On 22 July 2013 14:11, Marc Tompkins wrote: The error's in your error message: Python has decoded the string properly, > but (since you haven't specified an encoding) is trying to encode to the > default, which in Python < 3 is 'ascii'... which has a great big blank > space where all characters over 128 should be. > Thanks. I now have the full tree, with nice little DOS graphics, visible in Notepad++. That's useful. Besides having learned a bit of unicode. Win 7 doesn't show the full tree and if you expand it, it's kind of tiny. Why, in an aging population, is everyone assuming you can see spidery light grey text on websites and tiny symbols and offsets in a file explorer ;') Since I have python(x,y) which makes for an enormous tree, this makes it easier to go through and see what I've got and where it is ;') -- Jim What the Nations of the World Fail to Realize, is That we are Living in a Closed System: In March 2013, researchers from the Stanford University Hopkins Marine Station issued a report on Bluefin tuna caught off the California coast and tested for radioactive cesium. The report found that Bluefin tuna were 100 per cent contaminated, that not one was cesium-free. The report did not address such questions as whether cesium would continue to accumulate in tuna or whether it was appearing in other fish species. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Mon Jul 22 23:26:27 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 22 Jul 2013 14:26:27 -0700 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: On 22 July 2013 14:15, Marc Tompkins wrote: > > > You could do worse... Again, my issue with TREE is that it willfully > throws away information (non-ASCII characters in filenames) before passing > it on to you. As a result, the tree you print out may not correspond to > the actual filesystem structure, and there's nothing you can do about it. > If that's not a problem for you, go for it. > Oh, I meant walk the tree in python, not using DOS at all. Not everyone has it so it's a good habit to get into not using it. I was just wondering the best way to print the tree. I'll probably have to resort to graphics if I don't like the DOS box characters. Jim > > > -- Jim What the Nations of the World Fail to Realize, is That we are Living in a Closed System: In March 2013, researchers from the Stanford University Hopkins Marine Station issued a report on Bluefin tuna caught off the California coast and tested for radioactive cesium. The report found that Bluefin tuna were 100 per cent contaminated, that not one was cesium-free. The report did not address such questions as whether cesium would continue to accumulate in tuna or whether it was appearing in other fish species. -------------- next part -------------- An HTML attachment was scrubbed... URL: From eryksun at gmail.com Mon Jul 22 23:34:14 2013 From: eryksun at gmail.com (eryksun) Date: Mon, 22 Jul 2013 17:34:14 -0400 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: On Mon, Jul 22, 2013 at 3:48 PM, Marc Tompkins wrote: > > You'd be better off skipping TREE entirely and going pure-Python. TREE - > being Unicode-naive - can't deal with any foreign-alphabet characters beyond > the few baked in alongside the box-drawing characters; they all get turned > into question marks. I'm guessing that's not an issue on your own computer, > but if you ever want to deal with files from other people... Just to clarify, tree isn't completely Unicode naive. It writes Unicode to the console, presuming you're using a font that supports it, such as Consolas. The problem is that it doesn't configure ULIB to use the current console code page when streaming to a pipe or file. It leaves it at the default setting, which is to encode Unicode to the OEM code page (e.g. 437). If it would use the current code page instead, you could try setting it to 65001 (a somewhat-buggy UTF-8 code page). Streaming the UTF-16 directly (like cmd's /U option) would be even better, but that would require modifying ULIB. That said, I doubt anyone at Microsoft cares. It's a dinky utility written back in the early 90s. Probably no one has even looked at the code in years. From marc.tompkins at gmail.com Mon Jul 22 23:45:49 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Mon, 22 Jul 2013 14:45:49 -0700 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: On Mon, Jul 22, 2013 at 2:34 PM, eryksun wrote: > > Just to clarify, tree isn't completely Unicode naive. It writes > Unicode to the console, presuming you're using a font that supports > it, such as Consolas. > Interesting! Indeed - I just moved my test Cyrillic file to a different folder (so TREE wouldn't scroll off the page) and sho 'nuff there it is - ????, ? ???? ?????.RSP. The problem is that it doesn't configure ULIB to use the current > console code page when streaming to a pipe or file. It leaves it at > the default setting, which is to encode Unicode to the OEM code page > (e.g. 437). If it would use the current code page instead, you could > try setting it to 65001 (a somewhat-buggy UTF-8 code page). Streaming > the UTF-16 directly (like cmd's /U option) would be even better, but > that would require modifying ULIB. > > That said, I doubt anyone at Microsoft cares. It's a dinky utility > written back in the early 90s. Probably no one has even looked at the > code in years. > So it does the job correctly, as long as everything fits on one screen. Good to know... -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Mon Jul 22 23:50:18 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 22 Jul 2013 14:50:18 -0700 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: > Just to clarify, tree isn't completely Unicode naive. It writes > Unicode to the console, presuming you're using a font that supports > it, such as Consolas. > I'm sticking to 20 pt Lucida Console on a big, full screen DOS box with navy letters and cyan background. If you have a big screen might as well use it. Why the DOS box defaults to being so tiny is beyond me. To heck with this nutty new design idea of black backgrounds and tiny greyish letters. I didn't like it in DOS and I don't like it on the web. It's like being in the Negative Zone. Must be some Goth thing. Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From ramit.prasad at jpmorgan.com Mon Jul 22 23:41:11 2013 From: ramit.prasad at jpmorgan.com (Prasad, Ramit) Date: Mon, 22 Jul 2013 21:41:11 +0000 Subject: [Tutor] Interpret the contents of a line In-Reply-To: References: Message-ID: <5B80DD153D7D744689F57F4FB69AF474185B0F68@SCACMX008.exchad.jpmchase.net> Makarand Datar wrote: > Hi, > > I am working on a parser and my input file is a description about bunch of things written line by > line. So for instance, consider the following two lines and what the corresponding variables in python > should look like. > > example Line1: position = 1, 1, rotation = 90, 0, 0, mass = 120; Here I want the variables to be > position = [1,1,0], rotation = [90,0,0], mass = [120] > example Line2: position = 1, 1, 2, mass = 120, rotation = 90, 0; Here I want the variables to be > position = [1,1,2], rotation = [90,0,0], mass = [120] > example Line3: mass = 120, rotation = 90, 0; Here I want the variables to be position = [0,0,0], > rotation = [90,0,0], mass = [120] > > I know the maximum number of arguments possible for each variable. For example, in the first line > above, only two numbers for position ares specified; that means that the third entry in position list > is zero. So I need to handle these cases while reading in the file as well. Additionally, the text > might not always be in the same order; like shown in the second line above. And finally, sometimes > numbers for one variable might not exist at all; like shown in line 3. Here, the position is then read > as position = [0,0,0]. > How do I implement such stuff? Is there some smart way of doing this? All I have in mind is this: read > a line as comma or a space delimited list of words variable and then some how use if else + len(list) > to figure out whats going on. But that seems way too tedious and I feel that there might be an easier > way to read such things. > > Any help is highly appreciated. > Thank you You can use the regular expression library to split. If you can delimit the keys ("position") that would work best but if you cannot change source that is fine too. Based on your sample lines, you can split on text and convert to dictionary. >>> import re >>> line 'position = 1, 1, rotation = 90, 0, 0, mass = 120' >>> re.split('([a-z]*)', line) ['', 'position', ' = 1, 1, ', 'rotation', ' = 90, 0, 0, ', 'mass', ' = 120'] >>> l = _ >>> dict(zip(l[1::2], l[2::2]) ) # skip first element {'position': ' = 1, 1, ', 'rotation': ' = 90, 0, 0, ', 'mass': ' = 120'} Now your key is the element name and the value is the string. Now is remove equals sign and commas and split on space (or however you want to do it). And if the returned list in split is not big enough you just add zero(es) to the end. Ramit This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email. From alan.gauld at btinternet.com Tue Jul 23 00:12:23 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Mon, 22 Jul 2013 23:12:23 +0100 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: On 22/07/13 22:50, Jim Mooney wrote: > I'm sticking to 20 pt Lucida Console on a big, full screen DOS box with > navy letters and cyan background. If you have a big screen might as well > use it. Why the DOS box defaults to being so tiny is beyond me. Depends what you mean by tiny. On a 1024x768 display its not that small, and with a multi-tasking OS you want to be able to do more than one thing at a time don't you? Single window display is a throwback to the stone age of computing ..... err, or maybe forward to Windows 8? Go figure.... > letters. I didn't like it in DOS and I don't like it on the web. It's > like being in the Negative Zone. When I used an old VT220 on a VAX I always used to reverse the display to show black characters on a light (green or amber) screen. It used to freak out my colleagues who were traditionalist green on black men... When I got upgraded to a VT340(!) I also set it to display 132 chars wide. Now that was really heretical! :-) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From cybervigilante at gmail.com Tue Jul 23 00:22:00 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 22 Jul 2013 15:22:00 -0700 Subject: [Tutor] close, but no cigar In-Reply-To: References: Message-ID: On 22 July 2013 14:11, Marc Tompkins wrote: > > One way to deal with this is to specify an encoding: > newchar = char.decode('cp437').encode('utf-8') > Works fine, but I decided to add a dos graphics dash to the existing dash to expand the tree visually. Except I got a complaint from IDLE that I should add this: # -*- coding: utf-8 -*- Will that always work? Setting coding in a comment? Or am I looking at a Linux hash line? Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Tue Jul 23 00:27:40 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 22 Jul 2013 15:27:40 -0700 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: On 22 July 2013 15:12, Alan Gauld wrote: When I used an old VT220 on a VAX I always used to reverse the display to > show black characters on a light (green or amber) screen. It used to freak > out my colleagues who were traditionalist green on black men... > I think a lot of old sonar guys went to computer school on the WWII GI bill. Thought they were still in a submarine ;') -- Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From eryksun at gmail.com Tue Jul 23 01:30:03 2013 From: eryksun at gmail.com (eryksun) Date: Mon, 22 Jul 2013 19:30:03 -0400 Subject: [Tutor] close, but no cigar In-Reply-To: References: Message-ID: On Mon, Jul 22, 2013 at 6:22 PM, Jim Mooney wrote: > Works fine, but I decided to add a dos graphics dash to the existing dash to > expand the tree visually. Except I got a complaint from IDLE that I should > add this: I presume you copied a Unicode box character from a console window (not DOS), prompting IDLE to save your file as UTF-8 and insert an encoding declaration. > # -*- coding: utf-8 -*- > > Will that always work? Setting coding in a comment? Or am I looking at a > Linux hash line? Yes, it always works. It's handled by the parser: >>> src = '# -*- coding: utf-8 -*-' >>> st = parser.suite(src) >>> parser.st2list(st) [339, [257, [0, '']], 'utf-8'] >>> symbol.sym_name[339] 'encoding_decl' From alan.gauld at btinternet.com Tue Jul 23 01:31:23 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Tue, 23 Jul 2013 00:31:23 +0100 Subject: [Tutor] close, but no cigar In-Reply-To: References: Message-ID: On 22/07/13 23:22, Jim Mooney wrote: > # -*- coding: utf-8 -*- > > Will that always work? Setting coding in a comment? Or am I looking at a > Linux hash line? Same concept. It's a special comment that will only take effect when at the top of a file. Actually, I'm not sure how it plays with the hash-bang line - I assume being the second line is OK too... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From marc.tompkins at gmail.com Tue Jul 23 01:39:05 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Mon, 22 Jul 2013 16:39:05 -0700 Subject: [Tutor] close, but no cigar In-Reply-To: References: Message-ID: On Mon, Jul 22, 2013 at 3:22 PM, Jim Mooney wrote: > On 22 July 2013 14:11, Marc Tompkins wrote: > >> >> One way to deal with this is to specify an encoding: >> newchar = char.decode('cp437').encode('utf-8') >> > > Works fine, but I decided to add a dos graphics dash to the existing dash > to expand the tree > visually. Except I got a complaint from IDLE that I should add this: > > # -*- coding: utf-8 -*- > > Will that always work? Setting coding in a comment? Or am I looking at a > Linux hash line? > > I speak under correction here, but: what you're setting there is the encoding for the script file itself (and - the real point here - any strings you specify, without explicit encoding, inside the script), NOT the default encoding that Python is going to use while executing your script. Unless I'm very much mistaken, Python will still use the default encoding ('ascii' in your case) when reading strings from external files. -------------- next part -------------- An HTML attachment was scrubbed... URL: From davea at davea.name Tue Jul 23 03:13:55 2013 From: davea at davea.name (Dave Angel) Date: Mon, 22 Jul 2013 21:13:55 -0400 Subject: [Tutor] close, but no cigar In-Reply-To: References: Message-ID: On 07/22/2013 02:27 PM, Jim Mooney wrote: > Okay, I'm getting there, but this should be translating A umlaut to an old > DOS box character, according to my ASCII table, but instead it's print > small 'u': > > def main(): > zark = '' > for x in "????": > print(unichr(ord(u'x')-3), end=' ') > > result: u u u u > You're not using any of the A-umlaut characters in that code. You're repeatedly taking ord of the literal u'x' And naturally, u is 3 characters less than x. -- DaveA From sunil.techspk at gmail.com Tue Jul 23 04:44:33 2013 From: sunil.techspk at gmail.com (Sunil Tech) Date: Tue, 23 Jul 2013 08:14:33 +0530 Subject: [Tutor] 3 Dimensional Dictionaries In-Reply-To: References: Message-ID: On Tuesday, July 23, 2013, Sunil Tech wrote: > THANK YOU ALL for your time. > > The first format which I pasted was from the DB > > The second format(exactly the same), is to be sent to the view. > > If the logic can be fitted in One or two methods it'll help me to easily understand & to apply. > > so I request you to help... > > > On Sunday, July 21, 2013, Alan Gauld wrote: >> On 20/07/13 11:17, Sunil Tech wrote: >>> >>> Hi Everyone, >>> >>> I have a list of dictionaries like >>> >>> world = >>> [{'continent':'Asia','continent_code':1,'ocean':'Pacific','country':'India','country_code':1,'state':'Kerala', >>> 'state_pin':500001}, >> >>> >>> i am trying to to make it in this format >>> >>> world = [{'continent':'Asia', 'ocean':'Pacific', >>> 'countries':[{'country':'India', >>> 'states':[{'state':'Kerala', 'state_pin':500001}, >>> {'state':'Karnataka', 'state_pin':500002}] >>> }] >>> }, >> >>> Please help me in this regard. >> >> In what regard? Where do you need the help? >> You seem to know the data format you want? >> >> The only thing I'd suggest is to consider using classes if you are familiar with them. >> >> world => list of continents >> continent => class containing countries >> country => class containing states >> state => class containing data >> >> It then becomes easier to build helper methods to extract/manipulate the data you are interested in. >> >> Alternatively, if you have a large amount of data a database may be another option. Swap table for class above and use SQL to manage the data. >> >> But other than those suggestions I don't know what kind of >> help you want? >> >> -- >> Alan G >> Author of the Learn to Program web site >> http://www.alan-g.me.uk/ >> >> _______________________________________________ >> Tutor maillist - Tutor at python.org >> To unsubscribe or change subscription options: >> http://mail.python.org/mailman/listinfo/tutor >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc.tompkins at gmail.com Tue Jul 23 05:01:31 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Mon, 22 Jul 2013 20:01:31 -0700 Subject: [Tutor] close, but no cigar In-Reply-To: References: Message-ID: On Mon, Jul 22, 2013 at 6:13 PM, Dave Angel wrote: > On 07/22/2013 02:27 PM, Jim Mooney wrote: > >> Okay, I'm getting there, but this should be translating A umlaut to an old >> DOS box character, according to my ASCII table, but instead it's print >> small 'u': >> >> def main(): >> zark = '' >> for x in "????": >> print(unichr(ord(u'x')-3), end=' ') >> >> result: u u u u >> >> > You're not using any of the A-umlaut characters in that code. You're > repeatedly taking ord of the literal u'x' > > And naturally, u is 3 characters less than x. > Oooh - I missed that... -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Tue Jul 23 05:20:49 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 23 Jul 2013 13:20:49 +1000 Subject: [Tutor] unichr not working as expected In-Reply-To: References: Message-ID: <51EDF691.80404@pearwood.info> On 23/07/13 05:22, Jim Mooney wrote: > I already changed to u for the char, so I got a bigger number, and only > subtracted 3 from umlaut, which should have given me the dos line-drawing > dash, but now my problem is I can't seem to set encoding for that: > > import sys > sys.setdefaultencoding('cp437') > > gives me the error: > > AttributeError: 'module' object has no attribute 'setdefaultencoding' Don't touch setdefaultencoding. It is hidden for a reason. And if you insist on living dangerously, don't set it to weird legacy encodings like cp437. When Python starts up, it needs to set the encoding used, but you *cannot* set it to arbitrary encodings. Setting it to arbitrary encodings can cause all sorts of weird, hard to diagnose bugs, so to prevent that, Python deletes the setdefaultencoding function after using it. The documentation is clear that there are no user-serviceable parts here: http://docs.python.org/2/library/sys.html#sys.setdefaultencoding And in Python 3 it became a no-op, then finally deleted for good, gone forever, and thanks be to feck. http://bugs.python.org/issue9549 Apparently it only exists because when Unicode was first introduced to Python, the developers couldn't decide whether to use ASCII, Latin1 or UTF-8 as the internal encoding, which just goes to show that even the top Python devs can be foolish when it comes to Unicode. So they put in an experimental function to set the default encoding, and *literally forgot to remove it* for the public release. (That's according to the Effbot, Fredrik Lundh, one of the early Python luminaries.) -- Steven From steve at pearwood.info Tue Jul 23 06:00:34 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 23 Jul 2013 14:00:34 +1000 Subject: [Tutor] unichr not working as expected In-Reply-To: References: Message-ID: <51EDFFE2.8080605@pearwood.info> On 23/07/13 04:14, Jim Mooney wrote: > I tried translating the odd chars I found in my dos tree /f listing to > symbols, but I'm getting this error. The chars certainly aren't over > 10000, The ord is only 13 - so what's wrong here? > > def main(): > zark = '' > for x in "????": > zark += unichr(ord(x)-45) This is broken in three ways that I can see. Firstly, assuming you are using Python 2.7 (as you have said in the past), "????" does not mean what you think it means. In Python 3, this is a Unicode string containing four individual characters: LATIN CAPITAL LETTER A WITH GRAVE LATIN CAPITAL LETTER A WITH DIAERESIS LATIN CAPITAL LETTER A WITH DIAERESIS LATIN CAPITAL LETTER A WITH DIAERESIS Why you have duplicates, I do not know :-) But in Python 2, that's not what you will get. What you get depends on your environment, and is unpredictable. For example, on my system, using a Linux terminal interactively with the terminal set to UTF-8, I get: py> for c in "??": # removing duplicates ... print c, ord(c) ... ? 195 ? 128 ? 195 ? 132 Yes, that's right, I get FOUR (not two) "characters" (actually bytes). But if I change the terminal settings to, say, ISO-8859-7: py> for c in "??": ... print c, ord(c) ... ? 195 128 ? 195 132 the bytes stay the same (195, 128, 195, 132) but the *meaning* of those bytes change completely. So, the point is, if you are running Python 2.7, what you get from a byte string like "??" is unpredictable. What you need is a Unicode string u"??", which will exactly what it looks like. That's the first issue. Second issue, you build up a string using this idiom: zark = '' for c in something: zark += c Even though this works, this is a bad habit to get into and you should avoid it: it risks being unpredictably slower than continental drift, and in a way that is *really* hard to diagnose. I've seen a case of this fool the finest Python core developers for *weeks*, regarding a reported bug where Python was painfully slow but only for SOME but not all Windows users. The reason why accumulating strings using + can be slow when there are a lot of strings is because it is a Shlemiel the painter's algorithm: http://www.joelonsoftware.com/articles/fog0000000319.html? The reason why sometimes it is *not* slow is that CPython 2.3 and beyond includes a clever optimization trick which can *sometimes* fix this issue, but it depends on details for the operating system's memory handling, and of course it doesn't apply to other implementations like Jython, IronPython, PyPy and Nuitka. So do yourself a favour and get out of the habit of accumulating strings in a for loop using + since it will bite you one day. (Adding one or two strings is fine.) Problem number three: you generate characters using this: unichr(ord(x)-45) but that gives you a negative number if ord(x) is less than 45, which gives you exactly the result you see: py> unichr(-1) Traceback (most recent call last): File "", line 1, in ValueError: unichr() arg not in range(0x10000) (narrow Python build) (By the way, you're very naughty. The code you show *cannot possibly generate the error you claim it generates*. Bad Jim, no biscuit!) I don't understand what the ord(x)-45 is intended to do. The effect is to give the 45th previous character, e.g. the 45th character before 'n' is 'A'. But characters below chr(45) don't have anything 45 characters previous, so you need to rethink what you are trying to do. -- Steven From cybervigilante at gmail.com Tue Jul 23 06:43:49 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 22 Jul 2013 21:43:49 -0700 Subject: [Tutor] unichr not working as expected In-Reply-To: <51EDF691.80404@pearwood.info> References: <51EDF691.80404@pearwood.info> Message-ID: Steven D'Aprano When Python starts up, it needs to set the encoding used, but you *cannot* > set it to arbitrary encodings. Setting it to arbitrary encodings can cause > all sorts of weird, hard to diagnose bugs, so to prevent that, Python > deletes the setdefaultencoding function after using it. > Oh, I found that out already. I found a trick to set it anyway, using reload, then discovered python wasn't printing Anything, even "Hello, world", even after restarting the interpreter ;') I had awful visions of reinstalling python and re downloading python(x,y) on my Really slow connection, but the problem went away after awhile. So I will have to concur in not messing with it. Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Tue Jul 23 08:10:30 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Mon, 22 Jul 2013 23:10:30 -0700 Subject: [Tutor] unichr not working as expected In-Reply-To: <51EDFFE2.8080605@pearwood.info> References: <51EDFFE2.8080605@pearwood.info> Message-ID: On 22 July 2013 21:00, Steven D'Aprano wrote: > > (By the way, you're very naughty. The code you show *cannot possibly > generate the error you claim it generates*. Bad Jim, no biscuit!) > I know. I need a personal github. When I get frustrated I try so many things in quick succession I lose track. The worst is when I got something working, but was moving so fast, I forgot how ;') But I have my tree diagram. The next step is to walk the Py27 directory so I can make my own tree diagram, without DOS Tree and it's dumb graphics. Walking a directory seems to be a fave assignment of professors, judging by questions I see, but I don't want to look at anyone else's.solution. I'll just see what the walk generator puts out and try to figure it. That's the fun part. -- Jim When I was young dad told me if a moth fluttered onto my sleeping lips at night it would suck out my soul. Dad was such a kidder. But I still flee from moths. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Tue Jul 23 09:09:00 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Tue, 23 Jul 2013 00:09:00 -0700 Subject: [Tutor] object size in python is in what units? Message-ID: I've noticed that when I create a number of objects from a class, one after another, they're at different IDs, but the IDs all appear to be equidistant, so that they all appear to have the same size. But what does that size represent? is it bytes? ie - if I run this over and over I get different id numbers but they are always 40 apart. Or is that distance machine-specific as to units?: #Using Python 2.7 on Win 7 from __future__ import division, print_function class Kronk: def __init__(self, zar): self.zar = zar def getzar(self): return self.zar def getself(self): return id(self) lardKronk = Kronk('ziggle') newKronk = Kronk('barf') budKronk = Kronk('baloney') print(lardKronk.getzar()) print(newKronk.getzar()) print(budKronk.getzar()) print(lardKronk.getself()) print(newKronk.getself()) print(budKronk.getself()) '''result: ziggle barf baloney 41599624 - different results for each run but always 40 apart 41599664 41599704 ''' -- Jim When I was young dad told me if a moth fluttered onto my sleeping lips at night it would suck out my soul. Dad was such a kidder. But I still flee from moths. -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Tue Jul 23 09:30:20 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 23 Jul 2013 17:30:20 +1000 Subject: [Tutor] close, but no cigar In-Reply-To: References: Message-ID: <51EE310C.9010006@pearwood.info> On 23/07/13 09:39, Marc Tompkins wrote: > On Mon, Jul 22, 2013 at 3:22 PM, Jim Mooney wrote: > >> On 22 July 2013 14:11, Marc Tompkins wrote: >> >>> >>> One way to deal with this is to specify an encoding: >>> newchar = char.decode('cp437').encode('utf-8') >>> >> >> Works fine, but I decided to add a dos graphics dash to the existing dash >> to expand the tree >> visually. Except I got a complaint from IDLE that I should add this: >> >> # -*- coding: utf-8 -*- >> >> Will that always work? Setting coding in a comment? Or am I looking at a >> Linux hash line? >> >> > I speak under correction here, but: what you're setting there is the > encoding for the script file itself (and - the real point here - any > strings you specify, without explicit encoding, inside the script), NOT the > default encoding that Python is going to use while executing your script. > Unless I'm very much mistaken, Python will still use the default encoding > ('ascii' in your case) when reading strings from external files. Correct. The encoding declaration ONLY tells Python how to read the script. Remember, source code is text, but has to be stored on disk as bytes. If you only use ASCII characters, pretty much every program will agree what the bytes represent (since IBM mainframes using EBCDIC are pretty rare, and few programs expect double-byte encodings). But if you include non-ASCII characters, your text editor has to convert them to bytes. How does it do so? Nearly every editor is different, a plain text file doesn't have any way of storing metadata such as the encoding. Contrast this to things like JPEG files, which can store metadata like the camera you used to take the photo. So, some programmer's editors have taken up the convention of using so-called "mode lines" to record editor settings as comments in source code, usually in the first couple or last couple of lines. Especially on Linux systems, Emacs and Vim uses frequently include such mode lines. Python stole this idea from them. If the first or second line in the source code file is a comment containing something like "encoding = SPAM", then Python will read that source code using encoding SPAM. The form shown above -*- coding: utf-8 -*- is copied from Emacs. Python is pretty flexible though. However, the encoding must be a known encoding (naturally), and the comment must be in the first or second line. You can't use it anywhere else. Well, you actually can, since it is a comment, but it will have no effect anywhere else. -- Steven From hugo.yoshi at gmail.com Tue Jul 23 09:36:56 2013 From: hugo.yoshi at gmail.com (Hugo Arts) Date: Tue, 23 Jul 2013 09:36:56 +0200 Subject: [Tutor] object size in python is in what units? In-Reply-To: References: Message-ID: On Tue, Jul 23, 2013 at 9:09 AM, Jim Mooney wrote: > I've noticed that when I create a number of objects from a class, one > after another, they're at different IDs, but the IDs all appear to be > equidistant, so that they all appear to have the same size. But what does > that size represent? is it bytes? ie - if I run this over and over I get > different id numbers but they are always 40 apart. Or is that distance > machine-specific as to units?: > > In Python, the id is just a number that identifies the object. You can't count on it to represent object size, or anything else for that matter. The only requirement for python implementations is that different objects can not have the same id. The CPython implementation uses the memory address of the object as its id because it's simple and satisfies the above requirement. Still, i'd say it's very unsafe to try to infer object sizes from this. It depends on how memory addresses work on your machine, what type of memory allocator is used and how much overhead it has, and many more factors. Furthermore, other python implementations (I believe Jython and IronPython among them) do not even use memory addresses as id numbers, but simply assign consecutive integers to each created object. In general, relying on the id number to tell you anything other than whether two variables point to the same object is highly unreliable at best and impossible in the general case. HTH, Hugo -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Tue Jul 23 09:40:54 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 23 Jul 2013 17:40:54 +1000 Subject: [Tutor] object size in python is in what units? In-Reply-To: References: Message-ID: <51EE3386.10706@pearwood.info> On 23/07/13 17:09, Jim Mooney wrote: > I've noticed that when I create a number of objects from a class, one after > another, they're at different IDs, but the IDs all appear to be > equidistant, so that they all appear to have the same size. But what does > that size represent? is it bytes? No no no, a thousand times no!!! IDs are just numeric IDs, that is all, like your social security number or driver's licence number. Don't think of them as having any meaning at all, except that they are unique during the lifetime of the object. Some Python implementations never reuse IDs, and they are numbered consecutively. IronPython numbers them from 40 (if memory serves me right), so you get 40, 41, 42, 43, ... while Jython numbers them consecutively from 1, 2, 3, ... PyPy does something similar. CPython does reuse IDs, and uses the memory address of the base of the object, as reported by the operating system, which leads to annoying people saying that id() returns the address of the object. This is WRONG. It does not. id() returns an arbitrary ID number. >ie - if I run this over and over I get > different id numbers but they are always 40 apart. Or is that distance > machine-specific as to units?: Yes. It depends on whether CPython gets a chance to reuse the same memory address or not, whether you have a 32-bit or 64-bit Python compiler, whether any other objects are created in between each one, and many other factors. -- Steven From steve at pearwood.info Tue Jul 23 09:42:19 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 23 Jul 2013 17:42:19 +1000 Subject: [Tutor] slashes in paths In-Reply-To: References: Message-ID: <51EE33DB.5020202@pearwood.info> On 23/07/13 04:33, Marc Tompkins wrote: > On Mon, Jul 22, 2013 at 11:30 AM, Jim Mooney wrote: > >> On 22 July 2013 11:26, Marc Tompkins wrote: >> >>> >>> >>> >>> If you haven't already read it, may I suggest Joel's intro to Unicode? >>> http://www.joelonsoftware.com/articles/Unicode.html >>> >> >> I had a bad feeling I'd end up learning Unicode ;') >> > > It's not as painful as you might think! Try it - you'll like it! > Actually, once you start getting used to working in Unicode by default, > having to deal with programs that are non-Unicode-aware feels extremely > irritating. What he said! Unicode brings order out of chaos. The old code page technology is horrible and needs to die. It was just barely acceptable back in ancient days when files were hardly ever transferred from machine to machine, and even then mostly transferred between machines using the same language. Even so, it didn't work very well -- ask Russians, who had three mutually incapable code pages. The basics of Unicode are very simple: - text strings contain characters; - what is written to disk contains bytes; - you need to convert characters to and from bytes, regardless of whether you are using ASCII or Unicode or something else; - the conversion uses a mapping of character to byte(s), and visa versa, called an encoding; - ASCII is an encoding too, e.g. byte 80 <=> "P"; - use the encode method to go from text to bytes, and decode to go the other way; - if you don't know what encoding is used, you cannot tell what the bytes actually mean; - although sometimes you can guess, with a variable level of success. Remember those rules, and you are three quarters of the way to being an expert. -- Steven From marc.tompkins at gmail.com Tue Jul 23 10:13:51 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Tue, 23 Jul 2013 01:13:51 -0700 Subject: [Tutor] object size in python is in what units? In-Reply-To: References: Message-ID: On Tue, Jul 23, 2013 at 12:09 AM, Jim Mooney wrote: > I've noticed that when I create a number of objects from a class, one > after another, they're at different IDs, but the IDs all appear to be > equidistant, so that they all appear to have the same size. But what does > that size represent? is it bytes? ie - if I run this over and over I get > different id numbers but they are always 40 apart. Or is that distance > machine-specific as to units?: > > > 41599624 - different results for each run but always 40 apart > 41599664 > 41599704 > ''' > When I saved your code as a file on my machine (Python 2.7 on Win8 64-bit) and ran it from the command line, I got the same result as you - three IDs, 40 apart every time. However, I initially pasted your code into PyScripter, and when I ran it inside PyScripter the results were different: if the first ID was x, then the second was x+160, and the third was x+200. (160 is obviously divisible by 40... I have no idea what conclusion to draw from that.) I simplified the test-harness portion as follows: kronkList = [] for kronknum in xrange(0,100): kronkList.append(Kronk(kronknum)) print(kronkList[kronknum].getzar()) print(kronkList[kronknum].getself()) and now the ID is incremented by 40 each time regardless how I run the script. Each time I run it, the last digit stays the same throughout the run (as you would expect, given that we're adding 40 each time), but the last digits are NOT consistent _between_ runs. It appears to always be an even number, though. As an experiment, I added a couple of extra methods and attributes to the Kronk class, but it didn't change anything - IDs still incremented by 40 each time. eryksun will no doubt chime in to tell us exactly how object IDs are derived in cPython, but I'll go out on a limb and say that they have nothing to do with the size of the object; also that one would be very silly to make a program rely in any way on predicting the next ID, since external factors may throw off the calculations (as when I invoked your original script from inside PyScripter). All that you can rely on is that each unique object (i.e. doesn't pass an "is" comparison with any other object) has a unique ID... and, for all I know, not even that. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Tue Jul 23 10:17:18 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Tue, 23 Jul 2013 01:17:18 -0700 Subject: [Tutor] object size in python is in what units? In-Reply-To: <51EE3386.10706@pearwood.info> References: <51EE3386.10706@pearwood.info> Message-ID: On 23 July 2013 00:40, Steven D'Aprano wrote: > > No no no, a thousand times no!!! IDs are just numeric IDs, that is all, > like your social security number or driver's licence number. Don't think of > them as having any meaning at all, except that they are unique during the > lifetime of the object. > Okay, ID stands for a location fixed until the object disappears, but we don't know where that location is. But what about object offsets from self? Is the beginning of self.spam always the same distance from the beginning of self.eggs? Or can I just forget the hard ground of assembler-metaphors entirely as I float off into abstractville? I guess the fixed lengths I kept getting on re-runs were coincidental but not to be relied on. -- Jim An excellent animation/graph of the accelerating disappearance of arctic sea ice, which controls climate in the Northern hemisphere: https://www.youtube.com/watch?v=YgiMBxaL19M It is incontrovertible that Something is happening. -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc.tompkins at gmail.com Tue Jul 23 10:19:29 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Tue, 23 Jul 2013 01:19:29 -0700 Subject: [Tutor] object size in python is in what units? In-Reply-To: References: Message-ID: On Tue, Jul 23, 2013 at 1:13 AM, Marc Tompkins wrote: > As an experiment, I added a couple of extra methods and attributes to the > Kronk class, but it didn't change anything - IDs still incremented by 40 > each time. eryksun will no doubt chime in to tell us exactly how object > IDs are derived in cPython, but I'll go out on a limb and say that they > have nothing to do with the size of the object; also that one would be very > silly to make a program rely in any way on predicting the next ID, since > external factors may throw off the calculations (as when I invoked your > original script from inside PyScripter). All that you can rely on is that > each unique object (i.e. doesn't pass an "is" comparison with any other > object) has a unique ID... and, for all I know, not even that. > A couple of clarifications I wish I'd made before hitting Send: I shouldn't have said that IDs are "derived" or "calculated"; they're "assigned", presumably on a first-come, first-served basis; when I said "external factors may throw off the calculations" I meant "may make your calculations non-congruent with reality". -------------- next part -------------- An HTML attachment was scrubbed... URL: From marc.tompkins at gmail.com Tue Jul 23 10:24:23 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Tue, 23 Jul 2013 01:24:23 -0700 Subject: [Tutor] object size in python is in what units? In-Reply-To: References: <51EE3386.10706@pearwood.info> Message-ID: On Tue, Jul 23, 2013 at 1:17 AM, Jim Mooney wrote: > On 23 July 2013 00:40, Steven D'Aprano wrote: > >> >> No no no, a thousand times no!!! IDs are just numeric IDs, that is all, >> like your social security number or driver's licence number. Don't think of >> them as having any meaning at all, except that they are unique during the >> lifetime of the object. >> > > Okay, ID stands for a location fixed until the object disappears, but we > don't know where that location is. But what about object offsets from self? > Is the beginning of self.spam always the same distance from the beginning > of self.eggs? Or can I just forget the hard ground of assembler-metaphors > entirely as I float off into abstractville? I guess the fixed lengths I > kept getting on re-runs were coincidental but not to be relied on. > As Steven said: they are NOT locations, they are IDs. There is no distance involved. You CAN find the size of an object, and you can find its location in memory (but you really shouldn't bother, or rely on it) but neither of those things has any particular relation to the ID, except accidentally. -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Tue Jul 23 10:26:12 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Tue, 23 Jul 2013 01:26:12 -0700 Subject: [Tutor] slashes in paths In-Reply-To: <51EE33DB.5020202@pearwood.info> References: <51EE33DB.5020202@pearwood.info> Message-ID: On 23 July 2013 00:42, Steven D'Aprano wrote: My, that is simple. The Python docs should start with simple explanations like that so people would keep on reading instead of throwing up their hands in despair ;') > > - use the encode method to go from text to bytes, and decode to go the > other way; > My favorite part. Real bytes, not imaginary IDs ;') Jim An excellent animation/graph of the accelerating disappearance of arctic sea ice, which controls climate in the Northern hemisphere: https://www.youtube.com/watch?v=YgiMBxaL19M -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Tue Jul 23 10:57:16 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Tue, 23 Jul 2013 01:57:16 -0700 Subject: [Tutor] object size in python is in what units? In-Reply-To: References: Message-ID: On 23 July 2013 01:13, Marc Tompkins wrote: When I saved your code as a file on my machine (Python 2.7 on Win8 64-bit) > and ran it from the command line, I got the same result as you - three IDs, > 40 apart every time. However, I initially pasted your code into > PyScripter, and when I ran it inside PyScripter the results were different: > if the first ID was x, then the second was x+160, and the third was x+200. > (160 is obviously divisible by 40... I have no idea what conclusion to draw > from that.) > I like PyScripter, but I suspect it injects a lot of behind-the-scenes code. At times it does something odd or just chokes, so if something looks funny, I run it from Wing 101, which nearly always gives the same result as the command line and has yet to choke. And the command line, of course, is the final arbiter once a program is complete. So far, Wing 101 has fulfilled my expectations as a basic Python IDE, so it's the one I'll buy if I fall into money. PyCharm might be good but they don't have a free version and I'm not learning a tryout I might have to ditch. Which means the folks at Wing are smarter marketers by giving away the basic version ;') I think that Joel guy at .joelonsoftware.com mentioned something to that effect. An excellent site to peruse when I'm feeling brain-dead and need a break. Speaking of brain dead, I was taking a break from Python by looking at a video on finite state machines and the very first example was tennis scoring. I think the $#@! insane tennis scoring was harder to understand than the subject. Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From wolfrage8765 at gmail.com Tue Jul 23 12:50:58 2013 From: wolfrage8765 at gmail.com (Jordan) Date: Tue, 23 Jul 2013 06:50:58 -0400 Subject: [Tutor] object size in python is in what units? In-Reply-To: References: Message-ID: <51EE6012.3060907@gmail.com> > > Speaking of brain dead, I was taking a break from Python by looking at > a video on finite state machines and the very first example was tennis > scoring. I think the $#@! insane tennis scoring was harder to > understand than the subject. That got me laughing pretty good, since I too was recently trying to learn Finite State Machines. I play tennis occasionally and still forget how to score properly! > > Jim > > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor -------------- next part -------------- An HTML attachment was scrubbed... URL: From eryksun at gmail.com Tue Jul 23 14:01:46 2013 From: eryksun at gmail.com (eryksun) Date: Tue, 23 Jul 2013 08:01:46 -0400 Subject: [Tutor] close, but no cigar In-Reply-To: <51EE310C.9010006@pearwood.info> References: <51EE310C.9010006@pearwood.info> Message-ID: On Tue, Jul 23, 2013 at 3:30 AM, Steven D'Aprano wrote: > But if you include non-ASCII characters, your text editor has to convert > them to bytes. How does it do so? Nearly every editor is different, a plain > text file doesn't have any way of storing metadata such as the encoding. > Contrast this to things like JPEG files, which can store metadata like the > camera you used to take the photo. File systems can be sneaky about this. NTFS has alternate data streams, which were added for compatibility with Mac resource forks. C:\>echo some text > test.txt C:\>echo 1252 > test.txt:encoding C:\>more test.txt some text C:\>more < test.txt:encoding 1252 From davea at davea.name Tue Jul 23 16:14:14 2013 From: davea at davea.name (Dave Angel) Date: Tue, 23 Jul 2013 10:14:14 -0400 Subject: [Tutor] unichr not working as expected In-Reply-To: References: <51EDFFE2.8080605@pearwood.info> Message-ID: On 07/23/2013 02:10 AM, Jim Mooney wrote: > On 22 July 2013 21:00, Steven D'Aprano wrote: > >> >> (By the way, you're very naughty. The code you show *cannot possibly >> generate the error you claim it generates*. Bad Jim, no biscuit!) >> > > I know. I need a personal github. git is free, and is usually installed on individual machines. It's also easy to set up and use for a single user. http://githowto.com/setup Where git gets complex is when multiple users are making simultaneous changes. And even there, it's about as complicated as it needs to be, and very powerful. Once you've installed git, you don't even need internet access. Even if you're in a multiplayer project, you only need it for some operations, not for the run of the mill snapshots you'd like to do. > When I get frustrated I try so many > things in quick succession I lose track. The worst is when I got something > working, but was moving so fast, I forgot how ;') > One git feature I haven't used in a long time is the ability to test MANY versions against a particular test. If I recall correctly, the test should return a nonzero return code (eg. sys.exit(5)) for error. Anyway, you start up the git script, telling it the test name, and it'll repeatedly get versions till it finds the one you're looking for. -- DaveA From steve at pearwood.info Tue Jul 23 16:19:39 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 24 Jul 2013 00:19:39 +1000 Subject: [Tutor] object size in python is in what units? In-Reply-To: References: Message-ID: <51EE90FB.1020408@pearwood.info> On 23/07/13 18:19, Marc Tompkins wrote: > A couple of clarifications I wish I'd made before hitting Send: I > shouldn't have said that IDs are "derived" or "calculated"; they're > "assigned", presumably on a first-come, first-served basis; Actually, in CPython they are derived, from the memory location, which annoys me no end :-) But in Jython and IronPython they are assigned, when and as requested. Some day, I'll invent my own version of Python, and make object IDs be consecutive prime numbers. That'll teach people to rely on them. :-) -- Steven From oscar.j.benjamin at gmail.com Tue Jul 23 16:21:47 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Tue, 23 Jul 2013 15:21:47 +0100 Subject: [Tutor] object size in python is in what units? In-Reply-To: References: Message-ID: On 23 July 2013 09:13, Marc Tompkins wrote: > > As an experiment, I added a couple of extra methods and attributes to the > Kronk class, but it didn't change anything - IDs still incremented by 40 > each time. eryksun will no doubt chime in to tell us exactly how object IDs > are derived in cPython, but I'll go out on a limb and say that they have > nothing to do with the size of the object; also that one would be very silly > to make a program rely in any way on predicting the next ID, since external > factors may throw off the calculations (as when I invoked your original > script from inside PyScripter). All that you can rely on is that each > unique object (i.e. doesn't pass an "is" comparison with any other object) > has a unique ID... and, for all I know, not even that. > It has been pointed out that the value returned by id() should not be considered as a memory address since it isn't always a memory address in every Python implementation. However in CPython it is just the memory address i.e. the numeric value of the PyObject* pointer for the object. The reason for the patterns that you observe are related to CPython's memory pool allocator. You can read the long source comment describing this here: http://hg.python.org/cpython/file/a5681f50bae2/Objects/obmalloc.c#l367 Essentially anything over 512 bytes will be allocated using a call to the system (OS/runtime) malloc. Anything under 512 bytes will be allocated into a memory pool managed by the interpreter. The code I linked to shows the exact strategy used for this and if you read this you'll have a vague understanding of the results you see. You can check the size in bytes of an object with sys.getsizeof e.g.: >>> import sys >>> sys.getsizeof(lardKronk) 36 Note that I'm running Python 2.7, Windows XP, 32-bit to get that result and it is different on different platforms. So the size was 36 bytes but I also get the spacing of 40 bytes (when I run your script with 'python kronk.py'). The reason I get a spacing of 40 when only 36 bytes are needed for each allocation is that allocation uses 8-byte alingment so everything works in multiples of 8. This is described/defined here: http://hg.python.org/cpython/file/a5681f50bae2/Objects/obmalloc.c#l471 The reason that you get different results when running from pyscripter (it's the same with ipython) is presumably that in order to initialise pyscripter the interpreter will perform lots of other allocations before running your script. Consequently it will likely have lots of half-full memory pools with spaces that can fit a Kronk allocation here and there and uses those before creating a new pool. When you just run it as python kronk.py a lot less happens before your script runs. Oscar From steve at pearwood.info Tue Jul 23 16:28:06 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 24 Jul 2013 00:28:06 +1000 Subject: [Tutor] unichr not working as expected In-Reply-To: References: <51EDFFE2.8080605@pearwood.info> Message-ID: <51EE92F6.1040602@pearwood.info> On 24/07/13 00:14, Dave Angel wrote: > On 07/23/2013 02:10 AM, Jim Mooney wrote: >> On 22 July 2013 21:00, Steven D'Aprano wrote: >> >>> >>> (By the way, you're very naughty. The code you show *cannot possibly >>> generate the error you claim it generates*. Bad Jim, no biscuit!) >>> >> >> I know. I need a personal github. > > git is free, and is usually installed on individual machines. It's also easy to set up and use for a single user. I prefer Mercurial (also known as hg). The Python core developers use it for compiler development, and in my opinion, hg is easier to use than git. (Although, in fairness, they're both pretty easy.) hg is cleaner than git, being written in Python, while git is an unholy mess of C, Perl and bash that will suck your brain out if you try to understand it, unless you are Linus Torvalds. And there are some awesome hg tutorials out there: http://hginit.com/ http://mercurial.selenic.com/wiki/Tutorial (I trust that everyone remembers enough high school chemistry to understand why Mercurial is called hg? Hg is the chemical symbol for Mercury.) -- Steven From steve at pearwood.info Tue Jul 23 16:30:40 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 24 Jul 2013 00:30:40 +1000 Subject: [Tutor] object size in python is in what units? In-Reply-To: References: Message-ID: <51EE9390.6080104@pearwood.info> On 24/07/13 00:21, Oscar Benjamin wrote: > The reason for the patterns that you observe are related to CPython's > memory pool allocator. You can read the long source comment describing > this here: > http://hg.python.org/cpython/file/a5681f50bae2/Objects/obmalloc.c#l367 [...] Nice analysis! Thanks Oscar! -- Steven From steve at pearwood.info Tue Jul 23 16:38:20 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 24 Jul 2013 00:38:20 +1000 Subject: [Tutor] object size in python is in what units? In-Reply-To: References: <51EE3386.10706@pearwood.info> Message-ID: <51EE955C.5030305@pearwood.info> On 23/07/13 18:17, Jim Mooney wrote: > On 23 July 2013 00:40, Steven D'Aprano wrote: > >> >> No no no, a thousand times no!!! IDs are just numeric IDs, that is all, >> like your social security number or driver's licence number. Don't think of >> them as having any meaning at all, except that they are unique during the >> lifetime of the object. >> > > Okay, ID stands for a location fixed until the object disappears, but we > don't know where that location is. No, it does not stand for a location. It is an identification number, that is all. In CPython, every value (lists, strings, integers, custom classes) are objects which exist in one location until destroyed. In Jython and IronPython, values are objects which can move from location to location, in order to free up memory for new objects. In PyPy, not only can objects move, but they can also be invisibly turned into low-level data structures for efficient processing. So in CPython, Jython and IronPython, an object's identity is fixed. In PyPy, object identity is an illusion, and according to the PyPy developers, keeping that illusion working requires a lot of effort. > But what about object offsets from self? That's always zero, since self is the object :-) You can read up about the implementation of CPython here: http://docs.python.org/2/c-api/ but this describes only the C interface, not the concrete implementation. For that you pretty much have to read the source code. The C source code. > Is the beginning of self.spam always the same distance from the beginning > of self.eggs? Or can I just forget the hard ground of assembler-metaphors > entirely as I float off into abstractville? I guess the fixed lengths I > kept getting on re-runs were coincidental but not to be relied on. Understanding a little of the concrete implementation details will help you understand some of Python's design, and its limitations. But apart from that, it's not necessary, and sometimes it is actually counter-productive. Thinking in terms of low-level data can lead you wrong. For instance, when working in low-level languages, it is normal to write code to minimize moving data from place to place. For example, when sorting in C or assembly, comparing two values is cheap, but swapping them is expensive. In Python it is the opposite: swapping two values in a list just requires swapping two pointers, which is cheap, but the comparison is quite expensive and could call arbitrary user code. In CPython, objects are implemented in C, and are a block of memory that contains various fields, and potentially pointers to other objects. The nature of those fields will depend on which type of object they are. In Jython, objects are implemented in Java, and in IronPython they are implemented in C#, but otherwise are similar. But the layout of the fields will be different, and the fields themselves, except perhaps by accident. In PyPy, objects are implemented in RPython, a restricted and simplified version of Python, which is then automatically translated by the optimizing compiler into some low-level data structure behind the scenes. Since Python is defined by its behaviour, not any sort of concrete implementation, in principle one could invent a "Python Engine", like Babbage's Difference Engine, only thousands of times more complex, entirely out of clockwork. Or by water flowing through pipes, or reproducing DNA in a test tube, or some exotic quantum field. Or simply simulate a Python interpreter in your head, perhaps aided by pencil and paper. -- Steven From steve at pearwood.info Tue Jul 23 16:46:38 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 24 Jul 2013 00:46:38 +1000 Subject: [Tutor] close, but no cigar In-Reply-To: References: Message-ID: <51EE974E.4030000@pearwood.info> On 23/07/13 04:27, Jim Mooney wrote: > Okay, I'm getting there, but this should be translating A umlaut to an old > DOS box character, according to my ASCII table, I understand what you mean, but I should point out that what you say is *literally impossible*, since neither ? nor any box-drawing characters are part of ASCII. What you are saying is figuratively equivalent to this: ...should be translating ?????? to ???? according to my Latin to French dictionary... Even if the ancient Romans knew of the city of Moscow, they didn't write it in Cyrillic and you certainly can't get Japanese characters by translating it to French. Remember that ASCII only has 128 characters, and *everything else* is non-ASCII, whether they are line-drawing characters, European accented letters, East Asian characters, emoticons, or ancient Egyptian. People who talk about "extended ASCII" are confused, and all you need to do to show up their confusion is to ask "which extended ASCII do you mean?" There are dozens. For example, ordinal value 0xC4 (hex, = 196 in decimal) has the following meaning depending on the version of "extended ASCII" you use: ? LATIN CAPITAL LETTER A WITH DIAERESIS HEBREW POINT HIRIQ ? GREEK CAPITAL LETTER DELTA ? ARABIC LETTER WAW WITH HAMZA ABOVE ? BOX DRAWINGS LIGHT HORIZONTAL ? LATIN SMALL LETTER F WITH HOOK using encodings Latin1, CP1255, ISO-8859-7, ISO-8859-6, IBM866, and MacRoman, in that order. And there are many others. So the question is, if you have a file name with byte 196 in it, which character is intended? In isolation, you cannot possibly tell. As an English speaker, I've used at least four of the above six, although only three in file names. With single-byte encodings, limited to a mere 256 characters (128 of which are already locked down to the ASCII charset[1]), you can't have all of the above except by using Unicode[2]. The old "code pages" technology is sheer chaos, and sadly we'll be living with it for years to come. But eventually, maybe in another 30 years or so, everyone will use Unicode all the time, except for specialist and legacy needs, and gradually we'll get past this nonsense of dozens of encodings and moji-bake and other crap. [1] Not all encodings are ASCII-compatible, but most of them are. [2] Or something like it. In Japan, there is a proprietary charset called TRON which includes even more characters than Unicode. Both TRON and Unicode aim to include every human character which has ever been used, but they disagree as to what counts as distinct characters. In a nutshell, there are some tens of thousands or so characters which are written the same way in Chinese, Japanese and Korean, but used differently. Unicode's policy is that you can tell from context which is meant, and gives them a single code-point each, while TRON gives them three code-points. This is not quite as silly as saying that an English E, a German E and a French E should be considered three distinct characters, but (in my opinion) not far off it. -- Steven From davea at davea.name Tue Jul 23 17:08:10 2013 From: davea at davea.name (Dave Angel) Date: Tue, 23 Jul 2013 11:08:10 -0400 Subject: [Tutor] unichr not working as expected In-Reply-To: <51EE92F6.1040602@pearwood.info> References: <51EDFFE2.8080605@pearwood.info> <51EE92F6.1040602@pearwood.info> Message-ID: On 07/23/2013 10:28 AM, Steven D'Aprano wrote: > On 24/07/13 00:14, Dave Angel wrote: >> On 07/23/2013 02:10 AM, Jim Mooney wrote: >>> On 22 July 2013 21:00, Steven D'Aprano wrote: >>> >>>> >>>> (By the way, you're very naughty. The code you show *cannot possibly >>>> generate the error you claim it generates*. Bad Jim, no biscuit!) >>>> >>> >>> I know. I need a personal github. >> >> git is free, and is usually installed on individual machines. It's >> also easy to set up and use for a single user. > > I prefer Mercurial (also known as hg). The Python core developers use it > for compiler development, and in my opinion, hg is easier to use than > git. (Although, in fairness, they're both pretty easy.) hg is cleaner > than git, being written in Python, while git is an unholy mess of C, > Perl and bash that will suck your brain out if you try to understand it, > unless you are Linus Torvalds. And there are some awesome hg tutorials > out there: > > http://hginit.com/ > http://mercurial.selenic.com/wiki/Tutorial > > (I trust that everyone remembers enough high school chemistry to > understand why Mercurial is called hg? Hg is the chemical symbol for > Mercury.) > > My comment was in response to github. I've never investigated Mercurial. In various companies over the years, I've used probably a dozen different source control systems, and git was the latest, and to my thinking, best of them. I also installed git on this machine a couple of days ago, so the easy-start link was handy. -- DaveA From eryksun at gmail.com Tue Jul 23 17:25:32 2013 From: eryksun at gmail.com (eryksun) Date: Tue, 23 Jul 2013 11:25:32 -0400 Subject: [Tutor] object size in python is in what units? In-Reply-To: References: Message-ID: On Tue, Jul 23, 2013 at 10:21 AM, Oscar Benjamin wrote: > > The reason for the patterns that you observe are related to CPython's > memory pool allocator. You can read the long source comment describing > this here: > http://hg.python.org/cpython/file/a5681f50bae2/Objects/obmalloc.c#l367 > > Essentially anything over 512 bytes will be allocated using a call to > the system (OS/runtime) malloc. Anything under 512 bytes will be > allocated into a memory pool managed by the interpreter. The code I > linked to shows the exact strategy used for this and if you read this > you'll have a vague understanding of the results you see. Prior to 3.3 the upper limit is 256 bytes. Here's the link for 2.7.5: http://hg.python.org/cpython/file/ab05e7dd2788/Objects/obmalloc.c#l20 Pools for a given storage class are 4 KiB, and are allocated from the system in 256 KiB arenas. On 32-bit Windows Python, the pool overhead is 32 bytes. So for 40-byte blocks, a pool will allocate 101 objects, with 24 bytes left unused, i.e. 32 + 101*40 + 24 == 4096. 3.3 also switched to using anonymous mmap on POSIX (where supported), and as your link shows -- 3.4 looks to be switching to VirtualAlloc on Windows. These changes bypass malloc to avoid high watermark effects that prevent the heap from shrinking. On my Debian box 3.3 does a much better job releasing memory back to the system after allocating and deallocating thousands of objects. > You can check the size in bytes of an object with sys.getsizeof e.g.: > >>>> import sys >>>> sys.getsizeof(lardKronk) > 36 sys.getsizeof() includes the GC header that precedes the object base address. Only container objects (such as normal class instances) have this header, which is used by gc to resolve circular references. The header is 12 bytes on 32-bit Windows Python. If you want to exclude it, use the object's __sizeof__() method instead. From oscar.j.benjamin at gmail.com Tue Jul 23 18:19:31 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Tue, 23 Jul 2013 17:19:31 +0100 Subject: [Tutor] object size in python is in what units? In-Reply-To: References: Message-ID: On 23 July 2013 16:25, eryksun wrote: > > 3.3 also switched to using anonymous mmap on POSIX (where supported), > and as your link shows -- 3.4 looks to be switching to VirtualAlloc on > Windows. Why is it better to use mmap than malloc? I read a few things about this but the main thing I found seemed to be that on systems supporting mmap, malloc itself will use mmap for sufficiently large enough allocations. That would seem to indicate that CPython would be better off letting the system decide when to use it (I'm not arguing with the code, just confused). Oscar From marc.tompkins at gmail.com Tue Jul 23 19:01:13 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Tue, 23 Jul 2013 10:01:13 -0700 Subject: [Tutor] close, but no cigar In-Reply-To: <51EE974E.4030000@pearwood.info> References: <51EE974E.4030000@pearwood.info> Message-ID: On Tue, Jul 23, 2013 at 7:46 AM, Steven D'Aprano wrote: > This is not quite as silly as saying that an English E, a German E and a > French E should be considered three distinct characters, but (in my > opinion) not far off it. > I half-agree, half-disagree. It's true that the letter "E" is used more-or-less the same in English, French, and German; after all, they all use what's called the "Latin" alphabet, albeit with local variations. On the other hand, the Cyrillic alphabet contains several letters that are visually identical to their Latin equivalents, but used quite differently - so it's quite appropriate that they're considered different letters, and even a different alphabet. I don't know enough about the similarities and differences between various East Asian languages to know whether, say, Chinese and Korean are more like English and German or more like English and Russian - but that, rather than the visual similarity, would be my criterion for deciding. Spot the differences: A ? a ? B ? b ? C ? c ? E ? e ? ? ? ? ? K ? k ? M ? m ? n ? O ? o ? P ? p ? T ? t ? u ? X ? x ? Y ? y ? A few notes: - this won't look right unless your email client is Unicode-capable - no, I'm not saying that these letters are equivalent - some (T and ?, K and ?) basically are, others (E and ?, n and ?) definitely are not - I'm just saying that they are visually similar if not identical - just HOW similar they are depends on which typeface you use. -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan.gauld at btinternet.com Tue Jul 23 19:09:31 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Tue, 23 Jul 2013 18:09:31 +0100 Subject: [Tutor] unichr not working as expected In-Reply-To: References: <51EDFFE2.8080605@pearwood.info> <51EE92F6.1040602@pearwood.info> Message-ID: On 23/07/13 16:08, Dave Angel wrote: > >>> git is free, and is usually installed on individual machines. It's >>> also easy to set up and use for a single user. >> >> I prefer Mercurial (also known as hg). And I've used SVN in recent years and found it... OK. But my question is: Why is there such a rash of source code control systems just now? I used SCCS and RCS and then CVS for years and they seemed to work just fine (especially CVS for larger projects). So what is the itch that everyone is trying to scratch with these new systems? Personally I don't find them significantly better or even different to the previous generation. (Unlike the commercial products I've used like ClearCase(*) and Continuous which are vastly superior but vastly more expensive!) So what's the big deal with version control? (*)On Unix only, on Windows ClearCase is not significantly better than any other VCS -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From dwightdhutto at gmail.com Sat Jul 20 05:06:37 2013 From: dwightdhutto at gmail.com (David Hutto) Date: Fri, 19 Jul 2013 23:06:37 -0400 Subject: [Tutor] A list of 100+ projects to complete to master Python. In-Reply-To: References: Message-ID: The offsite backups, are just a safety measure for your programs, this can include things such as sourceforge so you don't lose your work to a glitch, which can happen without a moments notice. You want to master python, then reinvemnt the wheel sometimes. Most everything has beeen done. It's data, and data manipulation, so you can do it your way, or look at the code that original used, and do it your way. Remember the 'zen' of python, to reduce complex lower level macros into higher level reductions of these more complex codes. Just do it your own way. And as far as the API's, that is to utilize other programs that allow you to use functions, and python programming within other software. Like a cell phone will have a Python API so you can utilize python on individual devices, and other software that will translate, or have a python library in which you can use python code anywhere, as the language grows in it's utilization in several areas. On Wed, Jul 17, 2013 at 1:26 AM, Karan Goel wrote: > I'm not entirely sure if your reply was directed towards my message. > > - Karan Goel > Goel.im | Resume > | Github > > > On Wed, Jul 17, 2013 at 5:09 AM, David Hutto wrote: > >> First thing you should learn is offsite backups, I've lost several >> projects in the works because of a hd mishap. Secondarily, I would >> recommend using existing primarily used python projects to 'reinvent the >> wheel' so to speak. Thirdly, make sure the code is properly documented, and >> serves a purpose. And lastly, utilize other programs with python api's, >> such as blender, which can make your apps pop, or diversify into game >> development kits. >> >> >> On Tue, Jul 16, 2013 at 1:09 PM, Karan Goel wrote: >> >>> Hey guys and gals >>> >>> Just a plug here. My repo: https://github.com/thekarangoel/Projects >>> was one of the trending repos on Gh this week and I thought folks >>> on this list might be interested in knowing about it. >>> >>> In a nutshell, I'm trying to complete over a 100 practical language- >>> agnostic projects in Python only. I haven't read the project details yet, >>> and I'm not filtering things out. >>> >>> If you would like to do the same, join me. Fork or star the repo, >>> and start coding (in any language, really). >>> >>> https://github.com/thekarangoel/Projects >>> >>> Let me know if you have any questions or suggestions. >>> >>> - Karan Goel >>> Goel.im | Resume >>> | Github >>> >>> _______________________________________________ >>> Tutor maillist - Tutor at python.org >>> To unsubscribe or change subscription options: >>> http://mail.python.org/mailman/listinfo/tutor >>> >>> >> >> >> -- >> Best Regards, >> David Hutto >> *CEO:* *http://www.hitwebdevelopment.com* >> > > -- Best Regards, David Hutto *CEO:* *http://www.hitwebdevelopment.com* -------------- next part -------------- An HTML attachment was scrubbed... URL: From dwightdhutto at gmail.com Sat Jul 20 07:10:16 2013 From: dwightdhutto at gmail.com (David Hutto) Date: Sat, 20 Jul 2013 01:10:16 -0400 Subject: [Tutor] suggestions for splitting file based on date In-Reply-To: <87siz9rfvs.fsf@gmail.com> References: <87a9litkta.fsf@gmail.com> <87siz9rfvs.fsf@gmail.com> Message-ID: Why not use the actual month? With a simple x/y canvas in Tkinter you could plot by the months with polygon coordinates as your data visualization, or in 30 day /etc windows, just the price(y) being a derivative of x(the frequency of changes), and create simple line segments with polygon coordinates, and refine them to hourly/daily/weekly/monthly/yearly, or in 30/60/90 day increments with a little url scraping. -------------- next part -------------- An HTML attachment was scrubbed... URL: From dwightdhutto at gmail.com Sat Jul 20 07:26:48 2013 From: dwightdhutto at gmail.com (David Hutto) Date: Sat, 20 Jul 2013 01:26:48 -0400 Subject: [Tutor] suggestions for splitting file based on date In-Reply-To: References: <87a9litkta.fsf@gmail.com> <87siz9rfvs.fsf@gmail.com> Message-ID: You could also begin a little stats(I think steven D'aprano did pystats), which would show rhythms within those particular frequency windows, using y as a basis for your model On Sat, Jul 20, 2013 at 1:10 AM, David Hutto wrote: > Why not use the actual month? With a simple x/y canvas in Tkinter you > could plot by the months with polygon coordinates as your data > visualization, or in 30 day /etc windows, just the price(y) being a > derivative of x(the frequency of changes), and create simple line segments > with polygon coordinates, and refine them to > hourly/daily/weekly/monthly/yearly, or in 30/60/90 day increments with a > little url scraping. > -- Best Regards, David Hutto *CEO:* *http://www.hitwebdevelopment.com* -------------- next part -------------- An HTML attachment was scrubbed... URL: From dwightdhutto at gmail.com Tue Jul 23 13:12:06 2013 From: dwightdhutto at gmail.com (David Hutto) Date: Tue, 23 Jul 2013 07:12:06 -0400 Subject: [Tutor] 3 Dimensional Dictionaries In-Reply-To: References: Message-ID: You could find several url's to scrape the data from, and I believe it was mentioned you could use a list of dicts, but it seems that you might want to sub tier your dicts, It's been a while since I've pythoned so this a little longer that usual, but this may help if you pull in the countries/continents/states/provinces/zipcodes, however you may stil want to give a DB a try, which you can create on your own out of dictionaries/with list tuples, and subdicts. \but if you always wnat dynamic data, rescraping several url's at a personally optimized frequency into your db could help import random continents = ['china','u.s'] def world_stats(): world = {} for continent in continents: world[continent] = {'periodic_table_resources': {} } print(continent) for periodic_elements in range(0,119): rand = random.randint(0,101) world[continent]['periodic_table_resources'] = 'periodic_table_element %i=' % (periodic_elements) print(world[continent]['periodic_table_resources'], rand, '%') world_stats() This doesn't contain the url scraper yet. On Mon, Jul 22, 2013 at 10:44 PM, Sunil Tech wrote: > > > On Tuesday, July 23, 2013, Sunil Tech wrote: > > THANK YOU ALL for your time. > > > > The first format which I pasted was from the DB > > > > The second format(exactly the same), is to be sent to the view. > > > > If the logic can be fitted in One or two methods it'll help me to easily > understand & to apply. > > > > so I request you to help... > > > > > > On Sunday, July 21, 2013, Alan Gauld wrote: > >> On 20/07/13 11:17, Sunil Tech wrote: > >>> > >>> Hi Everyone, > >>> > >>> I have a list of dictionaries like > >>> > >>> world = > >>> > [{'continent':'Asia','continent_code':1,'ocean':'Pacific','country':'India','country_code':1,'state':'Kerala', > >>> 'state_pin':500001}, > >> > >>> > >>> i am trying to to make it in this format > >>> > >>> world = [{'continent':'Asia', 'ocean':'Pacific', > >>> 'countries':[{'country':'India', > >>> 'states':[{'state':'Kerala', 'state_pin':500001}, > >>> {'state':'Karnataka', 'state_pin':500002}] > >>> }] > >>> }, > >> > >>> Please help me in this regard. > >> > >> In what regard? Where do you need the help? > >> You seem to know the data format you want? > >> > >> The only thing I'd suggest is to consider using classes if you are > familiar with them. > >> > >> world => list of continents > >> continent => class containing countries > >> country => class containing states > >> state => class containing data > >> > >> It then becomes easier to build helper methods to extract/manipulate > the data you are interested in. > >> > >> Alternatively, if you have a large amount of data a database may be > another option. Swap table for class above and use SQL to manage the data. > >> > >> But other than those suggestions I don't know what kind of > >> help you want? > >> > >> -- > >> Alan G > >> Author of the Learn to Program web site > >> http://www.alan-g.me.uk/ > >> > >> _______________________________________________ > >> Tutor maillist - Tutor at python.org > >> To unsubscribe or change subscription options: > >> http://mail.python.org/mailman/listinfo/tutor > >> > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > > -- Best Regards, David Hutto *CEO:* *http://www.hitwebdevelopment.com* -------------- next part -------------- An HTML attachment was scrubbed... URL: From amandeep.qa at gmail.com Mon Jul 22 04:26:18 2013 From: amandeep.qa at gmail.com (Amandeep Behl) Date: Sun, 21 Jul 2013 19:26:18 -0700 Subject: [Tutor] Question Message-ID: def Move(label): label = "Anything" a_list = [1,2,3] Move(a_list) I don't see any error when executing above but when i manually assign [1,2,3] = "Anything" i get error Why? -------------- next part -------------- An HTML attachment was scrubbed... URL: From enmce at yandex.ru Tue Jul 23 14:27:10 2013 From: enmce at yandex.ru (enmce) Date: Tue, 23 Jul 2013 16:27:10 +0400 Subject: [Tutor] 2d rotation gives unexpected results. Message-ID: <1207841374582430@web10e.yandex.ru> Hello! This is my first post, nice to meet you all! I`m biology student from Russia, trying to learn python to perform some simple simulations. Here`s my first problem. I`m trying to perform some simple 2d vector rotations in pygame, in order to learn the basics of linear algebra and 2d transformations. So far i understand matrix multiplication pretty well, and probably all my math is right. Eventually i`m planning to write Poly class, and use it to rotate and translate some simple shapes. But when i try and write it in the program, i get very weird results, like all points of rectangle with coordinates [0,0],[0,100],[100,0],[100,100] start to go spiral and eventually shrink to the center instead of rotating right. Although even Excel calculations with this formulas give me right result. What is wrong with my code? [code]import pygame import math as m black = ( 0, 0, 0) white = ( 255, 255, 255) green = ( 0, 255, 0) red = ( 255, 0, 0) class Poly(): pos = [100,100] #x and y coordinates of a point rot = m.radians(1) #rotation in degrees def draw(self): #draw point pygame.draw.circle(screen,white,self.pos,10,0) def rotate(self): # rotation method sin = m.sin(self.rot) #calculationg sin and cos cos = m.cos(self.rot) x_rot = int(self.pos[0]*cos-self.pos[1]*sin) #mulpitplicating vector to rotation matrix y_rot = int(self.pos[0]*sin+self.pos[1]*cos) self.pos[0] = x_rot #set new coordinates to a point self.pos[1] = y_rot a = Poly() #Some simple sample points giving rectangle b = Poly() c = Poly() d = Poly() b.pos = [0,100] c.pos = [100,0] d.pos = [0,0] pygame.init() size = [700,500] screen = pygame.display.set_mode(size) done = False clock = pygame.time.Clock() while done == False: for event in pygame.event.get(): if event.type == pygame.QUIT: done = True a.rotate() #perform rotation b.rotate() c.rotate() d.rotate() screen.fill(black) a.draw() #draw point b.draw() c.draw() d.draw() pygame.display.flip() clock.tick(30) pygame.quit()[/code] P.S. I use Python 3.3 on Windows XP. Sorry for my english, bit rusty in that department. From vick1975 at orange.mu Mon Jul 22 19:45:50 2013 From: vick1975 at orange.mu (Vick) Date: Mon, 22 Jul 2013 21:45:50 +0400 Subject: [Tutor] hi In-Reply-To: References: <000001ce7ee7$d2233170$76699450$@orange.mu> Message-ID: <000901ce8703$4e599b30$eb0cd190$@orange.mu> Hello, Thanks for taking the time to reply! However, I had already solved the problem a few days ago on my own. The DOPRI 8(7)13 is the Dormand Prince 8th order with 13 stages method for solving ordinary differential equations. It is a member of the Runge-Kutta family of ODE solvers. I have been able to reproduce it in Python with a multi-variable capability. On a particular test equation the error of the RK4 is about 1e-4 whereas the DOPRI 8(7)13 is about 1e-13. I have then used the multi-variable DOPRI to compute for the n-body problem using 1st order ODEs which requires you to have 4 equations for 1 planet. I have already made a 10-body problem through an n-body solver in python and using DOPRI as my integrator. It works fine. Thanks again for your time and consideration. Regards Vick -----Original Message----- From: Oscar Benjamin [mailto:oscar.j.benjamin at gmail.com] Sent: Monday, 22 July, 2013 15:59 To: Vick Cc: Tutor at python.org Subject: Re: [Tutor] hi On 12 July 2013 11:08, Vick < vick1975 at orange.mu> wrote: > Hi, Hi Vick, Sorry for the delayed response I didn't see this email until a while after you sent it. > I have written a code to perform a numerical solution to 1st order > Ordinary Differential Equations (ODEs). I have written codes for the > famous Runge Kutta 4th order and the DOPRI 8(7)13. However the codes > are for 1 variable only; viz. the "y" variable. It would be good to show us the code for this in your email (not in an attachment). Also the code should be complete: the code uses a function rungkdp8 which is not shown. Where did that come from? Until you know what you're doing you should test your integrator on a simpler set of equations. Also when posting to a mailing list you should simplify the problem as much as possible which would mean using simpler ODEs. In short read this: http://sscce.org/ The code attached is incorrect as it applies each stage of the Runge-Kutta method to all time-steps sequentially before moving on to the next stage. It should apply each stage for one time step and then use the output of that for the next time-step. In other words you need to change the order of the loops. Here is an example of how to implement an Euler stepper that should hopefully get you started on the right track. I've deliberately avoided using numpy in this example even though it makes most of this easier. I did this to focus on the basics of Python which you should learn first. # integrator.py nan = float('nan') # indices for the two variables in the system POS = 0 VEL = 1 VARS = ['pos', 'vel'] # system has one parameter OMEGA OMEGA = 10 # Frequency in Hz is OMEGA / 2pi # function for the derivative of the system def shm(t, x, dxdt): '''Equation of motion for a mass in a spring with freq OMEGA''' dxdt[POS] = x[VEL] # Python uses square brackets for indexing dxdt[VEL] = - OMEGA**2 * x[POS] # Initial conditions for the problem x0 = [nan] * 2 x0[POS] = 1 x0[VEL] = 0 # This is the function that you should replace with e.g. rk4 or dopri8 def euler(t1, x1, t2): '''Take 1 Euler step from x1 at t1 to x2 at t2''' # Create empty lists for answer x2 = [nan] * len(x1) dxdt = [nan] * len(x1) # Call derivative function to fill in dxdt shm(t1, x1, dxdt) # Compute x2 and return dt = t2 - t1 for n in range(len(x1)): x2[n] = x1[n] + dt * dxdt[n] return x2 def solve(ts, x0): '''Compute states corresponding to the times in ts x0 is the state at ts[0] (the initial condition). ''' # Create an empty list of lists for the solution Xt = [x0[:]] # The initial conditions # Loop through timesteps computing the next value for n in range(len(ts) - 1): Xt.append(euler(ts[n], Xt[n], ts[n+1])) return Xt def print_line(*items): print(', '.join(str(i) for i in items)) def print_solution(ts, Xt, vars): print_line('t', *vars) for t, x in zip(times, Xt): print_line(t, *x) # Numerical parameters for the solution DT = 0.001 t0 = 0 T = 1 times = [t0] while times[-1] < T: times.append(times[-1] + DT) # Solve and print Xt = solve(times, x0) print_solution(times, Xt, VARS) > I have written another code in Excel VBA for DOPRI 8 for a > multi-variable capability. However I have not been able to reproduce > it in Python. I'm having trouble in making arrays or lists, I don't > know which is supposed to work. > > I have attached my Excel VBA code for a multi-variable numerical > integrator in PDF format. This does work in Excel. I have also attached my python code. I'm not familiar with Excel VBA but I think that this code suffers from the same problem that the loop over stages takes place outside the loop over time-steps so I believe that it is incorrect. > Can anyone help me out please? I happen to know quite a lot about this subject so feel free to ask more questions. Oscar -------------- next part -------------- An HTML attachment was scrubbed... URL: From wolfrage8765 at gmail.com Tue Jul 23 19:41:58 2013 From: wolfrage8765 at gmail.com (wolfrage8765 at gmail.com) Date: Tue, 23 Jul 2013 13:41:58 -0400 Subject: [Tutor] unichr not working as expected In-Reply-To: References: <51EDFFE2.8080605@pearwood.info> <51EE92F6.1040602@pearwood.info> Message-ID: Although I can not say for sure. It appears to me, they are trying to do a better job at merging from distributed branches to various distributed repositories. GIT being the shinning example at this particular task. I personally use BZR, because they aimed to be easier to use for a new programmer but with many of the strengths of GIT (Except for Speed). I am only just now reading up on HG. I am reading about another because I hope BZR does not die, but it seems to have fallen by the wayside, and in my experience GIT confused the crap out of me. On Tue, Jul 23, 2013 at 1:09 PM, Alan Gauld wrote: > On 23/07/13 16:08, Dave Angel wrote: >> >> >>>> git is free, and is usually installed on individual machines. It's >>>> also easy to set up and use for a single user. >>> >>> >>> I prefer Mercurial (also known as hg). > > > And I've used SVN in recent years and found it... OK. > > But my question is: Why is there such a rash of source code control systems > just now? > > I used SCCS and RCS and then CVS for years and they seemed > to work just fine (especially CVS for larger projects). So what is > the itch that everyone is trying to scratch with these new systems? > Personally I don't find them significantly better or even different to the > previous generation. (Unlike the commercial products I've used like > ClearCase(*) and Continuous which are vastly superior but vastly more > expensive!) > > So what's the big deal with version control? > > (*)On Unix only, on Windows ClearCase is not significantly better > than any other VCS > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor From ramit.prasad at jpmorgan.com Tue Jul 23 19:25:28 2013 From: ramit.prasad at jpmorgan.com (Prasad, Ramit) Date: Tue, 23 Jul 2013 17:25:28 +0000 Subject: [Tutor] unichr not working as expected In-Reply-To: References: <51EDFFE2.8080605@pearwood.info> <51EE92F6.1040602@pearwood.info> Message-ID: <5B80DD153D7D744689F57F4FB69AF474185B5DA1@SCACMX008.exchad.jpmchase.net> Alan Gauld wrote: > On 23/07/13 16:08, Dave Angel wrote: > > > >>> git is free, and is usually installed on individual machines. It's > >>> also easy to set up and use for a single user. > >> > >> I prefer Mercurial (also known as hg). > > And I've used SVN in recent years and found it... OK. > > But my question is: Why is there such a rash of source code control > systems just now? > > I used SCCS and RCS and then CVS for years and they seemed > to work just fine (especially CVS for larger projects). So what is > the itch that everyone is trying to scratch with these new systems? > Personally I don't find them significantly better or even different to > the previous generation. (Unlike the commercial products I've used like > ClearCase(*) and Continuous which are vastly superior but vastly more > expensive!) > > So what's the big deal with version control? > > (*)On Unix only, on Windows ClearCase is not significantly better > than any other VCS > > -- > Alan G > Author of the Learn to Program web site > http://www.alan-g.me.uk/ > Different source systems for different use cases. Git was developed by Torvolds because he wanted a source control that was better at handling a multitude of branches and probably aimed at the development style for the Linux kernel. I know from personal experience that some refactoring (e.g. file/directory renaming) is not supported by older VCS like CVS. Other VCS can support file renaming but not directory renaming and some do both. Ramit This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email. From dfjennings at gmail.com Tue Jul 23 19:45:05 2013 From: dfjennings at gmail.com (Don Jennings) Date: Tue, 23 Jul 2013 13:45:05 -0400 Subject: [Tutor] Question In-Reply-To: References: Message-ID: <16209487-F03D-4D68-A1B3-4B597DF17BC5@gmail.com> On Jul 21, 2013, at 10:26 PM, Amandeep Behl wrote: > > > def Move(label): > label = "Anything" > > a_list = [1,2,3] > Move(a_list) > > I don't see any error when executing above but when i manually assign [1,2,3] = "Anything" i get error > Why? Repeat after me: label is a name, label is a name, label is a name ;>) So, you misunderstand what is happening in your code. Your function accepts the argument named "label", but you do nothing with the object (a list of numbers in this case). Instead, you use the same name and assign it to the string object "Anything". Assignment works by assigning a name (on the left side of the operator), to an object on the right side. Take care, Don From cybervigilante at gmail.com Tue Jul 23 20:27:41 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Tue, 23 Jul 2013 11:27:41 -0700 Subject: [Tutor] object size in python is in what units? In-Reply-To: References: Message-ID: On 23 July 2013 07:21, Oscar Benjamin wrote: > > > You can check the size in bytes of an object with sys.getsizeof e.g.: > > >>> import sys > >>> sys.getsizeof(lardKronk) > 36 > And yet I still get 36 with this: bigKronk = Kronk('supercalifracilisticexpalidocious, even though the sound of it is really quite atrocious') sys.getsizeof(bigKronk) 36 So I guess the object is only pointing to its contents. Which I assume means an object is just a bunch of names bound to pointers. Jim An excellent animation/graph of the accelerating disappearance of arctic sea ice, which controls climate stability in the Northern hemisphere: https://www.youtube.com/watch?v=YgiMBxaL19M -------------- next part -------------- An HTML attachment was scrubbed... URL: From joel.goldstick at gmail.com Tue Jul 23 20:28:07 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Tue, 23 Jul 2013 14:28:07 -0400 Subject: [Tutor] Question In-Reply-To: <16209487-F03D-4D68-A1B3-4B597DF17BC5@gmail.com> References: <16209487-F03D-4D68-A1B3-4B597DF17BC5@gmail.com> Message-ID: On Tue, Jul 23, 2013 at 1:45 PM, Don Jennings wrote: > > On Jul 21, 2013, at 10:26 PM, Amandeep Behl wrote: > >> >> >> def Move(label): >> label = "Anything" >> >> a_list = [1,2,3] >> Move(a_list) >> >> I don't see any error when executing above but when i manually assign [1,2,3] = "Anything" i get error >> Why? Don's answer is great. The meaning of 'Name' seems to be the most common confusion to people learning to code python when having first learned other languages. To your question above: what do you expect to happen when you code this: [1,2,3] = "Anything" > > Repeat after me: label is a name, label is a name, label is a name ;>) > > So, you misunderstand what is happening in your code. Your function accepts the argument named "label", but you do nothing with the object (a list of numbers in this case). Instead, you use the same name and assign it to the string object "Anything". > > Assignment works by assigning a name (on the left side of the operator), to an object on the right side. > > Take care, > Don > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor -- Joel Goldstick http://joelgoldstick.com From cybervigilante at gmail.com Tue Jul 23 20:33:31 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Tue, 23 Jul 2013 11:33:31 -0700 Subject: [Tutor] close, but no cigar In-Reply-To: <51EE974E.4030000@pearwood.info> References: <51EE974E.4030000@pearwood.info> Message-ID: On 23 July 2013 07:46, Steven D'Aprano wrote:People who talk about "extended ASCII" are confused, and all you need to do to show up their confusion is to ask "which extended ASCII do you mean?" There are dozens. I think I was too impressed by all those wonderful old ASCII-art pics on bulletin boards. I bet Msoft didnt' predict That when they invented their box-drawing chars - more Unintended Consquences, but benign ones for a change. Some of them were truly amazing. I'll have to go do a webhunt for them. Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Tue Jul 23 20:37:10 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Tue, 23 Jul 2013 11:37:10 -0700 Subject: [Tutor] object size in python is in what units? In-Reply-To: <51EE90FB.1020408@pearwood.info> References: <51EE90FB.1020408@pearwood.info> Message-ID: On 23 July 2013 07:19, Steven D'Aprano wrote: > Some day, I'll invent my own version of Python, and make object IDs be > consecutive prime numbers. That'll teach people to rely on them. > --- Why "consecutive?" That's still way too predictable (Yes, the obvious flaw already occurred to me ;') Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Tue Jul 23 20:53:45 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Tue, 23 Jul 2013 11:53:45 -0700 Subject: [Tutor] unichr not working as expected In-Reply-To: <51EE92F6.1040602@pearwood.info> References: <51EDFFE2.8080605@pearwood.info> <51EE92F6.1040602@pearwood.info> Message-ID: On 23 July 2013 07:28, Steven D'Aprano wrote: > > (I trust that everyone remembers enough high school chemistry to > understand why Mercurial is called hg? Hg is the chemical symbol for > Mercury.) > And I recall my high school chemistry teacher claiming the noble gases could never combine, until someone just decided to cook up some Xenon Tetraflouride. Always fun when you get to contradict your teacher in class, and you're right - wel it was then. I'm a tad more mature, now - sometimes ;') There are always surprises. Jim -------------- next part -------------- An HTML attachment was scrubbed... URL: From alan.gauld at btinternet.com Tue Jul 23 21:02:12 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Tue, 23 Jul 2013 20:02:12 +0100 Subject: [Tutor] object size in python is in what units? In-Reply-To: References: Message-ID: On 23/07/13 19:27, Jim Mooney wrote: > And yet I still get 36 with this: > > bigKronk = Kronk('supercalifracilisticexpalidocious, even though the > sound of it is really quite atrocious') > sys.getsizeof(bigKronk) > 36 > > So I guess the object is only pointing to its contents. Python namespaces are effectively dictionaries binding a name with an object reference. The object in turn is like a dictionary referring to the values. So I guess you are seeing the object container size. > means an object is just a bunch of names bound to pointers. Pretty much everything in Python looks like a dictionary so I'd guess you are correct. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From alan.gauld at btinternet.com Tue Jul 23 21:20:14 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Tue, 23 Jul 2013 20:20:14 +0100 Subject: [Tutor] unichr not working as expected In-Reply-To: References: <51EDFFE2.8080605@pearwood.info> <51EE92F6.1040602@pearwood.info> Message-ID: On 23/07/13 18:41, wolfrage8765 at gmail.com wrote: > Although I can not say for sure. It appears to me, they are trying to > do a better job at merging from distributed branches Thanks for the memory jog. Yes i remember reading an interview with linus when he referred to the difficulty of using CVS on a distributed build and that was a prime motivator for git. Now what about the others? Are they also primarily aimed at distributed builds? Renaming files and folders should not be a big problem for any VCS, if you freeze/recreate the tree the folder names should be whatever they were at freeze. (The only issue is if you want to pull up version -N of a file and its history stops at version -(N+3) and you then need to pull up version (-3) of the old file and don't know its name! - bad commenting and history documentation, but not unknown. But the tool can do it, its only the management that's faulty! :-) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From eryksun at gmail.com Tue Jul 23 21:22:00 2013 From: eryksun at gmail.com (eryksun) Date: Tue, 23 Jul 2013 15:22:00 -0400 Subject: [Tutor] object size in python is in what units? In-Reply-To: References: Message-ID: On Tue, Jul 23, 2013 at 12:19 PM, Oscar Benjamin wrote: > On 23 July 2013 16:25, eryksun wrote: >> >> 3.3 also switched to using anonymous mmap on POSIX (where supported), >> and as your link shows -- 3.4 looks to be switching to VirtualAlloc on >> Windows. > > Why is it better to use mmap than malloc? > > I read a few things about this but the main thing I found seemed to be > that on systems supporting mmap, malloc itself will use mmap for > sufficiently large enough allocations. That would seem to indicate > that CPython would be better off letting the system decide when to use > it (I'm not arguing with the code, just confused). glibc malloc, for example, starts with a threshold at 128 KiB. But it adjusts this upward when you free larger blocks, up to DEFAULT_MMAP_THRESHOLD_MAX, which is 512 KiB on 32-bit systems and 4 MiB on 64-bit systems (as "man mallopt" informs me). That means in practice Python's 256 KiB small object arenas end up on the heap. That's subject to a dangling object allocated in the topmost arena (object references are pointers, so it can't be automatically moved to defragment the heap), even if (a big if) glibc would otherwise trim the heap. But also Python is already managing the arenas. Isn't it wasteful to have the C runtime also incurring overhead for these blocks? Reducing peak memory usage by a small amount (2-3%) appears to be the argument for the 3.4 switch to VirtualAlloc on Windows. mmap patch http://bugs.python.org/issue11849 VirtualAlloc patch http://bugs.python.org/issue13483 From steve at pearwood.info Tue Jul 23 21:23:56 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 24 Jul 2013 05:23:56 +1000 Subject: [Tutor] close, but no cigar In-Reply-To: References: <51EE974E.4030000@pearwood.info> Message-ID: <51EED84C.2050608@pearwood.info> On 24/07/13 03:01, Marc Tompkins wrote: > On Tue, Jul 23, 2013 at 7:46 AM, Steven D'Aprano wrote: > >> This is not quite as silly as saying that an English E, a German E and a >> French E should be considered three distinct characters, but (in my >> opinion) not far off it. >> > > I half-agree, half-disagree. It's true that the letter "E" is used > more-or-less the same in English, French, and German; after all, they all > use what's called the "Latin" alphabet, albeit with local variations. On > the other hand, the Cyrillic alphabet contains several letters that are > visually identical to their Latin equivalents, but used quite differently - > so it's quite appropriate that they're considered different letters, and > even a different alphabet. Correct. Even if they were the same, if legacy encoding systems treated them differently, so would Unicode. For example, \N{DIGIT FOUR} and \N{FULLWIDTH DIGIT FOUR} have distinct code-points, even though they are exactly the same character, since some legacy East-Asian encodings had separate characters for "full-width" and "half-width" forms. But I confess I have misled you. I wrote about the CJK controversy from memory, and I'm afraid I got it completely backwards: the problem is that the glyphs (images of the characters) are different, but not the meaning. Mea culpa. For example, in English, we can draw the dollar sign $ in two distinct ways, with one vertical line, or two. Unicode treats them as the same character (as do English speakers). "Han Unification" refers to Unicode's choice to do the same for many Han (Chinese, Korean, Japanese) ideographs with different appearance but the same meaning. For various reasons, some technical, some social, this choice proved to be unpopular, particularly in Japan. This issue is nothing new -- Unicode supports about 71,000 distinct East Asian ideographs, which is *far* more than the old legacy encodings were capable of representing, so if there is a Han character that you would like to write which Unicode doesn't support, chances are that neither does any other encoding system. More here: https://en.wikipedia.org/wiki/Han_unification http://www.unicode.org/faq/han_cjk.html http://slashdot.org/story/01/06/06/0132203/why-unicode-will-work-on-the-internet -- Steven From davea at davea.name Tue Jul 23 21:29:39 2013 From: davea at davea.name (Dave Angel) Date: Tue, 23 Jul 2013 15:29:39 -0400 Subject: [Tutor] 2d rotation gives unexpected results. In-Reply-To: <1207841374582430@web10e.yandex.ru> References: <1207841374582430@web10e.yandex.ru> Message-ID: On 07/23/2013 08:27 AM, enmce wrote: > Hello! > This is my first post, nice to meet you all! > I`m biology student from Russia, trying to learn python to perform some simple simulations. Hello, and welcome. However, please don't post identical messages on tutor and python-list. The response you got there from Peter Otten was right on. > > Here`s my first problem. > I`m trying to perform some simple 2d vector rotations in pygame, in order to learn the basics of linear algebra and 2d transformations. So far i understand matrix multiplication pretty well, and probably all my math is right. Eventually i`m planning to write Poly class, and use it to rotate and translate some simple shapes. But when i try and write it in the program, i get very weird results, like all points of rectangle with coordinates [0,0],[0,100],[100,0],[100,100] start to go spiral and eventually shrink to the center instead of rotating right. Although even Excel calculations with this formulas give me right result. > What is wrong with my code? > > [code]import pygame > import math as m > > black = ( 0, 0, 0) > white = ( 255, 255, 255) > green = ( 0, 255, 0) > red = ( 255, 0, 0) > > class Poly(): > pos = [100,100] #x and y coordinates of a point > rot = m.radians(1) #rotation in degrees > def draw(self): #draw point > pygame.draw.circle(screen,white,self.pos,10,0) > def rotate(self): # rotation method > sin = m.sin(self.rot) #calculationg sin and cos > cos = m.cos(self.rot) > x_rot = int(self.pos[0]*cos-self.pos[1]*sin) #mulpitplicating vector to rotation matrix > y_rot = int(self.pos[0]*sin+self.pos[1]*cos) > By calling int() on those two lines, you have just drastically changed the point's location. That won't be visible right away, but when you use that value repeatedly, the error will add up pretty quickly. In fact, just repeatedly calculating those values will eventually produce some noticeable errors. Those errors can be eliminated as well, but it'll be a bit harder. -- DaveA From davea at davea.name Tue Jul 23 21:31:20 2013 From: davea at davea.name (Dave Angel) Date: Tue, 23 Jul 2013 15:31:20 -0400 Subject: [Tutor] unichr not working as expected In-Reply-To: References: <51EDFFE2.8080605@pearwood.info> <51EE92F6.1040602@pearwood.info> Message-ID: On 07/23/2013 03:20 PM, Alan Gauld wrote: > On 23/07/13 18:41, wolfrage8765 at gmail.com wrote: >> Although I can not say for sure. It appears to me, they are trying to >> do a better job at merging from distributed branches > > Thanks for the memory jog. Yes i remember reading an interview with > linus when he referred to the difficulty of using CVS on a > distributed build and that was a prime motivator for git. > > Now what about the others? Are they also primarily aimed at > distributed builds? The three distributed version control systems I know of are: git, mercurial, and bazaar -- DaveA From eryksun at gmail.com Tue Jul 23 21:45:36 2013 From: eryksun at gmail.com (eryksun) Date: Tue, 23 Jul 2013 15:45:36 -0400 Subject: [Tutor] object size in python is in what units? In-Reply-To: References: <51EE3386.10706@pearwood.info> Message-ID: On Tue, Jul 23, 2013 at 4:17 AM, Jim Mooney wrote: > > Okay, ID stands for a location fixed until the object disappears, but we > don't know where that location is. But what about object offsets from self? > Is the beginning of self.spam always the same distance from the beginning of > self.eggs? Or can I just forget the hard ground of assembler-metaphors > entirely as I float off into abstractville? I guess the fixed lengths I kept > getting on re-runs were coincidental but not to be relied on. Instance attributes are stored either in a dict or directly as slots. In Cpython, the location of the slots that are used for the dict, weak references, and other slots that you define are relative to the object base address. Normally each instance of a class has a slot for a reference count (e.g. sys.getrefcount(obj)), a pointer to the type (e.g. type(obj)), a pointer to a dict (e.g. vars(obj)), and a pointer to a linked list of weak references (e.g. weakref.getweakrefs(obj)). You can override this for __dict__ and __weakref__ and add your own slots by defining __slots__: class Example(object): __slots__ = '__dict__', 'v' Instances of Example have a dict and a slot named 'v', but they don't allow weak references (i.e. the __weakref__ slot). >>> ex = Example() >>> ex.v, ex.w = 5, 10 >>> weakref.ref(ex) Traceback (most recent call last): File "", line 1, in TypeError: cannot create weak reference to 'Example' object Example's dict has a member_descriptor for the 'v' slot. Instead of just using ex.v, we can be weird and manually bind the descriptor: >>> vars(Example)['v'].__get__(ex) 5 And do the same for the __dict__ getset_descriptor: >>> vars(Example)['__dict__'].__get__(ex) {'w': 10} CPython doesn't actually use the __dict__ getset_descriptor from the type's dict. It already has the instance dict offset as a field in the type structure. From marc.tompkins at gmail.com Tue Jul 23 22:04:14 2013 From: marc.tompkins at gmail.com (Marc Tompkins) Date: Tue, 23 Jul 2013 13:04:14 -0700 Subject: [Tutor] unichr not working as expected In-Reply-To: References: <51EDFFE2.8080605@pearwood.info> <51EE92F6.1040602@pearwood.info> Message-ID: On Tue, Jul 23, 2013 at 12:31 PM, Dave Angel wrote: > The three distributed version control systems I know of are: > git, mercurial, and bazaar > Not to overplay my Joel Spolsky fanboiism, but are you aware of Kiln from Fog Creek? It started out as an enterprise-friendly wrapper around Mercurial (which is still Joel's favorite, apparently), but they decided to make it agnostic: it now round-trips seamlessly between git and Mercurial, with users of either not needing to know or care what other users prefer. Of course, I haven't actually used it myself, so I only have the developers' word to go by that it's the greatest thing since sliced bread - but I thought I'd mention it. http://www.joelonsoftware.com/items/2013/03/11.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Tue Jul 23 22:20:54 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Wed, 24 Jul 2013 06:20:54 +1000 Subject: [Tutor] Off Topic re: unichr not working as expected In-Reply-To: References: <51EDFFE2.8080605@pearwood.info> <51EE92F6.1040602@pearwood.info> Message-ID: <51EEE5A6.2040808@pearwood.info> On 24/07/13 04:53, Jim Mooney wrote: > On 23 July 2013 07:28, Steven D'Aprano wrote: > >> >> (I trust that everyone remembers enough high school chemistry to >> understand why Mercurial is called hg? Hg is the chemical symbol for >> Mercury.) >> > > And I recall my high school chemistry teacher claiming the noble gases > could never combine, until someone just decided to cook up some Xenon > Tetraflouride. Always fun when you get to contradict your teacher in class, > and you're right - wel it was then. I'm a tad more mature, now - sometimes > ;') Completely off-topic, you may get a kick out of "Things I Won't Work With", where a professional chemist periodically writes about chemical compounds that he won't work with for various reasons: - chemicals that set fire to sand, bricks, and asbestos; - chemicals that stink so unbearably that fogging the air with nitric oxide actually *improves* the air quality; - chemicals that detonate at -300?F; - chemicals so explosive that mixing them with TNT makes them more stable; - solutions that become more dangerous rather than less when you put them in the freezer; - chemicals more reactive than elemental fluorine -- some of which are made *from* elemental fluorine; - and chemicals which have short-term acute toxic effects, long-term chronic toxic effects if you survive the acute ones, and are carcinogenic if you survive the chronic toxicity. http://pipeline.corante.com/archives/things_i_wont_work_with/ -- Steven From eryksun at gmail.com Tue Jul 23 23:07:56 2013 From: eryksun at gmail.com (eryksun) Date: Tue, 23 Jul 2013 17:07:56 -0400 Subject: [Tutor] object size in python is in what units? In-Reply-To: References: Message-ID: On Tue, Jul 23, 2013 at 2:27 PM, Jim Mooney wrote: > > And yet I still get 36 with this: > > bigKronk = Kronk('supercalifracilisticexpalidocious, even though the sound > of it is really quite atrocious') > sys.getsizeof(bigKronk) > 36 I just noticed that you're using an old-style class. Remember to inherit from object in 2.x. The size of 36 bytes on 32-bit Windows Python gives it away. An old-style instance needs an extra pointer to the class object because its type is actually types.InstanceType. > So I guess the object is only pointing to its contents. Which I assume > means an object is just a bunch of names bound to pointers. That depends on the object. Built-in objects can have widely varying structure (beyond the base refcount and type tag). An immutable object such as a string or tuple will usually have its data allocated with the object itself, while a list has a pointer to an array, which can change while the base address of the list itself remains constant. When it comes to __sizeof__, most types include their variable size as well -- but not all do (e.g. a NumPy array doesn't). Typically the size doesn't include the size of referenced objects. You'd have to calculate that yourself by walking the references. It's up to you what to include in your own __sizeof__ method; just don't do something silly like this: class NegativeSize(object): __slots__ = () def __sizeof__(self): return -1 >>> sys.getsizeof(NegativeSize()) -1 From kbailey at howlermonkey.net Wed Jul 24 03:13:03 2013 From: kbailey at howlermonkey.net (Kirk Bailey) Date: Tue, 23 Jul 2013 21:13:03 -0400 Subject: [Tutor] script in Raspberry pi python Message-ID: <51EF2A1F.2020206@howlermonkey.net> Python works fine in pi. script works fine IF I invoke it high church: python ./RR.py but this fails: ./RR.py the shebang is #!/usr/bon/python which points at python. Now, someone with some clue and an ugly stick hammer my head, I already have some clue glue. Questions gladly answered, dumb looks still free. -- -Shaboom. Kirk Bailey CEO, Freehold Marketing LLC http://www.OneBuckHosting.com/ Fnord! From amitsaha.in at gmail.com Wed Jul 24 03:42:17 2013 From: amitsaha.in at gmail.com (Amit Saha) Date: Wed, 24 Jul 2013 11:42:17 +1000 Subject: [Tutor] script in Raspberry pi python In-Reply-To: <51EF2A1F.2020206@howlermonkey.net> References: <51EF2A1F.2020206@howlermonkey.net> Message-ID: On Wed, Jul 24, 2013 at 11:13 AM, Kirk Bailey wrote: > Python works fine in pi. script works fine IF I invoke it high church: > python ./RR.py > > but this fails: ./RR.py > the shebang is > #!/usr/bon/python bon -> bin Also, don't forget to chmod +x RR.py > which points at python. > > Now, someone with some clue and an ugly stick hammer my head, I already have > some clue glue. > > Questions gladly answered, dumb looks still free. > > -- > > -Shaboom. > > Kirk Bailey > CEO, Freehold Marketing LLC > http://www.OneBuckHosting.com/ > Fnord! > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor -- http://echorand.me From pasokan at talentsprint.com Wed Jul 24 05:40:57 2013 From: pasokan at talentsprint.com (Asokan Pichai) Date: Wed, 24 Jul 2013 09:10:57 +0530 Subject: [Tutor] Off Topic re: unichr not working as expected In-Reply-To: <51EEE5A6.2040808@pearwood.info> References: <51EDFFE2.8080605@pearwood.info> <51EE92F6.1040602@pearwood.info> <51EEE5A6.2040808@pearwood.info> Message-ID: On Wed, Jul 24, 2013 at 1:50 AM, Steven D'Aprano wrote: > On 24/07/13 04:53, Jim Mooney wrote: > >> On 23 July 2013 07:28, Steven D'Aprano wrote: >> >> >>> (I trust that everyone remembers enough high school chemistry to >>> understand why Mercurial is called hg? Hg is the chemical symbol for >>> Mercury.) >>> >>> >> And I recall my high school chemistry teacher claiming the noble gases >> could never combine, until someone just decided to cook up some Xenon >> Tetraflouride. Always fun when you get to contradict your teacher in >> class, >> and you're right - wel it was then. I'm a tad more mature, now - sometimes >> ;') >> > Way offtopic: It is Xenon Hexa Flouride IIRC. LOve that book 107 stories about CHemistry Asokan Pichai "Expecting the world to treat you fairly because you are a good person is a little like expecting the bull to not attack you because you are a vegetarian" -------------- next part -------------- An HTML attachment was scrubbed... URL: From paulrsmith7777 at gmail.com Wed Jul 24 06:11:06 2013 From: paulrsmith7777 at gmail.com (Paul Smith) Date: Wed, 24 Jul 2013 00:11:06 -0400 Subject: [Tutor] sqlite3 woes opening Moz, FFox sqlite db with only the inevitable 'DatabaseError: file is encrypted or is not a database' occurring... Message-ID: title = three days lost and counting... Thanks in advance all: Ok- I want to work with data in FFox, specifically a sqlite db found here.. 'C:\Users\Hive2\AppData\Roaming\Mozilla\Firefox\Profiles\zabt0uq4.default\places.sqlite' ...more specifically my FFox history found in a table 'moz_places'. Running Py 2.7.5 on a windows 7-64 machine and after days of *consulting -*google; various-boards ;stackoverflow; py.org; sqlite.org; mingw32?(prereq. for sqlite v. 2.6.3?); sphynx?(another prereq. for I forget); and *addressing* - possible version differences between sqlite3 and FFoxdb; file path confirmation; db creation - query using sqlite3(successful); etc. etc. We end up here... import sqlite3 as lite import sys import os print ("sqlite version info1:",lite.version)##clunky version confirm print ("sqlite version info2:",lite.sqlite_version,"\n")##clunk's cousin mypath = os.path.realpath("C:\\Users\\Hive2\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\zabt0uq4.default\\places.sqlite") mydb = (mypath)## I'm trying to confirm the path to the friggin' thing print ("This is my db path""\n"+(mydb)) con = None con = lite.connect(mydb) with con: cur = con.cursor() cur.execute("SELECT * FROM 'moz_places'")## it is a table in places.sqlite seems apropos print (con.fetchall()) ## print it all? con.close() Which produces... Python 2.7.5 (default, May 15 2013, 22:44:16) [MSC v.1500 64 bit (AMD64)] on win32 Type "copyright", "credits" or "license()" for more information. >>> ================================ RESTART ================================ >>> ('sqlite version info1:', '2.6.0') ('sqlite version info2:', '3.6.21', '\n') This is my db path C:\Users\Hive2\AppData\Roaming\Mozilla\Firefox\Profiles\zabt0uq4.default\places.sqlite Traceback (most recent call last): File "E:\pyurlseek\zedpathconfirm.py", line 15, in cur.execute("SELECT * FROM 'moz_places'") DatabaseError: file is encrypted or is not a database I just want to print out the table (or anything from the FFox db just to prove it is being opened/accessed?!) I am sure it is a simple error but being a real newb that I am I cannot escape the dreaded 'DatabaseError: file is encrypted or is not a database', time and time again. Help please Thanks in advance, PSmith Additional items: no I did not have FFox open when running; yes I can access the FFox sqlite db from 3 other programs like SQLite Expert etc. yes I do need some sleep now so, ciao. -------------- next part -------------- An HTML attachment was scrubbed... URL: From __peter__ at web.de Wed Jul 24 10:20:16 2013 From: __peter__ at web.de (Peter Otten) Date: Wed, 24 Jul 2013 10:20:16 +0200 Subject: [Tutor] sqlite3 woes opening Moz, FFox sqlite db with only the inevitable 'DatabaseError: file is encrypted or is not a database' occurring... References: Message-ID: Paul Smith wrote: > title = three days lost and counting... > > Thanks in advance all: > > Ok- I want to work with data in FFox, specifically a sqlite db found > here.. > > 'C: \Users\Hive2\AppData\Roaming\Mozilla\Firefox\Profiles\zabt0uq4.default\places.sqlite' > > ...more specifically my FFox history found in a table 'moz_places'. > > Running Py 2.7.5 on a windows 7-64 machine and after days of > *consulting -*google; various-boards ;stackoverflow; > py.org; sqlite.org; mingw32?(prereq. for sqlite v. 2.6.3?); > sphynx?(another prereq. for I forget); and *addressing* - possible version > differences between sqlite3 and FFoxdb; file path confirmation; db > creation - query using sqlite3(successful); etc. etc. > > We end up here... > > import sqlite3 as lite > import sys > import os > > print ("sqlite version info1:",lite.version)##clunky version confirm > print ("sqlite version info2:",lite.sqlite_version,"\n")##clunk's cousin > > mypath = > os.path.realpath("C: \\Users\\Hive2\\AppData\\Roaming\\Mozilla\\Firefox\\Profiles\\zabt0uq4.default\\places.sqlite") > mydb = (mypath)## I'm trying to confirm the path to the friggin' thing > print ("This is my db path""\n"+(mydb)) > con = None > con = lite.connect(mydb) > with con: > cur = con.cursor() > cur.execute("SELECT * FROM 'moz_places'")## it is a table in places.sqlite > seems apropos > print (con.fetchall()) ## print it all? > con.close() > > Which produces... > > Python 2.7.5 (default, May 15 2013, 22:44:16) [MSC v.1500 64 bit (AMD64)] > on win32 > Type "copyright", "credits" or "license()" for more information. >>>> ================================ RESTART > ================================ >>>> > ('sqlite version info1:', '2.6.0') > ('sqlite version info2:', '3.6.21', '\n') > This is my db path > C: \Users\Hive2\AppData\Roaming\Mozilla\Firefox\Profiles\zabt0uq4.default\places.sqlite > > Traceback (most recent call last): > File "E:\pyurlseek\zedpathconfirm.py", line 15, in > cur.execute("SELECT * FROM 'moz_places'") > DatabaseError: file is encrypted or is not a database > > I just want to print out the table (or anything from the FFox db just to > prove it is being opened/accessed?!) > > I am sure it is a simple error but being a real newb that I am I cannot > escape the dreaded 'DatabaseError: file is encrypted or is not a > database', time and time again. > > Help please > > Thanks in advance, > > PSmith > > Additional items: no I did not have FFox open when running; yes I can > access the FFox sqlite db from 3 other programs like SQLite Expert etc. > yes I do need some sleep now so, ciao. Trying my google-fu, looking for the error message I find http://trac.edgewall.org/wiki/PySqlite#DatabaseError:fileisencryptedorisnotadatabase """ DatabaseError: file is encrypted or is not a database There are typically three situations in which this somewhat cryptic error message can be seen: - when trying to modify the database file and the write permission is missing; fix the permissions - when accessing a SQLite 2.x database file with a SQLite 3.x library; see above for the upgrade instructions. - when accessing a database that has been created in or explicitly converted to WAL mode, using a version of SQLite older than 3.7.0 (i.e. with no WAL support); upgrade your bindings to use a recent SQLite """ Hm, what's WAL mode? There's a link on this very page, so I shun google and go directly to http://www.sqlite.org/wal.html """ The default method by which SQLite implements atomic commit and rollback is a rollback journal. Beginning with version 3.7.0, a new "Write-Ahead Log" option (hereafter referred to as "WAL") is available. [...] Activating And Configuring WAL Mode An SQLite database connection defaults to journal_mode=DELETE. To convert to WAL mode, use the following pragma: PRAGMA journal_mode=WAL; """ But does Firefox use WAL mode? Luckily I don't get the error over here: >>> import sqlite3 >>> db = sqlite3.connect("places.sqlite") >>> cs = db.cursor() >>> for row in cs.execute("pragma journal_mode;"): print row ... (u'wal',) So yes. You need a newer version of sqlite to read the database. I don't know what version of sqlite is shipped with Python 3.3 -- maybe a windows user can chime in. If you don't want to switch to Python 3.3 -- or if it doesn't feature an sqlite version that supports WAL -- you could use sqlite's commandline tools to switch off WAL on a copy (!) of places.sqlite. From wprins at gmail.com Wed Jul 24 10:43:11 2013 From: wprins at gmail.com (Walter Prins) Date: Wed, 24 Jul 2013 09:43:11 +0100 Subject: [Tutor] sqlite3 woes opening Moz, FFox sqlite db with only the inevitable 'DatabaseError: file is encrypted or is not a database' occurring... In-Reply-To: References: Message-ID: Hi Peter, On 24 July 2013 09:20, Peter Otten <__peter__ at web.de> wrote: > If you don't want to switch to Python 3.3 -- or if it doesn't feature an > sqlite version that supports WAL -- you could use sqlite's commandline > tools > to switch off WAL on a copy (!) of places.sqlite. I've just gone through similar steps as you and similarly determined that the reason for Paul's trouble is the version of sqlite3.dll included with the latest version of Python 2.7 doesn't support Firefox's DB. I'd like to add that an alternative solution to switching Python versions is to simply replace (rename the original first) sqlite3.dll in "c:\Python27\DLLs" with the latest on the SQLite site. I've just tested it and it works fine for my install of Python 2.7.2.5 32-bit with Firefox's places.sqlite after the update. The dll can be downloaded here under "Precompiled binaries for Windows": http://www.sqlite.org/download.html Walter -------------- next part -------------- An HTML attachment was scrubbed... URL: From __peter__ at web.de Wed Jul 24 11:03:33 2013 From: __peter__ at web.de (Peter Otten) Date: Wed, 24 Jul 2013 11:03:33 +0200 Subject: [Tutor] sqlite3 woes opening Moz, FFox sqlite db with only the inevitable 'DatabaseError: file is encrypted or is not a database' occurring... References: Message-ID: Walter Prins wrote: > Hi Peter, > > > On 24 July 2013 09:20, Peter Otten <__peter__ at web.de> wrote: > >> If you don't want to switch to Python 3.3 -- or if it doesn't feature an >> sqlite version that supports WAL -- you could use sqlite's commandline >> tools >> to switch off WAL on a copy (!) of places.sqlite. > > > I've just gone through similar steps as you and similarly determined that > the reason for Paul's trouble is the version of sqlite3.dll included with > the latest version of Python 2.7 doesn't support Firefox's DB. > > I'd like to add that an alternative solution to switching Python versions > is to simply replace (rename the original first) sqlite3.dll in > "c:\Python27\DLLs" with the latest on the SQLite site. I've just tested > it and it works fine for my install of Python 2.7.2.5 32-bit with > Firefox's > places.sqlite after the update. The dll can be downloaded here under > "Precompiled binaries for Windows": > http://www.sqlite.org/download.html That's indeed less roundabout. I had no idea you could update Python's sqlite version on windows without a C compiler... From nsivaram.net at gmail.com Wed Jul 24 18:49:14 2013 From: nsivaram.net at gmail.com (Sivaram Neelakantan) Date: Wed, 24 Jul 2013 22:19:14 +0530 Subject: [Tutor] suggestions for splitting file based on date References: <87a9litkta.fsf@gmail.com> <87siz9rfvs.fsf@gmail.com> Message-ID: <87bo5rucmt.fsf@gmail.com> On Sat, Jul 20 2013,David Hutto wrote: > Why not use the actual month? With a simple x/y canvas in Tkinter you could > plot by the months with polygon coordinates as your data visualization, or > in 30 day /etc windows, just the price(y) being a derivative of x(the > frequency of changes), and create simple line segments with polygon > coordinates, and refine them to hourly/daily/weekly/monthly/yearly, or in > 30/60/90 day increments with a little url scraping. [snipped 5 lines] I haven't looked into Tkinter much but will do. sivaram -- From wprins at gmail.com Thu Jul 25 01:50:43 2013 From: wprins at gmail.com (Walter Prins) Date: Thu, 25 Jul 2013 00:50:43 +0100 Subject: [Tutor] sqlite3 woes opening Moz, FFox sqlite db with only the inevitable 'DatabaseError: file is encrypted or is not a database' occurring... In-Reply-To: References: Message-ID: Hi, On 24 July 2013 17:08, Paul Smith wrote: > Thanks but still stuck... > > Walt- > Pete- > > I am trying to copy-update py 2.7.5 with latest sqlite3 for version fix > but am running win 7 64bit and am getting these errors on both regsrv32 and > WOW64 while trying to register new Dll's... > Are you running 32 bit Python or 64-bit Python? You need to match the dll to your Python build, not to the Windows OS build. I note that the official sqlite download page does NOT have a simple 64-bit build available. This together with your trouble trying to use the DLL implies to me that you're probably running 64-bit Python 2.7 which cannot directly use a 32-bit dll. Am I right in thinking you're using 64-bit Python build? > Python27\DLLs>regsvr32 sqlite3.dll > > the module"sqlite3.dll" was loaded but the entry-point DllRegisterServer > was not found > > make sure that sqlite3.dll is a valid dll or ocx file then try again > Why are you trying to register the dll? It's not a COM/ActiveX server, it's just a plain DLL. All you need to do is replace the old with the new. Done. Nothing else. ...so other than the attempted registering the new dll fail I also get a > ASCII error related to _init_ > > Installed sqlite 3.7.17 by itself and running from cmd line pointing to > FFox db - it works! so by itself the new sqlite 3.7.17 version works, now > to hammer it into py 2.7.5... > For reference, if you're indeed using a 64-bit build of Python, I've managed to track down a 64-bit build of the sqlite3.dll here: http://synopse.info/files/SQLite3-64.7z For reference, this was via the following page so is apparently legit: http://blogs.embarcadero.com/davidi/2013/01/25/42392 Aside, I notice the DLL file is compressed with 7Zip, so you'll need 7Zip (or another archiver that supports the 7z format) to decompress it: http://www.7-zip.org/ Alternatively to all the above, you can build a 64-bit DLL yourself. ;) Here's a relevant stack overflow question if you're interested in that rabbit hole...: http://stackoverflow.com/questions/10978020/compiling-sqlite-for-windows-64-bit Personally though, I'd suggest just using 32-bit Python unless you have a *really* good reason not to do so. Walter -------------- next part -------------- An HTML attachment was scrubbed... URL: From kbailey at howlermonkey.net Thu Jul 25 19:02:21 2013 From: kbailey at howlermonkey.net (Kirk Bailey) Date: Thu, 25 Jul 2013 13:02:21 -0400 Subject: [Tutor] script in Raspberry pi python In-Reply-To: References: <51EF2A1F.2020206@howlermonkey.net> Message-ID: <51F15A1D.6040402@howlermonkey.net> On 7/23/2013 9:42 PM, Amit Saha wrote: > On Wed, Jul 24, 2013 at 11:13 AM, Kirk Bailey wrote: >> Python works fine in pi. script works fine IF I invoke it high church: >> python ./RR.py >> >> but this fails: ./RR.py >> the shebang is >> #!/usr/bon/python > bon -> bin right. > > Also, don't forget to chmod +x RR.py already done... I think... yep, 755. RWXR.XR.X! Therefore, Fhat the Wuck over? > >> which points at python. >> >> Now, someone with some clue and an ugly stick hammer my head, I already have >> some clue glue. >> >> Questions gladly answered, dumb looks still free. >> >> -- >> >> -Shaboom. >> >> Kirk Bailey >> CEO, Freehold Marketing LLC >> http://www.OneBuckHosting.com/ >> Fnord! >> >> _______________________________________________ >> Tutor maillist - Tutor at python.org >> To unsubscribe or change subscription options: >> http://mail.python.org/mailman/listinfo/tutor > > -- -Shaboom. Kirk Bailey CEO, Freehold Marketing LLC http://www.OneBuckHosting.com/ Fnord! From davea at davea.name Thu Jul 25 20:19:53 2013 From: davea at davea.name (Dave Angel) Date: Thu, 25 Jul 2013 14:19:53 -0400 Subject: [Tutor] script in Raspberry pi python In-Reply-To: <51F15A1D.6040402@howlermonkey.net> References: <51EF2A1F.2020206@howlermonkey.net> <51F15A1D.6040402@howlermonkey.net> Message-ID: On 07/25/2013 01:02 PM, Kirk Bailey wrote: > > On 7/23/2013 9:42 PM, Amit Saha wrote: >> On Wed, Jul 24, 2013 at 11:13 AM, Kirk Bailey >> wrote: >>> Python works fine in pi. script works fine IF I invoke it high church: >>> python ./RR.py >>> >>> but this fails: ./RR.py >>> the shebang is >>> #!/usr/bon/python >> bon -> bin > right. Take a look at that file with a hex viewer. If you edited in Windows, it'll probably have 0d0a line endings. Those do NOT work with a shebang line. If that's the problem, then you can use dos2unix or equivalent utility to fix it up. Or your favorite Linux editor. If that's still not it, then how about using copy/paste to show us exactly what's happening. "Fails" is not very descriptive. Show us the console screen for something like: cat RR.py #!/usr/bin/python print("Hello world") ./RR.py As-many-as-5-lines-of-traceback error message, if it's a Python exception, and something different if it's a bash error. -- DaveA From wprins at gmail.com Thu Jul 25 21:45:52 2013 From: wprins at gmail.com (Walter Prins) Date: Thu, 25 Jul 2013 20:45:52 +0100 Subject: [Tutor] Fwd: script in Raspberry pi python In-Reply-To: References: <51EF2A1F.2020206@howlermonkey.net> <51F15A1D.6040402@howlermonkey.net> <51F1725E.9@howlermonkey.net> Message-ID: Hi, On 25 July 2013 19:45, Kirk Bailey wrote: > which python > /usr/bin/python > root at KirksPiBox1:/home/pi: ./RR.py > bash: ./RR.py: /usr/bin/python^M: bad interpreter: No such file or > directory > root at KirksPiBox1:home/pi:_ > Dave's correct. That "^M" represents ctlr-m, and is the displayable equivalent of the "Carriage Return" character. Windows text files uses CR/LF (Carriage Return, Line Feed) characters to delimit lines of text in text files. Unix uses only "Line Feed". So the extra carriage return character is taken to be part of the text at the end of the shebang line. And "/usr/bin/python" is not the same as "/usr/bin/python^M" Here's some background: http://mag-sol.com/articles/crlf.html How and where did you create the file? If with an editor, which editor? Walter -------------- next part -------------- An HTML attachment was scrubbed... URL: From cybervigilante at gmail.com Thu Jul 25 23:49:50 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Thu, 25 Jul 2013 14:49:50 -0700 Subject: [Tutor] use of the newer dict types Message-ID: If you do dict.keys() in 2.7 you get a list, which is quite handy. But if you do it in 3.3 you get this odd dict_keys type, for which I have yet to find a use, and which just requires an extra list step to use. The same for values. Since most changes from 2.7 to 3.3 were improvements, what am I missing here? What is the use of these types? -- Jim Eric Holder never saw a Bankster he didn't love. From davea at davea.name Fri Jul 26 00:37:53 2013 From: davea at davea.name (Dave Angel) Date: Thu, 25 Jul 2013 18:37:53 -0400 Subject: [Tutor] use of the newer dict types In-Reply-To: References: Message-ID: On 07/25/2013 05:49 PM, Jim Mooney wrote: > If you do dict.keys() in 2.7 you get a list, which is quite handy. But > if you do it in 3.3 you get this odd dict_keys type, for which I have > yet to find a use, and which just requires an extra list step to use. > The same for values. Since most changes from 2.7 to 3.3 were > improvements, what am I missing here? What is the use of these types? > The difference is analogous to the difference between range and xrange (in Python 2.x). The former is an iterator, rather than a list. If you're just going to use it in a loop anyway, xrange is smaller and faster. So 3.x just renamed xrange to range, and dropped the old range. For dictionaries, instead of returning a list, keys() returns a dictionary view. See http://docs.python.org/3/library/stdtypes.html#dict-views What's not clear to me is whether this dictionary view object is safe to use when the dictionary is changing. The page referenced above seems to say both yes and no. -- DaveA From eryksun at gmail.com Fri Jul 26 00:54:44 2013 From: eryksun at gmail.com (eryksun) Date: Thu, 25 Jul 2013 18:54:44 -0400 Subject: [Tutor] use of the newer dict types In-Reply-To: References: Message-ID: On Thu, Jul 25, 2013 at 6:37 PM, Dave Angel wrote: > > For dictionaries, instead of returning a list, keys() returns a dictionary > view. See http://docs.python.org/3/library/stdtypes.html#dict-views > > What's not clear to me is whether this dictionary view object is safe to use > when the dictionary is changing. The page referenced above seems to say > both yes and no. The view isn't an iterator. A separate iterator gets created for that: >>> d = {'a': 1} >>> keys = d.keys() >>> it = iter(keys) >>> d['b'] = 2 # invalidate "it" >>> list(it) Traceback (most recent call last): File "", line 1, in RuntimeError: dictionary changed size during iteration This creates and iterates a new dict_keyiterator: >>> list(keys) ['b', 'a'] From cybervigilante at gmail.com Fri Jul 26 01:03:28 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Thu, 25 Jul 2013 16:03:28 -0700 Subject: [Tutor] use of the newer dict types In-Reply-To: References: Message-ID: Dave Angel > What's not clear to me is whether this dictionary view object is safe to use > when the dictionary is changing. The page referenced above seems to say > both yes and no. Ah, now that makes some sense - they literally change as the dictionary does. That could be useful: >>> kv = {'b': 'beta', 'c': 'cappa', 'a': 'kark', 'd': 'delta', 'e': 'epsilon'} >>> kvkeys = kv.keys() >>> kvkeys dict_keys(['b', 'c', 'a', 'd', 'e']) >>> del kv['b'] >>> kv {'a': 'kark', 'c': 'cappa', 'd': 'delta', 'e': 'epsilon'} >>> kvkeys dict_keys(['c', 'a', 'd', 'e']) # b is missing Jim "Now Operation 7 must either bring out a result equal to zero (if n=1); or a result greater than zero, as in the present case; and the engine follows the one or the other of the two courses just explained, contingently on the one or the other result of Operation 7." --Ada Lovelace, the first computer programmer, describes the very first computable if-then. She was followed by Grace Hopper, the COBOL genius. Men were slower to take up the profession of programming ;') From davea at davea.name Fri Jul 26 01:51:28 2013 From: davea at davea.name (Dave Angel) Date: Thu, 25 Jul 2013 19:51:28 -0400 Subject: [Tutor] use of the newer dict types In-Reply-To: References: Message-ID: On 07/25/2013 06:54 PM, eryksun wrote: > On Thu, Jul 25, 2013 at 6:37 PM, Dave Angel wrote: >> >> For dictionaries, instead of returning a list, keys() returns a dictionary >> view. See http://docs.python.org/3/library/stdtypes.html#dict-views >> >> What's not clear to me is whether this dictionary view object is safe to use >> when the dictionary is changing. The page referenced above seems to say >> both yes and no. > > The view isn't an iterator. A separate iterator gets created for that: > > >>> d = {'a': 1} > >>> keys = d.keys() > >>> it = iter(keys) > > >>> d['b'] = 2 # invalidate "it" > > >>> list(it) > Traceback (most recent call last): > File "", line 1, in > RuntimeError: dictionary changed size during iteration > > This creates and iterates a new dict_keyiterator: > > >>> list(keys) > ['b', 'a'] OK, so the view is apparently an iterable. And it's apparently safe to change the dictionary after creating the view, UNTIL the iterator is created. And perhaps the dictionary isn't actually traversed till the iterator is created. I still don't see the advantage. Isn't the iterator going to be as big as the list would have been, and take just as long to build? -- DaveA From eryksun at gmail.com Fri Jul 26 02:05:24 2013 From: eryksun at gmail.com (eryksun) Date: Thu, 25 Jul 2013 20:05:24 -0400 Subject: [Tutor] use of the newer dict types In-Reply-To: References: Message-ID: On Thu, Jul 25, 2013 at 7:03 PM, Jim Mooney wrote: > Ah, now that makes some sense - they literally change as the > dictionary does. That could be useful: Just to be clear, the view object doesn't change. Its __repr__, __len__, rich comparison, and set methods are tied to the underlying dict. As long as the view exists, the dict won't be garbage collected. From eryksun at gmail.com Fri Jul 26 03:27:58 2013 From: eryksun at gmail.com (eryksun) Date: Thu, 25 Jul 2013 21:27:58 -0400 Subject: [Tutor] use of the newer dict types In-Reply-To: References: Message-ID: On Thu, Jul 25, 2013 at 7:51 PM, Dave Angel wrote: > > And it's apparently safe to change the dictionary after creating the view, > UNTIL the iterator is created. Notice the error the iterator raised in my example was just about the dict changing size. If the dict changes between steps while remaining the same size, the iterator doesn't have a clue: >>> d = dict.fromkeys('abcdef') >>> d2 = dict.fromkeys('ghijkl') >>> keys = d.keys() >>> it = iter(keys) >>> next(it) 'f' >>> next(it) 'e' >>> d.clear() >>> d.update(d2) >>> list(it) ['l', 'k', 'j', 'i', 'h'] 'g' got skipped. The result here depends on the iterator's current index into the hash table and the positions that the updated values hash to. > I still don't see the advantage. Isn't the iterator going to be as big as > the list would have been, and take just as long to build? It already has the hash table. It just has to iterate it looking for non-NULL values, and keep track of the previous index between calls to __next__(). Another bit of state the iterator maintains is a length hint, which helps to optimize memory allocation when using the iterator to create a new sequence: >>> d = dict.fromkeys('abcdef') >>> it = iter(d) >>> it.__length_hint__() 6 >>> next(it) 'f' >>> it.__length_hint__() 5 From davea at davea.name Fri Jul 26 03:38:08 2013 From: davea at davea.name (Dave Angel) Date: Thu, 25 Jul 2013 21:38:08 -0400 Subject: [Tutor] use of the newer dict types In-Reply-To: References: Message-ID: On 07/25/2013 09:27 PM, eryksun wrote: > On Thu, Jul 25, 2013 at 7:51 PM, Dave Angel wrote: >> >> And it's apparently safe to change the dictionary after creating the view, >> UNTIL the iterator is created. > > Notice the error the iterator raised in my example was just about the > dict changing size. If the dict changes between steps while remaining > the same size, the iterator doesn't have a clue: So in other words, it's unsafe to change the dictionary after creating the iterator, but it won't necessarily cause an exception; it might happen to work, or it might just get weird results. > > >>> d = dict.fromkeys('abcdef') > >>> d2 = dict.fromkeys('ghijkl') > > >>> keys = d.keys() > >>> it = iter(keys) > >>> next(it) > 'f' > >>> next(it) > 'e' > > >>> d.clear() > >>> d.update(d2) > >>> list(it) > ['l', 'k', 'j', 'i', 'h'] > > 'g' got skipped. The result here depends on the iterator's current > index into the hash table and the positions that the updated values > hash to. > >> I still don't see the advantage. Isn't the iterator going to be as big as >> the list would have been, and take just as long to build? > > It already has the hash table. It just has to iterate it looking for > non-NULL values, and keep track of the previous index between calls to > __next__(). In other words, no? The iterator assumes the dict is no longer going to change once the iterator is created, so it can just store iteration state. > > Another bit of state the iterator maintains is a length hint, which > helps to optimize memory allocation when using the iterator to create > a new sequence: > > >>> d = dict.fromkeys('abcdef') > >>> it = iter(d) > >>> it.__length_hint__() > 6 > >>> next(it) > 'f' > >>> it.__length_hint__() > 5 It'd sure be nice if the library reference actually specified the external behavior, though I have no problem with it omitting things like __length_hint__. Those are implementation details. -- DaveA From steve at pearwood.info Fri Jul 26 04:16:09 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Fri, 26 Jul 2013 12:16:09 +1000 Subject: [Tutor] use of the newer dict types In-Reply-To: References: Message-ID: <51F1DBE9.4010405@pearwood.info> On 26/07/13 07:49, Jim Mooney wrote: > If you do dict.keys() in 2.7 you get a list, which is quite handy. But > if you do it in 3.3 you get this odd dict_keys type, for which I have > yet to find a use, and which just requires an extra list step to use. > The same for values. Since most changes from 2.7 to 3.3 were > improvements, what am I missing here? What is the use of these types? Oh wow, deja vu. The same question was asked on comp.lang.python a day or two ago. Views are what Python dicts should have had 20 years ago, but didn't. Python 3 finally fixes that ancient design. In Python 2, the preferred way to iterate over a dictionary's keys is: for key in mydict: ... There is no need to extract the keys into a list, then iterate over the list. That does not change in Python 3. In Python 2, the preferred way to iterate over values or key/value pairs was to use an iterator, since that is more light-weight than a list: for value in mydict.itervalues(): ... is preferred over just mydict.values() since it doesn't make a full list copy of the values. For merely iterating over the items one at a time, it is wasteful and inefficient to build a list, and calling values() should be avoided. Likewise for iteritems() versus items(). On the rare occasions you actually need a list, simply call list() explicitly on the view. In Python 3, you just iterate over mydict.values() or items(), which return views. For plain iterating, the only difference between the two is that iterators can only be run through once, while views can be run over again and again. If you really need a one-shot iterator, just call iter() on the view. Views are similar to sets, and support some set operations, which is great for comparing elements of two dicts: py> d1 = dict.fromkeys([1, 2, 3, 4]) py> d2 = dict.fromkeys([3, 4, 5, 6]) py> d1.keys() & d2.keys() # keys that are in both {3, 4} py> d1.keys() ^ d2.keys() # keys not in both {1, 2, 5, 6} py> d1.keys() - d2.keys() # keys only in d1 {1, 2} py> d2.keys() - d1.keys() # keys only in d2 {5, 6} In Python 2.7, use viewkeys(). In Python 2.6, you're out of luck, and need to program that functionality yourself. Because views are set-like, they provide efficient, almost constant-time membership testing for keys and items. (But not values, for technical reasons.) Views also provide read-only, live access to dict keys, items and values. In both Python 2 and 3, it is safe to modify the value associated with a key, but not delete or add keys. Basically, it is unsafe to change the length of a container (dict, list, or set) while iterating over it. "Unsafe" means that doing so can lead you to unexpectedly skipping or repeating items. Dicts can detect some (but not all) changes, and explicitly raise an error. Consequently, iterating over a dict while deleting keys is one of the few cases where you actually do need to use a separate key list instead of a view. In that case, simply call for key in list(mydict): # or list(mydict.keys()) ... -- Steven From fomcl at yahoo.com Fri Jul 26 10:50:45 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Fri, 26 Jul 2013 01:50:45 -0700 (PDT) Subject: [Tutor] use of the newer dict types In-Reply-To: <51F1DBE9.4010405@pearwood.info> References: <51F1DBE9.4010405@pearwood.info> Message-ID: <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> >In Python 2, the preferred way to iterate over values or key/value pairs was to use an iterator, since that is more light-weight than a list: > >for value in mydict.itervalues(): ... if I have d = {1: 2}, I can do membership testing in the following two ways --but which one is the preferred way?: if 1 in d: ??? pass if d.has_key(1): ???? pass I find "in" (__contains__) more readable, but maybe has_key has advantages (speed)? From __peter__ at web.de Fri Jul 26 11:34:52 2013 From: __peter__ at web.de (Peter Otten) Date: Fri, 26 Jul 2013 11:34:52 +0200 Subject: [Tutor] use of the newer dict types References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> Message-ID: Albert-Jan Roskam wrote: > >>In Python 2, the preferred way to iterate over values or key/value pairs >>was to use an iterator, since that is more light-weight than a list: >> >>for value in mydict.itervalues(): ... > > if I have d = {1: 2}, I can do membership testing in the following two > ways --but which one is the preferred way?: if 1 in d: > pass > if d.has_key(1): > pass > I find "in" (__contains__) more readable, but maybe has_key has advantages > (speed)? "in" is faster (use timeit to verify) and your only option in Python 3. has_key() has been kept in Python 2 for backwards compatibility -- a long time ago (before 2.2, released in 2001) it was your only option. From alan.gauld at btinternet.com Fri Jul 26 13:12:19 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Fri, 26 Jul 2013 12:12:19 +0100 Subject: [Tutor] use of the newer dict types In-Reply-To: <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> Message-ID: On 26/07/13 09:50, Albert-Jan Roskam wrote: > if I have d = {1: 2}, I can do membership testing in the following > two ways --but which one is the preferred way?: > if 1 in d: > pass > if d.has_key(1): > pass In addition to being more readable (and the fact it's the only way in v3) 'in' has another big advantage - it's polymorphic: holds_1 = [] bigBox = [ {1:2,3:4}, ['a',2,;freddy',True], (1,2,3) ] for collection in bigBox: if 1 in collection: holds_1.append(collection) You can't do that using has_key(), the code has to test for dict type etc. So all in all 'in' is the way to go. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From sbjaved at gmail.com Fri Jul 26 13:22:50 2013 From: sbjaved at gmail.com (Saad Javed) Date: Fri, 26 Jul 2013 16:22:50 +0500 Subject: [Tutor] Rock, Paper, Scissors game Message-ID: I'm trying to understand how this code works. I've commented the line that I can't seem to understand. How did the guy come up with this? #!/usr/bin/env python import random #rock > scissor > paper > rock WEAPONS = 'Rock', 'Paper', 'Scissors' for i in range(0, 3): print "%d %s" % (i + 1, WEAPONS[i]) player = int(input ("Choose from 1-3: ")) - 1 cpu = random.choice(range(0, 3)) print "%s vs %s" % (WEAPONS[player], WEAPONS[cpu]) if cpu != player: if (player - cpu) % 3 < (cpu - player) % 3: #How does this line work? print "Player wins" else: print "CPU wins" else: print "tie!" Thanks. From davea at davea.name Fri Jul 26 14:20:35 2013 From: davea at davea.name (Dave Angel) Date: Fri, 26 Jul 2013 08:20:35 -0400 Subject: [Tutor] Rock, Paper, Scissors game In-Reply-To: References: Message-ID: On 07/26/2013 07:22 AM, Saad Javed wrote: > I'm trying to understand how this code works. I've commented the line > that I can't seem to understand. How did the guy come up with this? > > #!/usr/bin/env python > > import random > > #rock > scissor > paper > rock > > WEAPONS = 'Rock', 'Paper', 'Scissors' > > for i in range(0, 3): > print "%d %s" % (i + 1, WEAPONS[i]) > > player = int(input ("Choose from 1-3: ")) - 1 > cpu = random.choice(range(0, 3)) > > print "%s vs %s" % (WEAPONS[player], WEAPONS[cpu]) > if cpu != player: > if (player - cpu) % 3 < (cpu - player) % 3: #How does this line work? I'm not sure what aspect of it you're puzzled about, but % is the modulo operator. It produces the remainder upon integer division. So we're effectively dividing by 3, and keeping only the remainder, which will be 0, 1, or 2. It will only be zero if cpu==player, but we've already tested that. So the only two cases are 1 and 2. A little experimenting will show you that if the first part( (player-cpu)%3 ) is 1, the second will be 2, and vice versa. So the statement could be simplified to: if (player - cpu) % 3 == 1: Now is it clear? > print "Player wins" > else: > print "CPU wins" > else: > print "tie!" > BTW, that input statement should be raw_input, since you're running it on Python 2.x. -- DaveA From __peter__ at web.de Fri Jul 26 15:06:28 2013 From: __peter__ at web.de (Peter Otten) Date: Fri, 26 Jul 2013 15:06:28 +0200 Subject: [Tutor] Rock, Paper, Scissors game References: Message-ID: Saad Javed wrote: > I'm trying to understand how this code works. I've commented the line > that I can't seem to understand. How did the guy come up with this? > > #!/usr/bin/env python > > import random > > #rock > scissor > paper > rock > > WEAPONS = 'Rock', 'Paper', 'Scissors' > > for i in range(0, 3): > print "%d %s" % (i + 1, WEAPONS[i]) > > player = int(input ("Choose from 1-3: ")) - 1 > cpu = random.choice(range(0, 3)) > > print "%s vs %s" % (WEAPONS[player], WEAPONS[cpu]) > if cpu != player: > if (player - cpu) % 3 < (cpu - player) % 3: #How does this line work? I would never have come up with this line, but I think it can be pictured as follows: Given the cycle +-->rock-->paper-->scissors--+ | | +---------------<------------+ and only one direction allowed the winning "weapon" is always the one farther away from the loser, e. g. rock vs paper rock-->paper: 1 step paper-->scissors-->rock: 2 steps paper wins rock vs rock: 0 steps both tie The modulo operator helps with determining this "directed distance" by maintaining -n == N-n e. g. -1 % 3 == 3 - 1 == 2 You could anecdotally test the above outline by adding more weapons. For four weapons you should get ties between distinct weapons: +-->rock-->paper-->scissors-->banana--+ | | +---------------<---------------------+ rock vs scissors: rock-->paper-->scissors: 2 steps scissors-->banana-->rock: 2 steps tie You can modify the script to work with an arbitrary number of weapons by replacing 3 with length(weapons) if you like. > print "Player wins" > else: > print "CPU wins" > else: > print "tie!" > > Thanks. From eryksun at gmail.com Fri Jul 26 19:38:44 2013 From: eryksun at gmail.com (eryksun) Date: Fri, 26 Jul 2013 13:38:44 -0400 Subject: [Tutor] Rock, Paper, Scissors game In-Reply-To: References: Message-ID: On Fri, Jul 26, 2013 at 9:06 AM, Peter Otten <__peter__ at web.de> wrote: > You could anecdotally test the above outline by adding more weapons. For > four weapons you should get ties between distinct weapons: > > +-->rock-->paper-->scissors-->banana--+ > | | > +---------------<---------------------+ > > rock vs scissors: > rock-->paper-->scissors: 2 steps > scissors-->banana-->rock: 2 steps > tie > > You can modify the script to work with an arbitrary number of weapons by > replacing 3 with length(weapons) if you like. Nice analysis. I propose a rule to minimize the occurrence of ties and preserve the classic pairwise results when extending "rock, paper, scissors" using modulo (I'm sure this has been analyzed to death in academia, but I haven't research it). Extending to N items should require N to be odd, and rock, paper, and scissors should be at indexes 0, N//2, and N-1, respectively. The loser is the closer of the pair in the clockwise direction: +-->rock-->X-->paper-->Y-->scissors--+ | | +----------------<-------------------+ If it isn't a tie, the test for whether "player" wins is simply (other - player) % N > N // 2 For N == 3, this test is (other - player) % 3 > 1. Looking at it the other way around, it's (player - other) % 3 <= 1, i.e. (player - other) % 3 == 1, as Dave pointed out. The trick is to add items that make sense for all possible pairings. Each item has to beat the preceding N//2 items and lose to the subsequent N//2 items. So for N==5, we have Wins Loses X Scissors, Rock Paper, Y Y X, Paper Scissors, Rock I'll just plagiarize a certain TV show. X is Spock and Y is Lizard: Spock smashes scissors Spock vaporizes rock paper disproves Spock lizard poisons Spock lizard eats paper scissors decapitates lizard rock crushes lizard From alan.gauld at btinternet.com Fri Jul 26 20:58:43 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Fri, 26 Jul 2013 19:58:43 +0100 Subject: [Tutor] Rock, Paper, Scissors game In-Reply-To: References: Message-ID: On 26/07/13 12:22, Saad Javed wrote: > WEAPONS = 'Rock', 'Paper', 'Scissors' > if cpu != player: > if (player - cpu) % 3 < (cpu - player) % 3: #How does this line work? Try it in the interpreter... the two subtractions will always result in +/-1 or +/-2 >>> -2 % 3 1 >>> 2 % 3 2 >>> -1 % 3 2 >>> 1 % 3 1 Now see how those work in the if statement. It's not how I would implement it(*) but its neat enough and works. (*)I'd probably use a dictionary or look up table. HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From sbjaved at gmail.com Fri Jul 26 21:59:12 2013 From: sbjaved at gmail.com (Saad Javed) Date: Sat, 27 Jul 2013 00:59:12 +0500 Subject: [Tutor] Rock, Paper, Scissors game In-Reply-To: References: Message-ID: > I would never have come up with this line, but I think it can be pictured as > follows: > > Given the cycle > > +-->rock-->paper-->scissors--+ > | | > +---------------<------------+ > > and only one direction allowed the winning "weapon" is always the one > farther away from the loser, e. g. > > rock vs paper > rock-->paper: 1 step > paper-->scissors-->rock: 2 steps > paper wins > > rock vs rock: > 0 steps both > tie > > The modulo operator helps with determining this "directed distance" by > maintaining > > -n == N-n > > e. g. > > -1 % 3 == 3 - 1 == 2 > > You could anecdotally test the above outline by adding more weapons. For > four weapons you should get ties between distinct weapons: > > +-->rock-->paper-->scissors-->banana--+ > | | > +---------------<---------------------+ > > rock vs scissors: > rock-->paper-->scissors: 2 steps > scissors-->banana-->rock: 2 steps > tie > > You can modify the script to work with an arbitrary number of weapons by > replacing 3 with length(weapons) if you like. Thank you for taking out the time to explain. From fomcl at yahoo.com Fri Jul 26 22:22:29 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Fri, 26 Jul 2013 13:22:29 -0700 (PDT) Subject: [Tutor] use of the newer dict types In-Reply-To: References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> Message-ID: <1374870149.22579.YahooMailNeo@web163802.mail.gq1.yahoo.com> ----- Original Message ----- > From: Alan Gauld > To: tutor at python.org > Cc: > Sent: Friday, July 26, 2013 1:12 PM > Subject: Re: [Tutor] use of the newer dict types > > On 26/07/13 09:50, Albert-Jan Roskam wrote: > >> if I have d = {1: 2}, I can do membership testing in the following >> two ways --but which one is the preferred way?: >> if 1 in d: >> ? ? ? pass >> if d.has_key(1): >> ? ? ? pass > > > In addition to being more readable (and the fact it's the only way in > v3) 'in' has another big advantage - it's polymorphic: > > holds_1 = [] > bigBox = [ {1:2,3:4}, ['a',2,;freddy',True], (1,2,3)? ] > > for collection in bigBox: > ? ? if 1 in collection: > ? ? ? holds_1.append(collection) > > You can't do that using has_key(), the code has to test > for dict type etc. > > So all in all 'in' is the way to go. Hmm, I'm in (pun intended ;-). Those are three very good arguments. I'll have a look if my IDE (PyCharm) warns about the incompatibility with Python 3.x (t does so with several things, like u'). Thank you! From cybervigilante at gmail.com Fri Jul 26 23:55:08 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Fri, 26 Jul 2013 14:55:08 -0700 Subject: [Tutor] use of the newer dict types In-Reply-To: References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> Message-ID: On 26 July 2013 04:12, Alan Gauld wrote: > In addition to being more readable (and the fact it's the only way in v3) > 'in' has another big advantage - it's polymorphic: > > holds_1 = [] > bigBox = [ {1:2,3:4}, ['a',2,;freddy',True], (1,2,3) ] That is totally cool. I see something here every day that I know will be useful. Although Python choked on the semicolon in front of freddy. And using True in place of 1 is sneakily pedagogical ;') To ask about something entirely different, I downloaded Wing Personal, and was puzzled that it had no button to run the file in the interpreter, on the menu, unlike Wing 101. All it has is one to run debug. I wrote them and they seem to think that advanced programmers will only want to run debug. But that seems odd to me. Is that the case with seasoned programmers? I figure everyone likes to experiment by running a small program in the interpreter now and then, even if they're gurus. I found how to do that in Wing Personal after some annoyance, but it's awkward. Either a three-finger salute, highlighting, or a key-remap (which for some odd reason wasn't mentioned in the remap dictionary, so I had to wait for a reply to get the right remap clause.) But it still seems to me that just trying a short trial program to run in the interpreter should outnumber running a debug ten to one. After all, it might actually work, so why debug it ;') So I can't fathom their decision to drop it from the menu, which is almost empty anyway. I suggested that they restore the green run button and make the debug button red. Jim From alan.gauld at btinternet.com Sat Jul 27 00:48:40 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Fri, 26 Jul 2013 23:48:40 +0100 Subject: [Tutor] use of the newer dict types In-Reply-To: References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> Message-ID: On 26/07/13 22:55, Jim Mooney wrote: >> bigBox = [ {1:2,3:4}, ['a',2,;freddy',True], (1,2,3) ] > > That is totally cool. I see something here every day that I know will > be useful. Although Python choked on the semicolon in front of freddy. Oops, my bad. It should, of course be a quote sign. > And using True in place of 1 is sneakily pedagogical ;') It's not 'in place of' 1, it is a totally different value and type. It's only a happy coincidence that bool(1) returns True. So does bool(-9) after all... -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From steve at pearwood.info Sat Jul 27 06:41:33 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Sat, 27 Jul 2013 14:41:33 +1000 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> Message-ID: <51F34F7D.3010301@pearwood.info> On 27/07/13 08:48, Alan Gauld wrote: > On 26/07/13 22:55, Jim Mooney wrote: >> And using True in place of 1 is sneakily pedagogical ;') > > It's not 'in place of' 1, it is a totally different value and type. > It's only a happy coincidence that bool(1) returns True. So does bool(-9) after all... It's more than just a happy coincidence. True actually equals 1. Way back in Ancient Days, when dinosaurs walked the earth and men used Python 1.5, there was no bool type in Python, no True, no False. Instead, 1 and 0 were used as the canonical boolean values. So for example: [steve at ando ~]$ python1.5 -c "print 'c' in 'cat'" 1 [steve at ando ~]$ python1.5 -c "print 'c' in 'dog'" 0 Unfortunately, this leads to a little confusion, especially with functions like cmp() that return 1, 0 or -1. People got confused and thought it returned 1 for equal and 0 for not equal, instead of 0 for equal, 1 for greater than and -1 for less than. Also, people would define their own named bools in their modules: true = 1 false = not true etc. So eventually GvR changed his mind and added bools to Python, not without a *lot* of controversy: http://www.python.org/dev/peps/pep-0285/ For backwards compatibility, True == 1 and False == 0 had to remain the case. So the bool type is actually a subclass of int, but restricted to only two values. The side effect of this is you can do this: True + 2 => returns 3 which depending on your perspective is either a terrible thing or a great thing. -- Steven From cybervigilante at gmail.com Sat Jul 27 07:24:03 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Fri, 26 Jul 2013 22:24:03 -0700 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: <51F34F7D.3010301@pearwood.info> References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> Message-ID: On 26 July 2013 21:41, Steven D'Aprano wrote: > True + 2 > => returns 3 > > which depending on your perspective is either a terrible thing or a great > thing. I think the eminent logician, Lewis Carrol, would like it: snark = "What I tell you three times is true." snarklist = [snark, snark, snark] if len(snark): print(True) True Jim From cybervigilante at gmail.com Sat Jul 27 07:30:17 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Fri, 26 Jul 2013 22:30:17 -0700 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> Message-ID: > if len(snark): print(True) > > True Actually, that should have been len(snarklist), but the result is the same. Or to get even sillier: len([True]) == True True From cybervigilante at gmail.com Sat Jul 27 08:31:50 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Fri, 26 Jul 2013 23:31:50 -0700 Subject: [Tutor] dependencies Message-ID: I downloaded scipy for 3.3, installed numpy first, ran setup.py, but now I get this error: Blas (http://www.netlib.org/blas/) libraries not found. Okay, I went to the scipy site, but it doesn't say a thing about this dependency. Am I missing something? Shouldn't these packages tell you what all the dependencies are so you don't just flail around? Or are they somewhere I'm not looking? I'm looking at the scipy page: https://pypi.python.org/pypi/scipy/0.12.0 and don't see a thing about Blas. Is there something obvious I'm supposed to know that I've missed, regarding installs? At my level they seem to be a confusing mess. Jim From __peter__ at web.de Sat Jul 27 09:10:50 2013 From: __peter__ at web.de (Peter Otten) Date: Sat, 27 Jul 2013 09:10:50 +0200 Subject: [Tutor] dependencies References: Message-ID: Jim Mooney wrote: > I downloaded scipy for 3.3, installed numpy first, ran setup.py, but > now I get this error: Blas (http://www.netlib.org/blas/) libraries > not found. > > Okay, I went to the scipy site, but it doesn't say a thing about this > dependency. Am I missing something? Shouldn't these packages tell you > what all the dependencies are so you don't just flail around? Or are > they somewhere I'm not looking? > > I'm looking at the scipy page: > https://pypi.python.org/pypi/scipy/0.12.0 and don't see a thing about > Blas. > > Is there something obvious I'm supposed to know that I've missed, > regarding installs? I found http://www.scipy.org/scipylib/building/index.html > At my level they seem to be a confusing mess. So far my impression was that mere mortals like us are well catered with http://www.scipy.org/install.html From alan.gauld at btinternet.com Sat Jul 27 10:26:25 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Sat, 27 Jul 2013 09:26:25 +0100 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: <51F34F7D.3010301@pearwood.info> References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> Message-ID: On 27/07/13 05:41, Steven D'Aprano wrote: >> It's only a happy coincidence that bool(1) returns True. So does >> bool(-9) after all... > It's more than just a happy coincidence. True actually equals 1. > > For backwards compatibility, True == 1 and False == 0 had to remain the > case. So the bool type is actually a subclass of int, but restricted to > only two values. The side effect of this is you can do this: > > True + 2 > => returns 3 Most of what you said I knew, but I didn't realize it went so far as allowing arithmetic! Now I'm on the side of that being a bad thing. Booleans should be as distinct from numbers as chars are from strings... ;-) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From cybervigilante at gmail.com Sat Jul 27 19:21:08 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 27 Jul 2013 10:21:08 -0700 Subject: [Tutor] dependencies In-Reply-To: References: Message-ID: On 27 July 2013 00:10, Peter Otten <__peter__ at web.de> wrote: > http://www.scipy.org/scipylib/building/index.html I already installed pyzo. The query was more about this general business of dependencies. I don't see the dependency that failed on the scipy page, and this is not the first time I've installed something and it only mentions a dependency is missing during the install, but not on the download page. I'm just wondering if I'm missing something obvious, or there is something I should know that I don't. Jim From cybervigilante at gmail.com Sat Jul 27 20:03:11 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 27 Jul 2013 11:03:11 -0700 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> Message-ID: On 27 July 2013 01:26, Alan Gauld wrote: > Most of what you said I knew, but I didn't realize it went so far as > allowing arithmetic! Now I'm on the side of that being a bad thing. > > Booleans should be as distinct from numbers as chars > are from strings... ;-) I did this a while back and it took me some time to figure why I got opposite results: if [True]: print(True) else: print(False) # result: True if [True] == True: print(True) else: print(False) # result: False -- Jim From steve at pearwood.info Sat Jul 27 20:36:42 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 28 Jul 2013 04:36:42 +1000 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> Message-ID: <51F4133A.8050204@pearwood.info> On 28/07/13 04:03, Jim Mooney wrote: > I did this a while back and it took me some time to figure why I got > opposite results: > > if [True]: print(True) > else: print(False) > # result: True [True] is a non-empty list, and as a non-empty list, it is a truthy (true-like) value. You could have written [42], or ["hello world"], or even [False] or [0]. What matters is that the list is non-empty and therefore truthy. > if [True] == True: print(True) > else: print(False) > # result: False Obviously a list of one item is not equal to True, no matter what that item happens to be. Likewise [101] != 101, ["spam"] != "spam", and [[]] != []. A list containing an empty list is not equal to an empty list. However, this will test your knowledge: L = [] L.append(L) [L] == L True or false? Can you explain why? -- Steven From cybervigilante at gmail.com Sat Jul 27 21:00:00 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 27 Jul 2013 12:00:00 -0700 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: <51F4133A.8050204@pearwood.info> References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> <51F4133A.8050204@pearwood.info> Message-ID: On 27 July 2013 11:36, Steven D'Aprano wrote: > However, this will test your knowledge: > > L = [] > L.append(L) > [L] == L > > True or false? Can you explain why? I hate it when you make me think. Um, they both contain an empty list. Although the first one contains an empty list that contains an empty list, and the two fail an 'is' test. They are pointing to different things. But they test okay for equivalence since they both contain an empty object, which resolves to False, I think. It's getting a bit deep, here ;') Another thing that hit me is that although you can compare unequal types, as in if [True] == True: print(True) , which works: If you make a nonequivalence comparison, the program crashes: if [True] > True: print(True) Crash! So although they are both comparisons, they are a different type of comparison entirely. Jim From dfjennings at gmail.com Sat Jul 27 21:14:52 2013 From: dfjennings at gmail.com (Don Jennings) Date: Sat, 27 Jul 2013 15:14:52 -0400 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> <51F4133A.8050204@pearwood.info> Message-ID: On Jul 27, 2013, at 3:00 PM, Jim Mooney wrote: > On 27 July 2013 11:36, Steven D'Aprano wrote: > >> However, this will test your knowledge: >> >> L = [] >> L.append(L) >> [L] == L >> >> True or false? Can you explain why? > > I hate it when you make me think. > > Um, they both contain an empty list. Although the first one contains > an empty list that contains an empty list, and the two fail an 'is' > test. They are pointing to different things. But they test okay for > equivalence since they both contain an empty object, which resolves to > False, I think. In [20]: bool(L) Out[20]: True An empty list does evaluate to False, but a list which contains another list (even if the contained list happens to be empty) evaluates to True. Take care, Don From cybervigilante at gmail.com Sat Jul 27 21:17:24 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 27 Jul 2013 12:17:24 -0700 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: <51F4133A.8050204@pearwood.info> References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> <51F4133A.8050204@pearwood.info> Message-ID: On 27 July 2013 11:36, Steven D'Aprano wrote: > > L = [] > L.append(L) > [L] == L > > True or false? Can you explain why? Since I admitted to still being a bit confused, I though it out, and: >>> [[[[[L]]]]] == L True False propagates outward, since each containing bracket also contains an empty object. --- Jim Fair trade: Walmart sends our jobs to China. They send Chinese wages back to us. Seems fair to me ;') From fomcl at yahoo.com Sat Jul 27 21:19:59 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Sat, 27 Jul 2013 12:19:59 -0700 (PDT) Subject: [Tutor] PYTHONHASHSEED, -R Message-ID: <1374952799.78318.YahooMailNeo@web163804.mail.gq1.yahoo.com> Hi, In the little script below, are k and v guaranteed to be equal? # Python 2.7.3 (default, Sep 26 2012, 21:53:58)? [GCC 4.7.2] on linux2 # rnd.py d = {i : i for i in range(50)} k, v? = d.keys(), d.values() assert k == v, "keys and values not the same" I tried: python -R rnd.py (-R means Turn on hash randomization, see http://docs.python.org/2.7/using/cmdline.html) This does not raise an AssertionError, so it appears that they are indeed guranteed to be the same in CPython. Regards, Albert-Jan ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health, what have the Romans ever done for us? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~? From cybervigilante at gmail.com Sat Jul 27 21:29:41 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 27 Jul 2013 12:29:41 -0700 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> <51F4133A.8050204@pearwood.info> Message-ID: On 27 July 2013 12:14, Don Jennings wrote: > > An empty list does evaluate to False, but a list which contains another list (even if the contained list happens to be empty) evaluates to True. Now I'm really confused, since why am I getting this? (py3.3, since I forgot to mention that) >>> [[]] == True False Jim From cybervigilante at gmail.com Sat Jul 27 21:47:05 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 27 Jul 2013 12:47:05 -0700 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> <51F4133A.8050204@pearwood.info> Message-ID: On 27 July 2013 12:29, Jim Mooney wrote: > On 27 July 2013 12:14, Don Jennings wrote: Okay, I see it now: >>> bool([[]]) == bool([]) False >>> bool([[]]) True >>> bool([]) False >>> Those bool types are handy for resolving confusion. I totally forgot them. Jim From wprins at gmail.com Sat Jul 27 23:02:53 2013 From: wprins at gmail.com (Walter Prins) Date: Sat, 27 Jul 2013 22:02:53 +0100 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> <51F4133A.8050204@pearwood.info> Message-ID: Hi, On 27 July 2013 11:36, Steven D'Aprano wrote: > However, this will test your knowledge: > > L = [] > L.append(L) > [L] == L > > True or false? Can you explain why? On 27 July 2013 20:14, Don Jennings wrote: > In [20]: bool(L) > Out[20]: True > > An empty list does evaluate to False, but a list which contains another > list (even if the contained list happens to be empty) evaluates to True. > > True but besides the point for an expression where you're comparing 2 lists -- Lists are considered equal when corresponding values are equal, not if their truthyness is the same as the truthyness of the list on the other side of the expression. In the case above, you have 2 lists. On the right hand side of the boolean expression you have L, which is a plain list, and which (crucially) contains one element, a reference to a list L. On the *left* hand side, you have another list, which also contains 1 element, a reference to list L. So, since lists are considered equal when the *values* of their corresponding elements are the same, and since each list contains 1 element, being a reference to the object (list) L, it follows that the expression [L] and L would be considered equal by Python's comparison rules, which means the expression evaluates to True. Analogously, consider the following: l1=[1] l2=l1 l3=[l1] l4=[l2] l3==l4 # True or false? Why? Walter -------------- next part -------------- An HTML attachment was scrubbed... URL: From eryksun at gmail.com Sat Jul 27 23:49:11 2013 From: eryksun at gmail.com (eryksun) Date: Sat, 27 Jul 2013 17:49:11 -0400 Subject: [Tutor] PYTHONHASHSEED, -R In-Reply-To: <1374952799.78318.YahooMailNeo@web163804.mail.gq1.yahoo.com> References: <1374952799.78318.YahooMailNeo@web163804.mail.gq1.yahoo.com> Message-ID: On Sat, Jul 27, 2013 at 3:19 PM, Albert-Jan Roskam wrote: > In the little script below, are k and v guaranteed to be equal? > > d = {i : i for i in range(50)} > k, v = d.keys(), d.values() > assert k == v, "keys and values not the same" Yes, provided you compare list(k) and list(v), since in 3.x the views aren't equal. The dict isn't changing state, and the table traversal in each case is a single pass, looking for non-NULL values. I'm no language lawyer, but it seems the wording in the docs under the description of views guarantees that this is true across Python implementations: http://docs.python.org/2.7/library/stdtypes.html#dictionary-view-objects Keys and values are iterated over in an arbitrary order which is non-random, varies across Python implementations, and depends on the dictionary?s history of insertions and deletions. If keys, values and items views are iterated over with no intervening modifications to the dictionary, the order of items will directly correspond. This allows the creation of (value, key) pairs using zip(): pairs = zip(d.values(), d.keys()). Threading and asynchronous signal handlers may be an issue. A CPython thread defaults to holding the global interpreter lock for 100 virtual ops (see sys.setcheckinterval). So it might release the GIL in between calling list(keys()) and list(values()). In that case use your own lock. Or better yet, avoid using shared state if possible. > I tried: > python -R rnd.py > > (-R means Turn on hash randomization The way the objects hash into the table isn't relevant here. Also, hash randomization (_Py_HashSecret) isn't used by numbers. It's used by strings, and anything that hashes a string. For example, datetime objects use the hash of the string representation from __reduce__: >>> d1 = date(2013,7,28) >>> hash(d1) == hash(d1.__reduce__()[1]) True >>> d2 = datetime(2013,7,28,13,30) >>> hash(d2) == hash(d2.__reduce__()[1][0]) True date's hash is using the full tuple at index 1, while datetime's hash only uses the string data. CPython 3.3 defaults to enabling hash randomization. Set the environment variable PYTHONHASHSEED=0 to disable it. From cybervigilante at gmail.com Sun Jul 28 00:13:11 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 27 Jul 2013 15:13:11 -0700 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> <51F4133A.8050204@pearwood.info> Message-ID: On 27 July 2013 14:02, Walter Prins wrote: > So, since lists are considered equal when the *values* of their > corresponding elements are the same, I would have avoided being marvelously wrong by remembering Guido is a mathematician, and a set that contains an empty set contains one member. I'll have to remember Pythonic thinking is mathematical next time I'm puzzled. I'll evade your question, though, since I put all the values in the interpreter and that's cheating ;') But its good to know these nuts and bolts. It might help avoid disaster at a later time. That, and I'm going to use bool and the interpreter until I'm dead clear on something. Jim From cybervigilante at gmail.com Sun Jul 28 01:52:16 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 27 Jul 2013 16:52:16 -0700 Subject: [Tutor] dependencies In-Reply-To: References: Message-ID: On 27 July 2013 00:10, Peter Otten <__peter__ at web.de> wrote: > So far my impression was that mere mortals like us are well catered with > > http://www.scipy.org/install.html The Pyzo editor lets you switch between a py33 and py27 shell right in the editor, without any trouble, which is something I've been looking for, and others might find of use. Jim From alan.gauld at btinternet.com Sun Jul 28 01:53:46 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Sun, 28 Jul 2013 00:53:46 +0100 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> <51F4133A.8050204@pearwood.info> Message-ID: On 27/07/13 20:17, Jim Mooney wrote: > Since I admitted to still being a bit confused, I though it out, and: > >>>> [[[[[L]]]]] == L > True > > False propagates outward, since each containing bracket also contains > an empty object. Its not that false propagates out, it's how Python does a comparison of lists. The rules for equality are what matter here not the rules of boolean conversion. comparison operations return -1, 0 or 1. The boolean conversion of that result is effectively the inverse of the return value - ie 0 implies equality is true, -1 or 1 implies False. Try: >>> cmp([L],L) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From cybervigilante at gmail.com Sun Jul 28 02:33:12 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 27 Jul 2013 17:33:12 -0700 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> <51F4133A.8050204@pearwood.info> Message-ID: Ah, clarity. Just a note, but cmp is defined in 2.7 but not in 3.3 - being able to instantly switch between the two major shells in the Pyzo editor is really helpful, since my Lutz book is always comparing the two: > Try: > >>>> cmp([L],L) > Jim God does not know English From eryksun at gmail.com Sun Jul 28 02:33:35 2013 From: eryksun at gmail.com (eryksun) Date: Sat, 27 Jul 2013 20:33:35 -0400 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> <51F4133A.8050204@pearwood.info> Message-ID: On Sat, Jul 27, 2013 at 4:26 AM, Alan Gauld wrote: > > Most of what you said I knew, but I didn't realize it went so far as > allowing arithmetic! Now I'm on the side of that being a bad thing. > > Booleans should be as distinct from numbers as chars > are from strings... ;-) I wondered if the standard library uses arithmetic ops on bools. So I disabled the int ops (a quick ctypes hack on PyBool_Type's tp_flags) and ran the test suite. It turns out unittest itself uses this, so much that I didn't even bother to finish running Python's tests. For example: def three_way_cmp(x, y): """Return -1 if x < y, 0 if x == y and 1 if x > y""" return (x > y) - (x < y) http://hg.python.org/cpython/file/516303f32bad/Lib/unittest/util.py#l78 From eryksun at gmail.com Sun Jul 28 02:37:13 2013 From: eryksun at gmail.com (eryksun) Date: Sat, 27 Jul 2013 20:37:13 -0400 Subject: [Tutor] dependencies In-Reply-To: References: Message-ID: On Sat, Jul 27, 2013 at 7:52 PM, Jim Mooney wrote: > On 27 July 2013 00:10, Peter Otten <__peter__ at web.de> wrote: > >> So far my impression was that mere mortals like us are well catered with >> >> http://www.scipy.org/install.html > > The Pyzo editor lets you switch between a py33 and py27 shell right in > the editor, without any trouble, which is something I've been looking > for, and others might find of use. Do you mean IEP? http://www.iep-project.org IIRC, it even works with PyPy and Jython. From cybervigilante at gmail.com Sun Jul 28 03:35:16 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 27 Jul 2013 18:35:16 -0700 Subject: [Tutor] dependencies In-Reply-To: References: Message-ID: On 27 July 2013 17:37, eryksun wrote: > Do you mean IEP? > > http://www.iep-project.org > > IIRC, it even works with PyPy and Jython. Yes, I really like having the two major shells, 2.7 and 3.3, side by side. I've even seen authors confuse details on the two major py versions now and then, so it's nice to be able to unconfuse myself with the final arbiter, the interpreter ;'). I was about to buy Wing, but that even requires either running two wings or switching projects all the time. Just clicking on one shell tab or the other in IEP for a quick syntax check is a lot more intuitive, and exactly what I was looking for. And the price is right. -- Jim God does not know English. From amitsaha.in at gmail.com Sun Jul 28 03:43:11 2013 From: amitsaha.in at gmail.com (Amit Saha) Date: Sun, 28 Jul 2013 11:43:11 +1000 Subject: [Tutor] dependencies In-Reply-To: References: Message-ID: On Sun, Jul 28, 2013 at 9:52 AM, Jim Mooney wrote: > On 27 July 2013 00:10, Peter Otten <__peter__ at web.de> wrote: > >> So far my impression was that mere mortals like us are well catered with >> >> http://www.scipy.org/install.html > > The Pyzo editor lets you switch between a py33 and py27 shell right in > the editor, without any trouble, which is something I've been looking > for, and others might find of use. I randomly bumped into Pyzo last week and found it the easiest way to get a Python 3 environment for Matplotlib (which is what I was looking for) on Windows. Of course, it supports a bunch of other stuff and I hadn't yet come across the ability to use different shells. So thanks for the pointer. I think it's a great resource, especially on Windows. On Linux, thanks to the packagers, we don't have to worry so much. Oh, they are planning to use 'conda' as their package manager starting from their next release. I think this makes it even more attractive. From steve at pearwood.info Sun Jul 28 04:31:21 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 28 Jul 2013 12:31:21 +1000 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> <51F4133A.8050204@pearwood.info> Message-ID: <51F48279.7020605@pearwood.info> On 28/07/13 05:29, Jim Mooney wrote: > On 27 July 2013 12:14, Don Jennings wrote: >> >> An empty list does evaluate to False, but a list which contains another list (even if the contained list happens to be empty) evaluates to True. Don's explanation is mistaken. An empty list does not evaluate to False (the constant): py> [] == False False Instead, an empty list behaves *like* False *in a truth-context*. And similarly for non-empty lists behaving *like* True. To try to avoid confusion, we sometimes say that [] is "false-like", or "falsey", while non-empty lists like [0, 1] are "true-like" or "truthy". This is just duck-typing for true|false values. Any object can quack like the bool False, or like the bool True. Normally, that is all you need, but for those (rare!) cases where you want to convert a truthy value into a canonical True|False value, you can use bool: py> bool([]) False py> bool([0, 1]) True What on earth is a "truth-context"? Any place that Python needs to make a Boolean yes|no, true|false, 1|0 decision: if condition: ... elif condition: ... while condition: ... x if condition else y are the four obvious cases that spring to mind. In some languages, the condition must evaluate to an actual Boolean true|false value. Not Python, which supports duck-typing in truth-contexts. In effect, when you have a truth-context, Python automatically calls bool() on the condition for you. (Please don't learn the bad habit of manually calling bool, it is unnecessary and inefficient.) As a general rule, Python distinguishes objects which are "something" versus "nothing": # Objects which represent "nothing" are falsey: 0 0.0 '' None [] {} any other empty container # Objects which represent "something" are truthy: 1 2.0 'spam' object() non-empty lists non-empty dicts any other non-empty container > Now I'm really confused, since why am I getting this? (py3.3, since I > forgot to mention that) > >>>> [[]] == True > False A list containing anything, or nothing at all, is not equal to the constant True. This is probably what you wanted to test: py> bool([[]]) True -- Steven From cybervigilante at gmail.com Sun Jul 28 05:41:54 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 27 Jul 2013 20:41:54 -0700 Subject: [Tutor] dependencies In-Reply-To: References: Message-ID: Amit Saha > hadn't yet come across the ability to use different shells. So thanks > for the pointer. It's the very top left tab, which is very convenient. When I decided on Python I noticed that the schools and new books were all teaching 3.3. but a lot of modules and programs out in the real world were in 2.7. So it seemed being able to switch shells easily would be very helpful, unless you have an eidetic memory, which I don't. -- Jim From steve at pearwood.info Sun Jul 28 05:46:53 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 28 Jul 2013 13:46:53 +1000 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> <51F4133A.8050204@pearwood.info> Message-ID: <51F4942D.1020007@pearwood.info> On 28/07/13 08:13, Jim Mooney wrote: > On 27 July 2013 14:02, Walter Prins wrote: > >> So, since lists are considered equal when the *values* of their >> corresponding elements are the same, > > I would have avoided being marvelously wrong by remembering Guido is a > mathematician, and a set that contains an empty set contains one > member. If you put an empty bag inside another empty bag, the second bag is not empty any longer, is it? -- Steven From steve at pearwood.info Sun Jul 28 06:04:30 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 28 Jul 2013 14:04:30 +1000 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> <51F4133A.8050204@pearwood.info> Message-ID: <51F4984E.5090905@pearwood.info> On 28/07/13 09:53, Alan Gauld wrote: > Its not that false propagates out, it's how Python does a comparison of lists. The rules for equality are what matter here not the rules of boolean conversion. > > comparison operations return -1, 0 or 1. You're thinking of the cmp() builtin (Python 2.x only, not 3.x). General comparison operations such as == < <= is etc. return True or False. Comparison operators haven't depended on cmp() for a long time. You can google "rich comparison operators" for more info: https://duckduckgo.com/?q=rich%20comparison%20operators -- Steven From cybervigilante at gmail.com Sun Jul 28 06:22:02 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sat, 27 Jul 2013 21:22:02 -0700 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: <51F4984E.5090905@pearwood.info> References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> <51F4133A.8050204@pearwood.info> <51F4984E.5090905@pearwood.info> Message-ID: On 27 July 2013 21:04, Steven D'Aprano wrote: > Comparison operators haven't depended on cmp() for a long time. You can > google "rich comparison operators" for more info: >>> [5,6,7,8] < [1,1,1,1] False My, not crashing on that was a surprise. (works in 3.3 and 2.7) -- Jim From alan.gauld at btinternet.com Sun Jul 28 10:07:54 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Sun, 28 Jul 2013 09:07:54 +0100 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: <51F4984E.5090905@pearwood.info> References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> <51F4133A.8050204@pearwood.info> <51F4984E.5090905@pearwood.info> Message-ID: On 28/07/13 05:04, Steven D'Aprano wrote: > On 28/07/13 09:53, Alan Gauld wrote: > >> Its not that false propagates out, it's how Python does a comparison ... >> comparison operations return -1, 0 or 1. > > You're thinking of the cmp() builtin I was trying to use that for simplicity. (Although given Jim's penchant for understanding the inner workings, that was probably a mistake!) The real point I was trying to make was that the equality test does not work by comparing the boolean value of the expressions on each side but by applying comparisons to the actual values (so for lists it checks length and item values). By overloading the comparison operators the test rules are effectively arbitrary for any given class. The boolean result is then applied at the very end based on the test outcome. > Comparison operators haven't depended on cmp() for a long time. You can > google "rich comparison operators" for more info: > > https://duckduckgo.com/?q=rich%20comparison%20operators I only read the first two but one question remains: If cmp() is gone is __cmp__() still supported? I'm assuming it must be for backward compatibility? -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From steve at pearwood.info Sun Jul 28 10:33:36 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Sun, 28 Jul 2013 18:33:36 +1000 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> <51F4133A.8050204@pearwood.info> <51F4984E.5090905@pearwood.info> Message-ID: <51F4D760.1030100@pearwood.info> On 28/07/13 18:07, Alan Gauld wrote: >> Comparison operators haven't depended on cmp() for a long time. You can >> google "rich comparison operators" for more info: >> >> https://duckduckgo.com/?q=rich%20comparison%20operators > > I only read the first two but one question remains: > If cmp() is gone is __cmp__() still supported? I'm assuming it > must be for backward compatibility? Not in Python 3, it's gone. In Python 3, you have to define all six the rich comparison methods __eq__ __ne__ __lt__ __gt__ __le__ __ge__ if you wish to support the comparison operators. There's a helper function in functools to help simplify the job: @functools.total_ordering class MyClass: ... total_ordering will automatically fill in the missing comparison methods using rules like these: "<=" is equivalent to "not >" ">=" is equivalent to "not <" "!=" is equivalent to "not ==" "<=" is equivalent to "< or ==" ">=" is equivalent to "> or ==" -- Steven From memilanuk at gmail.com Sun Jul 28 17:01:14 2013 From: memilanuk at gmail.com (memilanuk) Date: Sun, 28 Jul 2013 08:01:14 -0700 Subject: [Tutor] dependencies In-Reply-To: References: Message-ID: On 07/27/2013 06:35 PM, Jim Mooney wrote: > On 27 July 2013 17:37, eryksun wrote: > >> Do you mean IEP? >> >> http://www.iep-project.org >> >> IIRC, it even works with PyPy and Jython. > > Yes, I really like having the two major shells, 2.7 and 3.3, side by > side. I've even seen authors confuse details on the two major py > versions now and then, so it's nice to be able to unconfuse myself > with the final arbiter, the interpreter ;'). I was about to buy Wing, > but that even requires either running two wings or switching projects > all the time. Just clicking on one shell tab or the other in IEP for a > quick syntax check is a lot more intuitive, and exactly what I was > looking for. And the price is right. > minor quibble: a quick browse thru the mailing list for it seems to indicate that it doesn't support ipython... which is a bummer. From cybervigilante at gmail.com Sun Jul 28 17:50:22 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sun, 28 Jul 2013 08:50:22 -0700 Subject: [Tutor] dependencies In-Reply-To: References: Message-ID: On 28 July 2013 08:01, memilanuk wrote: > minor quibble: a quick browse thru the mailing list for it seems to > indicate that it doesn't support ipython... which is a bummer. I'm looking at the pyzo33 directory and it has ipython_notebook.exe, and Pyzo33 > Lib > pyzo-packages has an Ipython directory. I don't know a thing about ipython or how to access it but that seems to indicate pyzo has something to do with it. Jim From cybervigilante at gmail.com Sun Jul 28 18:43:15 2013 From: cybervigilante at gmail.com (Jim Mooney) Date: Sun, 28 Jul 2013 09:43:15 -0700 Subject: [Tutor] dependencies In-Reply-To: References: Message-ID: On 28 July 2013 08:01, memilanuk wrote: it seems to indicate that it doesn't support ipython.. ===== I managed to load an ipython shell in pyzo's iep. click on shell on top > edit shell configurations The dropdown doesn't give you an ipython option but I erased what was there and entered the path to ipython manually. C:\pyzo33\ipython.exe , It still doesn't appear in the dropdown but it appears to work. I get the In: prompt and magic commands with ?, so I guess that's ipython. Like I said I don't know a thing about it, other than it seems to have better help, which is worth looking into. standard python's help leaves something to be desired. Once you do that the actual shell switching is in the upper left tab. Jim From memilanuk at gmail.com Sun Jul 28 20:03:59 2013 From: memilanuk at gmail.com (memilanuk) Date: Sun, 28 Jul 2013 11:03:59 -0700 Subject: [Tutor] dependencies In-Reply-To: References: Message-ID: On 07/28/2013 08:50 AM, Jim Mooney wrote: > On 28 July 2013 08:01, memilanuk wrote: > >> minor quibble: a quick browse thru the mailing list for it seems to >> indicate that it doesn't support ipython... which is a bummer. > > I'm looking at the pyzo33 directory and it has ipython_notebook.exe, > and Pyzo33 > Lib > pyzo-packages has an Ipython directory. I don't > know a thing about ipython or how to access it but that seems to > indicate pyzo has something to do with it. > Pyzo may be able to... I was referring to IEP, specifically: https://groups.google.com/forum/#!topic/iep_/tgW84YzUGXc While being able to switch between releases of python would be nice, and it appears that iep supports some of the 'magic' features in ipython... there appears to be a certain inflexibility about window positioning (like what if I want the shell in a sub-window in the lower right corner, not stretched across the whole top of the window - I figured out how to move some of the windows/frames, but putting the editor on top and the shell somewhere else is kickin' my butt), and it doesn't support any kind of integrated error checking (pep8/pyflakes/pylint) which for me personally, is more important than being able to flip between different versions of python on the fly. I'm *really* wanting spyder to hurry the heck up and support Python3 already... but it does an outstanding job with 2.7.x, so thats where I'm parked for now. From dfjennings at gmail.com Sun Jul 28 14:03:20 2013 From: dfjennings at gmail.com (Don Jennings) Date: Sun, 28 Jul 2013 08:03:20 -0400 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: <51F48279.7020605@pearwood.info> References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> <51F4133A.8050204@pearwood.info> <51F48279.7020605@pearwood.info> Message-ID: On Jul 27, 2013, at 10:31 PM, Steven D'Aprano wrote: > On 28/07/13 05:29, Jim Mooney wrote: >> On 27 July 2013 12:14, Don Jennings wrote: >>> >>> An empty list does evaluate to False, but a list which contains another list (even if the contained list happens to be empty) evaluates to True. > > Don's explanation is mistaken. Thanks for the correction. I learn so much more when I'm wrong :>) Take care, Don From kbailey at howlermonkey.net Thu Jul 25 20:45:50 2013 From: kbailey at howlermonkey.net (Kirk Bailey) Date: Thu, 25 Jul 2013 14:45:50 -0400 Subject: [Tutor] script in Raspberry pi python In-Reply-To: References: <51EF2A1F.2020206@howlermonkey.net> <51F15A1D.6040402@howlermonkey.net> Message-ID: <51F1725E.9@howlermonkey.net> which python /usr/bin/python root at KirksPiBox1:/home/pi: ./RR.py bash: ./RR.py: /usr/bin/python^M: bad interpreter: No such file or directory root at KirksPiBox1:home/pi:_ On 7/25/2013 1:47 PM, Walter Prins wrote: > Hi, > > > On 25 July 2013 18:02, Kirk Bailey > wrote: > > > On 7/23/2013 9:42 PM, Amit Saha wrote: > > On Wed, Jul 24, 2013 at 11:13 AM, Kirk Bailey > > > wrote: > > Python works fine in pi. script works fine IF I invoke it > high church: > python ./RR.py > > but this fails: ./RR.py > the shebang is > #!/usr/bon/python > > bon -> bin > > right. > > > Also, don't forget to chmod +x RR.py > > already done... I think... yep, 755. RWXR.XR.X! > > Therefore, Fhat the Wuck over? > > > Still not working? Strange. Any error message? What OS? > > What's the output of the following entered at the shell prompt: > > which python > > > > Walter -- -Shaboom. Kirk Bailey CEO, Freehold Marketing LLC http://www.OneBuckHosting.com/ Fnord! -------------- next part -------------- An HTML attachment was scrubbed... URL: From fomcl at yahoo.com Mon Jul 29 11:29:41 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Mon, 29 Jul 2013 02:29:41 -0700 (PDT) Subject: [Tutor] PYTHONHASHSEED, -R In-Reply-To: References: <1374952799.78318.YahooMailNeo@web163804.mail.gq1.yahoo.com> Message-ID: <1375090181.84232.YahooMailNeo@web163801.mail.gq1.yahoo.com> ----- Original Message ----- > From: eryksun > To: Albert-Jan Roskam > Cc: Python Mailing List > Sent: Saturday, July 27, 2013 11:49 PM > Subject: Re: [Tutor] PYTHONHASHSEED, -R > > On Sat, Jul 27, 2013 at 3:19 PM, Albert-Jan Roskam > wrote: >> In the little script below, are k and v guaranteed to be equal? >> >> d = {i : i for i in range(50)} >> k, v? = d.keys(), d.values() >> assert k == v, "keys and values not the same" > > Yes, provided you compare list(k) and list(v), since in 3.x the views > aren't equal. Hi Eryksun, Thank you for replying. Good to know these two things. The following question is almost new-thread-worthy, but: if I would like to make my app work for 2.x and 3.x, what is the best approach: (a) use "if sys.version_info.major...." throughout the code (b) use 2to3, hope for the best and fix manually whatever can't be fixed In any case, it will be good to use code that works in 2.x and 3.x (e.g., from __future__ import print_statement, ..print(..)). Approach (a) will result in one code base (a big plus) but code will be slightly slower (which may be relevant in e.g. busy loops. Approach (b) is not attractive wrt maintenance. If you find a bug, you have to fix it twice, you may need two sets of tests (en run each set every time you commit code). ? > The dict isn't changing state, So that's the criterion! Thanks! So as long as you don't use __setitem__ and __delitem__ (maybe also __setattribute__, __delattribute__, ...) the state does not change. and the table traversal in each case is > a single pass, looking for non-NULL values. I'm no language lawyer, > but it seems the wording in the docs under the description of views > guarantees that this is true across Python implementations: > > http://docs.python.org/2.7/library/stdtypes.html#dictionary-view-objects > > ? ? Keys and values are iterated over in an arbitrary order which is > ? ? non-random, That sounds like a contradictio in terminis to me. How can something be non-random and arbitrary at the same time? varies across Python implementations, and depends > ? ? on the dictionary?s history of insertions and deletions. > > ? ? If keys, values and items views are iterated over with no intervening > ? ? modifications to the dictionary, the order of items will directly > ? ? correspond. This allows the creation of (value, key) pairs using > ? ? zip(): pairs = zip(d.values(), d.keys()). This is indeed *exactly* what my question was about. > Threading and asynchronous signal handlers may be an issue. A CPython > thread defaults to holding the global interpreter lock for 100 virtual > ops (see sys.setcheckinterval). So it might release the GIL in between > calling list(keys()) and list(values()). In that case use your own > lock. Or better yet, avoid using shared state if possible. > >> I tried: >> python -R rnd.py >> >> (-R means Turn on hash randomization > > The way the objects hash into the table isn't relevant here. > > Also, hash randomization (_Py_HashSecret) isn't used by numbers. It's > used by strings, and anything that hashes a string. For example, > datetime objects use the hash of the string representation from > __reduce__: > > ? ? >>> d1 = date(2013,7,28) > ? ? >>> hash(d1) == hash(d1.__reduce__()[1]) > ? ? True > > ? ? >>> d2 = datetime(2013,7,28,13,30) > ? ? >>> hash(d2) == hash(d2.__reduce__()[1][0]) > ? ? True > > date's hash is using the full tuple at index 1, while datetime's hash > only uses the string data. > > CPython 3.3 defaults to enabling hash randomization. Set the > environment variable PYTHONHASHSEED=0 to disable it. So in addition to my "2to3" question from above, it might be a good idea to already set PYTHONHASHSEED so Python 2.x behaves like Python 3.x in this respect, right? Given that the environment variables are already loaded once Python has started, what would be the approach to test this? Call os.putenv("PYTHONHASHSEED", 1), and then run the tests in a subprocess (that would know about the changed environment variables)? Thanks again, Best wishes, Albert-Jan From fomcl at yahoo.com Mon Jul 29 13:33:16 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Mon, 29 Jul 2013 04:33:16 -0700 (PDT) Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: <51F4D760.1030100@pearwood.info> References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> <51F4133A.8050204@pearwood.info> <51F4984E.5090905@pearwood.info> <51F4D760.1030100@pearwood.info> Message-ID: <1375097596.79660.YahooMailNeo@web163803.mail.gq1.yahoo.com> ----- Original Message ----- > From: Steven D'Aprano > To: tutor at python.org > Cc: > Sent: Sunday, July 28, 2013 10:33 AM > Subject: Re: [Tutor] True and 1 [was Re: use of the newer dict types] > > On 28/07/13 18:07, Alan Gauld wrote: >>> Comparison operators haven't depended on cmp() for a long time. You > can >>> google "rich comparison operators" for more info: >>> >>> https://duckduckgo.com/?q=rich%20comparison%20operators >> >> I only read the first two but one question remains: >> If cmp() is gone is __cmp__() still supported? I'm assuming it >> must be for backward compatibility? > > Not in Python 3, it's gone. In Python 3, you have to define all six the rich > comparison methods __eq__ __ne__ __lt__ __gt__ __le__ __ge__ if you wish to > support the comparison operators. There's a helper function in functools to > help simplify the job: Wow, too bad __cmp__ is no longer present. Not so much for cmp(), but because it's such a compact way to define all the comparison methods. Btw, my book says "Python will supply the __ne__() (not equal) inequality operator automatically if wel implement __eq__ but don't implement __ne__()". [Programming in Python 3 (2009), M. Summerfield, p.213] ? > @functools.total_ordering > class MyClass: > ? ? ... > > > total_ordering will automatically fill in the missing comparison methods using > rules like these: > > "<=" is equivalent to "not >" > ">=" is equivalent to "not <" > "!=" is equivalent to "not ==" > > "<=" is equivalent to "< or ==" > ">=" is equivalent to "> or ==" > > > > -- > Steven > _______________________________________________ > Tutor maillist? -? Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > From francois.dion at gmail.com Mon Jul 29 13:54:17 2013 From: francois.dion at gmail.com (Francois Dion) Date: Mon, 29 Jul 2013 07:54:17 -0400 Subject: [Tutor] script in Raspberry pi python In-Reply-To: <51F1725E.9@howlermonkey.net> References: <51EF2A1F.2020206@howlermonkey.net> <51F15A1D.6040402@howlermonkey.net> <51F1725E.9@howlermonkey.net> Message-ID: Dont edit unix files on windows, else you might end up with that problem. It is looking for python^M and not python. Edit with vi and delete the control-M character. Also, If you need to edit a file with a GUI but have the raspberry pi headless check out: http://raspberry-python.blogspot.com/2012/09/sidekick.html http://raspberry-python.blogspot.com/2012/09/sidekick-2.html Then you can edit with geany or the like, but display on your windows pc. Francois Sent from my iPad On Jul 25, 2013, at 2:45 PM, Kirk Bailey wrote: > which python > /usr/bin/python > root at KirksPiBox1:/home/pi: ./RR.py > bash: ./RR.py: /usr/bin/python^M: bad interpreter: No such file or directory > root at KirksPiBox1:home/pi:_ > > > > > On 7/25/2013 1:47 PM, Walter Prins wrote: >> Hi, >> >> >> On 25 July 2013 18:02, Kirk Bailey wrote: >> >> On 7/23/2013 9:42 PM, Amit Saha wrote: >> On Wed, Jul 24, 2013 at 11:13 AM, Kirk Bailey wrote: >> Python works fine in pi. script works fine IF I invoke it high church: >> python ./RR.py >> >> but this fails: ./RR.py >> the shebang is >> #!/usr/bon/python >> bon -> bin >> right. >> >> >> Also, don't forget to chmod +x RR.py >> already done... I think... yep, 755. RWXR.XR.X! >> >> Therefore, Fhat the Wuck over? >> >> Still not working? Strange. Any error message? What OS? >> >> What's the output of the following entered at the shell prompt: >> >> which python >> >> >> >> Walter > > -- > > -Shaboom. > > Kirk Bailey > CEO, Freehold Marketing LLC > http://www.OneBuckHosting.com/ > > Fnord! > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor -------------- next part -------------- An HTML attachment was scrubbed... URL: From steve at pearwood.info Mon Jul 29 18:44:59 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 30 Jul 2013 02:44:59 +1000 Subject: [Tutor] PYTHONHASHSEED, -R In-Reply-To: <1375090181.84232.YahooMailNeo@web163801.mail.gq1.yahoo.com> References: <1374952799.78318.YahooMailNeo@web163804.mail.gq1.yahoo.com> <1375090181.84232.YahooMailNeo@web163801.mail.gq1.yahoo.com> Message-ID: <51F69C0B.6080402@pearwood.info> On 29/07/13 19:29, Albert-Jan Roskam wrote: >> The dict isn't changing state, > > So that's the criterion! Thanks! So as long as you don't use __setitem__ and __delitem__ (maybe also __setattribute__, __delattribute__, ...) the state does not change. Built-in dicts cannot take extra attributes: py> d = {} py> d.surprise = "what?" Traceback (most recent call last): File "", line 1, in AttributeError: 'dict' object has no attribute 'surprise' and even if they did, it would be irrelevant for traversing the key/value pairs. > and the table traversal in each case is >> a single pass, looking for non-NULL values. I'm no language lawyer, >> but it seems the wording in the docs under the description of views >> guarantees that this is true across Python implementations: >> >> http://docs.python.org/2.7/library/stdtypes.html#dictionary-view-objects Even before views, it has been a guarantee going as far back as I can remember that keys(), values() and items() will operate in the same order so long as they are called without modifying the dict between calls. > > Keys and values are iterated over in an arbitrary order which is >> non-random, > > That sounds like a contradictio in terminis to me. How can something be non-random and arbitrary at the same time? That's easy. Two nights ago, I had to decide between having fish or chicken for dinner. For some reason, I decided on fish. I can't tell you why -- I like both equally well, they both needed eating, I could have picked one or the other. Although I had no problem deciding that I wanted the fish, I have no insight into my reasons for that decision. Presumably there is something deep inside my unconscious brain that led me to decide that "I feel like fish tonight" rather than chicken, but since I'm totally unaware of the reasons for that choice, it was an arbitrary choice. Last night I had to choose between Italian or Chinese food. I really couldn't decide, so I tossed a coin[1]. That was a random choice, and I had no control over it. The result was entirely decided by the random process of tossing a coin. In the case of dicts, the iteration order is *deterministic* and follows from the internal structure of the dictionary's hash table, but it is unpredictable and arbitrary: - Python doesn't make any promises about that order, so it could change without warning from version to version. - Even if it doesn't change, Python doesn't tell you what that order is (although you could work it out from reading the source code). - Although that order depends on the history of the dict, the order that keys were added and deleted, and even on what keys it used to have but no longer does, not just the current keys. - To be precise, a dict includes *hidden state* which cannot be seen from Python code (namely the internal hash table) but that affects the order of key iteration. - Since the order is determined entirely by the (hidden) internal state of the dict, it will be the same each time you look, while a random order will likely change each time. For example, in Python 2.7, here are two dicts which have the same value, but iterate in opposite order: py> d1 = {-1: 'a'} py> d1[-2] = 'b' py> py> d2 = {-2: 'b'} py> d2[-1] = 'a' py> py> d1 == d2 True py> print d1; print d2 {-2: 'b', -1: 'a'} {-1: 'a', -2: 'b'} [1] Well, no, actually I didn't. I asked my wife to decide. -- Steven From eryksun at gmail.com Mon Jul 29 19:44:55 2013 From: eryksun at gmail.com (eryksun) Date: Mon, 29 Jul 2013 13:44:55 -0400 Subject: [Tutor] PYTHONHASHSEED, -R In-Reply-To: <1375090181.84232.YahooMailNeo@web163801.mail.gq1.yahoo.com> References: <1374952799.78318.YahooMailNeo@web163804.mail.gq1.yahoo.com> <1375090181.84232.YahooMailNeo@web163801.mail.gq1.yahoo.com> Message-ID: > question is almost new-thread-worthy, but: if I would like to make > my app work for 2.x and 3.x, what is the best approach: > (a) use "if sys.version_info.major...." throughout the code > (b) use 2to3, hope for the best and fix manually whatever can't be fixed (c) Use "six". BTW, sys.version_info.major doesn't work in <2.7. The names were added in 2.7/3.1. >> The dict isn't changing state, > > So that's the criterion! Thanks! So as long as you don't use > __setitem__ and __delitem__ (maybe also __setattribute__, > __delattribute__, ...) the state does not change. It's __getattribute__, __getattr__, __setattr__ and __delattr__. I guess that's relevant if we're talking about a dict that's functioning as a namespace, but that's a bit meta. Setting attributes on the dict itself (I guess it's a subclass we're talking about; normal dict instances don't have a dict) wouldn't affect the hash table it uses for the contained items. BTW, these slot wrappers generally aren't called in CPython, not unless you're overriding the built-in slot function. They're a hook back into the C API. They bind as a method-wrapper that has a function pointer to a C wrapper function that calls the C slot function. If you override __setattr__, your type's tp_setattro slot is set to slot_tp_setattro, which gets called by PyObject_SetAttr. If the value being 'set' is NULL, this function looks up "__delattr__" in your type. Since you didn't override this, it finds and binds the __delattr__ slot wrapper from your base class(es). This is also comes up with rich comparison functions, for which all 6 comparisons are handled by the single slot function, tp_richcompare. So if you subclass dict and override __lt__, then slot_tp_richcompare finds the slot wrapper for the other 5 comparisons by searching dict's dict: >>> type(vars(dict)['__gt__']) And binds it to the instance as a method-wrapper: >>> type(vars(dict)['__gt__'].__get__({})) After jumping through several hoops it ends up at dict_richcompare. For a regular dict, PyObject_RichCompare simply jumps straight to dict_richcompare. It doesn't use the slot wrapper. >> Keys and values are iterated over in an arbitrary order which is >> non-random, > > That sounds like a contradictio in terminis to me. How can something > be non-random and arbitrary at the same time? It's just worded generally to be valid for all implementations. They could have gone into the specifics of the open-addressing hash table used by CPython's dict type, but it would have to be highlighted as an implementation detail. Anyway, the table has a history (collisions with other keys, dummy keys) and size that affects the insertion order. It isn't ontologically random; it's contingent. But that's getting too philosophical I think. >> CPython 3.3 defaults to enabling hash randomization. Set the >> environment variable PYTHONHASHSEED=0 to disable it. > > So in addition to my "2to3" question from above, it might be a good > idea to already set PYTHONHASHSEED so Python 2.x behaves like > Python 3.x in this respect, right? Given that the environment > variables are already loaded once Python has started, what would be > the approach to test this? Call os.putenv("PYTHONHASHSEED", 1), and > then run the tests in a subprocess (that would know about the > changed environment variables)? Sorry, I don't see the point of this. PYTHONHASHSEED is to be set by a system administrator as a security measure. I think CPython is the only implementation that has this feature. With regard to tests that depend on the PYTHON* environment variables, Python's own tests use subprocess.Popen to run sys.executable with a modified "env" environment (e.g. checking the effect of PYTHONIOENCODING). From steve at pearwood.info Mon Jul 29 19:51:24 2013 From: steve at pearwood.info (Steven D'Aprano) Date: Tue, 30 Jul 2013 03:51:24 +1000 Subject: [Tutor] 2.x and 3.x in one code base [was Re: PYTHONHASHSEED, -R] In-Reply-To: <1375090181.84232.YahooMailNeo@web163801.mail.gq1.yahoo.com> References: <1374952799.78318.YahooMailNeo@web163804.mail.gq1.yahoo.com> <1375090181.84232.YahooMailNeo@web163801.mail.gq1.yahoo.com> Message-ID: <51F6AB9C.4080001@pearwood.info> On 29/07/13 19:29, Albert-Jan Roskam wrote: > The following question is almost new-thread-worthy, but: if I would like to make my app work for 2.x and 3.x, what is the best approach: > (a) use "if sys.version_info.major...." throughout the code > (b) use 2to3, hope for the best and fix manually whatever can't be fixed It depends on many factors. How many things that you need have changed? Do you need new features that exist in Python 3 but not 2? But generally I prefer a single code base. In most of my code, making it run in both is relatively simple. I usually support everything from 2.4 through 3.3 (and believe me, I can't wait to drop support for 2.4 -- on the other hand, I'm really grateful I don't need to support 2.3!), so I basically write code that mostly relies on 2.4 features. Then I add any extra, or changed, features in, at the top of the module. I prefer to use the feature if Python already has it, and only fall back on my own back-ported version if I have to, e.g.: try: from functools import wraps except ImportError: # Probably 2.4, so use my own version. from backports import wraps So I have a module, "backports.py", collecting various re-implemented or backported functions. Some things are trivial enough that I just put them at the top of my module: if type(map(chr, '')) is list: # Python 2.x from itertools import imap as map try: enumerate([], 2) # Starting value for enumerate. except TypeError: # Python 2.4 or 2.5 _enumerate = enumerate def enumerate(iterable, start=0): for i, obj in _enumerate(iterable): yield i+start try: strings = (str, unicode) except NameError: # No unicode, means Python 3. strings = (str,) Notice that I always test for the feature, not for the version number. This is defensive programming -- maybe some day my code will be running on an almost-but-not-quite compatible version of Python, and version checks will give the wrong idea. E.g. "AmazingPython 1.1" might have the same functionality of Python 3.3, but I'd never know it if I just check the version number. About the most complex such test I have is this one, to pre-define a couple of named constants for special numeric values: # Include special values. Prior to Python 2.6, this was messy, platform- # dependent, and not guaranteed to work. try: INF = float('inf') NAN = float('nan') except ValueError: # Possibly Windows prior to Python 2.6. try: INF = float('1.#INF') NAN = float('1.#IND') # Indeterminate. except ValueError: # Desperate times require desperate measures... try: INF = 1e3000 # Hopefully, this should overflow to INF. NAN = INF-INF # And this hopefully will give a NaN. except (ValueError, OverflowError): # Just give up. print('*** warning: unable to define INF and/or NAN floats ***') Very occasionally, I end up with *three* modules for some piece of functionality that uses completely different syntax, or is otherwise too hard to write as a single cross-version implementation. A trivial example: # === printer2.py module === import sys def do_print(obj, file=sys.stdout): print >>>file, obj # === printer3.py module === def do_print(obj, file=sys.stdout): print(obj, file=file) # === printer.py module === try: from printer2 import do_print except SyntaxError: from printer3 import do_print Then in the rest of my project, I just: import printer printer.do_print("something") -- Steven From wolfrage8765 at gmail.com Mon Jul 29 20:30:14 2013 From: wolfrage8765 at gmail.com (wolfrage8765 at gmail.com) Date: Mon, 29 Jul 2013 14:30:14 -0400 Subject: [Tutor] Shared State Message-ID: So this is not specifically Python related as it transcends languages. But still I perfer my examples to be in Python since that is the language I know best. I want to learn to program in a more functional manor, in fact I am now reading a book on Erlang, but I still mostly program in Python. I want to know how you reduce side-effects in a GUI program, and further how you handle side effects when you must. Following is a simple example for us to work through as it is my current use case. I have a small program that connects two users together to share files. This program is CLI based and thus uses text inputs and outputs to interact with the user. It uses threads because I still find myself needing to share state. A total of 4 threads: 1 thread waits on input() (User input), another thread performs background tasks like checking internet or transfering files, another thread displays and updates the view or the CLI display, and the final thread possibly un-needed acts as a Mediator where it notifies listeners of events or passes messages. To get to a more functional style program I implemented the mediator thread, it seems like the best option to reduce side-effects, but the reason it has not been fully implemented to date is that I felt like I was violating DRY, as every thread would be storing it's own copy of the state variables that it cared about. Is that the best way? I have more recently implemented multiprocessing.Manager().NameSpace to store the shared state across all of the threads. This is certianly easier and with less violations of DRY, but is it better, or is it dangerous (Dead-Lock)? Back to the program: It starts up and the worker thread immediately checks for an internet connection, while the (UI Thread and Input Thread) UI request that the user tell it if it will host the connection or be a client to another computer (Refered to as Mode). Once the Mode is set and a valid internet connection established the program then obtains the Host IP, either via a site like whatismyipaddress.com if host, or from user input if client. Once the host IP is obtained the program establishes a connection and transfers the files, during the transfer the UI is updated with the progress. So what is the best way to organize the program to reduce side-effects? Should I use message passing like the Mediator, or multiprocessing.Manager()? If you guys really want code I can show code, but the program is not even Beta quality, and my questions are more general to implementation. Plus my code would expand the scope of the conversation, since it does more than just what was explianed. Just in case, I am using Python 3.2+ on Linux Mint 64-bit, but my target market is Windows Users. Thank you for any and all feedback. From alan.gauld at btinternet.com Mon Jul 29 21:03:25 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Mon, 29 Jul 2013 20:03:25 +0100 Subject: [Tutor] Shared State In-Reply-To: References: Message-ID: On 29/07/13 19:30, wolfrage8765 at gmail.com wrote: > I want to learn to program in a more functional manor, in fact I am > now reading a book on Erlang, but I still mostly program in Python. I > want to know how you reduce side-effects in a GUI program, and further > how you handle side effects when you must. You can do a pure functional GUI but it will be slow..... In my experience functional is much better suited to batch processing scenarios - for example each thread launched by a GUI may be functional but the overall GUI is better with shared state, usually using OOP. Remember too that x = f(x) is the functional equivalent of f(x) # x is set inside f() So, GUI state can be encapsulated in a data structure (tuple, dict, class) and your functions can return an updated version of the state. But with multiple threads that means lots of update calls and thread safe synchronisation - hence a slow GUI. > was violating DRY, as every thread would be storing it's own copy of > the state variables that it cared about. Or you pass in the current values each time you call the update() function. > host IP is obtained the program establishes a connection and transfers > the files, during the transfer the UI is updated with the progress. The update needs to be as fast as possible and only done at the end of any extensive processing. Store the results in an intermediate container if necessary (temporary state - a compromise on pure functional but...). Personally I only use pure FP where it makes sense, trying to force FP into a solution that it doesn't fit well is a lot of pain for little gain IMHO. (The same is true for OOP of course! And the good news is that many of the places OOP does not suit fit very well with FP!) You might find it helpful to see how Lisp or Haskell? or erlang? programmers work with GUIs, there are a few web tutors for that... The biggest problem is that the GUI frameworks themselves are not FP friendly. You might find this link interesting: http://stackoverflow.com/questions/2672791/is-functional-gui-programming-possible -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From fomcl at yahoo.com Mon Jul 29 22:29:09 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Mon, 29 Jul 2013 13:29:09 -0700 (PDT) Subject: [Tutor] PYTHONHASHSEED, -R In-Reply-To: <51F69C0B.6080402@pearwood.info> References: <1374952799.78318.YahooMailNeo@web163804.mail.gq1.yahoo.com> <1375090181.84232.YahooMailNeo@web163801.mail.gq1.yahoo.com> <51F69C0B.6080402@pearwood.info> Message-ID: <1375129749.2048.YahooMailNeo@web163806.mail.gq1.yahoo.com> ----- Original Message ----- > From: Steven D'Aprano > To: tutor at python.org > Cc: > Sent: Monday, July 29, 2013 6:44 PM > Subject: Re: [Tutor] PYTHONHASHSEED, -R > > On 29/07/13 19:29, Albert-Jan Roskam wrote: > >>> The dict isn't changing state, >> >> So that's the criterion! Thanks! So as long as you don't use > __setitem__ and __delitem__ (maybe also __setattribute__, __delattribute__, ...) > the state does not change. > > Built-in dicts cannot take extra attributes: > > py> d = {} > py> d.surprise = "what?" > Traceback (most recent call last): > ? File "", line 1, in > AttributeError: 'dict' object has no attribute 'surprise' > > and even if they did, it would be irrelevant for traversing the key/value pairs. > > >> and the table traversal in each case is >>> a single pass, looking for non-NULL values. I'm no language lawyer, >>> but it seems the wording in the docs under the description of views >>> guarantees that this is true across Python implementations: >>> >>> > http://docs.python.org/2.7/library/stdtypes.html#dictionary-view-objects > > Even before views, it has been a guarantee going as far back as I can remember > that keys(), values() and items() will operate in the same order so long as they > are called without modifying the dict between calls. > > >> >? ? ? Keys and values are iterated over in an arbitrary order which is >>> ? ? ? non-random, >> >> That sounds like a contradictio in terminis to me. How can something be > non-random and arbitrary at the same time? > > > That's easy. Two nights ago, I had to decide between having fish or chicken > for dinner. For some reason, I decided on fish. I can't tell you why -- I > like both equally well, they both needed eating, I could have picked one or the > other. Although I had no problem deciding that I wanted the fish, I have no > insight into my reasons for that decision. Presumably there is something deep > inside my unconscious brain that led me to decide that "I feel like fish > tonight" rather than chicken, but since I'm totally unaware of the > reasons for that choice, it was an arbitrary choice. > > Last night I had to choose between Italian or Chinese food. I really > couldn't decide, so I tossed a coin[1]. That was a random choice, and I had > no control over it. The result was entirely decided by the random process of > tossing a coin. > > In the case of dicts, the iteration order is *deterministic* and follows from > the internal structure of the dictionary's hash table, but it is > unpredictable and arbitrary: Nice analogy. Thanks. My wife usually decides too, although I do tend to emphasize that there has got to be beer. ;-) ? From eryksun at gmail.com Mon Jul 29 23:07:40 2013 From: eryksun at gmail.com (eryksun) Date: Mon, 29 Jul 2013 17:07:40 -0400 Subject: [Tutor] PYTHONHASHSEED, -R In-Reply-To: <51F69C0B.6080402@pearwood.info> References: <1374952799.78318.YahooMailNeo@web163804.mail.gq1.yahoo.com> <1375090181.84232.YahooMailNeo@web163801.mail.gq1.yahoo.com> <51F69C0B.6080402@pearwood.info> Message-ID: On Mon, Jul 29, 2013 at 12:44 PM, Steven D'Aprano wrote: >>> a single pass, looking for non-NULL values. I'm no language lawyer, >>> but it seems the wording in the docs under the description of views >>> guarantees that this is true across Python implementations: >>> >>> http://docs.python.org/2.7/library/stdtypes.html#dictionary-view-objects > > Even before views, it has been a guarantee going as far back as I can > remember that keys(), values() and items() will operate in the same order so > long as they are called without modifying the dict between calls. I didn't mean to imply it has anything particularly to do with views. That's just where I found the wording I was looking for in the docs. dict views are stateless (they only have a pointer to the dict) and don't actually have intimate knowledge of the hash table; they create new sets and iterators and use abstract methods. Only the iterator objects work directly with the table. From fomcl at yahoo.com Mon Jul 29 23:08:08 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Mon, 29 Jul 2013 14:08:08 -0700 (PDT) Subject: [Tutor] 2.x and 3.x in one code base [was Re: PYTHONHASHSEED, -R] In-Reply-To: <51F6AB9C.4080001@pearwood.info> References: <1374952799.78318.YahooMailNeo@web163804.mail.gq1.yahoo.com> <1375090181.84232.YahooMailNeo@web163801.mail.gq1.yahoo.com> <51F6AB9C.4080001@pearwood.info> Message-ID: <1375132088.95980.YahooMailNeo@web163805.mail.gq1.yahoo.com> ----- Original Message ----- > From: Steven D'Aprano > To: tutor at python.org > Cc: > Sent: Monday, July 29, 2013 7:51 PM > Subject: [Tutor] 2.x and 3.x in one code base [was Re: PYTHONHASHSEED, -R] > > On 29/07/13 19:29, Albert-Jan Roskam wrote: >> The following question is almost new-thread-worthy, but: if I would like to > make my app work for 2.x and 3.x, what is the best approach: >> (a) use "if sys.version_info.major...." throughout the code >> (b) use 2to3, hope for the best and fix manually whatever can't be > fixed > > It depends on many factors. How many things that you need have changed? Do you > need new features that exist in Python 3 but not 2? But generally I prefer a > single code base. Hi Steven, That seems most attractive to me too. I wouldn't really know how much stuff I'd need to change. This article made me shudder a little though: http://python3porting.com/problems.html.? Worst-case-scenario would be that I'd gradually make fixes throughout the code only to find out the things have become one big convoluted messy bunch of spaghetti. Unicode and (for me) doctest seem two painspots. > In most of my code, making it run in both is relatively simple. I usually > support everything from 2.4 through 3.3 (and believe me, I can't wait to > drop support for 2.4 -- on the other hand, I'm really grateful I don't > need to support 2.3!), so I basically write code that mostly relies on 2.4 > features. Then I add any extra, or changed, features in, at the top of the > module. All the way back to Python 2.4? It seems frustrating not to be able not to use all the nice goodies added in later versions. 2.x vs 3.x is a different story, but why not stay up-to-date? (this is just my personal annoyance with our software in the office which is always emm, not quite up to date). > I prefer to use the feature if Python already has it, and only fall back on my > own back-ported version if I have to, e.g.: > > > > Notice that I always test for the feature, not for the version number. This is > defensive programming -- maybe some day my code will be running on an > almost-but-not-quite compatible version of Python, and version checks will give > the wrong idea. E.g. "AmazingPython 1.1" might have the same > functionality of Python 3.3, but I'd never know it if I just check the > version number. Good point: Python 2.7.2 (1.9+dfsg-1, Jun 19 2012, 23:23:45) [PyPy 1.9.0 with GCC 4.7.0] on linux2 Type "help", "copyright", "credits" or "license" for more information. And now for something completely different: ``why did you guys have to make the builtin fortune more interesting than actual work? i just catched myself restarting pypy 20 times'' >>>> import sys >>>> sys.version '2.7.2 (1.9+dfsg-1, Jun 19 2012, 23:23:45)\n[PyPy 1.9.0 with GCC 4.7.0]' There are probably situations where Python 2.x and pypy are not quite the same. So just using "if sys.version == '2' " is indeed not a good idea. E.g., yesterday I used pypy to run tests (using nosetests) and one test failed. Sliiight difference in ordering and layout of the actual result. > About the most complex such test I have is this one, to pre-define a couple of > named constants for special numeric values: > > > > # Include special values. Prior to Python 2.6, this was messy, platform- > # dependent, and not guaranteed to work. > try: > ? ? INF = float('inf') > ? ? NAN = float('nan') > except ValueError: > ? ? # Possibly Windows prior to Python 2.6. > ? ? try: > ? ? ? ? INF = float('1.#INF') > ? ? ? ? NAN = float('1.#IND')? # Indeterminate. > ? ? except ValueError: > ? ? ? ? # Desperate times require desperate measures... > ? ? ? ? try: > ? ? ? ? ? ? INF = 1e3000? # Hopefully, this should overflow to INF. > ? ? ? ? ? ? NAN = INF-INF? # And this hopefully will give a NaN. > ? ? ? ? except (ValueError, OverflowError): > ? ? ? ? ? ? # Just give up. > ? ? ? ? ? ? print('*** warning: unable to define INF and/or NAN floats > ***') > > > Very occasionally, I end up with *three* modules for some piece of functionality > that uses completely different syntax, or is otherwise too hard to write as a > single cross-version implementation. A trivial example: > > > # === printer2.py module === > import sys > def do_print(obj, file=sys.stdout): > ? ? print >>>file, obj > > > # === printer3.py module === > def do_print(obj, file=sys.stdout): > ? ? print(obj, file=file) > > > # === printer.py module === > try: > ? ? from printer2 import do_print > except SyntaxError: > ? ? from printer3 import do_print > > > > Then in the rest of my project, I just: > > import printer > > printer.do_print("something") > > > > -- > Steven > _______________________________________________ > Tutor maillist? -? Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > From eryksun at gmail.com Mon Jul 29 23:15:01 2013 From: eryksun at gmail.com (eryksun) Date: Mon, 29 Jul 2013 17:15:01 -0400 Subject: [Tutor] PYTHONHASHSEED, -R In-Reply-To: References: <1374952799.78318.YahooMailNeo@web163804.mail.gq1.yahoo.com> <1375090181.84232.YahooMailNeo@web163801.mail.gq1.yahoo.com> <51F69C0B.6080402@pearwood.info> Message-ID: On Mon, Jul 29, 2013 at 5:07 PM, eryksun wrote: > they create new sets and iterators and use abstract methods I meant the abstract C-API, such as PyObject_Size -- not the abstract methods of abstract base classes. Sorry. From fomcl at yahoo.com Mon Jul 29 23:27:24 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Mon, 29 Jul 2013 14:27:24 -0700 (PDT) Subject: [Tutor] PYTHONHASHSEED, -R In-Reply-To: References: <1374952799.78318.YahooMailNeo@web163804.mail.gq1.yahoo.com> <1375090181.84232.YahooMailNeo@web163801.mail.gq1.yahoo.com> Message-ID: <1375133244.44653.YahooMailNeo@web163806.mail.gq1.yahoo.com> ----- Original Message ----- > From: eryksun > To: Albert-Jan Roskam > Cc: Python Mailing List > Sent: Monday, July 29, 2013 7:44 PM > Subject: Re: [Tutor] PYTHONHASHSEED, -R > >> question is almost new-thread-worthy, but: if I would like to make >> my app work for 2.x and 3.x, what is the best approach: >> (a) use "if sys.version_info.major...." throughout the code >> (b) use 2to3, hope for the best and fix manually whatever can't be > fixed > > (c) Use "six". Ok, thanks, I will check this out. Btw, is Pythonbrew still the preferred way of working with different python versions? I read about it and it seems to be handy to use it to switch between versions. But I also read it is no longer maintained. > BTW, sys.version_info.major doesn't work in <2.7. The names were added > in 2.7/3.1. > >>> The dict isn't changing state, >> >> So that's the criterion! Thanks! So as long as you don't use >> __setitem__ and __delitem__ (maybe also __setattribute__, >> __delattribute__, ...) the state does not change. > > It's __getattribute__, __getattr__, __setattr__ and __delattr__. I > guess that's relevant if we're talking about a dict that's > functioning > as a namespace, but that's a bit meta. Setting attributes on the dict > itself (I guess it's a subclass we're talking about; normal dict > instances don't have a dict) wouldn't affect the hash table it uses > for the contained items. > > BTW, these slot wrappers generally aren't called in CPython, not > unless you're overriding the built-in slot function. They're a hook > back into the C API. They bind as a method-wrapper that has a function > pointer to a C wrapper function that calls the C slot function. > > If you override __setattr__, your type's tp_setattro slot is set to > slot_tp_setattro, which gets called by PyObject_SetAttr. If the value > being 'set' is NULL, this function looks up "__delattr__" in > your > type. Since you didn't override this, it finds and binds the > __delattr__ slot wrapper from your base class(es). > > This is also comes up with rich comparison functions, for which all 6 > comparisons are handled by the single slot function, tp_richcompare. > So if you subclass dict and override __lt__, then slot_tp_richcompare > finds the slot wrapper for the other 5 comparisons by searching dict's > dict: > > ? ? >>> type(vars(dict)['__gt__']) > ? ? > > And binds it to the instance as a method-wrapper: > > ? ? >>> type(vars(dict)['__gt__'].__get__({})) > ? ? > > After jumping through several hoops it ends up at dict_richcompare. > For a regular dict, PyObject_RichCompare simply jumps straight to > dict_richcompare. It doesn't use the slot wrapper. > >>> ? ? Keys and values are iterated over in an arbitrary order which is >>> ? ? non-random, >> >> That sounds like a contradictio in terminis to me. How can something >> be non-random and arbitrary at the same time? > > It's just worded generally to be valid for all implementations. They > could have gone into the specifics of the open-addressing hash table > used by CPython's dict type, but it would have to be highlighted as an > implementation detail. Anyway, the table has a history (collisions > with other keys, dummy keys) and size that affects the insertion > order. It isn't ontologically random; it's contingent. But that's > getting too philosophical I think. > >>> CPython 3.3 defaults to enabling hash randomization. Set the >>> environment variable PYTHONHASHSEED=0 to disable it. >> >> So in addition to my "2to3" question from above, it might be a > good >> idea to already set PYTHONHASHSEED so Python 2.x behaves like >> Python 3.x in this respect, right? Given that the environment >> variables are already loaded once Python has started, what would be >> the approach to test this? Call os.putenv("PYTHONHASHSEED", 1), > and >> then run the tests in a subprocess (that would know about the >> changed environment variables)? > > Sorry, I don't see the point of this. PYTHONHASHSEED is to be set by a > system administrator as a security measure. I think CPython is the > only implementation that has this feature. I was referring to what is at the bottom of this page (and I believe it's also mentioned in the doctest documentation): http://python3porting.com/problems.html Basically setting PYTHONHASHSEED will help you find doctests that are badly written, even prior to Python 3.3, but from 3.3 onwards they will fail no matter what. I hope I am not too vague now ;-) > With regard to tests that depend on the PYTHON* environment variables, > Python's own tests use subprocess.Popen to run sys.executable with a > modified "env" environment (e.g. checking the effect of > PYTHONIOENCODING). Ah, ok, glad to read that this is also sort of what I had in mind. From alan.gauld at btinternet.com Tue Jul 30 00:41:04 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Mon, 29 Jul 2013 23:41:04 +0100 Subject: [Tutor] 2.x and 3.x in one code base [was Re: PYTHONHASHSEED, -R] In-Reply-To: <1375132088.95980.YahooMailNeo@web163805.mail.gq1.yahoo.com> References: <1374952799.78318.YahooMailNeo@web163804.mail.gq1.yahoo.com> <1375090181.84232.YahooMailNeo@web163801.mail.gq1.yahoo.com> <51F6AB9C.4080001@pearwood.info> <1375132088.95980.YahooMailNeo@web163805.mail.gq1.yahoo.com> Message-ID: On 29/07/13 22:08, Albert-Jan Roskam wrote: > All the way back to Python 2.4? It seems frustrating not to > be able not to use all the nice goodies added in later versions. ... > just my personal annoyance with our software in the office > which is always emm, not quite up to date). Using latest versions is an indulgence only suited to people working on non mission critical systems with no legacy. Most corporate IT departments have a policy of using the Current-N version. In my last company the official python supported would be 2.5 and that only upgraded last year. (In fact we didn't officially support Python for live systems but we did use Ruby and VB and that was the policy there. Even for Windows we only went to Windows 8 this year... and most users are still on XP or 7 - we never used Vista). The cost of porting is far too high both financially and in terms of risk to follow versions. There needs to be a lot of evidence of maturity as well as a genuine benefit in the new version. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From kbailey at howlermonkey.net Tue Jul 30 03:50:30 2013 From: kbailey at howlermonkey.net (Kirk Bailey) Date: Mon, 29 Jul 2013 21:50:30 -0400 Subject: [Tutor] Fwd: script in Raspberry pi python In-Reply-To: References: <51EF2A1F.2020206@howlermonkey.net> <51F15A1D.6040402@howlermonkey.net> <51F1725E.9@howlermonkey.net> Message-ID: <51F71BE6.8030305@howlermonkey.net> originally edited on a windows box, then copied over with a usb stick. I suppose I better open it, copy it to mouse then paste it to a nw file- or what? Please advise. On 7/25/2013 3:45 PM, Walter Prins wrote: > > Hi, > > > > On 25 July 2013 19:45, Kirk Bailey > wrote: > > which python > /usr/bin/python > root at KirksPiBox1:/home/pi: ./RR.py > bash: ./RR.py: /usr/bin/python^M: bad interpreter: No such file or > directory > root at KirksPiBox1:home/pi:_ > > > Dave's correct. That "^M" represents ctlr-m, and is the displayable > equivalent of the "Carriage Return" character. Windows text files > uses CR/LF (Carriage Return, Line Feed) characters to delimit lines of > text in text files. Unix uses only "Line Feed". So the extra carriage > return character is taken to be part of the text at the end of the > shebang line. And "/usr/bin/python" is not the same as > "/usr/bin/python^M" > > Here's some background: > http://mag-sol.com/articles/crlf.html > > How and where did you create the file? If with an editor, which editor? > > Walter > > > > > > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor -- -Shaboom. Kirk Bailey CEO, Freehold Marketing LLC http://www.OneBuckHosting.com/ Fnord! -------------- next part -------------- An HTML attachment was scrubbed... URL: From kbailey at howlermonkey.net Tue Jul 30 04:42:40 2013 From: kbailey at howlermonkey.net (Kirk Bailey) Date: Mon, 29 Jul 2013 22:42:40 -0400 Subject: [Tutor] Fwd: script in Raspberry pi python In-Reply-To: References: <51EF2A1F.2020206@howlermonkey.net> <51F15A1D.6040402@howlermonkey.net> <51F1725E.9@howlermonkey.net> Message-ID: <51F72820.8020204@howlermonkey.net> ok, I read linked article, then copied the perl script text, which bombs. The TEXT: * #!/usr/local/bin/perl -w** ** use strict;** ** ** while () {** ** s/\r\n/\n/;** ** print;** ** }* This works nicely, and all problems are solved. I tried to download the program using apt-get, but the site for it is not answering th bell, so I just typed this in. Thank you, it i just what the doctor ordered. -- -Shaboom. Kirk Bailey CEO, Freehold Marketing LLC http://www.OneBuckHosting.com/ Fnord! -------------- next part -------------- An HTML attachment was scrubbed... URL: From oscar.j.benjamin at gmail.com Tue Jul 30 15:07:13 2013 From: oscar.j.benjamin at gmail.com (Oscar Benjamin) Date: Tue, 30 Jul 2013 14:07:13 +0100 Subject: [Tutor] 2.x and 3.x in one code base [was Re: PYTHONHASHSEED, -R] In-Reply-To: <1375132088.95980.YahooMailNeo@web163805.mail.gq1.yahoo.com> References: <1374952799.78318.YahooMailNeo@web163804.mail.gq1.yahoo.com> <1375090181.84232.YahooMailNeo@web163801.mail.gq1.yahoo.com> <51F6AB9C.4080001@pearwood.info> <1375132088.95980.YahooMailNeo@web163805.mail.gq1.yahoo.com> Message-ID: On 29 July 2013 22:08, Albert-Jan Roskam wrote: > Steven wrote: >> >> Notice that I always test for the feature, not for the version number. This is >> defensive programming -- maybe some day my code will be running on an >> almost-but-not-quite compatible version of Python, and version checks will give >> the wrong idea. E.g. "AmazingPython 1.1" might have the same >> functionality of Python 3.3, but I'd never know it if I just check the >> version number. > > Good point: > > Python 2.7.2 (1.9+dfsg-1, Jun 19 2012, 23:23:45) > [PyPy 1.9.0 with GCC 4.7.0] on linux2 > Type "help", "copyright", "credits" or "license" for more information. > And now for something completely different: ``why did you guys have to make the > builtin fortune more interesting than actual work? i just catched myself > restarting pypy 20 times'' >>>>> import sys >>>>> sys.version > '2.7.2 (1.9+dfsg-1, Jun 19 2012, 23:23:45)\n[PyPy 1.9.0 with GCC 4.7.0]' > > There are probably situations where Python 2.x and pypy are not quite the same. So just using "if sys.version == '2' " is indeed not a good idea. E.g., yesterday I used pypy to run tests (using nosetests) and one test failed. Sliiight difference in ordering and layout of the actual result. If you do want to test the Python version directly then you should use sys.version_info rather than sys.version: $ python Python 2.7.5 (default, May 15 2013, 22:43:36) [MSC v.1500 32 bit (Intel)] on win32 Type "help", "copyright", "credits" or "license" for more information. >>> import sys >>> sys.version_info sys.version_info(major=2, minor=7, micro=5, releaselevel='final', serial=0) >>> sys.version_info[:2] (2, 7) >>> sys.version_info >= (2, 7, 1) True >>> sys.version_info >= (3, 3) False AmazingPython, pypy etc. should use sys.version_info consistently with CPython to indicate the version of the Python language, not the version of the implementation. PEP 421 adds sys.implementation to the language so that AmazingPython 1.1 can store its version there: http://www.python.org/dev/peps/pep-0421/ Oscar From davea at davea.name Tue Jul 30 16:36:12 2013 From: davea at davea.name (Dave Angel) Date: Tue, 30 Jul 2013 10:36:12 -0400 Subject: [Tutor] Fwd: script in Raspberry pi python In-Reply-To: <51F72820.8020204@howlermonkey.net> References: <51EF2A1F.2020206@howlermonkey.net> <51F15A1D.6040402@howlermonkey.net> <51F1725E.9@howlermonkey.net> <51F72820.8020204@howlermonkey.net> Message-ID: On 07/29/2013 10:42 PM, Kirk Bailey wrote: > ok, I read linked article, then copied the perl script text, which bombs. > The TEXT: > > * #!/usr/local/bin/perl -w** > ** use strict;** > ** > ** while () {** > ** s/\r\n/\n/;** > ** print;** > ** }* > > This works nicely, and all problems are solved. I tried to download the > program using apt-get, but the site for it is not answering th bell, so > I just typed this in. Thank you, it i just what the doctor ordered. > Puzzled why you didn't just use dos2unix, as I suggested 5 days ago. It's in every Unix/Linux I've seen, and if not, it's as close as Synaptic or apt. -- DaveA From fomcl at yahoo.com Wed Jul 31 12:15:00 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Wed, 31 Jul 2013 03:15:00 -0700 (PDT) Subject: [Tutor] superscripts in a regex Message-ID: <1375265700.50164.YahooMailNeo@web163802.mail.gq1.yahoo.com> Hi, ? In the script below I want to filter out the digits and I do not want to retain the decimal grouping symbol, if there are any. The weird thing is that re.findall returns the expected result (group 1 with digits and optionally group2 too), but re.sub does not (it just returns the entire string). I tried using flags re.LOCALE, re.UNICODE, and re.DEBUG for solutions/clues, but no luck # -*- coding: utf-8 -*- #Python 2.7.3 (default, Apr 10 2012, 23:31:26) [MSC v.1500 32 bit (Intel)] on win32 import re regex = "(^\d+)[.,]?(\d*)[ \w]+" surfaces = ["79 m\xb2", "1.000 m\xb2", "2,000 m\xb2"] for i, surface in enumerate(surfaces): ??? #surface = surface.replace("\xb2", "2")? # solves it but maybe, some day, there will me (tm), (c), etc symbols! ??? print str(i).center(79, "-") ??? print re.sub(regex, r"\1\2", surface)? # huh?! ??? print re.findall(regex, surface)? # works as expected It's a no-no to ask this (esp. because it concerns a builtin) but: is this a b-u-g? Regards, Albert-Jan ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health, what have the Romans ever done for us? ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~? From __peter__ at web.de Wed Jul 31 14:24:59 2013 From: __peter__ at web.de (Peter Otten) Date: Wed, 31 Jul 2013 14:24:59 +0200 Subject: [Tutor] superscripts in a regex References: <1375265700.50164.YahooMailNeo@web163802.mail.gq1.yahoo.com> Message-ID: Albert-Jan Roskam wrote: > In the script below I want to filter out the digits and I do not want to > retain the decimal grouping symbol, if there are any. The weird thing is > that re.findall returns the expected result (group 1 with digits and > optionally group2 too), but re.sub does not (it just returns the entire > string). I tried using flags re.LOCALE, re.UNICODE, and re.DEBUG for > solutions/clues, but no luck > regex = "(^\d+)[.,]?(\d*)[ \w]+" > surfaces = ["79 m\xb2", "1.000 m\xb2", "2,000 m\xb2"] > print re.sub(regex, r"\1\2", surface) # huh?! > print re.findall(regex, surface) # works as expected Instead of "huh?!" I would have appreciated a simple Did... Expected... But got... Why? > It's a no-no to ask this (esp. because it concerns a builtin) but: is this > a b-u-g? No bug. Let's remove all the noise from your exposition. Then we get >>> re.sub("(a+)b?(c+)d*", r"\1\2", "aaaabccdddeee") 'aaaacceee' >>> re.findall("(a+)b?(c+)d*", "aaaabccdddeee") [('aaaa', 'cc')] The 'e's are left alone as they are not matched by the regexp. The fix should be obvious, include them in the bytes allowed after group #2: >>> re.sub("(a+)b?(c+)[de]*", r"\1\2", "aaaabccdddeee") 'aaaacc' Translating back to your regex, The byte "\xb2" is not matched by r"[ \w]": >>> re.findall(r"[ \w]", "\xb2") [] Include it explictly (why no $, by the way?) >>> re.sub(r"(^\d+)[.,]?(\d*)[ \w\xb2]+", r"\1\2", "1.000 m\xb2") '1000' or implicitly >>> re.sub(r"(^\d+)[.,]?(\d*)\D+", r"\1\2", "1.000 m\xb2") '1000' and you are golden. PS: I'll leave nudging you to use unicode instead of byte strings to someone else. Only so much (on a console using utf-8): >>> re.findall("[?]", "???") ['\xc2', '\xb9', '\xc2', '\xc2'] >>> print "".join(_) ??? >>> re.findall(u"[?]", u"???") [u'\xb9'] >>> print _[0] ? From fomcl at yahoo.com Wed Jul 31 15:28:08 2013 From: fomcl at yahoo.com (Albert-Jan Roskam) Date: Wed, 31 Jul 2013 06:28:08 -0700 (PDT) Subject: [Tutor] superscripts in a regex In-Reply-To: References: <1375265700.50164.YahooMailNeo@web163802.mail.gq1.yahoo.com> Message-ID: <1375277288.935.YahooMailNeo@web163805.mail.gq1.yahoo.com> ----- Original Message ----- > From: Peter Otten <__peter__ at web.de> > To: tutor at python.org > Cc: > Sent: Wednesday, July 31, 2013 2:24 PM > Subject: Re: [Tutor] superscripts in a regex > > Albert-Jan Roskam wrote: > >> In the script below I want to filter out the digits and I do not want to >> retain the decimal grouping symbol, if there are any. The weird thing is >> that re.findall returns the expected result (group 1 with digits and >> optionally group2 too), but re.sub does not (it just returns the entire >> string). I tried using flags re.LOCALE, re.UNICODE, and re.DEBUG for >> solutions/clues, but no luck > >> regex = "(^\d+)[.,]?(\d*)[ \w]+" >> surfaces = ["79 m\xb2", "1.000 m\xb2", > "2,000 m\xb2"] > >> print re.sub(regex, r"\1\2", surface)? # huh?! >> print re.findall(regex, surface)? # works as expected > > Instead of "huh?!" I would have appreciated a simple > > Did... Expected... But got... Why? Ok, sorry. Expected: one or two groups of digits. If a decimal separator is present, there would be two non-empty?roups, else one. ? >> It's a no-no to ask this (esp. because it concerns a builtin) but: is > this >> a b-u-g? > > No bug. Let's remove all the noise from your exposition. Then we get > >>>> re.sub("(a+)b?(c+)d*", r"\1\2", > "aaaabccdddeee") > 'aaaacceee' >>>> re.findall("(a+)b?(c+)d*", "aaaabccdddeee") > [('aaaa', 'cc')] > > The 'e's are left alone as they are not matched by the regexp. The fix > should be obvious, include them in the bytes allowed after group #2: > >>>> re.sub("(a+)b?(c+)[de]*", r"\1\2", > "aaaabccdddeee") > 'aaaacc' > > Translating back to your regex, The byte "\xb2" is not matched by > r"[ \w]": > >>>> re.findall(r"[ \w]", "\xb2") > [] > > Include it explictly (why no $, by the way?) No $ because of possible trailing blanks. > >>>> re.sub(r"(^\d+)[.,]?(\d*)[ \w\xb2]+", > r"\1\2", "1.000 m\xb2") > '1000' > > or implicitly > >>>> re.sub(r"(^\d+)[.,]?(\d*)\D+", > r"\1\2", "1.000 m\xb2") > '1000' > > and you are golden. aaah, thank you so much! I had been staring at this too long. It seemed so strange that the same regex would result in different groupings, depending on use of re.sub vs. re.findall, but it isn't, after all. > PS: I'll leave nudging you to use unicode instead of byte strings to someone > > else. Only so much (on a console using utf-8): > >>>> re.findall("[?]", "???") > ['\xc2', '\xb9', '\xc2', '\xc2'] >>>> print "".join(_) > ??? > >>>> re.findall(u"[?]", u"???") > [u'\xb9'] >>>> print _[0] > ? > > > _______________________________________________ > Tutor maillist? -? Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > From eryksun at gmail.com Wed Jul 31 16:16:52 2013 From: eryksun at gmail.com (eryksun) Date: Wed, 31 Jul 2013 10:16:52 -0400 Subject: [Tutor] True and 1 [was Re: use of the newer dict types] In-Reply-To: <1375097596.79660.YahooMailNeo@web163803.mail.gq1.yahoo.com> References: <51F1DBE9.4010405@pearwood.info> <1374828645.89019.YahooMailNeo@web163806.mail.gq1.yahoo.com> <51F34F7D.3010301@pearwood.info> <51F4133A.8050204@pearwood.info> <51F4984E.5090905@pearwood.info> <51F4D760.1030100@pearwood.info> <1375097596.79660.YahooMailNeo@web163803.mail.gq1.yahoo.com> Message-ID: On Mon, Jul 29, 2013 at 7:33 AM, Albert-Jan Roskam wrote: > Wow, too bad __cmp__ is no longer present. Not so much for cmp(), but > because it's such a compact way to define all the comparison methods. > Btw, my book says "Python will supply the __ne__() (not equal) > inequality operator automatically if wel implement __eq__ but don't > implement __ne__()". [Programming in Python 3 (2009), M. Summerfield, > p.213] That's true in 3.x. The base object type has a default rich comparison that uses __eq__ to implement __ne__ (but not vice versa). The CPython tp_richcompare slot function is object_richcompare in Objects/typeobject.c. Here's a link to 3.3.2: http://hg.python.org/cpython/file/d047928ae3f6/Objects/typeobject.c#l3175 In 2.x the base object type doesn't implement rich comparisons. You'll get the default comparison in CPython (by id or type name), which won't necessarily be consistent with __eq__. Here's something to riddle out. Why does CPython 2.x make 22 method calls for the following cmp()? class A(object): def _print(self, op): print '%s.__%s__' % (type(self).__name__, op) def __eq__(self, other): self._print('eq') return NotImplemented def __lt__(self, other): self._print('lt') return NotImplemented def __gt__(self, other): self._print('gt') return NotImplemented def __coerce__(self, other): self._print('coerce') return NotImplemented class B(A): pass >>> cmp(A(), B()) B.__eq__ A.__eq__ A.__eq__ B.__eq__ B.__eq__ A.__eq__ B.__gt__ A.__lt__ A.__lt__ B.__gt__ B.__gt__ A.__lt__ B.__lt__ A.__gt__ A.__gt__ B.__lt__ B.__lt__ A.__gt__ A.__coerce__ B.__coerce__ B.__coerce__ A.__coerce__ -1 Why is the order BAABBA repeated 3 times? As a hint, I'll group it like this: BA--AB--BA. Jython does this 'right' (IMHO) with 8 calls. PyPy appears to be playing tight to what's technically allowed and steps over the line. It only uses 4 calls, but it doesn't attempt a __coerce__, which I think is technically wrong. From livingmike at gmail.com Wed Jul 31 22:12:50 2013 From: livingmike at gmail.com (Mike McTernan) Date: Wed, 31 Jul 2013 13:12:50 -0700 Subject: [Tutor] readline Message-ID: I am having problems with the readline command in Python 2.7. I have a text file that has 12 lines of text, each line represents a response to a variable called star_sign that is collected via raw_input. However, instead of printing the line 5 it prints the first 5 letters on line 1. Here is the program: from sys import argv script, zodiac = argv prediction = open(zodiac, "r") prompt = "> " def welcome(): print "Welcome to your daily horoscope" horoscope() def horoscope(): print "I can predict your future" print "What is your starsign?" star_sign = raw_input(prompt) if "Leo" in star_sign: print prediction.readline(5) else: print "You gonna die" welcome() From joel.goldstick at gmail.com Wed Jul 31 22:45:54 2013 From: joel.goldstick at gmail.com (Joel Goldstick) Date: Wed, 31 Jul 2013 16:45:54 -0400 Subject: [Tutor] readline In-Reply-To: References: Message-ID: On Wed, Jul 31, 2013 at 4:12 PM, Mike McTernan wrote: > I am having problems with the readline command in Python 2.7. > > I have a text file that has 12 lines of text, each line represents a > response to a variable called star_sign that is collected via > raw_input. > > However, instead of printing the line 5 it prints the first 5 letters on line 1. > > Here is the program: > > from sys import argv > > script, zodiac = argv > > prediction = open(zodiac, "r") > > prompt = "> " > > def welcome(): > print "Welcome to your daily horoscope" > horoscope() > > def horoscope(): > print "I can predict your future" > print "What is your starsign?" > star_sign = raw_input(prompt) > > if "Leo" in star_sign: > print prediction.readline(5) The above reads only the first 5 characters from the file. You probably want to use foo = readlines() which will read each line of a file into foo. Then print foo[5] or maybe foo[4]. I'm not sure what you are doing There is a good tutorial in the python docs on doing file stuff. It will help you understand this stuff better. > > else: > print "You gonna die" > > welcome() > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor -- Joel Goldstick http://joelgoldstick.com From davea at davea.name Wed Jul 31 22:57:29 2013 From: davea at davea.name (Dave Angel) Date: Wed, 31 Jul 2013 16:57:29 -0400 Subject: [Tutor] readline In-Reply-To: References: Message-ID: On 07/31/2013 04:12 PM, Mike McTernan wrote: > I am having problems with the readline command in Python 2.7. > > I have a text file that has 12 lines of text, each line represents a > response to a variable called star_sign that is collected via > raw_input. > > However, instead of printing the line 5 it prints the first 5 letters on line 1. > > Here is the program: > > from sys import argv > > script, zodiac = argv > > prediction = open(zodiac, "r") > > prompt = "> " > > def welcome(): > print "Welcome to your daily horoscope" > horoscope() > > def horoscope(): > print "I can predict your future" > print "What is your starsign?" > star_sign = raw_input(prompt) > > if "Leo" in star_sign: > print prediction.readline(5) That 5 limits the number of bytes that readline() will read to 5. I doubt if that's what you wanted. > > else: > print "You gonna die" > > welcome() I suggest instead that you replace the prediction line with infile = open(zodiac, "r") predictions = infile.readlines() infile.close() And the print line with print predictions[5] Notice that predictions is now a list of strings, each representing one line of the file. They also have newlines, so you may want to strip() them, but that's a minor issue. -- DaveA From alan.gauld at btinternet.com Wed Jul 31 23:05:33 2013 From: alan.gauld at btinternet.com (Alan Gauld) Date: Wed, 31 Jul 2013 22:05:33 +0100 Subject: [Tutor] readline In-Reply-To: References: Message-ID: On 31/07/13 21:12, Mike McTernan wrote: > I am having problems with the readline command in Python 2.7. > script, zodiac = argv > prediction = open(zodiac, "r") This code is very fragile, you should check that zodiac is actually being set by the user and ideally that it is a real file before using it. Alternatively use a try/except block to catch any error. The latter is probably the most Pythonic approach. if "Leo" in star_sign: print prediction.readline(5) > instead of printing the line 5 it prints the first 5 letters on line 1. What made you think it would read line 5? The help clearly says: readline(...) readline([size]) -> next line from the file, as a string. Retain newline. A non-negative size argument limits the maximum number of bytes to return (an incomplete line may be returned then). Return an empty string at EOF. (END) So the '5' limits the number of *bytes* read not the lines. The program is working exactly as you should expect. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ From artb at mail.pyramid.net Wed Jul 31 17:14:45 2013 From: artb at mail.pyramid.net (Art Bullentini) Date: Wed, 31 Jul 2013 08:14:45 -0700 Subject: [Tutor] Converting a string to an integer Message-ID: <201307310814100.SM6474454@VADER2> This message was originally HTML formatted. View in a HTML capable client to see the original version.\r\n\r\nI'm trying to convert from a string of digits such as x = [y:y+1] into an integer using int(x). the book says I can do this, but I get an error message saying no can do. Anybody tell me what's going on? -------------- next part -------------- An HTML attachment was scrubbed... URL: From bdourass at gmail.com Tue Jul 30 05:18:59 2013 From: bdourass at gmail.com (Bassam Dourassi) Date: Tue, 30 Jul 2013 11:18:59 +0800 Subject: [Tutor] Python to open a "example.jmp" file Message-ID: I am trying to open a JMP file using python. Here is the code I found online: # -*- coding: cp1252 -*- from win32com.client import Dispatch jmp = Dispatch("JMP.Application") doc = jmp.OpenDocument("C:/Users/bdourass/Desktop/Python/example.jmp") doc.CreateOneWay This is the error I get: Traceback (most recent call last): File "C:\Users\bdourass\Desktop\Python\RunJMPusingPython.py", line 3, in jmp = Dispatch("JMP.Application") File "C:\Python27\lib\site-packages\win32com\client\__init__.py", line 95, in Dispatch dispatch, userName = dynamic._GetGoodDispatchAndUserName(dispatch,userName,clsctx) File "C:\Python27\lib\site-packages\win32com\client\dynamic.py", line 108, in _GetGoodDispatchAndUserName return (_GetGoodDispatch(IDispatch, clsctx), userName) File "C:\Python27\lib\site-packages\win32com\client\dynamic.py", line 85, in _GetGoodDispatch IDispatch = pythoncom.CoCreateInstance(IDispatch, None, clsctx, pythoncom.IID_IDispatch) com_error: (-2147221005, 'Invalid class string', None, None) >>> I couldn't find much resources about the interaction between python and JMP pro software online All inputs are appreciated. Thanks in advance. Sam -------------- next part -------------- An HTML attachment was scrubbed... URL: From jayhong123 at gmail.com Thu Jul 25 01:23:52 2013 From: jayhong123 at gmail.com (Jonathan Hong) Date: Wed, 24 Jul 2013 16:23:52 -0700 Subject: [Tutor] Python Trouble, Please Help! Message-ID: Hello, I am currently working with the Python programming language and had a very basic question regarding the start up of Python. For some reason, when I open up the Command Prompt in my Windows 7 computer, I am unable to access python files. So when I type in the python command, I am able to start writing in python, but I cannot call other python files such as saying python helloworld.py. I think it might have something to do with where I am placing my files and the path of python? Not too sure though because of my lack of computer knowledge. When I do put in python helloworld.py into the command prompt line, the computer gives me a response of: "python: can't open file 'hello.py': [Errno 2] No such file or directory." I know for a fact that there does exist a file called hello.py though. Please help me if you can! Sincerely, Python_Beginner -------------- next part -------------- An HTML attachment was scrubbed... URL: From renedatta at gmail.com Mon Jul 29 18:29:40 2013 From: renedatta at gmail.com (Rene Datta) Date: Mon, 29 Jul 2013 10:29:40 -0600 Subject: [Tutor] Online references for Python Message-ID: Hi: I was wondering if anyone could suggest a good quality online resource tool or compendium to provide a beginner's level of information to start learning the Python language. Thank you. Rene Datta renedatta at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: From rokbrown at ucdavis.edu Thu Jul 25 04:36:43 2013 From: rokbrown at ucdavis.edu (Rocko Brown) Date: Wed, 24 Jul 2013 19:36:43 -0700 Subject: [Tutor] (no subject) Message-ID: Hello, I am trying to generate a 2D histogram for two arrays of data in CSV format. The code is: x, y = '5k_vec.csv','tcd_5k.csv' H, xedges, yedges = np.histogram2d(x, y) H.shape, xedges.shape, yedges.shape # We can now use the Matplotlib to visualize this 2-dimensional histogram: extent = [yedges[0], yedges[-1], xedges[-1], xedges[0]] import matplotlib.pyplot as plt plt.imshow(H, extent=extent, interpolation='nearest') # plt.colorbar() # plt.show() And the errors I am getting on Wakari are: TypeError Traceback (most recent call last) in () 1 x, y = '5k_vec.csv','tcd_5k.csv'----> 2 H, xedges, yedges = np.histogram2d(x, y) 3 H.shape, xedges.shape, yedges.shape 4 5 # We can now use the Matplotlib to visualize this 2-dimensional histogram: /opt/anaconda/envs/np17py27-1.5/lib/python2.7/site-packages/numpy/lib/twodim_base.pyc in histogram2d(x, y, bins, range, normed, weights) 609 xedges = yedges = asarray(bins, float) 610 bins = [xedges, yedges]--> 611 hist, edges = histogramdd([x,y], bins, range, normed, weights) 612 return hist, edges[0], edges[1] 613 /opt/anaconda/envs/np17py27-1.5/lib/python2.7/site-packages/numpy/lib/function_base.pyc in histogramdd(sample, bins, range, normed, weights) 307 smax = ones(D) 308 else:--> 309 smin = atleast_1d(array(sample.min(0), float)) 310 smax = atleast_1d(array(sample.max(0), float)) 311 else: /opt/anaconda/envs/np17py27-1.5/lib/python2.7/site-packages/numpy/core/_methods.pyc in _amin(a, axis, out, keepdims) 12 def _amin(a, axis=None, out=None, keepdims=False): 13 return um.minimum.reduce(a, axis=axis,---> 14 out=out, keepdims=keepdims) 15 16 def _sum(a, axis=None, dtype=None, out=None, keepdims=False): TypeError: cannot perform reduce with flexible type Any help would be appreciated! -- *Rocko Brown*, MS, EIT PhD Candidate Department of Land, Air, & Water Resources University of California, Davis -------------- next part -------------- An HTML attachment was scrubbed... URL: From zubair.alam.jmi at gmail.com Tue Jul 23 20:22:23 2013 From: zubair.alam.jmi at gmail.com (zubair alam) Date: Tue, 23 Jul 2013 18:22:23 -0000 Subject: [Tutor] Question In-Reply-To: <16209487-F03D-4D68-A1B3-4B597DF17BC5@gmail.com> References: <16209487-F03D-4D68-A1B3-4B597DF17BC5@gmail.com> Message-ID: what exactly you want to do by [1,2,3] = "Anything" the above code is called unpacking of sequence, so there must be some variable on left side of assignment operator. but you have list of integers, why? On Tue, Jul 23, 2013 at 11:15 PM, Don Jennings wrote: > > On Jul 21, 2013, at 10:26 PM, Amandeep Behl wrote: > > > > > > > def Move(label): > > label = "Anything" > > > > a_list = [1,2,3] > > Move(a_list) > > > > I don't see any error when executing above but when i manually assign > [1,2,3] = "Anything" i get error > > Why? > > Repeat after me: label is a name, label is a name, label is a name ;>) > > So, you misunderstand what is happening in your code. Your function > accepts the argument named "label", but you do nothing with the object (a > list of numbers in this case). Instead, you use the same name and assign it > to the string object "Anything". > > Assignment works by assigning a name (on the left side of the operator), > to an object on the right side. > > Take care, > Don > > _______________________________________________ > Tutor maillist - Tutor at python.org > To unsubscribe or change subscription options: > http://mail.python.org/mailman/listinfo/tutor > -------------- next part -------------- An HTML attachment was scrubbed... URL: